Help with EMAIL

[Controls Guy]

Setting up my first email from Do-More in a long time, and I can't get a GMail account that works from other automation devices to work from a BRX.    I know Gmail has some weirdnesses, so for troubleshooting I'm trying with my own personal email account.

The device config screen tests the setup and account login OK and says my settings are good, but then during the TLS handshake when emailing from ladder, there's a 15 second pause and then I lose the server.

DMLogger file attached.   I have no idea how to troubleshoot something like this.

[BobO]

Some servers dump the connection after 15 seconds if the handshake isn't complete. Depending on scan time and the size of the associated certificate, it can easily exceed that. From the SMTP device config page you can disable certificate validation which might save some time.

The bottom line? SSL/TLS isn't well suited for small embedded controllers and some servers don't do us any favors.

[BobO]

I just added the TLS timeslice value as a configurable DST. Currently the fixed value is 1666us per scan (weird, but makes sense in system clocks). I've set the new DST to a range of 100-20000us. I might open that up further. It is likely that bumping this up would help your situation.

[Controls Guy]

I just added the TLS timeslice value as a configurable DST. Currently the fixed value is 1666us per scan (weird, but makes sense in system clocks). I've set the new DST to a range of 100-20000us. I might open that up further. It is likely that bumping this up would help your situation.

This program runs just under 10ms average.     I was thinking of dumping all the other logic to see if that's the issue, then optimizing if it is.

[BobO]

This program runs just under 10ms average.     I was thinking of dumping all the other logic to see if that's the issue, then optimizing if it is.

At 10ms, you are getting less than 20% of the processor working on the TLS handshake. Good chance that is the issue. This new feature would make it easier to get it done without screwing with logic.

A faster processor would make these issues go away. Eventually.

[Controls Guy]

OK, so getting enough processor attention within the server timeout window was an issue, and doing so makes my personal email (Cox) work OK.

However, no combination of settings that I've tried with Gmail will make it work.   I'm probably not holding my mouth right.

With any login method except 3, the server says I have to execute a STARTTLS command first.   If I use 3 (PB4S), the EMAIL instruction times out (I believe at the "server timeout" device setting, not the actual server timeout delay) with no conversation or any other error indication.

Anyone have a device config / ladder template that's known to work and less than, say, 6 months old?

[BobO]

The only thing that comes to mind is that GMail needed some of the account security settings relaxed to work.

[ATU]

Is anything showing up in the advanced security settings? It may be terminating communications because it doesn't recognize the device.

[Controls Guy]

The only thing that comes to mind is that GMail needed some of the account security settings relaxed to work.

I think you're probably talking about enabling access via "Less Secure Apps" (LSAs).    This is an existing account, and does have LSA's enabled, and works with other automation gear.

However, Googling about the issue (you find a lot of forum queries from people somewhat analgous to us who are trying to get their scanners to email them scans, etc.), I found the following about a phase-out of LSA access by GMail.    This is pasted in from https://www.reddit.com/r/gsuite/comments/ebjrwl/less_secure_nonoauth_apps_will_cease_to_work/, but I saw the same information direct from Google.

So this still isn't an explanation for me of this problem this time (I have LSAs enabled and it's working elsewhere), but seems potentially quite relevant for users emailing from Do-More and other automation platforms.    Does this mean that after February we'll no longer be able to use GMail with Do-More unless you guys implement OAuth2?

Less Secure (non-OAuth) Apps Will Cease to Work February 2021

We?re constantly working to improve the security of your organization?s Google accounts. As part of this effort, and in consideration of the current threat landscape, we?ll be turning off access to less secure apps (LSA) ? non-Google apps that can access your Google account with only a username and password, without requiring any additional verification steps. Access through only a username and password makes your account more vulnerable to hijacking attempts. Moving forward, only apps that support a more modern and secure access method called OAuth will be able to access your G Suite account.

Access to LSAs will be turned off in two stages:

June 15, 2020 - Users who try to connect to an LSA for the first time will no longer be able to do so. This includes third-party apps that allow password-only access to Google calendars, contacts, and email via protocols such as CalDAV, CardDAV and IMAP. Users who have connected to LSAs prior to this date will be able to continue using them until usage of all LSAs is turned off.

February 15, 2021 - Access to LSAs will be turned off for all G Suite accounts.

What do I need to do?

To continue using a specific app with your G Suite accounts, users in your organization must switch to a more secure type of access called OAuth. This connection method allows apps to access accounts with a digital key instead of requiring a user to reveal their username and password. We recommend that you share the user instructions (included below) with individuals in your organization to help them make the necessary changes. Alternatively, if your organization is using custom tools, you can ask the developer of the tool to update it to use OAuth. Developer instructions are also included below.

[BobO]

I think you're probably talking about enabling access via "Less Secure Apps" (LSAs).    This is an existing account, and does have LSA's enabled, and works with other automation gear.

However, Googling about the issue (you find a lot of forum queries from people somewhat analgous to us who are trying to get their scanners to email them scans, etc.), I found the following about a phase-out of LSA access by GMail.    This is pasted in from https://www.reddit.com/r/gsuite/comments/ebjrwl/less_secure_nonoauth_apps_will_cease_to_work/, but I saw the same information direct from Google.

So this still isn't an explanation for me of this problem this time (I have LSAs enabled and it's working elsewhere), but seems potentially quite relevant for users emailing from Do-More and other automation platforms.    Does this mean that after February we'll no longer be able to use GMail with Do-More unless you guys implement OAuth2?

Less Secure (non-OAuth) Apps Will Cease to Work February 2021

We?re constantly working to improve the security of your organization?s Google accounts. As part of this effort, and in consideration of the current threat landscape, we?ll be turning off access to less secure apps (LSA) ? non-Google apps that can access your Google account with only a username and password, without requiring any additional verification steps. Access through only a username and password makes your account more vulnerable to hijacking attempts. Moving forward, only apps that support a more modern and secure access method called OAuth will be able to access your G Suite account.

Access to LSAs will be turned off in two stages:

June 15, 2020 - Users who try to connect to an LSA for the first time will no longer be able to do so. This includes third-party apps that allow password-only access to Google calendars, contacts, and email via protocols such as CalDAV, CardDAV and IMAP. Users who have connected to LSAs prior to this date will be able to continue using them until usage of all LSAs is turned off.

February 15, 2021 - Access to LSAs will be turned off for all G Suite accounts.

What do I need to do?

To continue using a specific app with your G Suite accounts, users in your organization must switch to a more secure type of access called OAuth. This connection method allows apps to access accounts with a digital key instead of requiring a user to reveal their username and password. We recommend that you share the user instructions (included below) with individuals in your organization to help them make the necessary changes. Alternatively, if your organization is using custom tools, you can ask the developer of the tool to update it to use OAuth. Developer instructions are also included below.

No clue. This situation is very frustrating for resource constrained devices (and companies). They want to connect every device to the Net of the future, and yet seem oblivious to the fact that many device are built on low performance micros.

I have a vision of a cloud connected PLC where virtually all of the Internet comms are performed by the cloud itself. I still think that is the only viable answer long term. Make Internet connectivity an Internet problem, and leave factory floor devices to focus on control. It's easy for a controller to say to the cloud server "I want to send this message to X, Y, and Z", and then the cloud itself knows how to do that...even when how to do that changed five times in the last five years. Trying to shove the protocol of the moment into the PLC will always be a losing bet.

[Controls Guy]

No clue. This situation is very frustrating for resource constrained devices (and companies). They want to connect every device to the Net of the future, and yet seem oblivious to the fact that many device are built on low performance micros.

Yeah, I know.   All the support glibly says "Oh, you know, just update to the latest version of TBird or Outlook and go back to watching Netflix.  You'll be fine.", totally ignoring all the embedded stuff.

I have a vision of a cloud connected PLC where virtually all of the Internet comms are performed by the cloud itself. I still think that is the only viable answer long term. Make Internet connectivity an Internet problem, and leave factory floor devices to focus on control. It's easy for a controller to say to the cloud server "I want to send this message to X, Y, and Z", and then the cloud itself knows how to do that...even when how to do that changed five times in the last five years. Trying to shove the protocol of the moment into the PLC will always be a losing bet.

Yeah, but how does that work?   The link from the embedded devices to the cloud comm server will still need to be secure to someone's satisfaction.   Somebody will figure out how to hack the cloud comm server to order stuff on your Ebay account, then the providers will want to make it more secure, and so on, and so on, and so on.   You'll have the same security protocol arms race, just with a different server, wouldn't you?

Or are you saying this cloud server is an edge device at the automation customer, so he has complete physical control over the link from the automation stuff to that server, so it [probably] doesn't need to be secure?    So the comm server is basically a comms coprocessor for the entire building?

[BobO]

Establishing a secure connection is easy. Authenticating is hard. Or is it? If you control both ends of the pipe, it is a manageable thing to make sure you are who you claim. There is also zero need to support every random encryption scheme, only yours, and it can be as obscure as you choose. Basically you embed a secret in the PLC's factory settings, and then talk to the cloud in ancient Sanskrit, using non-standard encryption.

[Controls Guy]

Establishing a secure connection is easy. Authenticating is hard. Or is it? If you control both ends of the pipe, it is a manageable thing to make sure you are who you claim. There is also zero need to support every random encryption scheme, only yours, and it can be as obscure as you choose. Basically you embed a secret in the PLC's factory settings, and then talk to the cloud in ancient Sanskrit, using non-standard encryption.

OK, so you're talking a cloud device dedicated to a finite universe of end devices vs. something global like an smtp server, whether defined by the user or the device vendor, correct?

[BobO]

Precisely. Think of the cloud service as being an agent for the PLC. PLC says "do this thing", and the agent knows the details of what that means in any given month or year. The agent can be maintained by server programmers, not embedded guys. It's a deeper labor pool and a more resource rich platform. Services would be simple to implement there. The PLC is really a thin client for web things, and can emphasize doing control things very well.

I also think there is room for the cloud appliance locally.

[Controls Guy]

Okay, here's what constitutes holding your mouth right in post-LSA GMail (guaranteed to work till at least the end of the day):

Gmail Setup

• Log in to your account
• Go to Checkerboard button at top right → Account → Security and enable 2-Step Verification.
• Go to App Passwords (you'll have to enter your account password again)
• At the App Password screen, go to the Select App pulldown, I selected Other (custom name) and named it PLC.   You don't have to do Select Device.
• Click Generate, and you get a random password generated by Gmail for devices that can't do OAuth authentication.
  

   
Do-More Setup

In the device configuration for your email device, enter the following

• SMTP Server Address
       Use Server Name
       smtp.gmail.com
• Security
       Enable SSL/TLS → Yes
       SSL
       Disable Certificate Validation → No
• Other Settings
       Server Port: 465
       Timeout: 12
       'From' Email Address:  Enter the name you want to show on the Email
• Account Authentication
       AUTH PLAIN
       Username:   <YourEmailAddress@Gmail.Com>
       Password:   Enter the password given to you earlier by GMail

I did some performance timing and will post the results as soon as I get a chance to write it up.