Gmail Send Failures

[Bolt]

It's time for BobO's favorite subject, Gmail and TLS!.  I have been experiencing email failures from my BRX this morning.  Some emails retry for 3 hours and then go through, others go through "instantly". 

Here's a typical DmLogger capture of the sequence.  It seems to loose connection during the TLS handshake process.  Any idea on what to look for?

Code: [Select]

12:39:37.621 ,"TCP connection failed or was unexpectedly closed in EMAIL @000013C0"

12:39:37.620 ,"SMTP disconnected!"

12:39:37.619 ,"Server connection lost. Shutting down..."

12:39:27.548 ,"Starting TLS handshake..."

12:39:27.547 ,"220 2.0.0 Ready to start TLS"

12:39:27.545 ,"STARTTLS.."

12:39:27.544 ,"250 SMTPUTF8"

12:39:27.543 ,"250-CHUNKING"

12:39:27.540 ,"250-PIPELINING"

12:39:27.538 ,"250-ENHANCEDSTATUSCODES"

12:39:27.538 ,"250-STARTTLS"

12:39:27.537 ,"250-8BITMIME"

12:39:27.536 ,"250-SIZE 35882577"

12:39:27.535 ,"250-smtp.gmail.com at your service, [aa.bb.cc.dd]"

12:39:27.462 ,"EHLO [ww.xx.yy.zz].."

12:39:27.461 ,"220 smtp.gmail.com ESMTP y61sm1240616ota.31 - gsmtp"

12:39:27.460 ,"SMTP connected!"

12:39:27.459 ,"Connecting to SMTP server..."

12:39:27.458 ,"DNS lookup complete"

12:39:27.457 ,"Beginning DNS lookup..."

[BobO]

It's time for BobO's favorite subject, Gmail and TLS!.  I have been experiencing email failures from my BRX this morning.  Some emails retry for 3 hours and then go through, others go through "instantly". 

Here's a typical DmLogger capture of the sequence.  It seems to loose connection during the TLS handshake process.  Any idea on what to look for?

Code: [Select]

12:39:37.621 ,"TCP connection failed or was unexpectedly closed in EMAIL @000013C0"

12:39:37.620 ,"SMTP disconnected!"

12:39:37.619 ,"Server connection lost. Shutting down..."

12:39:27.548 ,"Starting TLS handshake..."

12:39:27.547 ,"220 2.0.0 Ready to start TLS"

12:39:27.545 ,"STARTTLS.."

12:39:27.544 ,"250 SMTPUTF8"

12:39:27.543 ,"250-CHUNKING"

12:39:27.540 ,"250-PIPELINING"

12:39:27.538 ,"250-ENHANCEDSTATUSCODES"

12:39:27.538 ,"250-STARTTLS"

12:39:27.537 ,"250-8BITMIME"

12:39:27.536 ,"250-SIZE 35882577"

12:39:27.535 ,"250-smtp.gmail.com at your service, [aa.bb.cc.dd]"

12:39:27.462 ,"EHLO [ww.xx.yy.zz].."

12:39:27.461 ,"220 smtp.gmail.com ESMTP y61sm1240616ota.31 - gsmtp"

12:39:27.460 ,"SMTP connected!"

12:39:27.459 ,"Connecting to SMTP server..."

12:39:27.458 ,"DNS lookup complete"

12:39:27.457 ,"Beginning DNS lookup..."

It looks like the server is dumping the connection after 10 seconds of not completing the TLS handshake. The PLC is just slow at doing that kind of computation...crypto is heavy. You can try checking "Disable Certificate Validation" in the SMTP device config. That might save some time and get through the hand shake a bit quicker.

You can also increase $TLSTimeslice (DST68) from 1500 to something higher. That us the amount of time in microseconds the TLS code is allowed to run per scan.

[Controls Guy]

I copied this from another thread on emailing via Gmail.    I did some tests last year.   Hope it's helpful.

Performance observations:

Gmail seems to timeout after 9-10 seconds if the email is not sent successfully (thus the 12 second timeout in the Do-More SMTP config)

The transaction seems to take 1.85-1.90s of CPU time regardless of time slice (CPU fraction dedicated to the EMAIL).    Total CPU time is very consistent across a range of CPU fraction from just over 20% to 65% which was the highest we tested, so it looks like the process is bound by the Do-More and there's no time spent waiting on Gmail at these transaction times.  Internet bandwidth shouldn't be critical If we're Do-More bound for speed, but while I was testing this I had 10.8 Mbps down, 2.2 up.

The EMAIL needs a minimum of 21% of CPU time in order to allocate 1.90s of CPU time before the Gmail timeout, so for a 1666us TLS time slice, the average scan time while not emailing needs to be under 6.2ms.

[Bolt]

Turns out it's Google's fault.

https://community.automationdirect.com/s/question/0D53u00003kQtoaCAC/productivity-gmail-root-ca-certificate-update

[BobO]

Turns out it's Google's fault.

https://community.automationdirect.com/s/question/0D53u00003kQtoaCAC/productivity-gmail-root-ca-certificate-update

I saw that. I assumed that this was an ongoing thing and unrelated to the CA issue.

You can refresh the CA by going into the system config's SMTP device setup. Just retest the connection. While doing so, we update the certificate. Then write to the PLC.

[Controls Guy]

Got me, too.  Glad the instructions on what to do were spelled out.

[Bolt]

Yes, the fix was quite simple, only requires some down time to do a Program Mode write. Thanks BobO!

[Bolt]

You can refresh the CA by going into the system config's SMTP device setup. Just retest the connection. While doing so, we update the certificate. Then write to the PLC.

Happened to me again about an hour ago.  The test in the device setup just ran successfully, so I am assuming once I get a chance to do a program mode write, it will work again.  A nice little early Christmas gift from Google!

All in all, I'm growing tired of Gmail from a BRX.  Between this and the text messages to Verizon being very unpredictable/inconsistent, I'm looking for something more reliable.  Anyone have suggestions what works well for them?

[Bolt]

I'm playing with some "less secure" SMTP gateways, no SSL/TLS.  One thing I have noticed if I go back to the SMTP device config, it defaults back to Server Port 25 each time.  Can this be made to hold the setting?  It keeps all other fields populated correctly.

[PLCwannabe]

All my emails quit working a while back, so I just checked "Disable certificate validation", now they all work again. Are there any risks with having that disabled?

[BobO]

All my emails quit working a while back, so I just checked "Disable certificate validation", now they all work again. Are there any risks with having that disabled?

It means that the PLC is skipping the part where you normally ensure that the email server is really who you think it is. It opens the possibility for someone to spoof the server, but in practical terms, nothing much changes. Right now email is the only thing doing certificate validation, HTTPCMD and MQTT both skip it. Unless you are emailing credit cards or bank account information, I wouldn't worry much, and even then, there is a pretty low likelihood of a server being spoofed.

[BobO]

I'm playing with some "less secure" SMTP gateways, no SSL/TLS.  One thing I have noticed if I go back to the SMTP device config, it defaults back to Server Port 25 each time.  Can this be made to hold the setting?  It keeps all other fields populated correctly.

That's a bug. It's supposed to reset the port when the TLS/SSL settings change, but not upon entering the dialog.

[durallymax]

Not specific to Do-More ( I usually send emails through C-More to avoid the limitations in Psuite), but I run into issues with periods of inactivity. Google will reset the "allow less secure apps" and at times require you to change your password. However, there is no prompt to change your password. Re-enabling less secure apps has not been successful for me all year, always had to change password and then things would start up again. To get around it, I just have them send a monthly email to a junk email.

[BobO]

I'm playing with some "less secure" SMTP gateways, no SSL/TLS.  One thing I have noticed if I go back to the SMTP device config, it defaults back to Server Port 25 each time.  Can this be made to hold the setting?  It keeps all other fields populated correctly.

Fixed.

Pretty sure it was only port 587 with TLS disabled that would do it. You hit the jackpot.