News:

  • February 01, 2023, 07:50:32 AM

Login with username, password and session length

Author Topic: IO Reinitialization  (Read 587 times)

Dmdtrain

  • Newbie
  • *
  • Posts: 5
IO Reinitialization
« on: April 25, 2022, 08:37:39 PM »
Hello,

Is there anyway to reinitialize IO besides using the designer software? Is there something that can be programmed to do the same thing?

Thanks

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #1 on: April 25, 2022, 09:52:09 PM »
Hello,

Is there anyway to reinitialize IO besides using the designer software? Is there something that can be programmed to do the same thing?

Thanks

Honestly, the only time you would ever need to do so is a bug. What condition are you trying to fix?
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #2 on: September 28, 2022, 11:17:40 AM »
I have an issue with a controller as a Modbus TCP server that will loose it's modbus tcp comms but will respond on the DMD UDP port 28784. I can reboot the controller via a REBOOT command and still no response on TCP port 502 while comms on UDP port 28784 are good. The only way I can get Modbus TCP communicating again is to do a Re-Initialize PLC I/O. Has this command been made available via the DMD software?

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #3 on: September 28, 2022, 01:14:54 PM »
I have an issue with a controller as a Modbus TCP server that will loose it's modbus tcp comms but will respond on the DMD UDP port 28784. I can reboot the controller via a REBOOT command and still no response on TCP port 502 while comms on UDP port 28784 are good. The only way I can get Modbus TCP communicating again is to do a Re-Initialize PLC I/O. Has this command been made available via the DMD software?

Are you sure you're actually REBOOTing? That is a full reset that does everything the re-init does and much more. I know of no scenario where re-init would fix something and a REBOOT does not.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #4 on: September 28, 2022, 04:11:11 PM »
...and what firmware version are you running?
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #5 on: September 28, 2022, 04:20:31 PM »
I'm using the REBOOT instruction, so however far that goes?
I'm monitoring the ModbusTCPserver.transactions and if no change in 10 minutes I'll perform a REBOOT via the instruction. My HMI software polls the PLC over Modbus TCP around every 30 seocnds.
Even after multiple REBOOTS the controller is still not connecting I dont know if it's a network issue, but the weird thing is that I am able to connect to it on the same ethernet port via DMD software on UDP port 28784 and if I Re-Initialize PLC I/O the controller connects backup
Also I'm looking into the ModbusTCPserver.lasterror and getting a 129. I cant find any documentation on that, do you have any idea what that error code is?

Firmware 2.8.5

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #6 on: September 28, 2022, 04:39:16 PM »
I'm using the REBOOT instruction, so however far that goes?
I'm monitoring the ModbusTCPserver.transactions and if no change in 10 minutes I'll perform a REBOOT via the instruction. My HMI software polls the PLC over Modbus TCP around every 30 seocnds.
Even after multiple REBOOTS the controller is still not connecting I dont know if it's a network issue, but the weird thing is that I am able to connect to it on the same ethernet port via DMD software on UDP port 28784 and if I Re-Initialize PLC I/O the controller connects backup
Also I'm looking into the ModbusTCPserver.lasterror and getting a 129. I cant find any documentation on that, do you have any idea what that error code is?

Firmware 2.8.5

So when you did the REBOOT, you get the normal power up light show? That is effectively the same as a power cycle. I/O re-init just resets the drivers, so I really don't understand that.

A 129 is garbage data in the Modbus frame. Generally should never happen, but does seem to happen with some radios.

How long does it take to fail?
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #7 on: September 28, 2022, 04:53:55 PM »
It's interminttent, could go weeks before it happens, other times the REBOOT does work, I dont understand it.
This setup is all over radio but like I said during the modbus tcp failures I am able to connect to it via DMD software like the comms are stable. I am using RX/WX commands on the same ethernet but that shouldnt matter, right?

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #8 on: September 28, 2022, 05:00:07 PM »
Would setting the Modbus Client Activity time to a lower or higher rate work better? I usually keep it stock at 60 seconds but maybe I need to tweak it.

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #9 on: September 28, 2022, 05:04:06 PM »
It's interminttent, could go weeks before it happens, other times the REBOOT does work, I dont understand it.
This setup is all over radio but like I said during the modbus tcp failures I am able to connect to it via DMD software like the comms are stable. I am using RX/WX commands on the same ethernet but that shouldnt matter, right?

This sounds exactly like a problem we have already been working on. In attempting to isolate what was going on, I added some diags to the firmware, then the problem stopped (typical). I'm toying with the idea of it being a software build problem, but I have no proof of that, just a hunch. After the diags caused the problem to disappear, I did a new clean build with them removed, and we are testing it on the other user's system. Only a few days in, but so far it hasn't failed. If you want to try it, I'm happy to send it to you.
« Last Edit: September 28, 2022, 05:08:13 PM by BobO »
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #10 on: September 28, 2022, 05:07:28 PM »
Would setting the Modbus Client Activity time to a lower or higher rate work better? I usually keep it stock at 60 seconds but maybe I need to tweak it.

Unless you are getting a large number of comm failures that result in the client abandoning sessions, the default is probably fine. You can know that through ModbusTCPServer.ActiveSessions. If that number starts hitting the max configured, clients will start getting errors when they try to connect.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #11 on: September 28, 2022, 06:19:51 PM »
Yes I would like to try it out. Thanks

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #12 on: December 01, 2022, 08:42:18 PM »
BobO,
After installing the new OS you provided I have not seen any issues until 11/29. It's a little different though, modbus communication will be active for a couple polls than drops out for hours then repeats (I can see this trend from my Historical Trend Charts in my HMI), unlike last OS where the comms would be down and never reconnect until the Re-initialize PLC I/O was enabled .
Today I connected to it remotely over radio with DmD (Same pathway as the Modbus Comms) and watched as the ModbusTCPServer.ActiveSessions = 0 (watched for a couple polls), once I enabled Re-initialize PLC I/O modbus communication instantaneously came back up, I watched for atleast 30 minutes with good comms? Let me know if I can provide any data from this controller that might help.


BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: IO Reinitialization
« Reply #13 on: December 02, 2022, 11:10:37 AM »
BobO,
After installing the new OS you provided I have not seen any issues until 11/29. It's a little different though, modbus communication will be active for a couple polls than drops out for hours then repeats (I can see this trend from my Historical Trend Charts in my HMI), unlike last OS where the comms would be down and never reconnect until the Re-initialize PLC I/O was enabled .
Today I connected to it remotely over radio with DmD (Same pathway as the Modbus Comms) and watched as the ModbusTCPServer.ActiveSessions = 0 (watched for a couple polls), once I enabled Re-initialize PLC I/O modbus communication instantaneously came back up, I watched for atleast 30 minutes with good comms? Let me know if I can provide any data from this controller that might help.

Drops for *hours* and then eventually comes back briefly, before dropping again? So strange. We have never been able to duplicate this, but as I mentioned, we have heard a few reports of it, always with radios. My theory is that something odd is happening with unstable radio connections that is depleting something in the TCP stack, which eventually prevents the client from connecting, but I have no idea what it is. Very frustrating, for users and us.

Once it's failed, do you see any change in the scan time or any other change in the PLC's behavior?

The one thing that might be useful is to see a Wireshark trace when it's failed. It would also be really helpful if I could see exactly how the TCP session is failing when the radio gets flaky, but it's hard telling how often that happens. In another place this is happening, the comms to the site are constantly erroring out due to the radio being less than perfect, but it still takes weeks/months before the problem shows up, so the offending failure may not happen very often. That might make catching it nearly impossible, or finding it in the trace...since we don't even know what we're looking for.

I have a version of firmware with some diagnostics in it. It isn't much, but it might give more info about what is failing. If you'd like to try it, I'd be happy to give it to you.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ryanastiefes

  • Sr. Member
  • ****
  • Posts: 50
Re: IO Reinitialization
« Reply #14 on: December 05, 2022, 12:17:22 PM »
BobO,
I have a Wireshark trace that I emailed you.
The Wireshark capture's both TCP port 502 and UDP port 28784.
Note that towards the end of the capture I Re-Initialize I/O in order to bring Modbus Comms back up.