News:

  • October 13, 2025, 02:05:00 AM

Login with username, password and session length

Author Topic: Dealing with an ill-behaved Modbus/TCP client  (Read 2448 times)

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3601
  • Darth Ladder
Dealing with an ill-behaved Modbus/TCP client
« on: April 13, 2022, 02:01:42 PM »
I have an application where the facility is passing me jobs via Modbus/TCP (MC coil per job per recipe).   But, their app sometimes has the  habit of opening sockets, abandoning them, and opening more before communicating.   I increased the max client setting and reduced the timeout on idle connections, which has mostly resolved the issue, but I think from time to time they may still use up all the connections.  I say "I think" because they have some other issues as well so I can't say with certainty this issue is still in play.

I've added the client count to the HMI so the customer can see if that issue is occurring, but is there a way I can shed connections from ladder?  I don't see anything relevant in DEVWRITE.  If so, is there a way to distinguish which connections get the axe, so I don't kill the HMIs?
« Last Edit: April 13, 2022, 02:03:21 PM by Controls Guy »
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #1 on: April 13, 2022, 02:10:51 PM »
Anyway to turn it around and have you to ask for the data periodically?

franji1

  • Bit Weenie
  • Host Moderator
  • Hero Member
  • *****
  • Posts: 3761
    • Host Engineering
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #2 on: April 13, 2022, 02:22:23 PM »
Stick ModbusTCPServer.ActiveSessions in a trend view and it should NOT continuously creep up - unless it is asking for TCP connections at a rate faster than your Idle timeout value.

OR

The client software may be so mis-behaved, it may try to re-open ALL "current" connections after they timeout, which would explain why the Idle timeout is effectively useless.  Their Client software: "You bad TCP Server!  You hung up on me 3 times!  No, wait, 4 times!  Well by gollee, I'm gonna call you back 4 times!  Maybe even 5, just to make sure I don't lose the connection."

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3601
  • Darth Ladder
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #3 on: April 13, 2022, 02:31:32 PM »
Anyway to turn it around and have you to ask for the data periodically?

I don't think so, because of the way the work comes to them.   Interesting thought though.  Also, it's purchased software and the vendor isn't the greatest at support and/or bug fixes.
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3601
  • Darth Ladder
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #4 on: April 13, 2022, 02:34:58 PM »
Stick ModbusTCPServer.ActiveSessions in a trend view and it should NOT continuously creep up - unless it is asking for TCP connections at a rate faster than your Idle timeout value.

OR

The client software may be so mis-behaved, it may try to re-open ALL "current" connections after they timeout, which would explain why the Idle timeout is effectively useless.  Their Client software: "You bad TCP Server!  You hung up on me 3 times!  No, wait, 4 times!  Well by gollee, I'm gonna call you back 4 times!  Maybe even 5, just to make sure I don't lose the connection."

I doubt that it's gradually creeping up.    I have the timeout at 5 seconds, max connections at the max of 16 and that has mostly (perhaps totally) resolved the issue.   It may be that this issue is totally resolved and the issues they've had have been others, I know at least some of their failures to communicate were due to queueing issues internal to their software.

This is more just for a belt-and-suspenders response to the saturation issue if it does ever occur again.   Is there a way to force-dump all the connections conditionally from ladder?
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.

franji1

  • Bit Weenie
  • Host Moderator
  • Hero Member
  • *****
  • Posts: 3761
    • Host Engineering
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #5 on: April 13, 2022, 03:54:23 PM »
This is more just for a belt-and-suspenders response to the saturation issue if it does ever occur again.   Is there a way to force-dump all the connections conditionally from ladder?

No.  As a TCP Server, it would be very difficult to specify WHICH connection to "hang up" on.  All of them (Remember, Modbus/TCP Server is just ONE POSSIBLE service)?  So then, all current clients of all services: Web Server, Modbus/TCP Server, EtherNet/IP Server, Custom)?  Maybe specify by server port # i.e. service (e.g. 502 to hang up on ALL Modbus/TCP clients?  80 on all Web clients, etc.).  Not sure how you could specify a SPECIFIC ill-behaved client.

I think the current timeout mechanism is fool-proof enough.  If that doesn't work, there's always REBOOT instruction  ;).
« Last Edit: April 13, 2022, 04:02:35 PM by franji1 »

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3601
  • Darth Ladder
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #6 on: April 13, 2022, 07:32:13 PM »
I thought that might be the case.   It makes sense, especially since all you have to do to reclaim those connections is disconnect the bad client long enough for all his connections to drop.
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #7 on: April 14, 2022, 10:53:27 AM »
How much flexibility is on their end? Do they have any settings for heartbeat detection or retries?

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3601
  • Darth Ladder
Re: Dealing with an ill-behaved Modbus/TCP client
« Reply #8 on: April 14, 2022, 12:57:06 PM »
How much flexibility is on their end? Do they have any settings for heartbeat detection or retries?

Good question. I'm not sure, I'll look into it some more.  The customer doesn't know absolutely everything about it cause it's a purchased package, but they can at least ask the vendor and hopefully get something.

And at this point it may or may not even be a problem any longer, it might just be the software's other issues.  The symptoms aren't 100% determinative.
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.