overworking a DL06 H0-ECOM100 with modbus tcp

[harncw]

h0ecom100_4_0_227
boot100_4_0_165

I posted my issue over on http://forum.automationdirect.com/showthread.php?p=33552#post33552
as I was unaware of this forum.

Currently in a test environment I'm able to get in the state where ECOM100 only replies with exception code 04.
Although earlier firmwares(224) did cause an error light on the ECOM100 I'm not seeing that since I updated my firmware. (224 to 227)

Basically what I'm doing is sending out way to many modbus tcp requests from a test applicaiton I wrote.
I'm trying to establish the boundaries of the device.

I seem to be able to get the device into a state where all I can do is powerclear the PLC.

any ideas?

are there any specs on Host's implementation of Modbus TCP, which I gather is relatively new functionality?

Granted I probably should not send so many Modbus TCP requests to the PLC at one time, but allowing it to get into this unrecoverable state is not good.

[harncw]

I've troubleshooted it down to only occurring when I send function 16 (Write Multiple registers) repeatedly.

[Greg]

What Modbus address are you writing to, and how many registers? Or does that even matter?

[harncw]

at the moment it's looking like

wireshark is showing me sending the plc a malformed packet
right before the ECOM100 stopped replying correctly

I need more time testing this, will post more when I know more.
Currently malformed packet means little to me.

It seems reproducible

All I'm sending are 2 identical requests over and over again.

Code:

0000   00 01 00 00 00 87 01 10 02 00 00 40 80 00 00 00
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00

0000   00 02 00 00 00 87 01 10 02 40 00 40 80 00 00 00
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00

At a rate of 200 mbtcp requests per second (Function 16's for 64 words a total length of 135 bytes) , it fails after a few minutes.
I tested roughly 10 minutes before failing one time and 2 minutes another.

[harncw]

What Modbus address are you writing to, and how many registers? Or does that even matter?

would a wireshark log be of any interest?

[Greg]

Yes! Please send to support@hosteng.com.

[Greg]

harncw, I received your e-mail and sent you one in return. Basically, I reviewed reviewed them and I found some very odd things happening from the PC. For instance, in the "f16 only cleaner example.PCAP" file, all is well until Telegram #812. At this point, the PC sends one telegram with 4 embedded Modbus TCP requests. Then immediately follows that telegram with another (#813) of the same. It is at this point that the ECOM100 starts sending Exception 0x04 codes back. Obviously the ECOM100 is complaining about the malformed packets. I don't, however, see the ECOM100 "locking up", but rather just getting preoccupied with sending back Exception codes.

In other words, it may simply be that the reason the ECOM100 appears to lock up is simply because he gets so far behind sending Exception codes. The ECOM100 is keeping pace with the rate at which you are sending packets, but he loses pace when he has to start sending Exception codes. It is very possible, if you stopped the PC from sending, that the ECOM100 could get caught up on the Exceptions he needs to send, and then simply start working again. But, I sent the traces to the engineer for him to review and I’ll let you know if he has anything further.

But until then, check your application on the PC and see if you can stop him from embedding 4 Modbus TCP requests in one telegram.

[Greg]

harncw, the ECOM100 engineer said he was going to spend some time looking at this because the ECOM100 should be able to handle the multiple Modbus TCP requests in one packet. For example, we handle the 3 in Telegram #767 and the 2 in Telegram #773. And Telegrams #812 & #813 that I pointed out actually are incomplete, and should generate an error. But he wants to try to understand why the ECOM100 seemingly gets stuck in a mode where he is always returning an Error 4.

I'll keep you posted.

[harncw]

Agreed It's not "locking up", it just seems unable to successfully respond to further ModBusTCP request.
I have in fact tested giving the PLC a "rest" in order to allow it to recover, to no avail.

It seems that in order to reproduce this issue, I have to bombard the PLC with way too much traffic.
I'm not sure what is causing multiple modbus requests for a given TCP packet, but it's agreed that that issue is on my side.

Code:

6) A TCP frame must transport only one MODBUS ADU. It is advised against sending multiple MODBUS requests or responses on the same TCP PDU

as per MODBUS MESSAGING ON TCP/IP IMPLEMENTATION GUIDE V1.0b

I can easily change my test program to not send so much at one time or put a pause in there after each packet send and/or figure out what is combining my individual packets...  In fact add a 10ms pause after I do my packet send, seems to have corrected this issue easily.

My overall concern is the ability of the PLC to recover from an "illegal" request.
IMHO the ECOM100 should be armored to ignore illegal requests, not go on vacation when an illegal request occurs.

so I started to type this up then you had your second reply...

we are both on the same page and thank you for your efforts

[Greg]

We are currently testing firmware which we believe fixes this issue.

The main issue here was multiple requests in a single packet where the last request was split across two packets. The ECOM100 supports multiple requests per packet as long as all the requests are complete. The new firmware will return an error code for the request that was split and ignore any bytes that don't form a valid request. In the process of fixing this, we added code to validate all incoming requests to make sure the request was valid and the data was all there.

This firmware will probably be fully tested and released 2nd week of August 2010.

[Greg]

harncw, ECOM100 firmware addressing this issue is now released and available for download (NetEdit --> File --> Live Update). Let us know if there are further issues or not.

Firmware releases:
H0-ECOM100 v4.0.269
H2-ECOM100 v4.0.1473
H4-ECOM100 v4.0.1473

[harncw]

It's looking great so far on my burn test

Hey thanks for turning this around so promptly!

[WRT]

Thanks guys for figuring this out.  So far the results of your efforts (269) seem rock solid.  I use over a hundred of these cards across thousands of miles of Arctic tundra, so any kind of lock-up is difficult and expensive to deal with.