IO Error causes and fixes

[CReese]

Hello,

I noticed in an otherwise 100% operational system that the CPU was starting up in Program mode. When I got into the PLC, it was displaying errors on a couple cards, a F2-08AD-2 and a F2-04AD-2. The error says that the channels have failed, when they are clearly still functional.

What are the possible causes for this error, and is there anything to do except replace the cards?

Thanks,
Colin

[plcnut]

Re-seat all the cards in the base.

[CReese]

Definitely tried with the offending cards. Will give it another shot. Couldn't hurt.

[plcnut]

Also make sure you have the latest firmware in the CPU.

[jwbaker3]

Are you using all of the channels on the cards? I had an error on an analog input card when I left one of the input channels unhooked. Don't know if that may be your problem but the error did not make the CPU come up in program mode. I saw it when updating the program.

JW

[BobO]

Do-more CPUs come up in program mode when there is a change to the I/O configuration, which could be due to glitchy backplane connection or module. You might check $IOVerifyLastTO (DST27) to see if it contains anything other than 0. If so, it means that it got the wrong answer for at least 1 scan while verifying the module.

They can also come up in program mode if they have watchdogged repeatedly. If the CPU has watchdogged it will bump $WatchdogReboots (DST385). When it hits 10 (I think) it will drop into program mode on restart. This is to prevent the CPU from winding up in a unrecoverable crash loop.

Failed channels and 24vdc are status bits read from the module. The CPU has no control over it...we just report it.

[CReese]

This is interesting. I am getting the error:

Master 0: DL205 Local I/O Master
Base 0: DL205 Base
Slot 0: 8 AI [X0-7/WX0-7]
The following channels have failed on this module: 1, 2, 3, 4, 5, 6, 7, 8

I get this error any time I plug an F2-08AD-1 into any base. I've swapped bases, bought new cards and inserted them with no connectors, and still the error.

This error alternates with the "There is an error with the module error"

These cards were in slots 2 & 4. With no point terminations connected, I swapped 2 & 3. This gave errors on modules 2,3, & 4. I then swapped them back, and showed errors on 2, 3, 4, & 5. Interestingly, while these errors were reported, the PLC did come up in Run when I power cycled it repeatedly.

This seems like some sort of firmware compatibility issue. Any ideas which direction to go?

Colin

[MikeS]

the 4-20mA input modules will report broken transmitter alarms for any channel that is enabled and is NOT connected to a current source. Assuming all 8 channels are enabled, (the default jumper setting), when you plugged in modules without the connectors you would get 8 broken transmitter alarms.

if you look at the I/O Mappings page of the System Config, you'll see there are 8 X inputs along with the 8 WX inputs for that module. The X bit associated with each channel will be ON when a broken transmitter is detected for that channel.

if this is the problem you're having, in Do-more Designer you should be seeing "Warning" on the status bar with a yellow background. Click on warning to open the System Information dialog. You should then see the "Open I/O System View" button also with a yellow background. click that button open I/O System View which will in turn display all of the current errors and warnings for all of the I/O modules. That's where you'll see the channels that are reporting broken transmitters.

[CReese]

That is all fine and good, but my CPU should not tell me that it is a failed channel input. This seems to imply a hardware problem, not a sensor problem. This is a VERY important distinction that should be clear from the error message.

Finally, this sort of error should not cause the CPU to not start in the mode it was in when it was powered off. I'm not clear that it ever was, now that it's starting up with no problems. Another lost question mark.

C

[BobO]

The CPU reports what the module tells it. The bits we are reporting are channel failure bits. Not sure what else to call them.

Channel fail bits will not prevent the CPU from returning to RUN, but general module failures do and any change to the I/O config does. The rules are well defined and well tested. If there is a deviation from those rules we certainly need to know about it and will be happy to fix it.

[CReese]

'channel signal loss', 'channel disconnected', 'channel current loss', or even 'channel current fault' are all more accurate.

I understand if this is a general condition that could denote actual hardware failure at the card, but as it exists it does not clearly indicate the range of problems that could exist. When I see "channel has failed", this in no way indicates that I could simply have it disconnected. Sure, RTFM, but I think this could be better indicated in DMD.

C

[MikeS]

the term 'channel failure' comes the module manufacturer's documentation, and yes, it is a very generic term.

having the entire module fail - while still reporting it's module id properly - is very rare. these modules have a single A-to-D convertor that gets multiplexed across the number of channels, so again having a single channel fail in the module hardware - while having the other channels work - is also extremely rare. what typically happens when the A2D fails is you always get 0's for channel values but no broken wire indications.

by far, what happens to generate these errors is a broken wire on one or more of the channels.

All that said, we will be changing the "channel failure" indication to "broken transmitter" in the I/O System View to hopefully point more directly at the problem being the external wiring and not the module itself.

[CReese]

Do-more CPUs come up in program mode when there is a change to the I/O configuration, which could be due to glitchy backplane connection or module. You might check $IOVerifyLastTO (DST27) to see if it contains anything other than 0. If so, it means that it got the wrong answer for at least 1 scan while verifying the module.

They can also come up in program mode if they have watchdogged repeatedly. If the CPU has watchdogged it will bump $WatchdogReboots (DST385). When it hits 10 (I think) it will drop into program mode on restart. This is to prevent the CPU from winding up in a unrecoverable crash loop.

Failed channels and 24vdc are status bits read from the module. The CPU has no control over it...we just report it.

Can you help me understand what 'watchdogged repeatedly' means? I do see that watchdog reboots happened ten times before it came up in program mode.

Thanks,
C

[BobO]

'Watchdogged' = watchdog timer rebooted a crashed controller. 'Repeatedly' = more than once, in this case, repeatedly = 10. Since your counter is at 10, it crashed and restarted 10 times, which is why it dropped into program mode. A crashing/rebooting PLC can't do comm, so dropping back to program mode improves your chances of being able to fix the issue from off site.

There were some networking issues fixed several firmware revisions ago. I would update and clear the reboot counter. If you don't ever want it to drop into program, just clear the reboot counter in you first scan code.

[CReese]

So the obvious next question is: what is causing the crash in the controller? How do I debug?

Colin