News:

  • September 24, 2026, 12:29:43 PM

Login with username, password and session length

« previous next »
Pages: [1] 2 3

Author Topic: Power Outage Issues  (Read 53733 times)

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Power Outage Issues
« on: May 11, 2023, 11:50:32 AM »
Looking for general guidance.

I had a power outage this morning. Power was out for about 45 minutes, BRX PLC lost power during this time. At time of outage, controller was in RUN mode and was actively running piece of equipment. After power was restored, controller was in PROGRAM mode. I can't figure out why it didn't go back to Run mode when power was restored. The BRX has 6 remote EBC100s. I checked the options and the BRX is set to allow RUN mode even if remotes aren't connected. PLC log shows power off event and power on event. Any thoughts?
Logged

jwbaker3

  • Hero Member
  • *****
  • Posts: 144
Re: Power Outage Issues
« Reply #1 on: May 11, 2023, 04:13:11 PM »
Did the power outage cause a surge? electrical noise and surges have cause problems like this for me in the past. (not just BRX AB also) I always install a power line filter and surge protectors with only the plc connected to them. Could be the problem maybe??

JW
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #2 on: May 11, 2023, 04:24:12 PM »
Quote from: RBPLC on May 11, 2023, 11:50:32 AM
Looking for general guidance.

I had a power outage this morning. Power was out for about 45 minutes, BRX PLC lost power during this time. At time of outage, controller was in RUN mode and was actively running piece of equipment. After power was restored, controller was in PROGRAM mode. I can't figure out why it didn't go back to Run mode when power was restored. The BRX has 6 remote EBC100s. I checked the options and the BRX is set to allow RUN mode even if remotes aren't connected. PLC log shows power off event and power on event. Any thoughts?

Nothing else in the log?

Newest version of the firmware should be giving more info on the startup modes transitions, but earlier versions didn't give as much. Generally it's a bad battery resulting in loss of retentive state, I/O config change, or the Ethernet master issue you referred to.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #3 on: May 11, 2023, 04:55:59 PM »
Quote from: BobO on May 11, 2023, 04:24:12 PM
Quote from: RBPLC on May 11, 2023, 11:50:32 AM
Looking for general guidance.

I had a power outage this morning. Power was out for about 45 minutes, BRX PLC lost power during this time. At time of outage, controller was in RUN mode and was actively running piece of equipment. After power was restored, controller was in PROGRAM mode. I can't figure out why it didn't go back to Run mode when power was restored. The BRX has 6 remote EBC100s. I checked the options and the BRX is set to allow RUN mode even if remotes aren't connected. PLC log shows power off event and power on event. Any thoughts?

Nothing else in the log?

Newest version of the firmware should be giving more info on the startup modes transitions, but earlier versions didn't give as much. Generally it's a bad battery resulting in loss of retentive state, I/O config change, or the Ethernet master issue you referred to.


Attached is the event log. As best I can tell, it looks like retentive memory for some UDTs was lost as well. I've been trying to figure out today if the UDT loss was a result of the power outage or some programming thing and I want to blame it on the power issue. A retentive non UDT bit was off as well which I would have expected to be on. I'm running OS 2.9.6. $BatteryLow bit is not on.

« Last Edit: May 11, 2023, 04:58:51 PM by RBPLC »
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #4 on: May 11, 2023, 05:36:53 PM »
Quote from: RBPLC on May 11, 2023, 04:55:59 PM
Attached is the event log. As best I can tell, it looks like retentive memory for some UDTs was lost as well. I've been trying to figure out today if the UDT loss was a result of the power outage or some programming thing and I want to blame it on the power issue. A retentive non UDT bit was off as well which I would have expected to be on. I'm running OS 2.9.6. $BatteryLow bit is not on.

According to the log the PLC was already in program mode.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #5 on: May 11, 2023, 09:28:26 PM »
Quote from: BobO on May 11, 2023, 05:36:53 PM
Quote from: RBPLC on May 11, 2023, 04:55:59 PM
Attached is the event log. As best I can tell, it looks like retentive memory for some UDTs was lost as well. I've been trying to figure out today if the UDT loss was a result of the power outage or some programming thing and I want to blame it on the power issue. A retentive non UDT bit was off as well which I would have expected to be on. I'm running OS 2.9.6. $BatteryLow bit is not on.

According to the log the PLC was already in program mode.

Is there a scenario where the log file would be incorrect? The controller/equipment had been running continuously since 4/28/23 in RUN mode until the power outage.
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #6 on: May 11, 2023, 09:44:11 PM »
Quote from: RBPLC on May 11, 2023, 09:28:26 PM

Is there a scenario where the log file would be incorrect? The controller/equipment had been running continuously since 4/28/23 in RUN mode until the power outage.

I'll never say never, but I don't know of any way to get to run mode without creating a log entry.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #7 on: August 09, 2023, 11:09:19 AM »
Same issue. This morning, my facility power very briefly went out twice. The attached log file time is correct. After the first outage, I noticed (through other means) that the controller didn't appear to be running. Before I could debug, the power went out again. After the second outage, I got a chance to remote in and look at the log file. This is similar to the first post about this issue. At present time, I haven't done anything with the controller other than log in to look at the Sys Info Log. It's currently in Program mode.

Edit: I just noticed from the first post that the log file entries appear to have been overwritten. I don't remember doing anything to clear the System Log the first time around.
« Last Edit: August 09, 2023, 11:20:27 AM by RBPLC »
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #8 on: August 09, 2023, 11:23:16 AM »
What is your hardware version?
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #9 on: August 09, 2023, 11:33:25 AM »
$HardwareRev = 16696.
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #10 on: August 09, 2023, 11:43:44 AM »
I'm a bit puzzled. Everything I can identify that would go from from run to program as a result of a power cycle should be leaving something in the log.

Can you dump DST400-409 into a Data View and post a picture of that?
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #11 on: August 09, 2023, 11:57:04 AM »
Values attached.
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #12 on: August 09, 2023, 12:23:31 PM »
Those values are a push down stack of the current state and the last 4 power cycles. Basically at startup we grab the top 8 values (4 states) and copy them to the lower 8 values. The top 2 values contain the current state. During initialization, mode changes, etc. the PLC OS writes values after each operation. Those values allow us to know the last initialization state, thus have a pretty good idea where the PLC is or failed at before the restart.

The 1400 values indicates the PLC was in program mode when it was power cycled.

The 608 value got reused (that shouldn't be, but ::) ) and the PLC was either validating the program during goto run, or was in program mode and was updating the sys config flash. One might have left the PLC flash corrupted (incomplete write to flash), and the other would have caught the PLC in transitional limbo which wouldn't corrupt but wasn't a good spot to reboot.

The 806 value is the last state published on a normal transition to run mode.

So the PLC appeared to be running, then power was cycled and while initializing or writing flash was power cycled again, and after that was in program mode three times.

Per the log and those post mortem registers I see no evidence that the PLC has been in run more for a while. If retentive memory was wiped that would be in the log. If retentive memory wasn't wiped, the log and those post mortem states should be believable.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

RBPLC

  • Hero Member
  • *****
  • Posts: 586
Re: Power Outage Issues
« Reply #13 on: August 09, 2023, 12:56:57 PM »
I don't know what to tell you. After the first instance of this I went in and fixed the UDT memory that appeared to have been dumped. It was absolutely in Run when I left it. We have been running this piece of equipment almost continuously since May when this first happened. It was running this morning as well. Y8 of the BRX is tied into main control power for the piece of equipment. If that output is not on, the piece of equipment doesn't run. Also, WY0 is outputting a process variable that would cause erratic operation if the BRX wasn't running. I can't explain the log file, just that something isn't correct.   
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6176
  • Yes Pinky, Do-more will control the world!
Re: Power Outage Issues
« Reply #14 on: August 09, 2023, 01:14:37 PM »
Do you have more than one PLC on the same network? Is it possible that we aren't looking at the correct one? Is it possible that an IP address has gotten reused?

The only way 1400 gets into that post mortem state is in program, going to run leaves different values, and the log shows no mode transitions to either mode. The log and the post mortem appear to agree and I've never seen a case when they didn't. I know of no possible way the PLC could be running with both of those at the state they are at. That doesn't mean it isn't, just that it would require multiple failures in different systems for that memory state to be present when the PLC is in run mode and I've never seen such a thing. That suggests another explanation.

Just for reference, what are the log entries prior to the three power cycles?
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO
Pages: [1] 2 3
« previous next »