News:

  • February 01, 2023, 07:04:27 AM

Login with username, password and session length

« previous next »
Pages: [1] 2

Author Topic: Hardware Watchdog Timeout  (Read 740 times)

Bolt

  • Hero Member
  • *****
  • Posts: 492
Hardware Watchdog Timeout
« on: May 31, 2022, 11:30:01 AM »
Over the weekend I had a BRX on 2.9.3 with a hardware watchdog timeout resulting in the PLC dropping to PROGRAM mode.  I have over a year's worth of entries in my Event Logs, and no other watchdogs to be found. Do you think I just unknowingly reached the 10 timeout limit over the last 5ish years?

Can you help me decipher the DST400 records to learn anything?
DST 400 806
DST401 1075183616
DST402 518
DST403 1075183616
DST404 1603
DST405 1802249
DST406 806
DST407 1835023
DST408 518
DST409 16918021
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #1 on: May 31, 2022, 12:01:21 PM »
We report the count in $WatchdogReboots (DST385). You can reset it from code. The purpose of it is to prevent a continuous reboot (from corruption, busted program execution, etc) from making it impossible to communicate with the PLC. By dropping into program, you have a much better chance of remote recovery.

All of the codes look normal, so the reboot didn't come during a transitional state. We have more codes for development, but we only leave the transitional state codes in product.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

Bolt

  • Hero Member
  • *****
  • Posts: 492
Re: Hardware Watchdog Timeout
« Reply #2 on: May 31, 2022, 12:29:50 PM »
Yeah, I've since thought about resetting the count if no change in a certain number of days, etc.  I have already added the count to my "PLC Restarted" email.
Logged

PLCwannabe

  • Hero Member
  • *****
  • Posts: 156
Re: Hardware Watchdog Timeout
« Reply #3 on: June 06, 2022, 12:58:49 AM »
I had the same issue happen yesterday on a plc controlling a milk tank. Plc unexpectedly went into program mode. I cycled the run/program/off switch and the 24vdc power supply, and it started working again. However after reading this thread I checked dst385, and it was at zero. The event log showed 4 or 5 watchdog timeouts in the last couple months since it was last cleared. Is there any more info to be gleaned from the attached screenshot?
 Is it ok to just clear dst385 every day?
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #4 on: June 06, 2022, 10:10:54 AM »
Quote from: PLCwannabe on June 06, 2022, 12:58:49 AM
I had the same issue happen yesterday on a plc controlling a milk tank. Plc unexpectedly went into program mode. I cycled the run/program/off switch and the 24vdc power supply, and it started working again. However after reading this thread I checked dst385, and it was at zero. The event log showed 4 or 5 watchdog timeouts in the last couple months since it was last cleared. Is there any more info to be gleaned from the attached screenshot?
 Is it ok to just clear dst385 every day?

That's odd. DST385 should be incrementing each time it watchdogs. It absolutely shouldn't be zero. Which is kinda beside the point...shouldn't be watchdogging either.

Seems like something else is going on there.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

PLCwannabe

  • Hero Member
  • *****
  • Posts: 156
Re: Hardware Watchdog Timeout
« Reply #5 on: June 07, 2022, 12:25:32 AM »
Does a power cycle perhaps clear dst385?
I've had a lot of buffer overflow errors on this specific plc. Could that have anything to do with this issue?
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #6 on: June 07, 2022, 08:40:38 AM »
Quote from: PLCwannabe on June 07, 2022, 12:25:32 AM
Does a power cycle perhaps clear dst385?
I've had a lot of buffer overflow errors on this specific plc. Could that have anything to do with this issue?

No. It's retentive.

We should be preventing overflows from breaking anything, but if there is something uncaught, it could definitely cause unpredictable behavior.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #7 on: June 08, 2022, 09:40:37 AM »
Quote from: PLCwannabe on June 06, 2022, 12:58:49 AM
I had the same issue happen yesterday on a plc controlling a milk tank. Plc unexpectedly went into program mode. I cycled the run/program/off switch and the 24vdc power supply, and it started working again. However after reading this thread I checked dst385, and it was at zero. The event log showed 4 or 5 watchdog timeouts in the last couple months since it was last cleared. Is there any more info to be gleaned from the attached screenshot?
 Is it ok to just clear dst385 every day?

Can you post a screen shot of the general info page with firmware and hardware revisions?
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

PLCwannabe

  • Hero Member
  • *****
  • Posts: 156
Re: Hardware Watchdog Timeout
« Reply #8 on: June 08, 2022, 03:07:13 PM »
Done.
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #9 on: June 08, 2022, 04:11:33 PM »
Quote from: PLCwannabe on June 08, 2022, 03:07:13 PM
Done.

There are definitely fixes in 2.9.x. We'll be dropping to 2.9.5 within a week or so. Might wanna update to that after we do.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

eman5oh

  • Full Member
  • ***
  • Posts: 45
Re: Hardware Watchdog Timeout
« Reply #10 on: June 27, 2022, 02:17:35 PM »
Any word on when 2.9.5 will be out?
Logged

JeffS

  • Sr. Member
  • ****
  • Posts: 67
Re: Hardware Watchdog Timeout
« Reply #11 on: November 15, 2022, 06:20:59 PM »
I have a PLC that seems to have watchdog hardware timeouts about twice a day.  I am on firmware 2.9.6. 

I saw this error "panic @008EBA6C: exit_hook" on the status page and figured it could possibly be contributing.  I have no idea what this error means though, can anyone shed some light on what this could mean?
Logged

brucek

  • Sr. Member
  • ****
  • Posts: 55
Re: Hardware Watchdog Timeout
« Reply #12 on: November 16, 2022, 08:32:52 AM »
Just and observation but in the second screen shot of the list of reboots it seems to happen every 11ish hours. Do you have a task or something that happens every 11ish hours?
Logged

JeffS

  • Sr. Member
  • ****
  • Posts: 67
Re: Hardware Watchdog Timeout
« Reply #13 on: November 16, 2022, 09:58:38 AM »
Nope, I don't see anything that would occur every 11ish hours.   I had thought it could be a comms issue that was causing this, but I have since disabled all the comms and it still giving watchdog reboots.
Logged

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5788
  • Yes Pinky, Do-more will control the world!
Re: Hardware Watchdog Timeout
« Reply #14 on: November 16, 2022, 01:42:24 PM »
It is a fault in the TCP/IP stack, apparently failing during some kind of termination. Never heard of it before and have no idea what could cause it. If it is an option, disconnect the Ethernet cable from the PLC. I'd like to rule out any external input as the cause.
Logged
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO
Pages: [1] 2
« previous next »