Host Engineering Forum
General Category => ECOMs and ECOM100s => Topic started by: MarkTTU on August 08, 2011, 01:11:16 PM
-
For the first time I'm making use of the ECIPSUP, ECWRMID, and ECWRNAM iBoxes. In the office and during testing in the office my code worked ok. Sometimes it took a while to get things set on the ECOMs, but eventually they would get set and life would go on smoothly.
I first ran into an issue when trying this out in our shop. I'd put new ECOM100s into my PLC (I'm using 3 in a 9 slot base) and when I powered up and the setup stage ran I'd occasionally wind up with an ECOM that had a red error light that wouldn't even respond to NetEdit. Through trial and error I found that if I let the ECOMs cool off a bit they seemed to work fine. Since it was ~120 in the shop that day I didn't worry about it too much as our customers never operate in those kinds of conditions.
Today one of our guys is setting up the first of these new products in the field. Its a 90 degree day and raining and he's running into the same problem so now I'm very concerned that the ECOM iBoxes may be doing something to corrupt my ECOMs.
I've attached the stage where I do ECOM setup. My program jumps to this stage and doesn't do anything until the stage is reset at the end.
Anyone have any advice?
-
First question - is it possible the PLC is being powered down? There was a customer who was "setting up" the CPUs by powering them up, downloading the program, switching to RUN, then powering down. The ECOM100s write this data to FLASH, and that takes time. Each "write" IBox takes time, so they all have to finish.
It's possible to corrupt your ECOM when configuring them either through NetEdit or via IBox instructions if you power down the module/base in the middle of writing to it.
Looking at your logic, it looks like you are doing this with multiple ECOMs serially, which is okay - just make sure you are not powering down anytime during this sequence.
How often does Stage S1 get run - hopefully just once at power up?
-
Its possible they were powered down, but they would have been powered for a while (a minute or two at least) before being powered down.
Once the red error light is on and the module doesn't show up in NetEdit anymore I found that after putting the module inside in the air conditioning for a few hours I could then "see" it with NetEdit and reload the firmware (it was v0).
S1 only runs at power up and no other stages are allowed to run until S1 is reset.
How long should S1 take at worst case?
-
How long should S1 take at worst case?
A long time if the FLASH has been written to Xthousands of times. Flash gets slower and slower. So if you write to it a million times, it will be REALLY SLOW (progress from seconds to minutes, etc.). If you can close-loop it, as long as S1 is ON, do not power down. Or, look at the ROM LED on the ECOM100 modules.
-
In the case of what happened today in the field they were brand new ECOMs so the FLASH should have had very few writes to it.
ROM LED on the ECOM100? Which one is the ROM LED?
-
My bad - there's no ROM LED on the ECOM100s.
-
No worries Mark.
I just changed my code to look like the attached and it sure is faster to get everything set... at least in my nice cool office. Is what I'm doing here any better or worse than what I was doing before?
-
The earlier version was clearer (less dependency on T0 in first version). The 2nd file you posted has lots of documentation - you may want to remove it from that post.
-
In the earlier version if one of the ECOMs didn't have success then they'd all have to be triggered again; I'm wondering if writing to the FLASH over and over again in a 1 second interval was overflowing a buffer or leading to excessive heat (I keep coming back to heat, but its all I've got). In the new version once a setting is written successfully I stop trying to write that setting and everything doesn't have to succeed in 1 second or risk being done again... at least that was my thinking behind the new version... think I'm running down the wrong path?
Thanks for the note about the file, I've edited it and replaced with one that only has the pertinent documentation.
-
A long time if the FLASH has been written to Xthousands of times. Flash gets slower and slower. So if you write to it a million times, it will be REALLY SLOW (progress from seconds to minutes, etc.). If you can close-loop it, as long as S1 is ON, do not power down. Or, look at the ROM LED on the ECOM100 modules.
That's good to know in a case like this where the intent is to write on every Program-run transition / power up. Is there a way to read the same registers and not write if they're already at the desired values?
-
We suspect it is not being caused by the PLC instructions. To test this, is it possible to disable running the EC* IBoxes and power up and see if the red light still comes on, on the "hot" units?
-
...and we would really love to hear a specific description of the lights while the units are coming up/failing. The sequence of LEDs can tell us a great deal about what might be happening.
-
My guy on the ground has already moved on to the next customer location (they won't actually be running for a few more weeks), but he will be back either later today or later this week. I'll get him to shoot a video using his phone of the ECOM lights as the PLC powers up. I'll also have him power up the ECOMs without S1 enabled.
He did try two new ECOMs he had with him and each of them wound up with their ERROR lights on and unresponsive after being powered up with the first program I attached.
He's also tried moving an ECOM that has an error light to a different PLC and the error light stayed on and it was still unresponsive to NetEdit. Taking an ECOM from this other PLC and putting it in the one that's been causing problems caused the "good" ECOM to stop responding to NetEdit and have its error light come on.
-
Hmmm...maybe that does sound like code after all...
The other info we haven't mentioned is that we are chasing a strange Ethernet power up issue on a 275, where it gets hung on power up when it is warm...never happens cold. We think it is related to the PHY chip, which happens to be the same one on the ECOM100. There was enough similarity that we were considering a possible common problem. We do not really think this is systemic, as we only see it on a single 275 CPU, so it may just be a bad PHY chip...but we have learned to be sensitive to failure reports since they have a habit of cropping up at 3 or 4 places at the same time.
-
Oh how I know that one... I had a software bug crop up in a core function of one of my products that had been on the market for 5-6 years before it reared its ugly head. Then of course it reared its head at several places within a few days of each other. We threw parts at the problem for a while before finding the bug in the code. How it had never been an issue before baffled us all.
At any rate, my guy is back and has shot two videos of the ECOMs at power up. One has the CPU in the base so my S1 code is being executed. For the other he just pulled the CPU out and powered it up with no CPU so S1 shouldn't have run that time ;)
-
Second video...
-
OK, my guy in the field replaced the ECOMs again with brand new ones and tried using the second code set I posted. It took a while (3-4 minutes), but eventually all three ECOMs were initialized and NetEdit could see them just fine. I did the same thing here in the office using three new ECOMs and it took about 4 seconds to be up and running. All other PLC cards are identical and the software is identical as well.
Not sure what the difference is between here and there other than 20 degrees...
-
Now wondering if the erase and/or programming time of the flash is affected by temperature. I know they get slower the more times they have been erased, but generally that take many, many write cycles to begin to affect them. We'll look into it.
-
Thanks!
At least now I know something to tell my guys... be patient ;)
Wonder if hammering the ECOMs with that timer is good or bad too... maybe I should wait for an error before trying to set the info another time ???
-
The first version just used a timer to "start the process rolling", and then each successive EC* Write was dependent upon the previous one finishing.
What do you mean by "hammering"?
-
The timer was self resetting so every second it would start the process over again.
-
Whoa! So if it didn't finish in time, it would start over? NOT GOOD! Just let it run until it's finished (S1 turns OFF).
-
I've found that many times I have to create a rising edge to those instructions several times before they "take." Maybe I should just use the error bit as a rising edge to try again...
-
You should be able to turn them all on with SP1 (if order does not matter) and then have the SGRST when ALL the Success bits turn ON. Otherwise, leave S1 ON and look at the error bits at the top of your program and report it as an error of some kind.
-
I think the issue is that these instructions DO take time. Assume they take at least 100ms each (could be more, depending upon the age of the flash). Just use the success/fail bits to tell you when you're done.
-
As I understand it these instructions occur only once at rising edge. I've seen them fail a lot which is why I resorted to essentially hammering them until they worked. Since the UI is a C-More connected via Ethernet there isn't any way to announce a failure... that and other than try again I'm not sure what one would do about a failure.
-
Yeah, they might not work if S1 isn't enabled and executed on the first scan. I don't have the internal logic in front of me, but there might be a STRN SP0 (not first scan) on the "enable", but maybe not. But re-executing them again and again will basically lock-up the ECOM100 module while it writes to FLASH again and again.
-
In some cases I think its been a matter of the code execution happening before the ECOM actually got powered up which is why I went with a timer. Early on when I was first testing these instructions I'd frequently run into times when I'd get neither success nor error. I assumed those times were times when the instruction fired before the ECOM was powered up, but that was an assumption and I know all to well what assuming anything does ::)
-
No, it's like you said - it's edge triggered, but it has to see the OFF and then ON. When they are used in a Stage, this makes it very difficult to edge trigger. One way to make this work is at the top of your program before ANY stage:
STR S1
OUT C0
So C0 would be OFF every scan S1 is NOT enabled. Then in the Stage logic for S1, use STRPD C0 (or actually just STR C0) which would cause all those instructions see the edge on the 2nd scan that S1 runs.
-
it's edge triggered, but it has to see the OFF and then ON.
That little jewel of info is VERY helpful! I'll try a couple of things tomorrow and see what happens with my reliability.
-
I just checked the code that's generated - it must see an OFF to ON transition. There are also issues if the IBox does not get scanned on first-scan, which can be an issue within a Stage. The Workspace register gets cleared on "first scan" for any IBox that has a "workspace" register. However, if this instruction exists within a Stage, but that stage is not active on "the first scan", then it's going to initialize in a strange state.
You may want to clear the Workspace registers "on first scan" at the top of your ladder for the IBoxes that are used within a Stage.
-
OK, the attached is what I've come up with that seems to work every time. I've got a 1s timer that essentially waits for the ECOMs to power up and let the ECOMs see an off to on transition. I've also got a 30s timer that will force the command with another off to on transition if it didn't work the first time. Thus far this code has proven reliable in testing. Anyone see issues with how I'm doing this?
-
OK, the attached is what I've come up with that seems to work every time. I've got a 1s timer that essentially waits for the ECOMs to power up and let the ECOMs see an off to on transition. I've also got a 30s timer that will force the command with another off to on transition if it didn't work the first time. Thus far this code has proven reliable in testing. Anyone see issues with how I'm doing this?
I would NOT do the 30s timer. Just check all of the success and error bits. If all the success bits are ON, it's succesful (even if it took 1 minute). Otherwise, if any of the error bits are on, you have a bad/missing module. If you want to speed it up, since you have 3 ECOM100 modules, you can configure these 3 simultaneously, not serially.
-
I've got them all setup to do it simultaneously. The only issue I have with not using the 30s timer (even if its 60s or 90s or whatever) is that sometimes neither the success bit nor the error bit comes on. In those cases I have to have a way to try again or my program will just hang.
-
That's fixing the symptom.
On first scan, clear all of the workspace parameters of the IBoxes that are used in S1.
STR SP0
LD K0
OUT V711
OUT V713
OUT V715
OUT V717
OUT V721
OUT V723
OUT V725
OUT V727
OUT V731
OUT V(see my ECRDMID below)
Then do something like READING the Module ID and retry that until it works in Stage 100, THEN jump to S1
SG S100
TMR T100 K10
STR T100
ECRDMID
STR Success
JMP S1
STR Failure
RST T100 // just reset the timer to try reading again
This way, you're READING to see if the module is OK, not Writing. No flash writes. You could stick the workspace clears in S100 and do them unconditionally, instead of on first scan at the top of your code.
-
That's fixing the symptom.
Ya, I know, but its been the best thing I could come up with.
On first scan, clear all of the workspace parameters of the IBoxes that are used in S1.
...
You could stick the workspace clears in S100 and do them unconditionally, instead of on first scan at the top of your code.
I've always assumed I should leave those alone, but I guess not. Wouldn't weird things start happening if I'm clearing those VMEMs every scan?
Then do something like READING the Module ID and retry that until it works in Stage 100, THEN jump to S1
This way, you're READING to see if the module is OK, not Writing. No flash writes.
I've done that in another program. In that one I read the module ID as set by the DIP switches and then set the appropriate IP address. Even there I still have the issue; even after reading the module ID sometimes I get neither success nor error when writing an IP.
Its almost like sometimes the backplane drops some data transfer that was supposed to make it from the 260 to the ECOM or vice-versa. ???
-
marksji, OK, but if you pile up writes to the flashROM, then you'll have a worse thing happen... the red light you were experiencing before. How about if it doesn't work after 30 seconds (or perhaps longer) that instead of retrying automatically, that you set some kind of error flag, let the "operator" know initialization of the ECOM100 didn't work as expected and manually chose what to do. What do you think?
-
The key issue is that the workspace parameter of these IBoxes are not being executed on the first scan (SP0) because they are used in Stages. Hence, their "old" values are there whenever you go back to RUN mode (after a new download, program to run transition), meaning that whatever IBox thinks it "had the token" (or whatever state it was in), will continue to think "it has the token" when it "starts" executing, when it really doesn't, and will be waiting for a response in ITS error code parameter, but it will never see it because it really doesn't have the token, and it will be "hung".
The IBoxes were designed to work with Ladder, but are not "stage aware". This is something we've done with MX and made many of these "IBox" type "native" instructions be VERY "stage aware" in MX. Stage awareness is something that must be designed up-front at the firmware leve. IBoxes are just Ladder Macros, so in 450/260/250/06/05-land, there are some limitations. In MX-land, it's gonna be GOOD.
If you do the RESET of the workspace registers in an S100 stage (with the READ MODULE ID IBox), that should "reset" your S1 IBoxes so they "never" hang (except for a bad or missing module).
-
Based on what I'm reading here, what if I just stick these at the top of my program and let them be what fires off my initial stage? From experience I know I'll have to wait for the ECOM to get powered up for a couple of seconds, but after that I could just wait. I think I could probably program something that's pure C-More where if the C-More sits on my "splash" screen for more than x seconds a message pops up letting the user know they need to power cycle the PLC...
-
ISG or at top of your program above all stages (but driven by a single, leading edge event), that code would work. When it's in a non-ISG stage or not part of the "main" ladder scan, then the IBoxes don't get a chance to re-set on Program->Run transition, so you end up with IBoxes initially in "strange states", i.e. "hanging".
-
So if inside my ISG stage (first stage in the ladder) I'm using SP0 to fire my ECOM setup stage (immediately follows my ISG stage) then I should be ok?
-
Yup, since it is below the ISG in your ladder
-
Cool. I'll do that then since its already almost written that way anyway. Easy fix.