Host Engineering Forum
General Category => Do-more CPUs and Do-more Designer Software => Topic started by: jcottrill on December 27, 2017, 11:59:54 PM
-
I hope you all are enjoying a break this time of year. I have the following challenge for your consideration when you get back in the office. Sorry for the length so the TL;DR version is I am looking for auto serialized output of user data types to JSON into a given memory block of strings.
Background...I am using the BRX PLC as a generalized platform for automated test equipment to make it easier for our young Test Engineering team to turn out functional testers with repeatable results. The base programs and tasks control the general flow for things like validating operators, work orders, products etc. HMI functionality is provided via a modified version of plcnut's web server. The primary page displays the devices under test (DUTs) and is updated periodically with AJAX requests to the PLC. These base programs, tasks and supporting data structures will be considered sacred and protected from being modified by the Test Engineers. They will be able to add in additional data structures, programs, etc to round out their testers but the core should remain untouched to ensure a base level of consistency.
So far all is going extremely well, at least through simulation. This is my first big project with PLCs so there is still a lot of learning along the way. Luckily Host Engineering has built an amazing community so that process has been pretty painless.
My next hurdle that has me spinning a bit is if it will be possible to generalize the DUT reporting back to the HMI and/or final log collection server. Lets assume the Engineer has defined the following fields in the DutResult user data type.
DutResult
DutID unsigned byte
T1_Start dword
T1_End dword
T1_Expected real
T1_Actual real
T2_Start dword
T2_End dword
T2_Expected real
T2_Actual real
I'd like to be able to generate a JSON record of a given DutResults memory block location (ex. DutResults0-9) without explicitly building the string(s). Below is an example of the conversion.
{
"DutId": 0,
"T1_Start": 234123432,
"T1_End": 234123450,
"T1_Expected": 5.00,
"T1_Actual": 5.01,
"T2_Start": 234123451,
"T2_End": 234123455,
"T2_Expected": 1.500,
"T2_Actual": 1.495
}
I am looking for something similar to using Newtonsoft.Json.JsonConvert.SerializeObject(someObject) in C#. By dynamically generating this I could avoid the need to teach our Test Engineers the finer points of JSON and also wouldn't have to worry about missing a field in the output. We could just focus on building testers.
Fundamentally, I think everything exists on the PLC in terms of the documentation, nicknames, etc. What I am not sure of is the ability to query and iterate through the fields of a structure, get the names of the fields and then get the associated values. I think this could be useful for handling web based requests for data and for logging data to external endpoints without the need for any middle-ware. I'd like to hear the thoughts of the team and community on this to see if there is any merit or if you have some better thoughts. Thanks in advance and keep up the great work!
-
I am not sure that I totally understand what you are wanting, but I have built JSON strings very similarly to how I build the csv strings in the latest sample code for the Do-more webserver. http://forum.hosteng.com/index.php/topic,1117.15.html
-
Hey plcnut,
To start, thanks for your work on the Web Server. It saved me quite a bit of time and allowed me to get to the meat of my project quicker. When I get some free time I will submit my modifications such as replying with 405 messages for missing pages instead of just redirecting to a default page. Sometimes that auto redirect can be confusing to people.
I took a look at your updated version and see what you are doing with your CSV data. This is similar to how I have handled JSON outputs for concrete data and could just as well do so here but I am trying to add in some mistake proofing for my teammates.
I will try to use your example to better illustrate what I am looking for. In your example you appear to be collecting logs and sending them off in mass whereas I will be doing more of a real-time stream but it should be close enough. I will also be using a single memory block of a user structure instead of multiple memory blocks of discrete values. The struct is the really the key to what I am proposing here.
Lets say I am the manufacturer of the freezer/machine that you are monitoring in your PLC code. I want an Engineer to do some functional or reliability testing on 10 of these machines and ultimately log the results. I provide them with the base PCL code that includes the following structure that defines the base information we must report back. This structure is then mapped to a memory block to track this data for all 10 machines. Also in the PLC code is a logging function to provide this data in JSON format as needed.
(https://i.imgur.com/zFp0yG0.png)
(https://i.imgur.com/4HlODqn.png)
The Engineer then adds the fields he needs (LastSampledTS, WaterPressure, PumpOn) to the structure for his specific situation as shown below.
(https://i.imgur.com/fZMqxQM.png)
The problem is, that the Engineer must remember to also add these additional fields to then logging code to ensure it is passed on. Teams can implement checks and validation to ensure this is correct but it is still dependent on people to get it right. I don't know about you, but in the crunch we silly humans sometimes miss some of these smaller details as we spend most of our energy focusing on safety, stability and overall functionality.
Instead of creating a manual concatenation:
DUTs[v_CurrentDut].DutId + "," + DUTs[v_CurrentDut].StartedTS + "," + ... + DUTs[v_CurrentDut].PumpOn
I am proposing a method to convert a given structure element directly as shown below. Using psuedo code instead of ladder logic to try to condense. It would reply on needing access to the program documentation to get the friendly names of the fields in the struct. With a tool such as this, the structure could be modified but you'd never need likely never need to touch or think about your logging functions as it would iterate through all the fields of the struct.
function Main {
...
dutCSV = StructToCSV(DUTs[v_CurrentDut]);
dutJSON = StructToJSON(DUTs[v_CurrentDut]);
...
}
function StructToJSON(struct) {
var result = "{";
var isFirst = true;
foreach (field in struct) {
if (!isFirst) {
result += ",";
isFirst = false;
}
result += "$"" + field.Name + "$":" + field.Value;
}
result += "}";
return result;
}
-
Not sure how many out there will find this useful but figured I would share it. For anyone out there that is logging user defined struct data to an external destination and need to do so using JSON, I have created a tool to auto create the string you need to paste into your STRPRINT block in the designer. This is a bit overkill if your struct only has a few properties but may help save your eyes if you have a dozen or more properties. You just need to provide it the name of the memory block or heap item representing your data. You can also provide the name of an iterator if using a memory block to use this inside a for-loop or set of stages. Finally past the contents of an export from a data view for the object and click convert. From there you can copy and past the results into your STRPRINT block.
https://jsfiddle.net/JCFiddle/ojcwd5bc/
(https://i.imgur.com/57QpUUW.png)
(https://i.imgur.com/0ccgLa0.png)
-
Old bump...
Since we now have MQTT/IoT, the subject of JSON has come up. We have had short discussions on how we might approach that and already have a decent idea of how to solve it. If some of y'all that use JSON could give me a sense of what your typical records look like and how you use it, it would be helpful to coming up with a good set of tools.
-
Nobody got thoughts on JSON? Fine. You'll get what we give you!
Aside: I had an old barber who used to tell me that when I attempted to tell him how I wanted my hair: "Well, I reckon you'll get what I give you." Indeed. Fortunately, while he was kinda grumpy, he was an excellent barber. Rest in peace, Zeke.
We were also talking about adding functions for HTTP GET and POST. Our thinking is that HTTP, JSON, and MQTT are all complementary pieces of an evolving vision to tie Do-more PLCs to enterprise data systems. Anything you could share with us about how you do these things now, or how you would do these things if you had direct support in the PLC, would be helpful.
-
Yes,
But the difference between a bad hair cut and a good haircut is about 6 weeks.
-
The Post and Get functionality would interface nicely with PHP. Go for it.
-
Hey BobO,
Sorry for the late reply. Haven't been keep an eye on the site much in the last week or so. Primarily I am serializing my user defined data types but there are some cases where I need to sneak in a string storage location with that data. Manually building the JSON string when there are 20+ fields in the record (or even 10 for that matter) is a bit of a pain and makes my eyes burn :) A slightly better approach would be if you could provide a table like the data mapping and a few other tools where we could provide the desired JSON attribute name and a memory location. Not sure how to handle nested JSON structures though unless you could reference a JSON map. Some of these JSON strings get pretty large so managing the strings could be an issue. Hopefully this makes sense and isn't end of day rambling.
-
Well, here's what we've implemented.
JSONBUILD is a new instruction lets you built a JSON record of fields or array items, whose values can be objects, strings, numeric values, or Booleans. It's a table instruction, with up to 50 entries.
In the JSONBUILD editor, you bring up a dialog where you can then add a group of JSON fields by simply referencing a single structure. It adds all the proper fields of that structure to the table. The attachments are an example of a UDT based on your DUTs. I created a block of 10 DUTs called MyDUTs and told it to create a JSON record based off the data in MyDUTs[V1] and all its "useful" fields. See the first screen shot for the JSONBUILD editor's sub-dialog where you add the record for the fields of a specific structure element (e.g. MyDUTs[V1]).
You end up with the following in the JSONBUILD instruction (see the 2nd attachment), which you can use MQTT or other instructions (some new). The JSONBUILD editor will let you add/edit more fields, not just limit you to the ones in the one structure.
For bit fields, note that you can tell the JSONBUILD instruction to treat the values as numeric (1, 0) or as JSON Boolean (true, false).
-
This looks pretty slick!
-
Where is this instruction set? I am trying to implement MQTT into Inductive Automation with the BRX platform. Has this been rolled out or still in process? Also, any thoughts are creating direct drivers for the Do-More platform for Inductive Automation?
-
Where is this instruction set? I am trying to implement MQTT into Inductive Automation with the BRX platform. Has this been rolled out or still in process? Also, any thoughts are creating direct drivers for the Do-More platform for Inductive Automation?
MQTT was added in 2.3. JSONBUILD/JSONPARSE are coming out very soon.
-
Yes I see the PUB/SUB, but right now we can only send string values...which I am trying to figure out how to send numeric values (reals) by others posts on the forum...I'll keep working!
-
Yes I see the PUB/SUB, but right now we can only send string values...which I am trying to figure out how to send numeric values (reals) by others posts on the forum...I'll keep working!
Reals expressed how? As the raw binary data, or as a string representation?
As used with MQTT, strings are just buffers and can contain anything.
-
BobO in all fairness I'm new to trying the MQTT and learning more and more each day. What I am trying to do is a simple exchange and PUB R0 (which is just randomly generating numbers). From what I gather I'll need to be using some String instruction blocks to make this conversion before I can PUB. What I'd like to do is simply PUB R0 (as a real) without any additional conversion.
Also, any thoughts on direct drivers for the Do-More platform and Ignition? I know Indusoft has them, just wanted to see if anything else was in the works for other platforms...
-
We originally had it spec'ed to make simple elements be valid payloads. That descended into discussions about the many possible ways R0 could get expressed (string with one or more formatting options or raw with all the variants of bytes ordering...it gets complicated fast) and we decided to leave it as a simple string buffer for now. You can use STRPRINT or STRPUTB to stuff anything into that string buffer, so you can build whatever payload you want. Yes, it is two steps, but it is also infinitely flexible. We would still like to allow you to stuff an element in as the payload, probably just supporting the most obvious string representation of the number.
As for Ignition, it has come up several times, and if it becomes a 'standard' or at least a defacto standard, then yes I'd say we'll do it. We have chased new stuff before only to waste a bunch of time on stuff that fell by the wayside, so we are a little more reluctant now to jump on every new things that seems interesting.
I will have someone here look into doing an app note and/or sample project that would generate/consume Ignition-friendly payloads.
-
I have seen Ignition and SparkPlug mentioned in the same breath on several occasions, so I assumed that they were associated. Apparently not. It appears that Ignition is just a SCADA system that happens to support SparkPlug.
The short answer is that we would be happy to work with Ignition to do a native driver, but I don't know what their interest level is.
SparkPlug is another (apparently unrelated) technology, and is a messaging standard used with SCADA, built on top of MQTT. We are trying to understand it more.
-
Is the Do-More SIM capable of MQTT PUB/SUB instruction? I keep getting rejected by the broker, but actually didn't know if the SIM was my problem or if I am making the mistake. The broker is just on localhost, so I thought it would be pretty straight forward. I am going to pull a BRX together and try the same settings...
-
BobO I reached out on the Ignition Forum...I let you know what they come back with.
-
Is the Do-More SIM capable of MQTT PUB/SUB instruction? I keep getting rejected by the broker, but actually didn't know if the SIM was my problem or if I am making the mistake. The broker is just on localhost, so I thought it would be pretty straight forward. I am going to pull a BRX together and try the same settings...
The SIM has the same implementation as hardware. It should work fine. Are you using the local host address, or resolving a name?
-
Local host address. I am felt pretty confident it is in my setup somewhere...but I'll keep working. I am just using ignitions standard MQTT broker module.
-
I tried both by name and by 127.0.0.1 and was able to hit the broker running on my machine. I'm running a fairly stock Mosquitto. The other one I usually test with in the public Mosquitto test server test.mosquitto.org.
-
I'll try those and report back. Thanks BobO.
Nailed it!...Finally....
-
I have seen Ignition and SparkPlug mentioned in the same breath on several occasions, so I assumed that they were associated. Apparently not. It appears that Ignition is just a SCADA system that happens to support SparkPlug.
The short answer is that we would be happy to work with Ignition to do a native driver, but I don't know what their interest level is.
SparkPlug is another (apparently unrelated) technology, and is a messaging standard used with SCADA, built on top of MQTT. We are trying to understand it more.
Let?s get in touch about a native driver for Ignition. I don?t know what the timeline would be but if it provides tangible benefits over using our Modbus driver, such as the ability to browse/interrogate the tags on the controller, then it would be worth looking into.
-
Let?s get in touch about a native driver for Ignition. I don?t know what the timeline would be but if it provides tangible benefits over using our Modbus driver, such as the ability to browse/interrogate the tags on the controller, then it would be worth looking into.
No browsing or interrogation yet, but full access to everything in the controller, which can happen by numerical reference for highest performance or by symbol name for better abstraction. We do generate a full export of all tags, as well as complex data type definitions, and we are happy to do a custom export if that would be helpful.
We will be implementing true tags within the next year, which will necessitate the inclusion of a symbol table, which would make browsing painless to implement. I'm sure we would benefit from a conversation about that at the right time. PM me your contact info and I'll contact you.
-
BobO any update here (from the Ignition side)? Also, any thoughts on implementing the Sparkplug B specification into the BRX? It would be awesome to have something similar to documentation editor that allowed us to define a topic and then select memory location as our defined tags (or if you guys create a tag base system) that would reside under that topic. The Sparkplug B specification can be found here:
https://s3.amazonaws.com/cirrus-link-com/Sparkplug+Topic+Namespace+and+State+ManagementV2.1+Apendix++Payload+B+format.pdf
You may have already looked into this or know the spec, so sorry if this information is redundant.
-
BobO any update here (from the Ignition side)? Also, any thoughts on implementing the Sparkplug B specification into the BRX? It would be awesome to have something similar to documentation editor that allowed us to define a topic and then select memory location as our defined tags (or if you guys create a tag base system) that would reside under that topic. The Sparkplug B specification can be found here:
https://s3.amazonaws.com/cirrus-link-com/Sparkplug+Topic+Namespace+and+State+ManagementV2.1+Apendix++Payload+B+format.pdf
You may have already looked into this or know the spec, so sorry if this information is redundant.
I looked at the spec a bit and I like the idea, but I think that for the near future we already have more than we can get done.
-
Bump. Has native integration into Ignition been pursued?
-
Not as far as I'm aware.
We still like the idea of adding Sparkplug B support, but no definite timeline.
-
Please oh please add Sparkplug B support! Is there any other work around that we could implement prior to??