News:

  • March 28, 2024, 03:59:49 AM

Login with username, password and session length

Author Topic: Programming Has Made Me Lazy  (Read 7987 times)

jcottrill

  • Full Member
  • ***
  • Posts: 43
Programming Has Made Me Lazy
« on: December 27, 2017, 11:59:54 PM »
I hope you all are enjoying a break this time of year.  I have the following challenge for your consideration when you get back in the office. Sorry for the length so the TL;DR version is I am looking for auto serialized output of user data types to JSON into a given memory block of strings.

Background...I am using the BRX PLC as a generalized platform for automated test equipment to make it easier for our young Test Engineering team to turn out functional testers with repeatable results.  The base programs and tasks control the general flow for things like validating operators, work orders, products etc.  HMI functionality is provided via a modified version of plcnut's web server.  The primary page displays the devices under test (DUTs) and is updated periodically with AJAX requests to the PLC.  These base programs, tasks and supporting data structures will be considered sacred and protected from being modified by the Test Engineers.  They will be able to add in additional data structures, programs, etc to round out their testers but the core should remain untouched to ensure a base level of consistency.

So far all is going extremely well, at least through simulation.  This is my first big project with PLCs so there is still a lot of learning along the way.  Luckily Host Engineering has built an amazing community so that process has been pretty painless. 

My next hurdle that has me spinning a bit is if it will be possible to generalize the DUT reporting back to the HMI and/or final log collection server.  Lets assume the Engineer has defined the following fields in the DutResult user data type.

Code: [Select]
DutResult
    DutID          unsigned byte
    T1_Start       dword
    T1_End         dword
    T1_Expected    real
    T1_Actual      real
    T2_Start       dword
    T2_End         dword
    T2_Expected    real
    T2_Actual      real

I'd like to be able to generate a JSON record of a given DutResults memory block location (ex. DutResults0-9) without explicitly building the string(s).  Below is an example of the conversion.

Code: [Select]
{
    "DutId": 0,
    "T1_Start": 234123432,
    "T1_End": 234123450,
    "T1_Expected": 5.00,
    "T1_Actual": 5.01,
    "T2_Start": 234123451,
    "T2_End": 234123455,
    "T2_Expected": 1.500,
    "T2_Actual": 1.495
}

I am looking for something similar to using Newtonsoft.Json.JsonConvert.SerializeObject(someObject) in C#.  By dynamically generating this I could avoid the need to teach our Test Engineers the finer points of JSON and also wouldn't have to worry about missing a field in the output.  We could just focus on building testers. 

Fundamentally, I think everything exists on the PLC in terms of the documentation, nicknames, etc.  What I am not sure of is the ability to query and iterate through the fields of a structure, get the names of the fields and then get the associated values.  I think this could be useful for handling web based requests for data and for logging data to external endpoints without the need for any middle-ware.  I'd like to hear the thoughts of the team and community on this to see if there is any merit or if you have some better thoughts.  Thanks in advance and keep up the great work!

plcnut

  • Hero Member
  • *****
  • Posts: 803
    • premiersi.com
Re: Programming Has Made Me Lazy
« Reply #1 on: January 05, 2018, 09:32:13 PM »
I am not sure that I totally understand what you are wanting, but I have built JSON strings very similarly to how I build the csv strings in the latest sample code for the Do-more webserver. http://forum.hosteng.com/index.php/topic,1117.15.html
Circumstances don't determine who we are, they only reveal it.

~Jason Wolthuis
Premier Systems Integration, LLC
http://premiersi.com

jcottrill

  • Full Member
  • ***
  • Posts: 43
Re: Programming Has Made Me Lazy
« Reply #2 on: January 06, 2018, 01:22:28 PM »
Hey plcnut,

To start, thanks for your work on the Web Server.  It saved me quite a bit of time and allowed me to get to the meat of my project quicker.  When I get some free time I will submit my modifications such as replying with 405 messages for missing pages instead of just redirecting to a default page.  Sometimes that auto redirect can be confusing to people.

I took a look at your updated version and see what you are doing with your CSV data.  This is similar to how I have handled JSON outputs for concrete data and could just as well do so here but I am trying to  add in some mistake proofing for my teammates.

I will try to use your example to better illustrate what I am looking for.  In your example you appear to be collecting logs and sending them off in mass whereas I will be doing more of a real-time stream but it should be close enough.  I will also be using a single memory block of a user structure instead of multiple memory blocks of discrete values.  The struct is the really the key to what I am proposing here.

Lets say I am the manufacturer of the freezer/machine that you are monitoring in your PLC code.  I want an Engineer to do some functional or reliability testing on 10 of these machines and ultimately log the results.  I provide them with the base PCL code that includes the following structure that defines the base information we must report back.  This structure is then mapped to a memory block to track this data for all 10 machines.  Also in the PLC code is a logging function to provide this data in JSON format as needed.




The Engineer then adds the fields he needs (LastSampledTS, WaterPressure, PumpOn) to the structure for his specific situation as shown below.



The problem is, that the Engineer must remember to also add these additional fields to then logging code to ensure it is passed on.  Teams can implement checks and validation to ensure this is correct but it is still dependent on people to get it right.  I don't know about you, but in the crunch we silly humans sometimes miss some of these smaller details as we spend most of our energy focusing on safety, stability and overall functionality.

Instead of creating a manual concatenation:

Code: [Select]
DUTs[v_CurrentDut].DutId + "," + DUTs[v_CurrentDut].StartedTS + "," + ... + DUTs[v_CurrentDut].PumpOn
I am proposing a method to convert a given structure element directly as shown below.  Using psuedo code instead of ladder logic to try to condense.  It would reply on needing access to the program documentation to get the friendly names of the fields in the struct.  With a tool such as this, the structure could be modified but you'd never need likely never need to touch or think about your logging functions as it would iterate through all the fields of the struct.

Code: [Select]
function Main {
    ...
    dutCSV = StructToCSV(DUTs[v_CurrentDut]);
    dutJSON = StructToJSON(DUTs[v_CurrentDut]);
    ...
}

function StructToJSON(struct) {
    var result = "{";
    var isFirst = true;

    foreach (field in struct) {
        if (!isFirst) {
            result += ",";
            isFirst = false;
        }       

        result += "$"" + field.Name + "$":" + field.Value;
    }

    result += "}";
    return result;
}



jcottrill

  • Full Member
  • ***
  • Posts: 43
Re: Programming Has Made Me Lazy
« Reply #3 on: January 20, 2018, 01:11:07 PM »
Not sure how many out there will find this useful but figured I would share it. For anyone out there that is logging user defined struct data to an external destination and need to do so using JSON, I have created a tool to auto create the string you need to paste into your STRPRINT block in the designer.  This is a bit overkill if your struct only has a few properties but may help save your eyes if you have a dozen or more properties.  You just need to provide it the name of the memory block or heap item representing your data.  You  can also provide the name of an iterator if using a memory block to use this inside a for-loop or set of stages.  Finally past the contents of an export from a data view for the object and click convert.  From there you can copy and past the results into your STRPRINT block. 

https://jsfiddle.net/JCFiddle/ojcwd5bc/




BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5975
  • Yes Pinky, Do-more will control the world!
Re: Programming Has Made Me Lazy
« Reply #4 on: June 20, 2018, 12:32:51 PM »
Old bump...

Since we now have MQTT/IoT, the subject of JSON has come up. We have had short discussions on how we might approach that and already have a decent idea of how to solve it. If some of y'all that use JSON could give me a sense of what your typical records look like and how you use it, it would be helpful to coming up with a good set of tools.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5975
  • Yes Pinky, Do-more will control the world!
Re: Programming Has Made Me Lazy
« Reply #5 on: June 25, 2018, 02:44:21 PM »
Nobody got thoughts on JSON? Fine. You'll get what we give you!

Aside: I had an old barber who used to tell me that when I attempted to tell him how I wanted my hair: "Well, I reckon you'll get what I give you." Indeed. Fortunately, while he was kinda grumpy, he was an excellent barber. Rest in peace, Zeke.


We were also talking about adding functions for HTTP GET and POST. Our thinking is that HTTP, JSON, and MQTT are all complementary pieces of an evolving vision to tie Do-more PLCs to enterprise data systems. Anything you could share with us about how you do these things now, or how you would do these things if you had direct support in the PLC, would be helpful.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

Garyhlucas

  • Hero Member
  • *****
  • Posts: 398
Re: Programming Has Made Me Lazy
« Reply #6 on: June 25, 2018, 08:09:22 PM »
Yes,
But the difference between a bad hair cut and a good haircut is about 6 weeks.

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2121
  • YKPAIHA
    • ATU, Inc.
Re: Programming Has Made Me Lazy
« Reply #7 on: June 25, 2018, 08:39:07 PM »
The Post and Get functionality would interface nicely with PHP. Go for it.

jcottrill

  • Full Member
  • ***
  • Posts: 43
Re: Programming Has Made Me Lazy
« Reply #8 on: June 26, 2018, 08:12:51 PM »
Hey BobO,

Sorry for the late reply.  Haven't been keep an eye on the site much in the last week or so.  Primarily I am serializing my user defined data types but there are some cases where I need to sneak in a string storage location with that data.  Manually building the JSON string when there are 20+ fields in the record (or even 10 for that matter) is a bit of a pain and makes my eyes burn :)  A slightly better approach would be if you could provide a table like the data mapping and a few other tools where we could provide the desired JSON attribute name and a memory location.  Not sure how to handle nested JSON structures though unless you could reference a JSON map.  Some of these JSON strings get pretty large so managing the strings could be an issue.  Hopefully this makes sense and isn't end of day rambling.

franji1

  • Bit Weenie
  • Host Moderator
  • Hero Member
  • *****
  • Posts: 3650
    • Host Engineering
Re: Programming Has Made Me Lazy
« Reply #9 on: January 09, 2019, 01:57:41 PM »
Well, here's what we've implemented.

JSONBUILD is a new instruction lets you built a JSON record of fields or array items, whose values can be objects, strings, numeric values, or Booleans.  It's a table instruction, with up to 50 entries.

In the JSONBUILD editor, you bring up a dialog where you can then add a group of JSON fields by simply referencing a single structure.  It adds all the proper fields of that structure to the table.  The attachments are an example of a UDT based on your DUTs.  I created a block of 10 DUTs called MyDUTs and told it to create a JSON record based off the data in MyDUTs[V1] and all its "useful" fields.  See the first screen shot for the JSONBUILD editor's sub-dialog where you add the record for the fields of a specific structure element (e.g. MyDUTs[V1]).

You end up with the following in the JSONBUILD instruction (see the 2nd attachment), which you can use MQTT or other instructions (some new).  The JSONBUILD editor will let you add/edit more fields, not just limit you to the ones in the one structure.

For bit fields, note that you can tell the JSONBUILD instruction to treat the values as numeric (1, 0) or as JSON Boolean (true, false).

jcottrill

  • Full Member
  • ***
  • Posts: 43
Re: Programming Has Made Me Lazy
« Reply #10 on: February 07, 2019, 05:43:38 PM »
This looks pretty slick!

ddubs2248

  • Sr. Member
  • ****
  • Posts: 64
Re: Programming Has Made Me Lazy
« Reply #11 on: March 26, 2019, 08:22:34 AM »
Where is this instruction set?  I am trying to implement MQTT into Inductive Automation with the BRX platform.  Has this been rolled out or still in process?  Also, any thoughts are creating direct drivers for the Do-More platform for Inductive Automation?

franji1

  • Bit Weenie
  • Host Moderator
  • Hero Member
  • *****
  • Posts: 3650
    • Host Engineering
Re: Programming Has Made Me Lazy
« Reply #12 on: March 26, 2019, 08:39:10 AM »
Where is this instruction set?  I am trying to implement MQTT into Inductive Automation with the BRX platform.  Has this been rolled out or still in process?  Also, any thoughts are creating direct drivers for the Do-More platform for Inductive Automation?

MQTT was added in 2.3.  JSONBUILD/JSONPARSE are coming out very soon.

ddubs2248

  • Sr. Member
  • ****
  • Posts: 64
Re: Programming Has Made Me Lazy
« Reply #13 on: March 26, 2019, 09:13:43 AM »
Yes I see the PUB/SUB, but right now we can only send string values...which I am trying to figure out how to send numeric values (reals) by others posts on the forum...I'll keep working!

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 5975
  • Yes Pinky, Do-more will control the world!
Re: Programming Has Made Me Lazy
« Reply #14 on: March 26, 2019, 09:16:55 AM »
Yes I see the PUB/SUB, but right now we can only send string values...which I am trying to figure out how to send numeric values (reals) by others posts on the forum...I'll keep working!

Reals expressed how? As the raw binary data, or as a string representation?

As used with MQTT, strings are just buffers and can contain anything.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO