JSONBuild, one element at a time

[etElMadr]

I need to build a JSON Array of JSON Objects, one object at a time. Right now, I'm trying to FIFOLoad the data from a user_block to a fifo_block using the same Struct. Once all data is loaded, FIFOUnload each to JSONBuild a JSON Object. Then I need to add that Object to a JSON Array. FIFO.depth gives me the number of elements, but I'm unsure how to "build" the array one Object at a time as the fifo queue unloads. 


Example JSON Array below. I need to add each JSON Object / keypair to the JSON Array.

[
    {
        "Name": "Kazuhito Yokoi",
        "Age": "35",
        "Favourite Color": "red",
        "Date Of Birth": "12/09/1983"
    },
    {
        "Name": "Oli Bob",
        "Age": "12",
        "Favourite Color": "cyan",
        "Date Of Birth": "12/08/2017"
    }
]

[franji1]

It appears you will have 3 copies of each record, one in the user block, one in the fifo block, and one in the JSONBUILD buffer.

I would guess we can eliminate the middle one - your FIFO can just store the user block INDEX, not the DATA (unless you are massaging it???).

So your FIFO may then just have user block index values

3
2
9
4
5
18
7
11

vs. copies of your data.  You would then use array indexing when doing FIFOUNLOAD (unload it to a V register).  So FIFOUNLOAD to V0, then reference UserBlock[V0] (so 3 would come out first, so V0 would equal 3 the first time you run FIFOUNLOAD, so UserBlock[V0] would reference UserBlock3).

From here it gets a little tricky, since JSONBUILD must be done in layers.  You do all the objects first, then do the array.  But since the number of records is variable (or is it constant?) and since each record is variable length (in terms of number of BYTEs), you either have to allocate a STRING for EACH RECORD with a MAX LENGTH equal to the MAX POSSIBLE RECORD size (100? 200? 1000?) x max number of records (10? 100?).  Let us know what these can be, WORST CASE, and we'll go from there.  You might need to use the RAMDRIVE as a cache if it's too big.

[etElMadr]

For each JSON Object record, lets say 129 characters not including quotes. Max record length can vary, but not by much
Number of records in the array, lets say max of 32. This can vary a lot.
so 32x129

I'm not massaging the data, but am filtering it from a larger dataset based on a value in the data field. The reason I tried FIFO was simply because every scan is looping thru the larger data set, so I figured in the process I'd grab the target data and throw it in the FIFO queue to JSONBuild later (but I do understand your suggestion of just storing the index.)


(future readers, just ignore this as its going the wrong direction)
I get converting each record to a string (JSON Object). What I'm stuck on is how to build the JSON Array out of a variable number of JSON Objects / String records. Maybe I loop thru the string records, using STRINSERT to add a Comma to the end and manually build the into a larger, comma delimited string.

[franji1]

I get converting each record to a string (JSON Object).

Good!

What I'm stuck on is how to build the JSON Array out of a variable number of JSON Objects / String records. Maybe I loop thru the string records, using STRINSERT to add a Comma to the end and manually build the into a larger, comma delimited string.

YES!  You see the issue.  Quotes are part of the data, so at least 4 quotes per field (field and data - numeric/boolean data does not need quotes BTW, your example for "Age" had quotes on the data "35", but it probably needs to be just 35??? I know this was just an example).  Say 4 fields, so 145 characters (129+16?).  That's not bad - you can use a STRING block of 32 strings max length of 148 called Obj (so strings Obj0 thru Obj31), then for the Array just use a BYTE block of 32x160 or just 5120 bytes, called ArrayBuff.  That should work.

So, to generate the individual objects, I like to use STAGE.  Create a PROGRAM code block called GenArray

Code: [Select]

SG S0
MOVE 0 V1  // Obj index, also V1 will equal the number of records in Obj when this program EXITs, so $Main can use it when doing the JSONBUILD for the Array to the ArrayBuff byte block
JMP S1

SG S1
STRE MyFIFO.Depth == 0  // equal-to contact
EXIT

STRGE V1 >= 32
SET C99 // error - more than 32 objects!
EXIT

STR ST1
JMP S2

SG S2
FIFOUNLOAD MyFIFO V0
JSONBUILD Object UserBlock[V0].Field1 UserBlock[V0].Field2 UserBlock[V0].Field3 ...  to Obj[V1]
INC V1
JMP S1

I think that's it? 

[franji1]

I rewrote it using Designer (instead of just typing it from my head) and included the Array generation also as part of the code-block (D42 will contain the BYTE length of the final ASCII JSON array text in ArrayBuff block of BYTEs).  See the attached "screen shot".

You probably want to import this into its own project first to look at it, then open up your project and then enter it into your project.  Use File->Import->Project.

Code: [Select]

// Ver(103) PCU(4) Do-more Designer Clip   Source Project: C:\Users\Public\Documents\Do-more\Designer2_8\Projects\MyUDT1    Format: Do-more Designer 2.00+  DO-MORE Ladder
PLC BX-DM1E-18ED13

$PRGRM GenArray

SG GenArray.S0

MOVE 0 V1
JMP GenArray.S1

SG GenArray.S1

STRE MyFIFO.Depth 0
JMP GenArray.S10
NOT
JMP GenArray.S2

SG GenArray.S2

STRGE V1 32
JMP GenArray.S99
NOT
FIFOUNLOAD MyFIFO FIFOBlock0 V0 0x1
JSONBUILD 0x0 "4 ""FieldA"" 0x1 UserBlock[V0].FieldA DST511 ""FieldB"" 0x1 UserBlock[V0].FieldB DST511 ""FieldB1"" 0x1 UserBlock[V0].FieldB1 DST511" Obj[V1] DST511 1 DST511
INC V1
JMP GenArray.S1

SG GenArray.S10

JSONBUILD 0x1 "4 SS0 0x2 Obj0 V1" SS0 ArrayBuff0 6000 D42
EXIT

SG S99
// it will hang here if this error ever occurs, you could put a SET C99/EXIT to terminate the code block in SG S99, but this should never happen.

$PGMEND GenArray

#BEGIN UDT_CONFIG
MyUDT
// Field Name, Data Type, Memory Layout, Read/Write, Display Format, Data View Detail
FieldA, BIT, 0:0, Read-Write, Native, Short
FieldB, BIT, 0:1, Read-Write, Native, Short
FieldB1, BIT, 0:2, Read-Write, Native, Short
#END

#BEGIN MEM_CONFIG
 UserBlock MyUDT decimal 32
 FIFOBlock UWORD decimal 32
 Obj STRING148 decimal 32
 ArrayBuff UBYTE decimal 6000
 GenArray PROGRAM  0 -1
 MyFIFO QUEUE  0
#END


[etElMadr]

Thank you. I stayed up way too late last night trying to get this to work, and realized this morning how many stupid mistakes I made!

Using the JSON Pretty tool in DmD, the array is being formed correctly. However, if I GET the PLC with:

Code: [Select]

http://192.168.PLC.IP/data/json?arrJSON=ArrayBuff0,6000
I get an array of numbers. Something like (lots more numbers then show below):

Code: [Select]

{
   "arrJSON": [
       91,
       123,
       34,
       122     
    ]
}

[franji1]

Sorry, I did not realize you were doing a GET of the element.  The GET is generating the array grammar, which you did in code, so it is treating the GET as an array of bytes (which is what your GET is requesting, HA!)

I think you don't need to generate the array version, so eliminate that last JSONBUILD in S10.  You simply need to do a GET of D42, then do a GET of Obj0,<the value of D42> and the GET generates the JSON array grammar.  So if D42 equaled 27, then

Code: [Select]

http://192.168.PLC.IP/data/json?arrJSON=Obj0,27

should give you the JSON array.

[etElMadr]

OK, so in the receiving device I should be able to convert bytes to ascii. And, I have to do that for each object in the array. Something like:

payload.toString("ascii")

Code: [Select]

Buffer.from(payload);


GET from Obj0 adds extra quotes to each object, which is throwing off the receiving device:

{
"objJSON": [
    "{
        "Name": "Kazuhito Yokoi",
        "Age": "35",
        "Favourite Color": "red",
        "Date Of Birth": "12/09/1983"
    }",
    "{
        "Name": "Oli Bob",
        "Age": "12",
        "Favourite Color": "cyan",
        "Date Of Birth": "12/08/2017"
    }"
 ]
}

[franji1]

I guess that would work - that's on your end.  Have you tried it?

Also, the payload of your GET is HUGE.  You probably need to do what I recommended.  There is a limit to what the BRX can do (4000 number values would be at least 2x to 3x as big as the JSON record form, so 12000 characters?)

Code: [Select]

[{"A":123,"B":456}]
becomes
[123,42,65,42,58,49,50,51,44,42,66,42,58,52,53,54,125]

Did you try the obj query?

[franji1]

GET from Obj0 adds extra quotes to each object, which is throwing off the receiving device:

{
"objJSON": [
    "{
        "Name": "Kazuhito Yokoi",
        "Age": "35",
        "Favourite Color": "red",
        "Date Of Birth": "12/09/1983"
    }",
    "{
        "Name": "Oli Bob",
        "Age": "12",
        "Favourite Color": "cyan",
        "Date Of Birth": "12/08/2017"
    }"
 ]
}

Sorry - missed that.  Not sure you could post process that data and replace "{ with just { and }" with just }

[franji1]

I think we've come full circle.  You don't need to use JSONBUILD at all.  Let the REST API do all the work

User_Block, Index FIFO => Output_Block of same UDT as User_Block

Similar algorithm of GenArray, but you are just going to do
MEMCOPY UserBlock[V0] to Output_Block[V1]
instead of doing JSONBUILD

Then just query
Output_Block0, 27 <V1>

[etElMadr]

(updated) Sorry for the delayed response, but was pulled out of the office for awhile. Thank you for your help.  I am experimenting with several routes based on your advice. Actually, the first thing I tried (before posting here) was using the REST API to get records from the larger data set. However, it only returns one record. So, I tried the JSONBuild route.

On the receiving device, I did get the decimal values of the JSON Array to convert using Buffer.from(payload). I just tried MEMCOPY, but ran into the previous problem that the Rest API only returns one record, regardless of the number requested. Next, I may try sending the JSON Array via UDP. 


http://192.168.PLC.IP/data/json?tblDATA=DATA1,4

returns the same single record as

http://192.168.PLC.IP/data/json?tblDATA=DATA1


{
"tblDATA":
    {
        "Name": "Kazuhito Yokoi",
        "Age": "35",
        "Favourite Color": "red",
        "Date Of Birth": "12/09/1983"
    }
}

[etElMadr]

What I've ended up doing is:
JSONbuild each Object from each record of the structured data
CRC the JSONObject string, and compare it to previous CRC value which is stored in the data structure for that record
If different, then FIFOload JSON Object to a message queue
FIFOunload each JSON Object and MQTTPub to a topic

The subscribed device builds the array as it receives the Objects