FILEREAD as Lookup Table

[Bolt]

I have a device that sends ASCII error codes to BRX via Modbus TCP.  Then I have a CSV file that references said error code to a error description.  Currently, I am hosting this CSV on a webserver, sending a HTTPCMD containing error code, cross referencing this to error description, and returning that to the BRX.  Works great.
However, due to changing network architecture, I would like to do this all on the BRX, have the lookup table on the SDCard, and use FILEREAD to do the looking up.  How would I go about doing this?

[franji1]

I have a device that sends ASCII error codes to BRX via Modbus TCP.  Then I have a CSV file that references said error code to a error description.  Currently, I am hosting this CSV on a webserver, sending a HTTPCMD containing error code, cross referencing this to error description, and returning that to the BRX.  Works great.
However, due to changing network architecture, I would like to do this all on the BRX, have the lookup table on the SDCard, and use FILEREAD to do the looking up.  How would I go about doing this?

How large is this database?  Is it just a few hundred error codes and 80 characters per error message?  You can do that easily with just 2 data blocks, one with an error code number and one with strings, up to 80 characters long, where the index of each correspond to each other.  No need for the SD Card/File system (unless you need it to LOAD the data on first scan).  If it is static, stick it in code, don't bother with the SD card.  Initialize the 2 blocks in $tFirstScan via ladder logic, definitely no SD card needed there.

To look up the error code text, use FOR/NEXT loop to find error code in the error code block, get that V index, and use that index in the string error text table.

STRPRINT  ErrorText   "Unknown Error:" ErrorCode  // assume ErrorCode is not in our table
// loop through ErrorCodeTable, looking for ErrorCode
FOR V42 0 199
  STRE ErrorCode ErrCodeTable[V42]  // found it?
  STRCOPY ErrorTable[V42] ErrorText  // copy it to ErrorText
  BREAK  // done! break the FOR/NEXT loop
NEXT
// ErrorText will either be assigned the found error code, or it will be "Unknown Error
RETURN

If the error codes are sequential, say 0..199 (i.e. they are not sparse), you do not even need the error code table, the error code IS the string table V array index.

However, if there are HUGE number of error codes (more than 500) with HUGE error messages (some with more than 500 characters), then the SD Card mechanism will be necessary, parsing the text.  But there are ways to implement a database lookup function in a PLC (e.g. have an Error code table with a corresponding FILE OFFSET DWORD table, where the lookup doesn't return a string, it returns the INDEX INTO THE FILE WHERE THAT ERROR TEXT EXISTS.  That's sort of what databases do on Server (table "keys" => corresponding data locations).

[Bolt]

This is an interesting angle, and I could likely make it all fit by condensing the text strings (there are ~240 codes, and many have a lengthy description), limiting the oddball errors that will likely not occur, etc, but .

What if I did a hybrid approach?  Error codes in PLC data block (4 or 5 alpha numeric characters), lookup a numeric value, then reference a SD Card file to get the full description?  Then it would be a matter of which line the result is on?  How would I do that (the FILEREAD line number part)?

[franji1]

I would not bother with hybrid - you run into issues of inconsistency between the HARDCODED  error codes and the error codes in the FILE.  You could do a hybrid of the hybrid (ha!)

Store the short and the long in the file, but when we are parsing for the long text file offset, we can build up the table of strings of the short text:
error code 1; short text 1; long text 1
error code 2; short text 2; long text 2
e.g.
21; RC; Right fielder, Pittsburgh Pirates
42; JR; Secondbase, Brooklyn Dodgers

Note I am using semicolons as delimiters.  I did not use commas in case you need to use commas in your error message text - so do NOT use semicolons in your short/long descriptions!!

On first scan, you would need to generate 3 tables
UNSIGNED WORD ErrorCodes block 256 long
STRING MAX 8 ShortErrorText block 256 long
SIGNED DWORD LongFileOffset block 256 long

1. Open the file
2. Read a line (CR/LF delimiter)
3. tokenize the line (semicolon delimits tokens), getting the 3 pieces
4. parse the 3 pieces and put into the 3 blocks
5. go back to step 2

It's not hard.  What's great is that you can do all this with the Simulator and make sure the logic all works before dealing with the physical SD Card in a BRX

Do step 1, 2, and 5 for now.  Try to get that working using FILEOPEN, with FILEREAD inside FOR/NEXT loop, and FILECLOSE after the NEXT.

You also have some possible issues you have to deal with.  Just think about them, we can keep these in mind
1. What happens if the SD card is not there, or the desired file does not exist?  Missing SD card is a definite point of failure; I say you just set a flag and when you look up error codes, you just get the error number as text in the short description (maybe with a question mark, e.g. "42?") and "Unknown Error: 42" for the long description???
2. What happens if some of the file content is invalid; set a flag, but just keep parsing the file (unknown error handling from 1 can be utilized here also)

[Bolt]

That will get me started.  I don't have much time to work on it today, but will report back.  Thanks for getting me in the right direction!

[Bolt]

I would not bother with hybrid - you run into issues of inconsistency between the HARDCODED  error codes and the error codes in the FILE.  You could do a hybrid of the hybrid (ha!)

Store the short and the long in the file, but when we are parsing for the long text file offset, we can build up the table of strings of the short text:
error code 1; short text 1; long text 1
error code 2; short text 2; long text 2
e.g.
21; RC; Right fielder, Pittsburgh Pirates
42; JR; Secondbase, Brooklyn Dodgers

Note I am using semicolons as delimiters.  I did not use commas in case you need to use commas in your error message text - so do NOT use semicolons in your short/long descriptions!!

On first scan, you would need to generate 3 tables
UNSIGNED WORD ErrorCodes block 256 long
STRING MAX 8 ShortErrorText block 256 long
SIGNED DWORD LongFileOffset block 256 long

1. Open the file
2. Read a line (CR/LF delimiter)
3. tokenize the line (semicolon delimits tokens), getting the 3 pieces
4. parse the 3 pieces and put into the 3 blocks
5. go back to step 2

It's not hard.  What's great is that you can do all this with the Simulator and make sure the logic all works before dealing with the physical SD Card in a BRX

Do step 1, 2, and 5 for now.  Try to get that working using FILEOPEN, with FILEREAD inside FOR/NEXT loop, and FILECLOSE after the NEXT.

You also have some possible issues you have to deal with.  Just think about them, we can keep these in mind
1. What happens if the SD card is not there, or the desired file does not exist?  Missing SD card is a definite point of failure; I say you just set a flag and when you look up error codes, you just get the error number as text in the short description (maybe with a question mark, e.g. "42?") and "Unknown Error: 42" for the long description???
2. What happens if some of the file content is invalid; set a flag, but just keep parsing the file (unknown error handling from 1 can be utilized here also)

Okay, I have 1-4 working, and can get it to return to step 2, but how do I have it read the next line of the file, and not re-read the first line over and over again?

[Bolt]

I got it to work, but I could not get it to run successfully in a FOR/NEXT loop arrangement.  It is almost like it ran too fast, and would never calculate FILESEEK position quick enough to read the correct line in the file.  Now I have it running as a Program with Stages, and manually incrementing the pointer after every line, and it works slick.

However, now I'm wondering what that advantage is to create these tables on the PLC.  Could I not run through this logic, and instead of writing to ErrorCodes[V0] (and LongFileOffset[V0], find the semicolon, do a STRCMP with the error code received via Modbus TCP, if matches, calculate offset and FILESEEK/FILEREAD the value, if no match, increment count and try again?  Is this simply too scan time intensive?

[Controls Guy]

I don't think it would be scan time intensive, I think It'd be slow.

Flash is faster than spinning media, but it's still not RAM.  Plus there's gonna be some software and hardware overhead/bottleneck getting the data off the flash.

Easier to move the whole table, or at least the index, to PLC memory.

[franji1]

Sorry, FOR/NEXT won't work.  I was thinking table lookup, not reading a file.  Oops.

A looping Stage program works
Program LoadErrorTables

SG S0
FILEOPEN OnSuccess JMP S1

SG S1
FILEREAD OnSuccess JMP S2 OnError JMP S99

SG S2
parse out pieces/process data
JMP S1  // loop back to FILEREAD stage

SG S99
FILECLOSE OnSuccess JMP S100


SG S100
EXIT

[Bolt]

I have this working to my satisfaction now.  Thanks for the help.

In Simulator:
Populating the data (298 rows) into some lookup tables takes ~150 milliseconds
Looking up the code in the tables and reading corresponding line from file takes ~2 to 75 milliseconds (first and last entries)
Looking up the code directly from the SD card without making lookup tables takes ~2 to 150 milliseconds

In BRX:
Looking up the code directly from the SD card without making lookup tables takes ~2 to 5700 milliseconds (with no impact to scan time)

Still fast enough for my purposes.

[franji1]

In BRX:
Looking up the code directly from the SD card without making lookup tables takes ~2 to 5700 milliseconds (with no impact to scan time)

Still fast enough for my purposes.

GREAT!!  You're now a BRX expert!

[franji1]

BobO mentioned to speed it up, you could see if it does not exist in the RAM drive, do a FILECOPY if it doesn't exist there, then always do the error lookup from the RAMDRIVE copy.

[Controls Guy]

Ooh, I like that idea.  Plus, if the codes are in order, or you can put them in order, at most 9 queries should find the record if random access is possible.

[Bolt]

BobO mentioned to speed it up, you could see if it does not exist in the RAM drive, do a FILECOPY if it doesn't exist there, then always do the error lookup from the RAMDRIVE copy.

Cool idea.  Shaves about 1 second off the first attempt (having to copy file from SD Card to RAM), and subsequent attempts take almost 2 seconds less than reading the whole file from the SD Card everytime.

Ooh, I like that idea.  Plus, if the codes are in order, or you can put them in order, at most 9 queries should find the record if random access is possible.

The error codes are ASCII, not numeric.  along the lines of AB-12 or XY-789, so that would be a little too tricky.


Now, my question is, what is the cleanest way to set my loop max index (in event of error code not found)?  It should be number of rows in file.  Using FILEQUERY, I don't think there's a way to see this, only bytes, but each line has varying byte count.  Would I have to loop through the file and count the number of <CF><LF> occurances?

[franji1]

Basically, if you reach the end of the file and have not found it, it's not found.

Not sure how you've written your code, but once you find the code, you should JMP out of the "loop" (so typically you should never reach the end of the file).

If the FileHandle structure's .EndOfFile bit is set, then you've reached the end of the file before finding the code.  You should be able to tell the difference.