Host Engineering Forum

General Category => Do-more CPUs and Do-more Designer Software => Topic started by: mfloresh on August 11, 2017, 07:11:20 PM

Title: Parsing comma separated values in long strings with DoMore
Post by: mfloresh on August 11, 2017, 07:11:20 PM

Hello,

I am parsing csvs in long strings using the STRFIND, STRSUB, etc functions, using stages. I am parsing about 30 long strings, averaging 8 values each string. This takes over one second for the DoMore to parse thru. Does anyone know of a more efficient way to parse these many parameters on these many strings? As a side note, I have old DL06s with coprocessors and they are actually faster at parsing the exact same data.

Thanks,

Miguel

Title: Re: Parsing comma separated values in long strings with DoMore
Post by: plcnut on August 11, 2017, 09:51:35 PM

I would think it should be much faster than that. Could you post some code to show how you are doing it now?

I have a database of 3000 strings that I parse through in a sortation system plus lots of 1000 character csv's and it blasts through them.

Title: Re: Parsing comma separated values in long strings with DoMore
Post by: Mike Nash on August 12, 2017, 10:14:33 AM

If you are doing the parsing in a program or task that is NOT $Main and you have any looping instructions (yielding - NEXT, UNTIL, WEND, YIELD or backwards GOTO) then the default timeslice of Yield Every 100 microseconds will make it very slow. If the rest of your program can tolerate slightly longer scan times, bump this up until you like the speed or the scan times as a whole get longer than you like.

You can experiment with this in a data view by adding and changing taskname.Timeslice where taskname is your task or program name for that code-block. Then change in the Configure Code-Block right click menu for the block.

Title: Re: Parsing comma separated values in long strings with DoMore
Post by: plcnut on August 12, 2017, 10:40:41 AM

I saw the OP mentioned stages, and it makes me wonder if he is using JMP's and iterating through once per scan.

Title: Re: Parsing comma separated values in long strings with DoMore
Post by: BobO on August 12, 2017, 11:56:52 AM

Quote from: mfloresh on August 11, 2017, 07:11:20 PM

As a side note, I have old DL06s with coprocessors and they are actually faster at parsing the exact same data.

I'm sure this is due to the way your program is written. Do-more CPUs are at least 20x faster than DL06.