News:

  • June 27, 2026, 02:56:51 PM

Login with username, password and session length

Author Topic: Parsing comma separated values in long strings with DoMore  (Read 7632 times)

mfloresh

  • Newbie
  • *
  • Posts: 8
Parsing comma separated values in long strings with DoMore
« on: August 11, 2017, 07:11:20 PM »
Hello,

I am parsing csvs in long strings using the STRFIND, STRSUB, etc functions, using stages. I am parsing about 30 long strings, averaging 8 values each string. This takes over one second for the DoMore to parse thru. Does anyone know of a more efficient way to parse these many parameters on these many strings? As a side note, I have old DL06s with coprocessors and they are actually faster at parsing the exact same data.

Thanks,

Miguel

plcnut

  • Hero Member
  • *****
  • Posts: 814
    • premiersi.com
Re: Parsing comma separated values in long strings with DoMore
« Reply #1 on: August 11, 2017, 09:51:35 PM »
I would think it should be much faster than that. Could you post some code to show how you are doing it now?

I have a database of 3000 strings that I parse through in a sortation system plus lots of 1000 character csv's and it blasts through them.
Circumstances don't determine who we are, they only reveal it.

~Jason Wolthuis
Premier Systems Integration, LLC
http://premiersi.com

Mike Nash

  • Hero Member
  • *****
  • Posts: 652
Re: Parsing comma separated values in long strings with DoMore
« Reply #2 on: August 12, 2017, 10:14:33 AM »
If you are doing the parsing in a program or task that is NOT $Main and you have any looping instructions (yielding - NEXT, UNTIL, WEND, YIELD or backwards GOTO) then the default timeslice of Yield Every 100 microseconds will make it very slow. If the rest of your program can tolerate slightly longer scan times, bump this up until you like the speed or the scan times as a whole get longer than you like.

You can experiment with this in a data view by adding and changing taskname.Timeslice where taskname is your task or program name for that code-block. Then change in the Configure Code-Block right click menu for the block.
« Last Edit: August 12, 2017, 10:18:06 AM by Mike Nash »

plcnut

  • Hero Member
  • *****
  • Posts: 814
    • premiersi.com
Re: Parsing comma separated values in long strings with DoMore
« Reply #3 on: August 12, 2017, 10:40:41 AM »
I saw the OP mentioned stages, and it makes me wonder if he is using JMP's and iterating through once per scan.
Circumstances don't determine who we are, they only reveal it.

~Jason Wolthuis
Premier Systems Integration, LLC
http://premiersi.com

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6164
  • Yes Pinky, Do-more will control the world!
Re: Parsing comma separated values in long strings with DoMore
« Reply #4 on: August 12, 2017, 11:56:52 AM »
As a side note, I have old DL06s with coprocessors and they are actually faster at parsing the exact same data.

I'm sure this is due to the way your program is written. Do-more CPUs are at least 20x faster than DL06.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO