Host Engineering Forum

General Category => Do-more CPUs and Do-more Designer Software => Topic started by: Controls Guy on March 19, 2015, 07:07:21 PM

Title: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 19, 2015, 07:07:21 PM
I have the following in a FOR loop, building a table to email.

It works in general, but some lines seem to have extraneous carriage returns for some bizarre reason, as shown in the second screenshot, even though all the  lines are built by the same STRPRINT box.   ???

Any ideas?
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 19, 2015, 07:08:47 PM
It would also be nice if one of the Fmtxxx() functions had a mode for implied decimal.  That would have been useful here for example.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: plcnut on March 19, 2015, 07:27:40 PM
It would 'appear' to be a text wrap issue. Have you tried to view the data in a dataview that is set to view as Hex? This way you could confirm their is an actual CRLF.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 19, 2015, 07:43:47 PM
Well, it would seem to be, but the only place I'm putting one in, is at the very end.  There might be one embedded in the string value, but the extraneous break isn't in the part of the line that comes from the string value.  ?????
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: ADC Product Engineer on March 19, 2015, 07:48:49 PM
IIRC, it has something to do with not having a proper termination in the string (<CR><LF>).  The buffer fills and then at 72 characters (I think) it forces the <CR> <LF>.

This is from memory, so Franji1 or BobO may come in and correct me as to the particulars on it.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 20, 2015, 12:07:22 AM
Well then, I'm really confused, because there WAS an 0D0A at the end of the previous line (hence the carriage return) way less than 72 characters previous, plus all the other lines make it farther than the broken ones do, with no problem.  Guess I have to look at the actual hex contents of Report0.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: ADC Product Engineer on March 20, 2015, 08:46:32 AM
As a follow up, I was mostly correct in my previous post.  (It only took me a half hour of email digging to find the relevant information.)

If you are sending email, SMTP servers require a 0D0A every 80 characters or one will be forced into the string.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 20, 2015, 01:19:43 PM
OK, I'm getting confuseder by the minute.   :o

ADC PE, you seem to be almost right.  The prebuilt string does have CRLF's where expected, about every 39 characters or so, according to a Data View in DMD.  I checked and the first extra CR in the email is indeed exactly 80 characters from the beginning of the message (though the second one is 180 chars after??), so you'd think maybe the EMAIL box is failing to send them and SMTP server is inserting an extra......BUT, it's only the 80th character IF YOU COUNT the <CR>'s and <LF>'s that are supposed to be there, plus the previous lines break where intended!  So I know the server's getting the intended 0D0A's, so that hypothesis doesn't work.

So then I thought maybe it's a bug in that specific SMTP server (inbox.com), so I tried another email server (vfemail.net), and the received email had extra breaks in exactly the same positions.  Can't be a bug in TWO servers, can it?

So then I thought maybe it's in my email client (Thunderbird), and it's just rendering that way on screen, but I looked at the message on the server using Cox's webmail client, and the extraneous breaks are already in by that point.

At this point, don't know what to do, other than put the mirroring switch on and verify what Do-More's sending, byte by byte.

Thoughts?
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 20, 2015, 07:37:04 PM
BobO figured this out for me.  My MHR strings had ASCII NULLs (0x00's) in them, which broke the character counter, so if I replace the NULLs with spaces, all should be good.

Thanks, BobO!   :)
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: ADC Product Engineer on March 20, 2015, 10:41:04 PM
Do-more is the culprit.  It is strictly adhering to the SMTP spec which states that every 80 characters, there must be a CR/LF.

Good to know about the null's.  I hadn't thought of that, but it makes sense.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 20, 2015, 11:12:00 PM
Well, it really had nothing to do with CRLF frequency, or only indirectly.  There were plenty of CRLF's present that no additional ones were needed.  The central issue is that NULL's break the character counting, and that can be worked around.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: BobO on March 22, 2015, 10:32:45 AM
The line break code uses standard C string handling to look for CR/LF. It hits a NULL and concludes that there is none, then grabs 78 characters, looks for white space at the end, adds CR/LF, and spits out the line...which now has 2 line breaks and a bunch of illegal NULLs.

Do-more could be a bit more graceful about it, but the NULLs are not permitted in the email. I will look at adding some warnings...and maybe converting illegal characters to something else.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 22, 2015, 11:12:21 PM
For me at least, either or both of those (run time warning or replacement) would be fine.  If you do real-time replacement, you probably want to announce it somehow, or else someday, somehow, the replacement will bite someone in some mystifying way that's hard to diagnose.

Thank you again for figuring out what was going on here.  It really had me baffled.  :)
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: BobO on March 23, 2015, 08:31:33 PM
For me at least, either or both of those (run time warning or replacement) would be fine.  If you do real-time replacement, you probably want to announce it somehow, or else someday, somehow, the replacement will bite someone in some mystifying way that's hard to diagnose.

Thank you again for figuring out what was going on here.  It really had me baffled.  :)

If we did replace, we would certainly warn also. I am inclined to just warn.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on March 24, 2015, 02:01:03 AM
I think just warning would be fine.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on April 13, 2015, 03:27:07 PM
During the previous conversation about emailing, I believe I said somewhere that DEVWRITE didn't seem to be working for me.  I based that belief on the fact that if I opened the device configuration from DMD, the values in the dialog were not updated.

Now I'm back to this task (commissioning the email functions) and decided to confirm my earlier observation, but using DEVREAD.  All the parameters that I've verified using DEVREAD DID in fact write as expected, so the dialog is showing not current values, but values that (presumably) were buffered somewhere the last time the dialog was used to edit them.

It's not life-threatening, but it seems to me it would be both more helpful and more correct if the dialog refreshed with current values when opened, regardless of whether the values were manually entered or the result of a DEVWRITE, if it's an easy change.

Thanks!
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on April 13, 2015, 03:31:00 PM
Or....is there any way to view device parameters in a data view?  I haven't found any way to do so.

That might be a nice short-term workaround and/or supplementary fix to having the dialog refresh every time.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: BobO on April 13, 2015, 04:30:20 PM
During the previous conversation about emailing, I believe I said somewhere that DEVWRITE didn't seem to be working for me.  I based that belief on the fact that if I opened the device configuration from DMD, the values in the dialog were not updated.

Now I'm back to this task (commissioning the email functions) and decided to confirm my earlier observation, but using DEVREAD.  All the parameters that I've verified using DEVREAD DID in fact write as expected, so the dialog is showing not current values, but values that (presumably) were buffered somewhere the last time the dialog was used to edit them.

It's not life-threatening, but it seems to me it would be both more helpful and more correct if the dialog refreshed with current values when opened, regardless of whether the values were manually entered or the result of a DEVWRITE, if it's an easy change.

Thanks!

It isn't an easy or feasible change. The system configuration isn't changed from DEVWRITE, only the current state of those same settings in the runtime driver. Y'all have said repeatedly that you wanted program control of configurations and this was the manageable answer for doing so. Pushing that information back into the FLASH-based SysConfig simply isn't viable.

Having an extended device view that would read current settings out of the driver is do-able, but there are at least 15 things you want sooner.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on April 13, 2015, 04:33:24 PM
If it's not feasible to change that behavior, that's fine.  It's more important that the parameters be writable at runtime, as you noted.  I'll just try to remember not to configure a device using the dialog on a project where it needs to be written dynamically, so I don't get confused if I happen to open the dialog.

Thanks!
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: BobO on April 13, 2015, 04:37:54 PM
If it's not feasible to change that behavior, that's fine.  It's more important that the parameters be writable at runtime, as you noted.  I'll just try to remember not to configure a device using the dialog on a project where it needs to be written dynamically, so I don't get confused if I happen to open the dialog.

Thanks!


We don't really know the difference between an important runtime changes and not, which means we would have to write them all. So every time you looked up the current IP address of the Google SMPT server, we'd be pushing that back to flash. Running flat out, the flash would be constantly erased and rewritten...and might die in a month.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on May 23, 2015, 07:53:51 PM
How about just writing back to flash when someone opens the device config dialog?  Or giving the user the choice in the dialog which version to look at?

That would prevent wear on the flash as well as the slowdown from writing to it all the time.
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: BobO on May 24, 2015, 02:37:24 PM
It does write to flash when you download a new SysConfig, which is the result of editing the device config.


Lest you think I am being hard to get along with, please let me elaborate a bit. The SysConfig is a complex document containing lots of critical stuff, serving lots of subsystems in the controller. Think of it as a big ZIP file containing the Laws of Physics. When the system starts up, the various subsystems read their part of that file and initialize themselves to whatever was specified by the user. Writing that file is a Big Deal that only happens in program mode. Modifying that file at runtime is the stuff of nightmares...like changing the Speed of Light. Bad.

At startup, the SMTP Client Device Configuration provides the SMTP Client all of the nice stuff you specified. DEVREAD and DEVWRITE give you runtime access to the state of the driver, but do not access or modify the config. Modifying driver values at runtime is easy and safe. Modifying the SysConfig at runtime is Very Bad.

In short, it isn't happening. I am an old engineer. Part of my ability comes as an experience base that tells me when to *not* do something. This falls in that category.

I am well aware of how this way of handling the issue seems counterintuitive from a user's perspective. We really do see that and care about that...but...in the case, it is architecturally a bad plan that can screw many things up. My rule is simple: I'd rather apologize to 20% for what I chose not to do, than to apologize to 100% for what I did poorly. Please consider this an apology. ;)
Title: Re: STRPRINT or EMAIL formatting oddly
Post by: Controls Guy on May 24, 2015, 06:04:02 PM
I think in your previous explanation of why this is a bad idea, you didn't go into too much detail, so I thought the barriers stated were the only ones, and when a way around that occurred to me, I posted it.  I can see now in more detail why you don't want to do it.  Thanks!

No apology required.