Host Engineering Forum
General Category => Do-more CPUs and Do-more Designer Software => Topic started by: Andrew S on May 24, 2019, 10:41:57 AM
-
Perhaps I've missed something, but I'm doing an MQTT Publish which is working fine. I noticed though that every time the command runs it connects, publishes the data, and then disconnects.
MQTT is designed to have long-running TCP connections. This is how LWT functions operate on the broker - it looks for a disorderly disconnect. The way that (to my understanding) the current implementation works, the immediate disconnect means that there's no active TCP connection for the broker to manage, and therefore process an LWT if necessary.
Usually, MQTT clients have a CONNECT and a DISCONNECT command that get issued appropriately. Is there a way to make this happen in a future release?
On a separate but related subject, would it be possible to override the default MQTT client name in a future release? I'd find this very useful.
Thanks, Andrew
-
You are either dropping the Enable leg or you are terminating the containing code block. The connection is maintained as long as at least one instruction is enabled, and automatically closed when the last instruction is disabled.
It should be fairly easy to add a DEVWRITE property to override the client name.
-
Okay. That helps. Thanks Bob.
Andrew.
-
Okay. That helps. Thanks Bob.
Andrew.
For folks like yourself who are intimately acquainted with the implementation details of MQTT, we probably wouldn't have approached it that way, instead preferring a full API with individual instructions for connecting, disconnecting, etc. The reality is that vast majority of our users just want to make MQTT work, with zero insight into the sausage making process.
-
It should be fairly easy to add a DEVWRITE property to override the client name.
Did this make it into 2.6? I don't see it as a property in the GUI or the help docs.
Andrew.
-
It should be fairly easy to add a DEVWRITE property to override the client name.
Did this make it into 2.6? I don't see it as a property in the GUI or the help docs.
Andrew.
It didn't. I'm sorry. Just added...it'll be in 2.7.
-
Thanks Bob.
-
Bob, I'm still struggling with the CONNECT/DISCONNECT issue. The current design seems sensible but since I'm using Stage programming I think that's causing issues.
I have a project that uses a FIFO to send data via MQTT. If the Publish fails it runs a timer for 30 seconds then retries until successful. I'm finding that if the broker is down and the FIFO starts to fill, on reconnect the PLC attempts to publish each message as desired, but only every other message makes it to the broker. This might be a timing thing but the messages that get lost show in Logger as being processed during MQTT Disconnect. All very strange.
Since each individual message is connecting, publishing and then disconnecting, I still have the original concern about LWT processing not functioning properly.
Is there a way I can program this so that it works properly or do I need to request again that you add CONNECT and DISCONNECT functionality?
Logger output is below (latest at the top). I can email you the program export if necessary.
Thanks, Andrew.
---- LOGGER STARTS ----
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.799 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.798 ,"MQTT disconnected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.797 ,"Disconnecting MQTT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.745 ,"Sending DISCONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.743 ,"Stage S20 / 5 / 0 / 1561924365 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.740 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.739 ,"PUBACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.737 ,"Sending PUBLISH for V2/TEST/4"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.736 ,"CONNACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.733 ,"Sending CONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.729 ,"MQTT broker connected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.728 ,"Connecting to MQTT broker..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.721 ,"Stage S20 / 4 / 0 / 1561924364 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.721 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.720 ,"MQTT disconnected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.719 ,"Disconnecting MQTT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.648 ,"Sending DISCONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.648 ,"Stage S20 / 2 / 0 / 1561924363 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.647 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.646 ,"PUBACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.646 ,"Sending PUBLISH for V2/TEST/3"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.645 ,"CONNACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.645 ,"Sending CONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.644 ,"MQTT broker connected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.644 ,"Connecting to MQTT broker..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.643 ,"Stage S20 / 3 / 0 / 1561924362 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.643 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.642 ,"MQTT disconnected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.641 ,"Disconnecting MQTT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.502 ,"Sending DISCONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.501 ,"Stage S20 / 1 / 0 / 1561924362 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.501 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.499 ,"PUBACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.498 ,"Sending PUBLISH for V2/TEST/5"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.497 ,"CONNACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.495 ,"Sending CONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.493 ,"MQTT broker connected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.492 ,"Connecting to MQTT broker..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.490 ,"Stage S20 / 5 / 1 / 1561924360 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.489 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.487 ,"MQTT disconnected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.485 ,"Disconnecting MQTT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.413 ,"Sending DISCONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.412 ,"Stage S20 / 4 / 1 / 1561924360 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.412 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.411 ,"PUBACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.411 ,"Sending PUBLISH for V2/TEST/3"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.410 ,"CONNACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.410 ,"Sending CONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.409 ,"MQTT broker connected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.409 ,"Connecting to MQTT broker..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.409 ,"Stage S20 / 3 / 1 / 1561924359 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.408 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.408 ,"MQTT disconnected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.407 ,"Disconnecting MQTT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.262 ,"Sending DISCONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.261 ,"Stage S20 / 2 / 1 / 1561924358 "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.260 ,"MQTT OK "
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.258 ,"PUBACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.256 ,"Sending PUBLISH for V2/TEST/1"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.255 ,"CONNACK received"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.254 ,"Sending CONNECT..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.253 ,"MQTT broker connected!"
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.252 ,"Connecting to MQTT broker..."
10.0.75.1 Port 29298(0x7272),06/30/19 12:53:09.251 ,"Stage S20 / 1 / 1 / 1561924358 "
---- LOGGER ENDS ----
-
Can you add single dummy subscription globally, effectively a keep-alive?
Since the first item enabled will force the CONNECT, and the last item disabled will force the DISCONNECT, you can easily control that.
-
That helped Bob. I'm losing the first PUBLISH after reconnect though (other messages seem fine). I'm thinking that it's a timing issue. It feels like the success bit is being set before the entire dialog has completed - is this possible?
-
That helped Bob. I'm losing the first PUBLISH after reconnect though (other messages seem fine). I'm thinking that it's a timing issue. It feels like the success bit is being set before the entire dialog has completed - is this possible?
Success and error have a slightly different meaning in comm function like this. In continuous operation, they wiggle every time a pass completes. Continuous mode isn't really compatible with Stage. In edge mode, they should turn off as soon as the edge is latched in and one turn on in response to the completion of the box. Unless it is broken, setting the result is a critical part of the box ending. MQTT boxes are kinda fickle though, because the box doesn't really do the work, it interacts with the driver.
We banged on it pretty hard, but I know I didn't do it in stage. We'll do some testing.
-
That sounds good. I was able to get the first PUBLISH out to the broker on a couple of tests so it definitely seems like a timing option. I'll leave the 1-second delay in there as a workaround. Let me know if you need me to test any further.
-
I created a simple test with a dedicated publishing program that reads from a FIFO. I added a single MQTTSUB to $Main and use it to connect and disconnect. I can't see that it is ever dropping the first publish following a reconnect.
I modded the program to trigger the FIFO load directly from the same bit that triggers the MQTTSUB, forcing those two operations to happen at virtually the same time. It still doesn't fail for me ever. I'm talking to a local instance of mosquitto, so it could be different off-site.
-
Not sure why this didn't post...
-
It all seems to work fine if the broker's there. Try shutting the broker down, stick a bunch of things in the queue, then see what happens when the PLC reconnects after you restart the broker.
-
With the broker shut down, I would expect the MQTTPUB to fail and jump to the error stage. If it is important to publish everything without fail, you just need to jump back to the publish stage until it succeeds, rather than jumping back to the FIFOUNLOAD. If it is claiming success without completing the publish, that?s a bug.
I?ll play with it more.
-
If it is claiming success without completing the publish, that?s a bug.
That appears to be what I've seen. It doesn't happen every time but I believe I've seen it multiple times.
-
I tested two ways: 1) pulling the cable, but leaving everything on both sides running, and 2) killing the broker.
When killing the broker, I occasionally see a bit of weirdness on mosquitto's side, where he is publishing more than once back to the PLC (as part of the subscribe), suggesting he saw the publishes from the PLC, but he doesn't actually show the publishes from the PLC and the data sent to the PLC is old. That would appear to be mosquitto bug.
When leaving the broker running and pulling the cable, the PLC is giving me a high-five on the publish as long as the connection hasn't yet timed out. It starts returning errors when the connection closes. That's a bug on our side. I'll look into it.
-
That appears to be what I've seen. It doesn't happen every time but I believe I've seen it multiple times.
I figured out my issue. It was set to QoS 0, which doesn't have any form of error checking. the publish packets get dropped into the socket queue, and even though they don't really get sent, the application layer doesn't really know that. When I changed it to QoS 1, it is correctly reporting the error, and the retry code in my PLC program got the result through.
-
That's interesting. My QoS setting is already set to 1 though. I'll do some more testing when I have a moment.
-
That's interesting. My QoS setting is already set to 1 though. I'll do some more testing when I have a moment.
I'm not saying there isn't an issue, but I haven't been able to demonstrate a problem once I got the QoS correct. There was definitely an issue with mosquitto, but it only showed when I shut it down and started it.
How are you handling the retry in your FIFO publishing program? All I'm doing is logging the retry in the error stage and jumping right back to the publish stage. It doesn't go back to the FIFO fetch stage until it successfully completes.
-
I'm doing something similar. I pop a record from the FIFO and store it in a UDT Structure. Then I attempt to PUBLISH it. If it fails I jump to a new stage, wait a minute, and then jump right back to the PUBLISH stage and attempt to resend. It keeps doing this until it successfully sends it. On success it checks the FIFO again and everything moves forward.
The failure scenario I've been testing has been against a local dockerised Mosquitto broker that I kill to simulate a network/server failure. Then I dump some data into the FIFO while the broker's down, and I see what happens when the broker restarts.
The issue seems to be that the PUBLISH returns success before it's actually done processing (the same issue you mentioned). I'll see if I can grab some logs that include the lost-data scenario.
-
If you are restarting mosquitto, it could be their bug. What I saw in their verbose dump was them publish to the PLC (it was subscribed to the same topic I'm publishing) in response to the PLC publishing to mosquitto, but the log didn't show mosquitto ever receiving the publish from the PLC to start with. That can't be, because they wouldn't publish multiple times due to a single subscription, and it was showing more than one. It appears that they are getting tripped up during a pregnant moment at startup.
When it works correctly, I see both the publish from the PLC to mosquitto, and mosquito publishing back to the PLC's subscription. When it fails, I just see the effect, and not the cause.
If you aren't already, run mosquitto with -v.
-
Interesting. I'll see if it does the same thing against VerneMQ (probably later this week).
-
Messed with it a bit more today. I'm gonna walk some of that back. I do see cases where a publish isn't happening even though it should be.
And now I can't duplicate the behavior I was seeing from mosquitto. Weird.
-
Weird.
Welcome to my life ;-)