News:

  • October 18, 2026, 07:51:34 PM

Login with username, password and session length

Author Topic: MQTT/IoT  (Read 60629 times)

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
MQTT/IoT
« on: June 12, 2017, 08:02:15 PM »
Too much smoke to ignore. Between the multiple requests that have come from Do-more users, and companies like GE and IBM spending prime time advertising money to promote it, tells me that this going mainstream.

You have my attention. I am currently envisioning client only, we'll leave the brokers to others for now.

A few quick questions that come to mind:
How many topics would you expect to publish or subscribe to?
How would you like to receive notification of subscription state changes?
How would you expect to gate publishing...time, event, topic state changes?
How many brokers would you expect to connect to?
What QoS would you expect to support?

And feel free to offer answers to questions I'm not yet smart enough to ask...
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: MQTT/IoT
« Reply #1 on: June 12, 2017, 09:47:05 PM »
I have yet to come across anything that supports it, although I really haven't been looking for it.

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #2 on: June 12, 2017, 09:58:54 PM »
I have yet to come across anything that supports it, although I really haven't been looking for it.

They are getting to the factory floor with bridges, but the protocol is very simple and tailored to devices with very limited resources. We've had multiple requests and it's pretty painless to implement, so I'd like to go ahead and jump on it.

I'd like to do DNP3 soon too, but it isn't as simple as MQTT, so I think we'll start with MQTT.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

HB_GUY

  • Full Member
  • ***
  • Posts: 43
Re: MQTT/IoT
« Reply #3 on: June 14, 2017, 12:36:06 PM »
The way I thought about it goes kinda like this.
Have a new DM instructions for publish and subscribe, that will allow you to define the broker you are connecting to.
Then, like the current publish or subscribe commands that already exist, have a list of Topics, and the memory locations associated with each topic...

Quote
A few quick questions that come to mind:
How many topics would you expect to publish or subscribe to?
personally, I could see publishing probably 40 or 50 topics as of right now.
But of course, as I would start to implement this, I'm sure that would grow considerably...

Quote
How would you like to receive notification of subscription state changes?
email or text messages, set a bit and allow HMI or SCADA handle

Quote
How would you expect to gate publishing...time, event, topic state changes?
Maybe the command could be setup to either fire every scan, or at time intervals.
This way we could use logic to control whether it is active on a scan by scan basis...

Quote
How many brokers would you expect to connect to?
Me personally, probably only 1. MAYBE, 2.

Quote
What QoS would you expect to support?
I could see use cases for all 3 QOS levels.
« Last Edit: June 14, 2017, 12:44:27 PM by HB_GUY »

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #4 on: June 14, 2017, 01:09:12 PM »
That's where I think we go, with one minor difference. The protocol and associated session need to live in a device, so I think there is an MQTT client device that gets created for each broker, and then zero to n MQTTPUB/MQTTSUB instructions per client device. Each MQTTxxx instruction will be table based, and probably support up to 50 topics.

Since the MQTT model is event based, we'll probably allow publishing to occur either on time and/or state change. On subscription, I'm thinking an optional flag per topic that indicates when fresh data was received. All of that is subject to change as we start implementing.

I'm really surprised that there isn't more noise about this on the shop floor, but I think it is inevitable. I've only done a cursory review of the protocol, but it seems very easy, and makes it trivially easy to expose whatever the machine designer deems useful. It feels to me that it will be successful for the same reasons Modbus is, but with the added advantage of being standardized and pushed by a consortium of large companies with deep pockets.

I might be wrong on this, but it is still easy to justify due to its simplicity.
« Last Edit: June 14, 2017, 01:10:53 PM by BobO »
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

Controls Guy

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 3628
  • Darth Ladder
Re: MQTT/IoT
« Reply #5 on: June 14, 2017, 07:34:35 PM »
personally, I could see publishing probably 40 or 50 topics as of right now.
But of course, as I would start to implement this, I'm sure that would grow considerably...

As you can well imagine, no one will EVER need more than 640K RAM.   :D
I retract my earlier statement that half of all politicians are crooks.  Half of all politicians are NOT crooks.  There.

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: MQTT/IoT
« Reply #6 on: June 14, 2017, 09:12:51 PM »
Would this replace OPC?

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #7 on: June 14, 2017, 09:33:27 PM »
Would this replace OPC?

Not sure. Conventional PLC protocols (and by extension, OPC) allow you to read or write almost anything in the PLC. From a security standpoint, that's bad, but it is the model that most are familiar with.

Under MQTT, the client device publishes relevant data and subscribes to whatever it needs from other clients, but it has no ability to dictate that a remote device give it something or force it to take something. From a security standpoint that is desirable, but it will force designers to think about the problem differently.

It won't take a clever engineer long to figure out that a published topic could actually contain a request for another client to publish something, so there are definitely ways around it. Mr. Reuben Goldberg might be proud, but that's never stopped an engineer.

The other question is performance. I will reserve judgment on that until we get into the implementation.

What I do like though is how easy it will be to make data available to other machines and enterprise level systems. Once this starts getting into a machine designer's thought process, they will come up with many uses for IoT.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #8 on: February 08, 2018, 02:44:40 PM »
Bump.

Starting to think more specifically about this. Current vision (initial direction of implementation) is to have MQTT Publisher and MQTT Subscriber devices independently, with one or more MQTTPUB/MQTTSUB instructions per device. Only one instruction can be enabled on a device at a time and each instruction will support up to 32 topics. You will be able to switch from instruction to instruction, but this will drop the connection to the broker and reopen it. In general, it is best to think in terms of one MQTT Publisher to one MQTTPUB, and same for subscribers.

If you need more than 32 topics in one direction, you can create another client/box pair.

I'm a little fuzzy on topic data. According to the spec, the topic message is application specific, and can apparently be just about anything. This presents the potential for significant effort if you want a "be all/end all" conversion facility built into the instruction. On the other hand, I'm guessing that 99% of what's out there is string data, so it makes more sense to publish all numerical values (V, N, D, R, etc) as string, and attempt to convert subscription data to numerical based on the output parameter (string to int or string to real). We will definitely allow strings as both both pub and sub parameters, so in the case that that the message is actually binary, you could publish to a string and then use STRGETB to extract the bytes to the relevant data type. What we are trying to avoid is overly complicating it: options for binary or string, byte swap, word swap, radix, byte/word/dword data sizes...etc. It gets ugly fast, and you still end up failing when the inevitable oddball comes along. I would rather it be easy for 90%, with a decent workaround for the rest.

Publishers will define an update rate, and then have the option of updating a topic only if it has changed. So you might set the update rate to 1 second, but only the topics that have changed since the last update will be sent. Although thinking about it, that would only work for numerical topic values...since it isn't practical to buffer the strings. I would expect much of the published data to be numerical though, so I want to support an event based update for those.

We are thinking that the MQTTSUB will have an optional DWORD parameter, where as a topic is updated, we'll set the corresponding bit. User can clear after he dispositions.

Not sure how to handle LWT. For publishers, that's pretty easy. Subscribers, it gets ugly-ish. I guess we could do something like a string parm for the last LWT we received, and an optional DWORD parm where we set bits for each topic we received LWT from. Since most LWTs shouldn't come in at the same time, that would be a solid 9 of 10.

Thoughts? We will be starting this soon-ish.
« Last Edit: February 08, 2018, 02:51:41 PM by BobO »
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #9 on: February 14, 2018, 10:33:06 AM »
So nothing?
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

Tark

  • Jr. Member
  • **
  • Posts: 16
Re: MQTT/IoT
« Reply #10 on: February 22, 2018, 11:11:16 AM »
Ok, I’ll bite.

First let me say I love what you’ve done with the Do-More.  I’m replacing about 16 DL-250s with Do-Mores, they are fantastic.  The recent addition of the User Data Types just made it even better.  On a side note, it would be great if a User Data Type could include additional structures like timers, counters, or even other User Data Types.  It could be a Complex Data Type.  Needless to say, you’ve done a great job with the Do-More, give yourselves at least 50 pats on the back and a At-A-Boy for good measure.

What I envision being able to do with MQTT in Do-More is this – Take a Heap Item and make it a MQTT topic.  So I could create a Pump User Data Type and then create Heap Items Pump1, Pump2, Pump3.  Then I could publish those Heap Items as MQTT topics.  Now in my SCADA I can subscribe to Pump1, Pump2, Pump3.  If I make changes to the Pump User Data Type structure in the PLC, then this will be updated in my SCADA.  I could also have PLC_B subscribe to the Pump1, Pump2, Pump3 topics in PLC_A.

I also envision the Heap Item being a Pub/Sub, so the SCADA can subscribe to the heap item and publish to it as well.  So a heap item would have read only and read/write values.

I would implement the Sparkplug MQTT topic and payload definition.

And I would consider talking with the folks over at Inductive Automation.

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #11 on: February 22, 2018, 12:26:06 PM »
Great information!

At first read, it seems like that is considerably more complex that we were envisioning, at least for now. Given the relatively nascent state of MQTT at the shop floor, I am very reluctant to do anything more than a very generic MQTT publish and subscribe.

We are definitely kicking around the idea of doing a data to JSON and JSON to data conversion mechanism in a future release, so I could see other data conversions happen as well. Even at that, we would probably leave the MQTT pub/sub fairly generic and push the payload definition into other instructions. As long as we give you the ability to publish and subscribe to a raw data buffer, you should be able to funnel the payload in or out of data conversion functions and get pretty much exactly what you are wanting.

In the meantime, I don't see any reason that you couldn't implement their interface with a little elbow grease. Not as convenient as what you are describing, but not overly hard either. I think a major fitness function for our MQTT implementation is going to exactly this: can I, without crazy amounts of effort, get data to and from common formats? Then over time as certain standards prove they are in fact standard, we'll look at canning those in purpose-built instructions.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: MQTT/IoT
« Reply #12 on: February 24, 2018, 11:21:23 AM »
I might be wrong, but I think you might see more interest in supporting EIP implicit messaging for I/O and Implementing EDS file configurations. There are tons of products out there that support it, albeit Modbus is as well. However, from what I have been working with, EDS implementation of implicit EIP is widely accepted and it reduces programing time.

BobO

  • Host Moderator
  • Hero Member
  • *****
  • Posts: 6177
  • Yes Pinky, Do-more will control the world!
Re: MQTT/IoT
« Reply #13 on: February 24, 2018, 11:29:07 AM »
I might be wrong, but I think you might see more interest in supporting EIP implicit messaging for I/O and Implementing EDS file configurations. There are tons of products out there that support it, albeit Modbus is as well. However, from what I have been working with, EDS implementation of implicit EIP is widely accepted and it reduces programing time.

ADC suggested that we didn't want to tackle EIP implicit in the CPU due to complexity, but we are planning a module.

These are really different needs though. IoT isn't about factory floor control, it's about enterprise level integration. And it's a small fraction of the effort that EIP implicit is.
"It has recently come to our attention that users spend 95% of their time using 5% of the available features. That might be relevant." -BobO

ATU

  • Internal Dev
  • Hero Member
  • ****
  • Posts: 2126
  • YKPAIHA
    • ATU, Inc.
Re: MQTT/IoT
« Reply #14 on: February 24, 2018, 12:06:52 PM »