Routed Ethernet

[AngrySparky]

I have an odd setup that I can't seem to get to work and I want to verify that its an IT problem and not on my end. I have two facilities with ControlLogix equipment that I am gathering data from using a BRX. These facilities where and still are connected together with a T1 line with everything being on one network (AAA.BBB.CCC.). We want to move away from the T1 line so IT has firewalls and VPNs and stuff setup between the plants now, however in order to run through this new connection stuff has to be on a different network (AAA.BBB.DDD.) than the original network. I have a BRX installed at one location with the onboard ethernet port and an ethernet POM. The Onboard ethernet port is set to the new network (AAA.BBB.DDD.) and I have the gateway set and all that. The POM module is set to the original network (AAA.BBB.CCC.) and is connected to the process equipment for data logging. What I need to do is connect to the process PLC at the second location which is on the original network (AAA.BBB.CCC.) through the new vpn network setup (AAA.BBB.DDD.). As long as the gateway for the new network (AAA.BBB.DDD.) is setup in the BRX is there any reason this shouldn't work?

[franji1]

What specific Ethernet POM?
ECOMEX or ECOMLT?

What are the Subnet Masks for your Onboard and for your POM?

[BobO]

I have an odd setup that I can't seem to get to work and I want to verify that its an IT problem and not on my end. I have two facilities with ControlLogix equipment that I am gathering data from using a BRX. These facilities where and still are connected together with a T1 line with everything being on one network (AAA.BBB.CCC.). We want to move away from the T1 line so IT has firewalls and VPNs and stuff setup between the plants now, however in order to run through this new connection stuff has to be on a different network (AAA.BBB.DDD.) than the original network. I have a BRX installed at one location with the onboard ethernet port and an ethernet POM. The Onboard ethernet port is set to the new network (AAA.BBB.DDD.) and I have the gateway set and all that. The POM module is set to the original network (AAA.BBB.CCC.) and is connected to the process equipment for data logging. What I need to do is connect to the process PLC at the second location which is on the original network (AAA.BBB.CCC.) through the new vpn network setup (AAA.BBB.DDD.). As long as the gateway for the new network (AAA.BBB.DDD.) is setup in the BRX is there any reason this shouldn't work?

BRX is functionally one stack with two NICs. The gateway is for routing things that *aren't* on the specified network(s). Anything directed to A.B.C.? or A.B.D.? will go to the appropriate port, based on netmask. Anything directed to something *other* than those two networks will be directed to the gateway. There should be only one gateway. It can be on either the internal or POM, but if both are configured, the internal will be used.

Netmask defines normal network routing, gateway catches everything else.

[AngrySparky]

It is an EcomEX POM. The subnets for each port are set to .255.255.255.0.

Thanks Bob for the information. I need to think about what that means and how I can get this to work. There is a second network on the ControlLogix at each location. Maybe Ill have to move my POM to that network for local network.

[BobO]

It is an EcomEX POM. The subnets for each port are set to .255.255.255.0.

Thanks Bob for the information. I need to think about what that means and how I can get this to work. There is a second network on the ControlLogix at each location. Maybe Ill have to move my POM to that network for local network.

I said netmask but I meant the network address implied by the ANDing of the address and netmask. If your internal address is 1.2.3.100 and your netmask is 255.255.255.0, the internal network is 1.2.3.0. Likewise if your POM address is 1.2.4.100 with netmask of 255.255.255.0, the network is 1.2.4.0. The stack checks the target address against those networks and directs to the NIC that matches. If your target was 5.6.7.100 and didn't match either local network, the message would get forwarded to the gateway (which is likely something like 1.2.3.1 or 1.2.4.1) so the gateway can route it wherever.

[AngrySparky]

Ok I see from your explanation that I will have to change this up a little. In the current setup with the T1 line all the equipment was on the same network across both plants, 1.2.3.x. I'm dealing with a spider nest of an old SCADA system so I was trying to avoid changing the IPs on the process PLCs just yet. In order for the new connection to work with ITs firewall I can't have the two plants on the same network so we set up a new network 1.2.4.x. IT has the firewall configured to allow the 1.2.3.x network to talk to the 1.2.4.x network. My hope was that with the brx I could use one port on the brx on the 1.2.3.x network to connect to the process PLC locally. Then use the second port on 1.2.4.x to connect to the process PLC at the other plant which is on the 1.2.3.x network by routing it through the firewall. From your explanation I don't think this will work now. There is a second network on the process PLC 1.2.5.x. I'm thinking I will have to switch one of the brx ports to this 1.2.5.x network and use that to get data from the local process PLC. Then hopefully I can connect to the second process PLC on the 1.2.3.x network via the 1.2.4.x port on the brx and the routing setup between the 1.2.4.x and the 1.2.3.x network.

Am I on the right track?

[BobO]

Ok I see from your explanation that I will have to change this up a little. In the current setup with the T1 line all the equipment was on the same network across both plants, 1.2.3.x. I'm dealing with a spider nest of an old SCADA system so I was trying to avoid changing the IPs on the process PLCs just yet. In order for the new connection to work with ITs firewall I can't have the two plants on the same network so we set up a new network 1.2.4.x. IT has the firewall configured to allow the 1.2.3.x network to talk to the 1.2.4.x network. My hope was that with the brx I could use one port on the brx on the 1.2.3.x network to connect to the process PLC locally. Then use the second port on 1.2.4.x to connect to the process PLC at the other plant which is on the 1.2.3.x network by routing it through the firewall. From your explanation I don't think this will work now. There is a second network on the process PLC 1.2.5.x. I'm thinking I will have to switch one of the brx ports to this 1.2.5.x network and use that to get data from the local process PLC. Then hopefully I can connect to the second process PLC on the 1.2.3.x network via the 1.2.4.x port on the brx and the routing setup between the 1.2.4.x and the 1.2.3.x network.

Am I on the right track?

Sounds like it. As long as 1.2.4.x has a route to 1.2.3.x accessible through a 1.2.4.x gateway, and the PLC's gateway is set to the 1.2.4.x gateway (for the PLC's 1.2.4.x port), it should work. Just note that in that configuration there will be no routing on the 1.2.5.x network.

[AngrySparky]

Hmm, I am very confused now. I took a second brx that I had and setup a test. I set its address up to A.B.252.95, my process PLC is at A.B.245.120. The .252 and .245 networks are routed together. I put a ping instruction in and it worked perfectly, I setup up my ethernet IP module in the process PLC (controllogix) and it connected up just fine. Ok that tells me the connection should work. So I go to my other BRX and change it, the main port remains at A.B.252.40, I moved the POM module from A.B.245.40 to A.B.19.40, as the local process PLC has location of A.B.245.122 and A.B.19.10. Got everything converted and got connection to the local process PLC on the .19 network restored. I still can not connect to the remote process PLC on the .245 network. A Ping from the BRX will not work and the ethernet module from the controllogix will not connect.  Gateways and configurations should be the same between the test PLC at .95 and my actual PLC at .40. I just tried switching the address of my in use BRX from .40 to .95 and it still didn't work.

[BobO]

Hmm, I am very confused now. I took a second brx that I had and setup a test. I set its address up to A.B.252.95, my process PLC is at A.B.245.120. The .252 and .245 networks are routed together. I put a ping instruction in and it worked perfectly, I setup up my ethernet IP module in the process PLC (controllogix) and it connected up just fine. Ok that tells me the connection should work. So I go to my other BRX and change it, the main port remains at A.B.252.40, I moved the POM module from A.B.245.40 to A.B.19.40, as the local process PLC has location of A.B.245.122 and A.B.19.10. Got everything converted and got connection to the local process PLC on the .19 network restored. I still can not connect to the remote process PLC on the .245 network. A Ping from the BRX will not work and the ethernet module from the controllogix will not connect.  Gateways and configurations should be the same between the test PLC at .95 and my actual PLC at .40. I just tried switching the address of my in use BRX from .40 to .95 and it still didn't work.

Just an FYI, pings generally aren't routed.

Please screenshot the local and POM configs and post them.

[AngrySparky]

Ok, I've done some more testing this morning and convinced myself now that everything is working, its just my network connection is garbage right now. I actually ended up swapping PLCs and loading my full program onto the one I was using as a test yesterday. I was able to see some connections then. I loaded my simple 1 rung test program onto the PLC I had previously been using and saw some connections as well. Right now even when I do get a successful ping I'm at 1500-2000 ms with probably 75% that fails. This is on ITs end and comes and goes, they are supposed to be working on it. It does seem that the test program has higher success but its literally only pinging the other PLC once per minute, the actual program is streaming data to a PC with dmlogger, talking to the local process PLC, etc. so I guess this doesn't really surprise me that much. I think understanding how the gateway needed to work was the key on my end, now I think its the network connection not my setup causing the issues.