Yes, N memory closely matches the modbus holding registers. I would assign N0 to one motor and N1 to the other.
You can reference the 0 bit by casting, N0:0 is bit 0 of N0, N0:8 is bit 8. You can then toggle these on or off as a coil in your program, and as long as N0 gets written to your motor controller, it will receive that word containing those bits.
Where it gets confusing is you say you write a 2 to the word, and it starts the motor. The number to is actually 10 in binary, bit 1 true, bit 0 false. And when you say 512 controls reverses direction, that's actualy 1 0000 0000, or only bit 9 true.
As far as stepping from 2, 4, 8, etc, you're loosing me there. Do you have a manual for the controller? Typically each bit in the word means something, so by doing 2, 4, 8, etc you are actually advancing through each bit for a moment, so maybe that in itself is doing something?