RF Mote Web UI : Program all the RF things

Introduction

Deploying a bunch of remote nodes and a gateway into the wild presents a number of issues. What does each do? What is connected to it? How often should it tell me what’s up?

Typically wrapping up a group of remote sensors requires a unique program for each node, programming and debugging each node locally with a serial interface, and programming a gateway/data aggregator to understand each individually. This can be a bit of a nightmare, and we really wanted a solution where we would never have to get out our FTDI cable to program a micro directly!

We’ve addressed all of these issues with our UniMote code and our gateway UI for messaging remote nodes. Read on.

UniMote for universal code and remote config

One thing we all loathe is repeating ourselves. Another is constant maintenance of multiple instances of code, even if they are quite similar (and often even more so if they are). For this reason, we’ve built a UniMote sketch that does, well, everything. It works on a battery-powered mote as well as it does on a sensor gateway. You can configure it over serial or RF with simple commands. You can read about that here and here.

Gateway SerialHandler to dispatch and process

On a gateway, we have a mote connected by serial, often to a Raspberry Pi, for example on our Pi HAT.  On our gateway, we need to both listen for incoming messages from our mote, as well as send messages when necessary.

To facilitate this, we set up a pretty simple serial message handler. It continuously listens, and dispatches messages as they show up in a queue set up in a database, This database houses received messages, queued messages, as well as those that have been sent, time-stamped of course. It’s in python, uses pyserial, and is pretty darned straightforward. In the form you’ll find it, it does use our pilib, but you can easily cut and paste if you want a standalone version.

 serialhandler.py

UI to queue serial commands

Our serial handler does nothing except process commands put into a queue, so all we need to do with a UI is to present a way of inserting messages into it. This is pretty easy, and based on things we have talked about previously. We create a UI that executes a simply wsgi function to insert entries in the serial queue. We add some css, and we get the UI below:

Terminal UI for entering sending messages to our motes.
Terminal UI for entering sending messages to our motes.

We can pretty easily see the messages as they go out and are received. Simple stuff.

Mote Control Panel

Now, on top of this basic goodness, we’re going to build something that resembles a control panel, where we can view each node’s vitals, update data, and view what the status of all the IO are. We do it using the same basic methods above, combined with the databased data retrieval we’ve talked about here and here. The end result is a browser-determined data retrieval sequencing that complements the node-side code that is set up to deliver data when the node feels like it. This will be the bedrock that our remote thermostat/PLC code rests firmly on. Here are a couple videos with an explanation and demo of the control panel interface:

Overview:

Demo:

UniMote v2 : Read all the RF things

Introduction

Code discussed below can be found here: newest mote sketch

In our last UniMote installment, we outlined our UniMote program. A one-size fits all sketch for our avr-based, RF-enabled microcontrollers, our sketch has the following main features:

  • Numbered, sequential access to all available IO on R4 and Moteino Mega microcontrollers
  • Remote, programmable IO modes (DI, DO, AIN, PWM, 1Wire)
  • 8 channels with configurable positive/negative feedback and deadband
  • Configurable IO and channel status reporting
  • Remote command for RF send
  • Daisy-chainable commands, e.g. send command to mote to send command to mote to send command, etc.
  • Support for low power sleep modes
  • Configurable program control
  • Remote (RF) and local (serial) modification of all control parameters

What we wanted to add, however, were a few more features.

  • RF parameter readout: While we already had local parameter readout (with the listparams command), we didn’t have a readout feature for communication over RF. For devices that don’t sleep, it’s very useful to poll parameters when necessary. So with the previous feature set, we have configurable bidirectional communication. We can confirm commands, read out all setpoints, process values, io/channel statuses, and program configuration. This is great for a status UI.
  • Command response with status: Like an ACK with useful information, all commands to the remote will now receive a response both acknowledging the command as well as a status response. This will alert us to errors with our command, should we try to set a parameter incorrectly.
Code cleanup:

In addition to the above, there was a lot of code cleanup and optimization to be done. We removed all string objects and replaced them with character arrays to save space and avoid seg faults. We also employed a great library that stores strings (where necessary) in flash memory, to avoid the RAM usage that results in unpredictable behavior when collisions occur. It saves a ton of memory, especially compared to sprintf. Best of all, we fit all of the above into the memory space of at ATMega328P. With no further ado, let’s introduce the features.

Gateway/Mote Universality

Another thing we didn’t like was that we had to have unique gateway and mote sketches. We’ve done away with that. A gateway and a mote are the same. Just change the NODEID and sleep parameters, if you wish.

RF Parameter Readout

In the previous version of UniMote, we had a listparams command that would list all the Mote parameters. It looked something like this:

~listparams
Command character received
NODEID:0,
GATEWAYID:0,
NETWORKID:0,
LOOPPERIOD:2000,
SLEEPMODE:0,
iomode:[0,0,0,0,0,3,0,3,0,0,0,0,0],
ioenabled:[0,0,0,0,0,0,0,0,0,0,0,0,0],
ioreadfreq:[10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000],
ioreportenabled:[0,0,0,0,0,0,0,0,0,0,0,0,0],
ioreportfreq:[0,0,0,0,0,0,0,0,0,0,0,0,0],
chanenabled:[0,0,0,0,0,0,0,0],
chanmode:[0,0,0,0,0,0,0,0],
chanposfdbk:[0,0,0,0,0,0,0,0],
channegfdbk:[-1,0,0,0,0,0,0,0],
chandeadband:[0,0,0,0,0,0,0,0],
chanpvindex:[5,0,0,0,0,0,0,0],
chansv:[15.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00]

This worked well, but this didn’t work for reporting over RF, as each message is only 61 characters long. OOPS. So we broke the reporting into a number of “modes” that determined what would be reported in each chunk. This makes it more efficient anyway: we don’t always need all of the data. In fact, we rarely do. We made all of these functions work for both local and radio readout. So a listparams command has the following format:

~listparams;iovals;0

This would list the first chunk of io values to the Serial display:

cmd:lp,iov:0000.0000,0000.0000,0000.0000,0000.0000,0000.0000

If we change the third parameter to something non-zero, the data will also be sent to that node number over RF. Simple as that. Each chunk is formatted to be 61 characters or less, and is listed in the table below. It’s all json-encoded, too, which makes it easy to extract with our gateway serial handler.

Listparams commands:
Command Description Example Output
cfg mote configuration cmd:lp,node:10,gw:1,nw:10,loop:1000,slpmd:0,slpdly:1000
iomd io modes cmd:lp,iomd:0,0,0,0,0,0,0,0,0,0,0,0,0
ioen io enabled cmd:lp,ioen:0,0,0,0,0,0,0,0,0,0,0,0,0
iov io values 0-4 cmd:lp,iov:0000.0000,0000.0000,0000.0000,0000.0000,0000.0000
iov2 io values 5-8 cmd:lp,iov2:0000.0000,0000.0000,0000.0000,0000.0000,0000.0000
iov3 io values 9-13 cmd:lp,iov3:0000.0000,0000.0000,0000.0000,0000.0000,0000.0000
iordf io read frequencies (100ms increments) cmd:lp,iordf:10,10,10,10,10,10,10,10,10,10,10,10,10
iorpf io report frequencies (100ms increments) cmd:lp,iorpf:10,10,10,10,10,10,10,10,10,10,10,10,10
chen channel enabled statuses cmd:lp,chen:0,0,0,0,0,0,0,0
chmd channel mode (reserved) cmd:lp,chmd:0,0,0,0,0,0,0,0
chpf channel positive feedback indices cmd:lp,chpf:00,00,00,00,00,00,00,00
chnf channel negative feedback indices cmd:lp,chnf:00,00,00,00,00,00,00,00
chdb channel deadband values cmd:lp,chdb:00,00,00,00,00,00,00,00
chpv channel process values 0-3 cmd:lp,chpv:0000.0000,0000.0000,0000.0000,0000.0000
chpv2 channel process values 4-7 cmd:lp,chpv2:0000.0000,0000.0000,0000.0000,0000.0000
chsv channel setpoint values 0-3 cmd:lp,chsv:0015.0000,0000.0000,0000.0000,0000.0000
chsv2 channel setpoint values 4-7 cmd:lp,chsv2:0015.0000,0000.0000,0000.0000,0000.0000

Command response with status

Since we’ve built the UniMote to receive commands from a remote, it’s very useful to receive a reply with two key pieces of information:

  • An echo of the command sent to confirm that it was received as sent
  • A status to indicate success, failure, and an error description of the sent command

So, when a command is sent, we get a short reply indicating the command, and the status. So, for example, if we want to change the loopperiod of the mote, we would see the following:

Issued command:
~sendmsg;10;;~lp;cfg;1

This command to the gateway (NODEID=1), sends a message to node 10 with the content “~lp;cnf;1”. When the node receives the command, it will send listparams configuration to node 1 (back to the gateway). On the remote node 10, we see on serial output:

BEGIN RECEIVED
nodeid:1, ~lp;cfg;1, RX_RSSI:-26
END RECEIVED
SENDING TO 1
cmd:lp,node:10,gw:1,nw:10,loop:1000,slpmd:0,slpdly:1000
SEND COMPLETE
Response from the mote:

And, sure enough, back on the gateway, we get the response:

BEGIN RECEIVED
nodeid:10, cmd:lp,node:10,gw:1,nw:10,loop:1000,slpmd:0,slpdly:1000,RX_RSSI:-23
END RECEIVED

Now let’s try a set command, where we expect a status response

Issued command:
~sendmsg;10;;~modparam;loopperiod;10

This command to the gateway (NODEID=1), sends a message to node 10 requesting that the loopperiod be changed to 1s (100ms increments). Sure enough, we receive the command, it is executed, and a reply message is sent that confirms the action:

BEGIN RECEIVED
nodeid:1,~modparam;loopperiod;10,RX_RSSI:-23
END RECEIVED
cmd:mp,param:loopperiod,setvalue:10,status:0
Sending message to replynode: 1

And back on the sending node, we receive the reply:

BEGIN RECEIVED
nodeid:10,cmd:mp,param:loopperiod,setvalue:10,status:0,RX_RSSI:-34
END RECEIVED

Our node has received confirmation! Logic ensues.

Now, let’s test a parameter set command that would receive an error. Let’s try to set the iomode to 9:

~sendmsg;10;;~modparam;iomode;0;9

The response from the node is:

cmd:mp,param:iomode,ionum:0,iomode:9status:1,error:moderange
Sending message to replynode: 1

Our sending note receives the status of 1, with an error of moderange. Enough said!

Cleanup Notes

Important things we did to optimize memory and to clean things up:

  • Elimination of all String objects: All strings objects have been converted to character arrays (people often refer to these as strings). This has helped RAM tremendously, avoided segfault issues with overruns that result in restart of the micro.
  • Use of FLASH library to take strings out of RAM: RAM is a precious resource (we report it locally after each command), and overrun of available RAM is a really easy way to get very unpredictable behavior, such as variable gibberish and sporadic reboots. Use of the native FLASH library string write and copy functions saves immensely on the tiny bit (2k) of ram available on an ATMega328P
  • Reuse of code for local (serial) output and for Radio commands: All code is built around the same output for local and remote programming. This not only saves loads of code space, but also makes the output the same for both methods.

Reference

The above makes use of the open source libraries available on github:

Explanation and installation here: