LED Bar Graph with Arduino and shift registers

Intro

So, this article was supposed to be called ‘LED Bar Graph with MAX7221’. We’ve got a pile of these neat little 12-segment bar graphs over here that have red and green LEDs in each digit. As it turns out, however, we’ve got common anode, while the MAX7221/7219 only drives common cathode. This is a bummer, and probably why Adafruit discontinued the common anode and only carries the common cathode now. We’d just pick up some common cathode and be done with it, except, like I said, we’ve got a pile of them. And it seemed like a cool project to make it work with even simpler components than our MAX7221.

Multiplexing?

Connecting up a bunch of LEDs on a shift register really isn’t newsworthy, in fact we posted about it earlier. The key here is that, while we can control all LEDs on one anode individually (each anode of three controls four segments), we can’t turn one LED in one group and one in another group without turning a bunch of others on inadvertently. Looking at our datasheet, which is similar to this one, reveals that we have 14 pins total, three of which are duplicated. So we’ve got eight cathodes and three anodes.

Some basic math is illuminating: we have 24 individual LEDs, each with two states. That gives 2^24 possible state combinations. We only have 14 pins, 3 of which are duplicates, leaving 11. This is only 2^11 possible pin states. We obviously need some trickery. And that’s where LED multiplexing comes in. It works the same way as a row/column setup does: each LED is lit very quickly in succession. It tricks our eyes, and allows fewer pins for a fixed number of LEDs. Take a typical 7-segment plus dot digit, for example. If you have eight eight-segment digits (what you can put on a MAX7221), you have 8×8=64 segments. Individually wired, this would take 128 pins, or 64 with a common anode or cathode. But with only eight cathodes and eight anodes, one per digit and one per segment type for a total of 16, you can multiplex it and operate it at one fourth the pin count. You could do the same thing with two eight-bit shift registers, such as the 74HC595 we’re using here. In our case, we have eight total cathodes per group (four green, four red), and three anodes. We’ll use one register for the cathodes (as we did above), and use the other register for our three anodes. So on to it.

Shift Registers

Each shift register gives us eight bits (we’ll use 74HC595s), so we can get away with two of them and have five bits to spare. They’re about $0.10 apiece. A MAX7221 is something like a dollar, so it’s marginally cheaper.

Testing reveals that driving these guys from 5V seems to be just fine, so our wiring looks a bit like this:

Wiring schematic for two shift registers and a 12-segment bar display.

Notice that we’ve wired our output enable pin to ground, which will make it permanently active. This is fine, since we aren’t sharing our SPI lines with other devices.

We’ve hooked up our clock and data lines on outputs 8-10 on the ‘duino, and we borrowed and modified some code to allow us to set the color on each of the first four digits (eight LEDs):

 void setDisplay(int byte1, int byte2) {
  int numtoset1 = 255-byte1;
  int numtoset2 = byte2;
  digitalWrite(latchPin, LOW);
  
    // shift out the bits:
    shiftOut(dataPin, clockPin, MSBFIRST, numtoset2);
    shiftOut(dataPin, clockPin, MSBFIRST, numtoset1);  

    //take the latch pin high so the LEDs will light up:
    digitalWrite(latchPin, HIGH);
}

Note that we’ve added provision for two bytes, which we’ll need for our remaining digits (actually, for controlling the anodes, but we’ll get to that later). This means that our closest shift register will get the second byte we send, meaning for our one shift register testing, we’ll put our test value in byte1. To start, we have set up the first four bits for red, and the second four for green. Our cathodes are the outputs, so low turns them on, but we take the complement so we can use 1 for ‘on’. This yields the following values to turn on each LED:

segment color value
1 R 1
1 G 16
2 R 2
2 G 32
3 R 4
3 G 64
4 R 8
4 G 128

So to get yellow, we just apply both red and green. 255 is all on, 0 is all off. We don’t need to remember these, as we can just use the built-in bit() function, passing 0-3 for red and 4-7 for green. We wrap it up again in a function that allows us to set, using two arguments: digit and color. We’ll modify this later when we create control for all digits. Here is the wrapper function:

void setDigit(int digit, int colorint) {
  int byte2 = 0;
  int byte1 = 0;
  digit = digit %4;
  if (colorint > 0){
    if (colorint == 1){
      byte1 = bit(digit);
    }
    else if (colorint == 2){
      byte1 = bit(digit + 4);
    }
    else if (colorint == 3){
      byte1 = bit(digit) + bit(digit +4);
    }    
  }  
  setDisplay(byte1,byte2);
}

We set up a basic function to set the first digits:

void loop() {
  setDigit(0,0);
  setDigit(1,1);
  setDigit(2,2);
  setDigit(3,3);
}

This will set the digits one at a time. Persistence of vision dictates that we can’t tell the difference whether or not they are on simultaneously or in sequence (as long as the frequency is >25Hz (!)) … and we get what we expect:

Our segments, lit in series, appear to be lit simultaneously, the core principle of multiplexed LED illumination.
Our segments, lit in series, appear to be lit simultaneously, the core principle of multiplexed LED illumination.

Neither I nor the camera can tell that they aren’t all constantly on. Now, the astute reader should ask: “Why are you turning the digits on separately, instead of all together?” This is a great question, and really the point of this post. See Mutiplexing, above. The short answer is that we don’t have enough pins to set all states simultaneously and uniquely, so we have to shuffle through them.

Now on to the remaining digits. We’ll modify our setDigit() routine to tolerate digits greater than four, and set the proper anode. Note the use of the mod operator to bring all digit values into the range of 0-3 for our cathodes.

void setDigit(int digit, int colorint) {
  int byte2 = 0;
  int byte1 = 0;
  if (digit < 4){
    byte2 = 1;
  }
  else if (digit < 8){
    byte2 = 2;
  }
  else if (digit < 16) {
    byte2 = 4;
  }
  digit = digit %4;
  if (colorint > 0){
    if (colorint == 1){
      byte1 = bit(digit);
    }
    else if (colorint == 2){
      byte1 = bit(digit + 4);
    }
    else if (colorint == 3){
      byte1 = bit(digit) + bit(digit +4);
    }    
  }  
  setDisplay(byte1,byte2);
}

We’ll call it like so:

setDigit(0,0);
  setDigit(1,1);
  setDigit(2,2);
  setDigit(3,3);
  setDigit(4,0);
  setDigit(5,1);
  setDigit(6,2);
  setDigit(7,3);
  setDigit(8,0);
  setDigit(9,1);
  setDigit(10,2);
  setDigit(11,3);

This seems fine, but what we get are fluttering LEDs, as shown here:

Obviously, 8×8 matrices make eight calls work, so our timing is probably suspect, or the difference between eight and twelve calls breaks the camel’s back. We don’t feel like doing the math. We don’t need to troubleshoot that, however. Remember that we’re making eight calls for only three anodes, so we can further optimize. We can combine the twelve calls into three (there are only three anodes). We’ll just restructure our algorithm a bit.

First, we define a quick function to return a value from digit and color:

int getByteFromColorInt(int digit, int colorint){
  int returnvalue=0;
  if (colorint > 0){
    if (colorint == 1){
      returnvalue = bit(digit);
    }
    else if (colorint == 2){
      returnvalue = bit(digit + 4);
    }
    else if (colorint == 3){
      returnvalue = bit(digit) + bit(digit +4);
    }    
  }
  return returnvalue;
}

Then, we split up the calls into groups:

void setBarDigits(int barColors[]) {
  int i;
  int byte21 = 0;
  int byte22 = 0;
  int byte23 = 0;
  for (i=0;i<4;i++) {
    byte21 += getByteFromColorInt(i, barColors[i]);
  }
  Serial.print("byte21: ");
  Serial.println(byte21);
  setDisplay(byte21,1);
  for (i=4;i<8;i++){
    byte22 += getByteFromColorInt(i%4, barColors[i]);
  }
  Serial.print("byte22: ");
  Serial.println(byte22);
  setDisplay(byte22,2);
  for (i=8;i<12;i++){
    byte23 += getByteFromColorInt(i%4, barColors[i]);
  } 
  Serial.print("byte23: ");
  Serial.println(byte23);
  setDisplay(byte23,4);
}

And simply call it like so:

void loop() {
  int barColors[12]={0,1,0,1,0,1,0,1,0,1,0,1};
  setBarDigits(barColors);
}

And we have success:

 

Alternating LEDs show that our segment control of multiple anodes is workign as advertised.
Alternating LEDs show that our segment control of multiple anodes is working as advertised.

One thing to remember is that the setBarDigits() function is running three states in series, and ending on one. So if you throw in a delay after the call, or do something time-consuming between when it ends and when it’s called again, it’s going to remain in the last state. So doing this, for example:

 void loop() {
  int barColors[12]={0,1,0,1,0,1,0,1,0,1,0,1};
  setBarDigits(barColors);
  delay(1000);
}

Will give you this:

 

Which is probably not what you want. So you need to run it often. This is why separate chips dedicated to this like the MAX7219/7221 make sense: they do all of this elsewhere so you don’t have to worry about it. Anyway, so let’s have some fun with some animation. First, we modify our setBarDigits() function to accept a runtime argument. This way the function will run its loop to maintain an set of digits for the time we specify:

void setBarDigits(byte barColors[], int runtime) {
  int i;
  int byte21 = 0;
  int byte22 = 0;
  int byte23 = 0;
  long starttime = millis();
  boolean doRun = true;
  
  while (doRun){
    byte21 = 0;
    byte22 = 0;
    byte23 = 0;
    for (i=0;i<4;i++) {
      byte21 += getByteFromColorInt(i, barColors[i]);
    }
    Serial.print("byte21: ");
    Serial.println(byte21);
    setDisplay(byte21,1);
    for (i=4;i<8;i++){
      byte22 += getByteFromColorInt(i%4, barColors[i]);
    }
    Serial.print("byte22: ");
    Serial.println(byte22);
    setDisplay(byte22,2);
    for (i=8;i<12;i++){
      byte23 += getByteFromColorInt(i%4, barColors[i]);
    } 
    Serial.print("byte23: ");
    Serial.println(byte23);
    setDisplay(byte23,4);
    
    if (millis()-starttime < runtime) {
      doRun = true;
    }
    else {
      doRun = false;
    }
  }
}

Then we create an array of states we’d like to see and pass them one by one:

void loop() {
  byte barColors1[12]={1,0,0,0,1,0,0,0,1,0,0,0};
  byte barColors2[12]={0,1,0,0,0,1,0,0,0,1,0,0};
  byte barColors3[12]={0,0,1,0,0,0,1,0,0,0,1,0};
  byte barColors4[12]={0,0,0,1,0,0,0,1,0,0,0,1};
  byte barColors5[12]={2,0,0,0,2,0,0,0,2,0,0,0};
  byte barColors6[12]={0,2,0,0,0,2,0,0,0,2,0,0};
  byte barColors7[12]={0,0,2,0,0,0,2,0,0,0,2,0};
  byte barColors8[12]={0,0,0,2,0,0,0,2,0,0,0,2};
  byte barColors9[12]={3,0,0,0,3,0,0,0,3,0,0,0};
  byte barColors10[12]={0,3,0,0,0,3,0,0,0,3,0,0};
  byte barColors11[12]={0,0,3,0,0,0,3,0,0,0,3,0};
  byte barColors12[12]={0,0,0,3,0,0,0,3,0,0,0,3};
  
  byte* barColorArray[12];
  barColorArray[0] = barColors1;
  barColorArray[1] = barColors2;
  barColorArray[2] = barColors3;
  barColorArray[3] = barColors4;
  barColorArray[4] = barColors5;
  barColorArray[5] = barColors6;
  barColorArray[6] = barColors7;
  barColorArray[7] = barColors8;
  barColorArray[8] = barColors9;
  barColorArray[9] = barColors10;
  barColorArray[10] = barColors11;
  barColorArray[11] = barColors12;

  for (int i=0;i<12;i++){
    setBarDigits(barColorArray[i],200);
  }
}

And we get this charming display:

But we still need to verify that we can set digits independently. The above doesn’t really demonstrate that, as we could be tying all the anodes together. So let’s switch it up a bit:

Perfect!

Direct Raspberry Pi IO with Apache and WiringPi

So last time, we demonstrated how to control your Pi IO with an intermediate database, using either our CuPID Controls UI and interface or a barebones installation with IO setpoints. We like this arrangement, for many reasons, such as error-handling, safety interlocks, datalogging, race conditions, and many other bonus features. It’s what we use for all of our applications.

We have come to realize, however, that there are advantages and elegance to an even simpler approach. Specifically, if you want the fastest possible response and minimal infrastructure, you may want to manipulate the Raspberry Pi IO directly. You may not want to rely on an intermediate database and daemon script to enact your commands.  For each application there is a best (or better) approach.

For this reason, we decided to find an effective way to bypass the typical problem of manipulating IO from a web interface: scripts run by anybody other than root, www-data for example, cannot execute the typical API, RPi.GPIO, to enable or disable GPIO outputs. This is currently possible using specialty web services, such as WebIOPi, but we would like to do so on a standard web server, such as Apache. This offers the possibility of doing so over not just your local network, but from anywhere you wish, as it is over a universal web interface, using javascript and html.

WiringPi offers this possibility, using the IO access from /sys/class/GPIO, as documented here. We don’t do anything special here except set up a wsgi method to execute these commands and call it using some javascript/jQuery. These are detailed below.

 The Requirements

  • Apache with mod_wsgi enabled
  • WiringPi. Installation is covered here
  • wsgi script alias directed to your wsgi script, something like this in your site configuration file:
 WSGIScriptAlias /wsgigpioactions /usr/lib/iicontrollibs/wsgi/wsgigpioactions.wsgi
  • A wsgi script, at location above, outlined below
  • An html document sourcing jQuery, so we can use the ajax call

WiringPI

We won’t bore you with stuff better covered elsewhere, like all of the features of WiringPi, or how to install it. The main feature we exploit is listed here:

  • wiringPiSetupSys (void) ;

This initialises wiringPi but uses the /sys/class/gpio interface rather than accessing the hardware directly. This can be called as a non-root user provided the GPIO pins have been exported before-hand using the gpio program. Pin numbering in this mode is the native Broadcom GPIO numbers – the same as wiringPiSetupGpio() above, so be aware of the differences between Rev 1 and Rev 2 boards.

The key feature, in bold, is that we can manipulate GPIO without root access. This is key, as our scripts executed from a web page via wsgi will be run as www-data.

Another nice feature of the package is that it comes with a handy command-line utility, gpio, that allows us to do our manipulation quite easily. So the code below will set up GPIO 18 for non-root access, and turn it on and then off:

gpio export 18 out
gpio -g write 18 1
gpio -g write 18 0

It doesn’t really get much easier than that. The export command automatically uses BCM numbering, and the ‘-g’ will let our gpio write command know to also use this scheme.

Although there is a python wrapper for this utility, we call it directly from our python scripts with a couple simple functions. More on that later.

The WSGI script

The server-side script is the special sauce. It is what will take our web page command and read or write on GPIO using WiringPi with the basic commands listed above. The mechanics of the wsgi call are explained here and here, but the gist is that with some leading headers and a return of data as requested, we can pass data to a wsgi script in json format, execute as we see fit in python, and return the data in a dictionary to do as we see fit in our web page. Take a look here for the complete wsgi script. We pass an object to the script in json defining an action, and act on our action as we see fit. Here, for example, is our action on an output toggle:

if d['action'] == 'wptoggleGPIOvalue':
    try:
        BCMpin = int(d['BCMpin'])
    except KeyError:
        data['message'] = 'No pin sent with command'
    else:
        from subprocess import check_output, call

        output = int(check_output(['gpio','-g','read','18']))
        # call(['gpio','export',BCMpin,'output'])
        if output == 0:
            call(['gpio','-g','write',BCMpin,'1'])
        else:
            call(['gpio','-g','write',BCMpin,'0'])

This is really basic stuff. The command assumes the gpio has already been exported as writable by /sys/class/,  reads the state, and writes the inverse. Piece of cake. We’ll add in some error-handling and export py-fu, but these are the nuts and bolts.

Polling data from the web page using jQuery

Using some basic ajax, we can execute calls to our wsgi script to get our data. The basic function is below:

function wsgiwpgpioactions (commands,callback) {
    // Get the data
    commands=commands || {action:'testaction'}
    callback = callback || logdone;
    $.ajax({
        url: "/wsgigpioactions",
        type: "post",
        datatype:"json",
        data: commands,
        success: function(response){
            callback(response)
        }
    });
}

We just need to pass it an action to carry out and a callback to execute on the data. It will execute asynchronously, execute the command we have issued, and return whatever data we have requested. Linking this to buttons, we can set up the jQuery action on a keypress:

$("testbutton").click(function)(){
    var command = {action:'wptoggleGPIOvalue',GPIOid:18}
    wsgigpioactions(command)
}

So in the above function, if we create a button with id of ‘testbutton’ and click it, it will execute a toggle of GPIO 18 as listed in the wsgi script above. Easy enough. If we want to get a little more complicated with it, we can apply the same key behavior to multiple button classes with different gpio IDs. The function below uses a helper function to extract a GPIO id from the DOM based on where the click came from. This way we don’t have to write  a button behavior for each:

$('.canpress').click(function(){
        var clickid = this.id
        var result = getGPIOfromid(clickid)
        if (result.mode == 'mode'){
            var action = 'wptoggleGPIOmode'
        }
        else if (result.mode == 'value') {
            var action = 'wptoggleGPIOvalue'
        }
        var command = {action:action,GPIOid:result.GPIOid}

        if (runqueries) {
//            alert('running command:')
//            alert('action: ' + action + ' , GPIOnum: ' + result.GPIOnum)
            wsgiwpgpioactions(command,updatestatusdata)
        }
        else {
            alert('queries not activated for command:')
            alert('action: ' + action + ' , GPIOnum: ' + result.GPIOnum)
        }
    });

Our helper function:

function getGPIOfromid(clickid) {
    if (clickid.search('mode') > 0){
            var mode = 'mode';
            // Chop up string
            // starts with GPIO
            // double-digit number
            if (clickid.search('mode') == 6) {
                var GPIOnum = clickid.slice(4,6);
                var GPIOid = clickid.slice(0,6);
            }
            // single digit number
            else {
                var GPIOnum = clickid.slice(4,5);
                var GPIOid  =clickid.slice(0,5);
            }
        }
        else if (clickid.search('value') > 0) {
            var mode = 'value';
            // Chop up string
            // starts with GPIO
            // double-digit number
            if (clickid.search('value') == 6) {
                var GPIOnum = clickid.slice(4,6);
                var GPIOid = clickid.slice(0,6);
            }
            // single digit number
            else {
                var GPIOnum = clickid.slice(4,5);
                var GPIOid  =clickid.slice(0,5);
            }
        }
    return {GPIOnum:GPIOnum, mode:mode, GPIOid:GPIOid}
}

 Putting it all together

Now, as we did before, we’ll set up a physical GPIO interface page, so we can monitor status and punch some buttons. First of all, we’ll need to get our gpio status regularly to update our buttons. So we wrap a text parser around the gpio readall function to return a dictionary array of current gpio statuses:

def getgpiostatus():

    from subprocess import check_output

    gpiolist=[]
    alloutput = check_output(['gpio','readall'])
    lines = alloutput.split('\n')[3:18]
    for line in lines:
        BCM1 = line[4:6].strip()
        wpi1 = line[10:12].strip()
        name1 = line[15:22].strip()
        mode1 = line[25:30].strip()
        val1 = line[32:34].strip()
        phys1 = line[36:39].strip()

        phys2 = line[42:44].strip()
        val2 = line[46:48].strip()
        mode2 = line[50:55].strip()
        name2 = line[57:65].strip()
        wpi2 = line[68:70].strip()
        BCM2 = line[74:76].strip()

        if BCM1 and BCM1 != '--':
            gpiolist.append({'BCM': BCM1, 'wpi': wpi1, 'name': name1, 'mode': mode1, 'value': val1, 'phys': phys1})
        if BCM2 and BCM2 != '--':
            gpiolist.append({'BCM': BCM2, 'wpi': wpi2, 'name': name2, 'mode': mode2, 'value': val2, 'phys': phys2})

    return gpiolist

We put that into our general-purpose library, pilib, and set up our wsgi script to call it (you can easily grab the function from our git blob and put it into your wsgi script if you don’t want pilib)

if d['action'] == 'wpgetgpiostatus':
    from pilib import getgpiostatus
    data = getgpiostatus()

And define a function in our web page to call the wsgi and return the data to the web page:

//// Update all GPIO data
function updategpioData() { wsgiwpgpioactions({action:'wpgetgpiostatus'},updatestatusdata)
}

This function runs our our ajax function wsgiwpgpioactions, with a callback of updatestatusdata, which will run once the ajax function is done. This function  takes the data as returned and inserts it into the DOM using some jQuery;

function updatestatusdata(statusdata){
    var text = '';
    $.each(statusdata, function(index,gpiodict){

        var value = gpiodict.value;
        var mode = gpiodict.mode;

        var valueelements =  $('.' + 'GPIO' + statusdata[index].BCM + 'value');
        valueelements.removeClass('on')
        valueelements.removeClass('off')
        if (value == '1'){
            text = 'On';
            valueelements.addClass('on')
        }
        else if (value == '0'){
            text = 'Off';
            valueelements.addClass('off')
        }
        else {
            text = 'Err';
        }
        valueelements.html(text);
        var modeelements = $('.' + 'GPIO' + statusdata[index].BCM + 'mode');

        modeelements.removeClass('input')
        modeelements.removeClass('output')
        if (mode == 'IN'){
            text = 'In';
            modeelements.addClass('input')
        }
        else if (mode == 'OUT'){
            text = 'Out';
            modeelements.addClass('output')
        }
        else if (mode == 'ALT0'){
            text = 'I2C';
            modeelements.addClass('I2C')
        }
        else {
            text = 'Err';
        }
        modeelements.html(text)
    })
}

So all we have to do is label our html elements properly with classes, and our update function will populate them nicely. For example:

<tr>
   <td>11</td>
   <td align="center"><div class="GPIO17value lightindicator canpress" id="GPIO17value">X</div></td>
   <td>&nbsp;</td>
   <td align="center"><div class="GPIO18value lightindicator canpress" id="GPIO18value">X</div></td>
   <td>12</td>
</tr>

You’ll notice above we added and removed ‘on’ and ‘off’ classes, and a few others, so that the buttons and elements look proper when they change values:

.lightindicator.on, .lightindicator.input {
        background:-webkit-gradient( linear, left top, left bottom, color-stop(0.05, #77ff77), color-stop(1, #00cc00) );
        background:-moz-linear-gradient( center top, #77ff77 5%, #00cc00 100% );
        filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#77ff77', endColorstr='#00cc00');
        background-color:#ededed;
    }
    .lightindicator.off {
        background:-webkit-gradient( linear, left top, left bottom, color-stop(0.05, #dddddd), color-stop(1, #777777) );
        background:-moz-linear-gradient( center top, #dddddd 5%, #777777 100% );
        filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#dddddd', endColorstr='#777777');
        background-color:#ededed;
    }

Demo

So let’s do something a bit more interesting with the framework that demonstrates we can do some real-time operation. We stripped down our Pi GPIO page EVEN FURTHER to contain only four buttons. We define them as 18, 23, 24, 25, in this version. We create four super-big buttons on our page, like so:

Our basic four GPIO button screen. Press a button, turn on an output.
Our basic four GPIO button screen. Press a button, turn on an output.

Next, we hook up some china special laser diodes on a board with some mosfets. We use one of our power output boards because it nicely breaks out our GPIO and power on the CuPID COM1 or COM2 bus. This is really nothing fancier than directly connecting our GPIO to the transistor gates. We’re just using what we have available here … and using our COM power lets us put our lights wherever we want (or as far as we care to stretch ethernet cable. Anyhow, here are a few pictures:

Our test setup. A CuPID and a few laser diodes, hooked up to four GPIO.
Our test setup. A CuPID and a few laser diodes, hooked up to four GPIO.
A close-up of our output setup. We're using a CuPID output board as a breakout for the GPIO on ethernet and to grab 5V and ground.
A close-up of our output setup. We’re using a CuPID output board as a breakout for the GPIO on ethernet and to grab 5V and ground. We machined up a little holder for our laser diodes.
The backside of our output breadboard. Four NPN 2N7000 mosfets, four 47ohm current limiting resistors, and four 10k pull downs on our outputs.
The backside of our output breadboard. Four NPN 2N7000 mosfets, four 47ohm current limiting resistors, and four 10k pull downs on our outputs.

And finally, let’s show it in motion. Beware, poor quality video.

CuPID Remote : Read, Database, Logging and UI

The Project

Building on our previous post, where we laid out the fundamentals for our Mote communication in a temperature Mote, and built up a modular Mote sketch that allows us to reprogram parameters via serial and radio, this time we’ll build up the processing back end that will get our data into our CuPID gateway and our User Interface – a web browser.

So we’ll set out to do the following:

  • Process IO data on gateway end into remote table
  • Process IO data into control and display algorithms alongside local data in the gateway

The Basics

We talked briefly about getting data into our CuPID from our gateway Moteino in another post, but we’ll rehash the basics here just to cover the bases.

We’ve got our gateway Moteino talking to our remotes, and passing on the data over serial to our Pi/CuPID, which is listening on the serial port /dev/ttyAMA0. It parses out the data between message demarcations and ends up with a nice json-formatted message, which it drops into a dictionary. We end up with something like this:

{'iovalue': '0', 'RX_RSSI': '-67', 'iomode': '01', 'iopin' : '04', 'nodeid': '2'}

This message comes from node 2, which is telling us that pin 4 is in mode 01 (digital output), and currently has a value of 0 (off). Cool. Let’s get it into tables, where it will be useful!

Tabling the Data

We are going to have a table of values brought in from the remotes, aptly called ‘remotes’, so we’ll throw this in there. Before we do so, however, let’s think about the structure of the table. We want enough fields to be present so that we can quickly identify whether a record should be inserted or replaced. To do this, we’ll set up a few auxiliary variables:

  • msgtype – this is metadata that will tell our scripts how to process
  • keyvalue – a unique value for the data to be reported. will vary with msgtype
  • keyvaluename – an identifier for the keyvalue, for handy use

Based on the keys present in the data, we’ll classify the message, and do a smart replace in the remote table.

So, for example, for two common message types:

runquery = False
        nodeid = datadict['nodeid']
        querylist = []
        if 'iovalue' in datadict:
            # iovalue type message
            try:
                msgtype = 'iovalue'
                keyvalue = datadict['iopin']
                keyvaluename = 'iopin'
            except:
                print('oops')
            else:
                runquery = True

        elif 'owdev' in datadict:
            # 1Wire message
            try:
                msgtype = 'owdev'
                keyvalue = datadict['owrom'][2:]
                keyvaluename = 'owrom'
            except:
                print('oops')
            else:
                runquery = True
        if runquery:
            deletequery = pilib.makedeletesinglevaluequery('remotes',{'conditionnames':['nodeid', 'keyvalue','keyvaluename'],'conditionvalues':[nodeid,keyvalue,keyvaluename]})
            insertquery = pilib.makesqliteinsert('remotes',  [nodeid, msgtype, keyvaluename, keyvalue, stringmessage, pilib.gettimestring()], ['nodeid', 'msgtype', 'keyvaluename', 'keyvalue', 'data', 'time'])
            querylist.append(deletequery)
            querylist.append(insertquery)
            pilib.sqlitemultquery(pilib.controldatabase, querylist)

So here, for each message type, we create a key with a value and a name. For io values, this is ‘iopin’ and then the number of the pin. This uniquely identifies a table entry. If an entry exists with the same nodeid, key name, and key value, we will replace it. We do this by constructing a query with our auxiliary function, makedeletesinglevaluequery, which takes a list of conditions and values and constructs a query. We then create an insert query with our data, which consists of the above, a data field that contains the raw message, and a timestamp. We can do most of the above with foreign key constraints on the database, but we keep it in our script for now. Running our serial monitor with this processing sequence, we get what we expect in our remotes table:

Our basic motes table, viewed here in phpliteadmin
Our basic motes table, viewed here in phpliteadmin

We’ll need to add in a hygiene function to remove old entries, but this will work for now. We’re now ready to do some postprocessing on our table, but first let’s create a basic page to display our mote table. Some copy pasta of existing pages and a little jQuery later, and we have this:

Our basic Mote table.
Our basic Mote table.

Great! Now let’s get it into our IO tables, so we can view, log and process it just like everything else.

Integrate the IO

To get our remote data into our IO tables, we’ll add the mote as an interface into our interfaces table. This way, we can selectively enable and disable processing of individual motes, and we can add options for poll period, how stale data needs to be before we ignore it, and other good stuff.  For now, we create a basic interface entry:

Our interfaces table entry for our Mote will allow selectively setting options and enable/disable. It will be processed just like our other interfaces.
Our interfaces table entry for our Mote will allow selectively setting options and enable/disable. It will be processed just like our other interfaces.

Alright, so we just need to add a case for our updateio script to process the entries. This takes a little py-fu, but it’s not too bad. We’ll need to process each message type separately, and we cheat on the name of each input for the moment, as we don’t feel like setting up metadata entries in our ioinfo table just yet. We are pleasantly greeted:

Our mote entries, all wrapped up and inserted into our IO table. We're now ready to process them as if they were local inputs!
Our mote entries, all wrapped up and inserted into our IO table. We’re now ready to process them as if they were local inputs!

We can now use our remote inputs as if they were local! Let’s take a view in our dataviewer. We scroll down to find our io automatically logging (default of 100 points), and select the onewire sensor. We’re done!

Our imported data in our dataviewer. We're now transparently reading remote data as if it were local.
Our imported data in our dataviewer. We’re now transparently reading remote data as if it were local.

Icing

Let’s revisit the idea of metadata for these channels. We hardcoded colloquial names for our mote inputs based on their unique input IDs, but it would be ideal if we again treat them like our other IO and give them metadata entries. This will also allow us to add other options, such as scaling. So we add a text column to our ioinfo table, and create entries for our remote devices. We can do this simply by editing the text in the IO table and pressing “update”:

Updating metadata for our remote inputs.

Next, we’d like to add a scaling factor. Our analog input being reported here, iopin 20, is hooked up to a voltage divider that will monitor battery input. So the value, 512, is 513 / 1024 * Vcc, where Vcc is 3.3V. We’re reading about 1.65V. Our voltage divider is a 470k resistor and a 1M resistor, with the 470k on the ground side. In other words, if we are reading 1.65V, the voltage at the high side is 1.65V * (1000 + 470)/470 = 5.2V. This is about right, within error on the resistors. We probably shouldn’t be using 5V power for our 1Wire sensor which will put data at 5V where Vcc is 3.3V, but oh well. Seems ok. So this gives us a total scaling factor of (1470/470)/1024*3.3 =0.0101 . We should technically add 1 to the input value, as it’s zero-indexed, but we’ll ignore that for now. Later, we’ll parse formulas in python, but leave that for another day.

So back to how we’ll make this happen. We’ll put a json-encoded option of ‘scale’ into the options field of the ioinfo metadata table. Then, when we process our io, we’ll see if this field exists, and scale it as we process it into the io table. A little more py-fu and this happens. We make the entry manually using phpliteadmin, since we don’t have an edit page set up for this options field yet:

Adding our options field in manually in phpliteadmin. We add scale to our battery monitor channel to scale it properly.
Adding our options field in manually in phpliteadmin. We add scale to our battery monitor channel to scale it properly.

Now let’s flip back to our inputs page. Just like magic, we’re all scaled! Now we have an accurate battery monitor that will log!

Our Battery monitor is updated and scaled, as we specified in our metadata table.
Our Battery monitor is updated and scaled, as we specified in our metadata table.

 Next time:

  • Programming our Motes to send channel status data to our gateway
  • Setting up our gateway to reprogram our Motes when they come online, using a table of set values, and

Reference

The above makes use of the open source libraries available on github here:

Explanation and installation here:

 

Adventures in Moteino : Modular Communication for CuPID Remote (UniMote)

Introduction

So we got to the end of our last post, about getting a battery-powered Moteino into a tiny little waterproof box (see posts here and here), and we needed to do a demo. What this means for us is getting our data read from our remote into a CuPID Controller, where we can plot it and watch it from, you know, anywhere. We looked at our message formatting and were at that point where we could hack something together to make it work … or we could just start writing it the right way. Guess which we chose? We wanted the data coming through to seamlessly integrate into our IO structure. Asynchronously, we want our serial monitor running in the background, cataloguing remote messages as they come in from the RF unit. We then want our IO read and update scripts to come by, grab this data, and make it available as actionable IO.

More important, we want our Moteino programming to be modular. For most or all applications, we want the same code to run on the ATMega, so that we can just reconfigure IO, control units, and other functions by remove parameter changes — without reflashing the code if at all possible.

The Project

We set out with the following goals:

  • Create an ATMega sketch that allows read/write of all available Moteino IO in all allowable formats by change of variable values, i.e. no code change required
  • Allow OneWire read operations on all digital IO  (only for DS18B20s here)
  • Configure IO for read and report (broadcast) on remotely adjustable schedule
  • Configure channel control, with configurable setpoint and process values and positive and negative feedback
  • Set all key program parameters remotely (via radio) and locally (on serial)
  • Adjustable, metadata-containing, report format for IO, channels, and system parameters
  • Save system configuration to EEPROM and restore upon resume after loss of power

Phew. Is that all?

By the way, we’ll cover the CuPID Read, Database and Logging UI in another post. This thing just got too long.

Main()

So we’re going to do some reading, some writing, some RF sending/receiving. Let’s block this out to see what it looks like. We also want to determine how to handle variables. Some will be static, some will be stored in EEPROM, and we’ll need to be able to reprogram them all remotely. Again, we want to be able to only reflash a program via the bootloader if we are upgrading firmware, not just changing the read and/or control parameters.

 Program Structure
Initialize {
  if (INIT) {
    Initialize variables to init values
  }
  else {
    Read values from EEPROM
  }
  Initialize unstored values
}
Loop {
  Read values
  Report values

  Run Channels algorithms
  Report Channels

  wait for sleep command or timeout
  if (sleep) {
    power down for LOOPPERIOD
  }
  else {
    delay for LOOPPERIOD
  }
}

That should about do it. We’ll revisit this later.

Storing everything

The one caveat about user-programmable variables is that you need to find a way to retain them. Hardcoding control parameters and values is something that is part and parcel of most avr coding. If we set them in real-time, however, we have to be able to retrieve them as-modified.

Variables

We want to store the really important stuff in non-volatile memory, should we momentarily lose power. For control applications, the important stuff needs to stay put. For this reason, we’ll put in in EEPROM. We only have 512 bytes to work with, so we’ll be cheap with it. Many of the below values, because they are stored as bytes, will be cast into a format that makes most sense. For example, a setpoint value will have a built-in range limit of 0-255, and float setpoints will be cast as byte integers. Unsigned long values such as read frequencies, which are natively unsigned longs with a range of 0 to 4,294,967,295, are cast into seconds and capped with a maximum value of 255. This reduces the total range from 0 to 256,000 with increments of 1000. You get the picture. These values are defined further in the sections below.

Item bytes EEPROM location
NodeID 1 0
NetworkID 1 1
GatewayID 1 2
loopperiod 1 3
sleepmode 1 4
sleepdelay 1 5
serialrfecho 1 6
Encryption Key 16 10-25
iomode 13 30-42
ioenabled 13 43-55
ioreadfreq 13 56-68
ioreportenabled 13 69-81
chanenabled 8 82-89
chanmode 8 90-97
chanposfdbk 8 98-105
channegfdbk 8 106-113
chandeadband 8 114-121
chanpvindex 8 122-129
chansv 8 130-137

Other variables will be initialized as follows

  • ioreportenabled[13] — all set to values for ioreadenabled
  • ioreportfreq[13] — all set to 0, meaning report whenever read
  • chanstate[8] — initialized to zero (no action)
  • chanpv[8] — initialized to zero
  • ioreporttimer[13], ioreadtimer[13], chanreporttimer[8] — all zero
The Mechanics

Alright, so how do we actually do this? It’s pretty simple. With an include of EEPROM.h, we just drop in chunks of code like this:

For storage:

void storeparams() {
  EEPROM.write(0,NODEID);
  EEPROM.write(1,NETWORKID);
  EEPROM.write(2,GATEWAYID);
  
   // update object
//    radio.writeReg(REG_SYNCVALUE2, NETWORKID);
//    radio.setAddress(NODEID);
    
  // maximum initialized loop period is 256
  // in 10ms increments, this is 2.5s
  if (LOOPPERIOD/10 > 256){
    EEPROM.write(3,256);
  }
  else {
    EEPROM.write(3,LOOPPERIOD/10);
  }
  
  EEPROM.write(4,SLEEPMODE);
  EEPROM.write(5,SLEEPDELAY);
  EEPROM.write(6,serialrfecho);
  
  int i;
  for (i=0;i<16;i++) {
//      EEPROM.write(i+10,ENCRYPTKEY[i]);
  }
  byte mybyte;
  for (i=0;i<13;i++){    
    EEPROM.write(i+30,iomode[i]);
    EEPROM.write(i+43,ioenabled[i]);
    EEPROM.write(i+69,ioreportenabled[i]);
    if (ioreadfreq[i]/1000 > 256){
      EEPROM.write(i+56,256);
    }
    else {
      mybyte = ioreadfreq[i] / 1000;
      EEPROM.write(i+56,mybyte);
    }
  } // for num io
  
  // channels data
  for (i=0;i<8;i++) {
    EEPROM.write(i+82,chanenabled[i]);
    EEPROM.write(i+90,chanmode[i]);
    EEPROM.write(i+98,chanposfdbk[i]);
    EEPROM.write(i+106,channegfdbk[i]);
    EEPROM.write(i+114,chandeadband[i]);
    EEPROM.write(i+122,chanpvindex[i]);
    EEPROM.write(i+130,chansv[i]);
  } // for channels
}

Likewise for retrieval:

void getparams() {
    Serial.println(F("Getting parameters"));
    NODEID = EEPROM.read(0);
    NETWORKID = EEPROM.read(1);
    GATEWAYID = EEPROM.read(2);    
    
    // update object
    // We DON'T do this because it may not be instantiated yet.
    
//    Serial.println(REG_SYNCVALUE2);
//    radio.writeReg(REG_SYNCVALUE2, NETWORKID);
//    radio.setAddress(NODEID);
    
    LOOPPERIOD = EEPROM.read(3) * 10;
    SLEEPMODE = EEPROM.read(4);
    SLEEPDELAY = EEPROM.read(5);
    serialrfecho = EEPROM.read(6);
 
    // io
    int i;
    for (i=0;i<13;i++){
      iomode[i] = EEPROM.read(i+30);
      ioenabled[i] = EEPROM.read(i+43);
      ioreadfreq[i] = EEPROM.read(i+56)*1000;  
      ioreportenabled[i] = EEPROM.read(i+69);    
    } // for io 
    
    //channels
    for (i=0;i<8;i++) {
      chanenabled[i] = EEPROM.read(i+82);
      chanmode[i] = EEPROM.read(i+90);
      chanposfdbk[i] = EEPROM.read(i+98);
      channegfdbk[i] = EEPROM.read(i+106);
      chandeadband[i] = EEPROM.read(i+114);
      chanpvindex[i] = EEPROM.read(i+122);
      chansv[i] = EEPROM.read(i+130);
    } // for channels
}

Piece of cake!

Moteino Modular

Taking a quick look at the pinout on the R4, which we're using here, gives us the following list of available pins and functions:

Label Pin Name Function Available
0 PD0 D0 RX N
1 PD1 D1 TX N
2 PC0 D2 DI/DO N
3 PD3 D3 INT1 Y
4 PD4 D4 DI/DO Y
5 PD5 D5 DI/DO/PWM Y
6 PD6 D6 DI/DO/PWM Y
7 PD7 D7 DI/DO Y
8 PB0 D8 FLASH N
9 PB1 D9 LED N
10 PB2 D10 XFM N
11 PB3 D11 XFM N
12 PB4 D12 XFM N
13 PB5 D13 XFM N
A0 PC0 D14/A0 DI/DO/AIN Y
A1 PC1 D15/A1 DI/DO/AIN Y
A2 PC2 D16/A2 DI/DO/AIN Y
A3 PC3 D17/A3 DI/DO/AIN Y
A4 PC4 D18/A4 DI/DO/AIN/SDA Y
A5 PC5 D19/A5 DI/DO/AIN/SCL Y
A6 ADC6 A6 AIN Y
A7 ADC7 A7 AIN Y

We'll not implement I2C for now, and simply assume the following possible modes for pins above that are available, leaving us with 13 available IO:

UniMote Label Pin Name Function Available
0 3 PD3 D3 INT1 Y
1 4 PD4 D4 DI/DO Y
2 5 PD5 D5 DI/DO/PWM Y
3 6 PD6 D6 DI/DO/PWM Y
4 8 PD7 D7 DI/DO Y
5 A0 PC0 D14/A0 DI/DO/AIN Y
6 A1 PC1 D15/A1 DI/DO/AIN Y
7 A2 PC2 D16/A2 DI/DO/AIN Y
8 A3 PC3 D17/A3 DI/DO/AIN Y
9 A4 PC4 D18/A4 DI/DO/AIN/SDA Y
10 A5 PC5 D19/A5 DI/DO/AIN/SCL Y
11 A6 ADC6 A6 AIN Y
12 A7 ADC7 A7 AIN Y

That's pretty manageable. We've got 13 possible IO, with four possible modes. We can use the digital pins as 1Wire buses, so we'll call that a fifth mode, and define them:

Mode Function
0 Digital Input (DI)
1 Digital Output (DO)
2 Analog Input (AIN)
3 Pulse Width Modulation (PWM)
4 1Wire Master (OW)

Next for each IO item, we want to set a number of variables:

  • ioenabled : whether to process the pin
  • iomode : which of the above modes is set
  • ioreadfreq : how often to read the pin value into local memory
  • ioreportenabled : whether or not to report the pin value over RF
  • ioreportfreq : how often to broadcast the pin value stored in local memory over RF and serial. If this value is zero, the IO value will be reported each time it is read

We'll store the pin values as recorded into an array aptly named iovalue[13], which we'll write to when reading pins, and read from when writing pins (for digital outputs).

Globally, we'll have a number of variables that control the entire process structure, which we've noted above in the EEPROM storage:

  • GATEWAYID
  • NODEID
  • NETWORKID
  • ENCRYPTKEY[16]
  • SLEEPMODE
  • SLEEPDELAY
  • LOOPPERIOD

 Mote, let's talk

Alright, so we've really laid the foundation here, but to debug (and to, you know, send messages on Serial and RF), we'll need a message passing interface. Originally built into the gateway and mote sketches is a nice foundation for reading serial input and responding to single character commands, e.g. the character 'r' dumps all register values, 'd' flash values, etc. We decided to leave these commands in place, should anyone decide they need them. We structure our message as follows:

~command;arg1;arg2;arg3

It's pretty simple. A tilda (~) tells our Mote that we are issuing a command, and we follow the command character with a command and a series of optional arguments. The key chunk of parsing code is as follows:

void processcmdstring(String cmdstring){
  Serial.println(F("processing cmdstring"));
  Serial.println(cmdstring);
  
  int i;
  String str1="";
  String str2="";
  String str3="";
  String str4="";
  if (cmdstring[0] == '~'){
    Serial.println(F("Command character received"));
    int strord=1;
    for (i=1;i<cmdstring.length();i++){
      if (cmdstring[i] != ';' || strord == 4){
        if (cmdstring[i] != '\n' && cmdstring[i] != '\0' && cmdstring[i] != '\r'){
          if (strord == 1){
            str1+=cmdstring[i];
          }
          else if (strord == 2){
            str2+=cmdstring[i];
          }
          else if (strord == 3){
            str3+=cmdstring[i];
          }
          else if (strord == 4){
            str4+=cmdstring[i];
          }
          else {
            Serial.println(F("Error in argument string parse"));
          }
        }
      } // cmdstring is not ;
      else { // cmdstring is ;
        strord ++;
      }  //cmdstring is ;
    } // for each character  
     
    if (str1 == "listparams") {
      listparams(0,0);
    }
    else if (str1 == "rlistparams") {
      listparams(1,str2.toInt());
    }
    else if (str1 == "reset") {
      resetMote();
    }
    else if (str1 == "modparam") {
      if (str2 == "loopperiod") {
        long newvalue = str3.toInt()*1000;
        if (newvalue >= 0 && newvalue < 600000){
          Serial.print(F("Modifying loopperiod to "));
          Serial.println(newvalue);
          // deliver in seconds, translate to ms
          LOOPPERIOD = newvalue;
          storeparams();
        }
        else {
          Serial.println("Value out of range");
        }
      } // loopperiod
      if (str2 == "nodeid") {
        long newvalue = str3.toInt();
        if (newvalue >= 2 && newvalue < 256){
          Serial.print(F("Modifying nodeid to "));
          Serial.println(newvalue);
          // deliver in seconds, translate to ms
          NODEID = newvalue;
          radio.setAddress(NODEID);
          storeparams();
        }
        else {
          Serial.println("Value out of range");
        }
      } // nodeid
      if (str2 == "networkid") {
        long newvalue = str3.toInt();
        if (newvalue >= 1 && newvalue < 256){
          Serial.print(F("Modifying networkid to "));
          Serial.println(newvalue);
          // deliver in ~seconds, translate to ms
          NETWORKID = newvalue;
          radio.writeReg(REG_SYNCVALUE2, NETWORKID);
          storeparams();
        }
        else {
          Serial.println("Value out of range");
        }
      } // network
      if (str2 == "gatewayid") {
        long newvalue = str3.toInt();
        if (newvalue >= 1 && newvalue < 256){
          Serial.print(F("Modifying gatewayid to "));
          Serial.println(newvalue);
          // deliver in seconds, translate to ms
          GATEWAYID = newvalue;
          storeparams();
        }
        else {
          Serial.println("Value out of range");
        }
      } // network
      else if (str2 == "iomode") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying iomode: "));
          Serial.println(ionumber);
          int newvalue = str4.toInt();
          if (newvalue >= 0 && newvalue <5) {
            Serial.print(F("Changing iomode to: "));
            Serial.println(newvalue);
            iomode[ionumber]=newvalue;
            ioreadtimer[ionumber] = 9999999; // read now
            storeparams();
          }
          else {
            Serial.print(F("Value out of range: "));
            Serial.println(newvalue);
          }
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // iomode
      else if (str2 == "ioenabled") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying ioenabled: "));
          Serial.println(ionumber);
          int newvalue = str4.toInt();
          if (newvalue == 0) {
            Serial.println(F("Disabling io."));
            ioenabled[ionumber] = 0;
            ioreadtimer[ionumber] = 9999999; // read now
            storeparams();
          }
          else if (newvalue == 1) {
            Serial.println(F("Enabling io."));
            ioenabled[ionumber] = 1;
            ioreadtimer[ionumber] = 9999999; // read now
            storeparams();
          }
          else {
            Serial.print(F("Value out of range: "));
            Serial.println(newvalue);
          }
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // ioenabled
      else if (str2 == "ioreadfreq") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying io: "));
          Serial.println(ionumber);
          long newvalue = str4.toInt()*1000;
          if (newvalue >= 0 && newvalue <600000) {
            Serial.print(F("Changing io readfreq to"));
            Serial.println(newvalue);
            ioreadfreq[ionumber]=newvalue;
            storeparams();
          }
          Serial.print(F("Value out of range: "));
            Serial.println(newvalue);
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // ioreadfreq
      else if (str2 == "ioreportenabled") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying io: "));
          Serial.println(ionumber);
          int newvalue = str4.toInt();
          if (newvalue = 0) {
            Serial.print(F("Disabling ioreporting."));
            ioreportenabled[ionumber] = 0;
            storeparams();
          }
          else if (newvalue == 0) {
            Serial.print(F("Enabling ioreporting."));
            ioreportenabled[ionumber] = 1;
            storeparams();
          }
          else {
            Serial.println(F("Value out of range"));
          }
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // ioreport enabled
      else if (str2 == "ioreportfreq") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying io: "));
          Serial.println(ionumber);
          long newvalue = str4.toInt()*1000;
          if (newvalue >= 0 && newvalue <600000) {
            Serial.print(F("Changing io reportfreq to: "));
            Serial.println(newvalue);
            ioreportfreq[ionumber]=newvalue;
            storeparams();
          }
          else {
            Serial.println(F("Value out of range"));
          }
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // ioreportfreq
      else if (str2 == "iovalue") {
        int ionumber = str3.toInt();
        if (ionumber >=0 && ionumber <14) {
          Serial.print(F("Modifying iovalue: "));
          Serial.println(ionumber);
          
          int newvalue = str4.toInt();
          if (newvalue >= 0) {
            Serial.print(F("Changing io value to: "));
            Serial.println(newvalue);
            iovalue[ionumber]=newvalue;
            storeparams();
          }
          else {
            Serial.print(F("Value out of range: "));
            Serial.println(newvalue);
          }
        }
        else {
          Serial.println(F("IO number out of range"));
        }
      } // iovalues
      else if (str2 == "chansv") {
        int channumber = str3.toInt();
        if (channumber >=0 && channumber <=8) {
          Serial.print(F("Modifying channel: "));
          Serial.println(channumber);
          // need to allow for floats
          int newvalue = str4.toInt();      
          Serial.print(F("Changing sv to"));
          Serial.println(newvalue);
          chansv[channumber]=newvalue;
          storeparams();
        }
        else {
          Serial.println(F("chan number out of range"));
        }
      }// chansv
      else if (str2 == "chanenabled") {
        int channumber = str3.toInt();
        if (channumber >=0 && channumber <=8) {
          Serial.print(F("Modifying channel: "));
          Serial.println(channumber);
          int newvalue = str4.toInt();
          if (newvalue == 0) {
            Serial.print(F("Disabling channel."));
            chanenabled[channumber] = 0;
            storeparams();
          }
          else if (newvalue == 1) {
            Serial.print(F("Enabling channel."));
            chanenabled[channumber] = 1;
            storeparams();
          }
          else {
            Serial.println(F("Value out of range"));
          }
        }
        else {
          Serial.println(F("chan number out of range"));
        }
      } // chanenabled
    } // modparams
    else if (str1 =="sendmsg"){
      Serial.println(F("sending message: "));
      Serial.print(F("to dest id: "));
      Serial.println(str2.toInt());
      Serial.print(F("Reserved parameter is: "));
      Serial.println(str3);
      Serial.print(F("Message is: "));
      Serial.println(str4);
      Serial.print(F("Length: "));
      Serial.println(str4.length());
      str4.toCharArray(buff,str4.length()+1);
      radio.sendWithRetry(str2.toInt(), buff, str4.length()+1);
    }
    else if (str1 == "flashid"){
      Serial.println(F("Flash ID: "));
      for (i=0;i<8;i++){
        Serial.print(flash.UNIQUEID[i],HEX);
      }
    }
    else{
      Serial.println(F("unrecognized command"));
      Serial.println(str1);
      Serial.print(F("of length: "));
      Serial.println(str1.length());
      for (i=0;i<str1.length();i++){
        Serial.println(str1[i]);
      }
    }
  } // first character indicates command sequence 
  else { // first character is not command
    if (cmdstring[0] == 'r') //d=dump register values
      radio.readAllRegs();
      //if (input == 'E') //E=enable encryption
      //  radio.encrypt(KEY);
      //if (input == 'e') //e=disable encryption
      //  radio.encrypt(null);

    if (cmdstring[0] == 'd') //d=dump flash area
    {
      Serial.println("Flash content:");
      uint16_t counter = 0;

      Serial.print("0-256: ");
      while(counter<=256){
        Serial.print(flash.readByte(counter++), HEX);
        Serial.print('.');
      }
      while(flash.busy());
      Serial.println();
    }
    if (cmdstring[0] == 'e')
    {
      Serial.print(F("Erasing Flash chip ... "));
      flash.chipErase();
      while(flash.busy());
      Serial.println(F("DONE"));
    }
    if (cmdstring[0] == 'i')
    {
      Serial.print(F("DeviceID: "));
      word jedecid = flash.readDeviceId();
      Serial.println(jedecid, HEX);
    }
  }
}

Simple, unoptimized, and verbose, but functional. A nice feature of the subroutine is that we can run it on both serial arguments and also commands we receive on RF. So, for example, if we run the command ~listparams, we get the following:

~listparams
Command character received
NODEID:0,
GATEWAYID:0,
NETWORKID:0,
LOOPPERIOD:2000,
SLEEPMODE:0,
iomode:[0,0,0,0,0,3,0,3,0,0,0,0,0],
ioenabled:[0,0,0,0,0,0,0,0,0,0,0,0,0],
ioreadfreq:[10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000],
ioreportenabled:[0,0,0,0,0,0,0,0,0,0,0,0,0],
ioreportfreq:[0,0,0,0,0,0,0,0,0,0,0,0,0],
chanenabled:[0,0,0,0,0,0,0,0],
chanmode:[0,0,0,0,0,0,0,0],
chanposfdbk:[0,0,0,0,0,0,0,0],
channegfdbk:[-1,0,0,0,0,0,0,0],
chandeadband:[0,0,0,0,0,0,0,0],
chanpvindex:[5,0,0,0,0,0,0,0],
chansv:[15.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00]

That will work just fine. We'd get the same thing if we broadcast a command string over RF (we'll demo this later).
We've got a basic set of commands to manage internal parameters. We'll detail a few here below.

Command Set

~listparams
~rlistparams;destinationid

This command is as detailed above. It list all key mote parameters in a JSON-like format. ~rlistparams is the same, except it sends to a specified node id. Typically, this will be the gateway requesting station identification. We've not fleshed out all of the message formatting for this yet, as we'll need to break it up in chunks small enough to fit within the 61 character limit, but those are just details.

~modparam;paramname;index;value

This is the workhorse command. It will allow us to set all of our internal parameters, from Serial or radio. It's really as simple as it looks. For example, observe enabling and setting our input to digital output mode:

~modparam;ioenabled;1;1
Command character received
Modifying ioenabled: 1
Enabling io.
Time to check data for pin 4
Digital input configured for pin 4
SENDING
iopin:04,iomode:01,iovalue:0
SEND COMPLETE

We'll break out some examples later on on reading and processing IO.

~sendmsg;destinationid;reserved;message

This is a pretty basic send message command. We've reserved the third argument, both because we'll need it, and also for the following reason: because the final argument is not parsed by semicolon delimiter, you can send a full command as the final argument, as long as it fits in the 61-character message size. Here are a couple examples:

Simple message send, on the mote:

~sendmsg;1;0;hello there! 
Command character received
sending message: 
to dest id: 1
Reserved parameter is: 0
Message is: hello there! 
Length: 13

And on the gateway:

BEGIN RECEIVED
nodeid:4, hello there!,RX_RSSI:-28
END RECEIVED
 - ACK sent.

Note, importantly, that if we send data, our gateway will simply insert it into its parsed message, which it will eventually pass on to python, which will turn it into a dictionary. From the below, it's pretty obvious how our gateway will perform as an arbitrator: it takes the data, tacks on metadata, and passes it on.

Command on node:

~sendmsg;1;0;d1:val1,d2:val2

On the gateway:

BEGIN RECEIVED
nodeid:2, d1:val1,d2:val2,RX_RSSI:-28
END RECEIVED

Ok, now remember the part where we can send a command as the last part of a sendmsg command? Let's send a command by RF from the gateway to the node. Let's start with something simple. Let's tell it to list parameters on its serial output:

~sendmsg;2;0;~listparams

Sure enough, we see the list of output on the node:

BEGIN RECEIVED
nodeid:1, ~listparams ,RX_RSSI:-32
END RECEIVED
processing cmdstring
~listparams
Command character received
NODEID:2,
GATEWAYID:1,
NETWORKID:100,
LOOPPERIOD:1000,
SLEEPMODE:0,
...

Pretty simple. Now let's send it a message telling it to send us a message back!

On the gateway:

~sendmsg;2;0;~sendmsg;1;0;ping!
Command character received
sending message: 
to dest id: 2
Reserved parameter is: 0
Message is: ~sendmsg;1;0;ping!
Length: 18

On the mote:

BEGIN RECEIVED
nodeid:1, ~sendmsg;1;0;ping!,RX_RSSI:-32
END RECEIVED
processing cmdstring
~sendmsg;1;0;ping!
Command character received
sending message:
to dest id: 1
Reserved parameter is: 0
Message is: ping!
Length: 5

And again on the gateway:

BEGIN RECEIVED
nodeid:2, ping!
END RECEIVED

Pretty cool! If we didn't have that pesky 61 character message length limit, we could bounce messages back and forth indefinitely!

A Quick Example: The Power Supply Monitor

Just to demonstrate how easy this is to set up, we're going to add on a feature we've been meaning to put into our motes, and one that's been the topic of some discussion: how to monitor your power supply voltage. For our motes, this is especially important, as they're running off of battery power. For the example, we'll simulate the battery with a power supply at about 9V.  We went the KISS route and threw down 1M and 470k resistors attached to power and ground, respectively, and to each other, of course. We hooked up our divider to A6 on our Mote, and let's get to enabling read and report:

Change io to analog input:

~modparam;iomode;11;2
Command character received
Modifying iomode: 11
Changing iomode to: 2

Enable reporting:

~modparam;ioreportenabled;11;1
Command character received
Modifying io: 11
Enabling ioreporting.

And finally, enabling the IO:

~modparam;ioenabled;11;1
Command character received
Modifying ioenabled: 11
Enabling io.
Time to check data for pin 20, io number 11
Analog input configured for pin 20
Value: 950.00
SENDING TO 1
iopin:20,iomode:02,iovalue:0941
SEND COMPLETE

We've got about 9.3V on our voltage divider, and we're measuring about 2.9V, which upon our Vcc of ~3.3V into 1024 parts gives us about 900, so we're in the ballpark. Remember, this ADC reference is relative, and we've got no caps in place. All things considered, not too bad. Over on our gateway, we're reading loud and clear:

BEGIN RECEIVED
nodeid:2, iopin:20,iomode:02,iovalue:0949,RX_RSSI:-28
END RECEIVED

Zee Channels

Now, we've already built a pretty flexible IO node. But one of the beautiful elements of a super-reliable remote node is that you can program it to control something simply and without connectivity. If you lose power or have to do those operating system things like reboot or update the system, mission critical processes don't spill, light on fire, or ruin your beer.

For this purpose, we built in eight channels with simple on/off control. The variables used for these channels are below, and their EEPROM assignments were listed previously.

  • chanenabled : whether the channel is enabled
  • chanpvindex : which indexed io is the process value
  • chansv : what the current setpoint value is for the channel
  • chanpv : what the current process value is for the channel (this is redundant and included for error-correction)
  • chanposfdbk : the io used for positive feedback (-1 = none)
  • channegfdbk : the io used for negative feedback (-1 = none)
  • chanmode : reserved for future use for different algorithms
  • chandeadband : deadback for sv-pv before action is taken
  • chanstate : status of the channel's control, e.g. -1 for negative feedback, +1 for positive feedback

We think that, with the above, the algorithm is straightforward and not worth discussing here. Instead, we'll get straight into an example. We'll configure the following:

  • D14/A0, UniMote 5 - 1Wire DS18B20 temperature sensor
  • D4, UniMote 2 - Red LED, Positive Feedback
  • D5, UniMote 3 - Blue LED, Negative Feedback

This mimics  a pretty common control scenario: controlling temperature with positive and negative feedback mechanisms. Even more control systems are comprised of a single feedback. Hot water heaters, brew kettles, glycol fermenter jackets, boilers, and so many more.

So let's set up a channel. 1Wire master is mode 4:

~modparam;iomode;5;4
Command character received
Modifying iomode: 5
Changing iomode to: 4

Get it reading:

~modparam;ioenabled;5;1
Command character received
Modifying ioenabled: 5
Enabling io.
Time to check data for pin 14, io number 5
1Wire configured for pin 14
dsaddress:2811FDE203000092,
temperature:24.31

Set our channel inputs and outputs:

~modparam;chanpvindex;0;5
Command character received
Modifying channel: 0
Changing pvindex to 5
~modparam;chanposfdbk;0;1
Command character received
Modifying channel: 0
Changing posfdbk to 2

~modparam;channegfdbk;0;2
Command character received
Modifying channel: 0
Changing posfdbk to3

Set the outputs to the correct modes (this could easily be automated, but you should be smart about it anyway):

~modparam;iomode;2;1
Command character received
Modifying iomode: 2
Changing iomode to: 1

~modparam;iomode;3;1
Command character received
Modifying iomode: 3
Changing iomode to: 1

Finally, let's enable the channel, and then enable the outputs:

~modparam;chanenabled;0;1
 Command character received
 Modifying channel: 0
 Enabling channel.
 Channel 0 value: 
 24.19
 Setpoint value: 
 15.00
 Setting negative action
 Neg feedback: 2

Looks good. Enabling outputs:

~modparam;ioenabled;1;1
Command character received
Modifying ioenabled: 1
Enabling io.
~modparam;ioenabled;2;1 
Command character received
Modifying ioenabled: 2
Enabling io.

Neat. We've got negative feedback on, indicated by our red LED, trying to reach the setpoint value of 15.00, with a process value of 24.19. Let's get that up to something above its current value so we can watch the action:

~modparam;chansv;0;28
Command character received
Modifying channel: 0
Changing sv to 28

Now let's play with it to ensure it works. Sure enough, if we grab the temperature sensor until the temperature exceeds 28, the control switches to negative feedback, and our blue LED comes on. Take the heat away, and it returns to positive feedback with the red LED lit. Success!

 

Sleep, but listen first!

Alright, so we want to conserve power for our remote nodes. It's not as if the 'duino is power-hungry, but when you are running on battery, everything counts. Fortunately, there are many libraries and loads of examples on how to do this. The tricky part, however, has to do with our communications sequence. The key part is that all communications are initiated by the remote node. The reason for this is, in fact, that the motes will sleep for periods of time, pop up, report, and go back to sleep. If communication were one-way (only from mote to gateway), this is not an issue. If we want to reprogram our node's parameters, such as report frequency, whether channels are enabled, how long it sleeps for, etc., we'll need to account for that in the remote's 'wake sequence'. To accomplish this, we'll build in a short period after the node processes its IO and channels, but before it sleeps, during which the gateway can issue commands. Alternatively, if the gateway has no commands, it can just tell the node to go to sleep. Revisiting our controls structure from above:

loop {
  read IO
  read channels

  sleepdelaytimer = 0
  while (sleepdelay <= sleepdelaytimer) {
    listen to radio
    if (message received) {
      process message
      // reset wait time
      sleepdelaytimer = 0 
    }
  }
  sleep (LOOPPERIOD)
}

We'll add in a command called 'gosleep', which will max out the sleepdelaytimer and send our mote to sleep. Now, recognizing that this will slow down the loop, we will only use the SLEEPDELAY if SLEEPMODE is enabled. This means that if SLEEPMODE is not enabled, the LOOPPERIOD will determine how often IO and channels are processed. If SLEEPMODE is enabled, however, it means that the mote will wait for SLEEPDELAY to go to sleep, and then sleep for LOOPPERIOD. This means that the device IO poll, when in SLEEPMODE, will actually only take place every SLEEPDELAY + LOOPPERIOD. Just something to be aware of. The other option would be to subtract SLEEPDELAY from LOOPPERIOD and only sleep for that long, but we prefer for code simplicity and semantics to leave it as is. We just have to remember this fact and that when SLEEPMODE is enabled LOOPPERIOD can be interpreted as the sleep period.

Here are the vital bits of the code:

// If we're in SLEEPMODE, we wait for a set period to receive packets on serial and radio
   
int millistart;
SLEEPDELAYTIMER = 0;
if (SLEEPMODE) {
  millistart = millis();
  Serial.println(F("Entering receive sequence"));
}
// failsafe. must listen.
if (SLEEPDELAY < 1000){
  SLEEPDELAY = 1000;
}
  
while (SLEEPDELAYTIMER < SLEEPDELAY){
    
  // SERIAL RECEIVE AND PROCESSING
  // Check for any received packets

  << Serial processing code >>

  // RADIO RECEIVE AND PROCESSING
  // Check for any received packets

  << Radio processing code >>

  if (SLEEPMODE) {
    if (SLEEPDELAYTIMER < 65535){
      // if timer is at 65535, means we are intentionally exiting
      SLEEPDELAYTIMER = millis() - millistart;
    }
  }
  else { // exit loop after one iteration if not in sleep mode
    SLEEPDELAYTIMER = 65535;
  } 
} // SLEEPDELAY while
if (SLEEPMODE){
 Serial.println(F("Exiting receive sequence"));
} 

 Sleep - the mechanics

As we mentioned previously, there are a number of libraries that will help put our Mote to sleep. We like the LowPower library, a favorite of the Moteino community. Also built into the RFM69 library is a simple radio.sleep() function that will put our RF to rest. A typical call to the LowPower powerDown function is shown below:

LowPower.powerDown(SLEEP_8S, ADC_OFF, BOD_OFF);

The LowPower library has a number of fixed length sleep intervals, ranging from 15ms to 8S, in multiple increments. We can choose one and loop as many times as we like. We put in a simple algorithm that tracks how much longer is remaining to sleep, grabs the next time increment down in size, and sleeps for that long. This technique will optimize our up vs. downtime at a fixed sleep period:

// Do our sleep or delay
if (SLEEPMODE) {
  Serial.println(F("Going to sleep for "));
  Serial.println(LOOPPERIOD);
  
  period_t sleepperiod;
  unsigned int sleepremaining = LOOPPERIOD;
  
  while (sleepremaining > 0) {
    if (sleepremaining > 8000) {
      // Use 8s interval
      sleepperiod = SLEEP_8S;
      sleepremaining -= 8000;
    }
    else if (sleepremaining >= 4000) {
      // Use 4s interval
      sleepremaining-=4000;
      sleepperiod = SLEEP_4S;
    }
    else if (sleepremaining >= 2000) {
      // Use 2s interval
      sleepremaining-=2000;
      sleepperiod = SLEEP_2S;
    }
    else if (sleepremaining >= 1000) {
      // Use 1s interval
     sleepremaining-=1000;
      sleepperiod = SLEEP_1S;
    }
    else if (sleepremaining >= 500) {
      // Use 500ms interval
      sleepremaining-=500;
      sleepperiod = SLEEP_500MS;
    }
    else if (sleepremaining >= 250) {
      // Use 250ms interval
     sleepremaining-=250;
      sleepperiod = SLEEP_250MS;
    }
    else if (sleepremaining >= 120) {
      // Use 120ms interval
      sleepremaining-=120;
      sleepperiod = SLEEP_120MS;
    }
    else if (sleepremaining >= 60) {
      // Use 60ms interval
      sleepremaining-=60;
      sleepperiod = SLEEP_60MS;
    }
    else if (sleepremaining >= 30) {
      // Use 30ms interval
      sleepremaining-=30;
      sleepperiod = SLEEP_30MS;
    }
    else {
      // Use 15ms interval
      sleepperiod = SLEEP_15Ms;
      sleepremaining = 0;
    }
    Serial.flush();
    radio.sleep();
    LowPower.powerDown(sleepperiod, ADC_OFF, BOD_OFF);
    Serial.print(F("Sleep remaining: "));
    Serial.println(sleepremaining);
  }
  // Sleeptime eludes millis()
  looptime += LOOPPERIOD;
} // end if

The important bit is at the end here:

    Serial.flush();
    radio.sleep();

    LowPower.powerDown(sleepperiod, ADC_OFF, BOD_OFF); 
    // Sleeptime eludes millis()
    looptime += LOOPPERIOD;

The serial flush function fixes some serial garble issues with power down (see discussion here). Also note the last bit where we add in the LOOPPERIOD. As it turns out, millis() stops when we power down the Mote. As a result, we need to manually add the time in to the looptime. We use this in the IO and channels processing, so we want to keep (relatively) accurate time. After these adds, we get the following for a sleep time of 3000:

Going to sleep for 
3000
Sleep remaining: 1000
Sleep remaining: 0

From our logic above, our delay is set to 3000, which will select a delay of 2000ms, then 1000ms after that is complete. Perfect.

Alright, we still have a ton of code cleanup to do, but the bones are here. Next, we'll get all this data into our gateway controller and work on some visualization and presentation. Eventually, we'll set up some metadata tables so we can control the Mote parameters from our UI!

Reference

The above makes use of the open source libraries available on github here. Arduino sketches are in iicontrollibs/mote

Explanation and installation here: