Panther Logger 5: Sending Data with LoRa

The LoRa modem on the Panther Logger is the RAK Wireless 11720 (datasheet is here). It is located on the upper right side of the board between the headers for the cellular modem. Sending data with a LoRa modem requires that a gateway is nearby, much like WiFi requires a router nearby. However, unlike WiFi, LoRa can send data much farther, several miles (LoRa = long range radio communication) and with lower power usage, while WiFi is limited to several hundred feet and will use more power.  The tradeoff is that not as much data can be sent with LoRa and it must be sent in smaller packets. Considering that many monitoring applications only need to send a couple to a couple dozen data points every sampling interval, LoRa can then be a perfect solution.

Data is sent by the gateway to a network that routes it to a server of your choice. In this tutorial we are going to use The Things Network (TTN) to route the data to one of the IoT databases we setup for sending data with the WiFi modem (Ubidots). 

Requirements for LoRa in this tutorial:

1. LoRa antenna attached to the LoRa modem. Do not power the LoRa modem without an antenna attached as this could damage the modem. We provide a small flexible LoRa antenna glued to the modem antenna ipex connector. This is for two reasons. First, the reason stated above, do not power the modem without an antenna. Second, these connectors (ipex4) can become disconnected easily. Third, frequent connecting and disconnecting of the antenna on these small ipex4 connectors can damage it. They are popular with manufacturers like RAK Wireless because they do not take up a lot of space. We suggest to keep the current antenna sold with the board in place until it is absolutely necessary to change it to a larger antenna and then carefully replace it with an ipex4-to-SMA adapter cable, to which is then attached to an antenna with SMA connection. You could also attach an SMA-to-N-type adapter to an antenna with an N-type connector. See our antenna recommendations page here.
2. A LoRaWAN gateway You can check the the Things Network coverage map here for available gateways near your location. Better yet, you can help grow the network and purchase your own gateway and set it up on The Things Network. The folks at The Things Network have released a very affordable indoor home gateway (such as here).  Instructions are very easy to follow for setting it up on the Things Stack (see here).

Go to The Things Network (TTN) here, and click on the sign up button in the upper right. Then click on Join The Things Network for a free account (community edition). Follow instructions to setup a free account. After logging into your account go to the console (click on user name, then on console). Click on applications and then create an application (see video to the right). Click on register end device, then enter device specifics manually.

Give your device an ID, all lower case

Give your device a name and description

For the frequency plan select FSB2, 902 - 928 MHz United States

Choose specification 1.03

Enter the APPEUI and then the DEVEUI and APPKEY from the serial window

Click register end device and now you are finished. 

We sent data above to TTN, but if we want to collect the data in a database and display it in a dashboard online then we will need to put it somewhere. Below, we will setup a payload decoder so that we can send decimal data to a database . Go to your TTN account and then to Applications, then End Devices and click on your device listed by EUI number (see video to the right). Then click on the payload formatters tab. We are going to setup the uplink decoder. Choose custom javascript formatter in the drop down box. Copy/paste the following in the formatter code box:

function Decoder(payload, port) {

    if(port === 1) {

        return [

            {

                "temperature": ((payload[0] & 0x80 ? 0xFFFF<<16 : 0) | payload[0]<<8 | payload[1])+(((payload[2] & 0x80 ? 0xFFFF<<16 : 0) | payload[2]<<8 | payload[3]))/100

            }

    ];

    }

}

This is a decoder function to decode a hexadecimal string of numbers that represent sensor float decimal data that can be positive or negative. It expects 8 hexadecimal digits like our temperature string that was sent above, 00 17 00 38. See an example payload for a weather station here and example function to decode that payload here. That example provides a variety of use cases that should work for most environmental monitoring data. 

On the TTN javascript payloader page you can test the decoder on the right by entering a hexadecimel string. The formatter is expecting 8 hex digits so enter  the data string we will send in hexadecimal format representing temperature data, 00170038  in the byte payload box and click test decoder. You should see the following in the decoded payload test window.

{

  "0": {

    "temperature": 23.56

  }

}

The javascript code converts the hexadecimal payload to decimal. 

Click on save changes to save the payload decoder.

Go back to Arduino and assuming the terminal program is still loaded on the Panther Logger then in the serial window run the series of AT commands above ending with the AT+SEND command again. 

Now go back to TTN and look at the live data for your application for this device. You should see another message with the payload sent. Click on it and a window opens on the right with information about the message in JSON format. Scroll down and you will see a similar JSON result as we saw in the test decoder window. It will look like the following:

"uplink_message": {

      "session_key_id": "AYyJJIeA3C2yQqrw3qEZqQ==",

      "f_port": 1,

      "f_cnt": 1,

      "frm_payload": "ABcAOA==",

      "decoded_payload": {

        "0": {

          "temperature": 23.56

        }

      },

Notice that the temperature data is now decoded and formatted how we would expect, as decimal format. It is also in the format expected by Ubidots. Note that other databases might be expecting a slightly different format so the payload decoder might need to be modified. There is a lot of other information in this message including antenna signal strength ("rssi") and signal to noise ratio ("snr") as well as the timestamp and geolocation of the gateway that sent the data. This entire message will now be sent by TTN to Ubidots with a webhook.

We can now send sensor data over LoRa with our Panther Logger through a gateway on TTN. However, we have only done it manually with the terminal sketch. Let's automate this to happen on a set time schedule. It is important to keep in mind here that if you have a free account with TTN that you are bound by the fair use policy, which stipulates that you can have:

• An average of 30 seconds uplink time on air, per 24 hours, per device.
• At most 10 downlink messages per 24 hours, including the ACKs for confirmed uplinks.

In other words... "A good goal is to keep the application payload under 12 bytes, and the interval between messages at least several minutes." This is explained here.

We will automate a script to send mock temperature and battery data to Ubidots over LoRa about every 5 minutes.

Go to our Github page for the LoRa_Send_Data sketch here. Just like code for sending data via cellular, this sketch has multiple tabs. Copy and paste each file into a new sketch with multiple tabs as was explained here for cellular sketches and save with a name of your choice. 

This sketch expects that the DS18B20 temperature sensor is attached and that you know its address. See previous tutorial on how to attach this sensor to the Panther Logger board and on reading these temperature sensors by their address in our tutorial on these sensors. The Send_Data_Ubidots_LoRa sketch will read these temperature sensors and battery voltage, convert the data to hexadecimal values and create the payload, then send the payload with the LoRa modem every five minutes.

Since this sketch will send more than one data variable we need to edit the payload formatter at TTN. Go to the formatter and replace it with the below code (it is also in our Github repository here).

function Decoder(payload, port) {
if(port === 1) {
return [
           {
"BATV": payload[0]+(payload[1]/100),
"Temp0": ((payload[2] & 0x80 ? 0xFFFF<<16 : 2) | payload[2]<<8 | payload[3])+(((payload[4] & 0x80 ? 0xFFFF<<16 : 0) | payload[4]<<8 | payload[5]))/100,
"Temp1": ((payload[6] & 0x80 ? 0xFFFF<<16 : 0) | payload[6]<<8 | payload[7])+(((payload[8] & 0x80 ? 0xFFFF<<16 : 0) | payload[8]<<8 | payload[9]))/100,
"Temp2": ((payload[10] & 0x80 ? 0xFFFF<<16 : 0) | payload[10]<<8 | payload[11])+(((payload[12] & 0x80 ? 0xFFFF<<16 : 0) | payload[12]<<8 | payload[13]))/100
           }
       ];
   }
}

This just takes the existing formatter and replicates the line of code to decode temperature data two more times and adds one more line first to decode battery voltage.  

Save the changes to the formatter and go back to the Arduino sketch.

To get started, find the code below at the beginning and replace the X's with your device's DEVEUI, APPEUI, and APPKEY:

char DEVEUI[] = "AT+DEVEUI=XXXXXXXXXXXXXX";
char APPEUI[] = "AT+APPEUI=XXXXXXXXXXXXXX";
char APPKEY[] = "AT+APPKEY=XXXXXXXXXXXXXXXXXXXXXXX";

Go to the code listed below and replace the address with your DS18B20 temperature sensor address repeated three times (or you can wire three different ones together!). 

byte Address[NSENSORS][8] = {
{0x28, 0xD7, 0xA8, 0xE2, 0x08, 0x00, 0x00, 0x39},
{0x28, 0xD7, 0xA8, 0xE2, 0x08, 0x00, 0x00, 0x39},
{0x28, 0xD7, 0xA8, 0xE2, 0x08, 0x00, 0x00, 0x39}
};

Note that you can change the interval between data transmissions, but we do not suggest a shorter interval than 5 minutes as it could violate The Things Network fair use policy.

Upload the sketch to the Panther Logger board and watch for new uplink data messages at TTN and new data in your device at Ubidots. 

There is a function, hexConvert1, in this sketch on the "Payload" tab to convert a floating point decimal number to hexadecimal where we expect no more than three numbers before the decimal and want two after the decimal. This also handles negative data. So temperature data would be a good example. 

char HexOutThreeNeg[10];
char* hexConvert1(float var){
char HexVar[10];
char FloatChar[10];
dtostrf(var,3,2,FloatChar);  
float VarConvert = atof(FloatChar);
int16_t Var1 = VarConvert;
int16_t Var2 = (VarConvert - Var1)*100;

sprintf(HexVar, "%04hX%04hX",
             Var1,
             Var2
);
strcpy(HexOutThreeNeg,HexVar);  
return(HexOutThreeNeg);
}

Decimal data needs to be converted to hexadecimal. To do this, we first need to make sure we know how many numbers to expect before and after the decimal. We can set this a priori with the dtostrf() function. This function will convert a floating point value to a character string and in the process allow us to set the the number of digits. 

dtostrf(var,3,2,FloatChar);  

var = the float value we want to convert

3 = number of digits before decimal

2 = number of digits to limit after decimal

FloatChar = Where to put the result, created with char FloatChar[10];

The result is a string representation of the float decimal with a set number of digits before and after the decimal.

Then we can convert back to a float with the atof function and continue with the conversion.

float VarConvert = atof(FloatChar);

We now have a float number called VarConvert with a defined number of digits before and after the decimal. Now we need to separate the numbers before the decimal from those after the decimal. We can get just the numbers before the decimal by converting to an integer:

int16_t Var1 = VarConvert; 

So if VarConvert was 23.56 then Var1 would now be 23.

Well if we subtract Var1 from VarConvert we get 23.56 - 23 = 0.56. And if we multiply by 100 we get 56, which we can store as an integer. We do this with the following:

int16_t Var2 = (VarConvert - Var1)*100; 

Now we have the digits before and after decimal separated. We can now convert to hexadecimal and a character string simultaneously using the sprintf function. 

sprintf(HexVar, "%04hX%04hX",
             Var1,
             Var2
);

The %04hX tells the sprintf function we want to output hex (this is the X) and use four digits. We use four because two will be used to represent a decimal number up to 255 (good enough for ambient temperature data) and another two to represent the sign as a negative or positive number.  We copy the result to a global character string that can be used outside of this function.

There are other functions to convert data of other types to hexadecimal such as data that is expected to have four digits before the decimal, but not negative data. 

The sketch uses these functions to convert sensor data to hexadecimal and then uses the AT commands we showed above to send the data to TTN. 

This sketch could be improved prior to deployment to do things like use millis() for more exact timing, possibly use sleep functions, include a software watchdog and other functions to ensure the program continues to send data during long deployments. 

<---LAST TUTORIAL