Getting started with LoraWAN, TheThingsNetwork and PlatformIO

Welcome back to the IoT hacking playground at ThingForward! From our last four blog post you should be familiar with building, testing & flashing embedded projects using PlatformIO, and its powerful features like Unit Testing and Remote and Continuous Integration applications by using TravisCI. Todays topic is a little bit different from embedded testing and older blogposts, nevertheless it is a vital topic for integration of IoT technology to the embedded devices and applications: We're going to look at LoRa and connecting a sensor to The Things Network.

Internet of Things will be full of machine-to-machine connections between nodes, and cloud services via gateways. One of the most important motivations is decreasing the power consumption on gateways and nodes. We basically have WiFi everywhere,but with the cost of high power consumption. For battery-powered sensor nodes, WiFi is not a proper transmission. At this point, LPWAN (low power wide area networks) protocols play a bigger role. LoRa, Sigfox and 6LoWPAN are the major protocols for IoT narrow band communication protocols. Engineering is a trade-off, low power and long range means that you have to sacrifice from your payload size or suffer from duty cycle and package loss during transmission. Because of these reasons, there is not only one best solution, there are several good solutions for different projects.

Todays topic is LoRaWAN. We're going to show

  • how to set up a sensor node with LoRa hardware
  • how to set up a gateway, making the LoRa connection from the node
  • and forward it to TTN, The Things Network.

By doing so, our sensor data will be reachable from everywhere via TTNs APIs. Let’s start with simple definitions we will use!

Intro

  • LoRa: Long Range, low power, low bandwidth and secure wireless modulation for M2M communication from Semtech. This is a patented, proprietary technology. The range can be up to 15 kilometers in rural areas and 2 kilometers in urban areas. The LoRa devices must wait for downlink until uplink is succeeded. The most important thing is the band frequency that must be used according to your country. As we are in Europe, you will see 868 MHz frequency in our blogpost. Please check your countries LoRa radio frequencies before buying the hardware.
  • LoRaWAN stands for “Long Range Wide Area Network”, and is the MAC layer on top of LoRa, being the physical layer. Its specification describes communcation between nodes, gateways and application platforms.
  • The Things Network: TTN has a vision to bring connectivity of networks of the Internet to the IoT field. To do so, TTN creates data connectivity for LoRaWAN applications. By using TTN, new gateways can be setup for coverage in your area, new applications and devices can be registered and the communication can be traced. The data is kept in TTN and reachable via Internet.
  • Gateway: A gateway is a physical machine which forwards the LoRa packets from nodes to TTN Network via Internet. In this project, we're going to use a Raspberry Pi 3 as gateway.* TTN Gateway: Gateway is a radio coverage for LoRaWAN which you need to configure directly in your location in order to setup connectivity on TTN. By doing that, you can reach your node data from basically anywhere from TTN console.
  • Node: A node is a microcontroller that has some sensors, which achieves processing and transmitting of sensor data to the gateway. For this project, an Arduino will be used and a BME280 sensor is selected.

Part 1 - Hardware

Here's the hardware that we are going to use today’s blogpost.

  • Raspberry Pi 3 Model B v1.2
  • Dragino LoRa/GPS Hat for RPi v1.3 @868 MHz *(see the note)
  • RPi PowerPack V1.2 (optional, you do not have to buy this)
  • Arduino Uno R3
  • Dragino LoRa Shield for Arduino v1.4 @ 868 MHz
  • Seeed Studio Base Shield for Arduino Uno
  • Blueberry BME280 Temperature-Humidity-Pressure Sensor

For our trainings and workshops we rely on sensors and connectors from Grove, so here we're also using a Grove Base Shield for Arduino and the Grove version of BME280. You can of course connect any other sensors, and you don't necessarily need the Grove Kit.

Image 1: From left to right; Raspberry Pi, LoRa Hat for RPi and RPi Power Pack Image 1: From left to right; Raspberry Pi, LoRa Hat for RPi and RPi Power Pack

Important Note: The LoRa Hat supports both RaspberryPi 2 and 3, BUT a small modification is needed when using a Pi3. You have to solder or jumper the Pins 22 to 24.

Image 2: LoRa/GPS Hat soldered for RPi Gateway

Image 2: LoRa/GPS Hat soldered for RPi Gateway

Here are the node components:

Image 3: From left to right; Arduino Uno, LoRa Shield for Arduino, Base Shield and BME280) Image 3: From left to right; Arduino Uno, LoRa Shield for Arduino, Base Shield and BME280

We do not need any modifications for LoRa Shield for Arduino. Keep in mind that the BME280 sensor will be connected to the Arduino via I2C ports. In the end, the hardware is ready and the structure diagram follows:

Image 4: From left to right; Gateway and Node Image 4: From left to right; Gateway and Node

Overview

Part 2 - Gateway setup

RPI Gateway

RPi will be used as a gateway and our RPi has Raspbian Stretch Lite OS. Please find the link to download it under the references section at the end of this post. After the usual setup of a fresh RPi (languages, locales, WiFi etc..) we need to enable SPI, an important setting for the LoRa Hat to function properly:

$ sudo raspi-config

Navigate to Interface Options -> SPI -> enable it.

Next comes downloading the WiringPi library:

$ sudo apt-get install wiringpi

We need git, too:

$ sudo apt-get install git

On to the LoRa software. We're using the Single Channel Packet Forwarder, tftelkamp/single_chan_pkt_fwd. This is a small c/cpp program to listen for communicating with a Semtech SX1276 transceiver, listenng for LoRa packets and forwarding them. This is how to compile it:

$ git clone https://github.com/tftelkamp/single_chan_pkt_fwd.git
$ cd single_chan_pkt_fwd
$ make
$ ./single_chan_pkt_fwd

On RPi, you will see the following output. Please follow the orange arrow and NOTE this Gateway ID. (Leave RPi working):

Image 5: Gateway ID on RPi Image 5: Gateway ID on RPi

TTN Network - Adding the gateway

As a second step, let us keep going with TTN user registration under https://www.thethingsnetwork.org

After registration, you can login with your account. Click on the Console on top left. Then you will see the following page:

Image 6: TTN Console Image 6: TTN Console

First thing is creating a coverage for our gateway. Please navigate yourself to the Gateways. Then create a new gateway by clicking:

Image 7: Gateways Image 7: Gateways

Here are the fields and how you should fill them:

  • Gateway ID: From the Image 5, you noted your RPi Gateway ID. This is a MAC address and it needs to be typed in without the colons, so our "b8:27:eb..." becomes "B827EB.." and so on.
  • I'm using the legacy packet forwarder, so that should be ticked.
  • Description is a basic description for your gateway.
  • Frequency Plan should be selected according to your country and rules. Please check it on TTN Network Wiki page which can you find under references.
  • Router is automatically selected as ttn-router-eu for our case.
  • Please find your location on the map. If you already have a gateway in your area, then you can also use that. You don't have to create a new one. If there is no existing gateway in your location, don't hesitate to create one. After finding yourself on the map, click on the location and you will see a blue icon on the map and your latitude and longitude degrees.
  • Select your antenna placement as indoor or outdoor.

Save it. It looks like so:

Image 8: Gateway Registration Image 8: Gateway Registration

If your RPi is still on duty, you can see that your gateway's status as successfully connected: Image 9: Gateway Overview Image 9: Gateway Overview

TTN Network - Adding Application

The next step is creating an application on TTN Network. Please click on Console again and choose Applications. Add an application by clicking on:

Image 10: Applications Image 10: Applications

Here are the fields and how you should fill them, then click on add application:

  • Application ID: Make yourself an ID number to your application. We will use something like: 200920170001
  • Description is a basic description for your application.

That is how it should look like: Image 11: Applications Registration Image 11: Applications Registration

After adding the application;

Image 12: Applications Overview Image 12: Applications Overview

TTN Network - Registering a Device

From Image 12, you will see the Devices tab. Please click on register device on this tab.

Here are the fields and how you should fill them:

  • Application ID: Make yourself an ID number to your application. We will use something like: 200920170011
  • Device EUI: Let TTN to generate it randomly, by clicking on the 2nd icon.
  • App Key: Let TTN to generate it randomly, by clicking the icon.

Click on register to end this process. That is how it should look like:

Image 13: Device Registration Image 13: Device Registration

After adding the device;

Image 14: Device Overview Image 14: Device Overview

The last thing is to do for device setup, on Image 14, go to right top and click on Settings and find Activation Method. It is automatically selected as OTAA but we're choosing ABP (Activation by Personalization) and save it. It looks like this:

Image 15: Device Activation Method Image 15: Device Activation Method

In the end of all TTN registrations, please see the Device EUI, Application EUI and App Key variables on Image 14. These will be used in the following parts of blogpost.

The next step is configuring the RPi packet forwarder code. We already started the program to get the gateway id, but we need to stop it. Open main.cpp file:

$ cd single_chan_pkt_fwd
$ nano main.cpp

On the line 59, you will see the part CONFIGURE THESE VALUES.

  • The freq value is the frequency that you will going to use. It is 868.1 MHz in Europe.
  • The lat, long and alt values are the values that you found on the TTN map. You can write them here too if you want.
  • The platform, email and description variables are informative fields. You can customise them as you wish.
  • SERVER1 is your TTN server IP address. To find that, open a new command line window and type:

$ ping router.eu.thethings.network for Europe server address. $ ping router.us.thethings.network for USA server address. $ ping router.cn.thethings.network for China server address. $ ping router.au.thethings.network for Australia server address.

  • SERVER2 is your local IP address. You can find it by running ifconfig on RPi and looking for the address of the network interface (i.e. your WiFi)

Here are the modifications, valid for our EU setup:

/*******************************************************************************
* Configure these values!
*******************************************************************************/
// SX1272 - Raspberry connections
int ssPin = 6;
int dio0 = 7;
int RST = 0;
// Set spreading factor (SF7 - SF12)
sf_t sf = SF7;
// Set center frequency
uint32_t freq = 868100000; // in Mhz! (868.1)
// Set location
float lat=51.24;
float lon=6.8;
int alt=0;
/* Informal status fields */
static char platform[24] = "Single Channel Gateway"; 
static char email[40] = "your@email.com"; 
static char description[64] = "Test"; 
// define servers
// TODO: use host names and dns
#define SERVER1 "52.169.76.203" // 
//#define SERVER2 "192.168.1.10" // local
#define PORT 1700 // The port on which to send data

Save main.cpp and build the file again, run the packet forwarder:

$ cd single_chan_pkt_fwd
$ make
$ ./single_chan_pkt_fwd

The gateway is configured and already connected to TTN. We'll leave it running and now turn to customizing our node.

Part 3 - Node Setup

The node consists of an Arduino Uno, the Dargino LoRa shield, Grove base shield and BME280 sensor to obtain temperature, humidity and pressure values. The objective is simple: Sense the weather and send it to the gateway. We've create a sample PlatformIO project for Arduino, feel free to clone and extend it. Open main.cpp

$ git clone https://github.com/emirez/lorawan-arduino-bme.git 
$ cd lorawan-arduino-bme
$ nano src/main.cpp

Under the project directory, you will see different libraries and their files:

  • aes, hal and lmic libraries are mandatory libraries for LoRa communication.
  • Seeed_BME280 header and cpp files are the library files for BME280 sensor. The communication of sensor data will be established via I2C.

Let's modify and analyse some parts of main.cpp. Here is the part that you need to modify first. On TTN Console, navigate to: Applications -> Select Application -> Devices -> 1 Registered Device -> Click on Device and see the following page:

Image 17: Device Address, Network Session Key and App Session Keys Image 17: Device Address, Network Session Key and App Session Keys

We need Device Address. Network Session key and App Session Key. Click the <> icon to see values in hexadecimal format (which is what we need), and copy/paste them in the following part of the code:

// LoRaWAN NwkSKey, network session key
static const PROGMEM u1_t NWKSKEY[16] = { ...values copied from above ...  };

// LoRaWAN AppSKey, application session key
static const u1_t PROGMEM APPSKEY[16] = { ...values copied from above ... };

// LoRaWAN end-device address (DevAddr)
static const u4_t DEVADDR = 0x26011B20 ; // <-- Change this address for every node!

You can also modify the transmission interval in seconds there. Please remember the duty cycle of LoRa is only %1 in g1 band which corresponds to (868.0 – 868.6 MHz) frequency band.

const unsigned TX_INTERVAL = 60;

Under the loop function, mydata[6] array is filled with BME280 sensor data and sent to gateway.

void loop() {
(...)
    mydata[0] = (temperature >> 8) & 0xFF;
    mydata[1] = temperature & 0xFF;
    mydata[2] = (barometer >>8) & 0xFF;
    mydata[3] = barometer & 0xFF;
    mydata[4] = (humidity >> 8) & 0xFF;
    mydata[5] = humidity & 0xFF;

    // Start job
    do_send(&sendjob);
    os_runloop_once();
    delay(15000);
}

Now it is time to upload our code to the Arduino from your local computer. Please remember that the RPi is still waiting for receiving the packages:

$ cd lorawan-arduino-bme
$ pio run -t upload && pio device monitor

Here is the result on Arduino's serial monitor:

[Thu Sep 21 13:04:57 2017] Processing uno (platform: atmelavr; board: uno; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
Collected 27 compatible libraries
Looking for dependencies...
Library Dependency Graph
|-- <Wire> v1.0
|-- <SPI> v1.0
Looking for upload port...
Auto-detected: /dev/cu.usbmodem1411
Uploading .pioenvs/uno/firmware.hex
avrdude: AVR device initialized and ready to accept instructions
Reading | ################################################## | 100% 0.00s
avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: reading input file ".pioenvs/uno/firmware.hex"
avrdude: writing flash (25626 bytes):
Writing | ################################################## | 100% 4.11s
avrdude: 25626 bytes of flash written
avrdude: verifying flash memory against .pioenvs/uno/firmware.hex:
avrdude: load data flash data from input file .pioenvs/uno/firmware.hex:
avrdude: input file .pioenvs/uno/firmware.hex contains 25626 bytes
avrdude: reading on-chip flash data:
Reading | ################################################## | 100% 3.29s
avrdude: verifying ...
avrdude: 25626 bytes of flash verified
avrdude: safemode: Fuses OK (E:00, H:00, L:00)
avrdude done. Thank you.
============================================================================ [SUCCESS] Took 9.81 seconds ============================================================================
--- Miniterm on /dev/cu.usbmodem1411 9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Starting
temperature: 2332
barometer: 5700
humidity: 5970
Packet queued
15962977: EV_TXCOMPLETE (includes waiting for RX windows)

Let's look at the RPi terminal:

pi@tfpi:~/single_chan_pkt_fwd $ ./single_chan_pkt_fwd
SX1276 detected, starting.
Gateway ID: b8:27:eb:ff:ff:27:d2:28
Listening at SF7 on 868.100000 Mhz.
------------------
stat update: {"stat":{"time":"2017-09-21 10:22:30 GMT","lati":51.24738,"long":6.76750,"alti":0,"rxnb":0,"rxok":0,"rxfw":0,"ackr":0.0,"dwnb":0,"txnb":0,"pfrm":"Single Channel Gateway","mail":"your@email.com","desc":"Test"}}
Packet RSSI: -32, RSSI: -108, SNR: 9, Length: 19
rxpk update: {"rxpk":[{"tmst":2030217195,"chan":0,"rfch":0,"freq":868.100000,"stat":1,"modu":"LORA","datr":"SF7BW125","codr":"4/5","lsnr":9,"rssi":-32,"size":19,"data":"QEEeASaAAAAB0uyBX4CDCNnorQ=="}]}

We can see that we just received a payload! Please consider that this payload is encrypted.

Let's see our payload on TTN interface. If the TTN registrations and pairing successful, then we will be able to see all the uplinks/downlinks on TTN interface. Please go to Console -> Applications -> Devices and see the following page:

Image 18: Device Overview Image 18: Device Overview

Great, our device was online 4 seconds ago. That means the connection between our setup and TTN is successful. Let us examine the payload. Please click on Data tab on right top. Here is what we see:

Image 19: Application Data and Decrypted Payload Image 19: Application Data and Decrypted Payload

And there is our payload! Please see that it is already decoded. (Remember the encrypted payload on RPi gateway output) Still, hexadecimal content is hard to parse for humans. What can we do now is to reformat our decoded payload in order to see the temperature, humidity and pressure values. Please navigate to TTN Console -> Applications -> Our Application -> Payload Formats from:

Image 20: Payload Reformatting Image 20: Payload Reformatting

You can use this decoder code directly:

function Decoder (bytes) {
var temperature = (bytes[0] << 8) | bytes [1];
var pressure = (bytes[2] << 8) | bytes [3];
var humidity = (bytes[4] << 8) | bytes [5];

return {

celcius: temperature / 100.0,
pressure: pressure,
humidity: humidity / 100.0

};
}

After copy/paste, please save function:

Image 21: Decoder Function Image 21: Decoder Function

Using the decoder function we can see clear-text data within the TTN console. Now on to the latest part, where we'd like to have all these values on the command line of our notebook. TTN offers an MQTT service where we can connect to and listen for updates. We're going to do this with the mosquitto client tools. You can find binary downloads on their website, on OSX its best using homebrew, so:

$ brew install mosquitto

For accessing this payload from TTN, we will need the access key and application ID. In order to find them, go to TTN Console -> Application -> Our Application and take them:

Image 22: Application ID and Access Key for Monitoring Image 22: Application ID and Access Key for Monitoring

Ready to go!

$ mosquitto_sub -h <SERVER_ADDRESS> -t '+/devices/+/up' -u '<DEVICE_ID>' -P '<ACCESS_KEY>' -v
$ //Here is an example:
$ //mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u '200920170001' -P 'ttn-account-v2.<YOUR KEY GOES HERE>' -v

After running this command, now restart your Arduino to trigger a packet-send. Then you will see the output:

$ mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u '200920170001' -P 'ttn-account-v2.<YOUR KEY GOES HERE>' -v
200920170001/devices/200920170011/up
{"app_id":"200920170001","dev_id":"200920170011","hardware_serial":"00D792ADB385DEE4","port":1,"counter":0,"is_retry":true,"payload_raw":"CRwCVRZE","payload_fields":{"celcius":23.32,"humidity":57,"pressure":597},"metadata":{"time":"2017-09-21T11:39:55.689736616Z","frequency":868.1,"modulation":"LORA","data_rate":"SF7BW125","coding_rate":"4/5","gateways":[{"gtw_id":"eui-b827ebffff27d228","timestamp":4118745314,"time":"","channel":0,"rssi":-36,"snr":9,"rf_chain":0,"latitude":51.24738,"longitude":6.7675,"altitude":50}]}}

The payload is decoded according to our decoder function, reachable via MQTT protocol that can be started on command line. That means, you do not necessarily have to use the TTN web console. So what did we achieve? Low cost single channel LoRaWAN gateway and a node is implemented together in TTN Network. Thanks to TTN, data flows and activities can be reached on its online TTN console or basically from the command line by a simple command.

So much for today, this concludes the fifth part of our blogpost series. But we're still not finished with LPWANs, stay tuned for upcoming posts (on, maybe, SigFox? :-)

Eren

Andreas

References: