How to best write an application that contains many nodes with different measure data types

Application Development
1

Hi,

I had a very brief look at the API’s that are available and also currently try the data store integration.

How should I structure applications, devices and sensor data structures?

In my understanding, an application can have many devices and thus diffenent sensors and their data. But the data integration is at application level.

Is it therefore practical to directly integrate the application data into one data store (having all data as a binary blob per message)?

Or is there a step between (probably the formatting) and then go to different data stores such as if I compare them with different tables?

Or should I create an application per sensor data type (set of values per measure)?

I am a database guy and would like to bring the data in such a type of relational store.

Thanks,

Lothar

2

As long as the first element in your data told you what type of device (sensor) it had come from so you know how to treat it, then you could use a single application. For example in your garden you might have a soil moisture sensor, a pond water level sensor, and a weather station.
If your data message structure was something like sensor_type, value1, value2, value n then your MQTT data collector for your app could interpret 3 different messages as
“S,123” - Soil sensor, reading 123
“P,27.3” - Pond sensor, water level 27.3cm
“W,18.2,24,11” - Weather station, temperature 18.2, humidity 24%, wind 11km/h

This means you only need 1 ttn app, and 1 data collector feeding into 1 database for your Garden Mgt application.

1 Like
3

Hi, instead of using Mark’s suggestion, you can also use the port number in the message and use that to determinate which message type.
I.e. port 1 for Soil, port 2 for Pond, 3 for Weather, etc.

In the LMIC code (LMIC_setTxData2(1, dataTX, sizeof(dataTX), 0); ) port is the 1st parameter, in this case 1.
Port is also available in the payload functions and in the message.

1 Like
4

In the console you can use a switch statement for the different ports to do different stuff with the payload. Reduces the data sent over the air:

function Decoder(bytes, port) {
switch(port) {
    case 1:
        return {
          payload: String.fromCharCode.apply(null, bytes)
        };
    case 2:
        return {
          // Whatever function...
        };
    default:
        return bytes;
  }
}
1 Like
5

How and where do I specify the port number? In my LoPy code I simply send some data (plain text or anything).
I haven’t seen any port number configuration either in LoPy code - yet, or in TTN app configuration.

Any tips?

Where can I find more examples (ttn related, I think LoPy is an issue more likely for the pycom forum)?

7

Assuming you’re not really suggesting to send text (don’t) and use field separators: one could use Cayenne’s LPP Low Power Payload which also has a TTN Arduino library and an integration in TTN Console for quick prototyping. But I don’t like the additional data such solutions need, especially when it’s not prototyping any more. (A two bytes value needs four bytes in LPP. But of course every LoRaWAN message needs a 13 bytes header anyhow, so the increase might be relatively small when not sending many values.)

When administering many nodes, you’ll surely also have other details to keep track of. Like maybe:

  • The device hardware version, software version, installation date, battery type
  • For devices on a fixed location: coordinates, height (see also below)
  • For each sensor in a device: calibration details, measurement intervals to detect broken sensors, serial numbers for warranties

So, I feel it is easy to also administer each node’s sensor setup.

Next, you might be using data from a mix of devices for actual (future) applications, where the very same sensor data is applicable to different use cases, or might even be enriched with third-party data. And when replacing a device you get an unrelated new DevEUI for the very same location, for which you might want to merge the data. Or when moving a device the very same DevEUI might be used for a new unrelated location, in which case you might want to separate measurements.

So I would define a single application in TTN Console and simply have TTN forward the raw data to my own server (say: subscribe using MQTT). I would decode the raw payload in my own code, based on the device details that are stored in my own database, matched with information that is always known for each uplink:

  • Device EUI to know which device sent the data (hardware_serial in TTN MQTT data)
  • Port number if a single device can use multiple formats (like: only including battery level in every Nth message)

Finally, I would always keep the raw measurement data (or the decoded measurement data) so one can improve any conversion formulas at a later time, and recalculate derived values. (Say: when measuring particle matter, temperature and humidity, be sure to always store those actual measurements, rather than only storing some derived indication for air quality.)

All above also allows you to move (some) devices to another network in the future.


For LMiC, the port number is the first parameter in:

LMIC_setTxData2(1, buffer, sizeof(buffer), 0);

In the TTN Arduino library, it’s the last parameter in:

ttn.sendBytes(buffer, sizeof(buffer), 1);

I don’t know for the LoPy. The port number is always included in the data, like for MQTT.

MQTT example 
{
   "app_id":"arjanvanb-app-1",
   "dev_id":"arjanvanb-bb-1",
   "hardware_serial":"70B3D5B020035CD1",
   "port":2,
   "counter":7165,
   "payload_raw":"CWUMqw0LDNM=",
   "metadata":{
      "time":"2017-04-27T10:07:27.207187339Z",
      "frequency":868.1,
      "modulation":"LORA",
      "data_rate":"SF7BW125",
      "coding_rate":"4/5",
      "gateways":[
         {
            "gtw_id":"eui-b827ebffff5fe05c",
            "timestamp":532315708,
            "time":"",
            "channel":0,
            "rssi":-57,
            "snr":9,
            "latitude":52.37447,
            "longitude":4.63576
         },
         {
            "gtw_id":"eui-5ccf7ff42f1970ec",
            "timestamp":1284851361,
            "time":"2017-04-27T10:07:26.494026Z",
            "channel":0,
            "rssi":-49,
            "snr":9,
            "rf_chain":1,
            "latitude":52.374523,
            "longitude":4.635771,
            "altitude":4
         }
      ]
   }
}
1 Like
8

For LoPy the port is set using method socket.bind(port_number)

Example:
    ...
        # create a LoRa socket
        s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)

        # set the LoRaWAN data rate
        s.setsockopt(socket.SOL_LORA, socket.SO_DR, 5)

        # bind to port
        s.bind(3)
    ...
1 Like
9

Hello, I try to send a data(15 bytes) across two different ports with LMIC library in Arduino uno, I created two different byte arrays named payloadT[13] and payload [2] to send the data
LMIC_setTxData2(1, payloadT, sizeof(payloadT), 0);
LMIC_setTxData2(2, payload, sizeof(payload), 0);
The previous code doesn’t work, but if I load all data on only one payload, without this line LMIC_setTxData2(2, payload, sizeof(payload), 0); (payloadT[15] por example), I recieve all data that i need. On the first case, i receive the data only of port 1. The reason that I want to do a fragmentation on my data is to send the temperature data on port 1 the GPS data on port 2 …

A fragment of my decode function on chirpstack is:
tempObj.temperatura=Number((B).toFixed(2));
** tempObj.altitud=Number(©.toFixed(2));**
** tempObj.presion=Number((D).toFixed(2));**
** tempObj.latitud=Number((E).toFixed(6));**
** tempObj.longitud=Number((F).toFixed(6));**
** tempObj2.LedYellow=Number((G).toFixed(2));**
** tempObj2.LedGreen=Number((H).toFixed(2));**
** if(port == 1){**
** return tempObj;**
** }else{**
** return tempObj2;**
** }**

Thanks in advance

10

This is the TTN forum, may-be you want to move this question to the chirpstack forum?

2 Likes
11

you are right, I am doing both projects in parallel on the chirpstack server and the TTN server. My real problem does not lie with the servers but with the sending of the data through a specific port

12

Have you tried sending the messages some time apart? LoRaWAN is a low bandwidth protocol, sending multiple messages in a short time might not be possible.

Also, consider using a mechanism that allows sending all values in one packet. LoRaWAN has an overhead of at least 13 bytes. Splitting values in multiple packets is wasting airtime. (And on a shared medium wasting resources is frowned upon)

2 Likes
13

In addition, I’d suspect the LMIC stack is likely to reject the second request, if it’s internal OS has had time to catch up with the reality of the first request - so literally anything could be happening under the hood.

1 Like
14

Thanks for your reply, both of you are right with time delay works but is a wasting resources as you say.