Extract the received value from downlink messages

Hello, I’m currently trying to send data to my device in order to work with it. I know there is a function in Serial.write that decode and transform the received payload to a value and is this one:

case EV_TXCOMPLETE:
      Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
      if (LMIC.dataLen) {
        // data received in rx slot after tx
        Serial.print(F("Data Received: "));
        Serial.write(LMIC.frame + LMIC.dataBeg, LMIC.dataLen);
        Serial.println();
      }

You can notice the function that transform the received message is this one: Serial.write(LMIC.frame + LMIC.dataBeg, LMIC.dataLen); This is useful if you want to show data in the terminal, but I need that data to do other things, There is a function that transform the message received in the device?

There isn’t any standard function to do that, no. But there are of course many capabilities in C++ and the Arduino libraries which would help.

Before you can figure out what would be useful, you have to be clear about what your downlink message is, and what it means. And remember you have not only the payload, but can also chose the custom port number to help represent the meaning.

2 Likes

Aside: if this gives you meaningful feedback in the serial monitor, then it seems you’re sending ASCII text in the downlink?

Thanks for the reply, what I’m doing is sending a number to my device in order to set a TX interval to send a job using the function os_SetTimedCallback.

I’m using node-red to inject the payload:

{
    "confirmed": true,
    "fPort": 10,
    "data": "NTAw"
}

As you can see, the format of the data is in base64, the value I’m sending is 500. In the debug console I can see the value, but I want to assign to a variable in order to set the TX_INTERVAL

That’s text. This decodes to the character 5, followed by the character 0, followed by another 0.

Maybe Change Timermillis() period on downlink for STM32L0 - #6 by arjanvanb can get you started, but: you should really encode the number in a binary way, and not send ASCII characters.

(And aside, given the data rather than payload_raw, and the lacking dev_id: are you using the deprecated TTN Node-RED library?)

Just send the number, it will only be re-encoded as Base64 again.

And then if you assign the result to an integer, you are ready to go with something like this:

int TxPeriod = ( LMIC.frame[LMIC.dataBeg] << 8 ) + LMIC.frame[LMIC.dataBeg+1];

3 Likes

Yes, even if I’m sending a number its processed as text. If I try to assign the Serial.write function to an integer it will return the number and the length and if I just print the value in the screen will result only in the number.

If I encode in binary will be something like this:

{
    "confirmed": true,
    "fPort": 10,
    "data": "01001110010101000100000101110111"
}

And the result is something ike this:

Received
�]4�]t�]5�]4�]4�M5�]u�]u24

@descartes I did that and I don’t get any package.

I don’t have any problem receiving the payload data, if I use the Serial.write(LMIC.frame + LMIC.dataBeg, LMIC.dataLen); it will print 500, now I want to store that 500 in a variable. I already tried to store the Serial.write in a int, but since 500 is treated as String, it will store the 500 and the length, resulting in a 5003. I tried to divide by 10 but for some reason the result is 5000 and can’t find a way to remove that extra digit.

That’s not encoding in binary, that’s a text string that only holds ones and zeroes, but it’s still very much text. Even 32 characters of text! (And next, there’s some more encoding happening that we don’t know of, as we don’t know what library or API you’re using.)

Maybe Working with Bytes is a good start.

I don’t know what library you’re using in Node-RED, but when using plain MQTT you’d need something like:

And bytes should then be two bytes for your number 500, maybe using writeUInt16BE:

// Allocate 2 bytes, to hold a single 16 bits unsigned integer; you
// may need more to send multiple values, or to use larger values.
const bytes = Buffer.allocUnsafe(2);
// 16 bits integer, 0..65,535
bytes.writeUInt16BE(500, 0);

Alternatively: start with scheduling downlinks using TTN Console. The number 500 would be the 2 bytes 0x01F4 when written in hexadecimal:

All of the above should decode fine using descartes’:

2 Likes

I understand, thats the case for payload_raw, what happens if I want to send an object?

{
  "port": 1,  
  "confirmed": false,   
  "payload_fields": {
    "txinterval":  500
  }
}

In the device code how can I manage it?

For the very simple case of sending just one value, encoding it as text isn’t terrible. Most of the packet’s actual air length will still be protocol overhead - there isn’t a big difference between 13 + 2 bytes and 13 + 3.

However, please don’t use confirmed downlink, instead just watch your node’s behavior and see if it has adopted the change by transmitting at a different interval.

This is when you really need to pack with care.

Likely you want to define a packed struct putting binary fields closely together without any padding in between. You’ll have to consider the precise way systems on each end represent data - eg, if they are big or little endian, and the bit width of data types. Something like the struct packing routines in python let you be explicit on the infrastructure side, and you can look up the encoding and width of Arduino data types on your particular platform.

Once you have the actual packed binary buffer, you’d base64 encode that and put it in your message.

Don’t forget that you can use much of the range of the “port” variable to identify the format of your message. Eg, you can decide that a message on port 1 is of type A, one one on port 2 is of type B, etc.

I very much disagree. Yes, you’re very right about the existing LoRaWAN overhead. But even if the payload length would be the same, for me that would still not be an excuse to consider sending text. Even more: it’s not making things any easier, neither when scheduling the downlink nor when handling it.

It’s even worse if people don’t understand when they’re sending text. (I know the latter does not apply to @cslorabox. :slight_smile:)

The example code or documentation from which you’ve taken this example (which is lacking dev_id), relies on defining an Encoder in the payload formats in TTN Console. That Encoder would then convert the message before it’s transmitted. And that conversion would yield the exact same bytes as mentioned before. So, if you would be sending the number 500 as a 16 bits integer, then the Encoder would again yield the bytes 0x01F4 (when shown in hexadecimal representation) or the bits 00000001 11110100 (when shown in binary representation).

In other words: with TTN, regardless whether you’re using payload_raw or payload_fields, there would be no difference in the LoRaWAN radio packet, nor in the handling in your device. I’d forget about such Encoder.

I understand why you are concerned about the “direction” implied with using a textual message - if it grows to be more than a few bytes, it soon becomes actually wasteful.

But “premature optimization” is a concern, too.

There’s little doubt how the bytes ‘5’ ‘0’ ‘0’ are to be interpreted, regardless of hardware platform, and it’s readily logged for human examination.

It’s no accident that textual formats like JSON are common in the Internet age. Granted, one must not send JSON or multi-field packets encoded as text over LoRaWAN!

But binary encoding rapidly introduces complication, too.

Is “500” 0x01 0xf4 or is it 0xf4 0x01?

I still don’t agree. :slight_smile:

To interpret that as such, one needs to know it’s text. For me, that’s just the same as knowing about any other encoding one used. It’s no accident that the in the Internet age many systems are using Content-Type headers. :wink:

There is also the issue of turning the string in to an actual integer that can be used in the device code base = more code = increasing potential bugs.

Which is sort of make work rather than premature optimisation.

I did it like this :

            if (LMIC.dataLen) {
              Serial.print(F("Received "));
              Serial.print(LMIC.dataLen);
              Serial.println(F(" bytes of payload"));      
      
              downlinkIntegerTotal=0;
              downlinkStringTotal="";

              for (int i = 0; i < LMIC.dataLen; i++) {
                downlinkInteger=LMIC.frame[LMIC.dataBeg + i];
                downlinkString=char(downlinkInteger);

                //build complete string from payload
                downlinkStringTotal = downlinkStringTotal + downlinkString;  

                //build total of integers from payload
                if(i==0){  // first byte does not have to be multiplied with a power of 16
                  downlinkIntegerTotal=downlinkIntegerTotal+downlinkInteger;
                }else{
                  downlinkIntegerTotal=downlinkIntegerTotal+downlinkInteger*pow(16,2*i); //the parts are grouped in 2, so use a power of 2 times i
                }
            
                Serial.print("ASCII value:");
                Serial.println(downlinkString); 
                Serial.print("integer:");
                Serial.println(downlinkInteger); 
                Serial.println("");
              }

              Serial.print("Total downlink string (if string in Base 64 format is sent in downlink) = ");
              Serial.println(downlinkStringTotal);
              Serial.print("Total downlink value (if integer in Base 64 format is sent in downlink)= ");
              Serial.println(downlinkIntegerTotal);
          }

There are parts of the world looking to implement long jail sentences for those that send text over LoRaWAN.

As you don’t show the definitions of downlinkString and downlinkStringTotal, I’d hazard a guess that they are defined as String. There are parts of the world looking to implement permanent exile to Elba for those that use String in Arduino code.

Where ?

Bolton, specifically Rumworth.