How to use "bitmap" in LoRa Serialization library?

Application Development
1

The LoRa Serialization library lets me encode for humidity, temperature, and others, but I’m having trouble with the bitmap.

So others see the full setup for the basics that work:
–>On the Payload Formats tab, encoder, use the following (modify accordingly):

function Encoder(json, port) {
  var bytes = encode([json.humidity, json.temperature], [humidity, temperature]);
  return bytes;
}

// https://github.com/thesolarnomad/lora-serialization/blob/master/src/encoder.js
// paste the entire contents of the encoder.js below

For the JSON fields:

{"humidity": 48.2, "temperature": 24.82 }

which gives

// encoded payload:
D4 12 09 B2

For bitmap (boolean?), I’m trying the following to be able to send on/off to various functions on the node. I believe I’m not formatting the bytes variable properly in the second line:

function Encoder(json, port) {
var bytes = encode([json.bitmap], [bitmap]);  //missing something ???
  return bytes;
}

with various attempts in the fields:

{"a": false}    //outputs 00
{"a": true}     //outputs 80
{"a": true, "b": true, "c": true, "d": true, "e": true, "f": false, "g": true, "h": true}   //outputs 00

Thank you again for the help–and your time.

2

Actually, this one is weird. (And I cannot reproduce that; looking at the code I expect, and get, 0x00 no matter what.)

Though indeed LoRa Serialization’s decoder returns properties named a, b, and so on, its encoder expects an array of booleans. Also, it needs to know in which JSON attribute that array is passed. So, specify an array with at most 8 boolean values:

{"myFlags": [true, true, true, true, true, false, true, true]}

…along with:

function Encoder(json, port) {
  return encode([json.myFlags], [bitmap]);
}

The above nicely yields 0xFB.

The encoder cannot just assume the flags are at the top level of the JSON object, as one could also specify multiple bitmap fields:

function Encoder(json, port) {
  return encode([json.myFlags, json.otherFlags], [bitmap, bitmap]);
}

…along with:

{
  "myFlags": [true, true, true, true, true, false, true, true],
  "otherFlags": [true, true]
}

This yields 0xFBC0.

But of course, the encoder could support input that matches the output that the decoder creates. The decoder for the above encoder reads:

function Decoder(bytes, port) {
  return decode(bytes, [bitmap, bitmap], ['myFlags', 'otherFlags']);
}

…and for 0xFBC0 would yield:

{
  "myFlags": {
    "a": true,
    "b": true,
    "c": true,
    "d": true,
    "e": true,
    "f": false,
    "g": true,
    "h": true
  },
  "otherFlags": {
    "a": true,
    "b": true,
    "c": false,
    "d": false,
    "e": false,
    "f": false,
    "g": false,
    "h": false
  }
}

So, indeed the encoder and decoder are not symmetric. Not too hard to change (or to also support the named attributes rather than an array), but I’m not sure if the author has a specific reason for this?

3

Giving it some more thought, it’s very unlikely that a Decoder (for uplinks) and Encoder (for downlinks) need to handle the same details. No symmetry as for the JSON needed/expected at all.

And maybe returning the named a, b, c and all is just a convenience, intended for further handling. Like:

function Decoder(bytes, port) {
  var decoded = decode(bytes,
    [humidity, temperature, bitmap],
    ['h', 't', 'flags']
  );

  // Process the bitmask results
  decoded.button1 = decoded.flags.a;
  decoded.button2 = decoded.flags.b;
  decoded.alarm = decoded.flags.c;

  // suppress the "flags" attribute in the result
  delete decoded.flags;

  return decoded;
}

For 0xD41209B280 this would yield (sorted for readability):

{
  "h": 48.2,
  "t": 24.82,
  "button1": true,
  "button2": false,
  "alarm": false
}

Likewise, the encoder could do some pre-processing:

function Encoder(json, port) {
  var flags = [json.enableThis, json.enableThat];
  return encode([flags, json.threshold], [bitmap, uint8]);
}

…along with:

{"enableThis": false, "enableThat": true, "threshold": 10}

Of course, the library could go to extremes to allow for specifying the intended names for each bitmask boolean too. But it’s easily done in the Payload Formats functions instead.

2 Likes
4

Thank you for all the information–it has been quite helpful, especially for a low-byte payload to turn on and off various items through different ports. I do have a question regarding the 0x prefix that should be coming through, although it might be academic in that it does not appear to affect my outcome on the node. Using your code, you get:

I am able to recreate, but only print in the serial monitor:

FB C0

Is that by design, or maybe I’m missing something?

I am decoding on the node using the general code from @GrumpyOldPizza void callback_onRecieve()

//from: https://github.com/GrumpyOldPizza/ArduinoCore-stm32l0/issues/67
void callback_onReceive() {
  onReceive = false;
  if (LoRaWAN.parsePacket()) {
    uint32_t size;
    uint8_t data[256];
    uint8_t portNum = LoRaWAN.remotePort();
    size = LoRaWAN.read(&data[0], sizeof(data));
// in this example, I am using port 4 to receive the data (see below)
    Serial.print("portNum = ");
    Serial.println(portNum);

    if (size) {
      Serial.print("data array[i], HEX: ");
      for (int i = 0; i < size; i++) {
        Serial.print(data[i], HEX);
        Serial.print(" ");
      }
      Serial.println();
    }

    //read and decode the bitmap from lora-serialization library
    if (portNum == 4) {
      Serial.println("Enter portNum == 4");
      if (size) {
        Serial.print("data array[i], HEX: ");
        for (int i = 0; i < size; i++) {
          Serial.print(data[i], HEX);
          Serial.print(" ");
        }
        Serial.println();
      }
    }

    if (size)
    {
      Serial.println(" ENTER if (size) ");
      Serial.println(LoRaWAN.read(&data[1], sizeof(data)));
      data[size] = '\0';
      Serial.print(", PORT: ");
      Serial.print(LoRaWAN.remotePort());
      Serial.print(", DATA: \"");
      Serial.print((const char*)&data[0]);
      Serial.println("\"");
      Serial.println();
      Serial.println(" LEAVE if (size) ");
    }
  }
}

The lack of 0x doesn’t seem to cause any issue, as the data reach the node in good order. The modifications work great, and using the bitmap will, hopefully, allow for triggering many different actions based on a single byte + overhead.

Thank you again!

5

0x is just notation, to make it clear that the value is in hexadecimal, removing ambiguity.

You can change
Serial.print("data array[i], HEX: ");
to
Serial.print("data array[i], HEX: 0x");
if you want to print 0x on serial.

1 Like
6

Note that you’re not really sending hexadecimal FB C0, but are sending binary data: the bits 11111011 11000000 which for a human can be shown as FB C0, or many other formats.

To see the bits, you could use Serial.print(data[i], BIN). Of course, that’s hardly useful for humans.

Also note that in computer code it is important to be explicit.

See also Working with bytes.

2 Likes
7

Ah! BIN is being received and printed to HEX. So does that explain when encoding with this json

{"delay": 241, "myFlags": [true, true, true, true, true, false, true, true]}

on TTN to downlink, I am receiving the encoded 3-byte payload “F1 00 FB” that prints the second byte “0” instead of “00”?

data array[i], HEX: F1 0 FB 
data array[i], BIN: 11110001 0 11111011

Using a delay greater than 255 …

{"delay": 600, "myFlags": [true, true, true, true, true, false, true, true]}

of course extends into the second byte:

data array[i], HEX: 58 2 FB 
data array[i], BIN: 1011000 10 11111011

Here are the TTN custom Payload Formats custom_decoder_2.txt (2.8 KB) and custom_encoder_2.txt (2.1 KB) .

On the node, this receive allowed for decoding when receiving downlink on port 2:

//from:  https://github.com/GrumpyOldPizza/ArduinoCore-stm32l0/issues/67
void callback_onReceive() {
  onReceive = false;
  if (LoRaWAN.parsePacket()) {
    uint32_t size;
    uint8_t data[256];
    uint8_t portNum = LoRaWAN.remotePort();
    size = LoRaWAN.read(&data[0], sizeof(data));
    Serial.println(" ");
    Serial.print("portNum = ");
    Serial.print(portNum);
    Serial.print("   size = ");
    Serial.print(size);
    Serial.println(" bytes");
    if (size) {
      Serial.print("   data array[i], HEX: ");
      for (int i = 0; i < size; i++) {
        Serial.print(data[i], HEX);
        Serial.print(" ");
      }
      Serial.println();
    }
    if (size) {
      Serial.print("   data array[i], BIN: ");
      for (int i = 0; i < size; i++) {
        Serial.print(data[i], BIN);
        Serial.print(" ");
      }
      Serial.println();
    }

    //read and decode the bitmap from lora-serialization library
    if (portNum == 2) {
      Serial.println("Port 2 downlink");
      if (size) {
        //see: https://www.thethingsnetwork.org/forum/t/decrypting-messages-for-dummies/4894/20
        int setDelay = data[1] << 8 | data[0];
        Serial.print("   Delay:  ");
        Serial.println(setDelay);
        uint8_t  bitVal = data[2];
        Serial.print("   Bit:  ");
        for (int i = 7; i > -1; i--) {
          Serial.print(bitRead(bitVal, i));
          Serial.print(", ");
        }
        Serial.println(" ");
      }
    }                    //end if portNum = 2
  }                    //end parsePacket = T
}                   //end callback on receive

The following links gave some assistance learning more about bytes and bits:

Thank you very much for your guidance @arjanvanb and your point @mfalkvidd!

2 Likes
8

I’m not sure if you’re still asking a question?

It’s just that the code does not print leading zeroes, which for humans often have no meaning. Each byte is always 8 bits, so you’re really sending 11110001 00000000 11111011. And hexadecimal F1 0 FB is the same as F1 00 FB, or f1 00 fb, or f100fb, and so on.

To print the leading zeroes, see other examples, like:

for (int i = 0; i < size; i++) {
  if (data[i] < 0x10) {
    Serial.print(F("0"));
  }
  Serial.print(data[i], HEX);
}

For binary, where the number of leading zeroes could be as many as 7, that’s a bit more difficult in Arduino. Many people on the Arduino forums use the bitRead like you used elsewhere in your code already, like, untested:

for (int i = 0; i < size; i++) {
  for (int bit = 7; bit >= 0; bit--) {
    Serial.print(bitRead(data[i], bit));
  }
  Serial.print(" ");
}

Fun fact: for computer code, a leading zero often denotes an octal number, just like a leading 0x denotes it’s a value in hexadecimal notation. Like in, e.g., C, C++, Java and JavaScript, 10 (decimal) and 010 (octal) are not the same. But in funny JavaScript, 80 and 080 are the same, while 70 and 070 are not… :thinking:

2 Likes
9

That answered the question–I wasn’t sure, but thought that the dropping of the leading 0 was acceptable.

I tested the code you posted for the bitRead() and leading 0’s worked:

{"delay": 62156, "myFlags": [false, false, false, true, true, true, false, false]}

properly output the following:

Delay:  62156
Bit:  11001100 11110010 00011100

Thank you for the leading 0 code and the associated links, too!

2 Likes