How can I correctly cast my value in byte before sending

End Devices (Nodes) LMIC
1

Hello,

I just heard that we can send struct with arduino-lmic library

I actually can prepapre my variable but I cn not have it in byte. I guess I have pain to understand how to have byte when my sensos return int-

First here is how I declare my struct for sending

typedef struct {
  byte value;
}Payload;

the arduino-lmic library has a function do_send(). I call my sensor functions and I get the sensor measure stored in int global variable variable. In fact, it’a a struct as well but with two elements and with active and inactive sensor.( Then I have to filter only the active sensor before sending)

struct mesure {
    uint16_t id_capteur;
    int valeur; // valeur mesurée (adapter le type)
    int var_size; // valeur mesurée (adapter le type)
    // autre chose de pertinent à mémoriser lors d'une mesure ?
  };
  
  const byte tailleHistorique = 11; // Ce chiffre doit correspondre aux nombres de capteurs. Voir capteur mesCapteur[] = {
  mesure mesMesures[nbMaxCapteurs][tailleHistorique];

The problem I have it’s how can I send a struct with byte as the doc say

To send data back and forth over The Things Network you’ll need to use bytes

and

Technically, you can send 51 bytes

Here is how I do.

First, I declare globaly where I only store active values and where the index is the ID of the sensor (See below)

typedef struct {
  byte value;
}Payload;

Then I in my do_send function

void do_send(osjob_t* j){
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
// Some exemple of function
get_battery()
/* 
result of get_battery is store in mesMesures[c_battery][capteur_battery_id].valeur
*/
get_baro();
/*
result of get_baro is store in
mesMesures[c_temperature][capteur_temperature_id].valeur
mesMesures[c_pression][capteur_presion_id].valeur
mesMesures[c_humidity][capteur_humidity_id].valeur
*/
get_luminosity();
/*
result of get_luminosity is store in
 mesMesures[c_luminosity][capteur_luminosity_id].valeur
*/

// Now I create a variable type Payload where I only store **the active value**
// Ican have up to 11active sensors and some of them have 3 digit: 33 Degre, 969hKa, 395.

Payload p[nbCapteurActif];

 for(int pp = 0; pp < nbMaxCapteurs; pp++)
        {
          if (mesCapteur[pp].actif)
          { 
            p[mesCapteur[pp].idDuCapteur].value =  mesMesures[pp][mesCapteur[pp].idDuCapteur].valeur;
          }
        }
// Seinding the value. I keep it comment for now   
//LMIC_setTxData2(1, p, sizeof(p), 0);
}

All work fine!! Excepted that values are not exact.

For exemple this :slight_smile:

Serial.println(F("Verify values to be sent"));
        
        for(int qq=0; qq<nbCapteurActif; qq++)
        {
          Serial.println(p[qq].value );
        }

print

27
201
53
20
0
43
0

but 201 should be 969
43 should be 429
And all toher should not be byte as 27 should be 32 37.

I still have pain to understand how I can format my data and make it ready to be sent in byte.

It will be really helpfully if you can help me to format Payload p according to
https://www.thethingsnetwork.org/docs/devices/bytes.html#how-to-send-big-numbers

How can I adapt and collect

 Payload p[nbCapteurActif]; 

     for(int pp = 0; pp < nbMaxCapteurs; pp++)
            {
              if (mesCapteur[pp].actif)
              { 
                p[mesCapteur[pp].idDuCapteur].value =  mesMesures[pp][mesCapteur[pp].idDuCapteur].valeur;
              }
            }
 // Sending the value. I keep it comment for now
//LMIC_setTxData2(1, p, sizeof(p), 0);

Keeping in mind that

mesCapteur[pp].idDuCapteur

is the ID of the sensor and will let me identify the sensor value in the ‘payload format’

I hope I cleary explain my isue and you can give me tips and hints

2

I think, first of my problem, as the sensor value are in int (two octet) it can not be store in byte (1 octet). It the reason why number over 255 aredestroy as the 969 into 201.

That correct?

In spite of the doc, I do not understand how I can adapt my struct ‘Payload p’ to make it ready for sending the sensors values in byte and not over 51byte.

Help will be appreciate :slight_smile:

3

True. Decimal 969 is hexadecimal 0x03C9, and when trying to put that into an 8 bits variable of type byte, uint8_t or int8_t, you’d only be storing 0xC9, which is decimal 201. You’ll need a 16 bits variable, like of type uint16_t or int16_t, to store two bytes.

Where did you hear that? You cannot really send structs.

What will happen is that each value in the struct is sent, without any meta information about that struct. So for 4 struct values, each with 3 values, you’ll just get a payload that holds a number of bytes that somehow you need to translate back into 12 values, and eventually separate those into 4 objects each holding 3 values at the JavaScript side in TTN Console (or your own application). A struct like:

struct mesure {
    uint16_t id_capteur;
    int valeur;
    int var_size;
};

…will yield 2 bytes (16 bits) for the first uint16_t value, another 2 or 4 bytes for the second int (depending on your Arduino version), and yet another 2 or 4 bytes for the third int. So, I don’t understand why your logging shows 7 bytes. That makes no sense at all.

On an Arduino don’t use int, but be specific about the number of bits you expect. Like with:

typedef struct {
  uint8_t a; // single byte, unsigned, 0..255
  int8_t b; // single byte, signed, -128..+127
  int16_t c; // two bytes, signed, -32,768..+32,767
} Measurement;

// Reserve space for at most 5 measurements
Measurement measurements[5];

…the following 3 measurements will yield an application payload of 01 02 0300 04 FB FAFF 07 F8 F7FF (when displayed as hexadecimal):

int count = 0;
measurements[count++] = {1, 2, 3};

// 8 bits decimal -5 = 0xFB, 16 bits decimal -6 = 0xFFFA
measurements[count++] = {4, -5, -6};

Measurement m;
m.a = 7;
// 8 bits decimal -8 = 0xF8
m.b = -8;
// 16 bits decimal -9 = 0xFFF7
m.c = -9;
measurements[count++] = m;

LMIC_setTxData2(1, (byte*)measurements, count * sizeof(Measurement), 0);

Note that multi-byte values are LSB (least significant byte first) in an Arduino, so your decoder needs to use LSB too. It also needs sign extension as JavaScript’s bitwise operators expect 32 bits. Something like:

function Decoder(bytes, port) {
  var measurements = [];
  while(bytes.length > 0) {
    var m = {};

    // Single byte, unsigned
    m.a = bytes[0];
    // Single byte, signed, with sign extension to get 
    // 32 bits for proper decoding of negative values:
    m.b = bytes[1]<<24>>24;
    // Two bytes, signed, LSB, with sign extension to get 
    // 32 bits for proper decoding of negative values:
    m.c = bytes[2] | bytes[3]<<24>>16;

    measurements.push(m);

    // Discard the 4 bytes we just handled above:
    bytes = bytes.slice(4);
  }

  return {
    measurements: measurements
  };
}

(If, like in your last example, each struct really only has a single value, then using structs is not making your node’s C++ code any more easy to read, and you’d better just use a single value. Not sure what you’re trying to do there, and too lazy to translate the French.)

4

I’m using this simple structure, maybe you can adapt it for your use case.

void get_config (uint8_t val) {
    ESP_LOGI(TAG, "Remote command: get configuration");
    int size = sizeof(configData_t);
    // declare send buffer (char byte array)
    unsigned char *sendData = new unsigned char[size];
    // copy current configuration (struct) to send buffer
    memcpy(sendData, &cfg, size);
    LMIC_setTxData2(RCMDPORT, sendData, size-1, 0); // send data unconfirmed on RCMD Port
    delete sendData; // free memory
    ESP_LOGI(TAG, "%d bytes queued in send queue", size-1);
};

This way you can send out structs of any type. My one looks like this:

// Struct holding devices's runtime configuration
typedef struct {
  uint8_t lorasf;                       // 7-12, lora spreadfactor
  uint8_t txpower;                      // 2-15, lora tx power
  uint8_t adrmode;                      // 0=disabled, 1=enabled
 ...
  char version[10];                    // Firmware version
  } configData_t;
5

However you need to keep in mind structs will use padding to align (some) data members on even boundaries for some processor architectures. So you will not be using all bytes in the sendData buffer (waste of airtime), you need to adjust the decoder accordingly.
Manually copying data to the output buffer (using shifts) is a much safer option.

3 Likes
6

It seems you’re subtracting 1 as the last value in your struct happens to be char version[10] and I assume you’re putting some null-terminated string into that? Indeed, when only sending text (hmmm, don’t unless really needed) then in LoRaWAN one might not need to send the terminator as LoRaWAN already includes another size indicator in its packet. So:

For future readers: when the data does not end with some text value, one should not subtract 1 from the size.

Also, why copy the value into a temporary buffer first? I’d say the following works too (where @kersing’s remarks still apply):

LMIC_setTxData2(..., (byte*)&cfg, ..., ...);
1 Like
7

Thank you for inspection of code.
Indeed the size-1 was a copy/paste error and is erraneous (and not harmful when only transmitting text, so was not recognized)

8

Hello!
Thank for your reply. Then I though using struct will make my code easier, but I will stop it now and use my previous code.

But I still has doubt about and if you can continue helping me about make a correct format, that would be great!

const uint8_t payloadSize = 51;
void do_send(osjob_t* j){
// gettinh here 8 to 11 value of sensors
/* 
     *  Prepare the dat into payload
    */
    uint8_t payload[payloadSize];
    uint8_t * ptr = payload;

    for(byte i = 0; i < nbMaxCapteurs; i++)
    {
      if (mesCapteur[i].actif)
      { 
        itoa(mesMesures[i][mesCapteur[i].idDuCapteur].valeur, (char *) ptr, 10); // We wrote to ptr the representation ASCII in base 10 the value of a sensor
        ptr += strlen((char *) ptr);
    
          *ptr = ',';             // add virgule
          ptr += sizeof(char);    // move right

      }
      else
     
          *ptr = ',';             // Add a last ,
          ptr += sizeof(char);
        
      }
    }
    *ptr = 0; // Close

}

and print what is going to be sent

/*
 * Print what is going to be sent
 */
        Si.sprint(F("Payload = {"),0);
        for (byte i = 0; i < payloadSize; i++) {
          //Serial.print("0x");
          if (payload[i] <= 0xF) Si.sprint(F("0"),0);
        
          Si.sprint(payload[i], HEX, 0);
          Si.sprint(F(" "),0);
        }
        Si.sprintln(F("}"),0);

The above code print me:

Payload = {32 37 2C 39 36 37 2C 35 34 2C 32 33 2C 30 2C 2C 2C 30 2C 2C 2C 34 33 30 2C 00 00 00 1F 00 00 00 E1 4B 01 00 14 08 00 20 01 00 00 00 E0 47 01 00 51 69 00 }

My payload format is

function Decoder(bytes, port) {
  var str=String.fromCharCode.apply(null,bytes);
  var astr = str.split(",");
  
  return{
    te:astr[0],
    pr:astr[1],
    hu:astr[2],
    lu:astr[3],
    w1:astr[4],
    w2:astr[5],
    w3:astr[6],
    da:astr[7],
    ts:astr[8],
    ga:astr[9],
    ba:astr[10],
  };
}

I am not fully happy with it, because it take bytes for unactive sensors
I need it because in my payload fromat the index 9 is ga, the index 2 hu.
If I unactive one or two sensors and I only send active sensor, the index will change depending if the number of active sensors, and no way to find the machting value in the payload format if the index 9 is not ga anymore, as the index change because we inactive some sensors.

Secondely, I do not like

2C 00 00 00 1F 00 00 00 E1 4B 01 00 14 08 00 20 01 00 00 00 E0 47 01 00 51 69 0

Juste after the last , (2C)

How would your recommand me to better work in order to send only

32 37 2C 39 36 37 2C 35 34 2C 32 33 2C 30 2C 2C 2C 30 2C 2C 2C 34 33 30 2C

Until the last 2C

and the best, how would recommand to send only active sensor values and to modify my payload format to be sure that 32 37 is temperature, 34 33 30 is battery (etc) whatever the index and the number of active sensor?

Do you understand my worries?

If I can not do than sensing active and inactive sensor is fine, but making sure there is no dusty after the last 2C

I hope I was clear :slight_smile:
Thank

9

So now you want to send a text value? (Like 27,967,54,23,0,,,0,,,430, in your example.) You really shouldn’t, that’s just a waste of air time.

10

To be sincerly with you, I am confuse now.
No, I want to send byte and I thought sending byte with my last code

That code prepare payload with byte, no??

const uint8_t payloadSize = 51;
void do_send(osjob_t* j){
// gettinh here 8 to 11 value of sensors
/* 
     *  Prepare the dat into payload
    */
    uint8_t payload[payloadSize];
    uint8_t * ptr = payload;

    for(byte i = 0; i < nbMaxCapteurs; i++)
    {
      if (mesCapteur[i].actif)
      { 
        itoa(mesMesures[i][mesCapteur[i].idDuCapteur].valeur, (char *) ptr, 10); // We wrote to ptr the representation ASCII in base 10 the value of a sensor
        ptr += strlen((char *) ptr);
    
          *ptr = ',';             // add virgule
          ptr += sizeof(char);    // move right

      }
      else
     
          *ptr = ',';             // Add a last ,
          ptr += sizeof(char);
        
      }
    }
    *ptr = 0; // Close

Where did you find this?

var str=String.fromCharCode.apply(null,bytes);

I am confuse

that cause to have string?

(char *)

Then how should I prepapre my payload variable to make sure, I send byte, then?

11

Are you kidding me? It’s in your very own post!

No. I’ve tried to explain to you September last year: What is the maximum number of characters I can send with LMiC? - #5 by pierrot10 Sorry, that’s all I’ll explain; signing off here.

1 Like
12

Oups, I am sorry. I was more concentrated on how to program my node/micro-controller to send value in bytes to avoid wasting of air time and first of all improve it.

You are right this

var str=String.fromCharCode.apply(null,bytes);

was in the payload formart (TTN console) and I have not wrote this. It’s an example I got. It’s the reason why I have not “match” my mind with this code. Sorry, no I am not kidding you :smile:

Any case, thank for your previous comment and sorry for the missunderstanding.