You are not required to do any decoding in the backend. You can get the ‘original’ data and decode it in your own code written in a programming language of your own choice. Decoding in the back end is just one of the options available to you.
Took me some fiddling, but it seems like I figured it out. I converted my float value (that had 2 digits before the decimal, and 2 after it) into a 16bit integer value, by multiplying it by 100.
After that, I converted that value into 2 bytes.
The part of my arduino sketch to do that looks like this:
static float temp;
si7021.getTemperature(temp);
si7021.triggerMeasurement();
int16_t int_temp = (int16_t)(temp * 100);
byte data[2];
data[0] = int_temp >> 8;
data[1] = int_temp & 0xFF;
myLora.txBytes(data, sizeof(data));
In my payload function, I than had to the the opposite:
function (bytes) {
var temperature = (bytes[0] << 8 | bytes[1]);
return {
temp: temperature/100.0
};
}
I now get the same values back that I’m sending 
A good help to figure this out was this youtube video for me: https://www.youtube.com/watch?v=MBcKblLODJQ&t=4s
@kersing you mention that I don’t have to use the decoding function in the ttn backend, can you explain how that would work? When I remove the payload function, the default payload function seems to be put back in.
1 Like
@erikHouten
The default function does not decode data, it simply passes the data on to the client which means you can do your own decoding in the client.
a 16bit integer is 2 bytes and a 2 byte array is 16bits. Ie you don’t need to create the 2 byte array. Just send the 16bit int.
You might also want to have a look at https://github.com/thesolarnomad/lora-serialization (as mentioned before in Approaches for efficient payload encoding). It should take care of any endiannes issues.
var msg2 = { payload: msg.payload.length };
msg2.payload = JSON.parse(msg.payload);
msg2.payload = new Buffer(msg2.payload.payload, ‘base64’).toString(‘hex’);
var lat = Buffer(msg2.payload, ‘hex’).readFloatLE(0);
var lon = Buffer(msg2.payload, ‘hex’).readFloatLE(4);
msg2.payload= “[{“lat”:” + lat + “,“lng”:” + lon + “}]”;//"{“lat”:lat,“lng”:lon};
return msg2;
This is how I am decoding the ttn payload via node-red. This assumes that the payload I am getting from ttn via mqtt is an 8 byte payload, coming out of the node-red mqtt node in base64, with the first four bytes being one float, and the second four bytes being the second float.
If you are trying to decode a raw 8 byte value, it’s even easier:
var lat = new Buffer(msg.payload).readFloatLE(0);
var lon = new Buffer(msg.payload).readFloatLE(4);
msg.payload= “[{“lat”:” + lat + “,“lng”:” + lon + “}]”;//"{“lat”:lat,“lng”:lon};
return msg;
I hate javascript and am terrible at doing anything with it, so there is probably a more efficient way to do this.
1 Like
@erikHouten
On staging.thethingsnetwork.org/applications I use this function:
function(bytes) {
var Bytes2Float32 = {
decodeFloat: function(bytes, signBits, exponentBits, fractionBits, eMin, eMax, littleEndian) {
var totalBits = (signBits + exponentBits + fractionBits);
var binary = "";
for (var i = 0, l = bytes.length; i < l; i++) {
var bits = bytes[i].toString(2);
while (bits.length < 8)
bits = "0" + bits;
if (littleEndian)
binary = bits + binary;
else
binary += bits;
}
var sign = (binary.charAt(0) == '1') ? -1 : 1;
var exponent = parseInt(binary.substr(signBits, exponentBits), 2) - eMax;
var significandBase = binary.substr(signBits + exponentBits, fractionBits);
var significandBin = '1' + significandBase;
var i = 0;
var val = 1;
var significand = 0;
if (exponent == -eMax) {
if (significandBase.indexOf('1') == -1)
return 0;
else {
exponent = eMin;
significandBin = '0' + significandBase;
}
}
while (i < significandBin.length) {
significand += val * parseInt(significandBin.charAt(i));
val = val / 2;
i++;
}
return sign * significand * Math.pow(2, exponent);
}
}
var value1 = [bytes[4], bytes[5], bytes[6], bytes[7]];
var value2 = [bytes[10], bytes[11], bytes[12], bytes[13]];
var value3 = [bytes[14], bytes[15], bytes[16], bytes[17]];
var value1float = Bytes2Float32.decodeFloat(value1, 1, 8, 23, -126, 127, false);
var value2float = Bytes2Float32.decodeFloat(value2, 1, 8, 23, -126, 127, false);
var value3float = Bytes2Float32.decodeFloat(value3, 1, 8, 23, -126, 127, false);
return {
payload : bytes,
batteryVoltage: value1float,
pressure: value2float,
temperature: value3float
};
}
It works for me. The decoded json is sent to the Azure IoTHub (with webjobs), there it goes to a SQL-server and a demo-web app shows the data (http://cloud.keller-druck.com).
Be aware that a Test doesn’t work (
TypeError(“Cannot access member ‘toString’ of undefined”) )
On console.thethingsnetwork.org/applications on the other hand, you must have a successful Test before saving the payload function. Therefore, I couldn’t get it working there.
1 Like
Hi,
Can someone send me a code for HX711 loadcell amplifier and DHT22 library?
I have tried this:
uint16_t weight = scale.get_units() * 100;
// Read sensor values and multiply by 100 to effictively have 2 decimals
uint16_t humidity = dht.readHumidity(false) * 100;
// false: Celsius (default)
// true: Farenheit
uint16_t temperature = dht.readTemperature(false) * 100;
// Split both words (16 bits) into 6 bytes of 8
byte message[6];
message[0] = highByte(temperature);
message[1] = lowByte(temperature);
message[2] = highByte(humidity);
message[3] = lowByte(humidity);
message[4] = highByte(weight);
message[5] = lowByte(weight);
LMIC_setTxData2(1, message, sizeof(message), 0);
}
On TTN side this is the payload function:
function (bytes) {
var decodedValue1 = (bytes[4] << 8) | bytes[5];
var decodedValue2 = (bytes[2] << 8) | bytes[3];
var decodedValue3 = (bytes[0] << 8) | bytes[1];
return { Weight: decodedValue1/100, Temp: decodedValue2/100, Hum: decodedValue3/100};
}
The result is this. It should be 4 decimal each:
Hum 20.3
Temp 1
Weight 654.36
You’re multiplying by 100, and storing that in memory as an “unsigned 16 bits integer” (uint16), hence losing all remaining decimals. For example: 12.34567 is stored and sent as 1234, which is probably just fine as most sensors really don’t have an accuracy of 4 decimals (or even 2 decimals). It even says this in your code:
// Read sensor values and multiply by 100 to effictively have 2 decimals
uint16_t humidity = dht.readHumidity(false) * 100;
As 2 bytes unsigned integers can only hold values 0 to 65535, you don’t really have room to multiply by, say, 1000 to get 3 decimals as then any value larger than 65.5 would cause an overflow with unexpected results.
Things are even more complicated if the values might be negative, in which case the JavaScript in the payload functions expects 4 bytes to do proper calculations for negative values:
// Signed 16 bits integer, -32767 up to +32767
int16_t temperature = dht.readTemperature(false) * 100;
…with in the payload function:
decodedValue3 = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
More details on the above “sign-extension” in Decrypting messages for dummies (and more links to examples in Is there any documentation on payload functions?).
You’re also mixing the values. You’re sending 2 bytes for temperature, humidity and weight. But you’re are showing that as humidity, temperate and weight. So, 20.3 is your temperature. I’d clean up and use the same order in your code:
// For staging-v1
function(bytes) {
// temperature might be negative; sign-extend to get 4 bytes:
var t = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
var h = bytes[2]<<8 | bytes[3];
var w = bytes[4]<<8 | bytes[5];
return {
temperature: t / 100,
humidity: h / 100,
weight: w / 100
}
}
I don’t understand why you’re getting a humidity of exactly 1. What does the false in the formula do? I guess the result of dht.readHumidity(false) is 1 or 1.00, which multiplied by 100 is sent as 100 and then decoded back to 1.
By the way, on the production environment the first line of a Decoder payload function needs to read:
function Decoder(bytes, port) {
And as Staging will be taken offline at the end of March, I strongly advise to use production.
2 Likes
Hi !
I have some difficult with the payload function.
Here is my Payload: 32352E3736302E32
This is for a DHT22 sensor. When i use python I can read values:
>>> binascii.unhexlify('32352E37')
b'25.7'
>>> binascii.unhexlify('36302E32')
b'60.2'
But I can’t make readable on the decoder. What can I do I tried … a lot !
thx (and sorry for my english)
That’s decoding text (where the hexadecimal 0x32 is the character 2, 0x35 is 5, 0x2E is the dot, and 0x37 is 7). But your node should not send text as that needs 8 bytes for just two values with a very limited range.
Still then, while you’ve not changed your node’s code to use a better encoding, and assuming that always 4 characters per value are used, so if:
- no negative values are sent
- zero is sent as
00.0, or as0.0with a leading space - values below 10 are sent with a leading space or zero, like
08.1 - integer values are sent with a fractional part, like
12.0
…then you could decode the text using:
function Decoder(bytes, port) {
// test with 32352E3736302E32
var text1 = String.fromCharCode.apply(null, bytes.slice(0, 4));
var text2 = String.fromCharCode.apply(null, bytes.slice(4, 8));
return {
text1: text1,
float1: parseFloat(text1),
text2: text2,
float2: parseFloat(text2)
}
}
…which will get you:
{
"float1": 25.7,
"float2": 60.2,
"text1": "25.7",
"text2": "60.2"
}
2 Likes
Oh I understand, so I need to optimise the code. I’m totally “noob” I need to learn. Thanks for your explanation.
BTW, here is the code
void onEvent (ev_t ev) {
Serial.print(os_getTime());
Serial.print(": ");
switch(ev) {
case EV_SCAN_TIMEOUT:
Serial.println(F("EV_SCAN_TIMEOUT"));
break;
case EV_BEACON_FOUND:
Serial.println(F("EV_BEACON_FOUND"));
break;
case EV_BEACON_MISSED:
Serial.println(F("EV_BEACON_MISSED"));
break;
case EV_BEACON_TRACKED:
Serial.println(F("EV_BEACON_TRACKED"));
break;
case EV_JOINING:
Serial.println(F("EV_JOINING"));
break;
case EV_JOINED:
Serial.println(F("EV_JOINED"));
break;
case EV_RFU1:
Serial.println(F("EV_RFU1"));
break;
case EV_JOIN_FAILED:
Serial.println(F("EV_JOIN_FAILED"));
break;
case EV_REJOIN_FAILED:
Serial.println(F("EV_REJOIN_FAILED"));
break;
case EV_TXCOMPLETE:
Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
if (LMIC.txrxFlags & TXRX_ACK)
Serial.println(F("Received ack"));
if (LMIC.dataLen) {
Serial.println(F("Received "));
Serial.println(LMIC.dataLen);
Serial.println(F(" bytes of payload"));
}
// Schedule next transmission
os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
break;
case EV_LOST_TSYNC:
Serial.println(F("EV_LOST_TSYNC"));
break;
case EV_RESET:
Serial.println(F("EV_RESET"));
break;
case EV_RXCOMPLETE:
// data received in ping slot
Serial.println(F("EV_RXCOMPLETE"));
break;
case EV_LINK_DEAD:
Serial.println(F("EV_LINK_DEAD"));
break;
case EV_LINK_ALIVE:
Serial.println(F("EV_LINK_ALIVE"));
break;
default:
Serial.println(F("Unknown event"));
break;
}
}
void do_send(osjob_t* j){
byte buffer[8];
float t = dht.readTemperature();
float h = dht.readHumidity();
dtostrf(t, 2, 1, buffer);
dtostrf(h, 2, 1, &buffer[4]);
String res = buffer;
res.getBytes(buffer, res.length() + 1);
Serial.println("");
Serial.print("Sending - temperature: ");
Serial.print(t);
Serial.print(", humidity: ");
Serial.print(h);
Serial.println("");
// Check if there is not a current TX/RX job running
if (LMIC.opmode & OP_TXRXPEND) {
Serial.println(F("OP_TXRXPEND, not sending"));
} else {
// Prepare upstream data transmission at the next possible time.
LMIC_setTxData2(1, (uint8_t*) buffer, res.length(), 0);
Serial.println(F("Packet queued"));
}
// Next TX is scheduled after TX_COMPLETE event.
}This goes horribly wrong for any value that does not convert to exactly four characters. (If you found that code elsewhere, please refer the author to this answer…)
Instead, use something like explained above:
// Read sensor values and multiply by 100 to effectively keep 2 decimals
// Signed 16 bits integer, -32767 up to +32767
int16_t t = dht.readTemperature() * 100;
// Unsigned 16 bits integer, 0 up to 65535
uint16_t h = dht.readHumidity() * 100;
This can be encoded into just 4 bytes for the 2 values, using some “shifting” as explained in more detail in Decrypting messages for dummies:
byte buffer[4];
buffer[0] = t >> 8;
buffer[1] = t;
buffer[2] = h >> 8;
buffer[3] = h;
LMIC_setTxData2(1, buffer, sizeof(buffer), 0);
And decoded like also explained in more detail in the link above:
function Decoder(bytes, port) {
var t = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
var h = bytes[2]<<8 | bytes[3];
return {
temperature: t / 100,
humidity: h / 100
}
}
You could probably even send humidity without any decimals, saving yet another byte:
// Unsigned 8 bits integer, 0 up to 255
uint8_t h = dht.readHumidity();
…with:
byte buffer[3];
buffer[0] = t >> 8;
buffer[1] = t;
buffer[2] = h;
LMIC_setTxData2(1, buffer, sizeof(buffer), 0);
…and:
function Decoder(bytes, port) {
var t = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
var h = bytes[2];
return {
temperature: t / 100,
humidity: h
}
}
3 Likes
Work Perfectly (with the optimized version for humidity) ! Big thank for all your explanations ! Clear and working. I’m getting smarter today 
Any ideas i could do what you are describing?
Let me first tell you that i use loriot server instead of ttn.
I tried to connect loriot backend with node red and ibm bluemix but i am struggling with javascript.
So any ideas about decoding with python and then connecting to an application server?
and you asked questions on the TTN forum ? … strange 
your welcome 
Just for future reference, another option to decode an IEEE-754 floating point in Decode float sent by Lopy as Node - #2 by arjanvanb.
1 Like