Thanks for your information. My device was/is working and i do get a payload which need to be encrypted. I can read the payload by hand (see above). But how can i put it in a decoder? I was looking at the tutorials the whole weekend but i do not understand. Cryptic world for me and i am looking for some help and explenation. As a problem i do see the length of my payload?!
Edit: its a normal mbus systematic as i see
I remember that I got stuck at the same point when I was starting with loraWan. If you know how, it is simple…
So, If you get data, you have created an application and added some devices. You will need one application for every device type.
In the application panel you see a button “Payload formats”. You will need a “decoder” to “decrypt” your payload. If you do not have a decoder, your payload is sent as a byte buffer.
So, place some js-code here, like:
function Decoder(bytes, port) {
// Decode an uplink message from a buffer
// (array) of bytes to an object of fields.
var decoded = {};
if (port === 100) {
decoded.open = bytes[0]&1;
decoded.batt = (25+(bytes[1]&15))/10;
decoded.temp = (bytes[2]&127)-32
decoded.time = (bytes[4]<<8)|bytes[3];
decoded.count = (bytes[7]<<16)|(bytes[6]<<8)|bytes[5];
return decoded;
}
}
Now you get your payload as a JSON-Object.
You should care for the port number, as most devices send different payloads for different ports. A port is just a number send by the device to quaify the kind of message.
To get your data out of TTN there are different ways. I have used node-red, which can use MQTT or a special TTN-Node.
@EFthings01
Hey thanks. Well i have my device installed up to the point “Payload Formats”. I do not have a written decoder to get the payload in JS (that is my problem!). Thats the point where i need some help. I can tell which part of payload is giving which information and how it should look but i can not translate it in a decoder to generate a JSON output.
Afterwards i am able to writer the JSON output via node red in an influxDB and make visuization with graphana 
that is working fine.
Your code kinda works…
I guess the payload is a hex string which will be converted?! The payload did not need to be converted. I need to take a number from a payload.
E.G. 0A5E 4104 = you can Skip 0A5E, now you have to take it backwards 04+41=0441=44,1°C
Edit: and the port is knowing (port===2)
I think, the code above is easy to adapt and you can use the decoder functions given above.
Anyway, you can also leave the decoder empty and write a decoder in node-red. As both are JS, the code will be pretty similar.
Please be careful where you put your “return decoded”. If you place it inside the if-statement, messages will only get forwarded if they are translated. If you place it outside, you will get a JSON-Payload for port 100 and a binary buffer for all other messages. This may give some errors in node-red .
*0A5E 4104
04 = will be bytes[4]? If i do use it like that i dont get 04 as result
I am so sorry but for me everything is spanish… the tutorials are not very usefull if u have 0 knowledges about programming :o at least it feels like
A few observations:
-
You’re not telling us which device you’re using? Please tell us, so future users can find this post. Also please add a link to the documentation so people don’t need to guess to help you.
-
As for “I guess the payload is a hex string”: no. LoRaWAN devices basically transmit bits, a.k.a. binary data. In your payload
4104is a hexadecimal representation of two bytes, each having 8 bits:01000001and00000100. (Often such presentation is prefixed with0xto indicate it’s shown as hexadecimal, like 0x4104.) But that is just a way to display the bytes. Most often, computers decode the above 2 bytes to the decimal number 16644. Please see How to send payload in Hex-format? and its links, like to Working with bytes, to make sure you understand. -
It’s really weird to decode those hexadecimal numbers 0x04 and 0x41 to the decimal 44.1 °C. But we’ve seen that before in Decoding Smart Waste Bin Detector DF702 payload:
(This encoding, when indeed true, makes handling unnecessary complex and makes the payload unnecessary long. The payload you’re showing is 42 bytes. The maximum at SF12 is 51 bytes; sending 42 will take a whopping 2.5 seconds.)
-
To use something like the above, you’d need to change the order, but above all you still need to find the specific bytes you need in the above calculation. You could try to find the hexadecimal pattern 0x0A5E, but the user manual should explain how to truly know how many bytes to skip.
-
As for:
In programming, the “index” for arrays (lists) is often zero-based, so for the above it should be
bytes[3](andbytes[0]would give you 0x0A). But there’s a lot more data before that0A5E 4104, which you need to take into account for that index too.
2 Likes
So, maybe you need to lern some Javascript first: https://www.w3schools.com/js/ or https://js.do/
1 Like
Hey sorry, you are right! The sensor is the CMi4110 by elvaco (https://www.elvaco.se/Image/GetDocument/en/286/cmi4110-users-manual-english.pdf). I am using the transmition mode standard and i know that the payload is long but i did not set it up that way. Its by factory.
function Decoder(bytes, port) {
return {
// Interpret hexadecimal 0x0230 as decimal 230, not decimal 560
Energieverbrauch:
((bytes[4] >> 4) * 1000
+ (bytes[4] & 0x0F) *100
+ (bytes[3] >> 4) * 10
+ (bytes[3] & 0x0F))/1000,
Volumen:
((bytes[11] >> 11) * 100000
+ (bytes[11] & 0x0F) * 10000
+ (bytes[10] >> 10) * 1000
+ (bytes[10] & 0x0F) * 100
+ (bytes[9] >> 9) * 10
+ (bytes[9] & 0x0F))/100,
};
}
I was trying this for my payload:
000C06575800000C14223902000B2D5701000B3B2008000A5A06060A5E41040C789938187002FD170000
First part of payload will be generated correctly
(get 5758 => 58+57 =>5857 => 5,857)
Second Part is not working 
I guess i am doing something wrong?!
(should be: get 223902 => 02+39+22 =>023922 => 239,22
now it generates 209.02 as value)
1 Like
Above, you’re “shifting” 11, 10 and 9 bits to the right, in >> 11 and all. But you should only shift half a byte to the right, hence 4 bits. Try this:
Volumen:
((bytes[11] >> 4) * 100000
+ (bytes[11] & 0x0F) * 10000
+ (bytes[10] >> 4) * 1000
+ (bytes[10] & 0x0F) * 100
+ (bytes[9] >> 4) * 10
+ (bytes[9] & 0x0F))/100,
As an aside, I could not get a definitive answer on how the values are encoded from the manual (the xxxx part in the image below). How did you get those details?
Also, the payload might not always use the prefix 0C06 for Energy, and 0C14 for Volume, and then needs a different value for the decimals, a different unit, and sometimes even supplies an additional byte for the value then? Or does one configure the device for a specific output?
Excerpt from the manual:
And, oops, 0C06xxxxxxxx is listed twice in that manual? 
1 Like
Aw perfect. No i understand what the 4 is makeing 
It Works. Thanks!!!
Well those xxxx are the values i do get from the sensor.
For Example my payload
000C06575800000C14[…] = 000C6xxxxxxxx0C14[…]
How i know that the 5758000 should be decoded to 5,857?
Well we can go and watch at the installed heatcounter at the Display where the sensor is installed and find out how it should be decoded in real So i can compare the real value and the payload value plus the information of the manual and thats how i know to use the unit MWh oder kWh (in this case) I hope u can understand what i mean (hard to explain in english for me - german language).
The prefixes CAN change but not in our system. We will always get those same information in the same unit and same decimals :o To include those informations in payload decoding it will be a massive rewriting of the decoder i guess?
Edit: yes our configuration gives the output in the same byte systematic (standard mode). The prefixes, units and decimals can change depending on where you install the sensor but not in our setting. So we alaway getting the same byte length, same units and same prefix Lets call my solution a semi automatic decoder 
1 Like
Indeed, creating a more generic decoder would require quite some effort, I think. You could add some sanity checks though, like to ensure that the first byte indeed indicates the “standard” message format, and that you see the expected “DIF” and “VIF” values for the readings that you want to extract.
Just for future reference:
Meanwhile I found that “BCD” refers to binary-coded decimal, which for M-Bus indeed uses half-bytes (nibbles; 4 bits) for each digit. And things like “BCD8” then refer to 8 digit values, which need 8 nibbles, hence 4 bytes. (So, for each digit in a BCD value you’ll only see the hexadecimal values of 0x0 thru 0x9; 0xA thru 0xF are never used, which is a slight waste of bandwidth. But well, you cannot change that.)
So, you’re right about how to get the actual values.
I could not quickly find a pure JavaScript library for some generic M-Bus decoding. Reading a bit more on the M-Bus/Meter-Bus protocol, it seems that generic decoding still needs a hardcoded decision tree for expected “VIF” values (whereas “DIF” values can be used to determine the length and type regardless the VIF, but it seems many possible DIF values are never used in this very device).
Even when limiting to this very device, it seems that small errors in the Elvaco documentation will require a lot of extra field testing. Like the length of xxxxxx does not always match the number of expected bytes: the excerpt of the documentation above claims Energy is always 6 bytes and BCD8, but nevertheless shows 7 bytes in, e.g., 0CFB00xxxxxxxx. So, that should either read 0CFB00xxxxxx which would then be BCD6, or Energy could be 8 bytes in some cases? Also 0C06xxxxxxxx is listed twice in the same short excerpt, and Power is documented as “BCD8” which should probably read “BCD6”. That’s already 3 errors while quickly browsing one page in the specification.
Finally, a generic decoder would need to include the unit in its output (like apparently Energy can be one of MWh, kWh, MJ or GJ) which then needs to be respected during further processing, or the decoder would need to convert it to a single unit.
Yes i will post my endversion soon with explanation for other people!
I totally agree that it is not that great but we have to deal with. Maybe for future projects we should look for other sensors which can be more modified or do handel the payload in an other way. Nevertheless i really want to thank you for your patient and help!!! We got the thinks working! Get the payload via node red into an InfluxDB and visualisation with grafana 
Thank your very much!!!
For Node-RED you might be able to find some libraries to decode M-Bus for you, rather than using a TTN Payload Format to do the preprocessing. On the other hand, you obviously only need the decoding part, not any communication, so any full M-Bus library might be overkill. Also, just in case the device’s format has some errors, you might not be able to fix those. Still then, the JavaScript in NodeJS is a bit more advanced too, and it would keep all your code in one place.
As for the Energy with 0x0CFB00 and 0x0CFB01, I think that indeed adds another byte to the payload to keep 8 digits for the value, as otherwise its existence makes no sense: 6 digits with one or zero decimals would also fit in the other 0x0C06, 0C07, 0C0E and 0C0F formats. So, the “extended VIF” 0xFB seems to be needed for large values. Which also makes me wonder:
So, the unit is not set for some specific output using some configuration? I think it might not be safe to assume the unit and number of decimals never change. If, over time, the values get to be too large to fit in the fixed number of digits, are you sure the device will not decrease its number of decimals, or select a larger unit, to prevent truncation of its readings?
In case you didn’t do that yet, one can easily create a helper for the packed BCD:
/**
* Convert the array of bytes into an unsigned integer, assuming packed
* binary-coded decimal (BCD) with an even number of nibbles, LSB.
*/
function bcdToUint(bytes) {
return bytes.reduceRight(function(acc, byte) {
return 100*acc + 10*(byte >> 4) + (byte & 0x0F);
}, 0);
}
The above can be used with, e.g, Volumen: bcdToUint(bytes.slice(9, 12)) / 100 (where index 12 is not included in the slice of the array, so passes an array of length 3, not 4).
Next, such code is less prone to programming errors when used along with a running index: if some var i has value 9, then it can be incremented by 3 on the fly using addition assignment, when used in bcdToUint(bytes.slice(i, i += 3)) to indicate one wants to consume the next 3 bytes.
And though the documentation seems to suggest a specific order of the data, each part is also uniquely identified by its “VIF”, and its “DIF” specifies the BCD length. That allows for such something quite generic, also supporting the other formats, like:
/**
* Decoder for M-Bus style payload of the Elvaco CMi4110 sensor.
*
* 2019-11-21: initial, PARTIAL, UNTESTED implementation
*
* Notes:
*
* - In M-Bus the length of BCD values can be determined from the data,
* but the value multiplier (the number of decimals) is part of the
* specification (along with the unit), and not in the data itself.
*
* - The documentation seems to suggest the order of DIBs is fixed, but
* this decoder does not rely on any order, using the VIF values.
*/
/**
* Convert the array of bytes into an unsigned integer, assuming packed
* binary-coded decimal (BCD) with an even number of nibbles, LSB.
*/
function bcdToUint(bytes) {
return bytes.reduceRight(function(acc, byte) {
return 100*acc + 10*(byte >> 4) + (byte & 0x0F);
}, 0);
}
/**
* Get a hexadecimal representation with leading zeroes for each byte.
*/
function hex(bytes, separator) {
return bytes.map(function (b) {
return ('0' + b.toString(16)).substr(-2);
}).join(separator || '');
}
function Decoder(bytes, port) {
var result = {};
var i = 0;
result.messageType = [
'standard',
'compact',
// 0x02: JSON; not supported in this decoder
undefined,
'scheduled daily redundant',
'scheduled extended'
][bytes[i++]];
if (!result.messageType) {
return {
error: 'unsupported message type',
unparsed: hex(bytes)
};
}
// Iterate the M-Bus Data Information Blocks, based on their Data
// Information Field and Value Information Field
while (i < bytes.length) {
var dif = bytes[i++];
var vif = bytes[i++];
var difData = dif & 0x0F;
var isAccumulated = (dif & 0x40) > 0;
// For this device, actually only BCD4, BCD6 and BCD8 are expected
var isBcd = (difData >= 0x09 && difData <= 0x0C) || difData === 0x0E;
var bcdLen = isBcd ? difData & 0x07 : undefined;
// VIF 0xFB is a special case for large values, handled below. For
// other BCD values extract the unsigned integer and increase the
// index here:
var bcdValue = isBcd && vif !== 0xFB
? bcdToUint(bytes.slice(i, i += bcdLen))
: undefined;
// We could switch on just VIF, but then unsupported DIF values
// will go unnoticed
var difVif = dif << 8 | vif;
switch (difVif) {
// Energy, or accumulated energy at 24:00
case 0x0C06:
case 0x0C07:
case 0x4C06:
case 0x4C07:
result[isAccumulated ? 'accumulatedEnergy' : 'energy'] = {
// 0x06 = 3 decimals, 07 = 2 decimals
value: bcdValue / Math.pow(10, 0x09 - vif),
unit: 'MWh'
};
break;
case 0x0C0E:
case 0x0C0F:
case 0x4C0E:
case 0x4C0F:
result[isAccumulated ? 'accumulatedEnergy' : 'energy'] = {
// 0x0E = 3 decimals, 0F = 2 decimals
value: bcdValue / Math.pow(10, 0x11 - vif),
unit: 'GJ'
};
break;
case 0x0CFB:
case 0x4CFB:
// Extended VIF, providing an additional byte after VIF
var vifExtension = bytes[i++];
bcdValue = bcdToUint(bytes.slice(i, i += bcdLen));
switch (vifExtension) {
case 0x00:
case 0x01:
result[isAccumulated ? 'accumulatedEnergy' : 'energy'] = {
// 0x00 = 1 decimal, 01 = no decimals
value: bcdValue / Math.pow(10, 0x01 - vifExtension),
unit: 'MWh'
};
break;
case 0x08:
case 0x09:
result[isAccumulated ? 'accumulatedEnergy' : 'energy'] = {
// 0x08 = 1 decimal, 09 = no decimals
value: bcdValue / Math.pow(10, 0x09 - vifExtension),
unit: 'GJ'
};
break;
default:
return {
error: 'unexpected extended VIF',
parsed: result,
unparsed: hex(bytes.slice(i - 3))
};
}
break;
// Volume
case 0x0C14:
case 0x0C15:
case 0x0C16:
result.volume = {
// 0x14 = 2 decimals, 15 = 1, 16 = no decimals
value: bcdValue / Math.pow(10, 0x16 - vif),
unit: 'm3'
};
break;
// Power
case 0x0B2B:
case 0x0B2C:
case 0x0B2D:
case 0x0B2E:
result.power = {
// 0x2B = 3 decimals, 2C = 2, 2D = 1, 2E = no decimals
value: bcdValue / Math.pow(10, 0x2E - vif),
unit: 'kW'
};
break;
// Flow
case 0x0B3B:
case 0x0B3C:
case 0x0B3D:
case 0x0B3E:
result.flow = {
// 0x3B = 3 decimals, 3C = 2, 3D = 1, 3E = no decimals
value: bcdValue / Math.pow(10, 0x3E - vif),
unit: 'm3/h'
};
break;
// Forward temperature
case 0x0A5A:
case 0x0A5B:
result.forwardTemperature = {
// 0x5A = 1 decimal, 5B = no decimals
value: bcdValue / Math.pow(10, 0x5B - vif),
unit: 'C'
};
break;
// Return temperature
case 0x0A5E:
case 0x0A5F:
result.returnTemperature = {
// 0x5E = 1 decimal, 5F = no decimals
value: bcdValue / Math.pow(10, 0x5F - vif),
unit: 'C'
};
break;
// Meter ID (using BCD values)
case 0x0C78:
// This will remove leading zeroes in the JSON output
result.meterId = bcdValue;
break;
// Meter date and time
case 0x046D:
// Not really implemented yet
result.dateTime = hex(bytes.slice(i, i += 4));
break;
// Error and warning flags
case 0x02FD:
// Skip the 0x17 in 02FD17 (we should probably validate it)
i++;
result.errorFlags = hex(bytes.slice(i, i += 2));
break;
// Error message
case 0x0E00:
result.error = 'CMi4110 unable to communicate with UH50/UC50';
break;
default:
return {
error: 'unexpected format',
parsed: result,
unparsed: hex(bytes.slice(i - 2))
};
}
}
return result;
}
For the payload from your first post, this yields (sorted for readability):
{
"messageType": "standard",
"energy": {
"unit": "MWh",
"value": 5.857
},
"volume": {
"unit": "m3",
"value": 239.22
},
"power": {
"unit": "kW",
"value": 15.7
},
"flow": {
"unit": "m3/h",
"value": 0.82
},
"forwardTemperature": {
"unit": "C",
"value": 60.6
},
"returnTemperature": {
"unit": "C",
"value": 44.1
},
"meterId": 70183899,
"errorFlags": "0000"
}
Of course: untested… So, future readers: if you ever use this, compare it to the documentation, and please report back here with some real payload values? Thanks.
2 Likes
I got a reply from Elvaco:
the module will use the same VIF code (unit) as is presented to it by the meter.
I don’t know if that means that the meter always uses the same VIF, or could change the VIF over time when totals get to be large values. But I guess whoever uses these meters knows.
It works. I do get
{
"energy": {
"unit": "MWh",
"value": 5.857
},
"errorFlags": "0000",
"flow": {
"unit": "m3/h",
"value": 0.82
},
"forwardTemperature": {
"unit": "C",
"value": 60.6
},
"messageType": "standard",
"meterId": 70183899,
"power": {
"unit": "kW",
"value": 15.7
},
"returnTemperature": {
"unit": "C",
"value": 44.1
},
"volume": {
"unit": "m3",
"value": 239.22
}
}
For my Payload
00 0C 06 57 58 00 00 0C 14 22 39 02 00 0B 2D 57 01 00 0B 3B 20 08 00 0A 5A 06 06 0A 5E 41 04 0C 78 99 38 18 70 02 FD 17 00 00
1 Like
Does the MWh for the energy match whatever is on the meter? (This is the 0x0C06 mentioned twice in the manual, with different units…)
"energy": {
"unit": "MWh",
"value": 5.857
}
Yes the meter does show MWh on its display and also in the payload
1 Like
does anyone have an idea if it is possible to use crypto function in a payload decoder?
I have a mbus message that is encoded with aes key … can aes key be decoded in ttn payload decoder?
There is no mechanism to require or import external libraries. But you can include any JavaScript code by using copy-paste.
However, you’ll need to administer the secrets, and you might not want to share those with TTN, and those might be different for each device in an application in TTN Console. So, it might be easier to do the decryption (and maybe all of the payload decoding) in your own application instead.