Currently I’ve some problems using cayenne lpp payload directly on my node, so I’m using a custom payload.
Is there a way to perform some payload conversion in the TTN console custom payload section in order to generate a lpp payload that I can use on Cayenne?
Because currently I have proper values in TTN console only, but can use them in Cayenne.
Maybe I’m mistaking and data is sent to Cayenne before being decoded locally on console, if so, do you have any solution for me?
what about a LoPy and this lib A Python library for Cayenne LPP 
data is decoded in the backend before sending to MyDevices
@kalon33
I have exactly the same need as you, so I tried, first in the console under payload format decoder. Got some JS code working returning an object with my data, until there all is fine.
Then I tried the converter (that seems fired after decoder) to convert my JS object to an array of bytes (compatible with cayenne LPP of course) and then I expected the output to be able to go to cayenne integration.
But seems we need to return an object from converter, so no luck, that’s weird because it would be a nice feature to be able to convert payload to bytes array (here cayenne LPP) from console. That would extend to any possibility.
@johan, any idea if it’s be possible?
Refer to the “Data Types” section here
https://mydevices.com/cayenne/docs/lora/#lora
The uplink Payload Structure is as follows
1 Byte 1 Byte N Bytes 1 Byte 1 Byte M Bytes …
Data1 Ch. Data1 Type Data1 Data2 Ch. Data2 Type Data2
Each data type can use 1 or more bytes to send the data according to the following table.
Type IPSO LPP Hex D.Size Data Resolution per bit
Digital Input 3200 0 0 1 1
Digital Output 3201 1 1 1 1
Analog Input 3202 2 2 2 0.01 Signed
Analog Output 3203 3 3 2 0.01 Signed
Illuminance Sensor 3301 101 65 2 1 Lux Unsigned MSB
Presence Sensor 3302 102 66 1 1
Temperature Sensor 3303 103 67 2 0.1 °C Signed MSB
Humidity Sensor 3304 104 68 1 0.5 % Unsigned
Accelerometer 3313 113 71 6 0.001 G Signed MSB per axis
Barometer 3315 115 73 2 0.1 hPa Unsigned MSB
Gyrometer 3334 134 86 6 0.01 °/s Signed MSB per axis
GPS Location 3336 136 88 9 Latitude : 0.0001 ° Signed MSB
Longitude : 0.0001 ° Signed MSB
Altitude : 0.01 meter Signed MSB
Check out how @kersing sends this data in his code here
Here is the code bit, you can see the first channel has 9 bytes for GPS( 0x88), and the 2nd channel has 2 bytes for the analog (battery voltage) value
if (fix) {
buffer[i++] = 0x01;
buffer[i++] = 0x88;
h = lat / 1000L;
buffer[i++] = (h >> 16) & 0xff;
buffer[i++] = (h >> 8) & 0xff;
buffer[i++] = h & 0xff;
h = lon / 1000L;
buffer[i++] = (h >> 16) & 0xff;
buffer[i++] = (h >> 8) & 0xff;
buffer[i++] = h & 0xff;
buffer[i++] = (alt >> 16) & 0xff;
buffer[i++] = (alt >> 8) & 0xff;
buffer[i++] = alt & 0xff;
}
measuredvbat *= 2; // we divided by 2, so multiply back
measuredvbat *= 3.3; // Multiply by 3.3V, our reference voltage
measuredvbat /= 1024; // convert to voltage
h = measuredvbat * 100;
Serial.print("VBat: " ); Serial.println(h);
buffer[i++] = 0x02;
buffer[i++] = 0x02;
buffer[i++] = (h >> 8) & 0xff;
buffer[i++] = h & 0xff;This is how he sets up Cayenne here
When you are at the Cayenne end, you can represent the data channel as you want e.g. an analog value can be represented as “Battery Voltage”
The whole GPS tracker works nicely!
Well done @kersing
That code runs on the node, it would be nice if we could transform data to lpp format in the console as well. I have some nodes where changing the code is hard (nodes are deployed, 40 minute drive away), being able to reformat the data and push it to cayenne would allow skipping the additional server now required to do the same thing.
Exactly what we need, change payload content on fly, and we all are coder, we know it’s do able easily since decoder and transformer funct already exists
you have seen version V2 of lpp … bi directional
Indeed, the idea of a decoder, converter and validator is that you can separate decoding binary data (i.e. bytes to voltage), converting units (voltage to temperature) and validating measurements (remove outliers).
We’re dropping the converter and validator in V3 as they’re not being used; you can put conversion in the decoder, and it makes more sense to drop/filter out invalid values higher upstream.
The requested feature is really being able to transform binary data to another binary format, right? You don’t really want to manually convert your binary payload to CayenneLPP in JavaScript right, just to update the Cayenne dashboard?
What is the goal? Let’s see if we can find a sensible solution that leads to that!
@johan
Thanks for your answer. Indeed it’s correct our final objective is to see data on Cayenne dashboard.
And for this, on nodes where we have hand, we can change the code and it’s fine
But sometimes, on other nodes where we do not have the source code and possibility to change payloads (when we buy devices) we need to convert known payload to CayenneLPP format, whatever solution is fine as soon we have hand to transform binary to another binary. JS is cool but another idea is fine, we would adapt I think.
I see. I would completely decouple the payload from Cayenne in that case.
Instead of using the TTN-Cayenne integration, consider using “Bring your own thing” and route data to Cayenne with MQTT. You can easily do this with Node-RED, taking data from TTN and routing it to Cayenne, without bothering with CayenneLPP.
Absolutely, the drawback of this is the need of another clouded server running MQTT client and some JS scripts (node red or native).
I’m fan of “less streams, less servers” => less things to check and less problems
But I will do that, got an NodeRed Instance running on my server
Thanks for the tip.
@johan
I tried your solution, unfortunately I’m not happy how cayenne handle that. While everyone is using user/pass in the MQTT broker and then ThingsID in the topic, cayenne NEEDS broker clientID set as ThingsID in the broker (and also in the topic).
This means if you have 100 devices, you need 100 different broker connexion (each with it’s own clientID). despite the fact than configuring 100 brokers in node red will be a nightmare, even if it could works, I can’t (don’t want sorry) use as many connexion as things, painfull
I know it’s not TTN fault but I’m back to my first step. I think adding the possibility to return a byte buffer from the converter will be really simple for everyone here to convert any payload.
But, I know simple for us does not mean simple for you.
If you could just think about the idea it would be very kind.
I see. It’s not the first IoT platform with that requirement. That MQTT is really designed for end device to cloud, not cloud to cloud.
What you can do is posting with TTN’s JSON format, i.e. the one you see in MQTT and HTTP integration, to: https://lora.mydevices.com/v1/networks/ttn/uplink. Cayenne looks at the payload_raw, in which you can put anything that you like. But, it has to be CayenneLPP.
Now, I would say this is the hacky solution and there’s no guarantee this API will stay available this way.
Exactly what I wanted to do, I thought if TTN can send LPP I should be able to do also just needed the undocumented interface
Thanks for sharing, since I’m doing from node red should be easy just to transform payload_raw between TTN mqtt and cayenne HTTPS post
Hope there is no auth or source ip filtering for this one
Thanks
There is indeed no auth nor IP filtering, and that is indeed what’s is wrong with this interface
@johan So
won’t be implemented? Currently I don’t have any Node RED running, so I can do what @Charles plans to do… 
In V2 we don’t allow altering payload_raw. It will, technically, be supported in V3 but not sure if it will be GA in the public Consoles
@johan I tried your solution with no luck, may be I’m missing something
I convert the received TTN payload_raw to cayenne LPP payload_raw => OK
Then I send the full msg object (converted to JSON) as new msg.payload (so http request node has all original message in it’s msg.payload)
The recode function (after decoding incoming payload) is
var buf = Buffer.alloc(4+4+11);
// Channel 1, Analog In (Battery)
buf.writeUInt16BE(0x0102, 0);
buf.writeUInt16BE(batt*100, 2);
// Channel 2, Temp sensor
buf.writeInt16BE(0x0267, 4);
buf.writeInt16BE(temp*10, 6);
// Channel 3, GPS
buf.writeUInt16BE(0x0388, 8);
buf.writeIntBE(lat*10000, 10,3);
buf.writeIntBE(lon*10000, 13,3);
buf.writeIntBE( 0*10000, 16,3);
// Save new payload format to current msg payload_raw
msg.payload_raw = buf.toString('hex');
var retmsg = {};
// Get full current message as JSON
retmsg.payload = JSON.stringify(msg);
return retmsg;
Looks fine, output of recode (going to http request to url you mentioned) looks like OK
and output of http request (cayenne) says ok with HTTP 200
But I have noting on cayenne dashboard, and my node is created of course
I tried just sending payload_raw and hardware_serial but in this case, cayenne return 500 error so my 1st output seems good.
We’re so close, that’s it’s a pity it does not work. Do you know what field cayenne needs? Did I missed something?
Thanks for your help
I’ve not got time atm - but I would try to emulate the HTTP using postman first (just quicker to test really) This could help?