Absolutely, the drawback of this is the need of another clouded server running MQTT client and some JS scripts (node red or native).
I’m fan of “less streams, less servers” => less things to check and less problems 
But I will do that, got an NodeRed Instance running on my server
Thanks for the tip.
@johan
I tried your solution, unfortunately I’m not happy how cayenne handle that. While everyone is using user/pass in the MQTT broker and then ThingsID in the topic, cayenne NEEDS broker clientID set as ThingsID in the broker (and also in the topic).
This means if you have 100 devices, you need 100 different broker connexion (each with it’s own clientID). despite the fact than configuring 100 brokers in node red will be a nightmare, even if it could works, I can’t (don’t want sorry) use as many connexion as things, painfull
I know it’s not TTN fault but I’m back to my first step. I think adding the possibility to return a byte buffer from the converter will be really simple for everyone here to convert any payload.
But, I know simple for us does not mean simple for you.
If you could just think about the idea it would be very kind.
I see. It’s not the first IoT platform with that requirement. That MQTT is really designed for end device to cloud, not cloud to cloud.
What you can do is posting with TTN’s JSON format, i.e. the one you see in MQTT and HTTP integration, to: https://lora.mydevices.com/v1/networks/ttn/uplink. Cayenne looks at the payload_raw, in which you can put anything that you like. But, it has to be CayenneLPP.
Now, I would say this is the hacky solution and there’s no guarantee this API will stay available this way.
Exactly what I wanted to do, I thought if TTN can send LPP I should be able to do also just needed the undocumented interface
Thanks for sharing, since I’m doing from node red should be easy just to transform payload_raw between TTN mqtt and cayenne HTTPS post
Hope there is no auth or source ip filtering for this one
Thanks
There is indeed no auth nor IP filtering, and that is indeed what’s is wrong with this interface
@johan So
won’t be implemented? Currently I don’t have any Node RED running, so I can do what @Charles plans to do… 
In V2 we don’t allow altering payload_raw. It will, technically, be supported in V3 but not sure if it will be GA in the public Consoles
@johan I tried your solution with no luck, may be I’m missing something
I convert the received TTN payload_raw to cayenne LPP payload_raw => OK
Then I send the full msg object (converted to JSON) as new msg.payload (so http request node has all original message in it’s msg.payload)
The recode function (after decoding incoming payload) is
var buf = Buffer.alloc(4+4+11);
// Channel 1, Analog In (Battery)
buf.writeUInt16BE(0x0102, 0);
buf.writeUInt16BE(batt*100, 2);
// Channel 2, Temp sensor
buf.writeInt16BE(0x0267, 4);
buf.writeInt16BE(temp*10, 6);
// Channel 3, GPS
buf.writeUInt16BE(0x0388, 8);
buf.writeIntBE(lat*10000, 10,3);
buf.writeIntBE(lon*10000, 13,3);
buf.writeIntBE( 0*10000, 16,3);
// Save new payload format to current msg payload_raw
msg.payload_raw = buf.toString('hex');
var retmsg = {};
// Get full current message as JSON
retmsg.payload = JSON.stringify(msg);
return retmsg;
Looks fine, output of recode (going to http request to url you mentioned) looks like OK
and output of http request (cayenne) says ok with HTTP 200
But I have noting on cayenne dashboard, and my node is created of course
I tried just sending payload_raw and hardware_serial but in this case, cayenne return 500 error so my 1st output seems good.
We’re so close, that’s it’s a pity it does not work. Do you know what field cayenne needs? Did I missed something?
Thanks for your help
I’ve not got time atm - but I would try to emulate the HTTP using postman first (just quicker to test really) This could help?
At first sight the payload_raw is hex and it should be base64
I got your code to work as an “integration” between a TTN node device to Cayenne - well done.
In my case, I started with a TTN node device running the “Cayenne_lpp” sketch with the integration “Cayenne” setup and running.
The first thing I did, was remove the “Cayenne” integration in the TTN console.
Next I set up a node-red as you have shown.
The only thing I did differently. was change a few things in your “Recode” function
i.e. assign the correct channels to data and use this line to encode to base64
msg.payload_raw = buf.toString('Base64');
After this, I could see the values change in Cayenne
N.B. Even if the data code isn’t mapped correctly to channels, I still saw that the RSSI and SNR change in Cayenne - so that proves that it is receiving data
*edit - I also got postman to work using as follows: (the trick is not to include Payload)
POST https://lora.mydevices.com/v1/networks/ttn/uplink
Authorization No Auth
Headers Content-Type application/json
Body the json after payload e.g. {"app_id":"xxx..... ...."temperature_5":23.1},"_msgid":"cc993c47.19999"}
For any node, this is the JSON, you need to emulate - no secret needed and you have to publicise the DEVEUI. You also “Base 64” the payload_raw from the byte array (that was set up as per the “Cayenne lpp” format)
N.B. It doesn’t seem to matter what the “metadata” values are, as long as they are there
{"app_id":"YOUR_APPLICATION_ID","dev_id":"YOUR_DEV_ID","hardware_serial":"YOUR_8_BYTE_DEVEUI","payload_raw":"BAIAKAAAAAAAAAAAAAAAAAAAAA==","metadata":{"gateways":[{"gtw_id":"TEST_1","rssi":-73,"snr":6.75}]}}
*edit - added curl command (windows)
curl -v -X POST --data "{\"app_id\":\"YOUR_APP_ID\",\"dev_id\":\"YOUR_DEV_ID\",\"hardware_serial\":\"1122334455667788\",\"payload_raw\":\"BAIAKAAAAAAAAAAAAAAAAAAAAA==\",\"metadata\":{\"gateways\":[{\"gtw_id\":\"TEST_1\",\"rssi\":-73,\"snr\":6.75}]}}" https://lora.mydevices.com/v1/networks/ttn/uplink -H "Content-Type: application/json"
That was the trick, noticed some strange things .
I used node base64 and json but @CurlyWurly just showed that base64 can be native I’ll integrate this in Recode function. I’m doing some check and post new Recode source code
@johan you saved my day, @CurlyWurly thanks also, I just saw your post after got it working 
@johan,
I’m still having problem with GPS, I looked around everywhere (in CayenneLPP.ccp and and Cayenne doc) and CayenneLPP.cpp source code is coded exactly how the documentation is stating
So I’ve done the same with my GPS position says (node red logs)
ls18014 lat=5.760811666666667 lon=45.60061666666667
So I multiply per 10000 and then construct LPP format channel 08 GPS 0x88 and display LPP payload buffer (as hex so see) in node console
fixed lat=57608.11666666667 lon=456006.1666666667 Payload => 098800e10806f546000000
and final payload send (with other sensors)
Final payload 0102018a02670036098800e10806f546000000
Doing reverse calculation of hex show me
lat = 0x00e108 = 57608 so / 10000 = 5.7608
lon = 0x06f546 = 456006 so / 10000 = 45.6006
So all looks good, any idea why cayenne show my position in africa (and not the same as lat=lon=0 so not the pb) ?
I’ve noticed that the map in Cayenne can be confusing - If someone can’t see the GPS point, these are the things I found I have to check
This code works for me in the node-red function (points to Chester in UK North-West).
var buf = Buffer.alloc(4+11);
var batt = 2.4;
var lat = 53.1934;
var lon = -2.8931;
var alt = 145;
var i = 0;
var h;
// Channel 4, Analog In (Battery)
buf[i++] = 0x04;
buf[i++] = 0x02;
buf[i++] = (batt*100 >> 8) & 0xff;
buf[i++] = (batt*100) & 0xff;
// Channel 8, GPS
buf[i++] = 0x08;
buf[i++] = 0x88;
//Lat
h = lat * 10000;
buf[i++] = (h >> 16) & 0xff;
buf[i++] = (h >> 8) & 0xff;
buf[i++] = h & 0xff;
//Lon
h = lon * 10000;
buf[i++] = (h >> 16) & 0xff;
buf[i++] = (h >> 8) & 0xff;
buf[i++] = h & 0xff;
//alt
buf[i++]= (alt >> 16) & 0xff;
buf[i++] = (alt >> 8) & 0xff;
buf[i++] = alt & 0xff;@CurlyWurly what stupid am I, of course I reversed lat and lon, btw I also checked signed value and I was decoding it with readUInt32BE instead of readInt32BE
So now the final working code is here, I also placed my decoding routine as example
flow
http request config, just set to POST and URL
recode node
// Return message
var retmsg = {};
// Get raw payload buffer (binary)
var b = msg.payload_raw;
// Decode an uplink message from a buffer
lon = b.readInt32BE(5)/600000;
lat = b.readInt32BE(9)/600000;
accel = { x:((b.readInt8(16)<<8)*2)/32768,
y:((b.readInt8(17)<<8)*2)/32768,
z:((b.readInt8(18)<<8)*2)/32768 };
batt = (b.readUInt16LE(27)* 0.006406) ;
temp = (b[25]<<8 | b[26]) & 0x0FFF ;
if (temp > 0x7FF ) {
temp = -(temp&0x7FF) ;
}
temp = temp*0.0625;
// Size here is LPP, depends on what you send
var buf = Buffer.alloc(4+4+8);
// Channel 1, Analog In (Battery)
buf.writeUInt16BE(0x0102, 0);
buf.writeUInt16BE(batt*100, 2);
// Channel 2, Temp sensor
buf.writeInt16BE(0x0267, 4);
buf.writeInt16BE(temp*10, 6);
// Channel 3, Accel
buf.writeInt16BE(0x0371, 8);
buf.writeInt16BE(accel.x*1000, 10);
buf.writeInt16BE(accel.y*1000, 12);
buf.writeInt16BE(accel.z*1000, 14);
// Channel 9, GPS (if fixed)
if (lat!==0 || lon!==0) {
lon *= 10000;
lat *= 10000;
var bufgps = Buffer.alloc(11);
bufgps.writeUInt16BE(0x0988, 0);
bufgps.writeIntBE(lat, 2, 3);
bufgps.writeIntBE(lon, 5, 3);
bufgps.writeIntBE( 0 , 8, 3);
buf = Buffer.concat([buf, bufgps]);
}
// Save converted payload format to current msg payload_raw in base64
msg.payload_raw = buf.toString('Base64');
delete msg._msgid; // Not needed
// Get FULL current message (not only payload) as JSON,
retmsg.payload = JSON.stringify(msg);
return retmsg;
Big thanks to @johan and @CurlyWurly, now we can reencode for cayenneLPP even only for specific devices in the same app, excellent 
That’s not solved for me as you need to setup a whole node RED instance to perform that, and indeed you need to have some hardware running for that…
Re “How to convert custom payload in TTN console”, Johan answered your original question before
“In V2 we don’t allow altering payload_raw. It will, technically, be supported in V3 but not sure if it will be GA in the public Consoles”
If you have a need to know about cloud solution options, then that is another thread title really
I changed the topic title, correct it’s not done in the console but my first need are done, it works, not the easiest way I agree, but it works. You do not need node red instance, you can do it with any SDK Api and using nodejs should be easy since majority of the code is already written, but yes you need a computer/server for that, they are plenty of autonomous linux box such as PI 3/Zero that can do the job fine for some bucks or using a corporate existing server is also an option.
Oh, just thought I was the thread initiator and it’s you, sorry about that, you can revert back topic title.
I reverted the change in the title (again), as the reference to TTN Console and the answers of Johan are quite useful for future reference, I feel. (Of course, all the workarounds are great too.)