Payload formatted V3 (from V2)

Sorry to hijack this thread. I might be completely stupid, but I indeed haven’t found a button (or other way) to create a new topic (strange).

I am also trying to migrate a simple tracking application from v2 to v3. I am having strange problems with the payload formatter. Maybe you can help me.

This is the formatter in v2:

function Decoder(bytes, port) {
// Decode an uplink message from a buffer
// (array) of bytes to an object of fields.

var latitude;//gps latitude,units: °
  latitude=(bytes[0]<<24 | bytes[1]<<16 | bytes[2]<<8 | bytes[3])/1000000;//gps latitude,units: °

var longitude;
  longitude=(bytes[4]<<24 | bytes[5]<<16 | bytes[6]<<8 | bytes[7])/1000000;//gps longitude,units: °

var alarm=(bytes[8] & 0x40)?"TRUE":"FALSE";//Alarm status

var batV=(((bytes[8] & 0x3f) <<8) | bytes[9])/1000;//Battery,units:V
if((bytes[10] & 0xC0)==0x40)
{
  var motion_mode="Move";
}
else if((bytes[10] & 0xC0) ==0x80)
{
  motion_mode="Collide";
}
else if((bytes[10] & 0xC0) ==0xC0)
{
  motion_mode="User";
}
else
{
  motion_mode="Disable";
}                                            //mode of motion

var led_updown=(bytes[10] & 0x20)?"ON":"OFF";//LED status for position,uplink and downlink

var Firmware = 160+(bytes[10] & 0x1f);  // Firmware version; 5 bits 

var roll=(bytes[11]<<8 | bytes[12])/100;//roll,units: °
var pitch=(bytes[13]<<8 | bytes[14])/100; //pitch,units: °

var hdop = 0;
if(bytes[15] > 0)
{
   hdop =bytes[15]/100; //hdop,units: °
}
else
{
   hdop =bytes[15];
}

var altitude =(bytes[16]<<8 | bytes[17]) / 100; //Altitude,units: °

return {
Latitude: latitude,
Longitude: longitude,
Roll: roll,
Pitch:pitch,
BatV:batV,
ALARM_status:alarm,
MD:motion_mode,
LON:led_updown,
FW:Firmware,
HDOP:hdop,
Altitude:altitude,
};
}

This is the migrated formatter. I kept it as much as possible identical.

function decodeUplink(input) {
 
var bytes;
  bytes = input.bytes;

var port;
  port = input.fport;

var latitude;//gps latitude,units: °
  latitude=(bytes[0]<<24 | bytes[1]<<16 | bytes[2]<<8 | bytes[3])/1000000;//gps latitude,units: °

var longitude;
  longitude=(bytes[4]<<24 | bytes[5]<<16 | bytes[6]<<8 | bytes[7])/1000000;//gps longitude,units: °

var alarm=(bytes[8] & 0x40)?"TRUE":"FALSE";//Alarm status
var batV=(((bytes[8] & 0x3f) <<8) | bytes[9])/1000;//Battery,units:V

if((bytes[10] & 0xC0)==0x40)
{
  var motion_mode="Move";
}
else if((bytes[10] & 0xC0) ==0x80)
{
  motion_mode="Collide";
}
else if((bytes[10] & 0xC0) ==0xC0)
{
  motion_mode="User";
}
else
{
  motion_mode="Disable";
}                                            //mode of motion

var led_updown=(bytes[10] & 0x20)?"ON":"OFF";//LED status for position,uplink and downlink

var Firmware = 160+(bytes[10] & 0x1f);  // Firmware version; 5 bits 

var roll=(bytes[11]<<8 | bytes[12])/100;//roll,units: °
var pitch=(bytes[13]<<8 | bytes[14])/100; //pitch,units: °

var hdop = 0;
if(bytes[15] > 0)
{
   hdop =bytes[15]/100; //hdop,units: °
}
else
{
   hdop =bytes[15];
}

var altitude =(bytes[16]<<8 | bytes[17]) / 100; //Altitude,units: °

 return {
    data: {
      Latitude: latitude,
      Longitude: longitude,
      Roll: roll,
      Pitch:pitch,
      BatV:batV,
      ALARM_status:alarm,
      MD:motion_mode,
      LON:led_updown,
      FW:Firmware,
      HDOP:hdop,
      Altitude:altitude
    },
    warnings: [],
    errors: []
  };
}

If the payload is

02D4D6C000807E9C0F4164

the v2 result is:

{
  "ALARM_status": "FALSE",
  "Altitude": 0,
  "BatV": 3.905,
  "FW": 164,
  "LON": "ON",
  "Latitude": 47.50304,
  "Longitude": 8.42102,
  "MD": "Move",
  "Pitch": 0,
  "Roll": 0
}

In V3 I get for the same payload:

 "decoded_payload": {
    "ALARM_status": "FALSE",
    "BatV": 3.905,
    "FW": 164,
    "LON": "ON",
    "MD": "Disable",
    "Pitch": 0,
    "Roll": 0,
    "altitude": 0,
    "hdop": null,
    "latitude": 47.50304,
    "longitude": 0
  },

Any idea?

BTW, I also haven’t found a way in v3 to test the payload formatter. In v2 you can test it with a given payload and see what the result is. Here, the only way seems to be to have some devices sending data, or did I overlook this function somewhere?

1 Like

Welcome, but its not strange at all - we ask that new users (and Neuer Users :slight_smile: ) spend a little time reading around the forum and familiarising before being granted basic rights to start creating new threads and posts. With (at the time of checking) just 25 mins under your belt you havent yet got that access capability, though as you have done you can start to post against existing threads. This is done as a security and anti-spam measure as the forum is under constant thread of spammer abuse.

Anyhow, hopefully you will now see some community response and assistance. It may be this post gets moved to another thread/category going forward. :wink:

Was the lowercasing of altititude, latitude and longitude deliberate in your example? Presumably there was a reason for the change.

There is actually no change in the ocde. It is exactly the same as in the v2 decoder. I really copy/pasted the working decoder from the v2 to the v3 and just changed slightly the result and the input part.

But there is a difference in the result reported from the deocer. There indeed the name “Altitude” was changed to “altitude” as well as “Latitude” and “longitude”.

I’d happily drop both decoders in to my test rig, but the phrase “Any idea?” is a bit vague - I get the difference in the output field names, but before I spend any time on this, what are the exact details of your concerns, tell us what differences you see that you want ‘ideas’ on.

Hi Nick

Thanks for your offer and sorry that I wasn’t clear enough.

The differences I see with the same payload and (more or less) the same decoder code is the following:

  • MD: v2 = “Move”, v3 =“Disable”
  • Longitude: v2 = “8.42102”, v3 = “0”
  • Altitude, Longitude, Latitude: v2 = uppercase first letter, v3 = lowercase first letter (this is less of a problem, though)

The first two points make the results unusable. As the code is really exactly the same (except a small adaptation of the different input and output formats), I am at the end of my understanding why this gives a different result. Maybe it is just a stupid mistake on my side, but I just don’t see it.

1 Like

You can’t test it on the decoder entry page, but you can save it and then simulate an uplink.

Pasting the decoder in to my test page it works and I tried it in v3 and it works OK. See:

Hi Nick

Thanks for your support. I tested the “simulate uplink”. When I send the above mentioned payload, I get the following result:

  "name": "as.up.data.forward",
  "time": "2021-04-12T20:35:40.601528548Z",
  "identifiers": [
    {
      "device_ids": {
        "device_id": "tracker-lgt92-01",
        "application_ids": {
          "application_id": "position-tracking-01"
        },
        "dev_eui": "A8404149C182AF65",
        "join_eui": "70B3D57ED00319C3"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.ApplicationUp",
    "end_device_ids": {
      "device_id": "tracker-lgt92-01",
      "application_ids": {
        "application_id": "position-tracking-01"
      },
      "dev_eui": "A8404149C182AF65",
      "join_eui": "70B3D57ED00319C3"
    },
    "correlation_ids": [
      "as:up:01F33WR1XP18BRBWC20CRY6339",
      "rpc:/ttn.lorawan.v3.AppAs/SimulateUplink:b1d4baa4-8dd3-4b38-8beb-61c05df985a1"
    ],
    "received_at": "2021-04-12T20:35:40.599393745Z",
    "uplink_message": {
      "f_port": 2,
      "frm_payload": "AtTWwACAfpwPQWQ=",
      "decoded_payload": {
        "ALARM_status": "FALSE",
        "BatV": 3.905,
        "FW": 164,
        "LON": "ON",
        "MD": "Disable",
        "Pitch": 0,
        "Roll": 0,
        "altitude": 0,
        "hdop": null,
        "latitude": 47.50304,
        "longitude": 0
      },
      "settings": {
        "data_rate": {}
      },
      "received_at": "0001-01-01T00:00:00Z"
    },
    "simulated": true
  },
  "correlation_ids": [
    "as:up:01F33WR1XP18BRBWC20CRY6339",
    "rpc:/ttn.lorawan.v3.AppAs/SimulateUplink:b1d4baa4-8dd3-4b38-8beb-61c05df985a1"
  ],
  "origin": "ip-10-100-14-162.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_APPLICATION_TRAFFIC_READ"
    ]
  },

As you can see the reported “decoded payload” is wrong again (same issues as mentioned before).

So, you verified that the decoder is working correctly, but it doesn’t seem to do that when data is uploaded by the device? Or do I misunderstand that?

This is what I get:

{
  "name": "as.up.data.forward",
  "time": "2021-04-12T19:51:41.192904897Z",
  "identifiers": [
    {
      "device_ids": {
        "device_id": "descartes-v3-device-abp-v3-stack3",
        "application_ids": {
          "application_id": "descartes-test01v3"
        },
        "dev_eui": "DE5CA72FFFFFFFFF"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.ApplicationUp",
    "end_device_ids": {
      "device_id": "descartes-v3-device-abp-v3-stack3",
      "application_ids": {
        "application_id": "descartes-test01v3"
      },
      "dev_eui": "DE5CA72FFFFFFFFF"
    },
    "correlation_ids": [
      "as:up:01F33T7GBWVYXH1VQX96RSS2TS",
      "rpc:/ttn.lorawan.v3.AppAs/SimulateUplink:4efe3b4a-b7ef-4517-a049-a4a2e23b2187"
    ],
    "received_at": "2021-04-12T19:51:41.180702781Z",
    "uplink_message": {
      "f_port": 1,
      "frm_payload": "AtTWwACAfpwPQWQ=",
      "decoded_payload": {
        "ALARM_status": "FALSE",
        "Altitude": 0,
        "BatV": 3.905,
        "FW": 164,
        "HDOP": null,
        "LON": "ON",
        "Latitude": 47.50304,
        "Longitude": 8.42102,
        "MD": "Move",
        "Pitch": 0,
        "Roll": 0
      },
      "settings": {
        "data_rate": {}
      },
      "received_at": "0001-01-01T00:00:00Z"
    },
    "simulated": true
  },
  "correlation_ids": [
    "as:up:01F33T7GBWVYXH1VQX96RSS2TS",
    "rpc:/ttn.lorawan.v3.AppAs/SimulateUplink:4efe3b4a-b7ef-4517-a049-a4a2e23b2187"
  ],
  "origin": "ip-10-100-14-162.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_APPLICATION_TRAFFIC_READ"
    ]
  },
  "unique_id": "01F33T7GC8XPD29DQ2DM9JM2EE"
}

As a total nerd, I did beautify / lint the javascript that’s on the Gist - you may want to try copynpasta from there just in case there some stray character messing up the processing.

I seem to be doing something completely wrong. No idea, what, however…

Ok, so I copy/pasted the complete "function decodeUplink(input) " from your gist and replaced my uplink payload formatter. Then I took the payload “02D4D6C000807E9C0F4164”, pasted this in the simulate uplink, set the fport to 2 and the result is still the wrong one I pasted in my post before???

Where is the difference? What did you do differently???

Are we both working on the same server? I am on “eu1”.

One other idea: I use the payload formatter for the application, not the device specific one. Did you also use the application formatter, or did you use the device payload formatter?

It seems I found the issue!!! It works correctly, if I use the payload formatter for the device, but not when I use the application payload formatter!!!

Now the question is: Is this working as expected or is this a bug? I would expect the same result, if the same payload decoder is used on the application level (with the device set to “use application payload formatter”).

I’m on EU1 and I was running it at device level so I didn’t need to change an app, but I’ve tried it at application level and it’s working OK for me.

Despite not having any code related to the port, I’ve also tried port 2, again, working here.

It’s an issue, it should work the same at both application & device level.

But you need to be 110% sure you’ve flushed out anything from the entry box for application, just in case, before we go to bug status :scream:

Now it becomes weird. If I switch back to using the application payload formatter, I also get a correct result. Even though I did not change nor edit the application payload formatter at all?!

If I would not have a history of the results on my live data screen, I would believe I am crazy.

We both are, why are we coding at 22:20 (BST) ???

1 Like

Haha, you are absolutely right (it’s even 23:20 here).

But it is really strange. I tested it so many times, always with the wrong result. I switched back to v2 to compare. And only after I enabled the device specific payload formatter (using the copy/pasted code), it started to work correctly, even if I switch the device formatter off again (back to the original state of “using the application formatter”.

Well, whatever. Thanks a lot for your help, Nick! Have a good night!

Michael