Best practices when sending GPS location data [HowTo]

Application Development
8

As requested on slack I will repeat my statement about SF12 here as well.

Moving nodes should switch off Adaptive Data Rate (or do they?), and always send with the slowest rate (SF12), and hence are bound to a maximum application payload length of 51 bytes2?

As you are developing a product for a moving object, it might not be a good option to use SF12. It is true SF12 can cover more distance but this has to come from somewhere. The thing with SF12 is, every symbol is coded with 4096 bits (also called chips with spread spectrum modulation). This results in a coding gain which can be received under the noise level, thus enabling greater distances to be covered. However this causes the package and airtime to be extremely long. Even with a few bytes it will be very sensitive for interference and as a moving object, it is likely to come across interfering objects.

SF11 would already be much better because the airtime is almost half of SF12 thus less chance for package loss due interference. However ideally SF7 would be used because this has the shortest air time. In addition SF7 uses less energy because of the shorter air time and of course you are able to send packages more frequently.

I actually still have to measure the difference in distance between SF7 and SF12. I am planning to do this within the next two weeks or so. I will share my findings here. If SF12 truly generates a significantly larger distance I would suggest that a moving node would first try to use SF7 and if that fails, it would then try a higher SF.

4 Likes
9

I have done some range testing by driving a node around in a car. Most transmissions using SF12 were not received while in motion. Transmissions while stationary (parked) were received reliably. I haven’t found the time to drive the same circuit using SF7 yet.
Transmissions with SF12 while walking were no problem.

3 Likes
10

I’m sorry, but I have to interject and say that is a genius idea, I was about to post another idea but that probably beats it hands down. I’ll put it here anyway.

In the UK they have an alternative geo system called the National Grid, it was used quite a lot in Government. It features 2 sets of numbers known as Eastings and Northings,
This grid sets the bottom left hand corner of the British Isles (a rock off the Scilly Isles) as 0,0. Go 10 metres to the East and 50 metres north and you are at 10,50. And so on till you get to the top of Scotland.

You could create a similar grid for a 40km area and map lat/lng to East/North. This then assumes your node is smart enough to do the donkey work itself, and you take into account the battery use this will cause.

Which brings me onto another question, seeing as I’m in “application development” anyway.

If we stick with the “pet lost locally” idea, what if you do not want to hear anything UNTIL the pet is lost, if the uplink message is just an hourly checkin (ping) will we be able to send back to the node a signal which can then be interpreted as “OK, wake the GPS up, now I want to know your exact location”. I mean, have I understood how LoRa in TTN will operate? (I have no kit, awaiting my kickstarter kit).

Thanks

11

Yes; after each uplink you can send data back to the node. And when only sending a ping once per hour and only responding when the pet is lost, that seems reasonable.

But a pet can run quite far in an hour, so you’d have to do some calculations about how often you can send such ping, or implement some geofencing to only send pings when not at home or when not close to the pet owner’s Bluetooth-enabled mobile phone. Also beware that both the uplink and the downlink might not be received (especially for a moving node), so the application might need to repeat the downlink if it still doesn’t get location data after it sent a downlink earlier. So one ping per hour might really be too few for this use case.

As an aside: for technical reasons, using downlinks will really need to be limited. The TTN Fair Access Policy might only allow 10 confirmed uplinks and downlinks per day (and other operators should impose such limits as well; LoRaWAN is just not suited for downlinks very well). But I guess pets don’t get lost that often, so most pings would not need a response.

See also Is anyone already working on the PetFinder use case?

12

…but beware that atmospheric circumstances can yield unexpected long ranges!

See the part about “Ducting”, most often happening at night and possibly yielding ranges of 200 km+, in @nmap’s very, very nice TTN webcast, Crash Course in Radio Theory.

13

Just for future reference, here is the c code for packing the GPS data into 48 bits.

uint8_t coords[6]; // 6*8 bits = 48
int32_t lat = float_latitude * 10000;
int32_t lon = float_longitude * 10000;

// Pad 2 int32_t to 6 8uint_t, skipping the last byte (x >> 24)
coords[0] = lat;
coords[1] = lat >> 8;
coords[2] = lat >> 16;

coords[3] = lon;
coords[4] = lon >> 8;
coords[5] = lon >> 16;

LMIC_setTxData2(1, coords, sizeof(coords), 0);

And here is some example PHP-code to decode it back into floats:

// Unpack into 6 bytes      
$data = unpack('C*', $payload);

// Combine each 3 bytes into a 24bit int      
$lat = ($data[1] + ($data[2] << 8) + ($data[3] << 16)) / 10000;      
$lon = ($data[4] + ($data[5] << 8) + ($data[6] << 16)) / 10000;      
var_dump( $lat, $lon );
4 Likes
14

…or to decode using TTN’s payload functions, to get an additional human readable fields in the MQTT payload:

See below for an improved version to support negative decimal coordinates.

// Version 2 (production) needs a function name; see post below for a fix
function (bytes) {
  // Does NOT handle negative values; see post below for a fix
  var lat = (bytes[0] + (bytes[1] << 8) + (bytes[2] << 16)) / 10000;
  var lng = (bytes[3] + (bytes[4] << 8) + (bytes[5] << 16)) / 10000;
  return {
    location: {
      lat: lat,
      lng: lng
    },
    love: "TTN payload functions"
  };
}

…to return something like the following in the MQTT message:

{
  "app_id": "my-app-id",
  "dev_id": "my-node",
  "hardware_serial": "42CF4601045CE277",
  "port": 1,
  "counter": 19,
  "payload_raw": "ANX6nBIX",
  "payload_fields": {
    "location": {
      "lat": -33.8688,
      "lng": 151.2092
    },
    "love": "TTN payload functions"
  },
  "metadata": {
    "time": "2017-01-16T22:59:20.373495512Z",
    "frequency": 868.1,
    "modulation": "LORA",
    "data_rate": "SF8BW125",
    "coding_rate": "4/5",
    "gateways": [
      {
        "gtw_id": "eui-5ccf7ff42f1970ec",
        "timestamp": 494534657,
        "time": "2017-01-16T22:59:20.296723Z",
        "channel": 0,
        "rssi": -46,
        "snr": 11,
        "latitude": 52.373141,
        "longitude": 4.892435,
        "altitude": 3
      }
    ]
  }
}
2 Likes
15

Even better! :slight_smile:

16

I think it is a very interesting approach, but it won’t work with negative integers… so forget about any coordinate to the west of Greenwich or to the South of the Ecuador… any suggestion to fix it?

1 Like
17

same problem

18

You could add 200 and then subtract them on the server/TTN side… that way you would always treat with positive numbers!

2 Likes
19

Great, we seem to have come to a good solution to transfer coördinates. Is there any to automatically get this data on a coverage map? Say on pade.nl/lora. Is there an application that we can assign our trackers to?

20

Check TTNmapper.org

21

When sending 3 bytes, we’re basically not sending the 4th byte of each integer, which should be 0xFF for negative numbers. Like for New York (40.712784, -74.005941, which would be sent as integers 407127 and -740059) we would send:

When decoding these 6 bytes back into two 32 bits signed integers, we need to compute the missing 4th and 8th bytes ourselves, to make JavaScript properly convert the negative values for us. Those bytes should be 0xFF if the most significant bytes have their “high bit” set, which is called “sign extending”:

// LSB, Least Significant Bit/Byte first
// Sign-extend the 3rd and 6th bytes into a 4th and 8th byte:
lat = (b[0] | b[1]<<8 | b[2]<<16 | (b[2] & 0x80 ? 0xFF<<24 : 0)) / 10000;
lng = (b[3] | b[4]<<8 | b[5]<<16 | (b[5] & 0x80 ? 0xFF<<24 : 0)) / 10000;

// MSB, Most Significant Bit/Byte first
// Sign-extend the 1st and 4th bytes into leading bytes:
lat = ((b[0] & 0x80 ? 0xFF<<24 : 0) | b[0]<<16 | b[1]<<8 | b[2]) / 10000;
lng = ((b[3] & 0x80 ? 0xFF<<24 : 0) | b[3]<<16 | b[4]<<8 | b[5]) / 10000;

Alternatively, shift the most significant byte 8 bits too far to the left, and then shift it back, which will do the sign extension on the fly, as the bitwise operator >> is the sign-propagating right shift:

lat = (b[0]<<24>>8 | b[1]<<8 | b[2]) / 10000;
lng = (b[3]<<24>>8 | b[4]<<8 | b[5]) / 10000;

Meanwhile, for the new production environment, payload functions should also include the function name, Decoder. So, to support 6 byte coordinates with possible negative values:

function Decoder(b, port) {

  // Amsterdam: 52.3731, 4.8924 = MSB 07FDD3 00BF1C, LSB D3FD07 1CBF00
  // La Paz: -16.4896, -68.1192 = MSB FD7BE0 F59B18, LSB E07BFD 189BF5
  // New York: 40.7127, -74.0059 = MSB 063657 F4B525, LSB 573606 25B5F4
  // Sidney: -33.8688, 151.2092 = MSB FAD500 17129C, LSB 00D5FA 9C1217

  // LSB, Least Significant Bit/Byte first! Your node likely sends MSB instead.

  // Sign-extend the 3rd and 6th bytes into a 4th and 8th byte:
  var lat = (b[0] | b[1]<<8 | b[2]<<16 | (b[2] & 0x80 ? 0xFF<<24 : 0)) / 10000;
  var lng = (b[3] | b[4]<<8 | b[5]<<16 | (b[5] & 0x80 ? 0xFF<<24 : 0)) / 10000;

  return {
    location: {
      lat: lat,
      lng: lng
    },
    love: "TTN payload functions"
  };
}

Alternatively, as coordinates in decimal degrees are -90…+90 for latitude, and -180…+180 for longitude: one could add 90 to the latitude and 180 to the longitude before sending, then send the positive values, and reverse that in the payload function.

And life can be made easy using libraries such as https://github.com/thesolarnomad/lora-serialization (though that one does not support 3 byte coordinates).

6 Likes
22

Look like the MGRS or the Maidenhead Locator System :wink:

1 Like
23

The AIS standard use 28 bits for longitude and 27 bits for latitude to report the position of a ship without loosing accuracy.
Could be packed in 7 bytes, giving one more bit to code something else.

TDOA could give us a approximative location, (or even the position of the base station itself), so the offset could be relative to the nearest square in a reference grid.

Example:
We are in the 4QFJ 1 6 MGRS square (precision level 10 km).
Exactly at location 4QFJ 12345 67890 (precision level 1 m).
So we transmit only 2345 7890, 0—9999 (dec): we need only 14 bits X 2
If we choose to have 12 bits X 2, the precision is near 2.5 meters (10000/2^12)

2 Likes
24

For information, see Low Power Payload (LPP) that allows the device to send multiple sensor data at one time.

GPS location is 9 bytes long:
3 bytes for Latitude: 0.0001 ° Signed MSB
3 bytes for Longitude: 0.0001 ° Signed MSB
3 bytes for Altitude: 0.01 meter Signed MSB

25

3 bytes for altitude is not best practices in my book. It also needs two additional bytes to indicate the “channel” and what the field type is. But indeed, TTN supports this as an integration as well.

1 Like
27

I think all GPS position reports must contain position error information. It is very important that the system receiving the position report have an understanding of how accurate the position report is.

2 Likes
28

Another idea:

If you need more frequent readings record a single GPS position set, then record at a higher frequency timestamp and position deltas. Only upload the message when it is full. Convert the delta’s back to absolute references in the incoming conversion function. (No need to hold any additional state in the Platform).

1 Like