Gateway timestamps in uplink metadata - how calculated?

Application Development
7

Below is what i get:

  1. “time” -> from TTN LoRaWAN server?
  2. “time” -> from gateways
  3. “timestamp” -> from gateways

If assume:

  1. is from TTN LoRaWAN server and is UTC time without leap secs.
  2. is from the gateways and is UTC time, calculated from gps time.
  3. is from the gateways and a monotic counter, in sync with LoRaWAN network

Ist this correct?

    {
        "topic": "test/devices/test/up",
        "payload": {
            "app_id": "test",
            "dev_id": "test",
            "hardware_serial": <deleted>",
            "port": 9,
            "counter": 0,
            "payload_raw": [
                1
            ],
            "payload_fields": {
                "timesync_seqno": 1
            },
            "metadata": {
                "time": "2019-03-10T18:25:18.591541462Z",
                "frequency": 867.9,
                "modulation": "LORA",
                "data_rate": "SF9BW125",
                "airtime": 164864000,
                "coding_rate": "4/5",
                "gateways": [
                    {
                        "gtw_id": "eui-12<deleted>",
                        "timestamp": 1733761908,
                        "time": "2019-03-10T18:25:17.555305Z",
                        "channel": 7,
                        "rssi": -49,
                        "snr": 11.5,
                        "rf_chain": 0,
                        "latitude": 52.<deleted>,
                        "longitude": 13.<deleted>,
                        "altitude": 60
                    },
                    {
                        "gtw_id": "eui-34<deleted>",
                        "timestamp": 550320884,
                        "time": "2019-03-10T18:25:18.393047Z",
                        "channel": 0,
                        "rssi": -59,
                        "snr": 13.25,
                        "rf_chain": 0,
                        "latitude": 52.<deleted>,
                        "longitude": 13.<deleted>,
                        "location_source": "registry"
                    }
                ]
            }
        },
        "qos": 0,
        "retain": false,
        "_msgid": "51e32a21.e32eb4"
    }
8

Partially. Check my answer above your message. Gateway time is just a wall time, could be GPS based. Could be NTP based, could be manually entered and totally wrong.

9

Yes, so it will be reliable only, if one has control over the gateway.

But at least it can be probed against current TTN servertime (which i assume as being reliable) in the same dataset, to estimate if it is a valid, recent, gateway time.

I tried to find the logic how the gateway list is sorted in the repo, but no chance without knowledge of the code structure. Can you point me?

10

As far as I know the order is that in which packets arrive at the back-end. However I never felt the need to check the code. You can take a look at the Go sources for V2 of the back-end at GitHub.

11

I found the v2 stack sources in github already, but where to look? tons of code…

12

A few dozen examples would likely yield insight.

  • if you see things that are not in signal strength order, it can’t be that…

  • if you see things that are not in gateway wall-clock order, it can’t be that…

Granted, you could see a pattern in a limited data set which turns out not to be true overall. But at the same time, it would be fair to ask “why do you care?”.

If it matters to you, and it is not explicitly clear in the specifications then you should probably sort the list yourself, to avoid any assumptions on behavior outside the specification on which it would be unsafe to rely, as it’s fair for unspecified behavior to change in a new version.

Incidentally, if a gateway based on a typical Linux has had a loss of Internet backhaul, and especially rebooted while under such loss, it is entirely possible when it gets back online for some packets to be reported with absurdly wrong wall clock times before the system gets an NTP fix and starts using accurate ones, unless it has specific logic to prevent that. Given that GPS can take time to lock, I could image a similar issue on a sufficiently cold start there as well.

3 Likes
13

Well, you’re right, i should parse the complete list and put my own secret sauce on it, instead relying on a unknown one.

14

Fun fact: https://www.thethingsnetwork.org/forum/t/can-one-gateway-be-listed-twice-in-the-ttn-metadata-gateways-array/12364

Including future roaming. :slight_smile:

15

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

17

Hi all,

I am trying to create an application where different LoRa devices are synchronised based on a time from TTN server.

For example lets say my GW gives me the bellow information:

{
  ...
  "metadata": {
    "time": "2020-01-15T14:41:55.369687496Z",
    "frequency": 867.3,
    "modulation": "LORA",
    "data_rate": "SF7BW125",
    "airtime": 71936000,
    "coding_rate": "4/5",
    "gateways": [
      {
        "gtw_id": "-----",
        "timestamp": 1880628275,
        "time": "2020-01-15T14:41:55.337396Z",
        ...
      }
    ]
  }
}

For instance, the time from metadata where is it coming from?
How is it calculated?

Best,
Haris

18

The time field in the metadata is generated by the backend when the (first instance of the) packet arrived from a gateway.

19

Do you have any feeling how much time this is taking? tens or hundreds of msecs?

20

If you’re trying to make the nodes transmit at the same time: don’t. That will create network issues. Instead, the node should always apply some randomness when it wants to transmit something.

If you’re trying to make the nodes know the current time, then remember that a downlink is always RX1 or RX2 seconds after the uplink, which can be used to synchronize clocks with a time you create yourself, rather than relying on some timestamp generated by some network server. But beware that a downlink that is scheduled after some uplink might not be transmitted right away, so might be transmitted after the next uplink, which you need to detect by including some counter. For all this, see Time synchronisation of a Node.

21

I am trying to synchronize the data sampling more or less and exploring the latter approach. Thanks I will check the other link.

22

A post was merged into an existing topic: Time synchronisation of a Node

23

We had some issues with buffering gateways, so this is what we found:

Usually you can evaluate at least 3 different times from a message:

  • metadata.gateways[0].time: This is the time, the message was received on the gateway
  • metadata.time: This is the time, the message was send to the backend
  • The current time when the message is received by the final application

Most of the time the differences on our applications are below 300 ms, mainly caused by the transfer time between gateway and internet, but we have seen delays up to 10 s when the backend was very busy.

So, given all devices are synchronized to an ntp, the gateway[].time is most accurate. Specially if the gateways does some buffering and retransfer (like the Kerlink CPF), it may be the only usefull time.

On some gateways it may happen, that the time stamp is zero (Nil) for some unknown reason. Then it may be useful to use the other times as a fallback.

24

Hi. I understand your point of view. Nevertheless, I don’t know how to set the right UTC region in order to manage that information coming in the payload. I am using a Pygate Gateway setting to AU915 but I am in Brazil. I am showing the date in to the payload on Node-red. For me, is interesting to see this date in the right format.

This is the Date as I am seeing in the Metadata payload: 2021-04-04T12:33:50.178819382Z. Right now is + 3 hours. Also my pointing router is brazil.thethings.network

25

It is not a point of view, it is the reality. How you set your PyGate to provide the right time and set the time zone is probably better asked at the pycom forum, they know the firmware and there will probably be more pygate owners over there.

26

Thank you.