Packet received time off by years (1999)

Gateways MultiTech Gateways
1

Hello folks.
A colleague notified me that packets received by my gateway (a MultiTech Conduit with FW 1.7.4) are showing as being received in 1999:
17 (Screenshot taken by my colleague)
When investigating the reason I couldn’t find anything being wrong on the gateway. It is synced to a couple of NTP servers and shows the right time. Also when monitoring the traffic from the gateway in the TTN console, the timestamps do seem plausible.
This is the second time this happened. The last time, I rebooted the gateway and the issue went away. This time after not finding anything, I took the chance to install the new 5.0.1 firmware, which also led to a reboot.
However the question remains why these packets showed times way off. Any ideas how this could have happened?

Addendum: Not sure whether this has anything to do with this, but I configured the forwarder to use the Conduit’s GPS with the lines:
“gps”: true,
“gps_tty_path”: “/dev/ttyXRUSB2”,
in /var/config/lora/local_conf.json. But the log file contains many messages like:
### [GPS] ###
# Invalid gps time reference (age: 1567791042 sec)
# no valid GPS coordinates available yet

2

I’m one step closer!
The MultiTech gateway has a partition /var/volatile which also contains the log files under /var/log (which is symlinked to /var/volatile/log). This partition was full, and as soon as I deleted the lora-pkt-fwd.log file and restarted the packet forwarder, the date problem went away.
However before deleting it the lora-pkt-fwd.log file was full of lines like:
22:08:02 WARNING: [gps] read() returned value 0
This probably blew up the log file until the partition was full.
So the next question is: how do I configure the Multitech’s GPS receiver correctly to work with TTN? After a reboot, the log file still contains messages like:
### [GPS] ###
# Invalid gps time reference (age: 1568130362 sec)
# no valid GPS coordinates available yet
Every now and then, these messages appear:
15:47:02 INFO: [TTN] ttn.opennetworkinfrastructure.org RTT 6
15:47:02 INFO: [TTN] send status success for ttn.opennetworkinfrastructure.org
15:47:32 INFO: Disabling GPS mode for concentrator’s counter…
15:47:32 INFO: host/sx1301 time offset=(1568129909s:417908µs) - drift=3132µs
15:47:32 INFO: Enabling GPS mode for concentrator’s counter.
I don’t know how to interpret these messages. And also not the “read() returned value 0” messages mentioned before. Can somebody provide me with some insight, please?

3

Are you sure the path specified for the GPS is right? To check, stop the packet forwarder and run ‘cat < /dev/…’ (specify the device you think should be used). If the device is correct you should see ascii NMEA messages,

4

I’ve set up things according to these instructions, where ttyXRUSB2 is mentioned. Also, on the Multitech Conduit there’s a symbolic link from /dev/gps0 to /dev/ttyXRUSB2. So I suppose the path should be right. I tried to run the tests you suggested, but they don’t seem to work:
admin@mtcdt:~$ sudo cat < /dev/ttyXRUSB0
-bash: /dev/ttyXRUSB0: Permission denied
admin@mtcdt:~$ sudo cat < /dev/ttyXRUSB1
-bash: /dev/ttyXRUSB1: Permission denied
admin@mtcdt:~$ sudo cat < /dev/ttyXRUSB2
-bash: /dev/ttyXRUSB2: Permission denied
admin@mtcdt:~$ sudo cat < /dev/ttyXRUSB3
-bash: /dev/ttyXRUSB3: Permission denied
admin@mtcdt:~$
The packet forwarder has been stopped before executing the tests, an I tried both with started and stopped gpsd.
Speaking of gpsd - it is enabled by default. Is it needed for the packet forwarder supporting TDOA timestamps or does it interfere with it?

5

GPSD is not required and might interfere. Try running the command after it has been stopped.

1 Like
6

OK, I stopped gpsd and tried again, with the same result at first. When trying a different command, namely “sudo minicom -b 9600 -D /dev/ttyXRUSB2”, I got the desired NMEA output. It did however not work with the cat < /dev/ttyXRUSBx commands.
So yes, /dev/ttyXRUSB2 appears to be the correct device name.
Here’s some more messages from the log:
07:50:59 INFO: GPS is disabled
[…]
07:50:59 INFO: GPS serial port path is configured to “/dev/ttyXRUSB2”
07:50:59 INFO: GPS is enabled
07:50:59 INFO: Using real GPS if available.
[…]
07:50:59 INFO: [main] TTY port /dev/ttyXRUSB2 open for GPS synchronization
[…]
07:51:03 INFO: Disabling GPS mode for concentrator’s counter…
07:51:03 INFO: host/sx1301 time offset=(1568188260s:609208µs) - drift=1211559096µs
07:51:03 INFO: Enabling GPS mode for concentrator’s counter.
[…]

And with every lora packet forwarded, I also get a message like:
Invalid gps time reference (age: 1568188294 sec)
no valid GPS coordinates available yet

The time of ~1.568E9 seconds equals to nearly 50 years. (Edit: obviously that’s the timespan since Jan 1st of 1970, ie the Unix time epoch).
Just an educated guess, but could the behavior have something to do with the GPS week rollover of 2019?

7

One more step closer (or maybe I’ve even found it):
Instead of manually stopping gpsd every time I rebooted the gateway, I now completely disabled it so that it does not start at all on boot.
To do that, I set the line ENABLED=“no” in /etc/default/gpsd
The packet forwarder now reports a successful GPS fix at every forward of a packet.

Since this step is not mentioned in the instructions linked above, I might have enabled gpsd by myself and forgot about it, or maybe some update messed things up.

I’ll keep an eye on the gateway and report again. Thanks for your help so far, @kersing!

1 Like
8

Your ‘sudo cat < /dev/tty…’ does not work because only the cat command is executed with root privileges. The shell still attempts to open the file using the user privileges because the redirect is not part of what is executed using sudo.
Easiest fix would be to use ‘sudo -i’ to become root and next execute ‘cat < /dev/tty…’.

1 Like