Which low cost gateway has GPS timestamp?

LoRaTracker · October 25, 2019, 10:55pm

Exactly.

You dont need a UART to set the timing from the 1PPS signal.

The so called ‘time’ from the GPS and read by the UART can only really tell you what second you are in.

Verkehrsrot · October 26, 2019, 7:20am

I know that. But i assume the RAK7258 runs the original Semtech Packet Forwarder. I looked in the source code on github. It works by grabbing the timestamp from NMEA data on serial port. Does not use systime.

bsiege · October 26, 2019, 8:13am

As far as i understand from taking a quick look on specs and some forwarder sources:

Concentrator has own internal clock, set via gateway over SPI. But i cannot find the logic for enabling the internal timestamping of concentrator. I assume you activated NTP already and therefore the gateway should/could set the internal clock of concentrator. From this moment i would expect timestamps, even if they are not in the needed quality. I cannot believe that concentrator has any mechanisms built in to validate the given time from gateway. Only thing that i can imagine that some logic inside concentrator excepts PPS mandatory for adding HW-Timestamps.

Verkehrsrot · October 26, 2019, 9:11am

The gateway has ntp synched systime, i checked this via console.
And it does fill in the “time” field in the cyclic statistc uplinks, but not the field “time” in packet uplinks.

Source code shows, why:

The timestamp for the statistic uplink is generated from systime:

github.com

Lora-net/packet_forwarder/blob/master/lora_pkt_fwd/src/lora_pkt_fwd.c#L1252


sigaction(SIGQUIT, &sigact, NULL); /* Ctrl-\ */
sigaction(SIGINT, &sigact, NULL); /* Ctrl-C */
sigaction(SIGTERM, &sigact, NULL); /* default "kill" command */


/* main loop task : statistics collection */
while (!exit_sig && !quit_sig) {
    /* wait for next reporting interval */
    wait_ms(1000 * stat_interval);


    /* get timestamp for statistics */
    t = time(NULL);
    strftime(stat_timestamp, sizeof stat_timestamp, "%F %T %Z", gmtime(&t));


    /* access upstream statistics, copy and reset them */
    pthread_mutex_lock(&mx_meas_up);
    cp_nb_rx_rcv       = meas_nb_rx_rcv;
    cp_nb_rx_ok        = meas_nb_rx_ok;
    cp_nb_rx_bad       = meas_nb_rx_bad;
    cp_nb_rx_nocrc     = meas_nb_rx_nocrc;
    cp_up_pkt_fwd      = meas_up_pkt_fwd;
    cp_up_network_byte = meas_up_network_byte;

But the field “time” for uplink packet json data ist generated using a gps time reference, grabbed from a struct time_reference_gps.

github.com

Lora-net/packet_forwarder/blob/master/lora_pkt_fwd/src/lora_pkt_fwd.c#L1612


if (j > 0) {
    buff_index += j;
} else {
    MSG("ERROR: [up] snprintf failed line %u\n", (__LINE__ - 4));
    exit(EXIT_FAILURE);
}


/* Packet RX time (GPS based), 37 useful chars */
if (ref_ok == true) {
    /* convert packet timestamp to UTC absolute time */
    j = lgw_cnt2utc(local_ref, p->count_us, &pkt_utc_time);
    if (j == LGW_GPS_SUCCESS) {
        /* split the UNIX timestamp to its calendar components */
        x = gmtime(&(pkt_utc_time.tv_sec));
        j = snprintf((char *)(buff_up + buff_index), TX_BUFF_SIZE-buff_index, ",\"time\":\"%04i-%02i-%02iT%02i:%02i:%02i.%06liZ\"", (x->tm_year)+1900, (x->tm_mon)+1, x->tm_mday, x->tm_hour, x->tm_min, x->tm_sec, (pkt_utc_time.tv_nsec)/1000); /* ISO 8601 format */
        if (j > 0) {
            buff_index += j;
        } else {
            MSG("ERROR: [up] snprintf failed line %u\n", (__LINE__ - 4));
            exit(EXIT_FAILURE);
        }

time_refrence_gps is synched to gps time, triggered by gps pps, in a separate thread:

github.com

Lora-net/packet_forwarder/blob/master/lora_pkt_fwd/src/lora_pkt_fwd.c#L2683


    pthread_mutex_lock(&mx_concent);
    i = lgw_get_trigcnt(&trig_tstamp);
    pthread_mutex_unlock(&mx_concent);
    if (i != LGW_HAL_SUCCESS) {
        MSG("WARNING: [gps] failed to read concentrator timestamp\n");
        return;
    }


    /* try to update time reference with the new GPS time & timestamp */
    pthread_mutex_lock(&mx_timeref);
    i = lgw_gps_sync(&time_reference_gps, trig_tstamp, utc, gps_time);
    pthread_mutex_unlock(&mx_timeref);
    if (i != LGW_GPS_SUCCESS) {
        MSG("WARNING: [gps] GPS out of sync, keeping previous time reference\n");
    }
}


static void gps_process_coords(void) {
    /* position variable */
    struct coord_s coord;
    struct coord_s gpserr;

All gps functions used in this thread are coded in the HAL, and there they are hardwired to a TTY gps port.

github.com

RAKWireless/rak_common_for_gateway/blob/198571a96e9426598819cb2c4ffabebebdc48cbe/lora/rak7243/loragw_gps.c

/*
 / _____)             _              | |
( (____  _____ ____ _| |_ _____  ____| |__
 \____ \| ___ |    (_   _) ___ |/ ___)  _ \
 _____) ) ____| | | || |_| ____( (___| | | |
(______/|_____)_|_|_| \__)_____)\____)_| |_|
  (C)2013 Semtech-Cycleo

Description:
    Library of functions to manage a GNSS module (typically GPS) for accurate
    timestamping of packets and synchronisation of gateways.
    A limited set of module brands/models are supported.

License: Revised BSD License, see LICENSE.TXT file include in the project
Maintainer: Michael Coracin
*/


/* -------------------------------------------------------------------------- */
/* --- DEPENDANCIES --------------------------------------------------------- */

This file has been truncated. show original

So no way to switch this to NTP, we need a gps on a TTY port. Too bad that ttyS1 and ttyS2 are not usable with the kernel, as stated above. Would be very easy then.

So I am considering to kill the console and wire a gps to console port ttyS0… kamikaze

Verkehrsrot · October 26, 2019, 9:29am

How could i test ttyS1 by using the very limited command set of busybox on the RAK7258, could you point me to a way to do this?

bsiege · October 26, 2019, 10:03am

This seems only be a conversion according to readme 5.1. Concentrator vs Unix time synchronization

As in our case GPS is enabled (LGW_GPS_EN==1), we need to have a way to get the actual concentrator current time, at any time. For this, a new thread has been added to the packet forwarder (thread_timersync) which will regularly: - Disable GPS mode of SX1301 counter sampler - Get current Unix time - Get current SX1301 counter - Compute the offset between Unix and SX1301 clocks and store it

Unix time stays Unix time no matter what the reference is.

Verkehrsrot · October 26, 2019, 10:06am

That’s right, but the Semtech code - as far as i see - does not deliver unix time in uplink packet field, but last time grabbed from NMEA, when pps triggered. So this function will work only if a serial gps is connected, and is feeding NMEA data?

bsiege · October 26, 2019, 11:22am

Implementation seems to differ from their description.
I read:

We also need to convert a SX1301 counter value to GPS UTC time when we receive an uplink, in order to fill the “time” field of JSON “rxpk” structure.>

That is a “simple” Unit conversion.

Verkehrsrot · October 26, 2019, 12:02pm

I checked the global_conf.json, which is generated by the Web UI on the gateway’s file system. Interestingly it contains

“gps_tty_path”: “/dev/ttyS1”

I will now try to connect a 3,3V GPS module to ttyS1 and see what happens.

Verkehrsrot · October 26, 2019, 1:48pm

Looks like it’s working:

JSON up: {"rxpk":[{"tmst":136481108,"time":"2019-10-26T13:47:57.677393Z","tmms":1256132895677,"chan":7,"rfch":1,"freq":868.500000,"stat":1,"modu":"LORA","datr":"SF7BW125","codr":"4/5","lsnr":7.8,"rssi":-79,"size":15,"data":"QIAuASaA4hIBXOCpJSCa"}]}

cslorabox · October 26, 2019, 5:01pm

Wow! Glad it worked.

When I tried to access ttyS1 and got error it must have been on a different build, possibly I only tried it on the first of a series of custom builds that would have still had default settings before I found the instructions on how to enable it.

It is logical that they’d enable it to support a GPS, glad to hear you found they actually did.

Verkehrsrot · October 26, 2019, 7:58pm

Here’s the cooking receipt to add GPS PPS synched time to the RAK7258:

I examined the pcb and found, that JP9 is not connected to the RAK634 module, thus it’s not ttyS1. I suppose it is wired to the Quectel EG95 GSM chip. I couldn’t verify this, because the pcb trace disappears under the footprint of the EG95.

I resolutely wired TX/RX of GPS module directly to RXD1 (pin 14) /TXD1 (pin 15) of RAK634, and grabbed 3V3/GND from JP9. Also wired PPS output of GPS to PPS input (pin 19) of RAK2754 module.

I used an old GPS module found in my stock. A “Global Top FGPMMOPA6C”, probably not the best buy, as it is has no backup battery. So it takes somes minutes after cold start to get time and PPS. Perhaps i will change this later with a Neo6M with battery backup.

It’s not necessary to change anything in the RAK’s configuration, since GPS port ttyS1 is already enabled in the global_conf.json found on the file system. Just disable the option “fake GPS” in Web UI. Then logon to console, enter logread -f and check the console output.

You should see repeated warnings, that GPS has no time:

Sat Oct 26 21:14:43 2019 user.info lora_pkt_fwd[1620]: WARNING: [gps] could not get GPS time from GPS

If you don’t see this warning, the GPS does not feed NMEA data. Check your wiring (RX/TX swapped?) and the boot configuration of the GPS module.

As soon as the GPS delivers time & pps, this warning disappears and the packet forwarder gets the time from GPS:

##### 2019-10-26 19:52:20 UTC #####
Sat Oct 26 21:52:20 2019 user.debug lora_pkt_fwd[1617]: ### [GPS] ###
Sat Oct 26 21:52:20 2019 user.debug lora_pkt_fwd[1617]: # Valid time reference (age: 1 sec)
Sat Oct 26 21:52:20 2019 user.debug lora_pkt_fwd[1617]: # GPS coordinates: latitude 52.XXXX, longitude 13.XXXX, altitude 102 m
Sat Oct 26 21:52:20 2019 user.debug lora_pkt_fwd[1617]: ##### END #####

After this modification you should see the field “time” in TTN console filled in by the gateway with precise PPS synched GPS time, also you see the location data.

"gateways": [
    {
      "gtw_id": "eui-60c5a8fffe76126a",
      "timestamp": 1380967060,
      "time": "2019-10-26T20:05:00.580895Z",
      "channel": 6,
      "rssi": -87,
      "snr": 9.3,
      "rf_chain": 1,
      "latitude": 52.XXXX,
      "longitude": 13.XXXX,
      "altitude": 104
    }

Verkehrsrot · October 26, 2019, 8:00pm

Pictures of my dirty hack:

2019-10-26%2020-39-54

IMG_20191026_203931

Verkehrsrot · October 26, 2019, 8:23pm

But now we see another issue: There is a delta of exactly 1 sec. between a MatchX 1701 and RAK7258 gateway. Both gateways have GPS/PPS.

"time": "2019-10-26T20:15:44.548435Z"
"time": "2019-10-26T20:15:45.548432Z"

Delta: 999,997 ms

Some time ago i saw the same effect with a Kerlink Wirnet gateway, it also was 1 sec. early.

Weird. Maybe a bug in the gps sync code of the semtech packet forwarder?

“gateways”: [
{
“gtw_id”: “eui-60c5a8fffe76126a”,
“timestamp”: 2040934931,
“time”: “2019-10-26T20:15:45.548432Z”,
“channel”: 1,
“rssi”: -77,
“snr”: 8.8,
“latitude”: 52.XXXX,
“longitude”: 13.XXXX,
“altitude”: 103
},
{
“gtw_id”: “eui-40d63cfffe032ed5”,
“timestamp”: 2182119899,
“time”: “2019-10-26T20:15:44.548435Z”,
“channel”: 4,
“rssi”: -50,
“snr”: 8.8,
“latitude”: 52.XXXX,
“longitude”: 13.XXXX,
“altitude”: 60
}

LoRaTracker · October 26, 2019, 8:33pm

The Ublox GPSs have a message you can poll to check if the GPSs value for leap seconds is correct, you can then tell if the time the GPS is showing is actually the correct UTC.

Not sure about the Global Tops,

Verkehrsrot · October 26, 2019, 8:43pm

But is this function supported by the code of semtech packet forwarder?

cslorabox · October 26, 2019, 9:41pm

If not it would be readily changed.

If you really want to avoid setting up to build code for the MT76x8 you could stick an MCU in between to fixup the GPS readings.

Verkehrsrot · October 26, 2019, 9:48pm

I don’t think the 1sec difference is a leap second issue. Then it would be more than 1 sec.

Verkehrsrot · October 26, 2019, 10:02pm

That makes no sense. I will change the GPS module, using a Ublox Neo 6M.

bsiege · October 26, 2019, 10:09pm

Well i think i had this in my early NTP days too:

Using the additional PPS feature of the NMEA driver is highly recommended for the GPS18x LVC: There is jitter of several milliseconds for the serial transmission, which makes synchronisation to this device very slow and sometimes even sluggish. The PPS-aware code of the NMEA driver after NTP-4.2.6p3-RC7 or NTP-4.2.7p72 can substitute the PPS time stamp for the serial receive time stamp when the difference between them is less than ±400ms. (This is not the full truth, but see below.) Since the delay between the end of the serial data and the PPS pulse is bigger than that, a fudge time2 of 500 to 600ms is needed to compensate for the delay. Otherwise the driver would either discard the PPS data or even lock to the next second pulse, which gets ntpd off by one second with micro-second accuracy. >

6.1.12.2. Garmin GPS18x LVC