The gateway tells the server about its relative time, "timestamp": 3928847795 in the log above. Next, TTN tells the gateway when to send the downlink, by giving a new relative time. For a regular downlink in RX1, it would add 1,000,000 microseconds and tell the gateway to transmit at 3,929,847,795. Likewise, it would add 2 seconds for RX2. For a Join Accept, it would add 5 seconds for RX1, or 6 for RX2.
So, the gateway needs an accurate clock as well. (It does not need to know the real time, a.k.a. wall time, but still needs an accurate clock.) You could compare the values of "timestamp" against the time between two uplinks. (Like if some uplinks are 20 seconds apart, then their timestamps should show a 20,000,000 microseconds difference.)
That’s why a debug log of the gateway will be very helpful, to tell what commands the gateway gets.