TTN GATEWAY central

Yes, I have used TTN mapper with multiple TTN gateways without any problem.
I specify where the gateway is located via the console. Then the TTN mapper software on the phone makes points on the map showing where I am when successful transmissions occur, using my phone’s location services, as I ride around on a bicycle with the phone in one pocket and a TTN sensor Node in another. It even includes indications of what the received signal strength was.
You might enjoy the TTN Mapper author’s presentation at the conference if you haven’t seen it yet:

1 Like

If you want (at a future date) Class B you will need a gateway with GPS too. The GPS time (pulse) is a requirement to synchronize beacon transmissions. GPS also allows for time stamping required for TDoA location determination. So GPS is useful even when the gateway is stationary. However GPS needs to see the sky which is an issue with indoor gateway.

3 Likes

Hi all,

FWIW, I just activated my Things Gateway (backers edition) earlier today and ran into the issue with the Lora circuit board not being seated properly. This was indicated by the ‘Gateway Card: ND’ message on the weblog of the gateway, and the boot cycle mentioned earlier in this thread.

While the lora circuitboard was seated quite firmly (and under a bit of ension in fact), after pushing it in a bit more the gateway was finally able to register with TTN and get online. To be honest I couldn’t really feel the circuitboard move when pushing it in, only bending a bit :grimacing:… I hope it continues to work. So far so good anyway.

Before figuring this out I saw a lot of timeouts during the activation procedure as well as the message about packets being blocked.

Anyway, thanks for the posts that highlighted the issue and what to do about it :+1:

Received my Things Gateway last week, got it activated without issues, but it never was stable for more then a day(not on wifi, not on ethernet). Sometimes it worked again after unplugging/plugging the power cord. Sometimes that did not work(config correct: false), and I needed to do a full reset and activation again.
Eventually I enabled beta firmware updates, hoping that would make things better, but that never seem do do anything. It always was on the firmware from june last year every time I looked.

For the last hour I can’t get it to work at all. Did a full reset, activated again, but nothing helps. Also I just noticed that there is a new firmware installed dated today.
Gateway Information

Version Info	
Hardware:	
v1
Bootloader:	
r1-7167873a (2017-06-02T13:48:18Z)
Firmware:	
v1.0.1-facdef23 (2018-02-14T08:16:22Z)
 	
Network	
Uptime:	
1111
Connected:	
true
Interface:	
Ethernet
Wifi SSID:	
Things-Gateway-001EC03B3B76
 	
Configuration	
Activation locked:	
true
Config correct:	
true
Region:	
EU_863_870
Gateway Card:	
868Mhz
 	
Packet forwarding	
Broker connection:	
true
Packets up:	
61
Packets down:	
0
 	
Miscellaneous	
External storage:	
false

Seems all right to me, but “last seen” status in the console says otherwise(and also my nodes aren’t working).
What to do? Thanks in advance.

I guess the new firmware is also released in the “beta” update channel?
Can’t get it activated in any way since the new firmware.

Edit: and after a “downtime” of 2 hours, I pulled the power cord, and plugged it back in. And now the gateway works again.

Hi, my gateway also received an OTA update to firmware “v1.0.1-facdef23 (2018-02-14T08:16:22Z)”.However this still not fixes the reboot loop issue. What I am observing is that as soon as a packet is received by the gateway there is a high probability that it will “crash” and revert to the reboot loop. However, the firmware does improve things somewhat in the sense that after a (variable) number of iterations the gateway will often achieve a stable connection again (previously it would loop indefinitely).

Also, as I have a working gateway AND a malfunctioning gateway, I am wondering if the following difference is relevant: the socket for the LoRa modem of the working gateway has a number etched on it: 1635D4, whereas the socket for the LoRa modem of the malfunctioning gateway has a different number: 1636D4. Can anyone compare this with his own gateway?

See picture below.

ttn gw sockets

3 Likes

Hi Marcel,

My malfunctioning gateway has also number 1636D4.

1 Like

Confirm, my malfunction Gateway: 1636D4

1 Like

Got a malfunctioning gateway too however the socket has number 1635D4.

malfunction Gateway: 1636D4

Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298

Build time: Feb 14 2018 08:16:44

Reboot reason: 0x10

Firmware: 1.01
Socket part serial: 1636D4

Same reboot loop. 5 lights, 8 second cycle.

Log:

SNTP: State change from 0 to 0
SNTP: State change from 0 to 0



**************************
*   The Things Network   *
*      G A T E W A Y     *
**************************
Firmware name: AmazingAckermann, type: 0, version: 1.0.1, commit: facdef23, timestamp: 1518596182
Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298
Build time: Feb 14 2018 08:16:44
Reboot reason: 0x03
BOOT: (persisted info) 6F 72 72 65 01 03 BB FD 8E 4D E4 2D 9A F7 F2 DC 




WIFI: Entering state 0
WIFI: Entering SCAN state 0

MAIN: Initialisation complete
LORA: Changing state from 0 to 0

MAIN: Leaving state 0
MAIN: Entering state 1
FLASH: Magic bytes found: wifi config present
FLASH: Magic bytes found: activation data present
FLASH: Magic bytes not found: no stored FOTA data present
FLASH: Loading Firmware Data
CNFG: (Firmware HASH (sha256)) FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 
FLASH: Loading WiFi Data
CNFG: WiFi SSID:      ***
CNFG: WiFi key:       ***
CNFG: WiFi conn_type: 1
CNFG: WiFi sec_type:  4
FLASH: Loading Activation Data
CNFG: Gateway ID:         ***
CNFG: Gateway Key:        ***
CNFG: Account Server URL: https://account.thethingsnetwork.org
CNFG: Locked:             true
CNFG: Locked first time:  false

MAIN: Leaving state 1
MAIN: Entering state 2
INET: State change to 0
LORA: Initialisation complete
LORA: Changing state from 0 to 1
WIFI: Entering state 1
ETH: IP Address: 0.0.0.0 
WIFI: Entering state 4
WIFI: Entering SCAN state 1
Scan is completed successfully
WIFI: Entering SCAN state 2
WIFI: Entering SCAN state 5
WIFI: Entering SCAN state 0
WIFI: Entering state 2
WIFI: Disabling modules
Head magic match void: trying to free an already freed block, ignore
WIFI: Entering state 3
SNTP: State change from 0 to 1
WIFI: Enabling modules for client
WIFI: Entering state 6


>WIFI: IP Address: 0.0.0.0 
CB: INET: Gateway has WiFi
INET: State change to 2
INET: Connected to a network, waiting for DHCP lease, checking validity with ping
SNTP: State change from 1 to 2
WIFI: IP Address: 192.168.0.115 
LORA: Wait init complete, waiting for application.
LORA: Changing state from 1 to 2
INET: State change to 3
INET: Ping probe
INET: Error sending probe on Eth
INET: Ping response from MRF24WN, set as default
INET: State change to 4
SNTP: State change from 2 to 3
MON: SYS Stack size: 3959
MON: heap usage: 147KB (156KB), free: 192KB
SNTP: State change from 3 to 4
SNTP: State change from 4 to 5
SNTP: State change from 5 to 6
SNTP: State change from 6 to 1
INET: Initiated NTP request.
SNTP: State change from 1 to 2
MON: SYS Stack size: 3959
MON: heap usage: 147KB (156KB), free: 192KB
SNTP: State change from 2 to 3
SNTP: State change from 3 to 4
SNTP: State change from 4 to 5
SNTP: State change from 5 to 6
SNTP: State change from 6 to 7
INET: State change to 5

MAIN: Leaving state 2
MAIN: Entering state 3

CNFG: Load online user config state change to 4
HTTP: Close active socket 0
HTTP: Starting connection
HTTPS: Connection Opened: Starting TLS Negotiation
HTTP: Wait for TLS Connect
HTTP: TLS Connection Opened: Starting Clear Text Communication
HTTP: Got 1289 bytes
HTTP: Connection Closed
HTTP: Close active socket 1
CONF: Parsing response token: HTTP/1.1 200 OK
CONF: ROUTER URL: mqtts://bridge.eu.thethings.network:8883

CNFG: Load online user config state change to 6
FREQ: APP_URL_Buffer: https://account.thethingsnetwork.org/api/v2/frequency-plans/EU_863_870
HTTP: Starting connection
HTTPS: Connection Opened: Starting TLS Negotiation
HTTP: Wait for TLS Connect
HTTP: TLS Connection Opened: Starting Clear Text Communication
HTTP: Got 1232 bytes
MON: SYS Stack size: 2870
MON: heap usage: 227KB (233KB), free: 111KB
HTTP: Connection Closed
HTTP: Close active socket 1

CNFG: L

Dear all

Earlier this week, @arjanvanb developed a patch for the reboot problem some of you have encountered on the gateways. After further investigation we’re confident we found the root cause of the problem. In short: the Gateway microcontroller uses its internal clock instead of the external clock. This configuration results in unstable firmware.

You can find a more in-depth explanation below, but first, what does this mean for your gateway (i.e. if you have the issue) and how can you get your gateway running?

We are working on a new firmware version that will fix this. To get there we are working together with Microchip and TWTG on highest priority. This will take some time, and we want to do some extensive tests before we make the change available through the over-the-air firmware updates for your gateway.

The fix developed by @arjanvanb solves the reboot issues for many gateways, although it doesn’t fix the root cause. The patch fixes the reboot issue but unfortunately the gateway firmware remains running on the internal clock and thus in some cases out of spec. This patch is already available through our “beta” over-the-air firmware update channel.

At the same time, we’re working on a different patch, which will halve the baudrate to the LoRa module from 115200 to 57600. This will let the gateway work within spec, but effectively reduces the total bandwidth of the gateway (not noticeable for 99% of the users). We hope to make this patch available through our “beta” over-the-air firmware update channel in a few days.

Both of these two patches will bring us a lot closer to resolving the gateway reboots while we work with Microchip and TWTG on a permanent fix.

Click here to subscribe to the mailing list to be informed about gateway firmware updates.

You can enable beta updates from The Things Network Console in the settings of your gateway. Note that beta updates can be unstable. If your gateway is working fine, you’ll probably want to stick with stable firmware updates.

Some gateways reboot too often for an automatic update. In order to update the gateway via SD perform the following steps:

  • Download the most recent beta firmware files: firmware.hex and checksums
  • Power off the gateway.
  • Take a micro SD card and make sure it’s FAT32 formatted. You’ll only need a few MB, so most SD cards should be fine.
  • Create a directory on the SD-Card called: update
  • Copy the firmware.hex and checkums files to the SD-Card update folder
  • Safely eject the card from your computer and insert it into the Gateway
  • Power on the gateway, if the first led is blinking, then it is updating.

Note: These patches need to be put on an SD-card. After flashing, the SD-Card needs to remain in the Gateway, otherwise other firmware may be reinstalled from our over-the-air update channels. If you update through SD-card, make sure to keep the SD-card inside the gateway until the fixes are also made available on our over-the-air update channels. Please subscribe to the gateway newsletter to automatically receive updates on the topic and receive a notification when the over-the-air fix is public and available.

A technical recap of the issue for the more technical users:

After a period of investigation supported by the community, we may have found the root cause of the reboot problem of the gateway.
@arjanvanb proved that the reboots are related to incorrect data flow over UART between the PIC32 and LoRa module. The PIC32-LoRa module communication contains errors that lead to board reboots, this can be explained by an incorrect (factory) clock configuration. The gateway was supposed to be running from the external 24 MHz clock, but instead is is running from the internal 8 MHz oscillator. While the operational speed of the gateway is the same in both cases, the clock accuracy varies. This introduces communication errors when using in-band clocked protocols such as UART.

If you need additional assistance or if something does not go as planned, please refer to the issues in Issues · TheThingsProducts/gateway · GitHub

We’re working around the clock (pun intended) to fix issues such as these and push them as fast as possible to you guys. Nonetheless, not all can be planned and we need to triple-check if we’re fixing the correct conflicting parts within the firmware. We’re sorry for any inconvenience that this have may caused you.


The Things Products Team

5 Likes

Since you posted before TTP’s post above: how did you get that? (Did it fetch it itself, so: did your gateway run fine for some time?)

I’ve added a full log to Gateway loses network connection and needs manual reset to recover, often initiated by MQTT problems or by reboot for firmware upgrade · Issue #8 · TheThingsProducts/gateway · GitHub which, for me, seems to be related to failing to use MQTT while it actually does have an internet connection. It’s not in a reboot loop for that issue though.

Same here: 1635D4 for a mostly working gateway (well, when using updated firmware).

Now installing the new firmware :slight_smile:

Auto-update from the network after blanking the gateway (Power on with MODE switch pressed). Can get through setup and connecting to WiFi/Ethernet, but then will hit reboot loop when talking to the module and stay in it until power-cycle.

After a power-cycle, about one minute of connecting and handshaking before 5 lights and 8 second loop again.

Looks like a better fix is coming though, so happy to wait for that :slight_smile:

Bummer, same here: today’s firmware (installed using an SD card) gets me the original problem again (LORA: Starting reconfiguration is followed by LORA: Configuration failed, retry and eventually LORA: RESET MODULE and Reboot reason: 0x10.

As debug logging is not enabled, I guess I’m going to build a new version myself. But not tonight.

So the downloads get me the old reboot loop. However, when building myself from the develop branch with debug logging enabled then all is fine: https://github.com/TheThingsProducts/gateway/issues/1#issuecomment-365746884

And: building myself without changing anything gets me the same problems!? Is enabling debug logging actually solving the issue!?

I would imagine that enabling debug logging has a similar effect as the proposed beta patch (halving the baudrate) in effect slowing communication / execution down sufficiently to mitigate the timing issues with the internal clock?

1 Like

The firmware mentioned in this post also gives me the same issue as before … I have confirmed that gateway has been updated through SD card, is running fw version v1.0.1-facdef23 (2018-02-14T08:16:22Z), and I have disabled OTA updates.

I also have a connector with etched number 1636D4 if that matters.

UPDATE 1: after dozen of reboots during 30+ minutes, gateway finally connected to the broker for the first time. Its been connected for a few minutes now which is an improvement compared to before where it rebooted every minute before even connecting to the broker.

UPDATE 2: after half a day the gateway is again rebooting. Sigh.

I confirm that my Malfunctioning gateway has a 1636D4 socket; my gateway seem no long able to get past the power led solid with the second led flashing rapidly; and its not visible on the WiFi or wired network or as an AP?

Also tried flash new firmware without success :’(

it would explain why its so hard to track down!? sounds like a timing issue that is resolved by logging; a missing lock/mutex perhaps ? wonder if its worth faking a few on the task to try and identify the area ie

run with only the lora see it reboots
repeat above adding wifi
repeat above adding ethernet

you get the picture; @arjanvanb thanks for the info; just hope many gateway can be fixed :’(