[solved] TTN Gateway has become unreliable

patmolloy · September 10, 2019, 3:44pm

I have one of the Kickstarter TTN gateways which drops off the network every few days. A power cycle brings it back on line reliably. I have several other (different tech) gateways co-located with the TTN gateway and they all stay connected. So it’s not local networking issues or internet connections issues (unless the TTN gateway is less tolerant somehow to possible transient issues?)

How can I troubleshoot this?

Interestingly I was just out at a customer meeting earlier, where they are experiencing the exact same problem with a TTN gateway.

Is this a known issue?

BoRRoZ · September 10, 2019, 4:11pm

I have one too but I don’t have this problem.
So I checked for you

TTN GATEWAY central 3

but it doesn’t seem to be a 'known common issue with all kickstarter gw’s

Jeff-UK · September 10, 2019, 7:07pm

WiFi or Wireline backhaul connectIon? If latter try swapping other GW’s with the network port used by TTN GW… I had a problem last summer (heat related I suspect) where a cheap 8 port 10/100hub on test bench had 2 flakey ports that were intermittent…2 of my gw’s under test proved unstable/intermittent until I found the problem - initally thought it was a poss bad cable (you’ve swapped that right? ) and only when plugged cable into wrong port by mistake did I find the issue!

jerylcook · September 10, 2019, 7:12pm

Not sure if this is your issue. but i had the same problem with my Rasbery PI Gateway.
I ended up doing a cron job restart ever morning, and purcahsed a USB wifi instead of using the onboard.

BoRRoZ · September 10, 2019, 7:36pm

absolutely NOT related - its not an RPI

BoRRoZ · September 10, 2019, 7:38pm

could it be the card connector inside ?

heat/cold/heat/cold … if its a bad or loose connection.
remove and then place back the card (push in firmly in the connector.

11%20PM

patmolloy · September 10, 2019, 9:08pm

Thanks! I will try various things … It’s a wired backaul, but maybe on a flakey port and/or flakey cable. I actually have the other gateway they sent too (because of delays on the KS they doubled the rewards - at least for me!) so I could possibly swap out. It was super reliable for a long time, and the fact that this customer today is seeing the same issue just made me stop and wonder whether some software push had happened to make them unhappy. Will report back … thanks for all the suggestions and help!

patmolloy · September 11, 2019, 9:04am

Ok, changed ethernet cable and now plugged into a different switch. It came up fine after a reboot last night, but seems to have stayed online only 3 or 4 hours. I’ll try the card connector tonight!

All of the other gateways are still fine. I might try changing the backhaul to WiFi too … maybe there is some going on with the Ethernet end of things??

Or maybe it has just become generally unhappy for some reason - possibly replace with the spare I have, but reluctant as I’d like to get to the bottom of this really!

BoRRoZ · September 11, 2019, 9:13am

my hifi amplifier stopped suddenly some time ago and of course I teared it apart.
Took me a long time, but I finally found a capacitor with a defect, but it was not defect all the time, only after some time I used the amp and things started heating up.
Just to say… it could be just one part in the GW base board/lora card , difficult to debug.

Indeed make sure you’re connections at that location are 100% (maybe test with another GW)
open the enclosure and use a hairdryer to heat the board/components a bit … see if that does something ?

patmolloy · September 14, 2019, 4:41pm

Now even more bizarre. Tried all suggestions. Gateway would come up and then go down. So I decided to try the other (unused gateway) and start from scratch. This one will not connect at all. I have tried latest stable and beta firmware. This is what is coming out of the debug port … can anyone tell me what the problem is? The first two leds are solid, the third slowly flashing. This goes on for a while until an automatic reboot and then into the same state.

> **************************
> *   The Things Network   *
> *      G A T E W A Y     *
> **************************
> Firmware name: AmazingAckermann, type: 0, version: 1.0.7, commit: e9d35a30, timestamp: 1560942019
> Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298
> Build time: Jun 19 2019 11:01:21
> Reboot reason: 0x40
> BOOT: (persisted info) 6F 72 72 65 01 03 AE 62 75 77 BF F5 14 7C 3C 10
> 
> 
> 
> 
> WIFI: Entering state 0
> WIFI: Entering SCAN state 0
> 
> MAIN: Initialisation complete
> LORA: Changing state from 0 to 0
> 
> MAIN: Leaving state 0
> MAIN: Entering state 1
> FLASH: Magic bytes found: wifi config present
> FLASH: Magic bytes found: activation data present
> FLASH: Magic bytes not found: no stored FOTA data present
> FLASH: Loading Firmware Data
> CNFG: (Firmware HASH (sha256)) FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
> FLASH: Loading WiFi Data
> CNFG: WiFi SSID:      M2M-UNIFI-UAP-AC-M
> CNFG: WiFi key:       ***
> CNFG: WiFi conn_type: 1
> CNFG: WiFi sec_type:  4
> FLASH: Loading Activation Data
> CNFG: Gateway ID:         ttn-gw02-868-new
> CNFG: Gateway Key:        ***
> CNFG: Account Server URL: https://account.thethingsnetwork.org
> CNFG: Locked:             false
> CNFG: Locked first time:  true
> 
> MAIN: Leaving state 1
> MAIN: Entering state 2
> INET: State change to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> WIFI: Entering state 1
> ETH: IP Address: 0.0.0.0
> WIFI: Entering state 2
> WIFI: Disabling modules
> Head magic match void: trying to free an already freed block, ignore
> WIFI: Entering state 3
> WIFI: Enabling modules for client
> WIFI: Entering state 6
> 
> 
> >WIFI: IP Address: 0.0.0.0
> SNTP: State change from 0 to 1
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> MON: SYS Stack size: 3967
> MON: TCPIP Stack size: 3882
> MON: APP Stack size: 8008
> MON: LoRa Stack size: 3909
> MON: heap usage: 178KB (188KB), free: 161KB
> CB: Disconnect
> CB: INET: Gateway has WiFi
> INET: State change to 2
> INET: Connected to a network, waiting for DHCP lease, checking validity with ping
> MON: SYS Stack size: 3959
> MON: TCPIP Stack size: 3882
> MON: APP Stack size: 8008
> MON: LoRa Stack size: 3909
> MON: heap usage: 178KB (188KB), free: 161KB
> SNTP: State change from 1 to 2
> WIFI: IP Address: 192.168.1.219
> INET: State change to 3
> INET: Ping probe
> INET: Error sending probe on Eth
> INET: Ping response from MRF24WN, set as default
> INET: State change to 4
> SNTP: State change from 2 to 6
> SNTP: State change from 6 to 1
> INET: Initiated NTP request.
> SNTP: State change from 1 to 2
> SNTP: State change from 2 to 3
> SNTP: State change from 3 to 4
> SNTP: State change from 4 to 5
> SNTP: State change from 5 to 6
> SNTP: State change from 6 to 7
> INET: State change to 5
> 
> MAIN: Leaving state 2
> MAIN: Entering state 3
> 
> CNFG: Load online user config state change to 4
> HTTP: Close active socket 0
> HTTP: Starting connection
> HTTPS: Connection Opened: Starting TLS Negotiation
> HTTP: Wait for TLS Connect
> MON: SYS Stack size: 2837
> MON: TCPIP Stack size: 3765
> MON: APP Stack size: 4243
> MON: LoRa Stack size: 3909
> MON: heap usage: 262KB (268KB), free: 76KB
> HTTP: TLS Connection Opened: Starting Clear Text Communication
> HTTP: Got 1328 bytes
> HTTP: Connection Closed
> HTTP: Close active socket 1
> CONF: Parsing response token: HTTP/1.1 200 OK
> CONF: ROUTER URL: mqtts://bridge.eu.thethings.network:8882
> 
> CNFG: Load online user config state change to 5
> FLASH: Lock Activation Data
> 
> CNFG: Gateway activation locked
> 
> CNFG: Load online user config state change to 6
> FREQ: APP_URL_Buffer: https://account.thethingsnetwork.org/api/v2/frequency-plans/EU_863_870
> HTTP: Starting connection
> HTTPS: Connection Opened: Starting TLS Negotiation
> HTTP: Wait for TLS Connect
> HTTP: TLS Connection Opened: Starting Clear Text Communication
> HTTP: Got 1266 bytes
> HTTP: Connection Closed
> HTTP: Close active socket 1
> 
> CNFG: Load online user config state change to 7
> 
> CNFG: Configuring LoRa module
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> MON: SYS Stack size: 2837
> MON: TCPIP Stack size: 3765
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3887
> MON: heap usage: 186KB (268KB), free: 153KB
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> MON: SYS Stack size: 2837
> MON: TCPIP Stack size: 3765
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 202KB (268KB), free: 137KB
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2837
> MON: TCPIP Stack size: 3765
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 218KB (268KB), free: 121KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> 
> **************************
> *   The Things Network   *
> *      G A T E W A Y     *
> **************************
> Firmware name: AmazingAckermann, type: 0, version: 1.0.7, commit: e9d35a30, timestamp: 1560942019
> Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298
> Build time: Jun 19 2019 11:01:21
> Reboot reason: 0x40
> BOOT: (persisted info) 6F 72 72 65 01 03 AE 62 75 77 BF F5 14 7C 3C 10
> 
> 
> 
> 
> WIFI: Entering state 0
> WIFI: Entering SCAN state 0
> 
> MAIN: Initialisation complete
> LORA: Changing state from 0 to 0
> 
> MAIN: Leaving state 0
> MAIN: Entering state 1
> FLASH: Magic bytes found: wifi config present
> FLASH: Magic bytes found: activation data present
> FLASH: Magic bytes not found: no stored FOTA data present
> FLASH: Loading Firmware Data
> CNFG: (Firmware HASH (sha256)) FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
> FLASH: Loading WiFi Data
> CNFG: WiFi SSID:      M2M-UNIFI-UAP-AC-M
> CNFG: WiFi key:       ***
> CNFG: WiFi conn_type: 1
> CNFG: WiFi sec_type:  4
> FLASH: Loading Activation Data
> CNFG: Gateway ID:         ttn-gw02-868-new
> CNFG: Gateway Key:        ***
> CNFG: Account Server URL: https://account.thethingsnetwork.org
> CNFG: Locked:             true
> CNFG: Locked first time:  false
> 
> MAIN: Leaving state 1
> MAIN: Entering state 2
> INET: State change to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> WIFI: Entering state 1
> ETH: IP Address: 0.0.0.0
> WIFI: Entering state 2
> WIFI: Disabling modules
> Head magic match void: trying to free an already freed block, ignore
> WIFI: Entering state 3
> WIFI: Enabling modules for client
> WIFI: Entering state 6
> 
> 
> >WIFI: IP Address: 0.0.0.0
> SNTP: State change from 0 to 1
> CB: INET: Gateway has WiFi
> INET: State change to 2
> INET: Connected to a network, waiting for DHCP lease, checking validity with ping
> SNTP: State change from 1 to 2
> WIFI: IP Address: 192.168.1.219
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> INET: State change to 3
> INET: Ping probe
> INET: Error sending probe on Eth
> INET: Ping response from MRF24WN, set as default
> INET: State change to 4
> SNTP: State change from 2 to 3
> MON: SYS Stack size: 3959
> MON: TCPIP Stack size: 3799
> MON: APP Stack size: 8008
> MON: LoRa Stack size: 3909
> MON: heap usage: 179KB (188KB), free: 160KB
> SNTP: State change from 3 to 4
> SNTP: State change from 4 to 5
> SNTP: State change from 5 to 6
> SNTP: State change from 6 to 7
> INET: State change to 5
> 
> MAIN: Leaving state 2
> MAIN: Entering state 3
> 
> CNFG: Load online user config state change to 4
> HTTP: Close active socket 0
> HTTP: Starting connection
> HTTPS: Connection Opened: Starting TLS Negotiation
> HTTP: Wait for TLS Connect
> HTTP: TLS Connection Opened: Starting Clear Text Communication
> HTTP: Got 1328 bytes
> HTTP: Connection Closed
> HTTP: Close active socket 1
> CONF: Parsing response token: HTTP/1.1 200 OK
> CONF: ROUTER URL: mqtts://bridge.eu.thethings.network:8882
> 
> CNFG: Load online user config state change to 6
> FREQ: APP_URL_Buffer: https://account.thethingsnetwork.org/api/v2/frequency-plans/EU_863_870
> HTTP: Starting connection
> HTTPS: Connection Opened: Starting TLS Negotiation
> HTTP: Wait for TLS Connect
> HTTP: TLS Connection Opened: Starting Clear Text Communication
> HTTP: Got 1266 bytes
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3799
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3909
> MON: heap usage: 262KB (268KB), free: 76KB
> HTTP: Connection Closed
> HTTP: Close active socket 1
> 
> CNFG: Load online user config state change to 7
> 
> CNFG: Configuring LoRa module
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> SNTP: StatLORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3787
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3887
> MON: heap usage: 202KB (268KB), free: 137KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3785
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 218KB (268KB), free: 121KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 234KB (268KB), free: 104KB
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 234KB (268KB), free: 104KB
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 250KB (268KB), free: 88KB
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 267KB (268KB), free: 72KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 283KB (283KB), free: 56KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing state from 3 to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> MON: SYS Stack size: 2870
> MON: TCPIP Stack size: 3763
> MON: APP Stack size: 3292
> MON: LoRa Stack size: 3857
> MON: heap usage: 299KB (299KB), free: 40KB
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> LORA: Changing state from 2 to 3
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: Starting reconfiguration
> LGMD:Command, cmd: 0x31, size: 0
> LGMD:Timeout on cmd: 0x31
> LORA: Configuration failed, retry
> LORA: RESET MODULE
> LORA: Changing sta
> *** assert ../src/system_config/TTN_Gateway_v1/rtos_hooks.c:112:fatal:vApplicationMallocFailedHook
> 
> SNTP: State change from 0 to 0
> SNTP: State change from 0 to 0
> 
> 
> 
> **************************
> *   The Things Network   *
> *      G A T E W A Y     *
> **************************
> Firmware name: AmazingAckermann, type: 0, version: 1.0.7, commit: e9d35a30, timestamp: 1560942019
> Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298
> Build time: Jun 19 2019 11:01:21
> Reboot reason: 0x40
> BOOT: (persisted info) 6F 72 72 65 01 03 AE 62 75 77 BF F5 14 7C 3C 10
> 
> 
> 
> 
> WIFI: Entering state 0
> WIFI: Entering SCAN state 0
> 
> MAIN: Initialisation complete
> LORA: Changing state from 0 to 0
> 
> MAIN: Leaving state 0
> MAIN: Entering state 1
> FLASH: Magic bytes found: wifi config present
> FLASH: Magic bytes found: activation data present
> FLASH: Magic bytes not found: no stored FOTA data present
> FLASH: Loading Firmware Data
> CNFG: (Firmware HASH (sha256)) FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
> FLASH: Loading WiFi Data
> CNFG: WiFi SSID:      M2M-UNIFI-UAP-AC-M
> CNFG: WiFi key:       ***
> CNFG: WiFi conn_type: 1
> CNFG: WiFi sec_type:  4
> FLASH: Loading Activation Data
> CNFG: Gateway ID:         ttn-gw02-868-new
> CNFG: Gateway Key:        ***
> CNFG: Account Server URL: https://account.thethingsnetwork.org
> CNFG: Locked:             true
> CNFG: Locked first time:  false
> 
> MAIN: Leaving state 1
> MAIN: Entering state 2
> INET: State change to 0
> LORA: Initialisation complete
> LORA: Changing state from 0 to 1
> WIFI: Entering state 1
> ETH: IP Address: 0.0.0.0
> WIFI: Entering state 2
> WIFI: Disabling modules
> Head magic match void: trying to free an already freed block, ignore
> WIFI: Entering state 3
> WIFI: Enabling modules for client
> WIFI: Entering state 6
> 
> 
> >WIFI: IP Address: 0.0.0.0
> SNTP: State change from 0 to 1
> CB: INET: Gateway has WiFi
> INET: State change to 2
> INET: Connected to a network, waiting for DHCP lease, checking validity with ping
> SNTP: State change from 1 to 2
> WIFI: IP Address: 192.168.1.219
> LORA: Wait init complete, waiting for application.
> LORA: Changing state from 1 to 2
> INET: State change to 3
> INET: Ping probe
> INET: Error sending probe on Eth
> INET: Ping response from MRF24WN, set as default
> INET: State change to 4
> S

BoRRoZ · September 14, 2019, 7:21pm

any changes in the network / router / firwall rules ect. ?

that second ‘new’ GW… was that working before on another location/network …

patmolloy · September 14, 2019, 9:10pm

Nope, no changes to anything at all on the network side. And the gateway is completely virgin - never unboxed before! Totally perplexed. Wondering if there is something rogue on my network that ONLY affects these TTN gateways – there is loads of other gear (incl other gateways from Sentrius, ic880a, RAK) that work fine!!

BoRRoZ · September 14, 2019, 9:22pm

well, I don’t know IF its related, but there also backend problems today.
Can you move , a working, not virgin, other GW to that location ?

niels80 · September 15, 2019, 8:05am

I had similar issues and tried to resolve it on the node side. But since there are currently massive problems on TTN side (see the thread on broken gateway-app communication) it might be as well a TTN issue. I would wait for these issue to be resolved to see if things become stable again.

Best,

Niels

DarrenO · September 15, 2019, 8:45am

Looking at your debug report you don’t have problems with WIFI:
CNFG: WiFi SSID: M2M-UNIFI-UAP-AC-M
The gateway is getting an IP address:
WIFI: IP Address: 192.168.1.219

Debug Reboot reason: 0x40

It’s failing several times trying to configure LORA:

LORA: Configuration failed, retry
LORA: Starting reconfiguration

I would query what Reboot reason: 0x40 is.

Its failing to configure LORA.

patmolloy · September 15, 2019, 8:52am

Thanks for all of the support - I wonder if it is a combination of the TTN side issues, some flakey hardware my end and/or some diagnosed horror on my local LAN/WiFi network?

I updated the original gateway with the develop firmware branch and it is behaving better. It joins, stays up for a while, then gets unhappy but manages (so far) to re-join. I’ll try to capture all of the debug messages to see if I can figure it out.

The other (virgin) Gateway just refuses to play ball. I installed the same develop branch firmware, but I just cannot get it activated via the ttn.fyi/activate procedure. Mystery!

patmolloy · September 15, 2019, 11:30am

Both of the Gateways are now online. For now. I really have no clue about the original problem, but I do have a feeling that the LoRa daughter board was not properly seated in the 2nd (virgin) gateway - hence the LoRa errors in the log. They are a devil to get out and I end up snipping off the plastic supports and having a real wrestle with the double sided foam. I am going to have to find a way of re-securing properly. I have a feeling a small jolt will cause issues.

They are both on wired connections.

I’ll see how this pans out but to get to this point has taken more or less 7 or 8 hours of endless piddling about.

BoRRoZ · September 15, 2019, 11:31am

indeed… coincidence or not… the backend problems seems solved to

sinux · February 19, 2023, 2:48pm

I got same issue after re-using my gateway (unconnected for a year or so).

Firmware name: AmazingAckermann, type: 0, version: 1.0.9, commit: eb004399, timestamp: 1666964731
Bootloader revision: 1, commit: 7167873a, timestamp: 1496411298
Build time: Oct 28 2022 13:46:22
…
LGMD:Command, cmd: 0x31, size: 0
LGMD:Timeout on cmd: 0x31
LORA: Configuration failed, retry
LORA: RESET MODULE
…

It was in fact a mechanical problem with the Lora module. A (gentle) pressure on the card fix the issue.

Hope it help