TTN GATEWAY central

Thank you, Yes I’m frustrated because IMHO while everyone is focused on this reboot issue and it seem noting is getting done with regards the issue I’m seeing, while I do understand the team has/is working hard.

I DO hope and trust the team will get round to the type of issue I’m having, but I’m at a lost to understand how I can resolve it as the gateway won’t update or show as a AP

If I have offended anyone here then please accept my sincere apologies, it was not intended; I have been trying to keep to facts and experience with my feelings as IMHO

With regards to the mission critical statement, this was a poor attempt and a personal opinion at possible failure during testing/quality control and/or the mindset of the overall project which I unreservedly apologise for if this caused offence.

Please, please could someone acknowledge the issue I’m seeing and provide some guidance.

4 Likes

I think they’re indeed focusing on the reboot issue, which seems to affect more people and therefore has higher priority. That doesn’t mean that we don’t care about your issue, but giving one problem the full attention will get it solved quicker than dividing time over two (unrelated) issues.

Your issue is tracked as TheThingsProducts/gateway#5. The last message in that issue (from you) says “after many attempts and reboots the gateway is now showing up as an AP and I’m able to connect; and the gateway now looks to have successfully updated it’s firmware” […] “however the #1 is still present” (where #1 is the reboot issue). If you are again seeing the AP issue, please comment on Github with a detailed message of what’s happening from the moment you plug in the gateway, so that the @TTP team hopefully has enough information to look into it when they have time. It would also be helpful to see what happens if you use an ethernet cable to connect the gateway. Please write all of that in the Github issue so that we have everything in one place, instead of information being spread across multiple places.

1 Like

Ok, have done, and described as possibly intermittent; will leave for extended period to see if that changes anything

To keep track of issues I propose to register issues at the github repo. It is the central location and will help to keep track and overview of everything.

2 Likes

Good post!
But if you talk about hundreds of gateways running fine the failure rate seems quite high, maybe even 10% if you count the issues reported here.

Yes, it first says ND, but changes to 868Mhz before rebooting.

The board seems to be properly connected.

Thanks for posting this. In the first place because you’re absolutely right. Secondly, you give some insight in what looks like a controlled process for identification and resolution of the known issues. That is good news. I have chosen not to dive into the logs and the code at this moment. Reason is that it will probably take an amount of my time which I can not spend now. So, I am patiently waiting for some updates from the @TTP team, and will be happy to do my part after that, for example by installing and testing beta releases.

This fix did not work on either Gateway (ref) device 1636D4 or 1710D4. 17/2/18

I have a brand new “The Things Network Gateway” which arrived today. I haven’t yet successfully made it through the full activation process. I have been looking through the forum, and haven’t yet found the solution.

After a few initial tries, I haven temporarily given up attempting to connect my TTN Gateway to my home router via WiFi. Instead, I currently have a regular old Cat5 ethernet cable hooking it to my home router. Eventually I want to connect it via WiFi, but to keep things simple for now I am using ethernet.

When I log into my home router, I can see that the TTN Gateway has obtained an IP, And using that IP address I am able to successfully browse to the internal web server on the TTN Gateway using my browser. On the “info” page, I can see the “uptime” continuing to increment. I can also see things like the following which appear to be happy:

Connected: true
Interface: Ethernet
Activation locked: true
Config correct: true

When I attempt to use the “activation process” at https://activate.thethingsnetwork.org/, it seems to reach “Step 4” of the process successfully, but I am continually seeing this message on that web site: “waiting for gateway to check in…”.

As for the gateway itself. I have seen everything from two solid LEDs with a blinking third, and then it reverted back to one blinking LED, and at the moment it shows 2 and sometimes 3. I am not sure if it is rebooting or not (sometimes the lights on the ethernet port go off and on, not just blinking, which might indicate a reboot is occurring).

I have looked for a way to connect some sort of serial port to my laptop in an attempt to get more information, but the USB port next to the “Mode” button doesn’t seem to be producing data (I don’t see any USB device listed in my linux host /dev/tty* list of devices).

I’ve tried resetting a few times by holding the “Mode” button while powering on, and so far it always gets me to the same spot.

Any hints are very most welcome.

Unfortunately it sounds exactly like the reboot issues the gateways are having

Yeah, after a deeper read of the forum, I agree. In the meantime, I’ve ordered a UART to USB converter to check and compare my own console messages with those here in the forum. But at this point, based on everything I am seeing, it seems as if I and others will be waiting until the team announces they have fixed it.

@htdvisser, I appreciate the updates you have posted. For my part, I find periodic updates on this issue to be helpful. If it doesn’t cause too much inconvenience on your end hopefully the updates can continue.

After I get the correct SD card, I’ll give the beta firmware a try on my end, and I can certainly test any proposed fixes if it helps (though I’d like to make sure I can revert to the previous firmware first and I am looking for notes to make sure I can do that)

Thanks.

just curious, what are the difficulties to select the proper clock-source in the PIC32?

1 Like

For microchip devices it is not uncommon to have oscillator settings in a configuration word that is written by the external programmer. (Applies to this device as well)

@TTP Would it be possible to program the intended settings if one has access to the appropriate Microchip programmer? Or would this just complicate the firmware maintenance? If we can update I will gladly offer to flash gateways during our monthly meetups.

1 Like

Today my Things Gateway has died. I had to move it, so briefly unplugged the power cable (as I’ve done several times before without issue), and now it’s stuck in the dreaded reboot loop. I’ve never had any issue with this gateway before (It’s been running for 6 weeks and transferred 120K+ packets). A full reset does not help. Using Ethernet does not help. The firmware is still the original (for me) v1.0.0-917719b9 (2017-06-26T17:59:33Z)

As far as I can see, reading through all the messages here, this may be the first report of a fully functional gateway randomly transitioning to this broken state?

I had the same problem this morning, tried different paths, with ethernet connection, wifi, nothing seemed to work. Then i just cycled my wifi router and TTN gateway power and after 10-15 minutes or so rebooting it worked again…

@Codion @Bokse0011 had same problem also this morning. In my case I had to temp turn off otherwise reliable TTN GW whilst adding in a multi-way power strip & 5 port switch to allow connection of other collocated GW’s for comparative tests in roof space. After adding power strip but before even touching network side GW powered up but then went in loop mode where it would cycle through to 4 leds then immediately fall back to 2nd led flashing 4/5 times (with 3rd led solid) before going to 4 then immediately falling back to 2nd led flashing…left alone for >1 hr still issue so brought down to office and directly connected to internet gateway - same problem…left running in loop for ~30 mins and suddenly stable (note updates and use of beta are turned off so it wasn’t an update loading issue). Then took back to roof space and quickly re-installed and immediately fine and has been good for hours since. :slight_smile: It is a lot colder in roof space (~7/8 deg c vs ~18/19 in office so suspect it may be a temp dependence - as GW was still warm when I took it back to restart in roof space. Given identified timing issues on the design wrt internal vs external timing source I suspect that may be a contribution. Will experiment more in coming days/weeks but doesn’t bode well for long term reliability over more extreme diurnal temp ranges, or quick self recovery after any power fails. Then again it could just be we all hit a TTN/Network issue? :wink: My suggestion is try warming or cooling your GW before trying again and see if that helps…if it does let us all know.

My gateway is partially working again. Some notes, in case anyone trying to diagnose this is interested:

I left the gateway in its reboot loop, also connected to ethernet, with the lid off to stay cool(er) and it appeared to recover itself after about 1 hour. Kind of. I had enabled the beta firmware option in the console, and it upgraded itself to v1.0.1-facdef23 (2018-02-14T08:16:22Z) and forwarded some LoRa packets again. Looked good. I then configured it for wifi (not because I really want to use wifi, but it seems this is currently the only way to get rid of the gateway’s hotspot?) and it got stuck in the reboot loop again. I left it in this state for about 10 minutes and it appeared to become “stable” again, on Wifi, but was quietly NOT forwarding any packets to TTN The LoRa LED would flash, but nothing was forwarded to my TTN application or shown in the console gateway traffic log. The “info” page on the gateway itself also showed “Packets up” increasing. After 10 minutes of this behaviour I dared to powercycle it again and after a few more reboot-loops it seems to be working once more - forwarding packets, but still occasionally rebooting too.

Really hope for another firmware update soon!

Hope for you guys that this is is a issue that is really fixable via firmware. Since the source is available i think a simple logic error would be already discovered…

1 Like

After building my own firmware (with debug logging turned on), my gateway stays up, at least for a little while. It has rebooted once (it seems to require a full power cycle to get it back up again). The uptick counter on the info page showed mid-2000s when it rebooted. And as I was typing, the second reboot occurred at around 1000-ish. Sometimes it comes back, sometimes not (ie. its not yet predictable for me).

My UART device hasn’t arrived yet, and thus I don’t have any logs to share yet. But, as already mentioned on several posts, seemingly this does point to a timing problem.

And on yet a different but related topic … This shows my ignorance on all things LoraWan (which I am trying to work on :slight_smile: ) … I have an Adafruit 32u4 with Lora, onto which I have compiled and seemingly successfully uploaded the LMIC stack and its ttn-abp (after modifications as described here).

During those time periods while the gateway is up, I have been trying to register the device and transmit “hello world”. Thus far seemingly unsuccessfully (I am seeing messages queued on the device, but nothing is showing up in the ttn console yet). Note I don’t have a HackRF yet to even know what kind of signal is leaving the Adafruit board, and I am unclear on many LoraWan topics such as device ids, eui’s, messaging, otaa vs abp, and other topics. I am not sure you have any pointers that might be useful for me to explore (right now its all about me googling, and lots of hit and miss :slight_smile: )