TTN Console gateway traffic and device data only sporadic

How to get started
1

Hello,

I am having a gateway registered with TTN for a few days and in the meanwhile also a couple of devices running some test data. I am having trouble most of the time to see any traffic in the TTN console for my gateway and the same goes for the device data. I have been reading that in some cases issues can come up with NAT and non ideal handling of UDP traffic. Since I can I have removed that problem entirely and my GW has a public IP address now. I still have the same issue, No or only sporadic traffic and data shown on the TTN Console.

My Setup:
GW: Mikrotik wAP LoRa with 6.5dB Omni, EU 868 channel plan. Console shows it connected and last seen is usually around 30 sec max. over 2000 received messages.

Devices
2x TTGO LoRa32 T3 v1.6 (there are other designations for the same board), Using the lmic library example code for ABP, sending a few bytes payload every 1min on one device, every 5 min on the 2nd device. Expecting one to go over the FUP limits and the other not.

[edit]
I can see the Received Messages counter on the Console for my GW clock up every time my devices send a message btw.
[/edit]

I can see both devices do connect to the gateway and I can see data from both devices sporadically in the console. Yesterday I have even seen the data for approx 2h coming in. As it stands right now i see no data for device 1 and only one line of data for device 2 from approx 30 min ago. The GW logs and traffic view does confirm that data was actually send.

I have tried different browsers, different computers, in Firefox I have tried private window and normal.I am at a loss here and can not figure out what is wrong here.

Any suggestions and help is appreciated.
Kind Regards
Jan P.

PS: Just a word re this forum. LoRa is not exactly a beginner friendly thing to get into. Adding this somewhat obscure limitation to the forum that prevents ppl from posting here for x minutes is beyond frustrating especially if one is not familiar with this particular forum system and tried to find the “new topic” button for ever.

2

Can you clarify if you mean you aren’t seeing consistent data in the web console

OR

Uplinks are being lost at some point?

For the former, the web console is more designed for point in time debugging, not so much for running as a monitoring system - if only because there are no filters (yet, possibly coming) so I find I get the occasional disconnect.

If the second, can you use an integration - MQTT client on your desktop machine (that doesn’t go to sleep) or Data Storage or a Webhook - to verify if you are losing uplinks.

As to the forum, I agree that LoRa(WAN) has many moving parts so it takes time for people to acquire enough information to make progress. And for that reason if people could just land here and post a question without searching the forum first, we’d be flooded with questions - even then many questions can usually be answered by searching the forum. Or reading the TTN docs. We are all volunteers here so we have to keep the forum rightsized.

3

Hello @descartes , thank you for your response.

I can see that my MCs do communicate with the GW.

13:30:36 lora gateway-0, Rx, Freq 868.500, CRC Ok, RSSI -26.0, Unconfirmed-Data-Up, 260113E9
13:30:50 lora gateway-0, Rx, Freq 867.100, CRC Ok, RSSI -17.0, Unconfirmed-Data-Up, 2601326A

I can also see the GW is sending UDP packets to the TTN server router.eu.thethings.network UDP 1700. Every time I can see data send I can also see the counter on the GW console count up. If I query the TTN via MQTT I receive nothing back.

 mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u 'XXXX' -P 'ttn-account-v2.XXXX' -v

X obviously replace with my credentials. The console shows nothing re GW traffic and device data. I would not think of the console as monitoring but as my only way of troubleshooting. And since GW traffic as well as device data is showing every now and then I can be confident that my set up is actually correct. The fact that the counter counts up would also suggest the data send actually reaches TTN. Where things go wrong is impossible for me to tell, all I can say is that data does reach TTN and is lost somehow.

GW: eui-3133303725005400
Devices: 260113E9 and 2601326A

Also for completeness, my GW receives a bunch of traffic, all but my own devices have a CRC Status of “Error”, my own devices show CRC Status “OK”. Watching the GW logs and the Received Message counter at the same time its evident that ONLY my own devices cause the counter to count up.

Kind Regards
Jan P.

4

As Nick says this is to encourage potential users to spend some time reading around the subject and also to use search as a form of self help… probably >50% of people’s questions can be answered without ever needing to post a question after 30mins of searching, and of the other portion a day or two of deeper digging solves >50% again. The other reason we do this (though reducing repeat questions and fragmentation of info and responses is great for right sizing and keeping the forum manageable as Nick stated) is to limit spamming activities. There a huge number of bots and human click farm generators in low salary countries who attempt to stuff the forum with adverts and spam and we have tried to learn from other forums to try and avoid this, even if we ask our new users to show some commitment and spend time before posting 1st questions or starting a new thread. Even with this time-out we mods still have to spend a fair proportion of our time removing & reviewing the ones that persist. I know I would rather spend my Sunday afternoon answering & helping vs. playing human spam filter and appreciate your patience :slight_smile:

5

Hi @Jeff-UK , I can appreciate what you and @descartes are saying. It somehow does just not stand out or is clear to a new user. Maybe that can be fixed somehow and some point.

Kind Regards
Jan P.

6

We recognise it is not always obvious and the Mods - one hero in particular - are in the process of writing, developing & reviewing a series of forum guides and how too’s to help newbies get started and to assist old hands who return after a period of absence to assist in evolving the way the forum is used & developed. We have had 5+ successful years and with the pending transition of the TTN underlying infrastructure from V2 to V3 through this year we want the Forum to evolve too. We are looking forward to the next successful 5 years and beyond. :slight_smile:

We welcome new members and contributors so please feel free to dive in and share what you learn and develop as you settle in to LoRaWAN & The Things Network, Labs write-ups, how too’s and use case or device developments are always welcome so please share. Remember you are the network…let’s build this thing together… :+1: :slight_smile:

2 Likes
7

It’s likely not your only problem, but that is an extremely high RSSI which indicates that your node is far too close to your gateway. This is known to result in overload which causes certain problems from time to time, but again, it likely does not explain your entire difficulty.

You probably want to move things so that you get RSSI’s below -40 if not below -50

If I query the TTN via MQTT

MQTT does not query but rather subscribe as such you won’t receive anything unless you have a subscription already active and continuing through the time at which the message is procesed.

I suspect there’s more to your difficulty though. Ideally you would be able to show full gateway logs of messages received there but missing from a gateway web console that was open before the message and continued through it, or messages showing up in the gateway console but not an application/device view similarly open throughout the time of the message.

8

OK @cslorabox, issue one is no problem. I have moved my devices and now have a lower signal.

16:18:40 lora gateway-0, Rx, Freq 868.300, CRC Ok, RSSI -66.0, Unconfirmed-Data-Up, 2601326A 
16:19:34 lora gateway-0, Rx, Freq 867.700, CRC Ok, RSSI -65.0, Unconfirmed-Data-Up, 260113E9

If the RSSI was a problem why would the messages then be counted on the console yet not displayed ? Also why would it be sporadic, work for a bit yesterday and not today ?

Re MQTT, I suppose that’s a terminology issue, subscribe, query client/server/broker. What I mean is that I have a CLI and issue the following command:

mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u 'XXXX' -P 'ttn-account-v2.XXXX' -v

which according to this information should give me something back. Again X replaced by the credentials. Its happily just sitting and doing nothing waiting for data from the TTN side until I cancel the command. I see nothing. If I intentionally give wrong credentials I get this back:

Connection Refused: not authorised.

I am unclear on the last part of your message, that sounds as if I have either TTN Console OR MQTT, not both at the same time ? Can you explain what you mean by the last sentences in different words ?

I can for some unknown reason suddenly see GW traffic on the TTN console since 15:48 (which is before I moved them still at high signal) up to current. Nothing from the devices. In the console Applications → Application ID → Devices → Device ID I can see Status 11h ago and 5h ago only. Still they show on the GW traffic just fine now since 15:48.

Kind Regards
Jan P.

9

Have you reset the counters for your ABP device at restart of the device? If not you will not see traffic until the uplink counter exceeds the highest counter of your previous ‘run’.
That is a security measure to prevent replay attacks which catches many new users out.

1 Like
10

:kissing_heart: @kersing

That’s it, now I see stuff on the CLI as well when I query via MQTT. So how does that get automated when I have a legit reboot of a ABP device ? Do I have to manually reset those counters every time ? Is there something like a “cron job” that I can set up that can do that on a hourly basis or something like that ?

I suppose I have to look into OTAA for my devices to fix this for good.

Well, I consider this problem solved, thank you all for your help !!

Kind Regards
Jan P.

11

Not. For security reasons you should not reset, your device should save the state. Use OTAA which resets the counters every time a new session (with new keys) is generated.
If you need to use ABP you should be able to find a setting for the device that states ‘frame counter checks’. Remove the checkmark (and save) to ‘continuously’ reset the counter. But be aware this is very insecure.

BTW, this issue with ABP gets mentioned on the forum every few weeks. So reading some more regarding ABP would have helped you find this yourself.

12

I can believe that but its is impossible to conclude to a search for ABP issues from my initial observation. My issue appeared to be console related hence my search for console device data. Its like asking for which shade of green a tree has, one 1st has to know that trees are actually green and not blue.

Anyway, thanks for the help !

Kind Regards
Jan P.

13

Hi Jan the search function is really powerful and as noted there are many years of experience logged in the forum, so please do try even if you think it might be impossible. Yes, you have to do some reading but the answers are often there :slight_smile:

remember I stated

Well I looked at your OP and saw you were using ABP, you could see increasing or incrementing counters in the console. It took me 10 secs to type those 3 words in the search function…and low and behold 4th item on the list tells you pretty much what Jac said above … and the post even had some pictures posted to give you more of a clue :wink: If you have other questions give search a go - if nothing else it will lead you through places where you can learn and find out lots of background which will help smooth your LoRaWAN & TTN experience. Of course if really stuck always come back and post as forumites always willing to pitch in and help. Another good start point is the TTN documetation https://www.thethingsnetwork.org/docs/index.html and if in doubt check out TTN CTO Johan Stockings recent LoRaWAN in 60 mins on YouTube frm the recent TTConf… wont solve issues like the counter issue directly but all helps with background and understanding. https://www.youtube.com/watch?v=ZsVhYiX4_6o Also I would encourage you to join in with one of the local TTN Communities - I guess its a long way frm the Batry Bay/Dumanus Bay area to either Cork (about a half dozen members) or Limerick (Larger at about a dozen). Of perhaps you guys out in West Cork can start another for the area ( I see another GW up in Bantry itself - from little acorns etc). And in post pandemic times (when they come!) always a good excuse to meet up and have a pint of the black stuff :wink:

14

Hi, I am not disputing that if I had known what I know now, that I could have searched with more relevant words but I also know from own experience that ppl that are in the know of something expect to much of “connection making” of someone that is totally new to a subject. I was indeed talking about a counter but that was the message counter of the GW, not of the devices. Also ABP was in any YT video I have seen shown as the easy to start with thing, nowhere have I seen that this will cause issues with counters. So looking for ABP issues would also not come to mind as a thing to look for. So all of you that are in the comfortable position to know a lot may have to just expect ppl not to be able to correlate 2 seemingly totally different ends of a problem. And I have also not just given it a shot with posting here because I have not looked elsewhere, I 1st of all spend days to find out how I can just compile example code in PlatformIO and to figure out what pin definitions are needed for the particular boards I have. I have also spend hours on Andreas Spiess YT channel to get information. That for example is the reason to buy a GW instead of building one to start. I felt it was enough of a challenge with the home brew clients and I needed a known working factor as GW. It is the norm for pretty much every forum I have seen since the beginning of my internet use that ppl unacquainted with the subject matter will not know the right words to search for, this forum here being no different. I am totally new to LoRa but I am not new to IT, my background is TCP/IP networking and WAN infrastructure. I like to think that I have a reasonable grasp on technology. Maybe you in the know expect to much from “new players”.

That is easy to say for someone intimate with LoRa,

So please all accept my apologies for not knowing the right words to search for. I am sure it wont be the last time and I am surely not the last user that experiences difficulties.

Again, thanks to all and maybe just reconsider what you expect from new users.
Kind Regards
Jan P.

15

Definitely will do once we can meet up again :smiley: I have the use of a number of prominent mountain radio mast sites to deploy GWs. My goal is to run a server my GWs connect to and handle my own data myself and forward all other clients that may connect to my GWs to TTN. I have not read in depth but I will have a closer look at the ChirpStack. Anyway, that’s for the future but its what I am aiming for.

Regards
Jan P.

16

That would put you outside the scope of the TTN, and naturally we would refer you to the Chirpstack forum in that event, for now lets assume we can persuade you to use and stick with TTN - afterall why have a dog & bark yourself (other than academic interest :wink: )

We all started somewhere - I spent the 1st 2-3 weeks after I joined just reading posts, absorbing and watching, and that after dipping in occasionally, annonimously, over a couple of months before that before I started asking questions … though I knew a little on LoRa/LoRaWAN before that so answered a few before ever asking my own. Like you I started with prebuilt GW’s then moved on to self builds as I gained enough knowledge/confidence to be dangerous, after a background that also included a bit of IT, networking and some wireless, though more sales/mktg/biz dev than deep techy - so I’m confident you will get there if I can.

Envy you your location and general West of Ireland position - love the area - used to be frequent visitor (mostly the Dublin/Cork/Limerick/Shannnon/Ennis triangle) for biz and occasionally pleasure. Also around Tralee, Ballybunion, Kilkenny/Killkee & Galway…so hopefully once we are plague free and all opens up again I will get back - might just look you up and buy you one of those pints to see how you are getting on. Have a look around the forum and the Irish communities as there are a number of active members in country and many started in last year or two as new to LoRa/LoRaWAN also so might help learning curve and share experiences :slight_smile:

Absolutely no need to apologise - all we ever ask is that you try : and as mentioned earlier we are currently underway with a project to hopefully make it easier for the new guys & gals to start up and find their way around

17

That depends entirely on how well I can deal with the airtime limitations. Since I have no practical experience yet I can’t tell. Running my own private would only be needed if I can see that the airtime becomes an issue and if I can configure that to suit my needs on my own system. But that’s future. For now I am fighting a battle again with PlatformIO and the example code for OTAA … I am Network and Systems engineer, not software developer so there are clearly my limitations.

Re location, I am only a blow in here, I am from DE originally and live here since 2003. West-Cork is no doubt one of the nicest places to be :smiley:

18

If you have a suite of devices that are going to be in excess of the 30s Tx FUP per device per day, LoRaWAN may not be right for you.

What sort of data are you hoping to collect? Best to do some calculations now rather than end up with kit & infrastructure and find out you’ve hit the ceiling?

19

@descartes I am not to bothered spending a few euro on a test setup. So having one or two GWs for testing and a hand full of clients would be OK. The data collected would be mostly environmental and possibly also downlink to react to a value I read back. LoRaWAN is pretty much the only possibility short of using dedicated 8xxMHz ISM band modems with RS232. For “good times” I can easy use devices such as Wireless Point to Point links but I need something more resilient and only low bandwidth. The Airtime limitations are kind of weird. If I for example need to read 5 values and send them from one device I might be bad on Airtime, Yet if I have 5 devices sending those 5 values I am OK if I understand the limits correct.

Well, I fight my OTAA issue 1st, that appears to be the most pressing now to avoid that counter issue I had with ABP and then deploy a few devices to see how I get on with Airtime.

Regards
Jan

PS: What other tech than LoRaWAN would consider as alternative ?

20

At the very least put the 5 readings in the same packet so that you only pay the penalty of the network headers once for all of them together, not for each individually

1 Like