Using TTN trafic monitor to troubleshoot a TTIG-based setup

Hello,

For starters, a bit of context :slight_smile: We are new to LoRaWAN, with which we are trying to use the TTIG to forward packets from Sensoterra autonomous soil moisture probes up to their servers. We have a fairly simple design with 8 probes positioned in an orchard around a gateway that is powered via a solar panel, and connected to the internet through a 4G dongle. Distance is quite low in comparison with what other people have been trying to do with TTIG, i.e. <100m. We can easily monitor the gateway through TTN v2 console so we know it is up and running, and we have other ways to check if there is a power loss on our setup.

Once we woke the sensors for the first time everyone was happily connected with a fairly good declared network power from the probes. But after a few days everybody progressively stopped emitting even though nothing changed in the setup. That is why are now trying to have a look at the TTN gateway trafic interface, on which we noticed that there were comparatively more and more join/accept requests and less and less uplink and downlink ones.

According to your experience, what could be the cause of this ? Is there any element we could have a look at in the trafic monitor or elsewhere to try understand what may be the issue ? I feel like the documentation is quite hard to reach and fully understand for newcomers, so my reason for being here today.

Thanks in advance !
J.

edit : in the meantime, iā€™m reading with attention topics such as this one. Also, this issue seems to be quite similar somehow.

edit 2 : we moved the antenna a bit further up, and now one probe woke up. Stragely enough, it is one of the farthest probes and declares a very good signal strenght.

This is NOT a simple design - it should work but there are at least two major extras here.

The TTIG can not inspire any devices to issue join-requests - this is usually due to power outage on the device or getting wet or some other influence. If the device has ended up having to rejoin, it does mean something has happened and they canā€™t send any uplinks until they have received the join-accept. If itā€™s battery issue, then they may power down, get enough reserve to send a join request but then fall off again so not be around for the join-accept.

So, concentrate on the devices first. What are they?

Preferably, get the TTIG on to WiFi via a wired internet connection and a couple of the devices at least 5m / behind a brick wall away to test. Letā€™s check the kit works before it runs on solar on a 4G dongle.

Thanks for your critical point of view. Weā€™ve been running tests before deploying in the fields, first in the office with TTIG connected to a wired internet connection through wifi, then using the wifi provided by the dongle. We repeated the experiment outside with the full solar panel setup without any issue. We havenā€™t observed any kind of obvious power loss, i.e. TTIG led switching off from fixed green.

My main source of surprise is that there is one probe performing nominally.

This ^

And this ^

Ok, please be patient as this is all new for me but we use Sensoterra soil moisture probes (technical sheet), that are using LoRaWAN 1.0.2, ADR enabled protocol. This document tells a bit more about how they behave in their power on/power off cycle.

Patience is my middle name**. We are all volunteers here, so we tend to be blunt when we ask questions and get a response that doesnā€™t even come close to answering them :wink:

Link to datasheet is very useful, interesting that they use a non-replaceable battery but it makes it easier to waterproof it. Shame they donā€™t appear to be sending battery level in their API. Even bigger shame they tried so hard to make it a closed system.

Iā€™m assuming you got the EUIā€™s & AppKey from the device and set that up on TTN. Can we see what a device console provides please? And the TTIG console as well, just so we can correlate. The devil is always in the detail.

Can you pull one out the ground and place it back in the ground about half way in about 10m away from the TTIG? And check the TTIG has the solid green?

** Actually, Danger is my middle name, but thatā€™s another story.

Thatā€™s completely understandable, and iā€™m thankful youā€™re taking some time to help !

They actually provide some kind of battery level info in their customer interface. Still, these are brand new probes.
image

Nope, as basic users we just turned everything on, shook the device to wake it up and data was forwarded for some time.

I donā€™t think there is any kind of device console, as it is entirely closed shut and inaccessible.

What weā€™ve seen is that there is a number of join requests/join accept like this one, but no up/downlink following up.
image
image

Weā€™ll try that when weā€™re back on the field but for reference hereā€™s the layout of the probes. The gateway is at Arbre 1. The first probes to answer this morning were #5 and #6.
image

OK, this is getting weird.

The ONLY way that these devices will be able to join TTN is if their details have been entered somewhere on to TTN by someone at some point. Does the manufacturer do this?

Did they suggest the TTIG?
Can you show us the RSSI & SNR on the Join Request?

The ā€œAbre 3 Orograndeā€ screen shot says it is "operationalā€™ and the last update was yesterday afternoon. Is it set for hourly updates? Is it trying to join (if you know itā€™s DevEUI).

When you say the first probes to answer this morning were #5 & #6, were these joins or uplinks? If they run hourly, why would they ā€œanswerā€ in the morning? Have the others checked in yet?

What does the manufacture say about this issue?
If the manufacturer handles the device registration, how will they be handling the move to v3?

Do you have a different device you can use to check end to end and see all the details on all the consoles?

I suppose, as they seem to have agreements with TTN and as it is specified in the datasheet that sensors come with three year LoRaWAN provisioning. This may be related to the fact that an app eui is showing by default in the requests ?

No, but they partner with/recommend some hardware providers.

Here is an example. Other values go 9+/-1 for SNR, -105+/-5 for RSSI.
image

Sensoterra consider as operational a sensor that did send data in the previous 48h. But they have an internal memory of 6 measurements and should send a reading every hour. This sensor is not seen as trying to join.

5 and 6 performed both joins and uplink this morning. I have no idea what triggered more than just the join. Certainly not the sun/powering up of the setup as it was at 5 am. The other ones havenā€™t checkind in since we first set everything up on the 15th. Hereā€™s a graph for 6 of them :slight_smile:
image

They seemed a bit puzzled as well and think it may be related to an issue in the packet forwarder ; they proposed to set up an endpoint on their side to perform verifications. We wanted to check if we did obvious mistakes before having them perform it as they will add a supplementary charge for that.

:thinking:

Nope, but this is something we could do ! If you could recommand a cheap, reliable device for that weā€™d have a look at it.

None available - there is no problem with commercial use but there is no SLA and this forum is the support channel - which is fine right up until a situation like this. Which is excellent for me as I can write this up and share with my prospects. Wonder why they choose three years, itā€™s not like it costs them anything, but youā€™d have to buy new ones when the battery runs out anyway.

The RSSI seems ridiculous low for a device that close in free air - one of the test devices I have running is one brick wall and two lots of windows away with a PCB antenna and the office gateway with a little stick antenna is hearing at -73.

Iā€™d go & pull a sensor out and try different placements to get some readings. Possible try the TTIG in different places and check itā€™s not been put in a metal box.

As for a simple device, something like the Dragino Water Leak sensor (or door sensor): LWL01 -- LoRaWAN Water Leak. Generally available in the EU from a variety of LoRaWAN shops. Basically you want something prebuilt that comes with the DevEUI, AppEUI and AppKey already configured so you donā€™t have to mess about with setup.

It wouldnā€™t hurt to have another gateway thatā€™s not a TTIG so you can eliminate the potential.

But fundamentally, the devices are doing the joins, something they should do only once in their lives as they have the batteries embedded in them. Once they have joined, they have no reason to do so again. If by design they do that, the firmware is way outside of the boundaries of normal LoRaWAN behaviour.

@descartes, thank you for sharing your thoughts. Weā€™ll continue exploring and update the topic because even though this falls into some level of commercial case, weā€™re actually only experimenting in the scope of using these (or other) LoRaWAN sensors for agricultural applications.

No problems with it being commercial and for you, you have a problem related to TTN.

Just makes me feel uneasy when companies use a non-SLA free service without full disclosure.

And you still have the migration to v3 to come with a new set of joins - I do hope their devices can be told to reboot / rejoin.

That isnā€™t possible.

Someone had to register the devices.

Where did the account / login you are using to access these come from?

it is specified in the datasheet that sensors come with three year LoRaWAN provisioning

Thatā€™s wrong in both directions - the manufacturer canā€™t provide 3 years on a version of TTN thatā€™s being turned off later this year in favor of the new V3, but they also canā€™t limit you to 3 years, since TTN is not their system.

My main source of surprise is that there is one probe performing nominally.

The decision to give up and do another join request is one made entirely by the node - LoRaWan has no concept of an explicit negative ack, it can only refuse to ack. So itā€™s up to a node to decide what to do when it doesnā€™t hear anything. Even if a connected node has a ā€œdisconnect and start overā€ command thatā€™s really just a suggestion that itā€™s up to the node to actual accomplish.

So even if thereā€™s a common issue or bug, different results on different nodes is entirely expected.

That all depends Nick, these soil sensors are often set very low to the ground typically with ant <<1m above surface - well inside the Freznel interference zone, and if GW isnt mounted very high (>10m ideally >20m) then that can severely limit RF signal, esp as though aerial photo shot shows land largely flat if it is undulating and with deep furrows and GW also low signal can get very attenuated, very quickly - also this is a TTIG - designed for indoor use and not weather proof so I assume deployed inside another enclosure? Is it modifiied to support an extenal antenna? If not then if the housing is in any way screenng signal the RSSI will be severly limited. It doesnt have to be a metalised/silvered plastic box- even if in what appears to be a standard plastic (ABS?) box, some boxes, esp if from recycled plastic content can severely attenuate (have seen 8-10dBm loss) - a problem where people put GWā€™s or nodes inside, e.g. plastic plumbing or sewerage pipes with limited high Frq/uWave RF transparency. (Often worth doing a quick uWave oven test on chosen materials to see what happens!). Also looks like some intervening trees/hedge row - is there perhaps a metal fence in way too?

So this part is fully managed by Sensoterra, then.

It is placed 2m high in a tree, all sensors are placed at ground level.

Yes, it is placed in a ABS box with an external antenna.

As weā€™re experimenting in orchards, wellā€¦ Still, having fruit trees as well as a row of very tall trees in between the two plots didnā€™t seem to worry much Sensoterraā€™s support team. Especially as at the time of initial setup every node woke up and declared good connectivity.

This is a bit of a head scratcher.

Iā€™d definitely think in terms of pulling the lot back to somewhere warm where you can check everything over, then find someoneā€™s garden close to hand to try to emulate the setup.

And if this is early days for you with evaluating ā€˜thingsā€™ then a different gateway and a couple of prebuilt but less closed (both physically so you can change the batteries and in terms of setting settings & registering it on TTN yourself) devices would be a good investment ~ā‚¬200. That way you can see what the logs look like for something else & can then compare & contrast.

Shame itā€™s an orchard. I like cider but if it was a vineyard, Iā€™d be more than happy to assist in person.

1 Like

We can still use the citrus peels to make fruit wine if you like thatā€¦ :slight_smile:
Joke aside, whatā€™d be a good more open gateway to start (again) with ?

I wouldnā€™t discount the modified TTIG as a solar based gateway, so donā€™t think of it as starting again, more about having something different to get to grips with more of the details.

Iā€™m not a gateway man, thatā€™s more @Jeff-UK who likes to build solar powered gateways with scaffold poles or @cslorabox who can go in to the details of the platform & OS updates.

If you want a purpose designed outdoor setup, then maybe a Dragino DLOS8 with 4G module.

But Iā€™m thinking more in terms of you having a gateway to do compare & contrast + testing. So you have two different gateways and as well as these water moisture sensors, at least one other device. Then you can test each combination and, as above, compare & contrast.

So a Dragino LPS8 LoRaWAN Pico indoor gateway for testing at the office. Or one of the RAK Pi based gateways.

Can we make Limoncello?

Ok, i see, weā€™ll have a look at your suggestion. Looking forward to read some from @Jeff-UK and @cslorabox as well, the ā€œworkbenchā€ series of topics look terrific. For the limoncello, we can do that (as long as we manage to gather soil water status and drive irrigation properly) !

Back to trying to understand the numbers, from the three devices that achieve uplink, i have :

Arbre 1 : ā€œrssiā€: -39, ā€œsnrā€: 11.25
Arbre 5 : ā€œrssiā€: -103, ā€œsnrā€: 8
Arbre 6 : ā€œrssiā€: -109, ā€œsnrā€: 7.25

Though i am also monitoring join requests from another probe (i donā€™t know which one because i donā€™t have at the moment the correspondance between serial nb and dev eui) with values ranging around :

ā€œrssiā€: -85, ā€œsnrā€: 9
ā€œrssiā€: -77, ā€œsnrā€: 8
ā€œrssiā€: -77, ā€œsnrā€: 9.25
ā€œrssiā€: -78, ā€œsnrā€: 14

So devices with more signal attenuation do communicate properly while other with less cannot join ā€¦?

And you decided to just ignore the question about how you got access to TTN?