Over-the-air-activation OTAA with LMIC

SvenKbach · August 3, 2016, 1:00pm

Thanks!

A life saver, but still pretty unstable, 50/50 it works, but when it does it always connects in max 20 seconds.

matthijskooijman · August 8, 2016, 9:44am

I just found a bug in the LMIC library, where it uses an invalid channel for sending join requests 50% of the time (because the list of join channels was reduced from 6 to the 3 in the LoRaWAN standard, but two places didn’t get updated properly). This means that in 50% of the cases, the first (and possibly second and third) join request would not be transmitted correctly. Once the channel wraps around from 5 to 0, it would be valid again and joining could again work. I’m committing a fix now, I expect to push it to my git repo today.

Note that this problem is not related to the SF used, so this does not explain what @Kibet is reporting (but perhaps this was the problem he was seeing and the channel selection just accidentally lined up with the SF selection). I haven’t been able to test joining at SF7 yet, since I’m too far away from my gateway for that to work now.

matthijskooijman · August 8, 2016, 10:17am

Also, there was another bug where, if joining was started automatically and the join request needed retransmission, the queued data packet would also be retransmitted a number of times (until an ack is received, but no ack was actually requested). This could cause the first EV_TXCOMPLETE after an EV_JOINED to be delayed for quite some time (up to 8 attempts with increasing SF’s). I’ll also have this fixed today.

Oliv · August 8, 2016, 10:09pm

Thanks for those bug corrections, did you pushed them already ?

Also I saw your comments on the license change, does it means that IBM will maintain LMIC or “just” allow a more permissive use of the stack ?
And I am not good in licenses, but does it also helps with the AES libraries you have tested ?

matthijskooijman · August 9, 2016, 9:59pm

@Oliv, I didn’t get around to pushing them before, but I just did. I posted an update in another topic, I think that should answer your questions.

matthijskooijman · August 9, 2016, 10:05pm

@Kibet, I just pushed quite some fixes to the lmic master branch, could you see if your join at SF7/SF8 persists? I’ve just tested this here, SF7 joins work for me. Two related remarks:

Perhaps your clock is inaccurate, but the error is negligable at the slower SF ratings? To check, add this to your sketch (somewhere during setup):

LMIC_setClockError(MAX_CLOCK_ERROR * 1 / 100);

This tells LMIC to make the receive windows bigger, in case your clock is 1% faster or slower.

If you set LMIC.dn2Dr before starting a join, I believe the RX2 window during the join will use that datarate. I do not think that TTN will use RX2 for join accepts (since the RX1 should always have equal or lower SF, so less airtime), but perhaps I’m mistaken and it does matter somehow.

gizmocuz · August 10, 2016, 8:59am

Tested with the latest merged code from matthijs.
Without the above patch (which i also did at line 879), the join process is faster, but still takes around 7 minutes.
I also noticed that the join request is now only performed on a few channels, not all.
Still, when the join process succeeds, it is always right after the request on 868.1Mhz
As experienced with a previous test
(LMIC-1.51 fitting in Arduino Atmega 328)

With the above patch from @Kibet , it is immediately (tested now 3x in a row)

EDIT:

@matthijs, your right!, setting the clock speed adter LMIC_reset() solved the problem also!
No need to patch the LMIC code

matthijskooijman · August 10, 2016, 10:35am

Ok, good to know. This would suggest either an inaccurate clock, or perhaps other (software) issues that are solved by increasing the receive window length. What hardware are you using? Is your sketch doing significant work between calls to osrunloop_once?

I would not expect this behaviour with git master, even when the clock is inaccurate (though perhaps it was just a coincidence).

If you want to dig in further, try setting LMIC_DEBUG_LEVEL to 2 (and uncomment the LMIC_PRINTF_TO line as well) to get more info about transmissions and settings used.[quote=“gizmocuz, post:37, topic:1921”]
I also noticed that the join request is now only performed on a few channels, not all.
[/quote]

The current version of LMIC only tries joining on the 868.1/3/5 Mhz channels. The LoRaWAN spec has three more frequencies, but the default gateway configurations and the TTN frequency plan does not include these frequencies, so LMIC does not use them.

gizmocuz · August 10, 2016, 3:41pm

My sketch is 99% your TTN-OTAA sketch, except it sends a text like “test 1”, “test 2”, etc…
It is not doing anything else at the moment

I now started the sketch about 20 times during the day, and all times the joining was instantly…

I am currently running some other tests, but tomorrow i will remove the clock fix, and will debug more with level 2, and report back. (But it’s working great now!, many thanks to you and all who made the pull requests)

gizmocuz · August 11, 2016, 9:54am

I did the test, this is the result without the clock adjustment call:

Starting OTAA New sample
RXMODE_RSSI
168: engineUpdate, opmode=0x808
Packet queued
1243: EV_JOINING
2412: engineUpdate, opmode=0xc
131206: engineUpdate, opmode=0xc
131511: TXMODE, freq=868100000, len=23, SF=7, BW=125, CR=4/5, IH=0
445987: RXMODE_SINGLE, freq=868100000, SF=7, BW=125, CR=4/5, IH=0
511463: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
521599: engineUpdate, opmode=0xc
4187723: engineUpdate, opmode=0xc
4188029: TXMODE, freq=868300000, len=23, SF=7, BW=125, CR=4/5, IH=0
4502376: RXMODE_SINGLE, freq=868300000, SF=7, BW=125, CR=4/5, IH=0
4567853: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
4577925: engineUpdate, opmode=0xc
8772865: engineUpdate, opmode=0xc
8773171: TXMODE, freq=868500000, len=23, SF=8, BW=125, CR=4/5, IH=0
9090827: RXMODE_SINGLE, freq=868500000, SF=8, BW=125, CR=4/5, IH=0
9156207: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
9166280: engineUpdate, opmode=0xc
16773798: engineUpdate, opmode=0xc
16774106: TXMODE, freq=868100000, len=23, SF=8, BW=125, CR=4/5, IH=0
17091570: RXMODE_SINGLE, freq=868100000, SF=8, BW=125, CR=4/5, IH=0
17156950: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
17166957: engineUpdate, opmode=0xc
24745192: engineUpdate, opmode=0xc
24745500: TXMODE, freq=868300000, len=23, SF=9, BW=125, CR=4/5, IH=0
25068940: RXMODE_SINGLE, freq=868300000, SF=9, BW=125, CR=4/5, IH=0
25134128: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
25245291: Setup channel, idx=3, freq=867100000
25245336: Setup channel, idx=4, freq=867300000
25247185: Setup channel, idx=5, freq=867500000
25250240: Setup channel, idx=6, freq=867700000
25253295: Setup channel, idx=7, freq=867900000
25256502: EV_JOINED
netid = 921102
25258759: engineUpdate, opmode=0x808
25261524: TXMODE, freq=867100000, len=20, SF=9, BW=125, CR=4/5, IH=0
Packet queued
25262523: engineUpdate, opmode=0x888
25331831: RXMODE_SINGLE, freq=867100000, SF=9, BW=125, CR=4/5, IH=0
25394330: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
25643508: EV_TXCOMPLETE (includes waiting for RX windows)
25643556: engineUpdate, opmode=0x800

This is the test with the clock adjustment call:

Starting OTAA New sample
RXMODE_RSSI
224: engineUpdate, opmode=0x808
Packet queued
1298: EV_JOINING
2467: engineUpdate, opmode=0xc
199891: engineUpdate, opmode=0xc
200195: TXMODE, freq=868500000, len=23, SF=7, BW=125, CR=4/5, IH=0
511503: RXMODE_SINGLE, freq=868500000, SF=7, BW=125, CR=4/5, IH=0
519002: Setup channel, idx=3, freq=867100000
519045: Setup channel, idx=4, freq=867300000
520635: Setup channel, idx=5, freq=867500000
523560: Setup channel, idx=6, freq=867700000
526485: Setup channel, idx=7, freq=867900000
529563: EV_JOINED
netid = 921102
531689: engineUpdate, opmode=0x808
534322: TXMODE, freq=867100000, len=20, SF=7, BW=125, CR=4/5, IH=0
Packet queued
535321: engineUpdate, opmode=0x888
596272: RXMODE_SINGLE, freq=867100000, SF=7, BW=125, CR=4/5, IH=0
658516: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
951196: EV_TXCOMPLETE (includes waiting for RX windows)
951239: engineUpdate, opmode=0x800

matthijskooijman · August 11, 2016, 11:22am

Without the adjustment, it seems that the RX1 window is opened slightly early:

>>> (445987-131511)*16 - 62000
4969616 us

This takes the time between TX and RX, measured in 16us clocks, subtracting 62ms for the TX airtime. The RX window starts at 5 seconds after TX end, reception is intended to start around the centre of the preamble (so slightly later than the RX window start). Perhaps the 62ms isn’t exactly correct, it would be interesting to see what LMIC calculates for that here, as well as what is written to txend here.

What hardware are you using?

gizmocuz · August 11, 2016, 2:38pm

Hardware: Arduino Uno with a Dragino Lora Shield (1.2, http://wiki.dragino.com/index.php?title=Lora_Shield)

Please find the requested debug info below:

Starting OTAA New sample
RXMODE_RSSI
219: engineUpdate, opmode=0x808
Packet queued
1294: EV_JOINING
2463: engineUpdate, opmode=0xc
98637: engineUpdate, opmode=0xc
airtime: 3856
98954: TXMODE, freq=868100000, len=23, SF=7, BW=125, CR=4/5, IH=0
LMIC.txend: 100062
412597: RXMODE_SINGLE, freq=868100000, SF=7, BW=125, CR=4/5, IH=0
478073: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
488145: engineUpdate, opmode=0xc
4354565: engineUpdate, opmode=0xc
airtime: 3856
4354885: TXMODE, freq=868300000, len=23, SF=7, BW=125, CR=4/5, IH=0
LMIC.txend: 4355736
4668271: RXMODE_SINGLE, freq=868300000, SF=7, BW=125, CR=4/5, IH=0
4733747: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
4743819: engineUpdate, opmode=0xc
8915252: engineUpdate, opmode=0xc
airtime: 7072
8915571: TXMODE, freq=868500000, len=23, SF=8, BW=125, CR=4/5, IH=0
LMIC.txend: 8919699
9232331: RXMODE_SINGLE, freq=868500000, SF=8, BW=125, CR=4/5, IH=0
9297710: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
9307782: engineUpdate, opmode=0xc
16254266: engineUpdate, opmode=0xc
airtime: 7072
16254587: TXMODE, freq=868100000, len=23, SF=8, BW=125, CR=4/5, IH=0
LMIC.txend: 16258587
16571218: RXMODE_SINGLE, freq=868100000, SF=8, BW=125, CR=4/5, IH=0
16636599: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
16646543: engineUpdate, opmode=0xc
24929268: engineUpdate, opmode=0xc
airtime: 12864
24929590: TXMODE, freq=868300000, len=23, SF=9, BW=125, CR=4/5, IH=0
LMIC.txend: 24939311
25252134: RXMODE_SINGLE, freq=868300000, SF=9, BW=125, CR=4/5, IH=0
25317322: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
25428487: Setup channel, idx=3, freq=867100000
25428532: Setup channel, idx=4, freq=867300000
25430381: Setup channel, idx=5, freq=867500000
25433436: Setup channel, idx=6, freq=867700000
25436491: Setup channel, idx=7, freq=867900000
25439699: EV_JOINED
netid = 921102
25441955: engineUpdate, opmode=0x808
airtime: 11584
25445369: TXMODE, freq=867100000, len=20, SF=9, BW=125, CR=4/5, IH=0
Packet queued
25446374: engineUpdate, opmode=0x888
LMIC.txend: 25452531
25515354: RXMODE_SINGLE, freq=867100000, SF=9, BW=125, CR=4/5, IH=0
25577918: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
25831583: EV_TXCOMPLETE (includes waiting for RX windows)
25831631: engineUpdate, opmode=0x800

Kibet · October 7, 2016, 3:25pm

I have been away for a while (holiday and more). I just tested this and it works nicely with SF7. Thank you for the bug fixing!

alexsh1 · October 14, 2016, 2:57pm

I have the following issue with LMIC on my RPi3:

pi@rpi3:~/arduino-lmic/examples/raspi/ttn-otaa $ sudo ./ttn*
ttn-otaa Starting
RFM95 device configuration
CS=GPIO8 RST=GPIO22 LED=GPIO23 DIO0=Unused DIO1=Unused DIO2=Unused
DevEUI : B8xxxxxxxxxxxxxxxx
AppEUI : 70B3D5xxxxxxxxxxx
AppKey : Fxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
RXMODE_RSSI
14:21:41: 606: engineUpdate, opmode=0x8
Packet queued
14:21:41: EV_JOINING
614: engineUpdate, opmode=0xc
411793: engineUpdate, opmode=0xc
411856: TXMODE, freq=868100000, len=23, SF=7, BW=125, CR=4/5, IH=0
728365: RXMODE_SINGLE, freq=868100000, SF=7, BW=125, CR=4/5, IH=0
793841: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
804257: engineUpdate, opmode=0xc
4523130: engineUpdate, opmode=0xc
4523187: TXMODE, freq=868300000, len=23, SF=7, BW=125, CR=4/5, IH=0
4839701: RXMODE_SINGLE, freq=868300000, SF=7, BW=125, CR=4/5, IH=0
4905177: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
4915592: engineUpdate, opmode=0xc
9366812: engineUpdate, opmode=0xc
9366868: TXMODE, freq=868500000, len=23, SF=8, BW=125, CR=4/5, IH=0
9686656: RXMODE_SINGLE, freq=868500000, SF=8, BW=125, CR=4/5, IH=0
9752036: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
9762451: engineUpdate, opmode=0xc
16799609: engineUpdate, opmode=0xc
16799665: TXMODE, freq=868100000, len=23, SF=8, BW=125, CR=4/5, IH=0
17119450: RXMODE_SINGLE, freq=868100000, SF=8, BW=125, CR=4/5, IH=0
17184830: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
17195250: engineUpdate, opmode=0xc
24465252: engineUpdate, opmode=0xc
24465342: TXMODE, freq=868300000, len=23, SF=9, BW=125, CR=4/5, IH=0
24791158: RXMODE_SINGLE, freq=868300000, SF=9, BW=125, CR=4/5, IH=0
24856346: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
24866796: engineUpdate, opmode=0xc
37921119: engineUpdate, opmode=0xc
37921220: TXMODE, freq=868500000, len=23, SF=9, BW=125, CR=4/5, IH=0
38247020: RXMODE_SINGLE, freq=868500000, SF=9, BW=125, CR=4/5, IH=0
38312208: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
38322631: engineUpdate, opmode=0xc
52958651: engineUpdate, opmode=0xc
52958746: TXMODE, freq=868100000, len=23, SF=10, BW=125, CR=4/5, IH=0
53295229: RXMODE_SINGLE, freq=868100000, SF=10, BW=125, CR=4/5, IH=0
53360033: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
53370481: engineUpdate, opmode=0xc
79742695: engineUpdate, opmode=0xc
79742784: TXMODE, freq=868300000, len=23, SF=10, BW=125, CR=4/5, IH=0
80079247: RXMODE_SINGLE, freq=868300000, SF=10, BW=125, CR=4/5, IH=0
80144051: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
80154480: engineUpdate, opmode=0xc
104734138: engineUpdate, opmode=0xc
104734201: TXMODE, freq=868500000, len=23, SF=11, BW=125, CR=4/5, IH=0
105099740: RXMODE_SINGLE, freq=868500000, SF=11, BW=125, CR=4/5, IH=0
105163776: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
105174191: engineUpdate, opmode=0xc
157643148: engineUpdate, opmode=0xc
157643206: TXMODE, freq=868100000, len=23, SF=11, BW=125, CR=4/5, IH=0
158008747: RXMODE_SINGLE, freq=868100000, SF=11, BW=125, CR=4/5, IH=0
158072783: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
158083201: engineUpdate, opmode=0xc
216662614: engineUpdate, opmode=0xc
216662672: TXMODE, freq=868300000, len=23, SF=12, BW=125, CR=4/5, IH=0
217070939: RXMODE_SINGLE, freq=868300000, SF=12, BW=125, CR=4/5, IH=0
217133439: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
217143858: engineUpdate, opmode=0xc
323541538: engineUpdate, opmode=0xc
323541596: TXMODE, freq=868500000, len=23, SF=12, BW=125, CR=4/5, IH=0
323949866: RXMODE_SINGLE, freq=868500000, SF=12, BW=125, CR=4/5, IH=0
324012366: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
15:48:06: EV_JOIN_FAILED

I have checked the keys many time over and over again. Anything else I should be looking at please?

galagaking · November 25, 2016, 3:07pm

Hi, just preparing for the Node building workshop (Setup with Pro Mini and RFM95/LMIC). I use the latest library. OTAA join lasts 7 minutes, if i use the ‘dirty’ fix: setDrJoin(DRCHG_SET,DR_SF7) in LMIC: several seconds!! Do you know why this dirty fix is stil needed? I will document the workshop, so it would be nice not to have too much ‘dirty fixes’. thanks, Frank @matthijs any clue?

JBraam · December 18, 2016, 9:10pm

Does this still work in the new backend? If yes: How?

Having an AppKey that can be used for initial OTAA for unknown nodes makes central provisioning possible…

578e295758953c7b0afd · January 5, 2017, 2:25am

Please tell what should I do to DEVADDR since I don’t need to fill it up. should I make it as note?? or fill it with something else??
thank you

mw12554 · January 5, 2017, 8:57am

For OTAA I just make DEVADDR { 0x00 } in LMIC. Like:

unsigned char DevAddr[4] = { 0x00 };

Maarten

578e295758953c7b0afd · January 5, 2017, 11:54am

did you change other part of the code?? or just change something else. I follow your method but it seems doesn’t work for my device. I am using RPi and Dragino LoRa GPS HAT… thanks

578e295758953c7b0afd · January 5, 2017, 11:56am

Thanks, I have some more question. for OTAA do I need change other part of the code or I just need to add APPKEY, APPEUI and DEVEUI to the code?? do I need to remove or add anything else??