Feather M0 with LMIC takes more than 2 minutes to transmit

End Devices (Nodes) LMIC
18

I may have just realized what’s the difference you’re talking about in terms of the transmission scheduling: in mine, I execute the LMIC_setTxData2() inside the do_send function, while in yours you execute it directly in a function you defined.

I haven’t really understood how the do_send function works, in particular with respect to its parameter (osjob_t* j), but I think that using it may somehow leave to the LMiC library the decision on when to send the transmission with all its implications; whereas directly calling the LMIC_setTxData2() as you did maybe will enforce the library to actually send asap, without taking into account the timing and other parameters that may cause issues like the one I’m experiencing.

Regarding possible duty cycle violations I don’t think I should be in danger of violating them, since the payload I’m sending each hour is really small (10 bytes) and using the TTN airtime calculator the airtime should range from 60 to 1500 ms, in any case way below the 1% duty cycle established for most of the bands (which should correspond to a maximum of 36 seconds per hour).

Unfortunately I won’t be able to work on this project until monday, but thanks to the advices you and @cslorabox gave me I have plenty of tests to prepare and run on my Feathers!

Anyway thanks again for sharing your insights and the results of your work, which by the way is quite impressive!

(Also thanks for the last link you sent, I only noticed it now and I’ll keep it in mind for the tests I’ll do next monday)

19

The LMIC has a mini-task scheduler built-in - not a RTOS but an element of one. Internally it creates, starts, stops and deletes jobs that the schedule will then process according to the jobs criteria.

They create two jobs for the user - one for data collection and one for sending. The data structure is osjob_t* and that is used by sendjob but it could be called anything because it’s only a placeholder to attach the next scheduled time and the function to call, which is do_send. This requires the job structure as a parameter as that’s how the scheduler calls it, as the structure can hold other information if need be.

All that said, calling do_send should be similar to me calling LMIC_setTXData2 but with some initial checks so it doesn’t try whilst it is already processing. My state flags tell me it won’t be, as my structure is either sleeping, creating a payload or LMIC active.

But I don’t have visibility on all your code so I can’t be sure what else is going on - for instance, I don’t even define the ‘sendjob’.

All of this has to be in the context of me trying many many things summer 2019, including trying to adjust timings and so forth. I stripped out various elements from the main example, had debug of the LMIC on full and read the source code. I arrived at a stable solution that works that I’m happy I’ve not warped time & space to achieve, pragmatic but not ideal. Read the README.md for my archive to get a better picture.

At present for co-developer (usually juniors), leveraging Arduino libraries, I use Uno/Nano/Pro Mini/Nano Every with a RAK4200 module on AT commands. I have done a project using a Nano Every (48K Flash, 6K RAM) with an RFM95 using LMIC to good effect. I’m trying out the Adafruit Feather M0 with RFM95 as this is a first class citizen of the MCCI LMIC repro but like many other of these projects, they tend to stop at the basic examples - your requirement for sleep is very common but it’s not been included as there are other things for the developers to do and it’s possibly a bit application specific.

I’ve been to “Planet Change LMIC Timing” and lost days of my life as I detailed on my repro. I’ve learn’t much since then but I’m not tempted back, I’d rather waste my time re-writing a LoRaWAN stack for smaller devices. I was highlighting that with a forum search you can find other people with similar issues over the years. Question is, do you feel lucky (punk), well do you?

20

It’s critical to remember that this only attempts to stage a packet and set state variables.

it does not send anything!

That only happens if and when LMiC decides it’s appropriate to during a future execution of the runloop.

Typically you wouldn’t call this until you were ready to send, but the actual sending is still going to happen sometime later which takes minimum intervals into account.

The reason one would want to use LMiC’s scheduler to plan the transmission for a future time is so that there can be one scheduling mechanism, which you can then modify so that if it has nothing to do for a while it sleeps.

Otherwise you end up with two scheduling mechanisms that aren’t well aware of each other and both of which need to be made compatible with the fact that at least one of them (the custom one) can cause a sleep which effects both.

21

Just to update, I tried various solutions without luck:

  • using the code mentioned above, trying both the “classic” do_send function and a custom function (essentially identical but without the osjob_t parameter), with the loop containing the os_runloop_once() function and with different values for the sleep_subperiod, ranging from 5 seconds to few minutes for each “subsleep” with the total sleep time always amounting to 1 hour; this resulted in even longer time periods between each transmission, especially in the case of lower subperiods;

  • following the solution illustrated in this link, which basically consisted in adding the following lines of code to the hal.cpp file of the LMiC library:

    u4_t os_cumulated_sleep_time_in_seconds=0L;

    u4_t hal_ticks () {
    static uint8_t overflow = 0;
    uint32_t scaled = (micros()+os_cumulated_sleep_time_in_seconds*1000000) >> US_PER_OSTICK_EXPONENT;
    …
    ]

    void hal_sleep_lowpower (u1_t sleepval) {
    os_cumulated_sleep_time_in_seconds += sleepval;
    }

This last solution resulted in transmission periods with seemingly randomic lenghts, but again often longer than 2 minutes.

At this point, I’m thinking of downgrading the LMiC library to a prior version, since it seems like before the most recent updates it was more “loose” in its scheduling routines.

I’ll keep you (and everyone else reading this post :slight_smile: ) updated.

1 Like
22

I’ve found my actual Feather (think archeological dig in my workspace) - I’ll implement a sleep routine to see what happens.

Using the MCCI LMIC straight out the packet I can totally trash the duty cycle / fair use policy - like 20 seconds - which indicates the comment-free code in that section isn’t working as expected but it’s very hard to figure out why (or indeed what it’s mean’t to do, and lacking any design docs, what it was intended to do).

23

Any duty cycle enforcement would seem to be here:

This is really a case where it would make sense to add some logging; unfortunately that’s a bit harder in a .c source file in an Arduino library, but I believe there’s a helper method that could be called.

Specifically what you’d want to track would be txbeg getting set to anything other than 0 or LMIC.txend in the processing of this “do we want to transmit” clause.

Particularly you’d want to study anytime this test fails:

24

I got as far as this:

25

Indeed, LMIC.bands[].avail tracks the next allowed transmit per “band” - more on that at the end

It’s set here, along with the global duty cycle version

So what is a “band” ? Well, it seems to be a way to track various duty cycle limits on various sub-channel groups.

This mention seems to have the most explanation though I think the actual values in use may be coming from calls elsewhere to LMIC_setupBand()

Given this logic is required I’m not going to get too deep into defeating it; if one were always going to sleep for hours sure, but many things have stay-awake phases, too.

You know what would be really simple? Sleep for 57 minutes and 40 seconds.

26

Really, thank you for your commitment in searching that deep into the libraries, unfortunately I’m not very skilled in the great field that is the firmware development; I’ll try my best to understand the lines of code both of you mentioned but I’m not really optimistic about the results that I’ll get by doing that.

Yesterday I tried to load my script using a downgraded version of LMiC (2.3.2), but again with no luck :confused:
Anyway I found out that the actual problem may be in the TX_COMPLETE event firing: I tried a script with a TX period of 100 seconds and it seemed that after the first 2-3 executions, where the TX period overhead was acceptable (< 30 seconds), the Feather woke up from the deep sleep, sent its message and only then stayed on for another few minutes, probably waiting for the TX_COMPLETE event to fire, since it’s where the sleep flag is set; I derived this information just from listening to the USB port connection and disconnection sounds, since this time the Feather was still connected to my laptop during the script execution, and by observing the messages related to transmissions in the Telegram channel I’m using for debugging.

Believe me, I’d love to solve this issue with a simple fix like that one :sweat_smile: that would work in the case my only concern was to have a more precise TX period, but actually my problem is that if the node stays on for more than 2 minutes, instead of the few seconds that many report with the Feather M0, its battery life will be drastically affected, and sadly that’s a key aspect of my project.

27

As this is the heart of the matter, probably worth writing the exact timings down - what you see in the serial port (the Arduino IDE can add timestamps) against the uplink timestamp - there should be no reason for TX_COMPLETE to not follow on ~2.5 seconds after the uplink - so something is going wrong if it isn’t.

If you can post all your code (without the keys/EUI’s), I can try it here.

28

I’d really appreciate it, thanks!
In order to replicate my configuration you should connect the IO1 pin to pin 6; regarding the sensors they shouldn’t have anything to do with my problem and even if you don’t connect anything to the board the firmware should handle their absence by sending some special values.
I had to rename the files to fit the format restrictions imposed by the forum, obviously you’ll need to remove the final “.txt” :sweat_smile:

Firmware_FeatherM0.ino.txt (9.1 KB) myHeader.h.txt (2.1 KB)

1 Like
29

Hi @descartes, did you manage to try the code on your Feather M0?

30

Yes, I’ll have to dig it out and re-run the tests as I don’t appear to have recorded the results anywhere.

31

Ok perfect, I’ll keep an eye on this post for the results then! Thanks for your time :slight_smile:

32

I’ve re-run the ABP test sketches as a first pass. I’m taking the original ABP examples from the latest MCCI LMIC and adding my own sleep code, see it here at:

Find / Grep for “Nick” to see where changes have been made.

I previously ran it without the sleep loop at 600 seconds and was seeing about 2 to 3 seconds variation in transmit time.

With the loop in the gist I’ve linked to that provides some user feedback with 10 seconds of sleep x 60, a flash of LED, rinse, repeat, I’m seeing 11 minutes 6 seconds, so the waking, flashing and sleeping is adding an overhead of around 10% - I’d expect that to diminish if I sleep for longer, but most humans, including me, can’t wait that long for some feedback, so your mileage will vary. However it is still consistent.

I’ll run OTAA tests in the morning.

Overall, you should think about starting with a plain vanilla out-the-box sketch, put some instrumentation (serial prints) in to track what’s taking up the time and then start adding in you original code modules for sensor readings and so forth.

HOWEVER, sleeping the Feather really borks the serial port - my custom serialBegin does its best to re-establish comms with the host computer for debug purposes, but as the Arduino IDE isn’t hugely performant, it’s easy for it to miss a wake up & serial print, especially if the host is busy or prone to sleeping.

If you look at the source for the ArduinoLowPower you’ll see that it does all the RTC & epoch ‘stuff’ for you - so the RTCZero code you have is somewhat redundant and may be confusing the issue / board / yourself.

33

First of all (again) thanks for your time.
I’m looking at the gist you posted, trying to understand what it is that breaks the timing in my case since I was almost sure that the issue was the LowPower.deepSleep. Are you using the standard LMiC library with no modifications to it?
Anyway, I’ll try to follow your advice regarding starting with a new sketch and see what I get debugging it.

For what concerns the RTCZero stuff, I’ll try to remove it if possible but as you can see from the goToSleep function in my original post I’m using it to keep track of the time elapsed during a sleep in the case such sleep cycle gets interrupted by an interrupt. From tests I ran a while ago (so I may be wrong on that, I’ll check again) the LowPower.deepSleep function stops its execution without resuming it when an interrupt is triggered. I’m thinking now that I may solve that issue using some kind of timer to indicate the end of the sleep cycle, maybe again with RTCZero - but that shouldn’t confuse things as what I’m doing now.

34

Yes, it’s all untouched out-the-box MCCI LMIC with the sleep code added. From my perspective, the core Arduino, Feather, LowPower and LMIC work as expected. So it will be a case of adding in some more code and testing.

I would start with a sketch that does all the sensor work that has plenty of prints in it so you can see how long things actually take although I doubt they add up to the missing 2 minutes.

I understand/understood what you were aiming for with the RTCZero - if you can bear to wait a few days I can add that in to my code as it is a useful technique - normally I do a send when I get an interrupt as I’ve not done any pulse counting on battery powered device.

LowPower is a small library so you can see exactly what it’s doing at a glance - on the SAMD, sleep & deepSleep are the same and from my reading, it is stopped by an interrupt so using the RTC is the best way to track remaining time.

35

I still have to do the tests with the fresh code since I’ve been busy setting up another device for a test deployment at the house of a colleague but to my great surprise, this device has been transmitting since yesterday at 17:47 without experiencing this “bug” of the 2’ 30" transmission time.
It transmitted correctly the first 2 times with a 8" offset between the arrival times (which I consider to be a great result, I expected even worse), then its packets went lost, probably because it has a “jumper antenna” (which I imagine to be much “weaker” wrt a proper antenna), it’s at 2-3 km from the nearest GTW and until this morning it was not placed outside the house.

This morning at 8:49 its packets started to be received again with an offset of 7-8" between each transmission (just a few minutes ago today’s fifth packet arrived); also the offset accumulated during the period in which the packets weren’t received reinforces the hypothesis that the transmission time has been more or less constant:

14 transmissions between 18:47:16 (excluded) to 08:49:07 (included), total offset = 111 s
offset per transmission = 111 / 14 = 7.9"

The only changes I applied to the code are the adaptation of my loop to the one you proposed in your gist (see code below; I also implemented the small modifications you made to the setup and the TX_COMPLETE event) and some little modifications to the sensor initialization and measurements (mainly some brief delays), which I don’t think can be relevant to the LMiC behaviour. I’ll study the differences between your state handling and the one I was using (which can be observed in the code I posted here) but, for now, it seems like it could have solved my issue!

void loop() {    
  os_runloop_once();

  if (State == STATE_OK_TO_SLEEP) {
    goToSleep(sleep_subperiod);
    sleep_cycles_done++;
    
    // If enough sleep cycles have been completed and the TX interval has been exceeded
    if (sleep_cycles_done * sleep_subperiod >= TX_interval) {
      State = STATE_TIME_TO_SEND;
    } 
  }
  
  else if (State == STATE_TIME_TO_SEND) {
    detachInterrupt(digitalPinToInterrupt(CONTACT_SENSOR_PIN));
    State = STATE_SENDING;
    sleep_cycles_done = 0;
    
    do_send(&sendjob);
  }
  
  else if (State == STATE_SENDING) {
    // This section runs for the duration of the radio doing it's thing
    if ((millis() & 256) != 0) digitalWrite(LED_BUILTIN, HIGH); else  digitalWrite(LED_BUILTIN, LOW); // Flash the LED
  }
}
36

Hi everyone, I’m posting just to thank everyone who contributed to this post (especially @descartes, your insights were really helpful!).
The firmware of the EN was a key part of the final thesis for my master in computer engineering, thanks to which I officially graduated last tuesday, achieving the maximum score for what concerned the thesis evaluation.
So, again, thank you and all the best! :slight_smile:

P.s. if anyone is interested in the thesis, which is about the design and development of a prototype for a monitoring system using LoRaWAN and IoT technologies, I uploaded the pdf here.

1 Like
37

Your very welcome!

Thank you for giving us the feedback, it’s always good to know when we’ve helped.

Good luck with your future plans.

1 Like