Lmic plus rfm95w staying awake for long periods

End Devices (Nodes)
1

Hi,

I’ve setup a small node that sends a one byte packet every 5 minutes. To get to 5 minutes it sleeps in a loop 50 seconds 10 times as the timer loop is a two byte counter. This works fine and it’s been running for a couple of weeks now. It wakes up and stays awake for a couple of seconds every 5 minutes, I know this because the led connected to the SCLK pin glows when it’s awake and spi is operating. However, I have seen that led stay on at times for minutes at a time, implying that either the hibernate didn’t work or that it was stuck in the lmic code for several minutes.

Has anyone else seen this sort of behavior? The hardware is a teensy LC and the rfm95w with the hallard branch of lmic.

On a similar note, I’ve also deployed the above with a pir sensor. The pir sensor draws upto 65ua is standby mode. This device was rolled out with 3x lithium ultimate AAA batteries but they were flat within a month. I would have expected much much longer, I’m wondering if the same behavior as the node that is simply sleeping between transmissions is going on.

Kim

2

I have had similar incidents with an interval from a few weeks. The first time two independent nodes got stuck exactly on the same time, consuming a lot of power. I posted this on the forum about a month ago:

https://www.thethingsnetwork.org/forum/t/behaviour-of-lmic-and-adr/6900

A few weeks later a single node changed its sending interval from 5 minutes to 30 minutes. Sending a downstream message fixed this behaviour. This incident repeated a few days later.

I added some debugging code to report the events, but it has not happened since that change :slight_smile:

The only cause I can think of is that some data is received that changes the LMIC state. I noticed that the beacon and ping LMIC code are NOT disabled by default, perhaps that causes this behaviour in rare circumstances.

1 Like
3

Interesting. The node that I saw stay active for minutes instead of seconds was at the time running on the kpn network. However, the node that ran 3x aaa batteries flat in a month’s time was running on ttn. The first node also locked up and wouldn’t come back out of a timer sleep, however at the time I put it down to the cpu getting into some strange state due to a static shock as I was handling it without an enclosure at the time and it happened very shortly after I picked it up. But who knows, I read you also had a node lock up.

Kim

4

Yes, I had a double simultaneous node lock up!

If you are still having the problem you could

define DISABLE_PING
define DISABLE_BEACONS

in lmic/config.h around line 43 to see if that fixes it.

1 Like
5

Ok, interesting. Although in my case I put the teensy LC into hiberate after sending a packet so that could only affect situations where a ping or beacon was sent just after I send a packet I guess. In any case, no harming in disabling it as it certainly can’t use it. Thanks.

Kim

6

Some more info on this. I’ve seen this now several times.I have nodes that wake both from timers and from a gpio interrupt. The timers always seem to come in, the gpio interrupt always wakes the node right away because the SPI clock is going through pin 13 on the teensy LC and so it makes the light go on. Once woken it should simply send a packet, that’s it. Confirm both by spectrum analyser and gateway logs show that the packet is not sent. Well… not sent for several minutes now it seems. Something is randomly holding up the sending of the packet to TTN. On the timer based sends I probably don’t notice if the send happens just delayed.

Maybe there’s an error in the code to handle duty cycling? In any case, for relatively time critical apps (Security sensors) this is a big deal.

There’s a nice challenge in here for someone to find this :slight_smile:
Kim

7

The LMIC code OS also takes care of duty cycle. When the MCU goes to sleep, this LMIC ticks that takes care of these duty cycle needs to be updated too (stops when you sleep). Myself is still looking into patching this to make it work properly with sleeping MCU.

8

Ok this is very interesting. That would make the next send be considered in essence “back to back” with the first. Still likely to be some “bugs” as a side effect of this use case as it doesn’t happen every time. Also I’m using SF7 so the duty cycle should only wait about 5s.

But thanks for confirming that there could be a problem in this area. Nice challenge for myself, though I don’t think I like to use the arduino ide for reading code so I’ll likely have to get familiar with arduino code in eclipse before I start :slight_smile:

Kim

9

When the mcu sleeps you will have to adjust timer0_overflow_count for original AVR microcontrollers, see the end of:

For other microcontrollers it will be different. Stm32 needs adjustment of systick_uptime_millis, see eg:

1 Like
10

Thanks very much tomtor for this helpful information. I’ll try it out and feedback the results when I have them.

Cheers,
Kim

11

Very interesting,
Would you mind detail this code, I imagine 8 is for 8s watchdog but why this 64 (prescaler?) ?

timer0_overflow_count+= 8 * 64 * clockCyclesPerMicrosecond();

Thanks

12

Hummm, that’s interesting, when I’m sleeping with my mini Lora Node I disable the timer0 and enable it back after sleep.

I’m doing this to achieve maximum low power and I never encounter this problem, is it repeatable or random ? I need to check.

Here a part of my sleeping code

/* ======================================================================
Function: goSleeping
Purpose : set the device to sleep mode
Input   : 
Output  : Global Status of what triggered wake up 
Comments: -
====================================================================== */
void goSleeping( uint8_t _wake_mode, uint8_t _wdt_period = APP_WATCHDOG_NONE)
{
  SERIAL_DEBUG.end();
  
  // light Off on board LED
  ulpn.setDevice(DEVICE_LED_OFF);

  // Turn off on board RGB LED
  ulpn.RGBShow(RGB_OFF);

  // turn off power from sensors 
  ulpn.setDevice(DEVICE_SENSORS_OFF); 

  // turn off power from RF Radio module
  ulpn.powerRadio(false);  

  // Disable all ATmel internal peripherals (for low power)
  ulpn.disableCPUDevices();
  
  // if we want to be wake up by the watchdog, set it up
  // immediatly after this call we're sleeping
  // ========================================
  ulpn.sleepDeviceWake( _wake_mode, _wdt_period); // zzz...
  // ========================================

  // We've been waked up (by a IRQ), disable ours until we done the job
  ulpn.setIRQ(IRQ_SWITCH_DISABLE | IRQ_EXTWAKE_DISABLE);

  // Enable back arduino core delay() function
  power_timer0_enable(); 

  // Enable sensors, RGB LED and onboard LED also
  ulpn.setDevice(DEVICE_SENSORS_ON );

  // Activate back Serial
  power_usart0_enable();
  SERIAL_DEBUG.begin(SERIAL_PORT_SPEED);

  ulpn.powerRadio(true);
}

Function disableCPUDevices for information

/* ======================================================================
Function: disableCPUDevices
Purpose : disable Atmel integrated devices (for low power)
Input   : -
Output  : - 
Comments: - 
====================================================================== */
void ULPNode::disableCPUDevices(void)
{
  // Disable ADC 
  ADCSRA &= ~_BV(ADEN)  ; 

  // disable Analog comparator  
  ACSR |= _BV(ACD); 
  
  // Disable digital input buffers on all ADC0-ADC5 pins
  //DIDR0 = 0x3F;    

  // set I2C pin as input no pull up
  // this prevent current draw on I2C pins that
  // completly destroy our low power mode

  //Disable I2C interface so we can control the SDA and SCL pins directly
  TWCR &= ~(_BV(TWEN)); 

  // disable I2C module this allow us to control
  // SCA/SCL pins and reinitialize the I2C bus at wake up
  TWCR = 0;
  pinMode(SDA, INPUT);
  pinMode(SCL, INPUT);
  digitalWrite(SDA, LOW);
  digitalWrite(SCL, LOW);  

  /*
  power_adc_disable();
  power_usart0_disable();
  power_spi_disable();  
  power_twi_disable();
  power_timer0_disable(); 
  power_timer1_disable();
  power_timer2_disable();
  */

  power_all_disable();
}

/* ======================================================================
Function: sleepDeviceWake
Purpose : prepare sleep mode for total power down and sleep
Input   : what IRQ could wake us
          Watchdog duration if we want it to wake us
Output  : - 
Comments: - 
====================================================================== */
void ULPNode::sleepDeviceWake(uint8_t mode, uint8_t wdt_period)
{
  uint8_t irq_mode = 0;
  
  // save current interrupt state
  uint8_t oldSREG = SREG;

  // switch all interrupts off while messing with their 
  // settings or doing something that can trigger one
  // we will sleep, WE DO NOT WANT to be interrupted before sleeping
  cli();  

  // Does the external device if any need to wake us ?
  irq_mode |= (mode & SLEEP_WAKE_EXT) ? IRQ_EXTWAKE_ENABLE : IRQ_EXTWAKE_DISABLE ;

  // Does the switch push need to wake us ?
  irq_mode |= (mode & SLEEP_WAKE_SWITCH)  ? IRQ_SWITCH_ENABLE : IRQ_SWITCH_DISABLE ;

  // Setting IRQ
  setIRQ(irq_mode);
  
  // Do we want watchdog to wake us with interrupt ?
  if (mode & SLEEP_WAKE_WATCHDOG) {
    // WDTCSR period msb bit watchdog period is not contiguous 
    // adapt wdt_period to be compatible with WDTCSR
    wdt_period = ((wdt_period & 0x08)?_BV(WDP3):0x00) | (wdt_period & 0x07);
    
    // allow changes on watchdog config
    // set interrupt mode and period, and do a fresh start
    WDTCSR = _BV(WDCE) | _BV(WDE) ;
    WDTCSR = _BV(WDIE) | wdt_period;    
    wdt_reset(); 
  } else {
    // we don't want to be waked by the watchdog, so be 
    // sure to disable it by changing the config
    WDTCSR = _BV(WDCE) | _BV(WDE);
    WDTCSR = 0;    
  }

  set_sleep_mode(SLEEP_MODE_PWR_DOWN);
  sleep_enable(); 
  
  // Sequence timed instruction don't try to optimize
  if (mode & SLEEP_BOD_OFF) {
    // turn off BOD (sequenced order see datasheet)
    // Don't change anything before we're waked
    MCUCR = _BV(BODS) | _BV(BODSE);
    MCUCR = _BV(BODS); 
    sei();          
    sleep_cpu();   
  } else {
    sei();          
    sleep_cpu();   
  }
   
  // ...........
  // ZZZZZZZZZZZ
  // ...........
  
  // Waked !!!
  sleep_disable();
 
  // Restore original Interrupts
  SREG = oldSREG;
}
1 Like
13

See

https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/cores/arduino/wiring.c

Not sure if my computation is correct, it is good if you double check.

14

@tomtor
Oh, I never thought it was incorrect, I’m just curious and would like to understand, at least trying to :wink:
Already checked timing in LMIC for arduino Zero, but never got it to understand the magic done ;-(

15

As already pointed by tomtor,

So the above formula may be inaccurate.

With a 1MHz CPU Clock, we get one Timer0 overflow every 64x256 µs.
So every second, we have 10^6/2^14 (roughly 61).
64 is a bit too fast

Do I forget something? Is it simply better to use a power 2 number?
Is the timing difference negligible to follow duty cycle policy?

16

Hi all
I know this thread is kind of stale but on the other hand, @Couin added something “recently”.
And, I do not see a resolution to the issue yet.

Thing is, I am experiencing the exact same thing with my Arduino Pro Minis and the RFM95 when putting the Arduino to sleep after transmissions.
After some time of running the nodes seem to randomly take minutes instead of seconds between queing a packet and the TX_complete.
Measuring the power needs, I found out that the RFM is on (4 mA) during that time causing the nodes to eat through the battery quite rapidly.
Of course I added the time0_overflow_count hack, adding back the amount of time I put the Arduino to sleep.
Thing is, this “cheating” seems to be related with the occurence of the effect in the first place.

To investigate, I added 600 seconds for every minute to the counter without putting the Arduino to sleep.
And this caused the problem to become enormous, causing huge delays after 2 or 3 sends already,

So, now my question:
Could it be that adding 8 seconds for every 8 seconds of sleep is too much?
And if so, why does it bother the LMIC?
As Couin pointed out, it might be more correct to only use 61 instead of 64 as multiplier.
Did anybody experience the behaviour and knows how to fix it?
Should I only use e.g. a multiplier of 60 to ensure I get more likely a slow running clock?
Sending every quarter of an hour to hour should not cause me problems with the duty cycles anyway.

17

As I supposed last time, it may be better to stick to 64.
My last post was more an explanation where this magic number comes from :slight_smile:

18

I remember reading some thoughtful discussions about how sleep should be handled.
It’s really easy to mess with LMIC scheduler.

I don’t remember this exact behaviour. I had issues I luckily managed to solve (can’t remember, it was nearly a year ago).
Could you share a (perhaps simplified) version of your code, the library and IDE used?

19

I did some more testing but do get nowhere.
I tried both the library from matthijskooijman and the supposedly newer (improved?) one here: mcci-catena/ arduino-lmic
but both behave the same.
The IDE used is Arduino 1.8.5.
Regarding the code, I am not sure if I can/should post a quite long code here. It is mainly the example otaa code with some modifications to put the Arduino to sleep.
What is very strange is, that the effect starts usually only after some hours.
Here is one example with a code that wakes up once per minute and tries to send once per hour:
Joining looks like that:

Starting
 sending...
4027: Packet queued
4080: EV_JOINING
11167: EV_TXSTART
332332: EV_JOINED
333199: EV_TXSTART
403949: EV_TXCOMPLETE (includes waiting for RX windows)
alt: 10:19:55
neu: 10:19:57
Received 3 bytes of payload
sent
 ... go to sleep
woken ... IRQ
-RTC- 20:0 ... go to sleep
woken ... IRQ
-RTC- 21:0 ... go to sleep

Then after one hour, a packet is sent:

-RTC- 58:0 ... go to sleep
woken ... IRQ
-RTC- 59:0 ... go to sleep
woken ... IRQ
-RTC- 0:0 sending...
151569065: EV_TXSTART
151569279: Packet queued
151639348: EV_TXCOMPLETE (includes waiting for RX windows)
alt: 11:0:1
neu: 11:0:11
Received 3 bytes of payload
sent
 ... go to sleep
woken ... IRQ
-RTC- 1:0 ... go to sleep
woken ... IRQ
-RTC- 2:0 ... go to sleep

This goes on for three hours and then this happens:

-RTC- 59:0 ... go to sleep
woken ... IRQ
-RTC- 0:0 sending...
278556535: Packet queued
329882220: EV_TXSTART
329952504: EV_TXCOMPLETE (includes waiting for RX windows)
alt: 14:13:41
neu: 14:14:1
Received 3 bytes of payload
sent
 ... go to sleep

The Packet queued message now comes way before the TX_START, because other than in the hours before, the LMIC seems to delay the sending for 13 minutes in this case.
It can be seen from the RTC printout that usually send is within the minute but here it is after 13 minutes.

Why is it doing this? And how can I avoid it.

The code where I schedule the sending is:

os_setTimedCallback(&sendjob, os_getTime() + ms2osticks(10), do_send);

and in the do_send I do this

Serial.println(F(" sending..."));
LMIC_setTxData2(1, message.getBytes(), message.getLength(), 0);
Serial.print(os_getTime());
Serial.print(": ");
Serial.println(F("Packet queued"));

message consists of 16 bytes put together by LoraMessage.h
After the send it runs the os_runloop_once(); in the loop() until TX_Complete and goes to sleep again.

20

What are you receiving in the downlink? Are those confirmed uplinks, application downlinks, or is TTN sending some MAC command that might confuse LMiC? (Click an item in TTN Console to see what’s sent.)

1 Like