Arduino LMIC and multitasking (on ESP32)

Hello,
I’m using a Heltec WiFi LoRa 32 (ESP32) board and I need some kind of multitasking as it has to monitor and write to a CAN bus, get GPS data, use Bluetooth LE and communicate with TTN.

I started with the ttn-otaa example and modified it.
I tried using freeRTOS tasks but weird things happen. The following code sends multiple join requests per second, I don’t understand why:

// LoRa libraries
#include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>

//------------- LoRa TTN (OTAA) config -------------//
// Device EUI (lsb format)
static const u1_t PROGMEM DEVEUI[8] = { hidden };
void os_getDevEui (u1_t* buf) {
  memcpy_P(buf, DEVEUI, 8);
}

// Application EUI (lsb format)
static const u1_t PROGMEM APPEUI[8] = { hidden };
void os_getArtEui (u1_t* buf) {
  memcpy_P(buf, APPEUI, 8);
}

// App key (msb format)
static const u1_t PROGMEM APPKEY[16] = { hidden };
void os_getDevKey (u1_t* buf) {
  memcpy_P(buf, APPKEY, 16);
}

// Pin mapping
const lmic_pinmap lmic_pins = {
  .nss = 18,
  .rxtx = LMIC_UNUSED_PIN,
  .rst = 14,
  .dio = {26, 33, 32},
};

// Data to be sent via LoRa and interval
static uint8_t mydata[] = "C2";
static osjob_t sendjob;
const unsigned TX_INTERVAL = 120; // seconds

void onEvent (ev_t ev) {
    switch(ev) {
        case EV_TXCOMPLETE:
            //Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
            // Schedule next transmission
            os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
            break;
    }
}

void do_send(osjob_t* j){
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        //Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
        // Prepare upstream data transmission at the next possible time.
        LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);
        //Serial.println(F("Packet queued"));
    }
    // Next TX is scheduled after TX_COMPLETE event.
}

void setup() {

xTaskCreate(
    ttnTask,           /* Task function. */
    "TTN Task",        /* name of task. */
    4096,                    /* Stack size of task */
    NULL,                     /* parameter of the task */
    1,                        /* priority of the task */
    NULL);                    /* Task handle to keep track of created task */
}

void loop() {

}

void ttnTask( void * parameter ){
    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();

    // Start job (sending automatically starts OTAA too)
    do_send(&sendjob);

    for (;;) {
        os_runloop_once();
    }
}

I also tried not using a FreeRTOS task. The ttn-otaa example works like this. But as soon as the arduino loop() function slows down (e.g. by updating the OLED display) the device cannot join the network (the join request is sent but no response received). I then tested with delay().

This works (device joins and payload is displayed on the TTN console):

void loop() {
    delay(1);
    os_runloop_once();
}

This does not work correctly (join request displayed on the TTN console but no message with payload). It might work after about 10 join requests:

void loop() {
    delay(10);
    os_runloop_once();
}

What is the proper approach I should use?
How have you integrated LMIC in your code so the arduino can do other things than only communicate with TTN?

Thanks !

Ben

From reading the LMIC pdf, it looks like one should create LMIC jobs, and call them using os_setCallback() or os_setTimedCallback().

The thing that somehow bothers me is this sentence:

Jobs must not be long-running in order to ensure seamless operation!
They should only update state and schedule actions, which will trigger new job or event callbacks.

Does someone know what “long-running” means exactly? Can I do all the processing that is not related to LoRa in these jobs? (e.g. CAN and Bluetooth stuff)

Up to now, this works well (might be of interest to others also starting with arduino-LMIC…):

#include <Arduino.h>

// OLED display library and pin mapping
#include <U8x8lib.h>
U8X8_SSD1306_128X64_NONAME_SW_I2C u8x8(/* clock=*/ 15, /* data=*/ 4, /* reset=*/ 16);

// LoRaWAN libraries (SX1276)
#include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>

// LoRa SX1276 pin mapping
const lmic_pinmap lmic_pins = {
    .nss = 18,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 14,
    .dio = {26, 33, 32},
};

// Jobs (used by LMIC OS)
osjob_t sendTTNjob;
osjob_t blinkLEDjob;

// Data to be sent via LoRa and interval
uint8_t mydata[] = "s7";
const unsigned TX_INTERVAL = 30; // seconds

// Send data to TTN
void sendTTN( osjob_t *j ) {
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        //Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
        // Prepare upstream data transmission at the next possible time.
        LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);
        //Serial.println(F("Packet queued"));
    }
    // Next TX is scheduled after TX_COMPLETE event.
}

// Display info on the OLED display
void displayMessage( String message ) {
    // Make sure the message is not longer than the display line
    if ( message.length() > 16 ) {
        message = message.substring(0, 16);
    }

    // Display the message on the bottom line of the OLED display
    u8x8.clearLine(7);
    u8x8.drawString(0, 7, message.c_str());
}

// Blink the LED at about 5Hz
void blinkLED(osjob_t *j) {
    // Toggle LED
    digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));

    // Call this function again in 100ms
    os_setTimedCallback(j, os_getTime() + ms2osticks(100), blinkLED);
}

// Function needed by LMIC
// TheThingsNetwork device EUI (lsb format)
void os_getDevEui (u1_t* buf) {
    static const u1_t PROGMEM DEVEUI[8] = { hidden };
    memcpy_P(buf, DEVEUI, 8);
}

// Function needed by LMIC
// TheThingsNetwork application EUI (lsb format)
void os_getArtEui (u1_t* buf) {
    static const u1_t PROGMEM APPEUI[8] = { hidden };
    memcpy_P(buf, APPEUI, 8);
}

// Function needed by LMIC
// TheThingsNetwork app key (msb format)
void os_getDevKey (u1_t* buf) {
    static const u1_t PROGMEM APPKEY[16] = { hidden };
    memcpy_P(buf, APPKEY, 16);
}

// Function needed by LMIC
// LMIC events handler
void onEvent(ev_t ev) {
    switch(ev) {
        case EV_TXCOMPLETE:
            // Schedule next transmission
            os_setTimedCallback(&sendTTNjob, os_getTime()+sec2osticks(TX_INTERVAL), sendTTN);
            displayMessage( "Message sent" );

            // EV_TXCOMPLETE includes waiting for RX windows. Process the received message here.
            if (LMIC.dataLen) {
                displayMessage( "Received message" );
            }
            break;

        case EV_JOINING:
            displayMessage( "Joining TTN" );
            break;

        case EV_JOIN_FAILED:
            displayMessage( "Join TTN failed" );
            break;

        case EV_JOINED:
            displayMessage( "TTN joined" );
            // Disable link check validation (automatically enabled
            // during join, but not supported by TTN at this time).
            LMIC_setLinkCheckMode(0);

            // Stop blinking the LED
            os_clearCallback(&blinkLEDjob);
            digitalWrite(LED_BUILTIN, LOW);

            // Start sending data to TTN
            os_setCallback(&sendTTNjob, sendTTN);
            break;
    }
}

// Arduino setup function
void setup() {
    // Digital pins init
    pinMode( LED_BUILTIN, OUTPUT );
    digitalWrite( LED_BUILTIN, LOW );

    // OLED display init (8 lines, 16 chars/line)
    u8x8.begin();
    u8x8.setPowerSave(0);
    u8x8.setFont(u8x8_font_pressstart2p_f); // https://github.com/olikraus/u8g2/wiki/fntlist8x8

    // Serial port init
    Serial.begin(115200);

    // LMIC init
    os_init();
    LMIC_reset();

    // Join the TTN network
    blinkLED( &blinkLEDjob );
    LMIC_startJoining();
}

// Arduino main loop
void loop() {
    // LMIC OS
    os_runloop_once();
}

Am I on the right track or should I code in another way?

Does someone know what “long-running” means exactly? Can I do all the processing that is not related to LoRa in these jobs? (e.g. CAN and Bluetooth stuff)

It’s a best practice to cut up jobs into small tasks. This allows the scheduler to give each task a chance to run.

It is possible to have parallel running rtos task with arduino lmic.
See my paxcounter example how to do it, it runs stable. Look in main.cpp, there you’ll find a structure you could adapt to your use case.

Thank you both for your help !

I had a look at the paxcounter, and found out that my code had two issues.

  • ttnTask never blocks, meaning that the IDLE task will never execute and will not feed the idle task watchdog. Solution is to add vTaskDelay(1) to the infinite loop (unless there is something cleaner).
    The serial port outputs the following:

Task watchdog got triggered. The following tasks did not reset the watchdog in time:
– IDLE (CPU 0)
Tasks currently running:
CPU 0: TTN Task
CPU 1: loopTask

  • ttnTask is running on core 0, arduino loop on core 1. I don’t know why this is a problem, but the solution is to pin ttnTask to core 1. (I also tried pinning it to core 0, it kept on sending multiple join requests even though the watchdog was not trigged anymore)

These are the two functions that I changed in my first example:

void setup() {
    xTaskCreatePinnedToCore(ttnTask, "ttnTask", 2048, ( void * ) 1, ( 5 | portPRIVILEGE_BIT ), NULL, 1);
}

void ttnTask( void * parameter ){
    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();

    // Start job (sending automatically starts OTAA too)
    do_send(&sendjob);

    for (;;) {
        os_runloop_once();
        vTaskDelay(1);
    }
}

It seems to work fine now, so I can carry on using freeRTOS. Thanks !

I also discovered problems when trying to set arduino-lmic running on core 0.
Don’t know, why.
Maybe sort of race condition if arduino core is executing parallel on core 1.

hi bdgraf
would it be possible, that you let us participate on your code on github?
I’ve the problem reading acceleration sensors (4 x 1000 a second) and the GPS (1 each second) and write them out. probably I could modify your multitasking source.
thanks

Great that you asked for the github, I wanted to do it for weeks… Here you go : https://github.com/marmotton/esp32-connected-car-lora

2 Likes

I want to reduce the time it takes between two LMIC_setxdata2 using a SX1262 module.

I don’t know what is the purpose of tx_rampup and os_setTimedCallback, is there any way to reduce from 30 seconds to less?

Do I have to change elsewhere?

Please do not hijack threads - you need to read a bit more of the forum before you can start your own topic.

Not without breaching the Fair Use Policy and possibly the law.

LoRaWAN is about sending sensor data every 15 minutes to a few hours, it’s not suitable for real time for all the reasons to be found on the forum.