On february the 24th, the TTN backend had a outage, as I can see differnt gateways on different locations stopped transfering messages.
After this outage, different devices on different locations (that are handled by different gateways) started increasing their SF and now are stuck on SF12. I see downlinks on SF9 to the devices with an LinkADRReq but all devices are not answering to that LinkADRReq.
Is it possible, that TTN uses a RX2 downlink window after the outage that are not configured before the outage so that the devices are not listening on it? And how to solve this without driving to all the devices and resetting them manually so that they’re joing with a new OTAA?
Your best bet here would be to find a situation where the downlinks are going through a gateway you control.
Look at what downlink SF is being used, that would help distinguish standard from TTN-variant RX2.
Look at how the gateway responds (ie, gateway’s on-board logs) if you suspect the power level setting might be invalid (or valid but considered invalid by the gateway due to not being an exact match for a table entry)
Look at the frame count in the downlink, if it’s a suspiciously small number that might suggest it has been reset which would lead to nodes rejecting the downlinks. Also ideally verify the MIC, while it’s relatively unlikely enough downlinks were previously sent for the lower 16 bits to roll over, if that happened and the high bits were lost on the network side it would also lead to the node rejecting the downlinks.
Ideally you’d have a node experiencing this which you could visit and connect to get a serial debug output from or something similar…
