This might help: https://www.thethingsnetwork.org/labs/story/save-your-data-using-nodejs-mqtt-mongodb
This will at least capture all your data and then allow analysis at your leisure 
1 Like
…but a gateway will capture many more packets… (Too bad there’s no such interface for gateways or the NOC.)
True in an urban environment - I live in the countryside and my gateways only capture data from my nodes talking to my app…
Get any gateway you like that supports the Semtech protocol, run a simple UDP capture program on your PC and forward packets to it using Semtech protocol. Part of the data is binary but the most important bits are json so simple code should work to capture and analyze.
Cheapest hardware solutions are probably RPi + RAK831/iMST for build it yourself or get the RG186/RG191 or the MultiTech MTCAP models.
2 Likes
I have been doing this for a while with my Lorank8. I just added a secondary UDP output to my server which hosts a UDP server program. Every received package is logged into a log file. the JSON is extracted and the payload is converted to readable hex. Just a simple C program.
That’s how I found out that a KPN user is sending every 20 seconds with a SF12 in my coverage area.
1 Like
I’m running the Semtech/UDP ttn-gateway software on RPi/IMST gateways. I connect the RPi/gateway to the Internet service via a cheap ethernet hub or an ethernet switch with port spanning or an ethernet tap. I then use Wireshark on a separate PC to collect the gateway UDP traffic into a pcap file and analyse all the traffic. I could run tcpdump/libpcap on the gateway RPi and collect the traffic but I prefer to leave the gateway unchanged.
1 Like
Sounds like you’re already doing what I’m looking for. 
In most cases by collecting the UDP traffic by either monitoring (via a hub) or forwarding it to an external server. What I would like to do is to parse the packets in the same physical box where I both have a database and a web server. Have done similar projects for Wi-Fi on RPi’s with a LAMP installation, but I’d rather use an existing HW for LoRaWAN. Will I be able to do this on either RG186/RG191 or the MultiTech MTCAP GW?
The RG1xx devices are ‘closed’, no access to add your own software without hacking them. The MultiTech devices allow adding software, however storage on flash is limited so probably not the best solution if you expect large volumes of data.
1 Like
Thanks. Not expecting large volumes of data, will just take snapshots for test purposes. Any idea how much storage that is available?
1 Like
This topic made me go and do some packet analysis. I set up a test “thing” (IMST iM880B) sending a test payload to TTN via an RPi/IMST-iC880A gateway running the “legacy” Semtech poly_pkt_fwd packet forwarder.
I collected uplink packets using Wireshark and, as Wireshark does not have a decoder for these packets, did some hand-analysis of an uplink packet using the poly_pkt_fwd source code. Copy attached in case anyone is interested.
5 Likes
Thanks a lot for sharing the packet capture. Will come handy.
When sniffing traffic in Wi-Fi, you can get something called a “Radio Tap Header” added on top of the 802.11 frames. These headers are provided by the wireless adapter and add useful information about the radio channel. Is it possible to get this kind of info out also in LoRa in some way?
I put “lora radio sniffer” into google and it reports about 560,000 results. I didn’t look at all of them but it looks like lots of people have adapted lora-based radio hardware into various types of sniffers. I suggest that you look there.
1 Like
I have done that too. As always with Google, you get a lot of good advice, a lot of bad advice and lot of so-so advice. I can use the Wi-Fi example again. Based on Google information I have ordered four different USB dongles the past year that should be capable of sniffing 802.11ac traffic on an RPi. Have spent probably a week in total trying to get drivers to work for these dongles (just so they can start capture traffic). In some cases it’s a HW upgrade from the manufacturer that removed the support, in other cases it’s simply wrong information in many DIY examples. Sometimes the driver kicks off the chipset but it drops packets. If I follow one of the DIY projects, I’m quite convinced I will end up a few hundred dollars poorer with a driver issue, a single channel capture or something similar.
So even though I agree Google normally holds the answer, I rather use these kinds of forums with experts to gather info and tips. Maybe someone points me at a good project they know works. I have already got a lot of useful info from this thread and hopefully it will be extended. If I build something that works I will also share it here.
Semtech’s own packet logger (util_pkt_logger) seems to work perfectly well. I’ve run it up here on spare RPi/iC880A gateway hardware.
Here’s the runtime dialogue and the first few captured packets:
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $ whoami
pi
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $ pwd
/opt/ttn-gateway/lora_gateway/util_pkt_logger
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $ sudo ./util_pkt_logger
loragw_pkt_logger: INFO: found global configuration file global_conf.json, trying to parse it
loragw_pkt_logger: INFO: global_conf.json does contain a JSON object named SX1301_conf, parsing SX1301 parameters
loragw_pkt_logger: INFO: lorawan_public 1, clksrc 1
loragw_pkt_logger: INFO: radio 0 enabled (type SX1257), center frequency 867500000, RSSI offset -166.000000, tx enabled 1
loragw_pkt_logger: INFO: radio 1 enabled (type SX1257), center frequency 868500000, RSSI offset -166.000000, tx enabled 0
loragw_pkt_logger: INFO: LoRa multi-SF channel 0 enabled, radio 1 selected, IF -400000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 1 enabled, radio 1 selected, IF -200000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 2 enabled, radio 1 selected, IF 0 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 3 enabled, radio 0 selected, IF -400000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 4 enabled, radio 0 selected, IF -200000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 5 enabled, radio 0 selected, IF 0 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 6 enabled, radio 0 selected, IF 200000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa multi-SF channel 7 enabled, radio 0 selected, IF 400000 Hz, 125 kHz bandwidth, SF 7 to 12
loragw_pkt_logger: INFO: LoRa standard channel enabled, radio 1 selected, IF -200000 Hz, 250000 Hz bandwidth, SF 7
loragw_pkt_logger: INFO: FSK channel enabled, radio 1 selected, IF 300000 Hz, 125000 Hz bandwidth, 50000 bps datarate
loragw_pkt_logger: INFO: global_conf.json does contain a JSON object named gateway_conf, parsing gateway parameters
loragw_pkt_logger: INFO: gateway MAC address is configured to AA555A0000000000
loragw_pkt_logger: INFO: found local configuration file local_conf.json, trying to parse it
loragw_pkt_logger: INFO: local_conf.json does not contain a JSON object named SX1301_conf
loragw_pkt_logger: INFO: local_conf.json does contain a JSON object named gateway_conf, parsing gateway parameters
loragw_pkt_logger: INFO: gateway MAC address is configured to AA555A0000000101
loragw_pkt_logger: INFO: concentrator started, packet can now be received
loragw_pkt_logger: INFO: Now writing to log file pktlog_AA555A0000000101_20171019T160004Z.csv
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $ cat pktlog_AA555A0000000101_20171019T160004Z.csv
“gateway ID”,“node MAC”,“UTC timestamp”,“us count”,“frequency”,“RF chain”,“RX chain”,“status”,“size”,“modulation”,“bandwidth”,“datarate”,“coderate”,“RSSI”,“SNR”,“payload”
“AA555A0000000101”,"",“2017-10-19 16:02:49.989Z”, 167529323, 868300000,1, 1,“CRC_OK” , 29,“LORA”,125000,“SF7” ,“4/5”,-91, +9.0,“403E2801-2680A70E-027BBC08-7E7B2724-7D4ED822-882CAAD3-59B61C9B-DB”
“AA555A0000000101”,"",“2017-10-19 16:03:34.443Z”, 211981435, 868500000,1, 2,“CRC_BAD”,237,“LORA”,125000,“SF7” ,“1/2”,-114,-11.8,“A653CD86-C12EA708-2F7D6AC9-CC081059-511F9FBD-D5BF914B-D6627C96-A03C2F46-EC780D3A-A6B9DD79-35AA5F45-502D99CF-E48F1C04-DA7014AE-B5800D8A-63201D33-CF9A0748-F710503E-A51AEB7D-4AA40981-FA54DB99-79AA6026-8CE9C249-6F99BB08-2CA590B4-A1B2A7FA-A0718586-629DDB3A-1EAD47F0-FD57B5C8-43B964FC-A45C7571-9CA08180-7D873E14-8B8DF629-11F69DAF-2436006F-C32DC846-92096B13-F7612439-E1D2C71B-F50AF862-BEFC960B-647AAAF1-4345F7B9-20558C17-CCBBE8D1-E48486BA-546BA9C9-0B8CA11E-7AF6347F-F7DD1BF1-B708040F-2DE4BDEE-8E145C65-0DC0C76E-96A93608-9A0B1BA8-177844D9-C5”
“AA555A0000000101”,"",“2017-10-19 16:05:50.011Z”, 347548731, 867300000,0, 4,“CRC_OK” , 29,“LORA”,125000,“SF7” ,“4/5”,-89, +8.8,“403E2801-2680A80E-02573111-AAC4D233-FF6A3894-41521806-C89DFE67-1B”
“AA555A0000000101”,"",“2017-10-19 16:08:16.243Z”, 493782891, 867100000,0, 3,“CRC_BAD”,203,“LORA”,125000,“SF7” ,"" ,-115,-11.8,“A0F04B7D-A9494EF2-F517A4CF-95BA5694-06A786A2-184B5250-8487CBEE-475BC60C-7C1D1751-80BC7F36-B53E4EDA-DEA5C3AA-96F83824-7D5B7976-A46EA8A9-F324E9AF-F1714AD4-5DACD87F-E94AA835-7C706C5E-B1139098-0E313EF7-E3C3AA74-4420D51B-D5591C6D-7A33E13A-881FE24A-2360B4BD-1CB18F69-C4C2EA56-4BD0E200-C2839EBF-776D0682-E8BED706-030E8F38-661A59D4-A58BAB1F-A5561041-C482E969-2EE18CED-739F395C-62AE4AA2-3C87BDE9-7846BEA7-0CF8C768-63315C2F-B8435DB7-61C5F995-0573BC8F-5CF0FCA5-BCED9A”
“AA555A0000000101”,"",“2017-10-19 16:08:49.993Z”, 527532444, 867500000,0, 5,“CRC_OK” , 29,“LORA”,125000,“SF7” ,“4/5”,-91, +7.0,“403E2801-2680A90E-023E938D-96D8A4C0-CA0E43C5-773963B6-00CB83F0-92”
“AA555A0000000101”,"",“2017-10-19 16:11:50.008Z”, 707546812, 867700000,0, 6,“CRC_OK” , 29,“LORA”,125000,“SF7” ,“4/5”,-91, +9.2,“403E2801-2680AA0E-0287B412-C81D0975-885F3361-381723C0-3CBD229C-D5”
“AA555A0000000101”,"",“2017-10-19 16:12:34.361Z”, 751899035, 868100000,1, 0,“CRC_BAD”,228,“LORA”,125000,“SF7” ,"" ,-115,-10.8,“7C9D23A2-7586029C-5046221A-A978E7E2-F7996F05-2AA86C9C-A2B1903D-D516FE65-CB0A3E51-9C55817B-9DF38A3B-6C19717B-6E927B1E-8938E1E5-47FDAE8B-AE7F78E5-FF4B4DEC-54AD4321-88CD4241-833014B6-8B439BCE-A823FCDC-BC25E682-407FDBCA-8F9181C6-845CFDD7-F2C2A84E-C1E6CE9A-D17166F6-E4153ECD-73F1A205-85490102-969C2E6D-DD7B0F4A-819A07A8-9F7561B0-10F8982E-50FD7568-FAD69C27-51357AB9-8CB8910C-A2F7E109-3FCD2287-AE1E8E3E-7B2EBCD7-FB2D6211-49107259-ED5545D0-167B3869-FB3CA588-A6DC823B-9BB9F90E-D0CF518D-00919636-3A9A3E94-1CF3345D-5B1C12C9”
pi@nes-acc-002:/opt/ttn-gateway/lora_gateway/util_pkt_logger $
2 Likes
I thought Now available: LoRa captures in Wireshark mean that wireshark was able to dissect LoRa packets? I haven’t tried it though.
1 Like
Yes, Wireshark is able to dissect LoRaWAN packets. However, it does so for packets which are in captures of linklayer type LoRaTap.
If you want to dissect LoRaWAN packets encapsulated in UDP, TCP or something like that I’ll have to add that. If somebody can supply me with a capture of such packets I’ll be happy to do so 
2 Likes
Maybe a stupid question, but will this also mean you can use dumpcap and then use tshark to extract stuff from the packets?
I have written a spec to encapsulate LoRa packets (https://github.com/eriknl/LoRaTap), I have some PoC hardware with a sx1272 that captures the raw data to pcap which can then be analyzed in Wireshark. Lack of time kept me from releasing…
1 Like
Yes, any part of Wireshark using the dissector can do so. But the dissector has to be set up to be ‘compatible’ with ethernet link layer captures before can recognize LoRaWAN packets on ethernet.
1 Like
Just in case parsing some hexadecimal or Base64 string from some log is easier, the following Node.js/JavaScript library can decipher the non-encrypted parts:
1 Like