Is it possible to more interesting statistics from a gateway than just the things you see in the packet forwarder log?
Things I would like to see for example:
statistics on the total number of received packets with good CRC, bad CRC, no CRC in the past minute/hour/day/week
total number of packets and bytes received/transmitted in the past minute/hour/day/week
total number of unique nodes received in the past minute/hour/day/week
statistics on the number of confirmed vs number of unconfirmed messages
maybe some kind of histogram of packet size
information on frequencies/channels used
etc.
I attended a build-your-own-gateway workshop and one of the things that was a bit tricky to find out was whether a gateway was actually working. Basically monitoring the number of packets per 30 seconds was the only way to check this independently from the backend.
For example, could I write my own local network server that only acts to keep statistics like these of forwarded packets (in combination with the poly packet forwarder)?
The log contains PUSH_ACK messages for data transmissions as well. If you look at the log:
##### 2016-09-07 18:51:37 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 0
# CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 0 (0 bytes)
# PUSH_DATA datagrams sent: 1 (502 bytes)
# PUSH_DATA acknowledged: 100.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 3 (100.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
### [GPS] ###
# Invalid gps time reference (age: 1473274297 sec)
# Manual GPS coordinates: latitude 53.18977, longitude 6.55738, altitude 8 m
##### END #####
INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 14 ms
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 13 ms
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 13 ms
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 13 ms
##### 2016-09-07 18:52:07 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 0
# CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
This a a situation where no data is received and forwarded. Immediately after the statistics are logged you will see a single [up] PUSH_ACK for the statistics.
When data is being forwarded the log looks like this:
##### END #####
INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 18 ms
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 14 ms
INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 13 ms
INFO: [down] for server router.eu.thethings.network serv_addr[ic]PULL_RESP received :)
INFO: [down] a packet will be sent on timestamp value 898695075
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 14 ms
INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 14 ms
INFO: [down] for server router.eu.thethings.network PULL_ACK received in 14 ms
As you can see the statistics PUSH_ACK is present, however there are two further [up] PUSH_ACK lines for the node, one for the OTAA registration request which results in the line ‘[down] a packet will be sent’ for the OTAA response and one line for a data packet transmitted by the node once the OTAA handshake completed.
@kersing thanks for your insightful reply! (as usual)
I’ll read up a bit more on the proposed NOC component.
Personally, I’d like to play with the idea of building a simple tool to get local statistics from the gateway directly and independently of the TTN backend, instead of just waiting things like this to be built by other people.
Looking at what the gateway software (packet forwarder) sends to the backend, it seems that a lot of interesting metadata is already there, and from the payload field we can extract things like the device address, message type (confirmed/unconfirmed) and FCNT.
So a possible first implementation could just act like a network server, but be passive in the sense that it just records the incoming packets in a database with an indexed timestamp. Things like getting a list of unique devices from the past week is then “just” a SQL query.
@bertrik
You solution is certainly possible, in fact the second link in my first message points to a post by ‘joris’ who has implemented what you are proposing augmented with a nice dashboard.