Hi, how do you access your received data in a consistent manner, without missing messages?
Currently, I can access the data of my sodaq one, running the tracker software by three different ways:
Through staging.thethingsnetwork.org, with the payload functions enabled I can read it right away. But there is no history kept. I need to leave the browser open to get everything.
Through an Android app called MQTT Dashboard, but if the Internet connection is off or the app is stopped or in the background it will stop collecting data.
Through node red. But that requires my computer running which I don’t like 24/7. Or the node red runs in the cloud on IBM blue mix, but that was only free for a month. Not a usable option.
I have tried a couple of dashboards like freeboard, but they are not compatible with mqtt.
So how can I get all messages, without missing one, and without being online my self all of the time? This becomes even more important if I set the node to send only a few times per day, to save the battery. Then I don’t like to miss any message. How are you, fellow TTNers, doing this at the moment?
It’s a bit extreme to say that Bluemix is not a usable option because it’s only free for a month. If you want 24/7 operation, you should either run a server yourself or be prepared to pay someone for that.
That said, I think there’s a completely free alternative. The guys from zeit.co have this service called now that does give you free 24/7 for node.js apps (also for dockerized apps, although I haven’t tested that one). This mean you could run node-red on their servers, or even simpler, write a tiny little TTN client that receives the data and stores it somewhere, like one of the many free dashboards, or some free tier of any cloud DB, or any other form of storage, like appending to a Google Spreadsheets directly from node.js.
It seemed to me that Bluemix was a little bit overkill. But it looks like that there are no easy and free solutions that just works. And that surprises me! It should not be so hard to just get all messages. Or if space is a constraint, only get the last 10 or all messages of the last 24 hours. I don’t mind programming something that works for me, but expected that someone had already found a solution for this common problem.
Your suggestion for zeit.co is very valuable to me! I will give it a try! Thanks.
@EelkeVisser
The current (staging) environment does not include storage of messages. However if you check the roadmap you will see the future brings a database component and REST API for data. For now you’ll need to use MQTT.
you will always loose messages, or at least you should be prepared to accept that you will.
There is no guarantee that every message sent by a node will be received on one (or multiple) gateways. As more nodes appear, there will be a certain number of messages being missed.
We are currently testing with multiple nodes (20-50) on the KPN network and not all messages arrive - this is not a flaw of the network, just real life …
The whole idea of LoRa is that nodes are created in such a way that it is not that important when a few messages are not being received. E.g: a weather forecast service using sensors placed in the field will use all data available but when messages are missed out, they will be skipped (or interpolated) during the forecast calculations.
Hi @Rob, I understand that messages can get lost when they are ‘on air’. But my question was more about what we can use to retrieve all messages from the staging server. Current options I have found, will only work if I have some sort of server running that is online 24/7. For me the storage or database component on the roadmap will probably solve the issue.
There are many dashboards available online, but I haven’t found one that can interface through the MQTT protocol (without websockets) that staging uses. And it is also not clear to me if these dashboards will gather and store the data when I am not watching the webpage.
Yep, the free storage with short retention and access through a REST API will solve your problem. MQTT is still the preferred way for streaming, but for batch processing you might want to get chunks from a database.