How many per day?
Good point. Put this as meta value in the table. This to avoid at start up time and poor initial values in the table a frequent exchange of messages to improve the table.
Could you give an example of what a table could look like?
Note that my writing is an idea and not a full definition of a proposal. So with the discussion we try to collect the items for in the definition as well the feasibility.
This reply is just to give you more info to explain the idea.
Description of the table:
The table has some meta data:
The client/node has an unique number so this ID can be used to identify the table. But the there is a need to identify say the version. Probably as well more data as time of creation, last modified time, flag for “in transition”, and maybe more. Meta data can be sent “off line”.
The other part is an ordered array of _name_s and _tuple_s.
The name is to discriminate the different measurements, e.g. a string: , , e.g. DHT22@temperature and a unit, e.g. Celcius.
The tuple is to be able to calculate the amount of bits, the marshalling, needed to decode/encode the value of the measurement: ,,. In this way the amount of bits needed to encode the value is always a positive integer (minimal 0, maximal 2^^n -1),
e.g. say the first array element of the table is:
“DHT22”, “temperature”, “Celcius”, (12,37.5,-1) will allow 8 bits to denote values between 12.0 up to 37.5 (256 different values) . The first 8 bits can be decoded to the temperature of the DHT22 sensor, measurement in degrees Celcius.
Say the second array element is defined as “switch@lamp”,“On/Off”, (0,1,0) will encode as 1 bit. The payload will be 9 bits.
Say the third array element is defined as “switch@dimmed”,“level”,(0,7,0) will encode 8 dimmed levels as 3 bits. The pay load now becomes now 12 bits.
The table is defined something as: { meta_data: { …}, data: [ { type: string, unit: string, measurement: string, tuple: [ base float, width float, decimals integer ], …]}
Discussion:
Use a minimum value minus 1 to denote a null reading?
The client/node will send the table number with say version zero to denote an out of width value, in order to alarm the server to redefine the table for some entry.
1 Like
Like noted before: note that downloads are very limited. You certainly cannot send a downlink for every uplink.
The table will probably be quite stable. Yes the initial table should be defined carefully. I do not expect that the download is more frequent as say once a month. As the table updates is done as changes to the table the amount of data to update a table is very very limited.
The problem is merely: how to make sure that the table at the server and client side are the same? Once a week a table checksum?
Remind you: this “table/algorithm” synchronisation problem already currently exists.
I like this concept.
In a way, it is a generalized version of thesolarnomad’s serialization library (the library was mentioned earlier by @arjanvanb) .
thesolarnomad’s library currently support the following serialization types:
- Unix time (4 bytes)
- GPS coordinates (8 bytes)
- Unsigned 8bit Integer (1 byte)
- Unsigned 16bit Integer (2 bytes)
- Temperature (2 bytes, -327.68 to +327.67)
- Humidity (2 bytes, 0-100 with two decimals)
- Bitmap (1 byte)
With the suggested table approach, a serialization library could be built to allow any table to be used. This would mean that I can save some bits if I want to save humidity with no decimals, that I can have a temperature that goes higher than 327 degrees, etc. And all this without having to modify the serialization library. All I need to do is to define a new (static) table when generating new firmware for my node.
So even if requiring deploying a new firmware, the table approach would add value since I would get greater flexibility in the serialization.
A further step might be to distribute table updates over LoRaWAN. An approach could be to not send updates of whole tables, but only updates of table rows. If we can encode one table row in the downlink, the device could request table updates once per 24 hours. If there is an update to the table, the first row (including the row’s index for identification) is sent in downlink. This would limit the update pace to one row per table per 24 hours but I think that could be sufficient.
Following step: update the table: yes, I did not have in mind to send the whole table (full tables could be sent out of band as e.g. at the initial time).
With sending only the change of value (last value minus previous) one could save more bits (similar as is done in the round robin database e.g. RRD graphs). A combination of the generalized table approach might even save more bits.
But with sending datagrams or telegrams which can be lost this will disrupt the server end value. Maybe a checksum send from the client side can help to detect data transport misses. Has someone experience with this approach?
Hello.
Another way to reduce the radio overhead and thus the band is to use a mathematical way to compress the data. And this is to express the large numbers into another Base.
Radioamateurs use radio to send position for more than 20 years and it’s made at a 300 or 1200 baud.
There is a method for compressing the information in BASE91 then send it as an ASCI string.
Take a look, it is very simple: http://he.fi/doc/aprs-base91-comment-telemetry.txt
For some specific application where the precision is not a must, a simplified form can be used: https://en.wikipedia.org/wiki/Maidenhead_Locator_System
Best regards/73 de Adrian YO3HJV
No, this yields text. With LoRaWAN you should never send text as that only uses a part of the available 8 bits per byte. LoRaWAN is perfectly able to handle binary data; there is no need to first convert the bits to some printable encoding.
http://base91.sourceforge.net/ tells you, emphasis mine:
The overhead produced by basE91 depends on the input data. It amounts at most to 23% (versus 33% for base64) and can range down to 14%, which typically occurs on 0-byte blocks. This makes basE91 very useful for transferring larger files over binary unsafe connections like e-mail or terminal lines.
2 Likes
Using https://github.com/thesolarnomad/lora-serialization I was able to get a Position, Altitude and HDOP down to 12 bytes. In my case, this was for TTN Mapper.
- Position (8 bytes) - Using LoraEncoder::writeLatLng
- Altitude (2 bytes) - Using LoraEncoder::writeUint16 (0-65535m)
- HDOP (2 bytes) - Using LoraEncoder::writeHumidity (0.00-99.99)
The decode function at the TTN end was just decoder.js with the following added to the end (before the closing brace of decoder.js):
var ll = latLng(bytes.slice(0, 8));
decoded.latitude = ll[0];
decoded.longitude = ll[1];
decoded.altitude = uint16(bytes.slice(8,10));
decoded.hdop = humidity(bytes.slice(10,12));
return decoded;
1 Like
Hello,
even so most areas of the earth are above sea level it wouldn’t hurt to use a signed integer for the altitude.
The lowest points in West Europe are 7 meters under sea level (see Lammefjorden and Zuidplaspolder).
Thank you, Tore
If you do that, do be careful to parse the output from the GPS properly, its not unusual to see the output of a GPS report negative altitude when you near sea level and the quality of the fix is on the poor side.
hello @darren-oc, could you explain the “position error” ? I 'm quite of new on this, and I’m developping a LoRaWAN device with gps using the <TinyGPS++.h> library, for now, I’m just sending lat, lgn, speed, altitude. ( altitude because i’m working in mines and I need to know this value).
The chip itself can report a % of error? Could you help me to understand? Or send to me where I could read about it?
As standard the TinyGPS++ library will give you HDOP, horizontal dilution of position, which is a measure of accuracy, under 1.0 is good. Dont see how you can translate that into an actual percentage position error, since you would need to know where you are to work out the percentage error.
You can add customisation to TinyGPS++ so you could extract the VDOP from the GPGSA sentence for instance.
mmmm you want to use a GPS in a mine ?
I would imagine there will be no signals after a very short distance underground, how far underground have you had a GPS working ?
Nice to know, I ll try to find how can I get this HDOP right now. Percentage was just a “guess” but I understood, near to 1 is good ! The mine that I’m talking about is an open pit mine.
There is several different levels (as you can see in this image), in order to identify which level of the road my sensor is I ll use the altitude.
Ah, an opencast mine, we even have those in South Wales!
Whilst the opencast mine might be ‘open’ once you go down the hole the GPS can loose sight of satellites around the horizon, so fewer satellites in view, accuracy is reduced and height errors in particular can become significant.
One of those things you would have to test for your particular circumstance.
What are you going to be tracking the postion of ?
1 Like
I did some tests and we had quite of good results. The line in red represents a commercial GPS for vehicles and the “circular points” are the coordinates from my lora device.
The climatic conditions were good in the day of the test Temperature min 24 ºC , max 32 ºC, the relative humidity was 22%-57%.
We would like to track some grader,hidraulic shovel,sandvik mobile screen, trucks. I think can be a nice project for lorawan, because is an open space. I need to pay attention in the GPS accuracy as you said, do you have any other tip? =)
The mine tip has around 400 m (depth)
I’m working in Brazil !
Tracking moving trucks might not be a good fit for TTN, remember the fair access limit of 30 seconds air time per day.
Yes, for this specific project I was thinking about a private gateway pointing my packet forward to a private chirpstack server. Unfortunately I think that use TTN will not be possible.