Data Storage not showing decoded data and only stores data two times a day

How to get started
9

Answering my own question: I guess they differ in null values?

It seems the Data Storage Integration does not support null values…

One can easily test by just returning some hardcoded JSON result in the Decoder, and then use Simulate uplink (with a random payload) on a device’s Overview page to trigger the Decoder and the integrations. No need to abuse the Fair Access Policy for that:

Simulate uplink

(To automate this, look at ttnctl devices simulate.)

Storage silently fails for this:

function Decoder(bytes, port) {
  return {
    "bytes": "ME9LIjE4MDk1Ljg3NzAibC9oIjE4MDk1Ljg3NzAia2cvaA==",
    "decode_data_hex": "0x30,0x4f,0x4b,0x22,0x31,0x38,0x30,0x39,0x35,0x2e,0x38,0x37,0x37,0x30,0x22,0x6c,0x2f,0x68,0x22,0x31,0x38,0x30,0x39,0x35,0x2e,0x38,0x37,0x37,0x30,0x22,0x6b,0x67,0x2f,0x68",
    "health_status": "OK",
    "mass_flow_unit": "kg/h",
    "mass_flow_value": 18095.877,
    "totalizer1_unit": null,
    "totalizer1_value": null,
    "totalizer2_unit": null,
    "totalizer2_value": null,
    "totalizer3_unit": null,
    "totalizer3_value": null,
    "volume_flow_unit": "l/h",
    "volume_flow_value": 18095.877
  }
}

Removing the attributes with null values, or using undefined, does work:

function Decoder(bytes, port) {
  return {
    ...,
    "totalizer1_unit": undefined,
    "totalizer1_value": undefined,
    "totalizer2_unit": undefined,
    "totalizer2_value": undefined,
    "totalizer3_unit": undefined,
    "totalizer3_value": undefined,
    ...
  };
}

I wonder if things changed, but I guess not.

I’ve always seen null values in the output of the Data Storage Integration, but that’s probably due to the following: changing the Decoder, or conditionally including specific attributes, also affects fetching existing/other data from the Data Storage Integration. Any attributes added after data was already stored, or outputted conditionally, are outputted with null values for old/other data. Likewise, any removed attributes are still outputted with null for new data (even when only new data is returned in the time period of the query).

In other words: it seems all returned items all have the very same attributes, though some may be null when not defined for that very item. That makes it looks like null was supported, but I guess it never was.

This does not explain why some items are stored without any decoded values, but I’ll leave that investigation to you. :wink:

Aside: note that the Decoder is executed upon receiving the data, not when fetching it from the Data Storage Integration. You can easily test by adding something like dummy: new Date().toISOString() in the output.

2 Likes