CayenneLPP Encoder and downlink payload fields

Hi all,

In my application the cayennelpp uplink messages from my *duino devices are decoded by the built in TTN decoder and I can see the nicely formatted values in the device data view.

I also want to send cayennelpp formatted messages to my devices so I populated the payload fields in the downlink message. I can see nicely formatted values in the device data view but they don’t arrive at the device.

I’m not certain I have the names in the right format used by the uplink decoder.

TTNPayloadFields

Odd thing is I can send raw payloads to the device just fine

TTNPayloadRaw

I have been staring at the documents so long I must be missing something to get this to work.

Any suggestions, tips on debugging etc.

@KiwiBryn

Can you show us your downlink encoder JavaScript please

First of all: Please don’t use confirmed messages for every uplink. You are allowed just 10 downlinks including ACKs a day.

Question: do you send a downlink with a field defined analog_in_1? Because that assumes TTN will encode your downlink just as it decodes your uplink. And that’s probably a wrong assumption because cayenne will not send the data in that format to TTN, it sends the encoded array of bytes as specified by the LPP format.

Hi

I’m using the built in CayenneLPP encoder

TTNApplicationEncoder

So I think the internals of the encoder/decoder are not exposed.

Thanks

@KiwiBryn

Hi,

I’m using my own private gateway so I had assumed me trying different settings to see what happened wouldn’t be impact others. My other Seeeduino LoRaWAN devices run unconfirmed.

For uplink messages the CayenneLPP decoder unpacks message and in the JSON.payload they are available as payload infields.

In the documentation on downlink messages there is sample JSON for a payload using payload_raw

{
  "port": 1,                 // LoRaWAN FPort
  "confirmed": false,        // Whether the downlink should be confirmed by the device
  "payload_raw": "AQIDBA==", // Base64 encoded payload: [0x01, 0x02, 0x03, 0x04]
}

There is also sample JSON using payload_fields

{
  "port": 1,                 // LoRaWAN FPort
  "confirmed": false,        // Whether the downlink should be confirmed by the device
  "payload_fields": {
    "led": true
  }
}

I had assumed that the process was symmetrical, that if I named the fields “analog_out_1” etc. matching the LPP naming scheme they would get “automagically” encoded.

If I want to use cayenneLPP + payload fields on the downlink messages it’s starting to look like I’ll have to use a my own Cayenne encoder + decoder implementation…

Thanks

@KiwiBryn

No such thing I’m afraid. You own your own gateway, but you are using a community infrastructure, so in return for you not having to run the back end servers and so you can make use of other “own private gateways”, others devices uplinks will pass through your gateway. ANY transmissions your gateway makes due to confirmed uplink ACKs or downlinks will make it unable to hear any devices, yours or anyone else while that’s going on.

As for downlinks, here’s the documentation:

https://www.thethingsnetwork.org/docs/applications/cayenne/#receiving-downlink

Hi @descartes

I had read the myDevices documentation as part of trying to figuring out how to to build my own TTN integration.

I’m actually working on TTNV2/V3 HTTP & MQTT based integrations for Azure IoT hubs/Azure IoT Central.

I have spent a lot of time reading the docs and trying to figure out the implementation details of JSON payloads, MQTT topics etc. I have struggled where the docs gloss over some of the detail I need or are in the odd case incorrect.

Overall, the TTN people and community have been very helpful which is great.

@KiwiBryn

Context is very important, it changes the way people answer.

I’ve no experience with Azure so I don’t know if there are any particular issues, tips or tricks involved.

But for any / all of the integrations, the JSON payload is pretty much the same. MQTT, if you use a desktop client to watch messages scroll by or if you are using a client & processor written in Python, is still connecting via MQTT and acting on the events.

So it may be you need to go back a little and ensure you have an MQTT or HTTP Integration working at the most basic level first, then look to getting the code up to Azure.

@descartes

Some more context for you…

I have a working Azure IoT Central/Azure IoT Hubs HTTP Integration which TTN has tweeted about. It has been stress & soak tested with 1000’s of devices and 100,000’s of messages.

I have built an MQTT data API client which subscribes to uplink and publishes to downlink topics etc…

I have a client for the restful TTN Application Manager API and I have been looking at whether it is worth building a GRPC version.

Over the last couple of days I have done some more research and answered my own question.

I had a look at the Go code and learnt a lot as I figured out how the bit I was interested in worked. I haven’t done any Go coding so it took a while to get comfortable with the syntax.

In core/handler/cayennelpp/encoder.go there were

func (e *Encoder) Encode(fields map[string]interface{}, fPort uint8) (byte, bool, error) and func (d *Decoder) Decode(payload []byte, fPort uint8) (map[string]interface{}, bool, error)

Which was a positive sign.

Then in core/handler/convert_fields.go there are these two methods (I assume they are called methods in Go)

> // ConvertFieldsUp converts the payload to fields using the application's payload formatter
> func (h *handler) ConvertFieldsUp(ctx ttnlog.Interface, _ *pb_broker.DeduplicatedUplinkMessage, appUp *types.UplinkMessage, dev *device.Device) error {
> 	// Find Application

and

> // ConvertFieldsDown converts the fields into a payload
> func (h *handler) ConvertFieldsDown(ctx ttnlog.Interface, appDown *types.DownlinkMessage, ttnDown *pb_broker.DownlinkMessage, _ *device.Device) error {

Then further down in the second method is this call

var encoder PayloadEncoder
	switch app.PayloadFormat {
	case application.PayloadFormatCustom:
		encoder = &CustomDownlinkFunctions{
			Encoder: app.CustomEncoder,
			Logger:  functions.Ignore,
		}
	case application.PayloadFormatCayenneLPP:
		encoder = &cayennelpp.Encoder{}
	default:
		return nil
	}var encoder PayloadEncoder
	switch app.PayloadFormat {
	case application.PayloadFormatCustom:
		encoder = &CustomDownlinkFunctions{
			Encoder: app.CustomEncoder,
			Logger:  functions.Ignore,
		}
	case application.PayloadFormatCayenneLPP:
		encoder = &cayennelpp.Encoder{}
	default:
		return nil
	}

Which I think calls

// Encode encodes the fields to CayenneLPP
func (e *Encoder) Encode(fields map[string]interface{}, fPort uint8) ([]byte, bool, error) {
	encoder := protocol.NewEncoder()
	for name, value := range fields {
		key, channel, err := parseName(name)
		if err != nil {
			continue
		}
		switch key {
		case valueKey:
			if val, ok := value.(float64); ok {
				encoder.AddPort(channel, float32(val))
			}
		}
	}
	return encoder.Bytes(), true, nil
}

Then right down at the very bottom of the call stack in keys.go

func parseName(name string) (string, uint8, error) {
	parts := strings.Split(name, "_")
	if len(parts) < 2 {
		return "", 0, errors.New("Invalid name")
	}
	key := strings.Join(parts[:len(parts)-1], "_")
	if key == "" {
		return "", 0, errors.New("Invalid key")
	}
	channel, err := strconv.Atoi(parts[len(parts)-1])
	if err != nil {
		return "", 0, err
	}
	if channel < 0 || channel > 255 {
		return "", 0, errors.New("Invalid range")
	}
	return key, uint8(channel), nil
}

At this point I started to hit the limits of my Go skills but with some trial and error (well mainly error) I figured it out…

The field names need to be formatted like this (ignore the C# implementation details). On each alternate line are the bytes which arrive on the *duino LoRaWAN device.

Dictionary<string, object> payloadFields = new Dictionary<string, object>();
payloadFields.Add(“value_0”, 0.0);
//00-00-00
payloadFields.Add(“value_1”, 1.0);
//01-00-64
payloadFields.Add(“value_2”, 2.0);
//02-00-C8
payloadFields.Add(“value_3”, 3.0);
//03-01-2C
payloadFields.Add(“value_4”, 4.0);
//04-01-90

payloadFields.Add(“value_0”, -0.0);
//00-00-00
payloadFields.Add(“value_1”, -1.0);
//01-FF-9C
payloadFields.Add(“value_2”, -2.0);
//02-FF-38
payloadFields.Add(“value_3”, -3.0);
//03-FE-D4
payloadFields.Add(“value_4”, -4.0);
//04-FE-70

The downlink payload values are sent as 2 byte floats with a sign bit, 100 multiplier and can be decoded on an Arduino with code this

byte data[] = {0xff,0x38} ; // bytes which represent -2 
float value = lpp.getValue( data, 2, 100, 1);
Serial.print("value:");
Serial.println(value);

So for readers who have the stamina to get to the end of this long post it is possible to use the baked in Cayenne Encoder/Decoder to send payload fields to a device…

@KiwiBryn

4 Likes

Wow, KiwiBryn, supersleuthing there. Brilliant. Thank you for your determination. I may well end up on this thread again. --!