I check with TTN and loriot.io
It’s fork from https://github.com/Lora-net/packet_forwarder
only with multi-server support (as old TTN poly_pkt_fwd)
Hi Eric @egourlao , it looks like you posted two different packet forwarders here, one at the top and another one mid-way. I have a Conduit AEP gateway and your two posts lead me to downloading 2 different packages:
admin@mtcdt:~# wget https://ttnreleases.blob.core.windows.net/packet-forwarder/master/multitech-conduit-pktfwd.tar.gz
admin@mtcdt:~# wget https://github.com/TheThingsNetwork/packet_forwarder/releases/download/v2.0.2/multitech-conduit-pktfwd.tar.gz
Can we just make this one package that is always the latest and greatest?
I’m currently having trouble getting your packet forwarder to work (I tried both of them). The gateway ID and Key are properly configured but on TTN the gateway still shows offline, even after an hour.
OMG I just wrote this long reply with detailed instructions and then I went to attach a picture and it deleted my whole post and just put the picture in it.
CTRL-Z does nothing but CTRL-F makes the search function in my browser not work and instead searches the whole forum when I’m only trying to search a page. This forum is the worst…
Hi - I can’t believe I am the first person to come across this, but I don’t see any reference to the following problem. I have a nanopi neo running xenial and have successfully installed go 1.9.
go get -u github.com/TheThingsNetwork/packet_forwarder
package github.com/TheThingsNetwork/ttn/api/gateway: cannot find package "github.com/TheThingsNetwork/ttn/api/gateway" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/gateway (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/gateway (from $GOPATH)
package github.com/TheThingsNetwork/ttn/api/discovery: cannot find package "github.com/TheThingsNetwork/ttn/api/discovery" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/discovery (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/discovery (from $GOPATH)
package github.com/TheThingsNetwork/ttn/api/fields: cannot find package "github.com/TheThingsNetwork/ttn/api/fields" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/fields (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/fields (from $GOPATH)
package github.com/TheThingsNetwork/ttn/api/protocol: cannot find package "github.com/TheThingsNetwork/ttn/api/protocol" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/protocol (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/protocol (from $GOPATH)
package github.com/TheThingsNetwork/ttn/api/protocol/lorawan: cannot find package "github.com/TheThingsNetwork/ttn/api/protocol/lorawan" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/protocol/lorawan (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/protocol/lorawan (from $GOPATH)
package github.com/TheThingsNetwork/ttn/api/router: cannot find package "github.com/TheThingsNetwork/ttn/api/router" in any of:
/usr/local/go/src/github.com/TheThingsNetwork/ttn/api/router (from $GOROOT)
/home/nicbkw/go/src/github.com/TheThingsNetwork/ttn/api/router (from $GOPATH)
There is a question similar to mine on StackOverflow (https://stackoverflow.com/questions/45843678/cannot-find-any-package) and the comments are shown below:
There is no protocol package in github.com/TheThingsNetwork/ttn/tree/develop/api. Referenced in github.com/TheThingsNetwork/packet_forwarder/blob/master/pktfwd/…. Quite a few references to packages that don’t exist. – captncraig Aug 23 at 15:26
@captncraig you found the references in master, but tried to resolve them in develop. They exist in master: github.com/TheThingsNetwork/ttn/tree/master/api – Adrian Aug 23 at 16:27
Hmm, it looks like develop is the default branch in github, and what gets pulled via go get. – captncraig Aug 23 at 16:35
Yep, looks like you’re right - package maintainer broke it by setting it to default develop. – Adrian Aug 23 at 16:53
So, help please!
I found help in TTN SLACK #packet-forwarder:
This error in particular occurs because the Go toolchain tries to retrieve the TTN API from the ttn GitHub repo, which has been removed in the latest versions. The best, at this point, is to ignore the error, and to just continue with the regular build instructions (https://github.com/TheThingsNetwork/packet_forwarder/blob/master/docs/INSTALL_INSTRUCTIONS/SPI.md). The error will fix itself with make deps.
1 Like
Hello everyone,
We’re going to put the development of our Packet Forwarder on hold for the moment. Everyone is still free to use it, and to choose the one they’d like to run on their gateway depending on the requirements. We realised developing our own packet forwarder wasn’t the right approach to the problems we wanted to solve in the first place.
We have started working on another solution for gateways. Our focus will be less on the packet forwarding logic, and more on the everyday maintenance of a gateway. Configuring a gateway, managing it, making sure it runs and stays connected is still a massive pain today (hence some discussions on the forum). There have already been great contributions from the community in that regard, such as the resin.io set up: we’re working on something that goes in the same direction of making gateways easier to manage.
4 Likes
So what’s the recommendation for the best packet forwarder going forward? Is the TTN protocol still in development moving forward or just the forwarder itself?
I’m setting up a few gateways at the moment and want to ensure they have the best supported solution.
4 Likes
We’re still developing our own protocol. We’ll update the TTN packet forwarder if there are minor changes to the API, but if we make major breaking changes to our protocol at some point, we might stop updating the TTN packet forwarder.
For this reason, I’d rather recommend installing Semtech-based packet forwarders if you’re aiming for low gateway maintenance. But depending on your requirements, you might still want to install the TTN packet forwarder for other reasons (e.g. if you need reliable connectivity).
As core team representative on this forum, I am appalled by your response. Will the core team please go have a meeting to discuss how best to support your users. There is an innate responsibility for TTN to provide a solution that meets the aims of the LoRa Alliance. TTN have built a great community over the past 2 years and facile comments about “Semtech code for low maintenance, TTN code for reliable connectivity” dismisses the efforts of the community to build stable solutions.
6 Likes
I think the most important question is: will the new developments still be open sourced, or will the community be left to fend with the Semtech forwarders while features such as secure forwarding and ease of gateway maintenance will be reserved for some premium scheme users?
3 Likes
I’m not happy with this response either. I’ve deployed a number of gateways recently using the new code and invested time and effort getting them setup and coverage mapped. Moving back to the Semtech forwarder seems a retrograde step but I need to ensure I have a supported solution.
There are already too many forwarder choices and the TTN solution was a good one for a stable and easy to configure setup. Is there some way support could be continued even through a community effort?
3 Likes
We’re planning to open-source what we’re working on, yes 
But we’re still early in the design and development stages. Our current idea revolves around an agent or a daemon that would run on-top of the Semtech packet forwarders, that could manage the PF, act as a bridge to the TTN protocol, and that could be used to control other parts of the gateway (OS, metrics, maybe even remote management?) - this again, in the objective of making gateways easier to manage.
Another change that motivated this decision was the gateway market, that’s getting more prolific - we’re seeing more and more cheap gateways and while that will help expand the global network (for community members that aren’t interested in building a RPi gateway or by the Things Gateway), it requires more documentation and integration work. It was manageable to have a single, go-to PF setup when there were a few go-to gateways like the Kerlink or the Conduit.
There’s now an ever-expanding list of commercial gateways with different systems and interfaces - some of them are even integrating different concentrator setups, with custom packet forwarders like the iBTS. We didn’t find it sustainable to try and integrate with every single one of the gateways - especially since manufacturers are now integrating the Semtech packet forwarder with most new gateways.
When I first read this, I was pretty outraged by the “We’re putting development of TTN packet forwarder on hold” thing. But not so much because of the actual stop of the development, but because you rushed into naming the Semtech one “legacy” everywhere, and now we’re left with legacy only.
I partially understand the need to concentrate efforts and work on what brings more value, but I disagree with the rationale you are putting forward. The gateway market might be expanding, but almost every gateway in the market uses the exact same SX1301 so they all work with the same code. And the beauty of open source is that you don’t have to be responsible for ALL the gateways out there, if you make it work properly for RPi and a generic SX1301, then others can pick up and fork/send you pull requests to extend it to other hardware.
From a technical perspective, having a layer on indirection between a piece of “legacy” code and TTN might work, but is not entirely efficient. I assume you would have the PF service sending UDP packets to localhost just to have another service re-packaging that into TTN’s protocol. Which sounds to me pretty wasteful and fragile (you suddenly have multiple, out-of-process points of failure in one logical piece of code).
The questions posed by @Epyon are also very appropriate and need a very strong response from your side.
Cheers
7 Likes
I actually think that the expanding number of gateways makes this new path much harder than the challenge of making a forwarder work.
Remote admin of different platforms is a much bigger challenge.
I’m almost thinking something along the lines of OpenWRT is needed with a very cutdown secure OS that runs the forwarder and management GUI with little else.
That would require custom Linux images for all those different gateway types with different hardware. I find building the MP forwarder for 4 different (may-be 5 in the future) to be a challenge and for the forwarder the base Linux system is in place. Creating an image for wildly different hardware platforms is not something I would want to tackle. (And what happens to the warranty of off-the-shelf products once you replace the OS image?)
4 Likes
I’m playing devils advocate a little here but I see it that most of the admin you might want to do is operating system related rather than the forwarder itself (assuming it’s running fine). The Pi isn’t ideal for this as it’s generally running behind NAT hence solutions like resin.io and others can make access easier but aren’t perfect.
Reducing the OS footprint limits the security risk and could make updating easier through an automated image based update like a wireless router etc or through the TTN console, thus ensuring all services were fully patched.
It’s a similar model to a wireless router, only that you may be managing them remotely.
I’d rather see the homebrew community running on limited hardware with better support and allowing others to branch support as needed ala OpenWRT
In terms of commercial gateways then the packet forwarder model or some middleware is the better option so not to interfere with any of the built in management
2 Likes
Thanks everyone. It may sound weird but I really appreciate pushback because that means that people actually care.
Gateways are still the biggest pain in the network and I think many Alliance members share that.
Before addressing your concerns, let’s take a few steps back.
Like everyone, we started with the Semtech UDP packet forwarder. Mostly because our focus was on other things. Now, UDP is not secure and not reliable. The forwarder is hard to configure, let alone remotely configurable easily, kept on introducing breaking changes (new formats, drop USB support) and as it’s written in C and the way it is written, it is hard to maintain. Also, the community developed forks to support multiple networks (a bad idea), new protocols (a good idea), we hosted one of those forks, but it diverged with Semtech’s reference design.
Because of all this, we saw a need for a new packet forwarder: use the HAL, a secure and authenticated TCP protocol, written in Go. Sounds simple but it’s not. Now, with a) the availability of so many commercial gateways and SPI settings and constrained hardware and b) the overhead and inevitable complexity, we decided to discontinue the forwarder.
But that is not the whole story. We are working on a major update of The Things Network, which will be announced soon. Part of this, is the Gateway Agent which runs on the Linux gateway and interfaces with the network server, and controls a Semtech based packet forwarder. It generates the global_conf.json file from data from the server: regional parameters, channel plan, the network server to connect to and gateway installation specific settings (max tx power). So it’s the Semtech packet forwarder + the Gateway Agent.
I don’t agree that we rushed with that, but maybe we can improve our communication here. Also, that specific fork is legacy.
We outsource the SX1301 interfacing to the Semtech’s reference design packet forwarder or the manufacturer’s variant of it, so that we can focus on a generic Gateway Agent. I believe that it becomes much easier instead.
Good question. There is no such thing as premium scheme users on The Things Network, and if that will ever exist, security will be there for everyone. The Gateway Agent will be open source. In fact, we are more committed to open source and there will be more open source than there is today, but more about that soon.
2 Likes
What about simply using @kersing’s forwarder then?
(TTN is asking for community help; seems he did a nice job, though I don’t have a gateway to use it on, so have not followed its developments nor have tested it. Even more, given the number of posts/feedback/support about it, at first I didn’t even know it was a different thing than the TTN forwarder, but I thought kersing was working on the “official” thing.)
7 Likes
I very much like the MP forwarder. I think that @kersing can confirm the issues I raised about maintainability and challenging build pipelines, especially when supporting 20+ gateways and versions. This forwarder is therefore more specific to MultiTech gateways. Also it uses our MQTT endpoint which is there to stay and is much better than UDP.
1 Like
Thanks for the reply. Does this mean the new agent will rely on the Semtech UDP protocol or will it convert to something more secure?
Is there a path that works for those of us with gateways now using the TTN forwarder to migrate while maintaining the same gateway IDs? This is important for apps such as ttnmapper.
2 Likes