Wow, thank you so much! This was sending me insane. Of course, Google is full of matches but they’re all about jq needing an explicit filter, which is a different issue.
I have a simple pipe chain like so:
mosquitto_sub <params> | jq <filter> | my_script.sh
Which silently fails. After a million attempts to narrow down the issue I’ve come to the same conclusion as you - jq does not pass its output through a pipe or redirect (but works fine to stdout) when called on the command line.
There’s two interacting weirdnesses going on here, which I’ll try to illustrate:
Works
mosquitto_sub <params> | jq <filter>
Fails
mosquitto_sub <params> | jq <filter> | tee out.txt
Works
mosquitto_sub <params> | tee in.txt
tail -f in.txt | jq <filter>
Fails
mosquitto_sub <params> | tee in.txt
tail -f in.txt | jq <filter> | tee out.txt
Fails
mosquitto_sub <params> | tee in.txt
tail.sh in.txt | jq <filter> | tee out.txt
Works
mosquitto_sub <params> | tee in.txt
tail_and_jq.sh | tee out.txt
Works
mosquitto_sub <params> | jq.sh
Fails
mosquitto_sub <params> | jq.sh | tee out.txt
Fails
mosquitto_sub <params> | tee in.txt
tail -f in.txt | jq.sh | tee out.txt
Summary: mosquitto passes output to jq fine, but then output is not available. mosquitto and jq in isolation are fine. But to get output from jq when fed by mosquitto, there needs to be a file inbetween AND the reading of that file and parsing by jq must be in the same script.
I don’t understand at all why this is the case, but at least we have a workaround! Pinning down the root cause is proving extremely elusive.