Troubleshooting

muxen-boat fails quietly. It has no control socket, no status command and no reply topics: everything it knows it either publishes on MQTT or prints in the journal. Those two are the whole toolbox.

systemctl status muxen-boat.service
journalctl -u muxen-boat -n 100                     # startup log and errors
journalctl -u muxen-boat -f                         # follow it
mosquitto_sub -h 127.0.0.1 -t 'system/time' -v      # is the bridge alive
mosquitto_sub -h 127.0.0.1 -t 'device/interface' -v # is the CAN link alive
mosquitto_sub -h 127.0.0.1 -t 'device/#' -v         # is the boat talking
candump can0                                        # is anything on the wire

This chapter is organised by what you see.

Nothing is published at all

Not even system/time. Work down this list; each step rules out everything above it.

Check	Command	What it means
Is the service running?	systemctl status muxen-boat	see the next two entries
Was it skipped?	systemctl status muxen-boat shows Condition check ... was not met	the CAN interface does not exist — below
Is it looping?	Restart=always with RestartSec=30, so a failing start repeats every 30 s	read the journal for the attempt
Is the broker up?	systemctl status mosquitto	the daemon needs it at startup

The service is skipped, not failed

Condition check resulted in Muxen CAN Reader being skipped.

The unit carries ConditionPathIsDirectory=/sys/class/net/can0. If can0 does not exist, systemd skips the unit rather than failing it, which is why nothing appears in the journal.

ip -details link show can0

Two causes: the interface is genuinely down or absent, or the boat uses a different interface name. Overriding MUXEN_INTERFACE in a drop-in changes what the daemon opens but not the condition, which is hardcoded to can0 because a systemd condition cannot expand a variable. On such a Brain the condition has to be adjusted as well.

It starts and exits immediately

The journal shows the config: summary and then nothing. The daemon brings up MQTT first and CAN second, so the last thing it printed says which one failed:

• The summary appears, then it exits — the broker was not reachable on 127.0.0.1:1883, or the CAN interface named by --interface could not be opened. Check systemctl status mosquitto and ip link show <interface>.
• config: invalid IP address '<value>' or config: invalid UDP port '<value>' — a bad --udp-ip or --udp-port. The daemon exits 1.
• A usage block on stderr — an unrecognised option. The daemon exits 2. This is the one to suspect after editing a unit file: a typo'd flag is a usage error, not a request for help.

Note that --udp-ip, --udp-port and --json-plain are long options only; there is no single-letter form for them.

system/time ticks, but no device/… topics

The bridge is up and connected; the bus is the question.

1. Is anything on the wire? candump can0 for a few seconds. Silence means the problem is upstream of this daemon — power, wiring, termination or bitrate.
2. Is it the right interface? The journal's config: interface = line says which one the daemon opened. Compare it with the interface the devices are actually wired to.
3. Is the bitrate right? candump showing traffic while nothing is published means the frames are not MUXEN frames — a wrong bitrate produces exactly that. Compare data.config.bitrate in device/interface against what the boat's boxes use.
4. Is the frame decoded at all? The daemon accepts every frame from the kernel but publishes only the frames it has a handler for. A device sending a frame the daemon does not know is silent on MQTT and invisible in the journal. Run it by hand with -v — it prints its handler tables at startup, which is the definitive list of what it decodes.

One device is missing, the rest are fine

Likely cause	How to tell	Fix
The device is off or unwired	absent from candump can0 too	power and wiring
Its function code has no handler	present in candump, absent from device/#	it is not a model this daemon decodes — see The device catalogue
Its instance is not the one you expect	it publishes, but under device/<f>/<other>/…	mosquitto_sub -t 'device/<f>/+/#' -v shows every instance of that function
Two devices share function and instance	one topic, values jumping between two plausible sets	the topic is built from the frame's source address, so two devices at the same address publish to the same topic; re-address one of them

The last one is worth ruling out early, because a topic fed by two boxes looks like one box behaving erratically rather than like an addressing fault.

A value on a screen is frozen

Publications are retained on the broker. A device that stops talking — or a daemon that has stopped — leaves its last payload there, and a client that subscribes afterwards receives it immediately and shows a perfectly plausible number forever.

That is what expireAfterSec is for. Compare the payload's rxdate against system/time:

mosquitto_sub -h 127.0.0.1 -t 'device/5/0/life' -v -W 3
mosquitto_sub -h 127.0.0.1 -t 'system/time' -v -W 3

If the payload arrives once and never repeats, nothing is producing it. If rxdate is far behind the clock, the device is silent and the consumer is failing to apply the expiry — see Freshness in Web clients.

A topic for equipment that has been removed from the boat stays on the broker until it is cleared explicitly:

mosquitto_pub -h 127.0.0.1 -t 'device/5/3/life' -r -n

A command does nothing

There is no acknowledgement and no error topic, so this needs to be worked from both ends.

1. Is the topic one the daemon listens to? The daemon subscribes to exactly three filters:

device/+/+/command
device/+/+/reset
device/+/+/rtr/+

Anything else never reaches it. In particular the two path segments must both be present and numeric: device/8/command and device/lighting/0/command are not delivered.

2. Does that function accept commands? Sixteen do: 0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 17, 20, 21, 25, 27 and 32. Publishing to device/12/0/command is accepted by the broker and silently ignored by the daemon, because navigation instruments have no command handler.

3. Was the payload usable? An empty payload, a truncated payload or invalid JSON is rejected by the parser and dropped. So is a valid document with nothing actionable in it. Run the daemon with -v and the reason is printed:

Message	Cause
encoder: bloc8: per-channel command with invalid channel <n>, ignoring	requestOn, requestOff or dimming without a channel in 0–7
encoder: bloc8: command with no actionable field, ignoring	e.g. {} or a bare {"publishPeriod":50}
encoder: converter: current limit <n> A out of range, ignoring	setCurrentAC1Max / setCurrentAC2Max outside the encodable range
encoder: motor: <n> conflicting mode requests, forcing off	more than one of the request* mode flags set
encoder: invalid channel index=<n>	lighting command with a channel outside 0–5
encoder: hvac: payload without a "command" key, ignoring	the HVAC command needs both command and parameter
remote_sw: unknown relay value "<s>" / unknown injection value "<s>"	a string other than "auto"; use a boolean to force

4. Did the frame go out? candump can0 while publishing shows whether a frame was transmitted. If it was, the command reached the device and the device did not act on it — which is a device question, not a bridge question.

5. Did the device confirm? The only confirmation is the device's own next published frame. Watch it while you publish:

mosquitto_sub -h 127.0.0.1 -t 'device/8/0/life' -v &
mosquitto_pub -h 127.0.0.1 -t 'device/8/0/command' -m '{"channel":0,"on":true}'

Several commands also resolve contradictions rather than refusing them — requestOn and requestOff together drop requestOn, for example — so a command built by concatenating flags can end up doing the opposite of what was intended. The per-device pages state the rule for each.

device/interface reports a problem

The report is read from the kernel every 10 seconds and is the daemon's own view of the link.

Reading	Meaning
up: false, state: STOPPED	the interface is down, or the query failed entirely. ip link set can0 up is outside this daemon's remit
state: ERROR_ACTIVE	normal operation
state: ERROR_WARNING, ERROR_PASSIVE	the controller is seeing errors: wiring, termination or a mismatched bitrate
state: BUS_OFF	the controller took itself off the bus. config.restart_ms says whether the kernel will bring it back by itself
stats.bus_errors climbing	the same causes, seen as a rate
stats.restarts climbing	the link is recovering repeatedly

On a virtual interface the driver exposes no state at all, and the report says ERROR_ACTIVE because that is what this topic has always published for that case — not a measurement.

stats and config are always present so the payload keeps one shape. They read as zeros when the interface could not be queried, so an all-zero report together with up: false means "no answer", not "a healthy idle bus".

A web page cannot connect

Check	How
The broker's WebSocket listener is up	ss -ltnp | grep 1884
The listener configuration is installed	/etc/mosquitto/conf.d/mosquitto-boat.conf exists and mosquitto was restarted after install
The site includes the snippet	the nginx site has include snippets/muxen-ws-boat.conf;
The route answers	browser dev tools: the /ws/boat request should upgrade, not 404

A page served over HTTPS cannot open a plain ws:// socket, so an HTTPS interface must go through the proxy. The snippet relies on a $connection_upgrade map that it does not define itself; it has to be present in the server's http block.

The clock looks wrong

Two different producers write system/time:

• muxen-boat.service, once a second, in the form YYYY-MM-DDTHH:MM:SSZ.
• muxen-boat-init.service, once at boot, using date --iso-8601=seconds --utc, which produces the +00:00 offset form rather than Z.

Both are valid ISO-8601 and the shipped clients parse either. A consumer that string-matches on a trailing Z will disagree with the boot message only.

If the timestamps themselves are wrong, the Brain's clock is wrong. The daemon reads the system clock and does not correct it, and every freshness decision in every client is made against it.

Getting more detail

journalctl -u muxen-boat -f
/usr/bin/muxen-boat --interface can0 -v       # by hand, verbose
candump can0                                  # the wire itself
mosquitto_sub -h 127.0.0.1 -t '#' -v          # everything on the broker

Verbose mode prints the CAN and MQTT handler tables at startup, the topic filters it subscribed to, and one line per encoder decision. Stop the service first — two instances on the same bus both transmit.

FAQ

Why does a number on the screen never change? Almost always a retained message: the equipment stopped talking and the broker kept its last value. The screen is supposed to grey it out once it is older than its own expiry; if it does not, that is a bug in the screen, not in the boat.

Why is the screen blank for a second when I open it? The page waits for the boat's clock before trusting anything. Until the first system/time tick arrives it cannot tell a live reading from an hours-old retained one, so it shows nothing rather than something wrong.

I pressed a button on the screen and nothing happened. The command is fire-and-forget: nothing tells the screen it failed. Work it from the bus end — watch the device's own topic while pressing, and if it does not change, look at whether a frame was transmitted at all.

Do I have to restart anything after adding a box to the bus? No. The daemon holds no list of devices. A new box starts publishing on its own topic as soon as it sends its first frame.

Is it safe to run these commands while we are under way? Subscribing is: it only reads. Publishing to a command topic really operates the equipment — a reset really restarts a box. Keep those for a quiet moment.

Two batteries are installed but only one shows. Either the second is silent, or both are set to the same instance and are overwriting each other on one topic. mosquitto_sub -t 'device/5/+/#' shows how many instances are actually publishing.

Can I read the boat's data from a laptop? Only if the network allows it. The broker should not be exposed beyond the Brain; the supported route is the web interface through the reverse proxy.

Does it keep history? No. The daemon stores nothing — no file, no database. What you can see is the last value of each topic on the broker.

What does "device 5, instance 0" mean? Function 5 is a battery monitor and instance 0 is the first one. The list of function codes is in The device catalogue.

Why does the daemon appear on the CAN bus itself? It transmits as function 9, instance 0 when it sends a command, a reset or a frame request. That address is fixed and cannot be changed.

Tips

Leave a subscription running during commissioning. A mosquitto_sub -h 127.0.0.1 -t 'device/#' -v in a spare terminal is the single most useful thing to have open while equipment is being wired: every box announces itself the moment it is powered.

Check the instance of each box before wiring the next one. Two devices at the same function and instance share one topic, and the resulting data looks like one flaky device rather than an addressing mistake.

Record the device/interface counters at handover. On a healthy installation bus_errors and restarts stay flat. Knowing the normal figures turns them into a diagnostic later.

Subscribe to exact topics in a screen, not to #. The daemon publishes a lot; a page that subscribes to everything pays for every message it discards. The daemon itself learned this — it subscribes to three narrow filters rather than # for the same reason.

Never publish a command topic with the retain flag. A retained command is re-delivered to the daemon on every reconnection, and the equipment acts on it again.

Clear retained topics for equipment you remove. Publish an empty retained message to the topic, or it stays on the broker for the next screen that subscribes.

Use rtr to fill a screen instead of waiting. The slow frames only arrive periodically; a Remote Transmission Request gets one now.

Check the Brain's clock first when freshness looks wrong. Every expiry decision, in the daemon and in every client, is made against it.

-v on a live bus is verbose by design. Use it against a quiet bus, or for a few seconds at a time, and remember to stop the service before running a second instance by hand.

Keep system/# in a client's startup subscriptions. Without system/time a client has no clock, and treats everything it receives as stale.