Disk budget

A datalogger that fills a disk takes the boat down. Not the logging — everything: the Brain cannot write its own state, services that need a temporary file start failing, and the failure looks like anything except a full disk.

This package has no retention policy of its own. It creates files and it never removes one. --maxRecordTime and --maxFileSizeBeforeCompression bound the size of one file; they do not bound the total, and no combination of them ever will. Bounding the total is the installer's job, and this chapter is how to do it with numbers rather than optimism.

That is a deliberate consequence of what this package is: legacy recording, enabled by hand for an investigation and disabled again afterwards. Nothing enables it at install time, and it is not intended to run continuously on a boat in service. The simplest disk policy is to stop the recorder when you are done and take the captures with you. Everything below is for the case where it has to keep running — a fault that only appears after days, say — and it is then a real risk to the filesystem, not a theoretical one.

The one-paragraph version

Left alone on a typical MUXEN bus, the raw recorder writes about 150 MB a day and keeps roughly 20 MB a day after compression. Both recorders enabled is twice that. Over a season that is several gigabytes, growing at a constant rate, with nothing to stop it. Decide how many days of history the boat needs, work out the space, and put a deletion rule in place before you leave.

How much one frame costs

The raw recorder writes one text line per frame. For a frame with a four-hex-digit identifier the line is:

31 + 3 × DLC   bytes

— 46 bytes for a 5-byte frame, 55 bytes for an 8-byte one, including the trailing CRLF. Measured across a real MUXEN capture the average comes out at 48 bytes per frame.

The decoded recorder writes one row per frame as well, so the number of rows is identical; the rows are wider because the values are expanded into named columns, and there is one header line per file. Treat its volume as the same order of magnitude and measure it rather than assume it.

Daily volume

At 48 bytes per frame, uncompressed, and at the measured gzip --best ratio for this traffic (about 7:1):

Frames per second	Raw per day	After compression
10	41 MB	~6 MB
36	150 MB	~20 MB
100	415 MB	~56 MB
500	2.1 GB	~280 MB
1000	4.1 GB	~560 MB

36 frames per second is what a real MUXEN installation measured, and is a reasonable starting assumption for a boat of ordinary size. It is not a promise: the rate scales with how many devices are on the bus and how often they talk.

Two corrections to apply to any figure in that table:

• The compressed column only applies to closed files. The file being written now is uncompressed, so at any moment there is up to --maxFileSizeBeforeCompression of uncompressed data on disk on top of the compressed history — 10 MiB by default for the raw recorder, and 10 MiB per open file for the decoded one, which keeps one open per device and frame kind.
• Both recorders enabled means both columns doubled.

Measure it on the actual boat

Two minutes of measurement beats any table.

Count the frames:

timeout 60 candump can0 | wc -l

Divide by 60 for frames per second, and read across the table above.

Or measure the recorder directly, which folds in everything — line lengths, compression, both recorders if both are running:

du -sb /var/lib/muxen/datalogger
sleep 600
du -sb /var/lib/muxen/datalogger

The difference over ten minutes, times 144, is the daily rate. Do this with the boat in its normal state: a bus at the dock with the systems off is not the bus under way.

How often files rotate

With the defaults, whichever limit comes first wins:

Frames per second	10 MiB reached after	Files per day
10	6 h 4 min	8 (the 3 h age limit wins)
36	1 h 41 min	~14
100	36 min	~40
500	7 min	~200

The crossover is at about 20 frames per second: below it the 3 hour age limit fires first and files are smaller than 10 MiB; above it, size decides and the age limit never fires.

File count matters on its own: the decoded recorder produces one series per device and frame kind, so several dozen files per rotation period is normal there, and a folder with tens of thousands of small files makes ls, du and any retention job slow.

Bounding the total

Pick one of these. Doing nothing is not one of them.

Delete by age — systemd-tmpfiles

The least surprising option, and the one to reach for by default:

sudo tee /etc/tmpfiles.d/muxen-datalogger.conf >/dev/null <<'EOF'
# path                          mode uid  gid  age
d /var/lib/muxen/datalogger     0755 root root 30d
EOF
sudo systemd-tmpfiles --clean

systemd-tmpfiles-clean.timer then runs it daily. Choose the age from the measured rate and the space you are willing to give up: 30 days at 20 MB/day is 600 MB; 30 days at 280 MB/day is 8.4 GB.

Check what a rule would remove before trusting it:

find /var/lib/muxen/datalogger -type f -mtime +30 | wc -l
du -ch $(find /var/lib/muxen/datalogger -type f -mtime +30) | tail -1

Delete by count

When what matters is "always keep the last N captures" rather than a number of days:

cd /var/lib/muxen/datalogger
ls -1t can0.*.log.gz | tail -n +200 | xargs -r rm --

Run it from a systemd timer. Note that deleting the oldest files does not change the recorder's behaviour, but deleting all of them resets the file index to 1.

Give it a partition of its own

The only mechanism that makes a runaway recorder harmless rather than merely slow. If /var/lib/muxen/datalogger is a separate filesystem — or a bind mount of one — then a recorder that outruns its retention job fills that filesystem and stops, and the rest of the Brain keeps writing. On a boat where recording is left permanently enabled this is worth the extra partition.

Do not rely on the file-size options

--maxFileSizeBeforeCompression 1048576 produces ten times as many files, each a tenth of the size, and exactly the same total per day. The option controls granularity, not consumption. Smaller files are still worth setting for a different reason — they shorten the window of uncompressed data that a power cut can damage.

What happens when the disk does fill

Two distinct failures, and the first one is silent.

Mid-file: the write fails and nothing says so. The recorder does not check the result of writing a row. The current file stops growing, the process stays up, systemctl status says active (running), and frames are lost without a message anywhere.

At the next rotation: the recorder exits. Creating the new file fails, the recorder prints

busmaster: failed to create file: /var/lib/muxen/datalogger/can0.42.log

and exits with status 0. Restart=always brings it back 30 seconds later, it fails the same way, and it loops. Because the exit status is zero the unit is never marked failed, so nothing that watches for failed units will notice.

Meanwhile everything else on the Brain that needs to write is failing too, and those failures are what you will actually be called about.

The check that catches it early:

df -h /var/lib
du -sh /var/lib/muxen/datalogger

Recommended posture

For a boat leaving with recording enabled:

• Enable one recorder unless there is a reason for both. The raw capture answers more questions and costs less.
• Measure the daily rate on the commissioned boat, not on the bench.
• Set a retention rule sized to the measured rate, and verify it fires once before you leave.
• Record the expected steady-state folder size at handover. A folder that later sits well above it means the retention job stopped, and that is much easier to see if somebody wrote the number down.
• If the recording is for one investigation, put an end date on it. The usual way this package fills a disk is a temporary capture that nobody turned off.