“No space left on device” error despite having plenty of space, on BTRFS.
Welcome to the world of BTRFS. It has some tantalizing features but also some infuriating issues.
First off, some info on your setup, it looks like you have four drives in a BTRFS “raid 10” volume (so all data is stored twice on different disks). This BTRFS volume is then carved up into subvolumes on different mount points. The subvolumes share a pool of disk space but have separate inode numbers and can be mounted in different places.
BTRFS allocates space in “chunks”, a chunk is allocated to a specific class of either data or metadata. What can happen (and looks like has happened in your case) is that all free space gets allocated to data chunks leaving no room for metadata
It also seems that (for reasons I don’t fully understand) that BTRFs “runs out” of metadata space before the indicator of the proportion of metadata space used reaches 100%.
This appears to be what has happened in your case, there is lots of free data space but no free space that has not been allocated to chunks and insufficient free space in the existing metadata chunks.
The fix is to run a “rebalance”. This will move data around so that some chunks can be returned to the “global” free pool where they can be reallocated as metadata chunks
btrfs fi balance start -dusage=5 /mnt/durable
The number after
-dusage sets how aggressive the rebalance is, that is how close to empty the blocks have to be to get rewritten. If the balance says it rewrote 0 blocks try again with a higher value of
If the balance fails then I would try rebooting and/or freeing up some space by removing files.