3

I recently built a system with a btrfs root partition. The most compelling reason for adopting btrfs compared to ext4, in my case, was live copy-on-write snapshots, with near-zero latency. Compared to ext4, when a full system backup entailed taking down the system, mounting from a live distribution, and building a partclone image on removable media, the promise of snapshots is that the snapshot can be captured on the backup media while the system is live.

To my astonishment, no tool exists, however, that will capture the entire snapshot to a single file on external media, such that if the system is restored from that file, all applications will have the same view of the filesystem as before the crash (except that the other snapshots or subvolumes are missing).

The documentation suggests mirroring the directory tree using a tool such as rsync, or using btrfs-send/btrfs-receive to capture the incremental changes on another system. In the first case, I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured. The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another btrfs file system is available, which is not always the case.

Are any suggestions available for saving or restoring a volume-level image, similar to a partclone file, but representing only a select subvolume?

brainchild
  • 245
  • 1
  • 5
  • 18

2 Answers2

1

btrfs send and btrfs receive are exactly the tools you need.

I have always found it nearly impossible to recreate all the metadata in a file tree exactly by mirroring a file tree rather than imaging the file system, and have little optimism that a restore would be very smooth. I always find that some metadata, be it permissions, timestamps, hidden files, or so on, are not properly captured.

Have you tried the tools? I have had no such problems with btrfs. btrfs send and btrfs receive do not work with files, they work with entire subvolumes, having access to all Btrfs metadata natively.

The problem is compounded when the transfer occurs across file systems of different types. The other suggestion assumes another Btrfs file system is available, which is not always the case.

You will need another Btrfs filesystem only if you want to replicate the subvolume there. This is often desirable, so you can mount it and get access to its files. And this is unavoidable, if you want to backup your subvolumes incrementally. You need the exact previous snapshot(s) in both places to incrementally send the next one.

With full dump, however, you get a standalone stream, which is exactly an image (but not a browsable backup, unless you btrfs restore it). Work with it like with any other stream:

btrfs send /source/subvolume >/another/filesystem/subvolume-image   # just a file
# (or you can gzip it and/or send with nc on the fly, whatever)
# then later
</another/filesystem/subvolume-image btrfs receive /some/btrfs/directory

Where /some/btrfs/directory may belong to the same Btrfs filesystem as /source.

Kamil Maciorowski
  • 81,893
1

I'm surprised the original question has gone unanswered for this long.

If what you need is an external subvolume backup as a file so that it can be stored in any FS you can treat the output of btrfs send and input of btrfs receive as regular streams and redirect them to/from a normal file with dd.

I imagine something along the following lines would answer your question:

For backing up to file:

# create read-only temporary snapshot of volume
btrfs subvolume snapshot -r /path-to-sv/subvol /path-to-sv/subvol.ro

send the subvolume to file, compress with parallel ZSTD & monitor progress


btrfs send /path-to-sv/subvol.snap.ro | 

    pv -c | pzstd -16 |  pv -c | 

    dd of=/path-to-external-backup/subvol.zstd.back


delete read-only snapshot


btrfs subvolume delete /path-to-sv/subvol.ro

To restore subvolume from backup we run the process in reverse:

# read backup file and decompress the stream, redirect to temporary read-only snapshot
dd if=/path-to-external-backup/subvol.zstd.back | \
    pv -c | pzstd -d | pv -c | \
    btrfs receive /path-to-sv/

make a RW subvolume


btrfs subvolume snapshot subvol.ro subvol


delete temporary snapshot


btrfs subvolume delete /path-to-sv/subvol.ro

Notice btrfs receive only takes the path to the destination folder as argument, since the name of the subvolume is passed in the stream with the rest of the data. Of course monitoring with pv -c is entirely optional, as is using (de)compression.

I tried it on my linux box and it seemed to work without issue.

paperclip optimizer
  • 161