Send stream format

Send stream format represents a linear sequence of commands describing actions to be performed on the target filesystem (receive side), created on the source filesystem (send side). The stream is currently used in two ways: to generate a stream representing a standalone subvolume (full mode) or a difference between two snapshots of the same subvolume (incremental mode). The stream can be generated using a set of other subvolumes to look for extent references that could lead to a more efficient stream by transferring only the references and not full data.

The stream format is abstracted from on-disk structures (though it may share some BTRFS specifics), the stream instructions could be generated by other means than the send ioctl.

Data types

Raw data types. Integer values are stored in little endian byte order.

Meaning

Size

Name

unsigned int

8 bit

u8

unsigned int

16 bit

u16

unsigned int

32 bit

u32

unsigned int

64 bit

u64

variable length binary data

variable

data

variable length string

variable

string

UUID

16 bytes

uuid

time specification

64bit seconds, 32bit nanoseconds

timespec

Stream structure

The stream starts with a descriptor bytes btrfs-stream followed by version stored as little endian u32. Then the sequence of commands starts. The expected start is a subvolume or snapshot followed by commands to change the data.

Command structure

  • u32 length

  • u16 command type

  • u32 checksum (CRC32C)

Note: the checksum initial seed is not 0xFFFFFFFF but 0x0. That is a slight mistake and not the recommended way, overlooked when the protocol was implemented. This does not have a big impact though.

Command data are structured as sequence of type-length-value (TLV):

  • TLV type

  • TLV length of the data

  • raw data

The TLVs are stored in the command data sequentially and the order is not mandatory, expected data for each command is looked up based on the type. Unrecognized TLV type may be ignored on the receive side. In the following documentation the TLVs are listed in the order as they’re emitted by kernel side generating the stream.

Subvolumes from the source and target filesystems are matched by the UUIDs.

All file or directory changes are matched by path, inode numbers may be different on the source and target filesystems but are present in the TLVs.

Attributes, TLV types

The TLV types are also called attributes of the command. The raw data type is one of the Data types listed above, the actual named type may be a generic one (like PATH) or specific for set of command (like CLONE_OFFSET).

The attributes represent the same raw data type when used in various commands, though this is not strictly necessary.

Version 1

Name

Number

Type

BTRFS_SEND_A_UNSPEC

0

invalid

BTRFS_SEND_A_UUID

1

uuid

BTRFS_SEND_A_CTRANSID

2

u64

BTRFS_SEND_A_INO

3

u64

BTRFS_SEND_A_SIZE

4

u64

BTRFS_SEND_A_MODE

5

u64

BTRFS_SEND_A_UID

6

u64

BTRFS_SEND_A_GID

7

u64

BTRFS_SEND_A_RDEV

8

u64

BTRFS_SEND_A_CTIME

9

timespec

BTRFS_SEND_A_MTIME

10

timespec

BTRFS_SEND_A_ATIME

11

timespec

BTRFS_SEND_A_OTIME

12

timespec

BTRFS_SEND_A_XATTR_NAME

13

string

BTRFS_SEND_A_XATTR_DATA

14

data

BTRFS_SEND_A_PATH

15

string

BTRFS_SEND_A_PATH_TO

16

string

BTRFS_SEND_A_PATH_LINK

17

string

BTRFS_SEND_A_FILE_OFFSET

18

u64

BTRFS_SEND_A_DATA

19

data

BTRFS_SEND_A_CLONE_UUID

20

uuid

BTRFS_SEND_A_CLONE_CTRANSID

21

u64

BTRFS_SEND_A_CLONE_PATH

22

string

BTRFS_SEND_A_CLONE_OFFSET

23

u64

BTRFS_SEND_A_CLONE_LEN

24

u64

Version 2

Name

Number

Type

BTRFS_SEND_A_FALLOCATE_MODE

25

u32

BTRFS_SEND_A_FILEATTR

26

u64

BTRFS_SEND_A_UNENCODED_FILE_LEN

27

u64

BTRFS_SEND_A_UNENCODED_LEN

28

u64

BTRFS_SEND_A_UNENCODED_OFFSET

29

u64

BTRFS_SEND_A_COMPRESSION

30

u32

BTRFS_SEND_A_ENCRYPTION

31

u32

Special cases

File creation (MKFILE) is done in two phases, where first a file with special name like o-123-45678 is created and then renamed (RENAME) to the target location.

Parent directory where file changes are currently done gets update of times (UTIMES) after each operation that affects it. This is not optimal as only the last one would be needed.

Raw data type is processed in a different way in protocol version 1 and 2. In 1 the maximum size of the TLV is 64KiB as the size is stored in u16. This is not sufficient for encoded write (ENCODED_WRITE) command. In 2 the data length is up to 4GiB (using the type u32) but the TLV must be last and the actual length is calculated as the delta between the whole command and the TLV (i.e. ignoring the TLV header length).

Stream version 1

BTRFS_SEND_C_UNSPEC (0)

Placeholder, invalid or ignored command.

BTRFS_SEND_C_SUBVOL (1)

Type

Name

Description

string

path

relative path of the subvolume

uuid

uuid

UUID of the sent subvolume

u64

ctransid

creation transaction

BTRFS_SEND_C_SNAPSHOT (2)

Start of commands of a given snapshot.

Type

Name

Description

string

path

relative path of the subvolume

uuid

uuid

UUID of the sent subvolume

u64

ctransid

creation transaction

uuid

clone_uuid

u64

clone_ctransid

BTRFS_SEND_C_MKFILE (3)

Create regular file. See also section Special cases.

Type

Name

Description

string

path

relative file path to create

u64

ino

inode number

BTRFS_SEND_C_MKDIR (4)

Create a directory.

Type

Name

Description

string

path

relative directory path to create

u64

ino

inode number

File creation is done in two commands, the first one contains a special file name that is later renamed to the final name. (WHY)

BTRFS_SEND_C_MKNOD (5)

Create a special file of type device node (mknod).

Type

Name

Description

string

path

relative file path to create

u64

mode

file mode parameter of mknod(2)

u64

rdev

rdev parameter of mknod(2)

BTRFS_SEND_C_MKFIFO (6)

Create a special file of type FIFO (mkfifo).

Type

Name

Description

string

path

relative file path to create

u64

ino

inode number

BTRFS_SEND_C_MKSOCK (7)

Create a special file of type socket (mknod S_IFSOCK).

Type

Name

Description

string

path

relative file path to create

u64

ino

inode number

BTRFS_SEND_C_RENAME (9)

Rename file path.

Type

Name

Description

string

path

relative source file path

string

path_to

relative target file path

BTRFS_SEND_C_RMDIR (12)

Remove directory.

Type

Name

Description

string

path

relative directory path

BTRFS_SEND_C_SET_XATTR (13)

Set a value of extended attribute.

Type

Name

Description

string

path

relative file path

string

xattr_name

name of the extended attribute

data

xattr_data

value of the extended attribute

BTRFS_SEND_C_REMOVE_XATTR (14)

Remove an extended attribute.

Type

Name

Description

string

path

relative file path

string

xattr_name

name of the extended attribute

BTRFS_SEND_C_WRITE (15)

Write file data to a given file offset.

Type

Name

Description

string

path

relative file path

u64

file_offset

where to write data

data

data

raw file data (variable length)

BTRFS_SEND_C_CLONE (16)

Clone extents from another file.

Type

Name

Description

string

path

relative file path

u64

file_offset

offset in the source file to clone from

u64

clone_len

length of cloned data

uuid

clone_uuid

u64

clone_ctransid

string

clone_path

clone target relative file path

u64

clone_offset

clone offset in the target file

BTRFS_SEND_C_TRUNCATE (17)

Truncate file to a given length.

Type

Name

Description

string

path

relative file path

u64

size

truncate to given size

BTRFS_SEND_C_CHMOD (18)

Chmod a file or directory.

Type

Name

Description

string

path

relative file path

u64

mode

new mode

BTRFS_SEND_C_CHOWN (19)

Change file owner (uid) and group (gid), specified by numeric id. The uid/gid must exist on the target filesystem, no mapping is done.

Type

Name

Description

string

path

relative file path

u64

uid

numeric used id

u64

gid

numeric group id

BTRFS_SEND_C_UTIMES (20)

Change file atime and mtime, nanosecond precision. While the ctime is also sent it’s not possible to change it using utimensat. The creation time is sent since protocol version 2 but cannot be changed on the target filesystem.

Type

Name

Description

string

path

relative file path

timespec

atime

file atime

timespec

mtime

file mtime

timespec

ctime

file ctime

timespec

otime

(since v2) file otime (creation time)

BTRFS_SEND_C_END (21)

Special command to denote end of one logical stream inside the whole stream sequence. May or may not be processed by receiver.

BTRFS_SEND_C_UPDATE_EXTENT (22)

When send is generated without data (BTRFS_SEND_FLAG_NO_FILE_DATA), this command informs about changed extent but does not send the actual data.

Type

Name

Description

string

path

relative file path

u64

file_offset

file offset where data were updated

u64

size

length of the data

Stream version 2

BTRFS_SEND_C_FALLOCATE (23)

Change file extents to preallocated, punch hole or zero fill.

Type

Name

Description

string

path

relative file path

u32

fallocate_mode

which fallocate operation to do

u64

file_offset

file offset where to apply the operation

u64

size

length of the range

BTRFS_SEND_C_FILEATTR (24)

File attributes, representing various flags (SETFLAGS ioctl, XFLAGS, BTRFS specific inode flags). The value is set from BTRFS inode bits and the stream format inherits that. Note that some flags like IMMUTABLE or APPEND may affect ability to change other flags and that for some flags there’s ready interface to set them.

BTRFS_SEND_C_ENCODED_WRITE (25)

File data encoded by the source filesystem and written directly to the target filesystem, without any other transformation. The data can be compressed or encrypted and the payload depends on presence of the TLVs.

Type

Name

Description

string

path

relative file path

u64

file_offset

file offset where to write the data

u64

unencoded_file_len

u64

unencoded_len

u64

unencoded_offset

u32

compression

(optional) compression type

u32

encryption

(optional) encryption type

data

data

encoded payload