oc-rsync

rsync re-implemented in Rust. Wire-compatible with upstream rsync 3.4.1 (protocol 32), works as a drop-in replacement.

Binary name: oc-rsync - installs alongside system rsync without conflict.

---

Status

Release: 0.5.9 (alpha) - Wire-compatible drop-in replacement for rsync 3.4.1 (protocols 28-32).

All transfer modes (local, SSH, daemon), delta algorithm, metadata preservation, incremental recursion, and compression are complete. Interop tested against upstream rsync 3.0.9, 3.1.3, and 3.4.1.

Component	Status
Transfer	Local, SSH, daemon push/pull
Delta	Rolling + strong checksums, block matching
Metadata	Permissions, timestamps, ownership, ACLs (-A), xattrs (-X)
File handling	Sparse, hardlinks, symlinks, devices, FIFOs
Deletion	--delete (before/during/after/delay), --delete-excluded
Compression	zlib, zstd, lz4 with level control
Checksums	MD4, MD5, XXH3/XXH128 with SIMD (AVX2, SSE2, NEON)
Incremental recursion	Push and pull directions
Batch	--write-batch / --read-batch roundtrip
Daemon	Negotiation, auth, modules, chroot, syslog, pre/post-xfer exec
Filtering	--filter, --exclude, --include, .rsync-filter, --files-from
Reference dirs	--compare-dest, --link-dest, --copy-dest
Options	--delay-updates, --inplace, --partial, --iconv, fuzzy matching
I/O	io_uring (Linux 5.6+), copy_file_range, clonefile (macOS), adaptive buffers
Platforms	Linux, macOS (full); Windows (partial - no ACLs/xattrs)

What's New (v0.5.9)

Performance

• Adaptive I/O buffers (8KB-1MB) scaled to file size
• Buffer pool passed to copy_file_range fallback path
• FileEntry memory reduced via Box<FileEntryExtras> for rarely-used fields
• Shared file list via Arc eliminates per-file clone overhead
• Precomputed sort keys remove per-comparison memrchr calls
• Parallel basis file signature computation in pipeline fill
• Batched receiver flush syscalls with path reuse
• Rayon-based parallel stat replaces tokio::spawn_blocking
• Upstream-matching compression negotiation precedence

Features

• io_uring wired into sender/generator for large file reads
• Protocol version guards for legacy flist read path (proto < 30)
• Batch file encoding replaced with protocol stream format
• Transfer statistics written to batch files

Fixes

• SSH transfer deadlocks and protocol compatibility resolved
• Client-mode multiplex activation matches upstream behavior
• INC_RECURSE capability direction corrected for daemon push
• 6 interop known failures resolved (dry-run, files-from, relative paths)

Interop Testing

Tested against upstream rsync 3.0.9, 3.1.3, and 3.4.1 in CI across protocols 28-32. Both push and pull directions verified for 30+ scenarios covering transfer modes, deletion, compression, metadata, reference dirs, file selection, batch roundtrip, path handling, device nodes, and daemon auth.

Performance

Threaded architecture replaces upstream's fork-based pipeline while keeping full protocol compatibility, reducing syscall overhead and context switches. Adaptive I/O buffers scale from 8KB to 1MB based on file size. Optional io_uring on Linux 5.6+ (--io-uring / --no-io-uring).

---

Installation

Homebrew

brew tap oferchen/rsync https://github.com/oferchen/rsync
brew install oferchen/rsync/oc-rsync

Prebuilt packages

Download from the Releases page:

Platform	Formats
Linux (x86_64, aarch64)	.deb, .rpm (with OpenSSL), static musl .tar.gz
macOS (x86_64, aarch64)	.tar.gz
Windows (x86_64)	.tar.gz, .zip

Linux static tarballs are available in two checksum variants:

Variant	Filename	Description
Pure Rust (recommended)	*-musl.tar.gz	Pure-Rust checksums, zero system dependencies
OpenSSL	*-musl-openssl.tar.gz	OpenSSL-accelerated MD4/MD5 checksums (vendored, statically linked)

Each release also includes three toolchain variants: stable (recommended, no suffix), beta (-beta), and nightly (-nightly).

Build from source

Requires Rust 1.88+.

git clone https://github.com/oferchen/rsync.git
cd rsync
cargo build --workspace --release

---

Usage

Works like rsync - drop-in compatible:

# Local sync
oc-rsync -av ./source/ ./dest/

# Remote pull (SSH)
oc-rsync -av user@host:/remote/path/ ./local/

# Remote push (SSH)
oc-rsync -av ./local/ user@host:/remote/path/

# Daemon pull
oc-rsync -av rsync://host/module/ ./local/

# Daemon push
oc-rsync -av ./local/ rsync://host/module/

# Run as daemon
oc-rsync --daemon --config=/etc/oc-rsyncd/oc-rsyncd.conf

# Delta transfer with compression
oc-rsync -avz --compress-level=3 ./source/ ./dest/

# Checksum-based sync with deletion
oc-rsync -avc --delete ./source/ ./dest/

# Batch mode (record and replay)
oc-rsync -av --write-batch=changes ./source/ ./dest/
oc-rsync -av --read-batch=changes ./other-dest/

For supported options: oc-rsync --help

---

Development

Prerequisites

• Rust 1.88.0 (managed via rust-toolchain.toml)
• cargo-nextest: cargo install cargo-nextest --locked

Build and test

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features --no-deps -D warnings
cargo nextest run --workspace --all-features

Project layout

src/bin/oc-rsync.rs     # Entry point
crates/cli/             # CLI flags, help, output formatting
crates/core/            # Orchestration facade, session management, config
crates/protocol/        # Wire protocol (v28-32), multiplex framing
crates/transfer/        # Generator (sender), receiver, delta transfer
crates/engine/          # Local copy executor, sparse writes, temp-file commit
crates/daemon/          # Daemon mode, module access control, systemd
crates/checksums/       # Rolling and strong checksums (MD4, MD5, XXH3, SIMD)
crates/filters/         # Include/exclude pattern engine, .rsync-filter
crates/metadata/        # Permissions, uid/gid, mtime, ACLs, xattrs
crates/rsync_io/        # SSH stdio, rsync:// TCP transport, handshake
crates/fast_io/         # Platform I/O (io_uring, copy_file_range, sendfile)
crates/compress/        # zstd, lz4, zlib compression codecs
crates/bandwidth/       # Bandwidth limiting and rate control
crates/signature/       # Signature layout and block-size calculations
crates/match/           # Delta matching and block search
crates/flist/           # File list generation and traversal
crates/logging/         # Logging macros and verbosity control
crates/logging-sink/    # Message sink and output formatting
crates/batch/           # Batch file read/write support
crates/branding/        # Binary naming and version metadata
crates/embedding/       # Programmatic entry points for library usage
crates/apple-fs/        # macOS filesystem operations (clonefile, FSEvents)
crates/windows-gnu-eh/  # Windows GNU exception handling shims

See cargo doc --workspace --no-deps --open for API documentation.

Architecture

cli -> core -> engine, daemon, transport, logging
                core -> protocol -> checksums, filters, compress, bandwidth -> metadata

Key crates: cli (Clap v4), core (orchestration facade), protocol (wire v28-32, multiplex framing), transfer (generator/receiver, delta pipeline), engine (local copy, sparse writes, buffer pool), checksums (MD4/MD5/XXH3, SIMD), daemon (TCP, auth, modules).

---

Security

Protocol parsing crates enforce #![deny(unsafe_code)]. Unsafe code is limited to:

• SIMD-accelerated checksums (with scalar fallbacks)
• Platform I/O operations (sendfile, io_uring, mmap -- with fallbacks)
• Metadata/ownership FFI (UID/GID lookup, chroot, setuid/setgid)
• Windows GNU exception handling

Not vulnerable to known upstream rsync CVEs (CVE-2024-12084 through CVE-2024-12088, CVE-2024-12747). OS-level race conditions (TOCTOU) remain possible at filesystem boundaries.

For security issues, see SECURITY.md.

---

Contributing

1. Fork and create a feature branch
2. Run cargo fmt, cargo clippy, and cargo nextest run
3. Open a PR describing behavioral changes and interop impact

---

License

GNU GPL v3.0 or later. See LICENSE.

---

Acknowledgements

Inspired by rsync by Andrew Tridgell and the Samba team. Thanks to Pieter for his heroic patience in enduring months of my rsync commentary. Thanks to Elad for his endless patience hearing rsync protocol commentary as I'm introduced to it.