ShaniOS Technical Documentation

What is ShaniOS?

ShaniOS is an immutable Linux distribution that brings enterprise DevOps practices to desktop computing. Built on Arch Linux with Btrfs filesystem, it provides atomic updates, instant rollback, and system integrity by design. It ships as two editions — GNOME and KDE Plasma — and works out of the box with no post-install tweaking required.

The core pillars of ShaniOS are:

• Immutability: The root filesystem (/) is mounted read-only. System binaries and libraries cannot be changed at runtime — by you, by software, or by malware. The only way to modify the OS is through the controlled shani-deploy update pipeline.
• Blue-Green Deployment: Two complete system images (@blue and @green) are maintained. While one is active, updates are written to the other. On reboot you switch to the updated image. The previous one is kept as a one-command rollback target.
• Atomic Updates: Updates are all-or-nothing. The running system is never touched during an update. If something goes wrong, boot failure is detected automatically and the system rolls back — your work is never interrupted.
• Selective Persistence: User data, configuration changes (/etc overlay), Flatpak apps, containers, VMs, and service credentials all live in dedicated Btrfs subvolumes that survive every update and rollback, always. A dedicated @nix subvolume is pre-created for use with the Nix package manager.
• Defence-in-Depth Security: Six Linux Security Modules run simultaneously (lsm=landlock,lockdown,yama,integrity,apparmor,bpf), LUKS2 argon2id full-disk encryption, TPM2 auto-unlock, Secure Boot with MOK, Intel ME disabled by default, and every OS image is SHA256+GPG verified before deployment.
• Zero Telemetry: No usage data, crash reports, analytics, or tracking of any kind — ever. The entire codebase is public on GitHub; every claim is independently verifiable.

3. System Optimizations

ShaniOS includes extensive performance, gaming, and reliability optimizations out of the box, eliminating the need for manual tweaking. These enterprise-grade optimizations are pre-configured and active from first boot.

Memory & Storage Management

• ZRAM Compression: Automatic RAM compression using the zstd algorithm (configured to use full RAM size) provides improved memory efficiency without requiring swap partitions. The system dynamically allocates compressed swap in RAM, reducing disk I/O and extending SSD lifespan.
• Optimized Swappiness: Tuned to 133 for balanced memory management on modern systems. This value encourages the kernel to use compressed ZRAM swap before evicting clean page cache, maintaining system responsiveness under memory pressure.
• Btrfs Maintenance: Automated filesystem maintenance runs in the background, including periodic scrubbing for data integrity, balance operations to optimize storage layout, and defragmentation for improved performance. These operations are scheduled to minimize impact on system usage.
• Profile Sync Daemon: Browser profiles are stored in tmpfs (RAM) for dramatically faster load times and reduced disk writes. Changes are periodically synchronized back to persistent storage, combining speed with data safety. This reduces SSD wear while improving browser responsiveness.
• Optimized Page Lock Unfairness: Set to 1 to reduce lock contention and improve multi-threaded application performance, particularly beneficial for databases and high-concurrency workloads.

Gaming & Performance Optimizations

ShaniOS is optimized for gaming performance with multiple layers of system tuning:

Gaming Hardware Support:

• game-devices-udev Package: Comprehensive udev rules for gaming peripherals with user-grade permissions, including:
  • Controllers: 8BitDo, PlayStation (DualShock 3/4, DualSense, DualSense Edge), Xbox (360, One, Series), Nintendo (Switch Pro, Joy-Cons, GameCube adapter), Google Stadia, NVIDIA Shield, Razer, Nacon, Hori, PowerA, PDP, Mad Catz, Astro C40, and more
  • Fight Sticks & Arcade: Razer Panthera, Mad Catz FightStick, Hit Box, and arcade-style controllers
  • VR Equipment: HTC Vive, PlayStation VR, Valve Index/SteamVR devices with full hardware access
  • Flight Sticks & Sim: VKBSim Gladiator, Logitech F310/F710, and other simulation controllers
  • uinput Support: Early creation of /dev/uinput for virtual device emulation (gamepad mappers, streaming tools)
• Additional RGB & Peripheral Support: OpenRGB udev rules (60-openrgb.rules) for comprehensive RGB control:
  • RGB Keyboards: ASUS Aura, Corsair, Razer, SteelSeries, Logitech G-series, MSI Mystic Light, Cooler Master, NZXT, and dozens more
  • RGB Mice: Gaming mice from all major manufacturers with LED control
  • RGB Mousepads: Corsair, Razer, SteelSeries, and other illuminated mousepads
  • RGB Accessories: Headset stands, mouse docks, PC case lighting, AIO coolers, and more
• Racing Wheel Support: Full force feedback and configuration access:
  • Logitech wheels (99-logitech-wheel-perms.rules): G29, G920, G923, G PRO, and legacy wheels (G25, G27, Driving Force, MOMO) with range, gain, LED, and force feedback tuning
  • Thrustmaster wheels (99-thrustmaster-wheel-perms.rules): T150, T300RS, T500RS, T248, TS-XW, TS-PC with spring, damper, and friction control
  • Fanatec wheels (99-fanatec-wheel-perms.rules): Full hardware access with deadzone removal via evdev-joystick
• Game Input Utilities:
  • AntiMicroX: Gamepad to keyboard/mouse mapper with uinput access (60-antimicrox-uinput.rules)
  • Steam Controller support: Native hardware access for Valve's gaming ecosystem

Kernel-Level Gaming Optimizations:

• Game Compatibility Kernel Parameters: Pre-configured for maximum compatibility:
  • Increased PID limit (65535) supports complex games with many processes
  • Expanded memory map areas (2147483642) accommodates large game worlds and high-resolution texture streaming
  • Enhanced inotify limits (1024 instances, 524288 watches) for games with extensive file monitoring
• Network Optimization: TCP FIN timeout reduced to 5 seconds (from default 60s) enables games to quickly reuse network ports when restarting. This Valve SteamOS optimization prevents "address already in use" errors in multiplayer games.
• CPU Scheduler Tuning: Multiple scheduler optimizations for gaming performance:
  • CFS bandwidth slice: 3000μs for improved interactivity and reduced input latency
  • Base slice: 3000000ns (3ms) for consistent frame times
  • Migration cost: 500000ns to prevent excessive CPU migration overhead
  • Migration limit: 8 tasks per balance operation for optimal multi-core utilization
  • Child runs first: Disabled (0) to prevent priority inversion in gaming scenarios
  • Autogroup scheduling: Enabled (1) to automatically group related processes for better desktop responsiveness
• Advanced Memory Management:
  • MGLRU (Multi-Gen LRU) enabled with aggressive settings for intelligent memory reclaim that preserves recently accessed game data
  • Transparent Hugepage set to madvise for selective large page usage, improving performance for games that explicitly request it
  • Optimized watermark scaling (500) and minimum free memory (1MB) for consistent response times under load
  • Reduced compaction proactiveness (0) minimizes latency spikes during memory allocation
  • Watermark boost factor: Set to 1 (minimal) to reduce aggressive memory reclaim
  • Zone reclaim disabled (0) prevents unnecessary NUMA overhead on multi-socket systems
• GameMode Integration: GameMode daemon enabled globally for all users, providing automatic performance optimizations when gaming. GameMode dynamically adjusts CPU governor, process priorities, I/O scheduling, and GPU performance levels. Games are automatically detected and optimized without manual configuration.
• High-Precision Timers: HPET and RTC frequencies increased to 3072 Hz (from default 64 Hz) for superior timing accuracy. This reduces input lag and improves frame pacing in games requiring precise timing, particularly beneficial for rhythm games and competitive shooters.

Process Management & Responsiveness

ShaniOS enables several system services for intelligent resource management:

• Ananicy-cpp: Automatic process priority management service with game-aware rules (enabled by default). Background tasks are automatically deprioritized when games or media applications are active, ensuring smooth performance without manual nice/renice commands.
• systemd-oomd: System-wide OOM (Out-Of-Memory) daemon enabled by default prevents system freezes. When memory pressure is detected, systemd-oomd selectively terminates low-priority processes based on memory usage patterns and process priorities, keeping critical services and active applications running.
• IRQBalance: Service enabled to optimize interrupt distribution across CPU cores, preventing a single core from becoming overwhelmed with hardware interrupts. This is particularly important for systems with high-bandwidth network cards or NVMe drives, ensuring balanced load across all cores.
• Increased File & Process Limits: Default limits raised to 1,048,576 for both open files (NOFILE) and processes (NPROC). This prevents "too many open files" errors in development environments, containerized workloads, and servers running multiple services.
• Fast Shutdown: Reduced timeout values (10s for stop, 10s for abort) ensure quick system shutdowns without waiting for unresponsive services. Services that fail to stop gracefully are automatically killed after the timeout.

Security Hardening

• AppArmor Mandatory Access Control:
  • AppArmor (apparmor.service): Enabled by default to confine system services and applications with security profiles
• Firewall Protection:
  • firewalld: Active firewall enabled by default, denying inbound connections while allowing essential services
  • Pre-configured rules: Automatic firewall configuration for KDE Connect/GSConnect and Waydroid during installation
• Kernel Hardening:
  • NMI watchdog disabled for faster boot times and reduced CPU overhead (this is safe on modern systems with other hang detection mechanisms)
  • Unprivileged user namespaces enabled for secure containerization (required for Podman, Flatpak sandboxing, and Distrobox)
  • Magic SysRq keys enabled for emergency recovery (Alt+SysRq+REISUB sequence for safe reboots during system hangs)
• Hardware Access Controls: The realtime group receives permission to access HPET and RTC devices directly. This is essential for professional audio production, real-time multimedia applications, and low-latency gaming. Users in the realtime group can run applications with real-time scheduling priorities.
• Blacklisted Kernel Modules:
  • PC speaker (pcspkr) disabled to eliminate annoying beeps
  • Intel Management Engine (mei, mei_me) disabled for enhanced privacy and security, removing Intel's remote management interface

Boot & System Efficiency

• Plymouth (BGRT Theme): Smooth graphical boot experience with the BGRT (Boot Graphics Resource Table) theme enabled by default. Plymouth provides a polished boot animation while hiding technical boot messages, presenting a clean interface for password entry during LUKS unlock. The BGRT theme displays your manufacturer's logo for a seamless firmware-to-OS transition.
• Journal Size Limit: SystemD journal capped at 50MB to prevent excessive disk usage from log accumulation. This is particularly important on systems with limited storage, while still maintaining sufficient logs for troubleshooting recent issues.
• Reduced Kernel Messages: Console printk level set to 3 (errors and critical messages only) for cleaner boot output. This eliminates informational noise while preserving important error messages needed for diagnostics.
• Optimized Service Startup: Parallel service initialization with optimized dependencies reduces boot time. Non-critical services are delayed or started on-demand, prioritizing services required for desktop readiness.
• Time Synchronization: systemd-timesyncd enabled by default for automatic network time protocol (NTP) synchronization, ensuring accurate system time without requiring a full NTP daemon.
• Socket Activation: Many services use socket activation for on-demand startup, reducing boot time and memory usage:
  • pcscd.socket: Smart card daemon (PC/SC) starts only when smart cards are accessed
  • lircd.socket: Linux Infrared Remote Control daemon for IR remote support
  • gpsd.socket: GPS daemon for location services and navigation hardware
  • cups.socket: Printing service starts on-demand when print jobs are sent
  • avahi-daemon.socket: Network service discovery (mDNS/DNS-SD) for zero-configuration networking
  • saned.socket: Scanner daemon starts when scanning is needed

Hardware & Peripheral Support

ShaniOS enables comprehensive hardware support services by default:

• Printing:
  • CUPS (cups.socket): Common UNIX Printing System for all printer support
  • cups-browsed: Automatic printer discovery on the network
  • ipp-usb: IPP-over-USB support for modern driverless printers
  • Avahi (avahi-daemon.socket): Network printer discovery via Bonjour/mDNS
• Scanning:
  • SANE (saned.socket): Scanner Access Now Easy for scanner support
  • ipp-usb: Also provides driverless scanning support
• Biometric Authentication:
  • fprintd: Fingerprint reader support for login and authentication
• Bluetooth:
  • bluetooth.service: Full Bluetooth stack for wireless peripherals, audio, and file transfer
• Mobile Devices:
  • usbmuxd: iOS device support for iPhone/iPad connectivity, file transfer, and tethering
• Networking:
  • NetworkManager: Intelligent network connection management for wired, wireless, VPN, and mobile connections
  • ModemManager: Mobile broadband modem support (3G/4G/5G)
• Gaming Peripherals:
  • ratbagd: Gaming mouse configuration daemon for DPI, button mapping, and LED control (libratbag)
  • inputattach: Serial input device support (legacy joysticks, game controllers)
• Location Services:
  • geoclue: Geolocation service for applications requiring location data
  • gpsd.socket: GPS hardware support
• Thunderbolt Security:
  • bolt: Thunderbolt 3 device authorization and management
• Power Management:
  • apcupsd: APC UPS monitoring and management for battery backup systems
  • power-profiles-daemon: System power profile management (performance, balanced, power-saver)
  • switcheroo-control: Hybrid graphics switching for laptops with dual GPUs (Intel + NVIDIA/AMD)
• Firmware Updates:
  • fwupd: Linux Vendor Firmware Service for automatic firmware updates
  • fwupd-refresh.timer: Periodic firmware update checks

Enhanced Shell Experience

ShaniOS provides a modern, feature-rich shell environment that rivals the best developer setups:

• Zsh as Default Shell: Powerful shell with advanced completion, globbing, and command-line editing. Zsh provides better defaults than Bash while maintaining compatibility with Bash scripts.
• Fish-Style Features:
  • Syntax highlighting shows valid/invalid commands in real-time as you type
  • Autosuggestions suggests commands from history with ghost text completion
  • History substring search allows arrow-up/down to search through history based on what you've typed
• Starship Prompt: Fast, minimal, and infinitely customizable prompt with intelligent git integration, showing branch status, ahead/behind commits, and repository state. The prompt adapts to display context-relevant information (Python virtualenv, Node.js version, Rust toolchain, etc.).
• McFly: Neural network-powered command history search learns from your usage patterns. McFly prioritizes frequently used commands in relevant directories and contexts, providing smarter suggestions than traditional reverse-i-search.
• FZF (Fuzzy Finder): Integrated fuzzy finding for files, command history, and directory navigation. Ctrl+R searches history, Ctrl+T inserts file paths, and Alt+C changes directories—all with fuzzy matching and preview windows.
• Multiple Shell Support: Bash and Fish shells included with completion support. Switch shells anytime with chsh while maintaining consistent functionality across all options.

Automated Maintenance

ShaniOS performs regular background maintenance to keep your system healthy without manual intervention. These systemd timers and services are enabled by default:

• btrfs-scrub.timer: Monthly scrubbing to detect and repair data corruption using filesystem checksums
• btrfs-balance.timer: Periodic filesystem balancing redistributes data across drives for optimal performance
• btrfs-defrag.timer: Automatic defragmentation runs on fragmented files to improve read performance
• btrfs-trim.timer: Regular TRIM operations for SSD optimization and lifespan extension
• bees (beesd) — Continuous Background Deduplication: The bees daemon performs ongoing block-level deduplication across all Btrfs subvolumes. It is auto-configured at every boot by beesd-setup.service, which writes a per-UUID config to /etc/bees/ and enables the beesd@<UUID>.service unit. The hash database size is automatically scaled to disk capacity (256 MB per TB, capped at 1 GB). This works alongside the CoW sharing between @blue and @green to maximise storage efficiency over time.
• mark-boot-in-progress / bless-boot / mark-boot-success / check-boot-failure: A coordinated set of services that implement boot health tracking. At every boot, mark-boot-in-progress plants a flag and clears previous results; bless-boot calls bootctl set-good early; mark-boot-success writes /data/boot-ok once multi-user.target is reached; and a 15-minute timer runs check-boot-failure to record the booted slot in /data/boot_failure if the system never reached a successful state.
• flatpak-update-system.timer / flatpak-update-user.timer: Two independent timers — one system-level, one per-user — update all installed Flatpak apps, uninstall unused runtimes (--delete-data), and run flatpak repair automatically. Both fire 5 minutes after boot then every 12 hours with a 15-minute randomized delay to avoid hammering servers simultaneously.
• profile-sync-daemon (psd): Browser cache synchronization from RAM to disk happens automatically (enabled globally for all users)
• systemd-timesyncd: Automatic time synchronization keeps system clock accurate
• ZRAM compression/decompression: Transparently handles memory pressure without user interaction

All maintenance operations are scheduled during low-usage periods and use minimal system resources to avoid impacting active work.

Migrating from a Traditional Linux Distro

If you're coming from Ubuntu, Fedora, Arch, or any mutable Linux distro, the main adjustment is how you install software and make system-level changes. Everything else — your dotfiles, shell, /etc configs, and all files under /home — works exactly as you'd expect.

The Mental Model Shift

Stop thinking about "installing software into the system." Think instead in terms of these four layers:

• Flatpak is your new apt install / pacman -S for GUI applications. Flathub is pre-configured and ready to use from day one.
• Distrobox is your escape hatch. It creates a full mutable Linux container (Ubuntu, Fedora, Arch — your choice) with seamless desktop integration. Use it for development tools, build environments, anything that needs apt or pacman, and apps that export to your launcher.
• Nix (pre-installed, on the dedicated @nix subvolume) covers CLI tools and language runtimes with pinned versions and zero conflicts. Think of it as a supercharged user-space package manager. The @nix subvolume is shared across both slots so installed Nix packages survive updates and rollbacks. Add a channel with nix-channel --add before installing packages for the first time.
• The /etc overlay still works exactly like a normal /etc. Edit any config file with sudo nano, sudo vim, etc. — changes persist across all updates.

What Cannot Be Done (and the Workaround)

Understanding Immutability

ShaniOS's immutability fundamentally changes how you interact with the system. Understanding this concept is key to using ShaniOS effectively.

What You CAN Do

• ✅ Install applications via Flatpak
• ✅ Edit configuration files in /etc
• ✅ Create and modify files in /home
• ✅ Run containers (Podman, Distrobox, LXC)
• ✅ Update the entire system atomically
• ✅ Store data in /data and persistent subvolumes

What You CANNOT Do

• ❌ Use pacman to install traditional packages (use Flatpak, Nix, or Distrobox)
• ❌ Modify files in / (root filesystem) (read-only by design)
• ❌ Edit files in /usr, /bin, /lib directly (use /etc overlay for configs, Distrobox for binaries)
• ❌ Install software that requires system-level changes (use containers or AppImages instead)
• ❌ Run sudo pip install globally (use pip install --user, Nix, or Distrobox)
• ❌ Run sudo npm install -g to system paths (use Nix or install inside Distrobox)
• ❌ Use make install to install built software into system directories (build and export from Distrobox)
• ❌ Modify files in /opt or /usr/share directly (/etc overlay for config; Distrobox for everything else)

Why This Design?

Blue-Green Deployment

ShaniOS implements blue-green deployment using Btrfs subvolumes—a strategy adapted from DevOps for desktop Linux.

How It Works

1. System maintains @blue and @green subvolumes
2. One subvolume is active (mounted as /), the other inactive
3. Updates apply to inactive subvolume
4. Bootloader updated to point to updated subvolume
5. Reboot switches to updated system
6. Previous version remains available for instant rollback

Advantages

• Atomic Updates: All-or-nothing updates
• Zero Downtime: Active system never modified
• Instant Rollback: Boot into previous version anytime
• Safe Testing: Validate updates before committing

Atomic Updates

ShaniOS uses an intelligent multi-layered update system with automatic checking, user notifications, and the shani-deploy tool for atomic system updates.

Persistence Strategy

ShaniOS selectively persists data across immutable system updates through bind mounts and dedicated subvolumes.

Bind-Mounted Service State

Critical service data is bind-mounted from @data subvolume:

# Example bind mounts from fstab
# All bind mounts use these options for correct boot ordering:
# bind,nofail,x-systemd.after=var.mount,x-systemd.requires-mounts-for=/data

# Network configuration
/data/varlib/NetworkManager /var/lib/NetworkManager none bind,nofail,x-systemd.after=var.mount,x-systemd.requires-mounts-for=/data 0 0

# Bluetooth pairings
/data/varlib/bluetooth /var/lib/bluetooth none bind,nofail,x-systemd.after=var.mount,x-systemd.requires-mounts-for=/data 0 0

# systemd state
/data/varlib/systemd /var/lib/systemd none bind,nofail,x-systemd.after=var.mount,x-systemd.requires-mounts-for=/data 0 0

What Persists — Full Bind Mount List

The following directories are bind-mounted from @data/varlib/ to /var/lib/, surviving all updates and slot switches:

• System Core: dbus (machine-id), systemd (unit states, timers, random seed), fontconfig (font cache)
• Networking: NetworkManager (WiFi passwords, VPN configs, connection profiles), bluetooth (paired devices), firewalld (rules), samba (shares, lock state), nfs (exports, client tracking)
• Remote Access & VPN: caddy (web server state), tailscale (VPN keys), cloudflared (tunnel credentials), geoclue (location permissions)
• Display Managers: gdm (GNOME session state), sddm (KDE session state), colord (color profiles), pipewire (audio state), rtkit (realtime scheduling)
• Hardware: cups (printer configs), sane (scanner configs), upower (battery history), fwupd (firmware metadata), tpm2-tss (TPM2 persistent objects and sealed keys)
• Authentication & Security: fprint (fingerprint enrollment), AccountsService (user avatars, session prefs), boltd (Thunderbolt authorization), sudo (auth timestamp cache), sshd (privilege separation), polkit-1 (authorization cache), fail2ban (jail database)
• Data Protection: restic (backup metadata), rclone (remote configs), appimage (AppImage cache)

Spool Directories

The following directories are bind-mounted from @data/varspool/ to /var/spool/:

• Scheduling: anacron (job timestamps), cron (user crontabs), at (one-time jobs)
• Queues: cups (print queue), samba (SMB print spool), postfix (mail queue)

Bind Mount Handling

ShaniOS intelligently handles bind mounts during configuration:

• Source check: Only mounts if source directory exists in @data
• Target check: Only mounts if service is installed in the system
• Graceful skipping: Missing services (e.g., sddm on GNOME) don't cause errors
• Optional services: Services not present in base system are automatically skipped

System Requirements

Disk Partition Layout

ShaniOS uses a simple two-partition layout — there are no separate /home, /var, or swap partitions. All subvolumes live within the single Btrfs partition.

When full-disk encryption is chosen, the Btrfs partition is wrapped in a LUKS2 container (/dev/mapper/shani_root). The ESP is never encrypted — only the root partition is.

Pre-Installation Setup

BIOS/UEFI Configuration

Configure your firmware before installation (typically accessed via F2, F10, Del, or Esc during startup):

Creating Installation Media

Download the ShaniOS ISO from shani.dev and write it to USB:

• Recommended: Balena Etcher (cross-platform, user-friendly)
• Windows: Rufus (advanced options available)
• Linux/Mac: dd command or GNOME Disks

Installation Steps

Installation takes approximately 10-15 minutes:

First Boot Configuration

Plymouth BGRT Boot Theme

ShaniOS uses the Plymouth BGRT boot theme. Plymouth provides a smooth graphical boot experience, suppressing kernel and systemd messages from the screen. The BGRT (Boot Graphics Resource Table) theme reads the manufacturer's logo directly from the UEFI firmware and displays it during boot — providing a seamless transition from firmware to OS that matches the device's branding. On laptops and OEM hardware this typically shows the manufacturer's logo; on custom-built machines it shows the motherboard vendor's logo or a fallback graphic.

If LUKS2 full-disk encryption is enabled, Plymouth presents the passphrase prompt over the boot animation — no raw terminal text. With TPM2 auto-unlock enrolled, even this prompt is skipped and the disk unlocks silently.

First Deployment — Automatic Background Setup

On first boot after installation, ShaniOS automatically runs the initial deployment in the background. This is a one-time process that takes a few minutes and requires no user interaction:

• All Btrfs subvolumes (@home, @data, @cache, @log, @flatpak, @containers, @nix, @snapd, @waydroid, @lxc, @lxd, @machines, @libvirt, @qemu, @swap, and more) are created and mounted
• The swapfile is created in @swap sized to your RAM, with CoW disabled — hibernation is automatically configured
• Both @blue and @green slots are prepared
• The beesd deduplication daemon is configured for your Btrfs volume UUID
• User groups are assigned and service state directories are initialised

After this completes, your system is fully configured — no manual post-install steps required.

Initial Setup Wizard

After first deployment completes, the Initial Setup wizard guides you through:

• Creating your user account and setting a password
• Configuring network connections (Wi-Fi, wired)
• Setting language, locale, and keyboard layout
• Setting privacy preferences
• Enabling location services (optional)
• Customising appearance settings

The wizard runs automatically. If you skip it, re-run with gnome-initial-setup (GNOME) or from System Settings → Welcome (KDE).

After the Wizard — Recommended First Steps

• Flathub is pre-configured. No flatpak remote-add needed — open GNOME Software or KDE Discover and browse apps immediately.
• Nix channel: Nix is pre-installed and running. Add a channel before installing packages: nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixpkgs && nix-channel --update. See the Nix section for full details.
• TPM enrollment (if available): If your machine has a TPM 2.0 chip, enroll your LUKS key for automatic disk decryption at next boot. See the TPM Encryption section.
• Waydroid (Android apps): Waydroid is pre-installed. Run sudo waydroid-helper init for automatic setup. Firewall rules are already configured. See the Android section.
• Secure Boot: If your BIOS supports it, enroll the MOK key and enable Secure Boot for the strongest TPM security guarantee. See the Secure Boot section.
• Check current slot: Run cat /data/current-slot to confirm whether you booted into @blue or @green.

OEM & Fleet Deployment

ShaniOS is designed for OEM and fleet use. Every machine imaging from the same signed ISO will boot into an identical, verified state. The Initial Setup wizard runs on first user login per machine, so user-specific personalisation (account, language, network) is captured without requiring per-device pre-configuration.

• Rollback never requires reimaging — the previous OS slot is always in the boot menu
• The boot-counting pipeline detects boot failures and automatically reverts the slot before the user sees an error, minimising support calls
• All user-facing changes (/etc customisations, systemd units, SSH keys, service configs) are in the @data OverlayFS and survive every update and rollback without reimaging
• The Plymouth BGRT theme shows the device manufacturer's logo automatically from UEFI — no per-model boot screen configuration needed
• passim (local content sharing daemon) broadcasts available fwupd firmware payloads via mDNS — machines on the same LAN avoid downloading the same firmware repeatedly

Btrfs Deep Dive

ShaniOS leverages advanced Btrfs features for immutability and efficiency.

Copy-on-Write (CoW)

Btrfs CoW minimizes storage duplication:

• Shared data blocks between @blue and @green
• Only ~18% overhead despite dual root system
• Modified blocks consume additional space
• Efficient updates even with two complete systems

Transparent Compression

# Default mount options
compress=zstd,space_cache=v2,autodefrag

• Reduces disk usage by 30-50%
• Minimal CPU overhead
• Improves SSD lifespan

Performance Optimizations

Specific subvolumes use nodatacow:

• @swap: CoW disabled for swap files (required)
• @libvirt: VM disk images benefit from direct writes

Mount Options by Subvolume

Why noatime?

All subvolumes use noatime to improve performance:

• Prevents writing to disk every time a file is read
• Significantly reduces SSD wear
• Improves battery life on laptops
• No impact on most applications (very few rely on access times)

Manual Btrfs Snapshots

Btrfs snapshots are instant, space-efficient copies of a subvolume at a point in time. Because they use CoW, a fresh snapshot consumes almost no additional space — only changes accumulate. ShaniOS automates slot snapshots via shani-deploy, but you can create and manage snapshots of your own data freely.

# Create a read-only snapshot of /home (best practice for backups)
sudo btrfs subvolume snapshot -r /home /data/snapshots/home-$(date +%Y%m%d)

# Create a writable snapshot (for testing changes to a subvolume)
sudo btrfs subvolume snapshot /home /data/snapshots/home-writable

# List all subvolumes and snapshots on the filesystem
sudo btrfs subvolume list /

# Show snapshot details (creation time, ID, parent)
sudo btrfs subvolume show /data/snapshots/home-$(date +%Y%m%d)

# Restore from a snapshot — replace @home contents with snapshot
# (Do this from a live USB or alternate slot to avoid conflicts)
sudo btrfs subvolume delete /home
sudo btrfs subvolume snapshot /data/snapshots/home-20250101 /home

# Delete an old snapshot to free space
sudo btrfs subvolume delete /data/snapshots/home-20240601

# Send a snapshot to another drive as a backup (incremental after first)
sudo btrfs send /data/snapshots/home-20250101 | sudo btrfs receive /mnt/backup/
# Incremental send (only sends the diff)
sudo btrfs send -p /data/snapshots/home-20250101 /data/snapshots/home-20250201 \
  | sudo btrfs receive /mnt/backup/

Checking Deduplication Status

The bees daemon runs continuously in the background and deduplicates data across all subvolumes. To check its activity and measure compression savings:

# Check bees daemon status
sudo systemctl status "beesd@*"

# View recent dedup activity from journal
sudo journalctl -u "beesd@*" --since today | grep -E "dedup|hash|block|crawl"

# Run an on-demand deduplication pass (in addition to background bees)
sudo shani-deploy --optimize

# Check compression ratio per subvolume
sudo compsize /
sudo compsize /home
sudo compsize /nix
sudo compsize /var/lib/flatpak

# Full storage usage report
sudo shani-deploy --storage-info

Filesystem Structure

ShaniOS uses Btrfs with a sophisticated subvolume layout:

Persistent Service State — Bind Mounts from @data

Because /var is volatile (systemd.volatile=state mounts a tmpfs over /var on every boot), all service state that must survive reboots is stored in the @data subvolume and bind-mounted back into place. The bind mounts are grouped by function in /etc/fstab:

Mount Options Reference

The root slots (@blue and @green) are not in fstab — they are mounted read-only by dracut/initramfs via the kernel command line (rootflags=subvol=@blue,ro,noatime,compress=zstd,space_cache=v2,autodefrag). All other Btrfs subvolumes use noatime,compress=zstd,space_cache=v2,autodefrag by default. VM disk subvolumes (@libvirt, @qemu) and the swap subvolume (@swap) use nodatacow,nospace_cache to avoid CoW fragmentation — note that compression is incompatible with nodatacow and is not applied to these subvolumes. Container and virtualisation subvolumes are mounted with nofail so the system boots cleanly even if they have not yet been created. The /etc overlay uses index=off,metacopy=off for maximum compatibility with the read-only lower layer. The /var tmpfs is provided by the systemd.volatile=state kernel parameter (not an fstab overlay entry, which exists in fstab but is commented out). All bind mounts use bind,nofail,x-systemd.after=var.mount,x-systemd.requires-mounts-for=/data to ensure ordering and graceful degradation.

Overlay Filesystem

OverlayFS enables writable /etc on read-only root.

# /etc overlay mount (from fstab)
overlay /etc overlay rw,lowerdir=/etc,upperdir=/data/overlay/etc/upper,workdir=/data/overlay/etc/work,index=off,metacopy=off,x-systemd.requires-mounts-for=/data 0 0

Overlay Directories

The /etc overlay is stored under @data with this structure:

@data/overlay/etc/
  ├── lower/      # (unused, just structure)
  ├── upper/      # Your changes stored here
  └── work/       # Overlay working directory

Viewing Changes

# List all overlay modifications
ls -la /data/overlay/etc/upper

# Compare with base system
diff -r /etc /data/overlay/etc/upper

Boot Process

Understanding how ShaniOS boots:

Early Boot Service Chain

After systemd takes over the root filesystem, ShaniOS runs a sequence of custom services before the desktop starts:

1. shanios-tmpfiles-data.service: Recreates required directories inside @data (overlay upper/work dirs, varlib, varspool) using systemd-tmpfiles. Runs after data.mount and before the /etc overlay is applied.
2. beesd-setup.service: Configures and enables the bees background deduplication daemon for the Btrfs volume. Runs after the root filesystem and etc-overlay.mount, before the daemon reload. Idempotent — skips if already configured at the same version.
3. etc-daemon-reload.service: Runs mount -a to apply all overlay and bind mounts from /etc/fstab, then issues a non-blocking systemctl daemon-reload so systemd discovers any new unit files that appeared in the overlay. Runs after beesd-setup.
4. etc-start-services.service: Starts any enabled services that were not yet running — specifically those whose unit files only became visible after the overlay was mounted. Runs once per boot (guarded by /run/start-overlay-services.done), with a 30-second timeout.
5. shani-user-setup.path / shani-user-setup.service: Watches /etc/passwd for changes. When a new regular user (UID 1000–59999) is detected, automatically adds them to the required groups (input, realtime, video, sys, cups, lp, scanner, nixbld, lxc, lxd, kvm, libvirt) and sets their shell to /bin/zsh.
6. startup-check.sh (autostart / systemd user): Runs 15 seconds after desktop login to give polkit and the session time to fully initialise. Reads /data/boot_failure and /data/current-slot; if a fallback boot is confirmed (booted slot ≠ current-slot and a matching failure file exists) it presents a GUI dialog (yad → zenity → kdialog → console) titled "Boot Failure Detected" prompting the user to rollback. If approved, it opens a terminal running pkexec shani-deploy --rollback, monitors completion via a status file, and on success offers to reboot immediately. Skips silently if /data/boot-ok is present or if no display is available.

Bootloader Configuration

ShaniOS uses systemd-boot with Unified Kernel Images (UKI):

• Timeout: 5 seconds
• Console mode: Maximum resolution
• Editor: Disabled for security (prevents boot parameter tampering)
• Default entry: Current active slot (@blue or @green)

Boot Entry Naming

Boot menu entries clearly show system state:

• shanios-blue (Active): Currently running system
• shanios-green (Candidate): Standby system, will be booted after next update

After each deployment, shani-deploy rewrites both boot entries — the newly updated slot becomes "Candidate" and the currently running slot stays "Active". The labels do not switch automatically at boot; they are explicitly updated by the deploy tool.

UEFI Boot Entry

ShaniOS creates a UEFI boot entry during installation:

• Label: shanios
• Loader: \EFI\BOOT\BOOTX64.EFI (Shim for Secure Boot)
• Fallback: Boot entry creation failures don't stop installation

Shim and Secure Boot Chain

The boot chain when Secure Boot is enabled:

UEFI Firmware
    ↓
shimx64.efi (signed by Microsoft — validates the next stage via MOK)
    ↓
mmx64.efi / grubx64.efi (systemd-boot, named grubx64.efi for shim compatibility, signed by MOK)
    ↓
Unified Kernel Image / shanios-blue.efi or shanios-green.efi (signed by MOK, built by gen-efi)
    ↓
Linux Kernel + Initramfs (embedded in UKI)

Kernel Command Line

ShaniOS uses these boot parameters, generated per-slot by gen-efi and saved to /etc/kernel/install_cmdline_<slot>. If that file already exists for a slot, gen-efi reuses it on subsequent UKI rebuilds rather than regenerating. The file is embedded directly into the UKI at build time by dracut.

quiet splash systemd.volatile=state ro
lsm=landlock,lockdown,yama,integrity,apparmor,bpf
rootfstype=btrfs
rootflags=subvol=@blue,ro,noatime,compress=zstd,space_cache=v2,autodefrag
root=/dev/mapper/shani_root  # If encrypted
rd.luks.uuid=... rd.luks.name=... rd.luks.options=tpm2-device=auto  # If TPM enrolled
rd.vconsole.keymap=...       # Injected from /etc/vconsole.conf KEYMAP= if set
resume=UUID=... resume_offset=...  # If swapfile exists (offset via btrfs inspect-internal map-swapfile)

Kernel Hardening

Security-focused kernel parameters are enabled by default:

lsm=landlock,lockdown,yama,integrity,apparmor,bpf

This enables six Linux Security Modules simultaneously — Landlock (filesystem sandboxing), Lockdown (kernel modification restrictions), Yama (ptrace scope restrictions), Integrity/IMA/EVM (runtime file integrity measurement), AppArmor (mandatory access control), and BPF LSM (dynamic eBPF security hooks). Most distributions enable one or two; ShaniOS runs all six concurrently. See the Security Features section for full details on each module.

• Landlock: Filesystem sandboxing for unprivileged processes
• Lockdown: Restricts kernel access and modification even by root
• Yama: Ptrace restrictions to prevent cross-process debugging attacks
• Integrity (IMA/EVM): Runtime file integrity measurement and extended attribute protection
• AppArmor: Mandatory Access Control — confines processes to what they legitimately need
• BPF LSM: eBPF-based dynamic security policy hooks

Kernel Parameter: systemd.volatile=state

The systemd.volatile=state kernel parameter creates a tmpfs (RAM-based) overlay for /var, making it volatile by default:

# From kernel command line
systemd.volatile=state

# Result: /var is a tmpfs, cleared on every reboot.
# Persistent data is restored via bind mounts into /var at boot:
#   /var/log        ← @log subvolume
#   /var/cache      ← @cache subvolume
#   /var/lib/flatpak    ← @flatpak subvolume
#   /var/lib/containers ← @containers subvolume
#   /var/lib/waydroid   ← @waydroid subvolume
#   /var/lib/machines   ← @machines subvolume
#   /var/lib/lxc        ← @lxc subvolume
#   /var/lib/libvirt    ← @libvirt subvolume
#   /var/lib/NetworkManager, bluetooth, etc. ← bind from @data/varlib/

Hibernation Support

ShaniOS automatically configures hibernation if a swapfile is created:

• Swapfile location: /swap/swapfile (in @swap subvolume)
• Size: Equal to RAM (or less if disk space limited)
• Resume offset: Automatically calculated using btrfs inspect-internal map-swapfile
• Btrfs CoW: Disabled on @swap subvolume for reliability

Security Features

ShaniOS implements defence-in-depth security across multiple independent layers. The read-only root prevents runtime modification of system files even by root. Six Linux Security Modules run simultaneously via a single lsm= kernel parameter — most distributions enable one or two. LUKS2 with argon2id protects data at rest with a memory-hard key derivation function specifically resistant to GPU and ASIC brute-force. Flatpak sandboxes applications. AppArmor confines system services with enforced profiles. firewalld denies all unsolicited inbound connections from first boot. Intel ME modules are blacklisted, removing the low-level hardware management channel from the attack surface. Every OS image is SHA256 + GPG verified before deployment. Together, these layers mean that even a fully compromised application cannot permanently damage the OS — a reboot to the unmodified, verified system image is always available.

Linux Security Modules — Full Detail

The six LSMs are activated together via a single kernel command-line parameter embedded in the Unified Kernel Image at build time:

lsm=landlock,lockdown,yama,integrity,apparmor,bpf

• Landlock: Filesystem sandboxing at the process level. Allows unprivileged processes to restrict their own filesystem access — a compromised process can be limited to only the paths it legitimately needs, even without root.
• Lockdown: Restricts kernel modifications even by root. Prevents unsigned kernel module loading, direct kernel memory writes, and other operations that could subvert kernel integrity. Two modes: integrity (prevent kernel modifications) and confidentiality (also prevent kernel data extraction).
• Yama: Restricts ptrace scope and other process tracing. Prevents one user's processes from attaching a debugger to another user's processes — closes a common privilege escalation path.
• Integrity (IMA/EVM): Runtime file integrity measurement. IMA (Integrity Measurement Architecture) records cryptographic hashes of files as they are accessed, building a runtime measurement log. EVM (Extended Verification Module) protects extended attributes (including security labels) from offline tampering.
• AppArmor: Mandatory Access Control that confines processes to the files, capabilities, and network operations they legitimately need. Profiles are shipped pre-written and enforced from first boot. snapd.apparmor.service loads Snap confinement profiles automatically.
• BPF LSM: eBPF-based security policy hooks. Allows dynamic, programmable security enforcement without modifying the kernel — security tools can attach BPF programs to kernel hooks to monitor and control behaviour at runtime.

Intel Management Engine — Disabled by Default

The Intel Management Engine (ME) is a separate, low-power processor embedded in Intel chipsets that operates independently of the main CPU and OS — including when the system is powered off (as long as power is connected). The ME runs its own firmware and has broad access to system resources.

ShaniOS blacklists the ME kernel modules (mei and mei_me) by default. This removes the OS-level communication channel to the ME, reducing the attack surface accessible from within a running Linux session. This is a meaningful privacy and security measure that most Linux distributions do not implement.

# Verify ME modules are blacklisted
cat /etc/modprobe.d/blacklist-intel-me.conf

# Confirm the modules are not loaded
lsmod | grep mei     # Should return nothing

Hardware Security Keys, Smart Cards & NFC

Enterprise and high-security authentication methods work at first boot — no setup required:

• FIDO2/U2F (libfido2): YubiKey and other FIDO2/U2F hardware security keys work for PAM login, sudo authentication, and web authentication (via browser WebAuthn) without any configuration. The relevant udev rules and pcscd.socket are pre-configured.
• Smart Cards (opensc + ccid + acsccid): PKCS#11 smart card authentication for login and sudo. CCID and ACS smart card readers are supported. pcscd.socket (PC/SC daemon) starts on demand when a card is accessed — no idle resource use.
• NFC (libnfc): NFC hardware is supported for authentication workflows that use NFC-based tokens or contactless smart cards.
• Fingerprint (fprintd): Biometric login and sudo authentication via PAM integration. Works at first boot on supported hardware.

# Test FIDO2 key detection
fido2-token -L

# List connected smart cards
opensc-tool -l

# Check fingerprint reader
fprintd-list $USER

# Check pcscd socket status
systemctl status pcscd.socket

Supply Chain Integrity & Image Verification

Supply chain attacks — injecting malicious code between a trusted source and the end user — are one of the most serious threats to software distribution. ShaniOS's update model is designed with this in mind.

Every OS image is GPG-signed before distribution. Before shani-deploy extracts a new image it verifies both the SHA256 checksum and the GPG signature. A tampered or corrupted image is rejected outright — the update aborts and nothing changes on your system. The verification uses the key ID 7B927BFF...8014792, which is published on public keyservers and can be independently fetched and verified.

# Manually verify an image (shani-deploy does this automatically)
# Fetch the public key from keyservers
gpg --keyserver keys.openpgp.org --recv-keys 7B927BFF8014792

# Verify the signature file against the image
gpg --verify shanios-image.zst.sig shanios-image.zst

# Verify SHA256 checksum
sha256sum -c shanios-image.zst.sha256

The build system, deploy scripts, and full signing workflow are public on GitHub. You can verify the full chain yourself — from build to signed image to deployment — without trusting any single party's word.

Secure Boot Setup

ShaniOS supports UEFI Secure Boot via Machine Owner Keys (MOK).

Automatic Secure Boot Enrollment (If Enabled During Install)

If you enabled Secure Boot during installation:

1. On first reboot, MOK Manager will appear automatically
2. Select "Enroll MOK"
3. Enter the password: shanios
4. Reboot to complete the enrollment
5. Secure Boot will be active with ShaniOS keys enrolled

Manual Secure Boot Enablement

To enable Secure Boot after installation:

sudo mokutil --import /etc/secureboot/keys/MOK.der

sudo mokutil --list-enrolled

TPM-Based Encryption

Enroll your LUKS encryption key to TPM 2.0 for automatic unlocking.

Prerequisites

• TPM 2.0 hardware module (enabled in BIOS/UEFI)
• LUKS2 encryption enabled during installation
• Recommended: Secure Boot enabled for maximum security

Enrollment Process

systemd-cryptenroll --tpm2-device=list

mokutil --sb-state

# Get the physical encrypted device
LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')
echo "Your encrypted device is: $LUKS_DEVICE"

# Example output: /dev/sda2 or /dev/nvme0n1p2

LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')
sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 "$LUKS_DEVICE"

LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')
sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0 "$LUKS_DEVICE"

# Regenerate only the currently booted slot (e.g., if booted into @blue):
sudo gen-efi configure blue

# To regenerate the OTHER slot's UKI as well, trigger a redeployment:
sudo shani-deploy --force

# Verify enrollment
LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')
sudo cryptsetup luksDump "$LUKS_DEVICE" | grep systemd-tpm2

Remove TPM Enrollment

To restore password-only unlocking:

LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')
sudo systemd-cryptenroll --wipe-slot=tpm2 "$LUKS_DEVICE"

# Regenerate UKI for the currently booted slot:
sudo gen-efi configure blue   # if booted into @blue
# or: sudo gen-efi configure green   # if booted into @green

# To update the other slot's UKI too:
sudo shani-deploy --force

Re-enroll After Enabling Secure Boot

If you enable Secure Boot after TPM enrollment:

# Get the device
LUKS_DEVICE=$(sudo cryptsetup status shani_root | grep 'device:' | awk '{print $2}')

# Remove old enrollment
sudo systemd-cryptenroll --wipe-slot=tpm2 "$LUKS_DEVICE"

# Re-enroll with Secure Boot protection
sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 "$LUKS_DEVICE"

# Regenerate UKI for the currently booted slot (e.g. @blue):
sudo gen-efi configure blue

# Then force-redeploy so the other slot also gets an updated UKI:
sudo shani-deploy --force

System Updates

Automatic Update Checker

ShaniOS includes a background service that automatically checks for updates:

• shani-update: Runs as a per-user systemd service (shani-update.timer), firing 5 minutes after login then every 2 hours. It is slot-aware: if the system detects it is running from the candidate slot (a post-update test boot), it silently exits and sends a notification asking the user to validate the new system before the next update cycle. When an update is available it presents a GUI dialog (yad for X11/Wayland with backend fallback, then zenity, kdialog, or console) asking the user to install or defer. If deferred, it schedules a follow-up via systemd-run --user --on-active=86400s (24 hours). If approved, it opens a terminal window running pkexec shani-deploy under systemd-inhibit, monitors completion via a status file, and on success presents a reboot prompt. Logs are kept in ~/.cache/shani-update.log with automatic 1 MB rotation.

Manual Update

sudo shani-deploy

The update process includes:

1. Self-update check: shani-deploy fetches the latest version of itself from GitHub and re-executes if a newer version is found (can be skipped with --skip-self-update)
2. System inhibit: prevents sleep, shutdown, and lid-close events for the duration of the update
3. Prerequisites check (root access, internet connectivity, disk space — minimum 10 GB free)
4. Boot validation: reads /data/current-slot and confirms it matches the running slot; detects and warns on slot mismatch
5. Mirror discovery (R2 CDN primary at downloads.shani.dev, SourceForge mirror as fallback)
6. Intelligent download with resume support (up to 5 attempts; R2 supports resume, SourceForge restarts on failure)
7. Verification: SHA256 checksum and GPG signature (key ID 7B927BFF...8014792)
8. Btrfs snapshot backup of the candidate slot before overwriting (named <slot>_backup_<timestamp>)
9. Deployment: extract new image into a temporary subvolume via zstd | btrfs receive, then snapshot to @candidate and set read-only
10. Filesystem verification: ensures all required subvolumes exist; creates missing ones (including swapfile at RAM size if @swap is new)
11. UKI generation: chroots into the new candidate slot and runs gen-efi configure <slot>
12. Boot entry update: candidate slot marked "Candidate", active slot stays "Active"; loader.conf default set to active slot
13. Cleanup: keeps only the latest backup per slot, removes old download files
14. Storage analysis: prints filesystem usage and per-subvolume compression stats via compsize

Update Channels

ShaniOS supports multiple update channels:

• stable (default): Thoroughly tested releases
• latest: Newest features (may be less stable)

Versions use a date-based format (YYYYMMDD, e.g. 20251201). The channel manifest (stable.txt or latest.txt) is fetched from SourceForge and contains the current image filename. Version comparison is lexicographic — newer dates are always greater — and the user-space checker (shani-update) also handles profile changes at the same version date as a valid update trigger.

Set your preferred channel:

# Choose channel
sudo shani-deploy -t latest

# Or set permanently
echo "latest" | sudo tee /etc/shani-channel

Update Options

# Show help
sudo shani-deploy --help

# Rollback to previous version (restores from Btrfs backup snapshot)
sudo shani-deploy --rollback

# Manual cleanup of old backups and downloaded images
sudo shani-deploy --cleanup

# Storage analysis: filesystem usage and per-subvolume compression stats
sudo shani-deploy --storage-info

# Manual deduplication run across @blue, @green, and backup subvolumes
# (bees handles continuous background dedup; this is for on-demand use)
sudo shani-deploy --optimize

# Dry run (simulate without making changes)
sudo shani-deploy --dry-run

# Force redeploy even if already on latest version
sudo shani-deploy --force

# Verbose output for troubleshooting
sudo shani-deploy --verbose

# Skip self-update of the deploy tool
sudo shani-deploy --skip-self-update

shani-deploy Quick Reference

Recovery: System Won't Boot

Firmware Updates (fwupd / LVFS)

fwupd is pre-installed and fwupd-refresh.timer runs automatically to check for new firmware. Update BIOS, NVMe controllers, SSD firmware, keyboard firmware, Thunderbolt devices, and other hardware supported by the Linux Vendor Firmware Service — no Windows, no USB boot drive, no manufacturer tools required.

# Refresh the LVFS metadata (normally automatic via fwupd-refresh.timer)
fwupdmgr refresh

# Check for available firmware updates
fwupdmgr get-updates

# Apply all available firmware updates
fwupdmgr update

# List all hardware devices fwupd can manage
fwupdmgr get-devices

# Show details and update history for a specific device
fwupdmgr get-details <device-id>

# Downgrade to a previous firmware version (if available)
fwupdmgr downgrade

All Ecosystem Updates — Quick Reference

# OS (always use shani-deploy — never pacman -Syu)
sudo shani-deploy

# Flatpak apps (also auto-updates every 12 hours via timer)
flatpak update

# Snap packages (also auto-updates in background)
snap refresh

# Nix packages (imperative profile)
nix-channel --update && nix-env -u '*'

# Nix flakes
nix flake update

# Home Manager (if installed)
home-manager switch

# Podman container images
podman auto-update

# Firmware (BIOS, NVMe, SSD controllers, Thunderbolt, etc.)
fwupdmgr refresh && fwupdmgr update

Shell & Environment

The default shell is Zsh. All user shell configuration lives in /home and is fully writable.

• Add environment variables permanently: append to ~/.zshrc (Zsh), ~/.bashrc (Bash), or ~/.config/fish/config.fish (Fish).
• System-wide environment variables: edit /etc/environment — this uses the /etc overlay and persists across all updates.
• Change your default shell: chsh -s /bin/fish (or /bin/bash, /bin/zsh). Changes take effect at next login. The shell binary must be listed in /etc/shells.
• Starship prompt customisation: edit ~/.config/starship.toml. Full reference at starship.rs/config.

McFly — Neural Network Shell History

McFly replaces the standard Ctrl+R history search with a neural network that learns from your usage patterns — prioritising commands by current directory, recent context, and exit codes. The longer you use it, the better it predicts what you need.

# Trigger McFly search (replaces standard Ctrl+R)
Ctrl+R

# McFly trains on your shell history automatically.
# The database grows smarter over time and is never cleared on updates.
# Database location:
ls ~/.local/share/mcfly/

# View McFly's training data size
du -sh ~/.local/share/mcfly/history.db

# Temporarily disable McFly and use default history search:
MCFLY_DISABLE=true zsh

# Delete McFly history and start fresh (rare)
rm ~/.local/share/mcfly/history.db

Profile Sync Daemon (PSD) — Browser Profiles in RAM

PSD moves your browser profiles to a tmpfs RAM filesystem, then syncs them back to disk periodically and on shutdown. The result is faster browser startup, faster tab loading, and substantially less SSD write wear from day-to-day browsing. It is enabled globally for all users on ShaniOS.

# Check PSD status
systemctl --user status psd

# Preview what PSD manages (dry run)
psd preview

# PSD supports all major browsers automatically:
# Vivaldi, Firefox, Chromium, Chrome, Brave, Opera, and more.
# Profiles are detected from their default locations.

# Manually sync profiles back to disk now (normally automatic)
psd sync

# PSD sync on shutdown is handled automatically by the systemd user unit.
# Your data is safe — sync runs every few minutes and on every logout.

# Log location:
journalctl --user -u psd -f

FZF — Fuzzy Finder Integration

# Fuzzy search shell history (alternative to McFly)
Ctrl+R

# Fuzzy insert a file path into the command line
Ctrl+T

# Fuzzy cd into a subdirectory
Alt+C

# Pipe any command output through fzf
ls | fzf
cat /etc/passwd | fzf

System Configuration

The /etc directory uses overlay filesystem, allowing modifications:

# Edit system configuration normally
sudo nano /etc/some-config-file

# Changes are stored in /data/overlay/etc/upper
# and persist across all system updates and slot switches

# List everything you've changed from defaults
ls -la /data/overlay/etc/upper

# Compare your changes with the current base
diff -r /etc /data/overlay/etc/upper 2>/dev/null | grep -v "^Only in /data"

# To revert a single file to its system default:
sudo rm /data/overlay/etc/upper/path/to/file

Software Management

ShaniOS uses Flatpak as the primary application delivery method, with Snaps for sandboxed cross-distro packages, AppImages for portable self-contained binaries, Nix for CLI tools, containers for development environments, and optionally Homebrew for cross-platform tooling.

# Search for applications
flatpak search keyword

# Install from Flathub
flatpak install flathub org.application.Name

# List installed apps
flatpak list

# Remove an app
flatpak uninstall org.application.Name

flatpak install flathub com.valvesoftware.Steam
flatpak install flathub com.heroicgameslauncher.hgl
flatpak install flathub net.lutris.Lutris
flatpak install flathub org.libretro.RetroArch

flatpak install flathub io.github.antimicrox.antimicrox  # Gamepad mapper
flatpak install flathub org.freedesktop.Piper            # Mouse configuration
flatpak install flathub org.openrgb.OpenRGB              # RGB lighting

flatpak install flathub io.github.benjamimgois.goverlay  # Overlay manager
flatpak install flathub org.freedesktop.Platform.VulkanLayer.MangoHud
flatpak install flathub org.freedesktop.Platform.VulkanLayer.gamescope

flatpak install flathub com.usebottles.bottles

# Search for a snap
snap find keyword

# Install a snap (strict confinement)
snap install app-name

# Install a snap with classic confinement
snap install app-name --classic

# List installed snaps
snap list

# Check available updates
snap refresh --list

# Update all snaps
snap refresh

# Update a specific snap
snap refresh app-name

# Roll back a snap to the previous revision
snap revert app-name

# Remove a snap
snap remove app-name

# Run a snap
snap run app-name

# List all interfaces for a snap
snap connections app-name

# Connect an interface manually
snap connect app-name:camera

# Disconnect an interface
snap disconnect app-name:camera

# List all available interfaces on the system
snap interface

# Check snapd status
sudo systemctl status snapd

# View snapd logs
journalctl -u snapd -f

# Check AppArmor confinement status
sudo apparmor_status | grep snap

# Developer tools
snap install code --classic          # VS Code
snap install sublime-text --classic  # Sublime Text
snap install android-studio --classic

# Communication
snap install slack --classic
snap install discord

# Utilities
snap install bitwarden
snap install multipass               # Ubuntu VM manager

# Open Gear Lever from your app launcher, or launch it from terminal
gear-lever

# Gear Lever automatically:
# - Registers the AppImage in your .local/share/applications
# - Creates a desktop entry with icon
# - Associates MIME types if declared in the AppImage
# - Checks for AppImageUpdate-compatible update feeds

# Make an AppImage executable
chmod +x MyApp-x86_64.AppImage

# Run it directly
./MyApp-x86_64.AppImage

# Optionally move it to a persistent location
mkdir -p ~/Applications
mv MyApp-x86_64.AppImage ~/Applications/

# Extract the contents without running (for inspection)
./MyApp-x86_64.AppImage --appimage-extract

# --- Basic Usage ---
# Pull an image
podman pull docker.io/library/nginx:latest
podman pull quay.io/fedora/fedora:latest

# Run a container interactively
podman run -it --rm ubuntu:22.04 bash

# Run a detached service (web server on port 8080)
podman run -d --name myapp -p 8080:80 nginx:latest

# List running containers
podman ps

# List all containers (including stopped)
podman ps -a

# Stop / start / restart
podman stop myapp
podman start myapp
podman restart myapp

# Remove container and image
podman rm myapp
podman rmi nginx:latest

# View logs
podman logs -f myapp

# Exec into a running container
podman exec -it myapp bash

# --- Volumes & Mounts ---
# Named volume (managed by Podman)
podman volume create mydata
podman run -d -v mydata:/app/data nginx:latest

# Bind mount (host path → container path)
podman run -d -v /home/user/website:/usr/share/nginx/html:Z nginx:latest
# :Z sets correct SELinux/AppArmor labels

# List volumes
podman volume ls
podman volume inspect mydata

# --- Networking ---
# Default bridge network
podman network ls
podman network create mynet

# Run two containers on the same network (they see each other by name)
podman run -d --name db --network mynet postgres:16
podman run -d --name app --network mynet -p 3000:3000 myapp:latest

# Port mapping
podman run -d -p 127.0.0.1:8080:80 nginx  # localhost only
podman run -d -p 8080:80 nginx             # all interfaces

# --- Pods (multi-container groups, like Kubernetes pods) ---
# Create a pod with a shared network namespace
podman pod create --name myapp-pod -p 8080:80

# Add containers to the pod
podman run -d --pod myapp-pod --name frontend nginx:latest
podman run -d --pod myapp-pod --name backend node:20

# Manage the entire pod
podman pod start myapp-pod
podman pod stop myapp-pod
podman pod rm myapp-pod

# List pods
podman pod ls

# --- Systemd Integration (auto-start containers at boot) ---
# Generate a systemd unit for a running container
podman generate systemd --new --name myapp > ~/.config/systemd/user/myapp.service

# Enable auto-start at login
systemctl --user daemon-reload
systemctl --user enable --now myapp

# Or use quadlet (modern declarative approach, Podman 4.4+)
# Create ~/.config/containers/systemd/myapp.container:
# [Unit]
# Description=My App
# [Container]
# Image=docker.io/library/nginx:latest
# PublishPort=8080:80
# Volume=/home/user/html:/usr/share/nginx/html:Z
# [Install]
# WantedBy=default.target
systemctl --user daemon-reload
systemctl --user start myapp

# Example docker-compose.yml — WordPress + MariaDB
# Save to ~/projects/wordpress/docker-compose.yml

# version: "3.9"
# services:
#   db:
#     image: mariadb:11
#     environment:
#       MYSQL_ROOT_PASSWORD: rootpass
#       MYSQL_DATABASE: wordpress
#       MYSQL_USER: wp
#       MYSQL_PASSWORD: wppass
#     volumes:
#       - db_data:/var/lib/mysql
#
#   wordpress:
#     image: wordpress:latest
#     ports:
#       - "8080:80"
#     environment:
#       WORDPRESS_DB_HOST: db
#       WORDPRESS_DB_USER: wp
#       WORDPRESS_DB_PASSWORD: wppass
#       WORDPRESS_DB_NAME: wordpress
#     depends_on:
#       - db
#
# volumes:
#   db_data:

# Run the stack
podman-compose up -d

# Or via the docker drop-in
docker compose up -d

# View logs
podman-compose logs -f

# Stop and remove
podman-compose down

# Stop and remove including volumes
podman-compose down -v

# --- Build from a Dockerfile ---

# Example Dockerfile (save as ~/myapp/Dockerfile):
# FROM node:20-alpine
# WORKDIR /app
# COPY package*.json ./
# RUN npm ci --only=production
# COPY . .
# EXPOSE 3000
# CMD ["node", "server.js"]

# Build the image (bud = Build Using Dockerfile)
buildah bud -t myapp:latest ~/myapp/

# Or using podman (calls buildah under the hood)
podman build -t myapp:latest ~/myapp/

# Build with build arguments
buildah bud --build-arg NODE_ENV=production -t myapp:prod .

# Build for a specific platform
buildah bud --platform linux/amd64 -t myapp:amd64 .
buildah bud --platform linux/arm64 -t myapp:arm64 .

# Multi-platform manifest
buildah manifest create myapp:multi
buildah bud --platform linux/amd64,linux/arm64 \
  --manifest myapp:multi .

# --- Scripted builds (buildah native API) ---
# More control than Dockerfile — useful for complex layering

# Start from a base image
ctr=$(buildah from ubuntu:22.04)

# Run commands inside (each becomes a layer)
buildah run $ctr -- apt-get update
buildah run $ctr -- apt-get install -y python3 python3-pip
buildah run $ctr -- pip3 install flask

# Copy files in
buildah copy $ctr ./app /opt/app

# Set metadata
buildah config --entrypoint '["python3", "/opt/app/main.py"]' $ctr
buildah config --port 5000 $ctr
buildah config --label maintainer="[email protected]" $ctr

# Commit to a named image
buildah commit $ctr myflaskapp:latest

# Remove the working container
buildah rm $ctr

# List images
buildah images

# Inspect an image
buildah inspect myflaskapp:latest

# --- Push images to a registry ---

# Push to Docker Hub
buildah push myapp:latest docker://docker.io/yourusername/myapp:latest

# Push to Quay.io
buildah push myapp:latest docker://quay.io/yourusername/myapp:latest

# Push to a local registry (e.g., Podman's built-in registry)
podman run -d -p 5000:5000 --name registry docker.io/library/registry:2
buildah push myapp:latest docker://localhost:5000/myapp:latest

# Push to an OCI archive (tarball)
buildah push myapp:latest oci-archive:/tmp/myapp.tar

# Tag an image
buildah tag myapp:latest myapp:1.0.0

# --- Create containers ---
# Ubuntu 22.04
distrobox create --name ubuntu --image ubuntu:22.04

# Fedora latest
distrobox create --name fedora --image fedora:latest

# Arch Linux (for AUR access)
distrobox create --name arch --image archlinux:latest

# With custom home directory
distrobox create --name dev --image ubuntu:22.04 --home ~/distrobox-dev

# List all boxes
distrobox list

# Enter a box
distrobox enter ubuntu

# Run a single command without entering
distrobox run --name ubuntu -- apt list --installed

# --- Inside the box: install anything ---
# In the ubuntu box:
sudo apt update && sudo apt install -y nodejs npm python3-pip gcc make

# In the arch box (full AUR access with yay):
sudo pacman -S yay
yay -S some-aur-package

# --- Export apps & commands to host desktop ---
# Export a GUI app to your host launcher (appears in GNOME/KDE app menu)
distrobox-export --app firefox       # from inside the box
distrobox-export --app code          # VS Code installed inside box

# Export a CLI binary to host PATH (~/.local/bin/)
distrobox-export --bin /usr/bin/node --export-path ~/.local/bin
distrobox-export --bin /usr/bin/python3 --export-path ~/.local/bin

# Remove an exported app/bin
distrobox-export --app firefox --delete
distrobox-export --bin /usr/bin/node --export-path ~/.local/bin --delete

# --- Manage boxes ---
# Stop a running box
distrobox stop ubuntu

# Remove a box (keeps /home data)
distrobox rm ubuntu

# Remove box and delete its home dir
distrobox rm ubuntu --rm-home

# Upgrade all packages inside a box
distrobox run --name ubuntu -- sudo apt upgrade -y

# Generate a systemd unit to auto-start a box at login
distrobox generate-entry ubuntu

# List available images
lxc image list images: | grep ubuntu

# Launch a container
lxc launch images:ubuntu/22.04 my-server

# Launch Fedora
lxc launch images:fedora/39 fedora-box

# List running containers
lxc list

# Open a shell in the container
lxc exec my-server -- bash

# Copy files to/from container
lxc file push ~/myconfig.conf my-server/etc/myconfig.conf
lxc file pull my-server/var/log/syslog ~/syslog.txt

# Start / stop / restart
lxc start my-server
lxc stop my-server
lxc restart my-server

# Snapshot and restore
lxc snapshot my-server snap0
lxc restore my-server snap0

# Delete container
lxc delete my-server

# Initialise LXD (first-time setup — runs an interactive wizard)
sudo lxd init

# List available images from the LXD image server
lxc image list ubuntu: | head -20

# Launch an Ubuntu container
lxc launch ubuntu:22.04 web-server

# Launch a full VM (not a container)
lxc launch ubuntu:22.04 my-vm --vm

# List all instances (containers + VMs)
lxc list

# Get a shell
lxc exec web-server -- bash

# Show resource usage
lxc info web-server

# Profiles — apply predefined resource limits
lxc profile list
lxc profile show default
lxc profile apply web-server default

# Storage pools
lxc storage list
lxc storage info default

# Snapshot and restore
lxc snapshot web-server clean-install
lxc restore web-server clean-install

# Stop and delete
lxc stop web-server
lxc delete web-server

# Pull an image from Docker Hub (converted to SIF format automatically)
apptainer pull docker://ubuntu:22.04
# Creates: ubuntu_22.04.sif

# Pull from a specific registry
apptainer pull docker://ghcr.io/owner/repo:tag

# Run a command inside a container
apptainer exec ubuntu_22.04.sif python3 script.py

# Run the container's default entrypoint
apptainer run ubuntu_22.04.sif

# Interactive shell in the container
apptainer shell ubuntu_22.04.sif

# Bind mount host directories (automatic: $HOME, /tmp, /proc)
apptainer exec --bind /data:/mnt/data ubuntu_22.04.sif ls /mnt/data

# Use GPU (NVIDIA) inside the container
apptainer exec --nv cuda_image.sif python3 gpu_script.py

# Build a custom SIF from a definition file
apptainer build myimage.sif myimage.def

# Build a sandbox (writable directory, for development)
apptainer build --sandbox myimage_sandbox/ myimage.def

# Inspect an image's metadata
apptainer inspect ubuntu_22.04.sif

# Cache location (default: ~/.apptainer/cache)
ls ~/.apptainer/cache/

# Clear cache
apptainer cache clean

# Bootstrap a Debian container into /var/lib/machines/debian12
sudo debootstrap stable /var/lib/machines/debian12 http://deb.debian.org/debian

# Start it as a machine (boots fully with systemd)
sudo machinectl start debian12

# Or start interactively
sudo systemd-nspawn -D /var/lib/machines/debian12 --boot

# List running machines
machinectl list

# Open a shell in a running machine
machinectl shell debian12

# Enable auto-start at boot
machinectl enable debian12

# Stop / poweroff
machinectl stop debian12
machinectl poweroff debian12

# Transfer files
machinectl copy-to debian12 ~/file.txt /root/file.txt
machinectl copy-from debian12 /etc/hostname ~/hostname.txt

# Add the nixpkgs-unstable channel (recommended for most users on ShaniOS)
nix-channel --add https://nixos.org/channels/nixpkgs-unstable nixpkgs

# Fetch the channel expressions and binary cache metadata
nix-channel --update

# Verify — should print: nixpkgs https://nixos.org/channels/nixpkgs-unstable
nix-channel --list

# Stable channel (replace 25.05 with the current release)
nix-channel --add https://nixos.org/channels/nixpkgs-25.05 nixpkgs
nix-channel --update

# List subscribed channels
nix-channel --list

# Add an additional channel (e.g. home-manager)
nix-channel --add https://github.com/nix-community/home-manager/archive/master.tar.gz home-manager

# Remove a channel
nix-channel --remove home-manager

# Update all channels (download latest expressions)
nix-channel --update

# Update a specific channel only
nix-channel --update nixpkgs

# List all channel generations (history of updates)
nix-channel --list-generations

# Roll back channels to the previous generation
nix-channel --rollback

# Roll back to a specific generation number
nix-channel --rollback 12

# Search for packages (after channel is set)
nix search nixpkgs ripgrep
nix-env -qaP | grep nodejs          # list all matching in channel

# Install a package into your default profile
nix-env -iA nixpkgs.ripgrep
nix-env -iA nixpkgs.nodejs_22
nix-env -iA nixpkgs.python312

# List installed packages
nix-env -q

# Upgrade a specific package
nix-env -uA nixpkgs.ripgrep

# Upgrade all installed packages to latest channel versions
nix-env -u '*'

# Uninstall a package
nix-env -e ripgrep

# Roll back the last nix-env operation (install/upgrade/remove)
nix-env --rollback

# List profile generations
nix-env --list-generations

# Switch to a specific profile generation
nix-env --switch-generation 42

# Drop into a temporary shell with specific packages (gone when you exit)
nix-shell -p python312 nodejs_22 gcc

# New-style equivalent (Nix flakes / nix CLI)
nix shell nixpkgs#python312 nixpkgs#nodejs_22

# Run a single command from a package without installing it
nix run nixpkgs#cowsay -- "Hello from Nix"
nix run nixpkgs#ffmpeg -- -i input.mp4 output.webm

# Start a per-project dev shell defined in a shell.nix file
nix-shell                           # reads shell.nix in current directory

# Step 1: fetch latest channel expressions
nix-channel --update

# Step 2: upgrade all installed packages to the versions now in the channel
nix-env -u '*'

# Or do both in one line
nix-channel --update && nix-env -u '*'

# Delete all old profile generations and collect unreachable store paths
nix-collect-garbage -d

# Collect garbage but keep the last N generations
nix-env --delete-generations old
nix-collect-garbage

# Just see how much space would be freed (dry run)
nix-collect-garbage --dry-run

# Check current store disk usage
du -sh /nix/store

export XDG_DATA_DIRS=$HOME/.nix-profile/share:$XDG_DATA_DIRS

# Add the home-manager channel matching your nixpkgs channel
nix-channel --add https://github.com/nix-community/home-manager/archive/master.tar.gz home-manager
nix-channel --update

# Install home-manager (standalone)
export NIX_PATH=$HOME/.nix-defexpr/channels:/nix/var/nix/profiles/per-user/root/channels${NIX_PATH:+:$NIX_PATH}
nix-shell '<home-manager>' -A install

# After install — edit your configuration
micro ~/.config/home-manager/home.nix   # or nano, vim, etc.

# Apply your configuration
home-manager switch

# List home-manager generations
home-manager generations

# Roll back to the previous generation
home-manager rollback

{ config, pkgs, ... }: {
  home.username = "alice";
  home.homeDirectory = "/home/alice";
  home.stateVersion = "25.05";

  home.packages = with pkgs; [
    ripgrep
    fd
    jq
    nodejs_22
    python312
  ];

  programs.zsh.enable = true;
  programs.git = {
    enable = true;
    userName  = "Alice";
    userEmail = "[email protected]";
  };
}

# Enable flakes and the new nix CLI (add to /etc/nix/nix.conf)
experimental-features = nix-command flakes

# Reload the daemon after editing
sudo systemctl restart nix-daemon

# With flakes enabled — use the new nix CLI
nix search nixpkgs#ripgrep
nix shell nixpkgs#ripgrep
nix run nixpkgs#hello

# Run the official install script (installs to /home/linuxbrew/.linuxbrew)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Add to shell config (~/.zshrc for zsh, ~/.bashrc for bash)
echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> ~/.zshrc

# Apply immediately in current session
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"

# Verify installation
brew doctor
brew --version

# Search for a package
brew search htop

# Install a package
brew install htop

# List installed packages
brew list

# Update Homebrew and all formulae
brew update && brew upgrade

# Remove a package
brew uninstall htop

# Show info about a package
brew info git

# Uninstall Homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)"

# Recommended: waydroid-helper automates the full init process
sudo waydroid-helper init

# Or manually, if you prefer:
sudo waydroid init

# Start the Waydroid session
sudo systemctl start waydroid-container
waydroid show-full-ui

# Check Waydroid status
waydroid status

# Install an APK from host (sideloading)
waydroid app install /path/to/app.apk

# List installed Android apps
waydroid app list

# Launch a specific app
waydroid app launch com.example.app

# Open the full Android UI
waydroid show-full-ui

# Get Waydroid IP (for ADB connection)
waydroid status | grep IP

# Connect ADB to Waydroid
adb connect 192.168.240.112:5555

# Now use standard adb commands against Waydroid
adb devices
adb logcat
adb shell

# Stop the Waydroid session
waydroid session stop

# Stop the container service
sudo systemctl stop waydroid-container

# Disable auto-start at boot (if you don't use Waydroid regularly)
sudo systemctl disable waydroid-container

# Reset Waydroid completely (wipes all Android data)
sudo waydroid init -f

# Should return /dev/kvm if VT-x / AMD-V is enabled in BIOS
ls /dev/kvm

# Check KVM kernel modules
lsmod | grep kvm

flatpak install flathub org.gnome.Boxes

flatpak install flathub org.virt_manager.virt-manager

# Or install the QEMU extension alongside for full feature set
flatpak install flathub org.virt_manager.virt-manager
flatpak install flathub org.virt_manager.virt-manager.Extension.Qemu

# Install via Nix
nix-env -iA nixpkgs.quickemu

# Create and start an Ubuntu VM
quickget ubuntu 22.04
quickemu --vm ubuntu-22.04.conf

# List available OS options
quickget --list

# Using quickemu (easiest)
quickget windows 11
quickemu --vm windows-11.conf

# Using GNOME Boxes: File → New → Download OS → Windows 11
# (Boxes handles VirtIO drivers automatically)

# For virt-manager: use the VirtIO disk and network drivers
# for best performance — download virtio-win.iso from Fedora

Web Hosting & Networking

ShaniOS lets you host websites and services directly from your desktop or laptop—even on home internet connections without a static IP. All tools listed here are installed for convenience and are not enabled or running by default.

How Desktop Web Hosting Works

Networking Tools Reference

Caddy Web Server

Modern, zero-config web server with automatic HTTPS via Let's Encrypt. Ideal for hosting static sites, reverse proxying containers, dashboards, and APIs directly from your desktop. Not active by default — start it when you need it.

Start & enable:

sudo systemctl enable --now caddy

# Reload config without downtime
sudo systemctl reload caddy

# View logs
sudo journalctl -u caddy -f

# Validate Caddyfile syntax before reloading
caddy validate --config /etc/caddy/Caddyfile

Static site with automatic HTTPS:

# /etc/caddy/Caddyfile

mysite.example.com {
    root * /var/www/html
    file_server
    encode gzip zstd
    log {
        output file /var/log/caddy/access.log
    }
}

# Local dev site (no certificate needed)
:8080 {
    root * /home/user/my-website
    file_server browse
}

Reverse proxy — forward to a container or app:

app.example.com {
    reverse_proxy localhost:3000
}

# Multiple subdomains → different backend services
api.example.com     { reverse_proxy localhost:8000 }
dashboard.example.com { reverse_proxy localhost:9090 }

# Load balance across replicas
lb.example.com {
    reverse_proxy localhost:3001 localhost:3002 localhost:3003 {
        lb_policy round_robin
        health_uri /health
        health_interval 10s
    }
}

Basic auth, headers, rate limiting:

# Generate bcrypt password hash
caddy hash-password --plaintext "yourpassword"

# Caddyfile with auth + security headers
secure.example.com {
    basicauth {
        admin $2a$14$YOUR_HASH_HERE
    }
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options nosniff
        X-Frame-Options DENY
        -Server
    }
    reverse_proxy localhost:8080
}

# Rate limit (requires caddy-ratelimit module, install via Flatpak/container)
# For production rate limiting, deploy Caddy inside a container with plugins

Serve from a Podman container:

# Run a web app container and proxy it with Caddy
podman run -d --name webapp -p 127.0.0.1:3000:3000 myapp:latest

# Caddyfile entry
myapp.example.com {
    reverse_proxy 127.0.0.1:3000
}

sudo systemctl reload caddy

TLS certificates and ACME state are stored in /data/varlib/caddy and persist across all system updates and rollbacks.

Cloudflared — Zero-Trust Tunnels

Creates an encrypted outbound-only tunnel to Cloudflare's edge, exposing local services publicly without opening inbound firewall ports or having a static IP. Disabled by default.

# Authenticate with your Cloudflare account
cloudflared tunnel login

# Create a named tunnel
cloudflared tunnel create my-tunnel

# Configure ingress — edit ~/.cloudflared/config.yml
# tunnel: <YOUR-TUNNEL-UUID>
# credentials-file: /home/user/.cloudflared/<UUID>.json
# ingress:
#   - hostname: mysite.example.com
#     service: http://localhost:8080
#   - hostname: api.example.com
#     service: http://localhost:3000
#   - service: http_status:404

# Route DNS to this tunnel (Cloudflare manages DNS record automatically)
cloudflared tunnel route dns my-tunnel mysite.example.com

# Test tunnel locally before enabling as service
cloudflared tunnel run my-tunnel

# Install and enable as a persistent systemd service
sudo cloudflared service install
sudo systemctl enable --now cloudflared

# Check tunnel status
cloudflared tunnel list
cloudflared tunnel info my-tunnel
sudo journalctl -u cloudflared -f

Tunnel credentials are stored at /data/varlib/cloudflared and survive all updates. No inbound firewall changes are needed — Cloudflared establishes only outbound HTTPS connections.

Tailscale — Private WireGuard Mesh

Builds a private peer-to-peer encrypted network between all your devices using WireGuard. All traffic is authenticated; devices are only reachable by other Tailscale nodes. Inactive until you sign in.

# Enable the daemon
sudo systemctl enable --now tailscaled

# Authenticate (opens browser)
sudo tailscale up

# Bring up with specific options
sudo tailscale up --accept-routes --ssh   # accept subnet routes; enable Tailscale SSH
sudo tailscale up --advertise-exit-node   # act as exit node (route all traffic)
sudo tailscale up --advertise-routes=192.168.1.0/24  # expose your LAN to the tailnet

# Status and peer list
tailscale status
tailscale netcheck           # diagnose NAT/relay connectivity

# Check your Tailscale IP
tailscale ip -4

# Ping a peer by its Tailscale hostname
tailscale ping myhostname

# Open UDP 41641 for best performance (optional — works without)
sudo firewall-cmd --add-port=41641/udp --permanent
sudo firewall-cmd --reload

# Built-in SSH (no OpenSSH needed on target)
tailscale ssh user@myhostname

Tailscale state is bind-mounted from /data/varlib/tailscale and persists across all updates — you stay authenticated and connected after every system update.

OpenSSH — Remote Shell & File Transfer

SSH server is not enabled by default. Enable it only when needed; prefer access via Tailscale rather than exposing port 22 publicly.

# Enable SSH server
sudo systemctl enable --now sshd

# Harden /etc/ssh/sshd_config (key settings)
sudo nano /etc/ssh/sshd_config
# PasswordAuthentication no      ← key-only auth
# PermitRootLogin no
# Port 2222                       ← non-default port (optional)
# AllowUsers youruser             ← restrict to specific users
# MaxAuthTries 3

sudo systemctl restart sshd

# Allow SSH in firewall (LAN only is safest)
sudo firewall-cmd --add-service=ssh --permanent
sudo firewall-cmd --reload

# Generate an ED25519 key pair on the CLIENT
ssh-keygen -t ed25519 -C "mydesktop"

# Copy public key to server
ssh-copy-id [email protected]
# or manually append ~/.ssh/id_ed25519.pub to server's ~/.ssh/authorized_keys

# Connect
ssh [email protected]
ssh -p 2222 [email protected]   # non-default port

# Tunnel a local port via SSH (port forward)
ssh -L 8080:localhost:3000 [email protected]   # access server's :3000 at localhost:8080

# Transfer files
scp localfile.txt [email protected]:/home/user/
rsync -avz /local/dir/ [email protected]:/remote/dir/

# SFTP interactive session
sftp [email protected]

firewalld — Dynamic Firewall

Active by default with restrictive rules. Manages nftables under the hood using a zone-based model. Pre-configured zones for KDE Connect and Waydroid are included.

# Status and current rules
sudo firewall-cmd --state
sudo firewall-cmd --list-all                      # active zone
sudo firewall-cmd --list-all-zones                # all zones

# Open a service (named)
sudo firewall-cmd --add-service=http --permanent
sudo firewall-cmd --add-service=https --permanent
sudo firewall-cmd --add-service=ssh --permanent
sudo firewall-cmd --reload

# Open a specific port
sudo firewall-cmd --add-port=8080/tcp --permanent
sudo firewall-cmd --add-port=41641/udp --permanent  # Tailscale
sudo firewall-cmd --reload

# Remove a rule
sudo firewall-cmd --remove-service=http --permanent
sudo firewall-cmd --remove-port=8080/tcp --permanent
sudo firewall-cmd --reload

# Lock down to a source IP (allow SSH only from LAN)
sudo firewall-cmd --add-rich-rule='rule family="ipv4" source address="192.168.1.0/24" service name="ssh" accept' --permanent
sudo firewall-cmd --reload

# Block an IP
sudo firewall-cmd --add-rich-rule='rule family="ipv4" source address="1.2.3.4" reject' --permanent
sudo firewall-cmd --reload

# GUI: firewall-config is pre-installed
sudo firewall-config

Fail2ban — Brute-Force Protection

Monitors logs and temporarily bans IPs that fail authentication repeatedly. Integrates with firewalld automatically. Not enabled by default — enable when running any public-facing service.

# Enable and start
sudo systemctl enable --now fail2ban

# Check overall status
sudo fail2ban-client status

# Check SSH jail
sudo fail2ban-client status sshd

# Manually ban/unban an IP
sudo fail2ban-client set sshd banip 1.2.3.4
sudo fail2ban-client set sshd unbanip 1.2.3.4

# View banned IPs from firewall perspective
sudo firewall-cmd --direct --get-all-rules

# Watch the fail2ban log live
sudo journalctl -u fail2ban -f

Custom jail for Caddy (HTTP brute-force):

# /etc/fail2ban/jail.d/caddy.conf
# [caddy]
# enabled  = true
# port     = http,https
# filter   = caddy
# logpath  = /var/log/caddy/access.log
# maxretry = 10
# bantime  = 3600
# findtime = 600

sudo systemctl restart fail2ban

NFS — Network File System

Native Linux file sharing at near-local disk speeds. Best for Linux-to-Linux sharing on a trusted LAN. State bind-mounted from /data/varlib/nfs.

# ── SERVER ──────────────────────────────────
sudo systemctl enable --now nfs-server

# Define exports in /etc/exports
# /home/user/shared   192.168.1.0/24(rw,sync,no_subtree_check)
# /data/media         192.168.1.0/24(ro,sync,no_subtree_check)
echo "/home/user/shared  192.168.1.0/24(rw,sync,no_subtree_check)" | sudo tee -a /etc/exports
sudo exportfs -arv

# Allow through firewall
sudo firewall-cmd --add-service=nfs --permanent
sudo firewall-cmd --add-service=rpcbind --permanent
sudo firewall-cmd --add-service=mountd --permanent
sudo firewall-cmd --reload

# Check active exports
sudo exportfs -v
showmount -e localhost

# ── CLIENT ──────────────────────────────────
# Temporary mount
sudo mount -t nfs 192.168.1.100:/home/user/shared /mnt/remote

# Automount at boot — add to /etc/fstab
# 192.168.1.100:/home/user/shared  /mnt/remote  nfs  defaults,_netdev,nfsvers=4  0 0

# Check mount
mount | grep nfs

Samba — SMB/CIFS (Windows / macOS / Linux)

SMB/CIFS shares visible to Windows, macOS Finder, and other Linux machines. Samba state bind-mounted from /data/varlib/samba.

# Enable Samba services
sudo systemctl enable --now smb nmb

# Edit /etc/samba/smb.conf — add a share
sudo nano /etc/samba/smb.conf

# Example share (append to smb.conf):
# [SharedDocs]
#    comment = My Documents
#    path = /home/user/Documents
#    browseable = yes
#    read only = no
#    valid users = youruser
#    create mask = 0664
#    directory mask = 0775

# Set Samba password (separate from system password)
sudo smbpasswd -a youruser

# Apply config changes
sudo systemctl restart smb nmb

# Test config syntax
testparm

# Allow through firewall
sudo firewall-cmd --add-service=samba --permanent
sudo firewall-cmd --reload

# List active Samba shares
smbclient -L localhost -U youruser

# ── MOUNTING REMOTE SHARES ──────────────────
# From another Linux machine
sudo mount -t cifs //192.168.1.100/SharedDocs /mnt/samba \
  -o username=youruser,uid=$(id -u),gid=$(id -g),vers=3.0

# Persistent (add credentials file for security)
# echo "username=youruser" > ~/.smbcredentials
# echo "password=yourpass" >> ~/.smbcredentials
# chmod 600 ~/.smbcredentials
# Then in /etc/fstab:
# //192.168.1.100/SharedDocs /mnt/samba cifs credentials=/home/user/.smbcredentials,uid=1000,gid=1000,_netdev 0 0

SSHFS — Mount Remote Directories over SSH

Mount any directory from an SSH-accessible machine as a local folder. Requires only an SSH server on the remote — no special server software needed.

# Mount a remote directory
sshfs [email protected]:/home/user/projects ~/mnt/remote-projects

# Mount with specific options
sshfs [email protected]:/data ~/mnt/server \
  -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3

# Unmount
fusermount -u ~/mnt/remote-projects

# Auto-mount at login (add to /etc/fstab)
# [email protected]:/home/user/projects /home/user/mnt/remote fuse.sshfs defaults,_netdev,reconnect,uid=1000,gid=1000,IdentityFile=/home/user/.ssh/id_ed25519 0 0

dnsmasq — Local DNS, DHCP & Split DNS

Lightweight DNS forwarder and DHCP server for homelab setups. Use it for custom .home domains, split DNS (different resolution for LAN vs internet), or local ad-blocking. Not active by default.

sudo systemctl enable --now dnsmasq

# /etc/dnsmasq.conf — key examples:

# Custom local hostnames
# address=/nas.home/192.168.1.10
# address=/printer.home/192.168.1.20
# address=/desktop.home/192.168.1.50

# Upstream DNS servers
# server=1.1.1.1
# server=8.8.8.8

# Local DHCP range (if dnsmasq acts as DHCP server)
# dhcp-range=192.168.1.100,192.168.1.200,12h
# dhcp-option=3,192.168.1.1  ← default gateway

# Block trackers/ads by routing to 0.0.0.0
# address=/ads.doubleclick.net/0.0.0.0

# Restart after changes
sudo systemctl restart dnsmasq

# Point NetworkManager to use dnsmasq
# Add /etc/NetworkManager/conf.d/dns.conf:
# [main]
# dns=dnsmasq
sudo systemctl restart NetworkManager

Avahi — Zero-Config mDNS/DNS-SD (Bonjour)

Avahi is active by default — your machine is immediately reachable as hostname.local on the LAN. Used by CUPS (printers), KDE Connect, DLNA, and SSH discovery.

# Discover services on your network
avahi-browse -a                          # all services
avahi-browse -at                         # one-shot (no live view)
avahi-browse _http._tcp                  # HTTP services only
avahi-browse _ssh._tcp                   # SSH servers
avahi-browse _smb._tcp                   # Samba shares
avahi-browse _ipp._tcp                   # Printers

# Look up a .local hostname
avahi-resolve --name myhostname.local
avahi-resolve --address 192.168.1.50

# Publish a custom service
# Create /etc/avahi/services/caddy.service:
# <?xml version="1.0" standalone='no'?>
# <service-group>
#   <name>My Web Server</name>
#   <service><type>_http._tcp</type><port>8080</port></service>
# </service-group>

sudo systemctl status avahi-daemon

KDE Connect & GSConnect — Phone Integration

KDE Connect (KDE edition) and GSConnect (GNOME extension, pre-installed on GNOME edition) connect your Android phone to your Linux desktop over the local network. Features: notification sync, clipboard sharing, file transfer, remote control, media control, SMS from desktop, and running commands on the PC from your phone. The necessary firewall ports are pre-opened on ShaniOS — no manual firewall configuration needed.

# ── SETUP ───────────────────────────────────────────────────────────────
# 1. Install KDE Connect on your Android phone from Google Play or F-Droid
# 2. Ensure phone and PC are on the same WiFi network
# 3. Open KDE Connect on KDE, or click the GSConnect icon in GNOME top bar
# 4. The phone appears automatically — click "Pair" on both devices

# ── KDE CONNECT CLI (kdeconnect-cli) ─────────────────────────────────────
# List paired devices
kdeconnect-cli --list-devices
kdeconnect-cli --list-available   # discoverable devices on LAN

# Send a file to your phone
kdeconnect-cli --device "Phone Name" --share ~/Documents/file.pdf

# Ping your phone (find it)
kdeconnect-cli --device "Phone Name" --ping

# Send a text/notification to phone
kdeconnect-cli --device "Phone Name" --ping-msg "Hey, check this"

# Lock phone screen
kdeconnect-cli --device "Phone Name" --lock

# Ring phone (find it in your couch)
kdeconnect-cli --device "Phone Name" --ring

# ── FIREWALL ─────────────────────────────────────────────────────────────
# Ports are pre-opened on ShaniOS (1714-1764 TCP+UDP in public zone)
# To verify:
sudo firewall-cmd --list-all | grep 1714

# ── GSCONNECT (GNOME) ────────────────────────────────────────────────────
# GSConnect is a GNOME Shell extension — it appears as an icon in the top bar
# after pairing. No CLI; use the extension menu or the phone app directly.
# To enable GSConnect if not already active:
gnome-extensions enable [email protected]

Rygel — DLNA/UPnP Media Server

Streams your music, videos, and photos to smart TVs, game consoles (PS4/PS5/Xbox), Kodi, and any DLNA renderer on the LAN. GNOME edition only. Not active by default.

# Start Rygel as a user service
systemctl --user enable --now rygel

# Configure — ~/.config/rygel.conf
# [MediaExport]
# enabled=true
# uris=/home/user/Music;/home/user/Videos;/home/user/Pictures

# Check discovery
avahi-browse -a | grep -i rygel

# View logs
journalctl --user -u rygel -f

NetworkManager — Connection & VPN Management

NetworkManager is pre-installed and active by default. All network connections — wired, Wi-Fi, mobile broadband, and VPN — are managed through it. Use nmcli for scripting, nmtui for a terminal UI, or the full nm-connection-editor GUI.

# --- Connection management ---
nmcli device status                        # show all interfaces
nmcli connection show                      # list all saved connections
nmcli connection up "MyWifi"               # activate a saved connection
nmcli device wifi list                     # scan for available Wi-Fi
nmcli device wifi connect "SSID" password "pass"

# --- Hotspot (share internet over Wi-Fi) ---
nmcli device wifi hotspot ssid "MyHotspot" password "secret"

# --- Static IP (example) ---
nmcli connection modify "eth0" ipv4.method manual ipv4.addresses 192.168.1.50/24 ipv4.gateway 192.168.1.1 ipv4.dns 1.1.1.1

# --- Terminal UI (interactive) ---
nmtui

VPN Protocols — All Pre-installed, Configured via GUI

All major VPN protocols are pre-installed as NetworkManager plugins. Connect to any VPN by opening Settings → Network → VPN → + and choosing your protocol — no manual package installation needed.

# --- OpenVPN (most common, .ovpn file) ---
# Import from GUI: Settings → Network → VPN → + → Import from file → select .ovpn
nmcli connection import type openvpn file /path/to/client.ovpn
nmcli connection up "MyOpenVPN"

# --- WireGuard (fast, modern) ---
# Import from GUI or from a .conf file:
nmcli connection import type wireguard file /etc/wireguard/wg0.conf
# Or use the NM GUI to paste your private key, peer public key, and endpoint.

# --- strongSwan / IKEv2 (corporate, certificate-based) ---
# Configure via nm-connection-editor: IPsec/IKEv2, enter server, your certificate.
nmcli connection up "CorpVPN"

# --- Cisco AnyConnect / OpenConnect ---
# GUI: Settings → Network → VPN → + → Cisco AnyConnect Compatible
# CLI:
openconnect --protocol=anyconnect vpn.example.com

# --- Fortinet SSL VPN ---
openfortivpn vpn.example.com:443 --username=you

# --- L2TP/IPsec, PPTP, SSTP, VPNC ---
# All configured through Settings → Network → VPN → +, select protocol.

# --- List and toggle VPN connections ---
nmcli connection show --active
nmcli connection up   "MyVPN"
nmcli connection down "MyVPN"

Remote Desktop — FreeRDP, kRDP, kRFB, gnome-remote-desktop

ShaniOS includes both client and server tools for remote desktop access.

# --- FreeRDP — connect TO a Windows/RDP server (pre-installed) ---
xfreerdp /v:192.168.1.100 /u:username /p:password /dynamic-resolution /gfx /rfx

# Full-screen RDP session:
xfreerdp /v:server.example.com /u:me /f /multimon

# RDP over SSH tunnel (recommended for security):
ssh -L 3389:192.168.1.100:3389 jumphost
xfreerdp /v:localhost /u:username

# --- kRDP (KDE RDP server — pre-installed on KDE edition) ---
# Enable: Settings → System → Remote Desktop → Enable Remote Desktop
# Uses GNOME RDP protocol; accessible from Windows "Remote Desktop Connection"
systemctl --user enable --now plasma-remotedesktop

# --- kRFB / krfb (KDE VNC server — pre-installed on KDE edition) ---
systemctl --user enable --now krfb
# Connect from any VNC viewer: vncviewer hostname:5900

# --- gnome-remote-desktop (GNOME edition — pre-installed) ---
# Enable: Settings → Sharing → Remote Desktop
# Supports both RDP and VNC protocols
grdctl rdp enable
grdctl rdp set-credentials username password
grdctl status

# --- SSH with X forwarding (run GUI apps remotely) ---
ssh -X user@host
ssh -Y user@host  # trusted (faster, less secure)

ModemManager — Mobile Broadband (3G/4G/5G)

ModemManager is pre-installed and integrates with NetworkManager for USB/PCIe mobile broadband modems and SIM cards. Manages LTE/5G, SMS, and signal monitoring.

# --- ModemManager status ---
mmcli -L                          # list detected modems
mmcli -m 0                        # detailed modem info (signal, tech, SIM)
mmcli -m 0 --signal-get           # signal strength

# --- Connect via NetworkManager ---
# NM auto-detects the modem; configure APN in Settings → Network → Mobile Broadband
nmcli connection show             # the modem connection will appear here

# --- SMS (if supported by modem) ---
mmcli -m 0 --messaging-list-sms
mmcli -s 0                        # read SMS message 0
mmcli -m 0 --messaging-create-sms="text=Hello&number=+1234567890"

Server Software via Podman Containers

Only Caddy is pre-installed as a server. All other server software runs as rootless Podman containers — this is the recommended approach on ShaniOS. Container data lives in the persistent @containers Btrfs subvolume and survives all system updates. Use --restart unless-stopped for services you want to auto-start after reboot, and optionally generate a systemd unit with podman generate systemd for tighter integration.

Nginx — high-performance web / reverse proxy:

podman run -d \
  --name nginx \
  -p 127.0.0.1:8081:80 \
  -v /home/user/www:/usr/share/nginx/html:ro,Z \
  -v /home/user/nginx.conf:/etc/nginx/nginx.conf:ro,Z \
  --restart unless-stopped \
  nginx:alpine

# Simple nginx.conf for serving static files
# /home/user/nginx.conf:
# server {
#   listen 80;
#   root /usr/share/nginx/html;
#   index index.html;
#   try_files $uri $uri/ =404;
# }

# Caddy proxies it publicly:
# mysite.example.com { reverse_proxy localhost:8081 }

Apache HTTPD — classic web server with .htaccess support:

podman run -d \
  --name apache \
  -p 127.0.0.1:8082:80 \
  -v /home/user/www:/usr/local/apache2/htdocs:ro,Z \
  --restart unless-stopped \
  httpd:alpine

# With custom config
podman run -d \
  --name apache \
  -p 127.0.0.1:8082:80 \
  -v /home/user/www:/usr/local/apache2/htdocs:Z \
  -v /home/user/httpd.conf:/usr/local/apache2/conf/httpd.conf:ro,Z \
  --restart unless-stopped \
  httpd:latest

Nginx + PHP-FPM — PHP apps (WordPress, Laravel, custom):

# ~/php-stack/compose.yml
# services:
#   nginx:
#     image: nginx:alpine
#     ports: ["127.0.0.1:8080:80"]
#     volumes:
#       - ./www:/var/www/html:ro,Z
#       - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro,Z
#     depends_on: [php]
#
#   php:
#     image: php:8.3-fpm-alpine
#     volumes:
#       - ./www:/var/www/html:Z
#     depends_on: [db]
#
#   db:
#     image: mariadb:11
#     environment:
#       MYSQL_ROOT_PASSWORD: rootpass
#       MYSQL_DATABASE: myapp
#       MYSQL_USER: appuser
#       MYSQL_PASSWORD: apppass
#     volumes: [db_data:/var/lib/mysql]
#
# volumes: {db_data: {}}

mkdir -p ~/php-stack/www
podman-compose -f ~/php-stack/compose.yml up -d

MariaDB / MySQL — relational database:

podman run -d \
  --name mariadb \
  -p 127.0.0.1:3306:3306 \
  -e MYSQL_ROOT_PASSWORD=strongpassword \
  -e MYSQL_DATABASE=mydb \
  -e MYSQL_USER=myuser \
  -e MYSQL_PASSWORD=myuserpass \
  -v mariadb_data:/var/lib/mysql \
  --restart unless-stopped \
  mariadb:11

# Connect from host
podman exec -it mariadb mariadb -u myuser -p mydb
# or use mysql client if installed via Distrobox

PostgreSQL — advanced relational database:

podman run -d \
  --name postgres \
  -p 127.0.0.1:5432:5432 \
  -e POSTGRES_USER=myuser \
  -e POSTGRES_PASSWORD=strongpassword \
  -e POSTGRES_DB=mydb \
  -v postgres_data:/var/lib/postgresql/data \
  --restart unless-stopped \
  postgres:16-alpine

# Connect
podman exec -it postgres psql -U myuser -d mydb

# pgAdmin web UI (optional)
podman run -d \
  --name pgadmin \
  -p 127.0.0.1:5050:80 \
  -e [email protected] \
  -e PGADMIN_DEFAULT_PASSWORD=admin \
  --restart unless-stopped \
  dpage/pgadmin4

Redis — in-memory cache & message broker:

podman run -d \
  --name redis \
  -p 127.0.0.1:6379:6379 \
  -v redis_data:/data \
  --restart unless-stopped \
  redis:7-alpine redis-server --appendonly yes

# Connect and test
podman exec -it redis redis-cli ping
podman exec -it redis redis-cli set mykey "hello"
podman exec -it redis redis-cli get mykey

# Redis with password
podman run -d \
  --name redis \
  -p 127.0.0.1:6379:6379 \
  -v redis_data:/data \
  --restart unless-stopped \
  redis:7-alpine redis-server --appendonly yes --requirepass strongpassword

MongoDB — document database:

podman run -d \
  --name mongodb \
  -p 127.0.0.1:27017:27017 \
  -e MONGO_INITDB_ROOT_USERNAME=admin \
  -e MONGO_INITDB_ROOT_PASSWORD=strongpassword \
  -v mongodb_data:/data/db \
  --restart unless-stopped \
  mongo:7

# Connect
podman exec -it mongodb mongosh -u admin -p strongpassword --authenticationDatabase admin

SQLite via Litestream — replicated SQLite:

# Litestream continuously replicates a SQLite database to S3/Backblaze/local
podman run -d \
  --name litestream \
  -v /home/user/app/db:/data:Z \
  -v /home/user/litestream.yml:/etc/litestream.yml:ro,Z \
  --restart unless-stopped \
  litestream/litestream replicate

# litestream.yml example:
# dbs:
#   - path: /data/app.db
#     replicas:
#       - url: s3://mybucket/app.db

Jellyfin — self-hosted media server (movies, TV, music):

podman run -d \
  --name jellyfin \
  -p 127.0.0.1:8096:8096 \
  -v /home/user/jellyfin/config:/config:Z \
  -v /home/user/jellyfin/cache:/cache:Z \
  -v /home/user/media:/media:ro,Z \
  --restart unless-stopped \
  jellyfin/jellyfin

# With hardware transcoding (Intel VA-API)
podman run -d \
  --name jellyfin \
  -p 127.0.0.1:8096:8096 \
  --device /dev/dri/renderD128:/dev/dri/renderD128 \
  -v /home/user/jellyfin/config:/config:Z \
  -v /home/user/media:/media:ro,Z \
  --restart unless-stopped \
  jellyfin/jellyfin

# Proxy: jellyfin.example.com { reverse_proxy localhost:8096 }

Navidrome — self-hosted music streaming (Subsonic-compatible):

podman run -d \
  --name navidrome \
  -p 127.0.0.1:4533:4533 \
  -v /home/user/navidrome/data:/data:Z \
  -v /home/user/Music:/music:ro,Z \
  -e ND_SCANSCHEDULE="@every 1h" \
  -e ND_LOGLEVEL=info \
  --restart unless-stopped \
  deluan/navidrome:latest

# Compatible with DSub, Ultrasonic, Symfonium apps on Android/iOS
# Proxy: music.example.com { reverse_proxy localhost:4533 }

Nextcloud — self-hosted Google Drive/Dropbox alternative:

# ~/nextcloud/compose.yml
# services:
#   db:
#     image: mariadb:11
#     environment:
#       MYSQL_ROOT_PASSWORD: rootpass
#       MYSQL_DATABASE: nextcloud
#       MYSQL_USER: nc
#       MYSQL_PASSWORD: ncpass
#     volumes: [db_data:/var/lib/mysql]
#     restart: unless-stopped
#
#   redis:
#     image: redis:7-alpine
#     restart: unless-stopped
#
#   nextcloud:
#     image: nextcloud:29
#     ports: ["127.0.0.1:8888:80"]
#     environment:
#       MYSQL_HOST: db
#       MYSQL_DATABASE: nextcloud
#       MYSQL_USER: nc
#       MYSQL_PASSWORD: ncpass
#       REDIS_HOST: redis
#       NEXTCLOUD_ADMIN_USER: admin
#       NEXTCLOUD_ADMIN_PASSWORD: changeme
#     volumes: [nc_data:/var/www/html]
#     depends_on: [db, redis]
#     restart: unless-stopped
#
# volumes: {db_data: {}, nc_data: {}}

mkdir -p ~/nextcloud
podman-compose -f ~/nextcloud/compose.yml up -d
# Proxy: cloud.example.com { reverse_proxy localhost:8888 }

Syncthing — peer-to-peer file sync (no central server):

podman run -d \
  --name syncthing \
  -p 127.0.0.1:8384:8384 \
  -p 22000:22000/tcp \
  -p 22000:22000/udp \
  -p 21027:21027/udp \
  -v /home/user/syncthing/config:/var/syncthing/config:Z \
  -v /home/user/sync:/var/syncthing/Sync:Z \
  -e PUID=$(id -u) \
  -e PGID=$(id -g) \
  --restart unless-stopped \
  syncthing/syncthing:latest

# Open firewall ports for Syncthing
sudo firewall-cmd --add-port=22000/tcp --permanent
sudo firewall-cmd --add-port=22000/udp --permanent
sudo firewall-cmd --add-port=21027/udp --permanent
sudo firewall-cmd --reload

# Web UI: http://localhost:8384
# Proxy (optional): sync.example.com { reverse_proxy localhost:8384 }

Filebrowser — web-based file manager:

podman run -d \
  --name filebrowser \
  -p 127.0.0.1:8085:80 \
  -v /home/user:/srv:Z \
  -v /home/user/filebrowser.db:/database.db:Z \
  --restart unless-stopped \
  filebrowser/filebrowser:s6

# Access at http://localhost:8085 (admin/admin — change immediately)
# files.example.com { reverse_proxy localhost:8085 }

Gitea — self-hosted Git with web UI, issues, CI:

podman run -d \
  --name gitea \
  -p 127.0.0.1:3000:3000 \
  -p 127.0.0.1:2222:22 \
  -v /home/user/gitea:/data:Z \
  -e USER_UID=$(id -u) \
  -e USER_GID=$(id -g) \
  --restart unless-stopped \
  gitea/gitea:latest

# git.example.com { reverse_proxy localhost:3000 }

# SSH push via non-default port
# In ~/.ssh/config on clients:
# Host git.example.com
#   Port 2222

Forgejo — community fork of Gitea:

podman run -d \
  --name forgejo \
  -p 127.0.0.1:3000:3000 \
  -p 127.0.0.1:2222:22 \
  -v /home/user/forgejo:/data:Z \
  -e USER_UID=$(id -u) \
  -e USER_GID=$(id -g) \
  --restart unless-stopped \
  codeberg.org/forgejo/forgejo:latest

Vaultwarden — self-hosted Bitwarden-compatible password manager:

podman run -d \
  --name vaultwarden \
  -p 127.0.0.1:8180:80 \
  -v /home/user/vaultwarden:/data:Z \
  -e WEBSOCKET_ENABLED=true \
  -e ADMIN_TOKEN=$(openssl rand -base64 48) \
  --restart unless-stopped \
  vaultwarden/server:latest

# HTTPS is REQUIRED — Vaultwarden refuses connections over plain HTTP
# vault.example.com {
#     reverse_proxy localhost:8180
#     reverse_proxy /notifications/hub localhost:3012
# }

Prometheus + Grafana — metrics collection and dashboards:

# Create a minimal Prometheus config first
mkdir -p ~/monitoring
cat > ~/monitoring/prometheus.yml <<'EOF'
global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']
EOF

# Node Exporter — expose system metrics (CPU, RAM, disk, network)
podman run -d \
  --name node-exporter \
  --network host \
  -v /proc:/host/proc:ro,rslave \
  -v /sys:/host/sys:ro,rslave \
  -v /:/rootfs:ro,rslave \
  --restart unless-stopped \
  prom/node-exporter \
  --path.procfs=/host/proc --path.sysfs=/host/sys

# Prometheus
podman run -d \
  --name prometheus \
  -p 127.0.0.1:9090:9090 \
  -v ~/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro,Z \
  -v prometheus_data:/prometheus \
  --restart unless-stopped \
  prom/prometheus

# Grafana
podman run -d \
  --name grafana \
  -p 127.0.0.1:3001:3000 \
  -v grafana_data:/var/lib/grafana \
  -e GF_SECURITY_ADMIN_PASSWORD=changeme \
  --restart unless-stopped \
  grafana/grafana

# grafana.example.com { reverse_proxy localhost:3001 }

Uptime Kuma — self-hosted service uptime monitoring:

podman run -d \
  --name uptime-kuma \
  -p 127.0.0.1:3001:3001 \
  -v /home/user/uptime-kuma:/app/data:Z \
  --restart unless-stopped \
  louislam/uptime-kuma:latest

# status.example.com { reverse_proxy localhost:3001 }

Netdata — real-time system performance monitoring:

podman run -d \
  --name netdata \
  -p 127.0.0.1:19999:19999 \
  --cap-add SYS_PTRACE \
  --security-opt apparmor=unconfined \
  -v netdata_config:/etc/netdata \
  -v netdata_lib:/var/lib/netdata \
  -v netdata_cache:/var/cache/netdata \
  -v /etc/passwd:/host/etc/passwd:ro \
  -v /proc:/host/proc:ro \
  -v /sys:/host/sys:ro \
  --restart unless-stopped \
  netdata/netdata

# metrics.example.com { reverse_proxy localhost:19999 }

Home Assistant — home automation platform:

podman run -d \
  --name homeassistant \
  --network host \
  -v /home/user/homeassistant/config:/config:Z \
  -e TZ=Europe/London \
  --restart unless-stopped \
  ghcr.io/home-assistant/home-assistant:stable

# Access at http://localhost:8123 for initial setup
# hass.example.com { reverse_proxy localhost:8123 }

# Note: --network host is recommended so HA can discover devices on LAN
# Open firewall if using host networking
sudo firewall-cmd --add-port=8123/tcp --permanent
sudo firewall-cmd --reload

Ntfy — push notifications server:

podman run -d \
  --name ntfy \
  -p 127.0.0.1:8090:80 \
  -v /home/user/ntfy/cache:/var/cache/ntfy:Z \
  -v /home/user/ntfy/etc:/etc/ntfy:Z \
  --restart unless-stopped \
  binwiederhier/ntfy serve

# ntfy.example.com { reverse_proxy localhost:8090 }

# Send a notification from the command line
curl -d "Your backup completed successfully" ntfy.example.com/my-alerts

# Subscribe in the ntfy Android/iOS app or browser

Gotify — self-hosted push notification server:

podman run -d \
  --name gotify \
  -p 127.0.0.1:8070:80 \
  -v /home/user/gotify/data:/app/data:Z \
  --restart unless-stopped \
  gotify/server

# push.example.com { reverse_proxy localhost:8070 }

Woodpecker CI — lightweight CI/CD for Gitea/Forgejo:

# ~/woodpecker/compose.yml
# services:
#   woodpecker-server:
#     image: woodpeckerci/woodpecker-server:latest
#     ports: ["127.0.0.1:8000:8000"]
#     environment:
#       WOODPECKER_OPEN: "true"
#       WOODPECKER_HOST: https://ci.example.com
#       WOODPECKER_GITEA: "true"
#       WOODPECKER_GITEA_URL: https://git.example.com
#       WOODPECKER_GITEA_CLIENT: <oauth2-client-id>
#       WOODPECKER_GITEA_SECRET: <oauth2-client-secret>
#       WOODPECKER_AGENT_SECRET: <random-secret>
#     volumes: [woodpecker_data:/var/lib/woodpecker]
#     restart: unless-stopped
#
#   woodpecker-agent:
#     image: woodpeckerci/woodpecker-agent:latest
#     environment:
#       WOODPECKER_SERVER: woodpecker-server:9000
#       WOODPECKER_AGENT_SECRET: <same-random-secret>
#     volumes: [/run/user/1000/podman/podman.sock:/var/run/docker.sock]
#     depends_on: [woodpecker-server]
#     restart: unless-stopped

podman-compose -f ~/woodpecker/compose.yml up -d

Pi-hole — network-wide DNS ad blocker:

podman run -d \
  --name pihole \
  -p 127.0.0.1:8083:80 \
  -p 53:53/tcp \
  -p 53:53/udp \
  -e TZ=Europe/London \
  -e WEBPASSWORD=changeme \
  -v /home/user/pihole/etc-pihole:/etc/pihole:Z \
  -v /home/user/pihole/etc-dnsmasq.d:/etc/dnsmasq.d:Z \
  --restart unless-stopped \
  pihole/pihole:latest

# Open DNS port in firewall (if using as LAN DNS)
sudo firewall-cmd --add-service=dns --permanent
sudo firewall-cmd --reload

# Point your router's DNS to this machine's IP
# Web UI: http://localhost:8083/admin

AdGuard Home — DNS-based ad/tracker blocker with DoH/DoT:

podman run -d \
  --name adguardhome \
  -p 53:53/tcp -p 53:53/udp \
  -p 127.0.0.1:3000:3000 \
  -p 853:853/tcp \
  -v /home/user/adguard/work:/opt/adguardhome/work:Z \
  -v /home/user/adguard/conf:/opt/adguardhome/conf:Z \
  --restart unless-stopped \
  adguard/adguardhome

# Initial setup wizard at http://localhost:3000
# Supports DNS-over-HTTPS and DNS-over-TLS out of the box

Nginx Proxy Manager — GUI-based reverse proxy with Let's Encrypt:

# ~/npm/compose.yml
# services:
#   app:
#     image: jc21/nginx-proxy-manager:latest
#     ports:
#       - "80:80"
#       - "443:443"
#       - "127.0.0.1:81:81"
#     volumes:
#       - /home/user/npm/data:/data:Z
#       - /home/user/npm/letsencrypt:/etc/letsencrypt:Z
#     restart: unless-stopped

podman-compose -f ~/npm/compose.yml up -d
# Web UI at http://localhost:81 ([email protected] / changeme)

Stirling PDF — self-hosted PDF manipulation tool:

podman run -d \
  --name stirling-pdf \
  -p 127.0.0.1:8080:8080 \
  -v /home/user/stirling/trainingData:/usr/share/tessdata:Z \
  -v /home/user/stirling/extraConfigs:/configs:Z \
  --restart unless-stopped \
  frooodle/s-pdf:latest

# pdf.example.com { reverse_proxy localhost:8080 }

Paperless-ngx — document management / OCR:

# ~/paperless/compose.yml
# services:
#   broker:
#     image: redis:7-alpine
#     restart: unless-stopped
#
#   db:
#     image: postgres:15-alpine
#     environment:
#       POSTGRES_DB: paperless
#       POSTGRES_USER: paperless
#       POSTGRES_PASSWORD: paperless
#     volumes: [pgdata:/var/lib/postgresql/data]
#     restart: unless-stopped
#
#   webserver:
#     image: ghcr.io/paperless-ngx/paperless-ngx:latest
#     ports: ["127.0.0.1:8000:8000"]
#     environment:
#       PAPERLESS_REDIS: redis://broker:6379
#       PAPERLESS_DBHOST: db
#       PAPERLESS_OCR_LANGUAGE: eng
#       PAPERLESS_TIME_ZONE: Europe/London
#     volumes:
#       - /home/user/paperless/data:/usr/src/paperless/data:Z
#       - /home/user/paperless/media:/usr/src/paperless/media:Z
#       - /home/user/Documents/inbox:/usr/src/paperless/consume:Z
#     depends_on: [broker, db]
#     restart: unless-stopped

podman-compose -f ~/paperless/compose.yml up -d
# docs.example.com { reverse_proxy localhost:8000 }

Miniflux — minimal RSS / Atom feed reader:

podman run -d \
  --name miniflux \
  -p 127.0.0.1:8080:8080 \
  -e DATABASE_URL="postgres://miniflux:password@localhost/miniflux?sslmode=disable" \
  -e RUN_MIGRATIONS=1 \
  -e CREATE_ADMIN=1 \
  -e ADMIN_USERNAME=admin \
  -e ADMIN_PASSWORD=changeme \
  --restart unless-stopped \
  miniflux/miniflux:latest

# rss.example.com { reverse_proxy localhost:8080 }

LinkWarden — collaborative bookmark manager:

podman run -d \
  --name linkwarden \
  -p 127.0.0.1:3000:3000 \
  -e DATABASE_URL="mongodb://localhost:27017/linkwarden" \
  -e NEXTAUTH_SECRET=$(openssl rand -base64 32) \
  -e NEXTAUTH_URL=https://links.example.com \
  -v /home/user/linkwarden/data:/data/data:Z \
  --restart unless-stopped \
  ghcr.io/linkwarden/linkwarden:latest

Make containers start automatically at login (systemd user service):

# Generate a systemd unit for a running container
podman generate systemd --name jellyfin --files --new

# Move the generated unit to your user systemd dir
mv container-jellyfin.service ~/.config/systemd/user/

# Enable it (starts at login, without needing root)
systemctl --user daemon-reload
systemctl --user enable --now container-jellyfin.service

# Enable lingering so services start at boot even without a login session
loginctl enable-linger $USER

# Check status
systemctl --user status container-jellyfin.service

rclone — Cloud Sync (40+ Providers)

Sync, copy, and mount cloud storage (S3, Backblaze B2, Google Drive, Nextcloud, SFTP, WebDAV, and 40+ more). Config stored in /data/varlib/rclone.

# Interactive setup wizard
rclone config

# Sync local → cloud (mirror; deletes files removed locally)
rclone sync /home/user/data remote:my-bucket --progress

# Copy local → cloud (never deletes remote)
rclone copy /var/www/html remote:backups --progress

# Mount cloud storage as local folder
mkdir -p ~/mnt/cloud
rclone mount remote:my-bucket ~/mnt/cloud --daemon --vfs-cache-mode writes

# Unmount
fusermount -u ~/mnt/cloud

# List remote files
rclone ls remote:my-bucket
rclone lsd remote:              # list buckets/containers

# Check sync differences without transferring
rclone check /local/dir remote:bucket

# Delete files older than 30 days on remote
rclone delete remote:old-logs --min-age 30d

# Filter by pattern
rclone sync /home/user/docs remote:docs --include "*.pdf" --exclude "*.tmp"

restic — Encrypted Deduplicated Backups

Content-addressed, encrypted, deduplicated backups to any storage rclone can reach. Config stored in /data/varlib/restic.

# Initialize repository (local or rclone backend)
restic init --repo /mnt/backup/myrepo
restic -r rclone:remote:restic-repo init

# Create a snapshot
restic -r /mnt/backup/myrepo backup /home/user /var/www/html /data/varlib

# Use RESTIC_REPOSITORY and RESTIC_PASSWORD env vars to avoid repetition
export RESTIC_REPOSITORY=/mnt/backup/myrepo
export RESTIC_PASSWORD=mysecretpassword
restic backup /home/user

# List snapshots
restic snapshots

# Restore latest snapshot
restic restore latest --target /tmp/restored

# Restore a specific snapshot (use ID from `restic snapshots`)
restic restore 1a2b3c4d --target /tmp/restored

# Mount snapshots as a FUSE filesystem (browse by time)
restic mount /mnt/restic-browse

# Check repository integrity
restic check

# Prune old snapshots (keep 7 daily, 4 weekly, 12 monthly)
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --prune

Automated daily backup via systemd timer:

# Create ~/.config/systemd/user/restic-backup.service:
# [Unit]
# Description=restic backup
# [Service]
# Type=oneshot
# Environment="RESTIC_REPOSITORY=/mnt/backup/myrepo"
# Environment="RESTIC_PASSWORD=mysecretpassword"
# ExecStart=restic backup /home/%u
# ExecStartPost=restic forget --keep-daily 7 --keep-weekly 4 --prune

# Create ~/.config/systemd/user/restic-backup.timer:
# [Unit]
# Description=Daily restic backup
# [Timer]
# OnCalendar=daily
# Persistent=true
# [Install]
# WantedBy=timers.target

systemctl --user daemon-reload
systemctl --user enable --now restic-backup.timer

Network Diagnostics & Troubleshooting Commands

All tools listed here are pre-installed. Packages: iproute2 (ip/ss), iputils (ping/arping), inetutils (traceroute/ftp/telnet), net-tools (ifconfig/netstat/arp/route), bind (dig/host/nslookup), iw, wireless_tools (iwconfig/iwlist/iwspy), wpa_supplicant (wpa_cli/wpa_passphrase), nmap, tcpdump, mtr, iperf3, iftop, nethogs, bandwhich (bandwidth by process and remote address), ethtool, socat, ngrep, whois, net-snmp (snmpwalk/snmpget), openbsd-netcat, curl, wget, aria2 (multi-protocol: HTTP/FTP/BitTorrent/Metalink), zsync, lsof, openldap (ldapsearch), openssl.

ping / arping — reachability & MAC detection:

# ICMP ping (from iputils)
ping -c 4 shani.dev
ping -c 4 -i 0.2 192.168.1.1        # fast ping, 200ms interval
ping -s 1400 -c 4 192.168.1.1       # large packet (MTU test)
ping6 -c 4 ipv6.google.com          # IPv6 reachability

# ARP ping — verify a host is alive at Layer 2 (LAN only)
sudo arping -I eth0 -c 3 192.168.1.1
sudo arping -I eth0 192.168.1.50    # find MAC address of an IP

traceroute / mtr — path tracing:

# Classic traceroute (from inetutils)
traceroute shani.dev
traceroute -n shani.dev             # no DNS reverse lookup (faster)
traceroute -T -p 443 shani.dev      # TCP traceroute (bypasses ICMP blocks)
traceroute -U -p 53 8.8.8.8         # UDP traceroute

# MTR — live traceroute + per-hop packet loss/latency (mtr package)
mtr shani.dev                       # interactive TUI
mtr --report --report-cycles 20 shani.dev  # non-interactive report
mtr -n --report shani.dev           # no DNS (pure IPs)
mtr --tcp --port 443 shani.dev      # TCP mode through firewalls

DNS resolution — dig / host / nslookup / resolvectl:

# dig (from bind package) — full DNS query
dig shani.dev                       # A record (default)
dig shani.dev AAAA                  # IPv6 record
dig shani.dev MX                    # mail exchange
dig shani.dev NS                    # nameservers
dig shani.dev TXT                   # TXT records (SPF, DKIM, etc.)
dig +short shani.dev A              # clean output, IP only
dig +trace shani.dev                # full recursive resolution trace
dig @1.1.1.1 shani.dev              # query specific resolver (Cloudflare)
dig @8.8.8.8 shani.dev              # query Google DNS directly
dig -x 1.2.3.4                      # reverse DNS (PTR lookup)

# host — simpler alternative
host shani.dev
host shani.dev 1.1.1.1              # via specific server

# nslookup — interactive or one-shot
nslookup shani.dev
nslookup shani.dev 8.8.8.8

# resolvectl — query systemd-resolved (what the system actually uses)
resolvectl status                   # per-interface DNS config
resolvectl query shani.dev          # resolved via system stub
resolvectl flush-caches             # flush DNS cache
cat /etc/resolv.conf                # current stub resolver config

ip — interfaces, addresses, routes, neighbours:

# Interfaces
ip link show                        # all interfaces (state, MAC)
ip -brief link show                 # compact: name + state + MAC
ip link show eth0                   # specific interface
ip link set eth0 up                 # bring interface up
ip link set eth0 down               # bring interface down

# Addresses
ip addr show                        # all IPs on all interfaces
ip -brief addr show                 # compact summary
ip addr show eth0                   # specific interface
ip addr add 192.168.1.50/24 dev eth0   # add temporary IP
ip addr del 192.168.1.50/24 dev eth0   # remove IP

# Routes
ip route show                       # routing table
ip route get 8.8.8.8                # which interface/gateway for a destination
ip route add 10.0.0.0/8 via 192.168.1.1  # add a route
ip route del 10.0.0.0/8             # remove a route

# Neighbours (ARP/NDP cache)
ip neigh show                       # ARP table (LAN MAC→IP mapping)
ip neigh flush all                  # clear ARP cache

# Statistics
ip -s link show eth0                # TX/RX byte and packet counters
ip -s -s link show eth0             # extended error counters

ss — socket statistics (replaces netstat):

ss -tlnp               # TCP listening sockets with process name/PID
ss -ulnp               # UDP listening sockets
ss -tlnp | grep :80    # who is listening on port 80
ss -tnp                # all established TCP connections
ss -s                  # summary: total sockets by state
ss -4 -tnp             # IPv4 only
ss -6 -tnp             # IPv6 only
ss -xnp                # Unix domain sockets
ss -tlnp src :443      # filter by local port
ss -tnp dst 192.168.1.100   # connections to a specific host

netstat / ifconfig / arp / route — legacy net-tools:

# netstat (net-tools — older but widely familiar)
netstat -tlnp          # listening TCP sockets (same as ss -tlnp)
netstat -s             # protocol statistics (TCP/UDP/IP error counters)
netstat -r             # routing table (same as ip route show)
netstat -i             # interface statistics

# ifconfig (legacy — use ip addr for new scripts)
ifconfig               # all interfaces
ifconfig eth0          # specific interface

# arp (legacy — use ip neigh for new scripts)
arp -n                 # ARP table without DNS resolution
arp -d 192.168.1.50    # remove an ARP entry

# route (legacy — use ip route for new scripts)
route -n               # routing table (numeric)

ethtool — NIC hardware & driver info:

# Link speed and duplex
ethtool eth0                        # speed, duplex, link detected
ethtool -i eth0                     # driver name and version
ethtool -S eth0                     # NIC statistics (errors, drops, missed)
ethtool -k eth0                     # offload features (TSO, GRO, etc.)
ethtool -a eth0                     # pause parameters (flow control)

# Wake-on-LAN
ethtool -s eth0 wol g               # enable WoL on magic packet
ethtool eth0 | grep "Wake-on"       # check WoL status

# Test link (loop-back self-test, if NIC supports it)
sudo ethtool -t eth0

nmap — port scanning & service discovery:

# Basic scans
nmap localhost                      # top 1000 TCP ports on localhost
nmap 192.168.1.100                  # remote host
nmap -p 22,80,443,8080 localhost    # specific ports

# Discovery
nmap -sn 192.168.1.0/24             # ping sweep: find live hosts (no port scan)
nmap -sn 192.168.1.0/24 --open      # only show up hosts

# Service & version detection
nmap -sV localhost                  # service version fingerprinting
nmap -sV -p 22 192.168.1.100        # SSH version on remote
nmap -A localhost                   # OS detect + version + scripts + traceroute

# Script-based probes
nmap --script=http-title 192.168.1.0/24    # grab HTTP page titles on LAN
nmap --script=ssh-hostkey 192.168.1.100    # get SSH host key fingerprint
nmap --script=ssl-cert 192.168.1.100 -p 443  # read TLS cert details

# Fast / quiet scans
nmap -F 192.168.1.100               # fast: top 100 ports only
nmap -T4 -F 192.168.1.0/24         # fast sweep of top 100 ports on subnet

# UDP scanning (requires root)
sudo nmap -sU -p 53,123,161 192.168.1.1  # DNS, NTP, SNMP

tcpdump — live packet capture:

# Capture on any interface
sudo tcpdump -i any -n              # all traffic, no DNS resolution
sudo tcpdump -i eth0 -n             # specific interface

# Filter by port
sudo tcpdump -i any port 80 -n -A  # HTTP traffic, ASCII body
sudo tcpdump -i any port 443 -n    # HTTPS (encrypted, shows handshake)
sudo tcpdump -i any port 53 -n     # DNS queries

# Filter by host
sudo tcpdump -i any host 192.168.1.50 -n
sudo tcpdump -i any src 192.168.1.50 -n   # only FROM that host
sudo tcpdump -i any dst 192.168.1.50 -n   # only TO that host

# Combine filters
sudo tcpdump -i any host 192.168.1.50 and port 80 -n

# Save to file for Wireshark analysis
sudo tcpdump -i any -w /tmp/capture.pcap
sudo tcpdump -i any -w /tmp/capture.pcap -C 10  # rotate at 10 MB

# Read saved capture
tcpdump -r /tmp/capture.pcap -n

# Show raw bytes
sudo tcpdump -i any -X port 80 -n | head -60

ngrep — grep over network traffic:

# Match HTTP method lines
sudo ngrep -d any -q "GET\|POST\|PUT\|DELETE" port 80

# Find passwords or tokens in plaintext traffic (internal audit use)
sudo ngrep -d any -qi "password\|token\|authorization"

# Match DNS queries
sudo ngrep -d any "" udp port 53

# Match on a specific host
sudo ngrep -d any -q "User-Agent" host 192.168.1.50 and port 80

# Quiet mode (only matched packets, no headers)
sudo ngrep -q "error\|fail" port 8080

socat — universal relay & TCP/UDP Swiss Army knife:

# Test TCP connectivity to a port
socat - TCP:192.168.1.100:3000
socat - TCP:192.168.1.100:8080,crnl   # with line-ending conversion

# Test UDP connectivity
socat - UDP:192.168.1.100:5000

# Simple TCP listener (simulate a server)
socat TCP-LISTEN:9999,fork -          # echo server on port 9999

# Port forward: local 8080 → remote 192.168.1.100:80
socat TCP-LISTEN:8080,fork TCP:192.168.1.100:80

# Test TLS/SSL connection (alternative to openssl s_client)
socat - OPENSSL:mysite.example.com:443,verify=0

# Bidirectional pipe between two ports
socat TCP-LISTEN:4444,fork TCP:192.168.1.100:22  # proxy SSH

nc (netcat) — port testing & simple transfer:

# Test if a port is open
nc -zv 192.168.1.100 22             # TCP: verbose, exit immediately
nc -zvw 3 192.168.1.100 443         # 3-second timeout
nc -zvu 192.168.1.100 53            # UDP port test

# Scan a port range
nc -zv 192.168.1.100 20-25

# Simple file transfer (no encryption — LAN only)
# Receiver:
nc -l -p 9999 > received_file.bin
# Sender:
nc 192.168.1.100 9999 < file_to_send.bin

# Banner grabbing (see what a service says on connect)
nc 192.168.1.100 25                 # SMTP banner
nc 192.168.1.100 21                 # FTP banner
nc 192.168.1.100 22                 # SSH version string

# Simple HTTP request
printf "GET / HTTP/1.0\r\nHost: example.com\r\n\r\n" | nc example.com 80

curl & wget — HTTP/HTTPS testing:

# curl
curl -v https://mysite.example.com               # verbose: headers + body
curl -I https://mysite.example.com               # HEAD: response headers only
curl -s -o /dev/null -w "%{http_code}\n" https://mysite.example.com  # status code only
curl -s -o /dev/null -w "HTTP %{http_code} | %{time_total}s | %{size_download} bytes\n" https://mysite.example.com
curl -k https://localhost:8443                   # ignore self-signed cert
curl -H "Host: mysite.example.com" http://127.0.0.1  # test vhost routing
curl -L https://example.com/redirect             # follow redirects
curl -u user:pass https://api.example.com/data   # HTTP basic auth
curl -X POST -H "Content-Type: application/json" \
     -d '{"key":"value"}' https://api.example.com/endpoint  # POST JSON
curl --resolve mysite.example.com:443:127.0.0.1 https://mysite.example.com  # test before DNS propagates
curl -w "@/dev/stdin" -o /dev/null -s https://mysite.example.com <<'EOF'
     time_namelookup:  %{time_namelookup}s\n
     time_connect:     %{time_connect}s\n
     time_starttransfer: %{time_starttransfer}s\n
     time_total:       %{time_total}s\n
EOF

# wget
wget -q --server-response -O /dev/null https://mysite.example.com 2>&1 | head -20
wget --spider https://mysite.example.com         # check if URL exists (no download)
wget -r -l 1 --spider https://mysite.example.com 2>&1 | grep broken  # check links

openssl — TLS/SSL certificate inspection:

# Inspect a server's TLS certificate
openssl s_client -connect mysite.example.com:443 </dev/null 2>&1 | openssl x509 -noout -text

# Check expiry date only
openssl s_client -connect mysite.example.com:443 </dev/null 2>/dev/null \
  | openssl x509 -noout -dates

# Test specific TLS version
openssl s_client -connect mysite.example.com:443 -tls1_2 </dev/null
openssl s_client -connect mysite.example.com:443 -tls1_3 </dev/null

# Inspect a local certificate file
openssl x509 -in /etc/caddy/certs/mysite.crt -noout -text
openssl x509 -in /etc/caddy/certs/mysite.crt -noout -dates -subject -issuer

# Test SMTP with STARTTLS
openssl s_client -connect mail.example.com:587 -starttls smtp

# Check if a certificate matches its private key
openssl x509 -noout -modulus -in cert.pem | md5sum
openssl rsa  -noout -modulus -in key.pem  | md5sum
# Both sums must match

whois — domain & IP registration info:

whois shani.dev                     # domain registration info
whois 8.8.8.8                       # IP WHOIS (ASN, ISP, abuse contact)
whois -h whois.arin.net 8.8.8.8    # query ARIN directly
whois AS15169                       # ASN info (Google's AS)

iperf3 — bandwidth & throughput testing:

# Run server on the remote/target machine
iperf3 -s                           # default port 5201
iperf3 -s -p 9999                   # custom port

# TCP throughput test (from client)
iperf3 -c 192.168.1.100             # default 10-second TCP test
iperf3 -c 192.168.1.100 -t 30       # 30-second test
iperf3 -c 192.168.1.100 -P 4        # 4 parallel streams (saturate link)
iperf3 -c 192.168.1.100 -R          # reverse: server sends to client

# UDP test (for latency-sensitive or wireless links)
iperf3 -c 192.168.1.100 -u -b 100M  # UDP at 100 Mbps
iperf3 -c 192.168.1.100 -u -b 1G    # UDP at 1 Gbps (gigabit test)

# JSON output for scripting
iperf3 -c 192.168.1.100 -J | jq '.end.sum_received.bits_per_second'

nethogs & iftop — per-process & per-connection bandwidth:

# nethogs — which process is using bandwidth (like top for network)
sudo nethogs                        # all interfaces
sudo nethogs eth0                   # specific interface
sudo nethogs -d 1                   # refresh every 1 second
# Inside nethogs: 'm' cycles units (KB/s, MB/s, GB/s); 'q' quits

# iftop — per-connection bandwidth usage (like top for flows)
sudo iftop                          # all interfaces, interactive
sudo iftop -i eth0                  # specific interface
sudo iftop -n                       # no DNS resolution (faster)
sudo iftop -P                       # show port numbers
sudo iftop -B                       # show in bytes instead of bits
# Inside iftop: 'n' toggles DNS; 'p' toggles ports; 'q' quits

ethtool — NIC hardware diagnostics:

ethtool eth0                        # link speed, duplex, auto-negotiate, link detected
ethtool -i eth0                     # kernel driver name and firmware version
ethtool -S eth0                     # detailed NIC statistics (rx_errors, tx_drops, etc.)
ethtool -k eth0                     # hardware offload features (TSO, GSO, GRO, LRO)
ethtool -a eth0                     # flow control (pause frames)
ethtool -c eth0                     # interrupt coalescing settings
sudo ethtool -s eth0 speed 1000 duplex full autoneg on  # force speed/duplex
sudo ethtool -t eth0                # run built-in NIC self-test (if supported)

# Find which physical port a NIC is on (LED blink)
sudo ethtool -p eth0 5              # blink NIC LED for 5 seconds

net-snmp — SNMP queries to network devices:

# Query a router/switch/AP via SNMP v2c
snmpwalk -v2c -c public 192.168.1.1                           # walk entire MIB
snmpget -v2c -c public 192.168.1.1 sysDescr.0                # system description
snmpget -v2c -c public 192.168.1.1 ifInOctets.1              # bytes in on interface 1
snmpwalk -v2c -c public 192.168.1.1 ifTable                  # all interfaces

# SNMP v3 (authenticated + encrypted)
snmpwalk -v3 -u admin -l authPriv -a SHA -A "authpass" \
  -x AES -X "privpass" 192.168.1.1 sysUpTime

# Translate OID numbers to names
snmptranslate .1.3.6.1.2.1.1.1.0
snmptranslate -On sysDescr.0        # OID number for a named OID

# SNMP trap listener (receive traps from devices)
sudo snmptrapd -f -Lo -c /etc/snmp/snmptrapd.conf

inetutils — ftp, telnet, rsh for legacy protocol testing:

# FTP (for testing FTP servers — use sftp/rsync for actual transfers)
ftp 192.168.1.100
ftp -n 192.168.1.100 <<EOF
quote USER anonymous
quote PASS [email protected]
ls
quit
EOF

# Telnet (for testing TCP services by hand — NOT for production access)
telnet 192.168.1.100 25    # test SMTP handshake
telnet 192.168.1.100 110   # test POP3
telnet 192.168.1.100 80    # type HTTP manually:
                            #   GET / HTTP/1.0
                            #   Host: 192.168.1.100
                            #   (press Enter twice)

# inetutils also provides: rlogin, rsh, rcp (for legacy systems)

lsof — open files, sockets, and processes:

# Network-related lsof usage
lsof -i                             # all network connections
lsof -i TCP                         # TCP only
lsof -i UDP                         # UDP only
lsof -i :80                         # what's using port 80
lsof -i :80-443                     # port range
lsof -i @192.168.1.100              # connections to/from a host
lsof -i TCP:443 -sTCP:LISTEN        # who is LISTENING on 443
lsof -i -n -P                       # all connections, no DNS, numeric ports
lsof -p $(pgrep caddy) -i           # all network files opened by caddy
lsof -u www-data -i                 # all connections by a specific user

iw & wireless_tools — WiFi scanning & diagnostics:

# iw — modern nl80211 wireless tool
iw dev                              # list wireless interfaces
iw dev wlan0 info                   # interface details (freq, channel, SSID)
iw dev wlan0 link                   # current association (signal, bitrate)
iw dev wlan0 station dump           # connected station info (AP or peers)
sudo iw dev wlan0 scan              # scan for nearby networks (requires root)
sudo iw dev wlan0 scan | grep -E "SSID|signal|freq"  # summarised scan
iw phy                              # physical device capabilities
iw phy phy0 info                    # supported bands, channels, features
iw reg get                          # current regulatory domain (country)
sudo iw reg set GB                  # set regulatory domain

# wireless_tools — legacy wext interface (still useful for older drivers)
iwconfig                            # show all wireless interfaces
iwconfig wlan0                      # specific interface: SSID, rate, signal
iwlist wlan0 scan                   # scan for networks (older method)
iwlist wlan0 scan | grep -E "ESSID|Quality|Frequency"
iwlist wlan0 rate                   # supported bit rates
iwlist wlan0 channel                # available channels
iwspy wlan0                         # per-MAC signal tracking

wpa_cli & wpa_passphrase — WPA supplicant management:

# wpa_cli — control wpa_supplicant (NetworkManager usually manages this)
sudo wpa_cli status                 # connection state
sudo wpa_cli scan                   # trigger scan
sudo wpa_cli scan_results           # view scan results
sudo wpa_cli list_networks          # configured networks
sudo wpa_cli disconnect             # disconnect
sudo wpa_cli reconnect              # reconnect

# wpa_passphrase — generate wpa_supplicant config blocks
wpa_passphrase "MySSID" "MyPassword"
# Output can be pasted into /etc/wpa_supplicant/wpa_supplicant.conf

zsync — delta download for large files:

# zsync downloads only the changed parts of a file using a .zsync metafile
# Used internally by shani-deploy for efficient OS image updates

# Download a file using zsync (only transfers changed blocks)
zsync https://example.com/largefile.iso.zsync

# Resume an interrupted zsync download
zsync -i partial_file.iso https://example.com/largefile.iso.zsync

# Specify output filename
zsync -o /tmp/output.iso https://example.com/file.iso.zsync

# Check how much data would be transferred (seed with existing file)
zsync -i existing.iso https://example.com/updated.iso.zsync

ldapsearch & openldap — LDAP directory queries:

# Query an LDAP directory (openldap package)
ldapsearch -x -H ldap://192.168.1.100 -b "dc=example,dc=com"

# Authenticated bind
ldapsearch -x -H ldap://192.168.1.100 \
  -D "cn=admin,dc=example,dc=com" -W \
  -b "dc=example,dc=com" "(objectClass=person)"

# Search for a specific user
ldapsearch -x -H ldap://192.168.1.100 \
  -b "dc=example,dc=com" "(uid=jsmith)"

# LDAPS (TLS)
ldapsearch -x -H ldaps://192.168.1.100 -b "dc=example,dc=com"

# Check if LDAP port is open first
nc -zv 192.168.1.100 389           # LDAP
nc -zv 192.168.1.100 636           # LDAPS

Service log inspection:

# Follow a service log live
sudo journalctl -u caddy -f
sudo journalctl -u sshd -f
sudo journalctl -u smb -f
sudo journalctl -u nfs-server -f
sudo journalctl -u tailscaled -f
sudo journalctl -u cloudflared -f
sudo journalctl -u fail2ban -f
sudo journalctl -u NetworkManager -f
sudo journalctl -u dnsmasq -f
sudo journalctl -u avahi-daemon -f

# Filter by priority (err, warning, info, debug)
sudo journalctl -u caddy -p err -n 50

# Kernel ring buffer — driver errors, firewall drops
sudo journalctl -k -f
sudo journalctl -k | grep -E "FINAL_REJECT|DROP|INVALID"
sudo journalctl -k | grep -E "eth0|wlan0|link"    # interface events

# AppArmor denials
sudo journalctl -k | grep apparmor

# All failed systemd units since boot
systemctl --failed

# Check a specific boot's journal (after rollback / kernel panic)
journalctl --list-boots                   # show all recorded boots
journalctl -b -1 -p err                   # previous boot, errors only

rsync & aria2 — File Transfer

rsync — incremental sync & deployment:

# Deploy a website to a remote server (delete files removed locally)
rsync -avz --delete /var/www/html/ user@server:/var/www/html/

# Sync with progress and checksums
rsync -avz --progress --checksum /src/ /dst/

# Dry run — preview changes without transferring
rsync -avz --dry-run /src/ /dst/

# Exclude patterns
rsync -avz --exclude="*.log" --exclude=".git/" /src/ /dst/

# Sync over SSH on a custom port
rsync -avz -e "ssh -p 2222" /src/ user@server:/dst/

# Mirror a remote directory locally
rsync -avz [email protected]:/home/user/docs/ ~/mirror/docs/

aria2 — parallel multi-protocol downloader:

# Download a file (HTTP/HTTPS/FTP/SFTP/BitTorrent/Magnet)
aria2c https://example.com/largefile.zip

# Multi-connection download (up to 16 connections per server)
aria2c -x 16 -s 16 https://example.com/largefile.iso

# Download from multiple mirrors simultaneously
aria2c https://mirror1.com/file.iso https://mirror2.com/file.iso

# Torrent download
aria2c /path/to/file.torrent
aria2c "magnet:?xt=urn:btih:..."

# Download list from a file
aria2c -i ~/urls.txt

# Run as JSON-RPC daemon (for AriaNG web UI)
aria2c --enable-rpc --rpc-listen-all=false --rpc-secret=mysecret --daemon