Automating Backups with BlueFish File Mirror: Step-by-Step Workflow

Troubleshooting BlueFish File Mirror: Common Issues and Fixes

BlueFish File Mirror is a tool for mirroring files between systems. When it misbehaves, problems typically fall into a few repeatable categories: connectivity, permissions, configuration, performance, and data integrity. Below are common symptoms, root causes, and step-by-step fixes to get your mirrors healthy again.

1. Connection failures / sync not starting

Symptoms:

• Mirror job remains in “queued” or “failed” state.
• Remote host unreachable or timeouts.

Likely causes:

• Network outage, DNS issues, firewall blocking, wrong host/port, SSH key or credential problems.

Fixes:

1. Check basic network reachability
   • Ping the remote host: ping remote-host
   • Test port: telnet remote-host 22 or nc -vz remote-host 22
2. Verify DNS
   • Resolve the hostname: nslookup remote-host or dig remote-host
   • Use IP directly in config to rule out DNS.
3. Check firewall rules
   • Ensure required ports (SSH/rsync/HTTP) are open on both ends and any intermediate firewall.
4. Validate credentials
   • Manually SSH to the remote host using the same user/key: ssh -i /path/key user@remote-host
   • If using password auth, confirm the password and that password auth is enabled server-side.
5. Confirm service endpoints
   • If connecting to an API or daemon, ensure the service is running and listening: systemctl status service-name and ss -tuln | grep :PORT.

2. Authentication errors

Symptoms:

• “Permission denied”, “Authentication failed”, or repeated password prompts.

Likely causes:

• Wrong SSH key, missing key on server, expired credentials, incorrect permissions on key files.

Fixes:

1. Permissions on SSH key
   • Ensure private key has strict permissions: chmod 600 /path/to/key
2. Public key on remote
   • Confirm the public key exists in ~/.ssh/authorized_keys for the target user.
3. Agent and passphrase
   • If using ssh-agent, ensure the key is loaded: ssh-add -l
   • If using passphrase-protected keys in unattended jobs, consider using an agent or key without passphrase secured by OS protections.
4. Credential expiration
   • Rotate or renew expired credentials (API tokens, service accounts).
5. Logs
   • Inspect server auth logs: /var/log/auth.log or /var/log/secure for hints.

3. Permission denied on files or directories

Symptoms:

• Errors like “Permission denied” when reading or writing files during sync.

Likely causes:

• Mirror process runs as a user without rights, ACL issues, immutable flags, SELinux/AppArmor restrictions.

Fixes:

1. Confirm user and permissions
   • Identify the mirror service user and run: sudo -u mirror-user ls -l /path/to/files
2. Fix ownership and mode
   • Chown/chmod as appropriate: chown -R correctuser:group /mirror/path and chmod -R 750 /mirror/path
3. Check ACLs
   • List ACLs: getfacl /path/to/file
   • Remove or update ACL entries if necessary.
4. SELinux/AppArmor
   • Check SELinux mode: getenforce and relevant audit logs (/var/log/audit/audit.log). Temporarily set to permissive for testing: setenforce 0 (restore after test).
   • For AppArmor, check profiles and logs.
5. Immutable flags
   • Check and remove immutable flag: lsattr /path and chattr -i /path/to/file.

4. Partial or corrupted files after sync

Symptoms:

• Files truncated, checksum mismatches, or corrupted content.

Likely causes:

• Interrupted transfers, disk space issues, file locking, or protocol mismatches.

Fixes:

1. Check disk space
   • On both ends: df -h and df -i for inodes.
2. Retry with checksums
   • Use rsync-style checksum verification where possible (e.g., rsync –checksum).
3. Atomic updates
   • Configure mirror to write to a temporary file then rename to avoid partial reads.
4. Increase timeout and retry
   • Lengthen timeouts and enable retries to avoid interruptions.
5. Verify transfer protocol
   • Ensure both ends use compatible protocol versions and options (binary mode for FTP, correct rsync flags).
6. Run integrity checks
   • Compare checksums before and after: sha256sum file on both sides.

5. Performance slowdowns

Symptoms:

• Syncs take much longer than expected or saturate CPU/disk.

Likely causes:

• Network congestion, high IO on source/target, inefficient configuration (e.g., single-threaded, small packet sizes).

Fixes:

1. Measure baseline
   • Use iperf for network, iostat/top/vmstat for system IO/CPU.
2. Enable parallelism
   • Increase parallel file transfers or threads if supported by BlueFish File Mirror.
3. Tune rsync/transfer options
   • Use compression only when CPU allows (-z), adjust block size, enable pipelining.
4. Network tuning
   • Increase TCP window sizes, enable jumbo frames (if supported).
5. Off-peak windows
   • Schedule heavy syncs during low-usage periods.
6. Avoid small-file overhead
   • Bundle small files into archives for transport, then extract remotely.

6. Configuration errors or unexpected behavior after updates

Symptoms:

• Jobs that previously worked now fail after upgrading or config changes.

Likely causes:

• Config syntax changes, deprecated options, defaults changed in new versions.

Fixes:

1. Review release notes
   • Check changelogs for breaking changes and updated options.
2. Validate config syntax
   • Run any provided config-check tools or test commands.
3. Revert to known good config
   • Temporarily roll back to previous config/version to confirm the update caused the issue.
4. Migrate settings
   • Update configuration to new schema; follow vendor migration guides.

7. Logs empty or not helpful

Symptoms:

• Lack of useful logs for diagnosing failures.

Fixes:

1. Increase log level
   • Set debug or verbose logging in the service config.
2. Centralize logs
   • Send logs to syslog/journal or remote logging for easier correlation.
3. Correlate timestamps
   • Ensure clocks are synced via NTP/chrony on both ends.
4. Capture packet traces
   • When necessary, capture traffic with tcpdump for protocol-level debugging.

8. Jobs stuck in queue or duplicate runs

Symptoms:

• Multiple concurrent runs for same job, or jobs never start.

Likely causes:

• Locking mechanism failed, job cleanup not executed, or scheduler misconfiguration.

Fixes:

1. Inspect lock files
   • Remove stale lock files after confirming no active process.
2. Check scheduler settings
   • Ensure cron/systemd timers or internal scheduler aren’t overlapping jobs.
3. Enforce single-run
   • Use PID files or atomic lock mechanisms to prevent duplicates.
4. Graceful cleanup
   • Configure timeouts and cleanup routines for failed jobs.

Quick checklist for any mirror problem

• Verify network and DNS.
• Confirm credentials and permissions.
• Check disk space and resource utilization.
• Inspect and increase logging for clues.
• Retry with checksum/verification enabled.
• Consult release notes after upgrades.

If the problem persists, collect these artifacts before seeking vendor support:

• Mirror service logs (with timestamps).
• System logs (auth, syslog/journal).
• Output from: df -h, ss -tuln, ps aux | grep bluefish, and sha256sum of problematic files.
• Exact config file and version information.