Backing up large WordPress sites

Most backup failures on large sites have the same causes: one request trying to do too much, and a database export that depends on tools the host does not provide. BackupScope is built around avoiding both.

Why large backups fail

Time limits. PHP stops a request after its max_execution_time, often 30 to 120 seconds. Zipping gigabytes in one request does not fit.
Memory limits. Building a large archive or export in memory runs out of PHP memory.
No mysqldump. Many hosts do not allow running command-line tools from PHP.
Large database rows. A single very large value can exceed the MySQL max_allowed_packet when the export is imported again.

Small, resumable steps

Every BackupScope backup is a job made of steps: preflight, scan, database export, archive, verify, finalize. Each step does a limited amount of work per request and saves its position, so no single request comes near the time limit. A lock makes sure two requests never work on the same backup at once.

If a request is cut off, by a time-out, a server restart or a closed browser tab, the next request continues from the last saved position instead of starting again.

Two archive engines, chosen by size

The scan estimates the backup size, and BackupScope picks the engine from that estimate:

Up to 5 GB:a standard ZIP built with PHP's ZipArchive.
Above 5 GB: a streaming ZIP64 archive, written in batches across many requests. ZIP64 is the ZIP format for archives and files larger than 4 GB, supported by all current unzip tools.

If the standard engine fails on a site below the threshold, BackupScope falls back to the streaming engine automatically.

5 GB is a routing threshold, not a size limit. It only decides which engine builds the archive. What limits a backup is your server: free disk space for the archive, and any file size limits of your host.

Developers can change the threshold in wp-config.php (value in bytes), for example to use the streaming engine for every backup above 1 GB:

define( 'BACKUPSCOPE_BATCH_THRESHOLD', 1073741824 );

The instabackup_batch_threshold_bytes filter does the same from code.

Database export without mysqldump

The database is exported through WordPress's own database connection, table by table and page by page, so it works where mysqldump is not available and never holds a whole table in memory. Very large values are written in chunks so that the export can be imported again with a default max_allowed_packet. Binary and BIT columns are exported in a form MySQL reads back exactly.

Verification

When the archive is complete, it is read back and every entry's CRC-32 checksum is checked. On a large site this also runs in resumable steps. A backup only becomes your latest backup after it passes.

Tips for large sites

Check free disk space. The scan shows the estimated backup size. The server needs at least that much free space, plus room for the database export.
Leave out what you do not need.Caches and other backup plugins' archives are excluded automatically. Use a Custom backup to untick large folders you keep elsewhere, for example a video archive.
Keep the page open in BackupScope Free. The backup runs while the page is open and resumes when you open it again. BackupScope Pro runs scheduled backups in the background.
Send copies off-site. BackupScope Pro uploads large backups to S3-compatible storage in parts (multipart upload), so an interrupted upload does not start from zero.
Restoring a large site by hand? The Restore Guide covers importing a large database.sql from the command line.