Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Backups

Regular backups are a critical component of maintaining a reliable Kinesis API installation. This guide covers how to properly back up your Kinesis API data and configuration, ensuring you can recover from unexpected failures or data corruption.

Database Backend Considerations

⚠️ Important: Backup strategies differ depending on your database backend configuration.

Kinesis DB and SQLite:

  • Kinesis API’s built-in backup system fully supports these backends
  • Database files are included in automated backups
  • One-click restore functionality available
  • All manual backup methods apply

MySQL and PostgreSQL:

  • Kinesis API’s built-in backup system does NOT backup database data for these backends
  • Only configuration files (.env), media (public/), and translations are backed up by the built-in system
  • You must use MySQL/PostgreSQL native backup tools for database backups
  • See the Database Backend Compatibility section below for details

Database Backend Compatibility

For MySQL Databases

If you’re using MySQL (DB_BACKEND=mysql), you must use MySQL’s native backup tools:

Creating MySQL Backups:

# Full database backup
mysqldump -u username -p database_name > kinesis_backup_$(date +%Y%m%d).sql

# Compressed backup (recommended)
mysqldump -u username -p database_name | gzip > kinesis_backup_$(date +%Y%m%d).sql.gz

# All databases
mysqldump -u username -p --all-databases > all_databases_backup.sql

Restoring MySQL Backups:

# Restore from SQL file
mysql -u username -p database_name < kinesis_backup.sql

# Restore from compressed backup
gunzip < kinesis_backup.sql.gz | mysql -u username -p database_name

Automated MySQL Backup Script:

#!/bin/bash
# MySQL Backup Script for Kinesis API

DB_USER="your_username"
DB_PASS="your_password"
DB_NAME="kinesis_db"
BACKUP_DIR="/path/to/mysql-backups"
BACKUP_FILE="$BACKUP_DIR/kinesis-mysql-$(date +%Y%m%d-%H%M%S).sql.gz"

# Create backup directory if it doesn't exist
mkdir -p "$BACKUP_DIR"

# Create compressed backup
mysqldump -u"$DB_USER" -p"$DB_PASS" "$DB_NAME" | gzip > "$BACKUP_FILE"

# Optional: Keep only last 7 days of backups
find "$BACKUP_DIR" -name "kinesis-mysql-*.sql.gz" -type f -mtime +7 -delete

echo "MySQL backup completed: $BACKUP_FILE"

For more information, see MySQL Backup and Recovery.

For PostgreSQL Databases

If you’re using PostgreSQL (DB_BACKEND=postgresql), you must use PostgreSQL’s native backup tools:

Creating PostgreSQL Backups:

# Plain SQL backup
pg_dump -U username -d database_name > kinesis_backup_$(date +%Y%m%d).sql

# Compressed backup
pg_dump -U username -d database_name | gzip > kinesis_backup_$(date +%Y%m%d).sql.gz

# Custom format (recommended - faster restore, compression, selective restore)
pg_dump -U username -Fc -d database_name > kinesis_backup_$(date +%Y%m%d).dump

# Directory format (parallel backup/restore)
pg_dump -U username -Fd -d database_name -f kinesis_backup_$(date +%Y%m%d)

Restoring PostgreSQL Backups:

# Restore from SQL file
psql -U username -d database_name < kinesis_backup.sql

# Restore from compressed SQL
gunzip < kinesis_backup.sql.gz | psql -U username -d database_name

# Restore from custom format
pg_restore -U username -d database_name kinesis_backup.dump

# Restore with clean (drop existing objects first)
pg_restore -U username -d database_name --clean kinesis_backup.dump

Automated PostgreSQL Backup Script:

#!/bin/bash
# PostgreSQL Backup Script for Kinesis API

DB_USER="your_username"
DB_NAME="kinesis_db"
BACKUP_DIR="/path/to/pg-backups"
BACKUP_FILE="$BACKUP_DIR/kinesis-pg-$(date +%Y%m%d-%H%M%S).dump"

# Create backup directory if it doesn't exist
mkdir -p "$BACKUP_DIR"

# Create custom format backup (includes compression)
pg_dump -U "$DB_USER" -Fc -d "$DB_NAME" > "$BACKUP_FILE"

# Optional: Keep only last 7 days of backups
find "$BACKUP_DIR" -name "kinesis-pg-*.dump" -type f -mtime +7 -delete

echo "PostgreSQL backup completed: $BACKUP_FILE"

For more information, see PostgreSQL Backup and Restore.

Backup Methods

Kinesis API offers multiple backup approaches to suit different needs and technical preferences:

New in version 0.31.0: Kinesis API now includes a comprehensive backup system accessible through the web interface and API.

⚠️ Database Backend Support: The built-in backup system provides full database backup support for Kinesis DB and SQLite only. For MySQL and PostgreSQL backends, it backs up configuration and media files but not database data.

For Kinesis DB and SQLite Users:

This is the recommended method as it provides:

  • Automated backup creation and management
  • Complete database backup including all data
  • Built-in compression and storage optimization
  • Integrated restore functionality
  • Expiration and retention management
  • User-friendly web interface

For MySQL and PostgreSQL Users:

The built-in system will backup:

  • Configuration files (.env)
  • Media files (public/)
  • Translations (translations/)

But you must separately backup your database using native tools (see Database Backend Compatibility).

Getting Started:

  1. Log in to your Kinesis API web interface as a ROOT user
  2. Navigate to the “Backups” section in the sidebar
  3. Click “Create a new backup” to generate your first backup

For detailed instructions on using the built-in backup system, see the Backup Management page.

Benefits:

  • No manual file handling required
  • Automatic compression reduces storage space
  • Built-in validation ensures backup integrity
  • Simple restore process with automatic engine restart (Kinesis DB/SQLite)
  • Integrated with the permission system for security

2. Manual File-Based Backup

For users who prefer direct file system control or need custom backup procedures:

Understanding Kinesis API’s Data Storage

Before discussing backup strategies, it’s important to understand where Kinesis API stores its data:

  1. Database Files: Located in the data/ directory
    • Kinesis DB: {db_name}.pages, {db_name}.blobs, {db_name}.blobs.idx (where {db_name} defaults to main_db or the value of DB_NAME environment variable)
    • SQLite: Database file at the path specified in DATABASE_URL (default: data/db.sqlite), plus WAL files (-shm, -wal) if they exist
    • MySQL/PostgreSQL: Data stored on external database server (not in data/)
  2. License and Instance Files: data/license.json, data/instance.json (if they exist)
  3. Configuration: Stored in the .env file
  4. Media Files: Stored in the public/ directory (entire directory)
  5. Translations: Stored in the translations/ directory (entire directory)

A complete backup strategy must account for your database backend:

  • Kinesis DB/SQLite: Back up all five components
  • MySQL/PostgreSQL: Back up components 2-5 plus separate database dumps

Note: The backup system only backs up files that actually exist. For example, if you’re using SQLite, the Kinesis DB files won’t exist and vice versa.

Manual Backup Methods

Docker Installation Backup:

If you’re running Kinesis API via Docker (the recommended method), follow these steps:

  1. Stop the container (optional but recommended for consistency):

    docker stop kinesis-api

  2. Create the backup directory:

    mkdir backup/

  3. Back up the data directory:

    cp -r data/ backup/data/

  4. Back up the environment file:

    cp .env backup/.env

  5. Back up the public directory (if you’ve uploaded media):

    cp -r public/ backup/public/

  6. Back up the translations directory (if you’re making use of the localization engine):

    cp -r translations/ backup/translations/

  7. Restart the container (if you stopped it):

    docker start kinesis-api

Native Installation Backup:

If you’re running Kinesis API directly on your host:

  1. Stop the Kinesis API service:

    # If using systemd
sudo systemctl stop kinesis-api

# Or if running directly
kill $(pgrep kinesis-api)

  2. Back up the required directories and files:

    cp -r /path/to/kinesis-api/data/ /path/to/backup/data/
cp /path/to/kinesis-api/.env /path/to/backup/.env
cp -r /path/to/kinesis-api/public/ /path/to/backup/public/
cp -r /path/to/kinesis-api/translations/ /path/to/backup/translations/

  3. Restart the service:

    # If using systemd
sudo systemctl start kinesis-api

# Or if running directly
cd /path/to/kinesis-api && ./target/release/kinesis-api &

Automated Backup Scripts

Simple Daily Backup Script

Create a file called backup-kinesis-api.sh:

#!/bin/bash
# Kinesis API Backup Script

# Configuration
KINESIS_DIR="/path/to/kinesis-api"
BACKUP_DIR="/path/to/backups"
BACKUP_NAME="kinesis-api-backup-$(date +%Y%m%d-%H%M%S)"

# Create backup directory
mkdir -p "$BACKUP_DIR/$BACKUP_NAME"

# Optional: Stop the container for consistent backups
docker stop kinesis-api

# Copy the data
cp -r "$KINESIS_DIR/data" "$BACKUP_DIR/$BACKUP_NAME/"
cp "$KINESIS_DIR/.env" "$BACKUP_DIR/$BACKUP_NAME/"
cp -r "$KINESIS_DIR/public" "$BACKUP_DIR/$BACKUP_NAME/"
cp -r "$KINESIS_DIR/translations" "$BACKUP_DIR/$BACKUP_NAME/"

# Restart the container
docker start kinesis-api

# Compress the backup
cd "$BACKUP_DIR"
tar -czf "$BACKUP_NAME.tar.gz" "$BACKUP_NAME"
rm -rf "$BACKUP_NAME"

# Optional: Rotate backups (keep last 7 days)
find "$BACKUP_DIR" -name "kinesis-api-backup-*.tar.gz" -type f -mtime +7 -delete

echo "Backup completed: $BACKUP_DIR/$BACKUP_NAME.tar.gz"

Make the script executable and schedule it with cron:

chmod +x backup-kinesis-api.sh
crontab -e

Add a line to run it daily at 2 AM:

0 2 * * * /path/to/backup-kinesis-api.sh

Restoring from Backup

Kinesis API provides multiple restoration methods depending on how your backup was created:

The built-in backup system provides the simplest restoration process:

Through Web Interface:

  1. Access the Backups Page: Log in as a ROOT user and navigate to /web/backups
  2. Select Backup: Find the backup you want to restore from the list
  3. Click Restore: Click the restore button (refresh icon) next to your chosen backup
  4. Confirm Action: Confirm the restoration in the warning dialog
  5. Automatic Process: The system will:
    • Replace all current data with backup data
    • Restart the database engine automatically
    • Verify the restoration was successful

Through API:

# Replace with your actual values
curl -X GET "http://your-api-url/backup/restore?uid=1&id=backup_id" \
  -H "Authorization: Bearer your-jwt-token"

Important Notes:

  • ⚠️ Warning: Restoring a backup will replace ALL current data with the backup data
  • This action cannot be undone
  • The system automatically handles database engine restart
  • No manual file manipulation is required

Choosing the Right Backup:

When selecting a backup for restoration:

  • Check Creation Time: Ensure you’re selecting the backup from the correct time period
  • Read Description: Use backup descriptions to identify the purpose (e.g., “Before major update”)
  • Consider Data Loss: Any data created after the backup timestamp will be lost
  • Verify Backup Age: Check that the backup hasn’t expired and been automatically cleaned up

2. Manual File-Based Restoration

For backups created using manual file methods:

Docker Installation Restoration:

  1. Stop the running instance:

    docker stop kinesis-api

  2. Replace the data with the backup:

    # If your backup is compressed
tar -xzf kinesis-backup.tar.gz

# Replace the current data
rm -rf data/ .env public/ translations/
cp -r backup/data/ .
cp backup/.env .
cp -r backup/public/ .
cp -r backup/translations/ .

  3. Restart the service:

    docker start kinesis-api

Native Installation Restoration:

  1. Stop the Kinesis API service:

    # If using systemd
sudo systemctl stop kinesis-api

# Or if running directly
kill $(pgrep kinesis-api)

  2. Replace the files:

    # Backup current state (optional safety measure)
cp -r /path/to/kinesis-api/data/ /path/to/kinesis-api/data.backup/

# Restore from backup
rm -rf /path/to/kinesis-api/data/
cp -r /path/to/backup/data/ /path/to/kinesis-api/
cp /path/to/backup/.env /path/to/kinesis-api/
cp -r /path/to/backup/public/ /path/to/kinesis-api/
cp -r /path/to/backup/translations/ /path/to/kinesis-api/

  3. Restart the service:

    # If using systemd
sudo systemctl start kinesis-api

# Or if running directly
cd /path/to/kinesis-api && ./target/release/kinesis-api &

3. Hybrid Restoration Approach

You can also combine methods for maximum flexibility:

Scenario: Restore Built-in Backup to External Environment

  1. Download Backup: Use the built-in system to create a backup
  2. Export Data: Access the backup files from the system
  3. Manual Transfer: Copy to your target environment
  4. Manual Restoration: Use file-based restoration on the target

Scenario: Import Manual Backup into Built-in System

  1. Create Manual Backup: Using your existing file-based process
  2. Import to System: Manually copy files to Kinesis API directory
  3. Create Built-in Backup: Use the web interface to create a new backup for future use
  4. Built-in Restoration: Use the integrated system for future restorations

Restoration Best Practices

Before Restoration:

  1. Create Current Backup: Always backup your current state before restoring
  2. Verify Backup Integrity: Ensure the backup you’re restoring from is not corrupted
  3. Check Dependencies: Verify that environment configurations match
  4. Plan Downtime: Coordinate restoration during maintenance windows

During Restoration:

  1. Follow Exact Steps: Don’t skip steps or modify the process
  2. Monitor Logs: Watch for error messages during the restoration
  3. Verify Completeness: Ensure all files are restored properly

After Restoration:

  1. Test Functionality: Verify that all features work as expected
  2. Check Data Integrity: Confirm that data is accessible and correct
  3. Update Documentation: Record the restoration in your maintenance logs
  4. User Communication: Notify users of any changes or data loss

Troubleshooting Restoration Issues:

Built-in System Issues:

  • Backup Not Found: Check that the backup hasn’t expired
  • Permission Errors: Ensure you’re logged in as a ROOT user
  • Restoration Fails: Check system logs for specific error messages

Manual Restoration Issues:

  • File Permission Errors: Ensure proper ownership and permissions
  • Database Corruption: Try using an earlier backup
  • Configuration Mismatches: Verify environment variables match

Recovery from Failed Restoration:

If a restoration fails:

  1. Stop the Service: Prevent further damage
  2. Restore Previous State: Use your pre-restoration backup
  3. Investigate Cause: Check logs and file integrity
  4. Try Alternative Method: Consider using the other restoration approach
  5. Seek Support: Contact support if issues persist

Restoration Testing

Regular Testing Schedule:

  • Test restoration procedures monthly in a non-production environment
  • Document the complete process and timing
  • Verify that all data types and features work correctly after restoration

Testing Environment Setup:

  1. Isolated Environment: Use a separate test environment
  2. Production-like Configuration: Mirror your production setup
  3. Test Data: Use anonymized production data for realistic testing