Important
This package currently contains minimal features and is a work-in-progress
This package installs and integrates litestream, the SQLite replication tool, as a Django command.
pip install django-litestreamAdd django_litestream to your Django INSTALLED_APPS.
The package integrates all the commands and options from the litestream command-line tool with only minor changes.
Note
Django 5.1 was released a few days ago (as of the time of writing). If you are looking for a good production configuration for SQLite, check out this blog post.
These are the available configurations for django-litestream:
# settings.py
LITESTREAM = {
"config_file": "/etc/litestream.yml",
"path_prefix": None,
"bin_path": "litestream",
"dbs": [],
"extend_dbs": [],
"logging": {},
"addr": "",
}The config_file is where the Litestream configuration file should be generated by the init command.
The config_file will be automatically passed to every command you run, so you can freely change the default location
without having to pass the -config argument manually each time.
For example, you could place it in your project directory:
# settings.py
LITESTREAM = {
"config_file": BASE_DIR / "litestream.yml",
...
}The path_prefix is a string that will be prepended to the path of every database in the dbs configuration. This is useful if you are replicating databases from different projects to the same bucket, you could set the path_prefix to the project name so that the databases are stored in different folders in the bucket.
The bin_path is the path to the Litestream binary. If you want to use a custom installation, specify it here.
The dbs, logging, and addr configurations are the same as those in the Litestream configuration file. You can read more about them here. This allows you to keep your litestream configuration in your Django settings.
The extend_dbs is a list of dictionaries with the same format as the dbs configuration, and,
as its name suggests, it will extend the dbs configuration when the final configuration is generated.
You can run python manage.py litestream to see all available commands.
python manage.py litestream initThis command will write the Litestream configuration to the indicated config_file based on your settings.
If you did not specify any values in the dbs key, it will automatically parse your Django DATABASES configuration
and write one s3 replica for each SQLite database it finds.
For example, if you have the following DATABASES configuration:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "db.sqlite3",
},
"other": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "other.sqlite3",
},
}And your BASE_DIR is /home/tobi/myproject, the generated configuration after running init will look like this:
dbs:
- path: /home/tobi/myproject/db.sqlite3
replicas:
- type: s3
bucket: $LITESTREAM_REPLICA_BUCKET
path: db.sqlite3
access-key-id: $LITESTREAM_ACCESS_KEY_ID
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY
- path: /home/tobi/myproject/other.sqlite3
replicas:
- type: s3
bucket: $LITESTREAM_REPLICA_BUCKET
path: other.sqlite3
access-key-id: $LITESTREAM_ACCESS_KEY_ID
secret-access-key: $LITESTREAM_SECRET_ACCESS_KEYYou can tweak these settings according to your preferences. Check the databases settings reference for more information.
If you have any entries in the dbs configuration, the init command won’t automatically parse the DATABASES configuration.
To extend the configuration generated by the init command, you should use the extend_dbs configuration, for example:
# settings.py
LITESTREAM = {
"config_file": BASE_DIR / "litestream.yml",
"extend_dbs": [
{
"path": BASE_DIR / "cache.sqlite3",
"replicas": [
{
"type": "s3",
"bucket": "$LITESTREAM_REPLICA_BUCKET",
"path": "cache.sqlite3",
"access-key-id": "$LITESTREAM_ACCESS_KEY_ID",
"secret-access-key": "$LITESTREAM_SECRET_ACCESS_KEY",
}
]
}
]
}You can omit the access-key-id and secret-access-key keys and litestream will automatically use any of the environment variables below if available:
AWS_ACCESS_KEY_IDorLITESTREAM_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYorLITESTREAM_SECRET_ACCESS_KEY
This works exactly like the equivalent litestream command and lists all the databases.
Examples:
python manage.py litestream databasesImportant
For the rest of the commands, wherever you are asked to specify the database path db_path,
you can use the Django database alias instead, for example, default instead of /home/tobi/myproject/db.sqlite3.
This works exactly like the equivalent litestream command.
Examples:
python manage.py litestream generations default
python manage.py litestream generations -replica s3 defaultThis works exactly like the equivalent litestream command, except it does not support the ability to replicate a single file.
Running litestream replicate db_path replica_url won't work. You can only run:
python manage.py litestream replicate
python manage.py litestream replicate -exec "gunicorn myproject.wsgi:application"This is the command you will run in production using a process manager as its own process. You can run it separately (the first example) or use it to run your main Django process (second example). It would basically act as a process manager itself and run both the replicate and the Django process. The replication process will exit when your web server shuts down.
This works exactly like the equivalent litestream command.
Examples:
python manage.py litestream restore default
python manage.py litestream restore -if-replica-exists defaultThis command verifies the integrity of your backed-up databases. This process is inspired by the verify command of the litestream-ruby gem.
The verification process involves the following steps:
- Add Verification Data: A new row is added to a
_litestream_verificationtable in the specified database. This table is created if it does not already exist. The row contains a unique code and the current timestamp. - Wait for Replication: The command waits for 10 seconds to allow Litestream to replicate the new row to the configured storage providers.
- Restore Backup: The latest backup is restored from the storage provider to a temporary location.
- Check Verification Data: The restored database is checked to ensure that the verification row is present. This ensures that the backup is both restorable and up-to-date.
If the verification row is not found in the restored database, the command will return an error indicating that the backup data is out of sync. If the row is found, the command confirms that the backup data is in sync.
Examples:
python manage.py litestream verify defaultThis check ensures that the restored database file Exists, can be opened by SQLite, and has up-to-date data.
This works exactly like the equivalent litestream command.
Examples:
python manage.py litestream snapshots default
python manage.py litestream snapshots -replica s3 defaultThis works exactly like the equivalent litestream command.
Examples:
python manage.py litestream wal default
python manage.py litestream wal -replica s3 defaultPrint the version of the Litestream binary.
python manage.py litestream versiondjango-litestream is distributed under the terms of the MIT license.