mirror of
https://github.com/jkaninda/mysql-bkup.git
synced 2026-11-13 08:37:15 +00:00
feat: mysql-bkup v2 on bkupkit
MysQL backup and restore built on the shared bkupkit core: streaming end to end, with no dump held in memory or written to local disk.
This commit is contained in:
1 parent
ccb3fcdf26
commit
28de0d7b52
65 files changed
+3453
-3871
No files matched your search
@@ -48,7 +48,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Azure Blob Configuration
|
||||
- AZURE_STORAGE_CONTAINER_NAME=backup-container
|
||||
|
||||
@@ -30,7 +30,7 @@ docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_PORT=3306" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup backup --all-databases
|
||||
```
|
||||
@@ -50,7 +50,7 @@ docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_PORT=3306" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup backup --all-in-one
|
||||
```
|
||||
|
||||
@@ -50,10 +50,10 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## FTP Configuration
|
||||
- FTP_HOST="hostname"
|
||||
- FTP_HOST=hostname
|
||||
- FTP_PORT=21
|
||||
- FTP_USER=user
|
||||
- FTP_PASSWORD=password
|
||||
|
||||
@@ -42,14 +42,14 @@ services:
|
||||
# In production, lock your image tag to a specific release version
|
||||
# instead of using `latest`. Check https://github.com/jkaninda/pg-bkup/releases
|
||||
# for available releases.
|
||||
image: jkaninda/pg-bkup
|
||||
image: jkaninda/mysql-bkup
|
||||
container_name: pg-bkup
|
||||
command: backup --storage s3 -d database --path /my-custom-path
|
||||
environment:
|
||||
- DB_PORT=5432
|
||||
- DB_HOST=postgres
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## AWS Configuration
|
||||
- AWS_S3_ENDPOINT=https://s3.amazonaws.com
|
||||
@@ -58,7 +58,7 @@ services:
|
||||
- AWS_ACCESS_KEY=xxxx
|
||||
- AWS_SECRET_KEY=xxxxx
|
||||
## Optional: Disable SSL for S3 alternatives like Minio
|
||||
- AWS_DISABLE_SSL="false"
|
||||
- AWS_DISABLE_SSL=false
|
||||
## Optional: Enable path-style access for S3 alternatives like Minio
|
||||
- AWS_FORCE_PATH_STYLE=false
|
||||
|
||||
@@ -91,7 +91,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## AWS Configuration
|
||||
- AWS_S3_ENDPOINT=https://s3.amazonaws.com
|
||||
@@ -104,11 +104,11 @@ services:
|
||||
## Optional: Delete old backups after a specified number of days
|
||||
#- BACKUP_RETENTION_DAYS=7
|
||||
## Optional: Disable SSL for S3 alternatives like Minio
|
||||
- AWS_DISABLE_SSL="false"
|
||||
- AWS_DISABLE_SSL=false
|
||||
## Optional: Enable path-style access for S3 alternatives like Minio
|
||||
- AWS_FORCE_PATH_STYLE=false
|
||||
|
||||
# Ensure the pg-bkup container is connected to the same network as your database
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
- web
|
||||
|
||||
@@ -122,5 +122,5 @@ networks:
|
||||
|
||||
- **Cron Expression**: Use the `--cron-expression` flag or `BACKUP_CRON_EXPRESSION` environment variable to define the backup schedule. For example, `0 1 * * *` runs the backup daily at 1:00 AM.
|
||||
- **Backup Retention**: Optionally, use the `BACKUP_RETENTION_DAYS` environment variable to automatically delete backups older than a specified number of days.
|
||||
- **S3 Alternatives**: If using an S3 alternative like Minio, set `AWS_DISABLE_SSL="true"` and `AWS_FORCE_PATH_STYLE="true"` as needed.
|
||||
- **S3 Alternatives**: If using an S3 alternative like Minio, set `AWS_DISABLE_SSL=true` and `AWS_FORCE_PATH_STYLE=true` as needed. In the Compose list syntax, do not quote values: the quotes would become part of the value.
|
||||
|
||||
@@ -23,15 +23,16 @@ This section explains how to set up and configure SSH-based backups.
|
||||
3. **Required Environment Variables**
|
||||
The following environment variables are mandatory for SSH-based backups:
|
||||
|
||||
- `SSH_HOST`: The hostname or IP address of the remote server.
|
||||
- `SSH_USER`: The username for SSH authentication.
|
||||
- `SFTP_HOST`: The hostname or IP address of the remote server.
|
||||
- `SFTP_USER`: The username for SSH authentication.
|
||||
- `REMOTE_PATH`: The directory on the remote server where backups will be stored.
|
||||
- `SSH_IDENTIFY_FILE`: The path to the private key file for SSH authentication.
|
||||
- `SSH_PORT`: The SSH port (default is `22`).
|
||||
- `SSH_PASSWORD`: (Optional) Use this only if you are not using a private key for authentication.
|
||||
- `SFTP_IDENTITY_FILE`: The path to the private key file for SSH authentication (`SFTP_IDENTITY_FILE` still works).
|
||||
- `SFTP_KNOWN_HOSTS`: A `known_hosts` file used to verify the server's host key. Host keys are verified by default since v2; `SFTP_INSECURE_IGNORE_HOST_KEY=true` turns the check off (not recommended).
|
||||
- `SFTP_PORT`: The SSH port (default is `22`).
|
||||
- `SFTP_PASSWORD`: (Optional) Use this only if you are not using a private key for authentication.
|
||||
|
||||
{: .note }
|
||||
**Security Recommendation**: Using a private key (`SSH_IDENTIFY_FILE`) is strongly recommended over password-based authentication (`SSH_PASSWORD`) for better security.
|
||||
**Security Recommendation**: Using a private key (`SFTP_IDENTITY_FILE`) is strongly recommended over password-based authentication (`SFTP_PASSWORD`) for better security.
|
||||
|
||||
---
|
||||
|
||||
@@ -54,16 +55,17 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## SSH Configuration
|
||||
- SSH_HOST="hostname"
|
||||
- SSH_PORT=22
|
||||
- SSH_USER=user
|
||||
- SFTP_HOST=hostname
|
||||
- SFTP_PORT=22
|
||||
- SFTP_USER=user
|
||||
- REMOTE_PATH=/home/jkaninda/backups
|
||||
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
|
||||
- SFTP_IDENTITY_FILE=/tmp/id_ed25519
|
||||
- SFTP_KNOWN_HOSTS=/config/known_hosts
|
||||
## Optional: Use password instead of private key (not recommended)
|
||||
#- SSH_PASSWORD=password
|
||||
#- SFTP_PASSWORD=password
|
||||
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
@@ -97,18 +99,19 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=postgres
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## SSH Configuration
|
||||
- SSH_HOST="hostname"
|
||||
- SSH_PORT=22
|
||||
- SSH_USER=user
|
||||
- SFTP_HOST=hostname
|
||||
- SFTP_PORT=22
|
||||
- SFTP_USER=user
|
||||
- REMOTE_PATH=/home/jkaninda/backups
|
||||
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
|
||||
- SFTP_IDENTITY_FILE=/tmp/id_ed25519
|
||||
- SFTP_KNOWN_HOSTS=/config/known_hosts
|
||||
## Optional: Delete old backups after a specified number of days
|
||||
#- BACKUP_RETENTION_DAYS=7
|
||||
## Optional: Use password instead of private key (not recommended)
|
||||
#- SSH_PASSWORD=password
|
||||
#- SFTP_PASSWORD=password
|
||||
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
@@ -124,6 +127,6 @@ networks:
|
||||
|
||||
- **Cron Expression**: Use the `--cron-expression` flag or `BACKUP_CRON_EXPRESSION` environment variable to define the backup schedule. For example, `0 1 * * *` runs the backup daily at 1:00 AM.
|
||||
- **Backup Retention**: Optionally, use the `BACKUP_RETENTION_DAYS` environment variable to automatically delete backups older than a specified number of days.
|
||||
- **Security**: Always prefer private key authentication (`SSH_IDENTIFY_FILE`) over password-based authentication (`SSH_PASSWORD`) for enhanced security.
|
||||
- **Security**: Always prefer private key authentication (`SFTP_IDENTITY_FILE`) over password-based authentication (`SFTP_PASSWORD`) for enhanced security.
|
||||
|
||||
---
|
||||
@@ -43,7 +43,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
@@ -64,7 +64,7 @@ You can also run backups directly using the Docker CLI:
|
||||
docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/pg-bkup backup -d database_name
|
||||
```
|
||||
@@ -92,7 +92,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Optional: Define a cron schedule for recurring backups
|
||||
- BACKUP_CRON_EXPRESSION=@midnight
|
||||
|
||||
@@ -46,7 +46,7 @@ spec:
|
||||
value: ""
|
||||
- name: DB_NAME
|
||||
value: ""
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: ""
|
||||
# Use Kubernetes Secrets for sensitive data like passwords
|
||||
- name: DB_PASSWORD
|
||||
@@ -104,20 +104,20 @@ spec:
|
||||
value: ""
|
||||
- name: DB_NAME
|
||||
value: "dbname"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "postgres"
|
||||
# Use Kubernetes Secrets for sensitive data like passwords
|
||||
- name: DB_PASSWORD
|
||||
value: ""
|
||||
- name: SSH_HOST_NAME
|
||||
- name: SFTP_HOST
|
||||
value: "xxx"
|
||||
- name: SSH_PORT
|
||||
- name: SFTP_PORT
|
||||
value: "22"
|
||||
- name: SSH_USER
|
||||
- name: SFTP_USER
|
||||
value: "xxx"
|
||||
- name: SSH_PASSWORD
|
||||
- name: SFTP_PASSWORD
|
||||
value: "xxxx"
|
||||
- name: SSH_REMOTE_PATH
|
||||
- name: REMOTE_PATH
|
||||
value: "/home/toto/backup"
|
||||
# Optional: Required if you want to encrypt your backup
|
||||
- name: GPG_PASSPHRASE
|
||||
@@ -161,20 +161,20 @@ spec:
|
||||
value: ""
|
||||
- name: DB_NAME
|
||||
value: "dbname"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "postgres"
|
||||
# Use Kubernetes Secrets for sensitive data like passwords
|
||||
- name: DB_PASSWORD
|
||||
value: ""
|
||||
- name: SSH_HOST_NAME
|
||||
- name: SFTP_HOST
|
||||
value: "xxx"
|
||||
- name: SSH_PORT
|
||||
- name: SFTP_PORT
|
||||
value: "22"
|
||||
- name: SSH_USER
|
||||
- name: SFTP_USER
|
||||
value: "xxx"
|
||||
- name: SSH_PASSWORD
|
||||
- name: SFTP_PASSWORD
|
||||
value: "xxxx"
|
||||
- name: SSH_REMOTE_PATH
|
||||
- name: REMOTE_PATH
|
||||
value: "/home/toto/backup"
|
||||
# Optional: Required if your backup was encrypted
|
||||
#- name: GPG_PASSPHRASE
|
||||
@@ -220,20 +220,20 @@ spec:
|
||||
value: ""
|
||||
- name: DB_NAME
|
||||
value: "test"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "postgres"
|
||||
# Use Kubernetes Secrets for sensitive data like passwords
|
||||
- name: DB_PASSWORD
|
||||
value: ""
|
||||
- name: SSH_HOST_NAME
|
||||
- name: SFTP_HOST
|
||||
value: "192.168.1.16"
|
||||
- name: SSH_PORT
|
||||
- name: SFTP_PORT
|
||||
value: "2222"
|
||||
- name: SSH_USER
|
||||
- name: SFTP_USER
|
||||
value: "jkaninda"
|
||||
- name: SSH_REMOTE_PATH
|
||||
- name: REMOTE_PATH
|
||||
value: "/config/backup"
|
||||
- name: SSH_PASSWORD
|
||||
- name: SFTP_PASSWORD
|
||||
value: "password"
|
||||
# Optional: Required if you want to encrypt your backup
|
||||
#- name: GPG_PASSPHRASE
|
||||
@@ -283,20 +283,20 @@ spec:
|
||||
value: ""
|
||||
- name: DB_NAME
|
||||
value: "test"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "postgres"
|
||||
# Use Kubernetes Secrets for sensitive data like passwords
|
||||
- name: DB_PASSWORD
|
||||
value: ""
|
||||
- name: SSH_HOST_NAME
|
||||
- name: SFTP_HOST
|
||||
value: "192.168.1.16"
|
||||
- name: SSH_PORT
|
||||
- name: SFTP_PORT
|
||||
value: "2222"
|
||||
- name: SSH_USER
|
||||
- name: SFTP_USER
|
||||
value: "jkaninda"
|
||||
- name: SSH_REMOTE_PATH
|
||||
- name: REMOTE_PATH
|
||||
value: "/config/backup"
|
||||
- name: SSH_PASSWORD
|
||||
- name: SFTP_PASSWORD
|
||||
value: "password"
|
||||
# Optional: Required if you want to encrypt your backup
|
||||
#- name: GPG_PASSPHRASE
|
||||
@@ -341,7 +341,7 @@ spec:
|
||||
value: "3306"
|
||||
- name: DB_NAME
|
||||
value: "dbname"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "username"
|
||||
- name: DB_PASSWORD
|
||||
value: "password"
|
||||
@@ -352,7 +352,7 @@ spec:
|
||||
value: "3306"
|
||||
- name: TARGET_DB_NAME
|
||||
value: "dbname"
|
||||
- name: TARGET_DB_USERNAME
|
||||
- name: TARGET_DB_USER
|
||||
value: "username"
|
||||
- name: TARGET_DB_PASSWORD
|
||||
value: "password"
|
||||
|
||||
@@ -42,7 +42,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Required to encrypt backup
|
||||
- GPG_PASSPHRASE=my-secure-passphrase
|
||||
@@ -58,7 +58,7 @@ networks:
|
||||
|
||||
## Using GPG Public Key
|
||||
|
||||
To encrypt backups using a GPG public key, set the `GPG_PUBLIC_KEY` environment variable to the path of your public key file. Backups encrypted with a public key require manual decryption before restoration.
|
||||
To encrypt backups using a GPG public key, set the `GPG_PUBLIC_KEY` environment variable to the path of your public key file. To restore such backups, set `GPG_PRIVATE_KEY` (a file path or the inline armored key; `/config/gnupg/private_key.asc` is picked up automatically) and, if the key is protected, `GPG_PRIVATE_KEY_PASSPHRASE` (defaults to `GPG_PASSPHRASE`).
|
||||
|
||||
### Example Configuration
|
||||
|
||||
@@ -78,7 +78,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Required to encrypt backup
|
||||
- GPG_PUBLIC_KEY=/config/public_key.asc
|
||||
@@ -116,5 +116,21 @@ gpg --output database_20240730_044201.sql.gz \
|
||||
## Key Notes
|
||||
|
||||
- **Automatic Restoration**: Backups encrypted with a GPG passphrase can be restored directly without manual decryption.
|
||||
- **Manual Decryption**: Backups encrypted with a GPG public key require manual decryption using the corresponding private key.
|
||||
- **Private key restore**: Backups encrypted with a GPG public key restore directly when `GPG_PRIVATE_KEY` is set; manual decryption also works.
|
||||
- **Fail closed**: if `GPG_PUBLIC_KEY` (or any configured key) cannot be read, the backup fails instead of being stored unencrypted.
|
||||
- **Security**: Always keep your GPG passphrase and private key secure. Use Kubernetes Secrets or other secure methods to manage sensitive data.
|
||||
|
||||
---
|
||||
|
||||
## Using age
|
||||
|
||||
[age](https://age-encryption.org) is a modern alternative to GPG. Set `AGE_RECIPIENTS` (one or more `age1...` or `ssh-ed25519 ...` public keys, comma or newline separated, or `AGE_RECIPIENTS_FILE`) and backups are saved as `.sql.gz.age`. Restore with the matching identity in `AGE_IDENTITY` (`AGE-SECRET-KEY-...`) or `AGE_IDENTITY_FILE` (default `/config/age/identity.txt`).
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- AGE_RECIPIENTS=age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
|
||||
# restore:
|
||||
# - AGE_IDENTITY_FILE=/run/secrets/age_identity
|
||||
```
|
||||
|
||||
`AGE_PASSPHRASE` encrypts with a passphrase instead (it needs about 256 MiB of memory for the key derivation). `ENCRYPTION=auto|none|gpg|age` chooses explicitly; with `auto` (default) GPG wins when both GPG and age keys are set. Decrypt by hand with `age -d -i key.txt file.sql.gz.age | gunzip`.
|
||||
+10
-7
@@ -45,14 +45,14 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
|
||||
## Target Database
|
||||
- TARGET_DB_HOST=target-postgres
|
||||
- TARGET_DB_HOST=target-mysql
|
||||
- TARGET_DB_PORT=3306
|
||||
- TARGET_DB_NAME=dbname
|
||||
- TARGET_DB_USERNAME=username
|
||||
- TARGET_DB_USER=username
|
||||
- TARGET_DB_PASSWORD=password
|
||||
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
@@ -78,14 +78,14 @@ Save your source and target database connection details in an environment file (
|
||||
DB_HOST=postgres
|
||||
DB_PORT=3306
|
||||
DB_NAME=dbname
|
||||
DB_USERNAME=username
|
||||
DB_USER=username
|
||||
DB_PASSWORD=password
|
||||
|
||||
## Target Database
|
||||
TARGET_DB_HOST=target-postgres
|
||||
TARGET_DB_HOST=target-mysql
|
||||
TARGET_DB_PORT=3306
|
||||
TARGET_DB_NAME=dbname
|
||||
TARGET_DB_USERNAME=username
|
||||
TARGET_DB_USER=username
|
||||
TARGET_DB_PASSWORD=password
|
||||
```
|
||||
|
||||
@@ -95,12 +95,15 @@ TARGET_DB_PASSWORD=password
|
||||
docker run --rm --network your_network_name \
|
||||
--env-file your-env \
|
||||
-v $PWD/backup:/backup/ \
|
||||
jkaninda/pg-bkup migrate
|
||||
jkaninda/mysql-bkup migrate
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Notes
|
||||
|
||||
- **Streaming**: the dump is piped straight into the target server; no file is written. The target database is created if it does not exist, and `TARGET_DB_NAME` defaults to the source name.
|
||||
- **All databases**: `migrate --all-databases` copies every database (system databases excluded, `--exclude-db` / `BACKUP_EXCLUDE_DATABASES` honoured) under the same names.
|
||||
- **MySQL and MariaDB**: source and target may be different flavours; each side uses its own client. Objects that only exist in one flavour (for example MySQL-only collations on old MariaDB versions) can make a cross-flavour migration fail.
|
||||
- **Irreversible Operation**: The `migrate` command directly transfers data from the source to the target database. Ensure you have a backup of the target database before proceeding.
|
||||
- **Network Configuration**: Ensure the `mysql-bkup` container is connected to the same network as your source and target databases.
|
||||
@@ -40,28 +40,28 @@ databases:
|
||||
- host: mysql1 # Optional: Overrides DB_HOST or uses DB_HOST_DATABASE1.
|
||||
port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_DATABASE1.
|
||||
name: database1 # Required: Database name.
|
||||
user: database1 # Optional: Overrides DB_USERNAME or uses DB_USERNAME_DATABASE1.
|
||||
user: database1 # Optional: Overrides DB_USER or uses DB_USER_DATABASE1.
|
||||
password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_DATABASE1.
|
||||
path: /s3-path/database1 # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
|
||||
|
||||
- host: mysql2 # Optional: Overrides DB_HOST or uses DB_HOST_LLAP.
|
||||
port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_LLAP.
|
||||
name: lldap # Required: Database name.
|
||||
user: lldap # Optional: Overrides DB_USERNAME or uses DB_USERNAME_LLAP.
|
||||
user: lldap # Optional: Overrides DB_USER or uses DB_USER_LLAP.
|
||||
password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_LLAP.
|
||||
path: /s3-path/lldap # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
|
||||
|
||||
- host: mysql3 # Optional: Overrides DB_HOST or uses DB_HOST_KEYCLOAK.
|
||||
port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_KEYCLOAK.
|
||||
name: keycloak # Required: Database name.
|
||||
user: keycloak # Optional: Overrides DB_USERNAME or uses DB_USERNAME_KEYCLOAK.
|
||||
user: keycloak # Optional: Overrides DB_USER or uses DB_USER_KEYCLOAK.
|
||||
password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_KEYCLOAK.
|
||||
path: /s3-path/keycloak # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
|
||||
|
||||
- host: mysql4 # Optional: Overrides DB_HOST or uses DB_HOST_JOPLIN.
|
||||
port: 3306 # Optional: Default is 5432. Overrides DB_PORT or uses DB_PORT_JOPLIN.
|
||||
name: joplin # Required: Database name.
|
||||
user: joplin # Optional: Overrides DB_USERNAME or uses DB_USERNAME_JOPLIN.
|
||||
user: joplin # Optional: Overrides DB_USER or uses DB_USER_JOPLIN.
|
||||
password: password # Optional: Overrides DB_PASSWORD or uses DB_PASSWORD_JOPLIN.
|
||||
path: /s3-path/joplin # Required: Backup path for SSH, FTP, or S3 (e.g., /home/toto/backup/).
|
||||
```
|
||||
|
||||
@@ -7,7 +7,7 @@ nav_order: 13
|
||||
|
||||
# Receive Notifications
|
||||
|
||||
You can configure the system to send email or Telegram notifications when a backup succeeds or fails.
|
||||
You can configure the system to send email (SMTP or Posta) or Telegram notifications when a backup succeeds or fails.
|
||||
|
||||
This section explains how to set up and customize notifications.
|
||||
|
||||
@@ -31,7 +31,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## SMTP Configuration
|
||||
- MAIL_HOST=smtp.example.com
|
||||
@@ -41,7 +41,8 @@ services:
|
||||
- MAIL_FROM=Backup Jobs <backup@example.com>
|
||||
## Multiple recipients separated by a comma
|
||||
- MAIL_TO=me@example.com,team@example.com,manager@example.com
|
||||
- MAIL_SKIP_TLS=false
|
||||
## Only for SMTP servers with a self-signed certificate
|
||||
#- MAIL_INSECURE_SKIP_VERIFY=true
|
||||
## Time format for notifications
|
||||
- TIME_FORMAT=2006-01-02 at 15:04:05
|
||||
## Backup reference (e.g., database/cluster name or server name)
|
||||
@@ -55,6 +56,44 @@ networks:
|
||||
|
||||
---
|
||||
|
||||
## Posta Notifications
|
||||
|
||||
[Posta](https://github.com/goposta/posta) can deliver the notification emails instead of an SMTP server. Set the Posta URL, an API key with the `send` scope and the recipients. By default the same HTML email is sent (with a plain-text version). To use a template stored in Posta instead, set `POSTA_TEMPLATE` (and optionally `POSTA_TEMPLATE_ERROR` for failures); the notification fields (`database`, `file`, `duration`, `storage`, `location`, `size`, `backup_size`, `sha256`, `encryption`, `error`, `time`, `success`, `tool`...) are passed as template data.
|
||||
|
||||
### Example: Posta Notification Configuration
|
||||
|
||||
```yaml
|
||||
services:
|
||||
mysql-bkup:
|
||||
image: jkaninda/mysql-bkup
|
||||
container_name: mysql-bkup
|
||||
command: backup
|
||||
volumes:
|
||||
- ./backup:/backup
|
||||
environment:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=db
|
||||
- DB_NAME=database
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Posta
|
||||
- POSTA_URL=https://posta.example.com
|
||||
- POSTA_API_KEY=psk_your_api_key
|
||||
- POSTA_FROM=Backup Jobs <backup@example.com>
|
||||
## Multiple recipients separated by a comma (default MAIL_TO)
|
||||
- POSTA_TO=me@example.com,team@example.com
|
||||
## Optional: templates stored in Posta
|
||||
#- POSTA_TEMPLATE=backup-success
|
||||
#- POSTA_TEMPLATE_ERROR=backup-failure
|
||||
#- POSTA_LANGUAGE=en
|
||||
## Only for account-wide API keys
|
||||
#- POSTA_WORKSPACE_ID=42
|
||||
## Only notify failures
|
||||
#- NOTIFY_ON=failure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Telegram Notifications
|
||||
|
||||
To send Telegram notifications, provide your bot token and chat ID. Notifications will be sent for both successful and failed backup runs.
|
||||
@@ -73,7 +112,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## Telegram Configuration
|
||||
- TG_TOKEN=[BOT ID]:[BOT TOKEN]
|
||||
@@ -191,7 +230,7 @@ Backup Reference: {{.BackupReference}}
|
||||
|
||||
## Key Notes
|
||||
|
||||
- **SMTP Configuration**: Ensure your SMTP server supports TLS unless `MAIL_SKIP_TLS` is set to `true`.
|
||||
- **SMTP Configuration**: Ensure your SMTP server supports TLS unless `MAIL_INSECURE_SKIP_VERIFY` is set to `true`. `MAIL_TLS=starttls|tls|none` selects the TLS mode. The 1.x variable `MAIL_SKIP_TLS` is still read with its documented meaning (`true` = skip certificate verification); 1.x had it inverted. Slack (`SLACK_WEBHOOK_URL`), Discord (`DISCORD_WEBHOOK_URL`), ntfy (`NTFY_URL`), generic webhooks (`WEBHOOK_URL`, `WEBHOOK_SECRET`) and healthchecks.io pings (`HEALTHCHECK_PING_URL`) are also available; `NOTIFY_ON=always|failure|success` filters them.
|
||||
- **Telegram Configuration**: Obtain your bot token and chat ID from Telegram.
|
||||
- **Custom Templates**: Mount custom templates to `/config/templates` to override default notifications.
|
||||
- **Time Format**: Use the `TIME_FORMAT` environment variable to customize the timestamp format in notifications.
|
||||
@@ -44,7 +44,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## AWS S3 Configuration
|
||||
- AWS_S3_ENDPOINT=https://s3.amazonaws.com
|
||||
@@ -56,7 +56,7 @@ services:
|
||||
- AWS_DISABLE_SSL=false
|
||||
## Optional: Enable path-style access for S3 alternatives like Minio
|
||||
- AWS_FORCE_PATH_STYLE=false
|
||||
# Ensure the pg-bkup container is connected to the same network as your database
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
- web
|
||||
|
||||
|
||||
@@ -45,16 +45,17 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
## SSH Configuration
|
||||
- SSH_HOST_NAME=hostname
|
||||
- SSH_PORT=22
|
||||
- SSH_USER=user
|
||||
- SSH_REMOTE_PATH=/home/jkaninda/backups
|
||||
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
|
||||
- SFTP_HOST=hostname
|
||||
- SFTP_PORT=22
|
||||
- SFTP_USER=user
|
||||
- REMOTE_PATH=/home/jkaninda/backups
|
||||
- SFTP_IDENTITY_FILE=/tmp/id_ed25519
|
||||
- SFTP_KNOWN_HOSTS=/config/known_hosts
|
||||
## Optional: Use password instead of private key (not recommended)
|
||||
#- SSH_PASSWORD=password
|
||||
#- SFTP_PASSWORD=password
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
- web
|
||||
@@ -70,5 +71,5 @@ networks:
|
||||
- **Supported File Formats**: The restore process supports `.sql`, `.sql.gz`, `.sql.gpg`, and `.sql.gz.gpg` files.
|
||||
- **SSH Path**: Use the `--path` flag to specify the folder on the SSH remote server where the backup file is located.
|
||||
- **Encrypted Backups**: If the backup is encrypted with GPG, ensure the `GPG_PASSPHRASE` environment variable is set for automatic decryption.
|
||||
- **SSH Authentication**: Use a private key (`SSH_IDENTIFY_FILE`) for SSH authentication instead of a password for better security.
|
||||
- **SSH Authentication**: Use a private key (`SFTP_IDENTITY_FILE`) for SSH authentication instead of a password for better security. Host keys are verified against `SFTP_KNOWN_HOSTS` (or set `SFTP_INSECURE_IGNORE_HOST_KEY=true`).
|
||||
- **Network Configuration**: Ensure the `mysql-bkup` container is connected to the same network as your database.
|
||||
@@ -45,9 +45,9 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=postgres
|
||||
- DB_NAME=database
|
||||
- DB_USERNAME=username
|
||||
- DB_USER=username
|
||||
- DB_PASSWORD=password
|
||||
# Ensure the pg-bkup container is connected to the same network as your database
|
||||
# Ensure the mysql-bkup container is connected to the same network as your database
|
||||
networks:
|
||||
- web
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_PORT=3306" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup backup -d database_name
|
||||
```
|
||||
@@ -42,7 +42,7 @@ docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_PORT=3306" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup backup --all-databases --disable-compression
|
||||
```
|
||||
@@ -60,7 +60,7 @@ docker run --rm --network your_network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=dbhost" \
|
||||
-e "DB_PORT=3306" \
|
||||
-e "DB_USERNAME=username" \
|
||||
-e "DB_USER=username" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup restore -d database_name -f backup_file.sql.gz
|
||||
```
|
||||
@@ -73,7 +73,7 @@ Below is an example of a `docker-compose.yml` file for running a one-time backup
|
||||
|
||||
```yaml
|
||||
services:
|
||||
pg-bkup:
|
||||
mysql-bkup:
|
||||
# In production, pin your image tag to a specific release version instead of `latest`.
|
||||
# See available releases: https://github.com/jkaninda/mysql-bkup/releases
|
||||
image: jkaninda/mysql-bkup
|
||||
@@ -85,7 +85,7 @@ services:
|
||||
- DB_PORT=3306
|
||||
- DB_HOST=mysql
|
||||
- DB_NAME=foo
|
||||
- DB_USERNAME=bar
|
||||
- DB_USER=bar
|
||||
- DB_PASSWORD=password
|
||||
- TZ=Europe/Paris
|
||||
networks:
|
||||
@@ -105,7 +105,7 @@ You can schedule recurring backups using the `--cron-expression` or `-e` flag:
|
||||
docker run --rm --network network_name \
|
||||
-v $PWD/backup:/backup/ \
|
||||
-e "DB_HOST=hostname" \
|
||||
-e "DB_USERNAME=user" \
|
||||
-e "DB_USER=user" \
|
||||
-e "DB_PASSWORD=password" \
|
||||
jkaninda/mysql-bkup backup -d dbName --cron-expression "@every 15m"
|
||||
```
|
||||
@@ -147,7 +147,7 @@ spec:
|
||||
env:
|
||||
- name: DB_HOST
|
||||
value: "mysql"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "user"
|
||||
- name: DB_PASSWORD
|
||||
value: "password"
|
||||
@@ -187,7 +187,7 @@ spec:
|
||||
env:
|
||||
- name: DB_HOST
|
||||
value: "mysql"
|
||||
- name: DB_USERNAME
|
||||
- name: DB_USER
|
||||
value: "user"
|
||||
- name: DB_PASSWORD
|
||||
value: "password"
|
||||
|
||||
+238
-69
@@ -6,81 +6,250 @@ nav_order: 3
|
||||
|
||||
# Configuration Reference
|
||||
|
||||
MySQL backup, restore, and migration processes can be configured using **environment variables** or **CLI flags**.
|
||||
mysql-bkup is configured with **environment variables**, **CLI flags** and an optional **YAML file**. For every setting the order is: flag, environment variable, deprecated name, YAML `env:` section, default. Every secret (`DB_PASSWORD`, `AWS_SECRET_KEY`, `GPG_PASSPHRASE`, ...) can also be read from a file with `<NAME>_FILE`.
|
||||
|
||||
## CLI Utility Usage
|
||||
Run `mysql-bkup config` to see the resolved configuration and where each value comes from (secrets redacted).
|
||||
|
||||
The `mysql-bkup` CLI provides commands and options to manage MySQL backups efficiently.
|
||||
## Commands
|
||||
|
||||
| Option | Short Flag | Description |
|
||||
|-------------------------|------------|-----------------------------------------------------------------------------------------|
|
||||
| `mysql-bkup` | `bkup` | CLI tool for managing MySQL backups, restoration, and migration. |
|
||||
| `backup` | | Executes a backup operation. |
|
||||
| `restore` | | Restores a database from a backup file. |
|
||||
| `migrate` | | Migrates a database from one instance to another. |
|
||||
| `--storage` | `-s` | Specifies the storage type (`local`, `s3`, `ssh`, etc.). Default: `local`. |
|
||||
| `--file` | `-f` | Defines the backup file name for restoration. |
|
||||
| `--path` | | Sets the storage path (e.g., `/custom_path` for S3 or `/home/foo/backup` for SSH). |
|
||||
| `--config` | `-c` | Provides a configuration file for multi-database backups (e.g., `/backup/config.yaml`). |
|
||||
| `--dbname` | `-d` | Specifies the database name to back up or restore. |
|
||||
| `--port` | `-p` | Defines the database port. Default: `3306`. |
|
||||
| `--disable-compression` | | Disables compression for database backups. |
|
||||
| `--cron-expression` | `-e` | Schedules backups using a cron expression (e.g., `0 0 * * *` or `@daily`). |
|
||||
| `--all-databases` | `-a` | Backs up all databases separately (e.g., `backup --all-databases`). |
|
||||
| `--all-in-one` | `-A` | Backs up all databases in a single file (e.g., `backup --all-databases --single-file`). |
|
||||
| `--custom-name` | `` | Sets custom backup name for one time backup |
|
||||
| `--help` | `-h` | Displays the help message and exits. |
|
||||
| `--version` | `-V` | Shows version information and exits. |
|
||||
| Command | Description |
|
||||
|---|---|
|
||||
| `backup` | Back up now, or as a daemon with `--cron-expression` |
|
||||
| `restore` | Restore a backup: `-f FILE [-d DB]` |
|
||||
| `migrate` | Copy a database (or every database with `-a`) into `TARGET_DB_*`, streamed |
|
||||
| `verify` | Download, decrypt and decompress a backup and check its sha256 |
|
||||
| `list` | Backup history from the index (`--json`, `--storage-scan`) |
|
||||
| `delete` | Delete a backup, its sidecar and its index entry (`-f FILE`) |
|
||||
| `prune` | Apply the retention policy (`--dry-run`) |
|
||||
| `reconcile` | Rebuild the backup index (`--from-index OLD`, `--dry-run`) |
|
||||
| `config` | Show the resolved configuration; `--env-docs` prints the tables below |
|
||||
| `health` | Exit 0 when the daemon is healthy (Docker `HEALTHCHECK`) |
|
||||
| `version` | Print the version |
|
||||
|
||||
---
|
||||
## Flags
|
||||
|
||||
## Environment Variables
|
||||
| Flag | Short | Commands | Variable |
|
||||
|---|---|---|---|
|
||||
| `--dbname` | `-d` | all | `DB_NAME` |
|
||||
| `--storage` | `-s` | all | `STORAGE` (`local`, `s3`, `gcs`, `azure`, `sftp`/`ssh`/`remote`, `ftp`) |
|
||||
| `--path` (`--remote-path`) | `-P` | all | `REMOTE_PATH` |
|
||||
| `--file` | `-f` | restore, verify, delete | `FILE_NAME` |
|
||||
| `--config` | `-c` | backup, list, prune, config | `BACKUP_CONFIG_FILE` |
|
||||
| `--cron-expression` | `-e` | backup | `BACKUP_CRON_EXPRESSION` |
|
||||
| `--all-databases` | `-a` | backup, migrate | `BACKUP_ALL_DATABASES` |
|
||||
| `--all-in-one` | `-A` | backup | `BACKUP_ALL_IN_ONE` |
|
||||
| `--exclude-db` | | backup | `BACKUP_EXCLUDE_DATABASES` |
|
||||
| `--custom-name` | | backup | `BACKUP_CUSTOM_NAME` (fixed name, no timestamp) |
|
||||
| `--name` | `-n` | backup, list, prune | `BACKUP_NAME` |
|
||||
| `--disable-compression` | | backup | `DISABLE_COMPRESSION` |
|
||||
| `--compression` | | backup | `COMPRESSION` (`gzip`, `zstd`, `none`) |
|
||||
| `--encryption` | | backup | `ENCRYPTION` (`auto`, `none`, `gpg`, `age`) |
|
||||
| `--index-name` | | all | `BACKUP_INDEX_NAME` |
|
||||
| `--result-file` | | all | `RESULT_FILE` |
|
||||
| `--help` | `-h` | all | |
|
||||
| `--version` | `-v` | root | |
|
||||
|
||||
| Name | Requirement | Description |
|
||||
|--------------------------------|--------------------------------------|----------------------------------------------------------------------------|
|
||||
| `DB_PORT` | Optional (default: `3306`) | Database port number. |
|
||||
| `DB_HOST` | Required | Database host. |
|
||||
| `DB_NAME` | Optional (if provided via `-d` flag) | Database name. |
|
||||
| `DB_USERNAME` | Required | Database username. |
|
||||
| `DB_PASSWORD` | Required | Database password. |
|
||||
| `DB_SSL_CA` | Optional | Database client CA certificate file |
|
||||
| `DB_SSL_MODE` | Optional(`0 or 1`) default: `0` | Database client Enable CA validation |
|
||||
| `AWS_ACCESS_KEY` | Required for S3 storage | AWS S3 Access Key. |
|
||||
| `AWS_SECRET_KEY` | Required for S3 storage | AWS S3 Secret Key. |
|
||||
| `AWS_BUCKET_NAME` | Required for S3 storage | AWS S3 Bucket Name. |
|
||||
| `AWS_REGION` | Required for S3 storage | AWS Region. |
|
||||
| `AWS_DISABLE_SSL` | Optional | Disable SSL for S3 storage. |
|
||||
| `AWS_FORCE_PATH_STYLE` | Optional | Force path-style access for S3 storage. |
|
||||
| `FILE_NAME` | Optional (if provided via `--file`) | File name for restoration (e.g., `.sql`, `.sql.gz`). |
|
||||
| `GPG_PASSPHRASE` | Optional | GPG passphrase for encrypting/decrypting backups. |
|
||||
| `GPG_PUBLIC_KEY` | Optional | GPG public key for encrypting backups (e.g., `/config/public_key.asc`). |
|
||||
| `BACKUP_CRON_EXPRESSION` | Optional (flag `-e`) | Cron expression for scheduled backups. |
|
||||
| `BACKUP_RETENTION_DAYS` | Optional | Delete backups older than the specified number of days. |
|
||||
| `BACKUP_CONFIG_FILE` | Optional (flag `-c`) | Configuration file for multi database backup. (e.g: `/backup/config.yaml`) |
|
||||
| `SSH_HOST` | Required for SSH storage | SSH remote hostname or IP. |
|
||||
| `SSH_USER` | Required for SSH storage | SSH remote username. |
|
||||
| `SSH_PASSWORD` | Optional | SSH remote user's password. |
|
||||
| `SSH_IDENTIFY_FILE` | Optional | SSH remote user's private key. |
|
||||
| `SSH_PORT` | Optional (default: `22`) | SSH remote server port. |
|
||||
| `REMOTE_PATH` | Required for SSH/FTP storage | Remote path (e.g., `/home/toto/backup`). |
|
||||
| `FTP_HOST` | Required for FTP storage | FTP hostname. |
|
||||
| `FTP_PORT` | Optional (default: `21`) | FTP server port. |
|
||||
| `FTP_USER` | Required for FTP storage | FTP username. |
|
||||
| `FTP_PASSWORD` | Required for FTP storage | FTP user password. |
|
||||
| `TARGET_DB_HOST` | Required for migration | Target database host. |
|
||||
| `TARGET_DB_PORT` | Optional (default: `5432`) | Target database port. |
|
||||
| `TARGET_DB_NAME` | Required for migration | Target database name. |
|
||||
| `TARGET_DB_USERNAME` | Required for migration | Target database username. |
|
||||
| `TARGET_DB_PASSWORD` | Required for migration | Target database password. |
|
||||
| `TARGET_DB_URL` | Optional | Target database URL in JDBC URI format. |
|
||||
| `TG_TOKEN` | Required for Telegram notifications | Telegram token (`BOT-ID:BOT-TOKEN`). |
|
||||
| `TG_CHAT_ID` | Required for Telegram notifications | Telegram Chat ID. |
|
||||
| `TZ` | Optional | Time zone for scheduling. |
|
||||
| `AZURE_STORAGE_CONTAINER_NAME` | Required for Azure Blob Storage | Azure storage container name. |
|
||||
| `AZURE_STORAGE_ACCOUNT_NAME` | Required for Azure Blob Storage | Azure storage account name. |
|
||||
| `AZURE_STORAGE_ACCOUNT_KEY` | Required for Azure Blob Storage | Azure storage account key. |
|
||||
## Exit codes
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| `0` | success |
|
||||
| `1` | failure |
|
||||
| `2` | configuration error |
|
||||
| `3` | database connection or authentication |
|
||||
| `4` | storage |
|
||||
| `5` | encryption or decryption |
|
||||
|
||||
## Environment variables
|
||||
|
||||
### storage
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `STORAGE` | `--storage` `-s` | `local` | | Storage backend: local, s3, gcs, azure, sftp (ssh, remote), ftp (local, s3, gcs, azure, sftp, ssh, ftp, remote) |
|
||||
| `STORAGE_PATH` | | `/backup` | | Root directory of the local storage backend |
|
||||
| `REMOTE_PATH` | `--path` `-P` | | `AWS_S3_PATH` (deprecated), `S3_PATH` (deprecated) | Folder inside the storage backend (without file name) |
|
||||
|
||||
### general
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `BACKUP_NAME` | `--name` `-n` | | | Base name of the artifact; a timestamp is appended (default: the database or volume name) |
|
||||
| `BACKUP_CUSTOM_NAME` | `--custom-name` | | | Fixed artifact name without timestamp (overwrites the previous one) |
|
||||
| `FILE_NAME` | `--file` `-f` | | | Backup file to restore, verify or delete |
|
||||
| `BACKUP_CONFIG_FILE` | `--config` `-c` | | | YAML configuration file (default /config/config.yaml if present) |
|
||||
| `BACKUP_TIMEOUT` | | | | Abort a backup after this duration (0 = none) |
|
||||
| `RESTORE_TIMEOUT` | | | | Abort a restore after this duration (0 = none) |
|
||||
| `TMP_DIR` | | `/tmp/backup` | `BACKUP_TMP_DIR` (deprecated) | Scratch directory (only for secret files and disk-based modes) |
|
||||
| `COMPRESSION` | `--compression` | `gzip` | | Compression: gzip, zstd or none (gzip, zstd, none) |
|
||||
| `COMPRESSION_LEVEL` | | | | Compression level (0 = default) |
|
||||
| `DISABLE_COMPRESSION` | `--disable-compression` | | | Same as COMPRESSION=none |
|
||||
| `RESTORE_VERIFY_FIRST` | `--verify-first` | `auto` | | Verify the whole artifact before restoring (auto: on for GPG-encrypted volumes) (auto, true, false) |
|
||||
| `LOG_LEVEL` | | `info` | | Log level (debug, info, warn, error) |
|
||||
| `LOG_FORMAT` | | `text` | | Log format (text, json) |
|
||||
| `TIME_FORMAT` | | `2006-01-02 15:04:05` | | Time layout used in notifications |
|
||||
| `RESULT_FILE` | `--result-file` | | | Write the run result as JSON to this path |
|
||||
|
||||
### scheduling
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `BACKUP_CRON_EXPRESSION` | `--cron-expression` `-e` | | | Run as a daemon on this cron schedule (5 or 6 fields, @daily, @every 6h) |
|
||||
| `BACKUP_RUN_ON_START` | | | | In daemon mode, run one backup immediately at start |
|
||||
| `BACKUP_RESCUE_MODE` | | | | Deprecated: runs always continue with the remaining targets after a failure |
|
||||
| `METRICS_ADDR` | | | | Serve Prometheus /metrics and /healthz on this address in daemon mode (e.g. :9090) |
|
||||
| `HEALTH_FILE` | | `/tmp/bkup-health.json` | | State file read by the health command |
|
||||
| `HEALTH_MAX_AGE` | | | | health fails when the last success is older than this (default: 2x the schedule interval) |
|
||||
|
||||
### retention
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `BACKUP_RETENTION_DAYS` | | `0` | | Delete backups older than N days (0 = keep) |
|
||||
| `BACKUP_RETENTION_COUNT` | | `0` | | Keep only the newest N backups (0 = no limit) |
|
||||
| `BACKUP_RETENTION_ALLOW_ROOT` | | | | Allow pattern-based retention at the storage root (index disabled) |
|
||||
| `BACKUP_RETENTION_ADOPT_LEGACY` | | | | Import pre-index backups matching this tool's naming into the index |
|
||||
| `BACKUP_INDEX` | | `true` | | Keep the backup index file and per-backup .meta.json sidecars |
|
||||
| `BACKUP_INDEX_NAME` | `--index-name` | `{tool}-metadata.json` | | Index file name; placeholders {tool} {engine} {name} {db} |
|
||||
|
||||
### storage-s3
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `AWS_S3_ENDPOINT` | | | `S3_ENDPOINT` (deprecated), `AWS_ENDPOINT_URL_S3`, `AWS_ENDPOINT_URL` | S3 endpoint (empty = AWS) |
|
||||
| `AWS_S3_BUCKET_NAME` | | | `BUCKET_NAME` (deprecated), `AWS_BUCKET_NAME` (deprecated) | S3 bucket |
|
||||
| `AWS_REGION` | | `us-east-1` | `AWS_DEFAULT_REGION` | S3 region |
|
||||
| `AWS_ACCESS_KEY` | | | `ACCESS_KEY` (deprecated), `AWS_ACCESS_KEY_ID` | S3 access key (empty = default AWS credential chain) |
|
||||
| `AWS_SECRET_KEY` | | | `SECRET_KEY` (deprecated), `AWS_SECRET_ACCESS_KEY`, `AWS_SECRET_KEY_FILE` | S3 secret key |
|
||||
| `AWS_SESSION_TOKEN` | | | `AWS_SESSION_TOKEN_FILE` | S3 session token |
|
||||
| `AWS_PROFILE` | | | | AWS shared config profile |
|
||||
| `AWS_DISABLE_SSL` | | | | Use http for the S3 endpoint |
|
||||
| `AWS_FORCE_PATH_STYLE` | | | | Path-style S3 addressing (MinIO) |
|
||||
| `S3_STORAGE_CLASS` | | | | S3 storage class |
|
||||
| `S3_SSE` | | | | Server-side encryption: AES256 or aws:kms (aes256, aws:kms) |
|
||||
| `S3_SSE_KMS_KEY_ID` | | | | KMS key id for aws:kms |
|
||||
| `S3_PART_SIZE` | | `64MiB` | | Multipart part size; max object size is 10000 x this |
|
||||
| `S3_UPLOAD_CONCURRENCY` | | `2` | | Parallel part uploads (memory = part size x this) |
|
||||
|
||||
### storage-sftp
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `SFTP_HOST` | | | | SFTP host |
|
||||
| `SFTP_PORT` | | `22` | | SFTP port |
|
||||
| `SFTP_USER` | | | | SFTP user |
|
||||
| `SFTP_PASSWORD` | | | `SFTP_PASSWORD_FILE` | SFTP password |
|
||||
| `SFTP_IDENTITY_FILE` | | | | SFTP private key file (the SSH agent is also used when SSH_AUTH_SOCK is set) |
|
||||
| `SFTP_IDENTITY_PASSPHRASE` | | | `SFTP_IDENTITY_PASSPHRASE_FILE` | Passphrase of the SFTP private key |
|
||||
| `SFTP_KNOWN_HOSTS` | | | | known_hosts file used to verify the SFTP host key (default ~/.ssh/known_hosts) |
|
||||
| `SFTP_INSECURE_IGNORE_HOST_KEY` | | | | Do not verify the SFTP host key (insecure) |
|
||||
|
||||
### storage-ftp
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `FTP_HOST` | | | `FTP_HOST_NAME` (deprecated) | FTP host |
|
||||
| `FTP_PORT` | | `21` | | FTP port |
|
||||
| `FTP_USER` | | | | FTP user |
|
||||
| `FTP_PASSWORD` | | | `FTP_PASSWORD_FILE` | FTP password |
|
||||
| `FTP_TLS` | | | | Use explicit FTPS |
|
||||
|
||||
### storage-azure
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `AZURE_STORAGE_ACCOUNT_NAME` | | | | Azure storage account |
|
||||
| `AZURE_STORAGE_ACCOUNT_KEY` | | | `AZURE_STORAGE_ACCOUNT_KEY_FILE` | Azure storage account key |
|
||||
| `AZURE_STORAGE_CONTAINER_NAME` | | | | Azure blob container |
|
||||
| `AZURE_STORAGE_ENDPOINT` | | | | Azure blob endpoint (Azurite, sovereign clouds) |
|
||||
| `AZURE_STORAGE_SAS_TOKEN` | | | `AZURE_STORAGE_SAS_TOKEN_FILE` | Azure SAS token (instead of the account key) |
|
||||
|
||||
### storage-gcs
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `GCS_BUCKET` | | | | GCS bucket |
|
||||
| `GCS_CREDENTIALS_FILE` | | | `GOOGLE_APPLICATION_CREDENTIALS` | GCS service account JSON file |
|
||||
| `GCS_CREDENTIALS_JSON` | | | `GCS_CREDENTIALS_JSON_FILE` | GCS service account JSON content |
|
||||
| `GCS_ENDPOINT` | | | | GCS endpoint (emulators) |
|
||||
|
||||
### encryption
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `ENCRYPTION` | `--encryption` | `auto` | | auto (from the keys set), none, gpg or age (auto, none, gpg, age) |
|
||||
| `ENCRYPTION_ARMOR` | | | | ASCII-armor encrypted output |
|
||||
| `GPG_PASSPHRASE` | | | `GPG_PASSPHRASE_FILE` | GPG symmetric passphrase |
|
||||
| `GPG_PUBLIC_KEY` | | | | GPG public key(s): file path or inline key (default /config/gnupg/public_key.{asc,gpg}) |
|
||||
| `GPG_PRIVATE_KEY` | | | `GPG_PRIVATE_KEY_FILE` | GPG private key for restore: file path or inline key (default /config/gnupg/private_key.{asc,gpg}) |
|
||||
| `GPG_PRIVATE_KEY_PASSPHRASE` | | | `GPG_PRIVATE_KEY_PASSPHRASE_FILE` | Passphrase of the GPG private key (default GPG_PASSPHRASE) |
|
||||
| `AGE_RECIPIENTS` | | | | age recipients (age1..., ssh-ed25519 ...), comma or newline separated |
|
||||
| `AGE_RECIPIENTS_FILE` | | | | File with age recipients |
|
||||
| `AGE_PASSPHRASE` | | | `AGE_PASSPHRASE_FILE` | age passphrase (scrypt) |
|
||||
| `AGE_IDENTITY` | | | `AGE_IDENTITY_FILE` | age identity (AGE-SECRET-KEY-...) for restore |
|
||||
| `AGE_IDENTITY_FILE` | | | | age identity file for restore (default /config/age/identity.txt) |
|
||||
|
||||
### notifications
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `MAIL_HOST` | | | | SMTP host |
|
||||
| `MAIL_PORT` | | `587` | | SMTP port |
|
||||
| `MAIL_USERNAME` | | | | SMTP user |
|
||||
| `MAIL_PASSWORD` | | | `MAIL_PASSWORD_FILE` | SMTP password |
|
||||
| `MAIL_FROM` | | | | Sender address |
|
||||
| `MAIL_TO` | | | | Recipients, comma separated |
|
||||
| `MAIL_SKIP_TLS` | | | | Deprecated: true skips certificate verification (use MAIL_INSECURE_SKIP_VERIFY) |
|
||||
| `MAIL_INSECURE_SKIP_VERIFY` | | | | Skip SMTP certificate verification |
|
||||
| `MAIL_TLS` | | `starttls` | | SMTP TLS mode (starttls, tls, none) |
|
||||
| `TG_TOKEN` | | | `TELEGRAM_BOT_TOKEN`, `TG_TOKEN_FILE` | Telegram bot token |
|
||||
| `TG_CHAT_ID` | | | `TELEGRAM_CHAT_ID` | Telegram chat id |
|
||||
| `POSTA_URL` | | | | Posta base URL (https://posta.example.com) for email notifications through Posta |
|
||||
| `POSTA_API_KEY` | | | `POSTA_API_KEY_FILE` | Posta API key with the send scope |
|
||||
| `POSTA_WORKSPACE_ID` | | | | Posta workspace id (only for account-wide API keys) |
|
||||
| `POSTA_FROM` | | | | Posta sender address (default MAIL_FROM) |
|
||||
| `POSTA_TO` | | | | Posta recipients, comma separated (default MAIL_TO) |
|
||||
| `POSTA_TEMPLATE` | | | | Posta template name for notifications; the event fields are passed as template data (default: built-in HTML) |
|
||||
| `POSTA_TEMPLATE_ERROR` | | | | Posta template name for failures (default POSTA_TEMPLATE) |
|
||||
| `POSTA_LANGUAGE` | | | | Posta template language |
|
||||
| `WEBHOOK_URL` | | | | Generic JSON webhook URL |
|
||||
| `WEBHOOK_SECRET` | | | `WEBHOOK_SECRET_FILE` | HMAC-SHA256 secret for the X-Bkup-Signature header |
|
||||
| `SLACK_WEBHOOK_URL` | | | `SLACK_WEBHOOK_URL_FILE` | Slack incoming webhook URL |
|
||||
| `DISCORD_WEBHOOK_URL` | | | `DISCORD_WEBHOOK_URL_FILE` | Discord webhook URL |
|
||||
| `NTFY_URL` | | | | ntfy topic URL |
|
||||
| `NTFY_TOKEN` | | | `NTFY_TOKEN_FILE` | ntfy access token |
|
||||
| `NOTIFY_ON` | | `always` | | When to notify (always, failure, success, never) |
|
||||
| `HEALTHCHECK_PING_URL` | | | | healthchecks.io-style URL pinged on start (/start), success and failure (/fail) |
|
||||
| `BACKUP_REFERENCE` | | | | Free text included in notifications |
|
||||
| `TEMPLATES_DIR` | | `/config/templates` | | Directory with custom notification templates |
|
||||
|
||||
### database
|
||||
|
||||
| Variable | Flag | Default | Fallbacks | Description |
|
||||
|---|---|---|---|---|
|
||||
| `DB_URL` | | | `DB_URL_FILE` | Connection URL mysql://user:password@host:port/database?ssl-mode=required (overrides host, port, user, password) |
|
||||
| `DB_HOST` | | `localhost` | | Database host (a path starting with / is a Unix socket) |
|
||||
| `DB_PORT` | | `3306` | | Database port |
|
||||
| `DB_NAME` | `--dbname` `-d` | | | Database name |
|
||||
| `DB_USER` | | | `DB_USERNAME` | Database user |
|
||||
| `DB_PASSWORD` | | | `DB_PASSWORD_FILE` | Database password |
|
||||
| `DB_SSL_MODE` | | `disabled` | | TLS mode: disabled, preferred, required, verify_ca, verify_identity (1.x values 0 and 1 mean disabled and required) |
|
||||
| `DB_SSL_CA` | | `/etc/ssl/certs/ca-certificates.crt` | | CA certificate file, used with DB_SSL_MODE=verify_ca or verify_identity |
|
||||
| `DB_SSL_CERT` | | | | Client certificate file |
|
||||
| `DB_SSL_KEY` | | | | Client key file |
|
||||
| `BACKUP_ALL_DATABASES` | `--all-databases` `-a` | | | Back up every database, one file each |
|
||||
| `BACKUP_ALL_IN_ONE` | `--all-in-one` `-A` | | | Back up every database into a single file |
|
||||
| `BACKUP_EXCLUDE_DATABASES` | `--exclude-db` | | | Databases to skip with --all-databases, comma separated |
|
||||
| `TARGET_DB_URL` | | | `TARGET_DB_URL_FILE` | migrate: target connection URL |
|
||||
| `TARGET_DB_HOST` | | | | migrate: target host |
|
||||
| `TARGET_DB_PORT` | | `3306` | | migrate: target port |
|
||||
| `TARGET_DB_NAME` | | | | migrate: target database (default: the source name) |
|
||||
| `TARGET_DB_USER` | | | `TARGET_DB_USERNAME` | migrate: target user |
|
||||
| `TARGET_DB_PASSWORD` | | | `TARGET_DB_PASSWORD_FILE` | migrate: target password |
|
||||
| `TARGET_DB_SSL_MODE` | | `disabled` | | migrate: target TLS mode (same values as DB_SSL_MODE) |
|
||||
| `TARGET_DB_SSL_CA` | | `/etc/ssl/certs/ca-certificates.crt` | | migrate: target CA certificate file |
|
||||
| `MYSQL_CLIENT` | | `auto` | | Client tools: auto (mariadb-dump for MariaDB, Oracle mysqldump for MySQL), mysql or mariadb (auto, mysql, mariadb) |
|
||||
|
||||
---
|
||||
|
||||
## Scheduled Backups
|
||||
|
||||
@@ -91,7 +260,7 @@ The `mysql-bkup` CLI provides commands and options to manage MySQL backups effic
|
||||
|
||||
### Cron Syntax
|
||||
|
||||
The cron syntax consists of five fields:
|
||||
The cron syntax consists of five fields (a sixth, leading seconds field is also accepted):
|
||||
|
||||
```conf
|
||||
* * * * * command
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
title: Upgrading to v2
|
||||
layout: default
|
||||
nav_order: 5
|
||||
---
|
||||
|
||||
# Upgrading from 1.x to v2
|
||||
|
||||
v2 rebuilds mysql-bkup on a shared streaming core: dump, compression, encryption and upload run in one pipe, without temporary files, whatever the size of the database. Existing commands, flags and environment variables keep working.
|
||||
|
||||
|
||||
Everything that worked in 1.x keeps working; a few behaviours changed on purpose.
|
||||
|
||||
- **Image:** Debian 13 slim instead of Alpine, with Oracle's `mysqldump` for MySQL servers (correct MySQL 8 dumps) and `mariadb-dump` for MariaDB. `/bin/sh`, `/backup`, `/config`, the `backup`/`restore`/`migrate` shortcuts and the `bkup` alias are still there; `tini` is now PID 1.
|
||||
- **Renamed variables** are still read, with a deprecation warning: `S3_ENDPOINT` → `AWS_S3_ENDPOINT`, `BUCKET_NAME`/`AWS_BUCKET_NAME` → `AWS_S3_BUCKET_NAME`, `ACCESS_KEY`/`SECRET_KEY` → `AWS_ACCESS_KEY`/`AWS_SECRET_KEY`, `AWS_S3_PATH`/`S3_PATH` → `REMOTE_PATH`, `FTP_HOST_NAME` → `FTP_HOST`, `BACKUP_TMP_DIR` → `TMP_DIR`.
|
||||
- **`DB_USER` is the database user variable** (also `TARGET_DB_USER`). `DB_USERNAME` and `TARGET_DB_USERNAME` are still read without a warning, including the per-database forms (`DB_USERNAME_<NAME>`, `<NAME>_DB_USERNAME`).
|
||||
- **`DB_SSL_MODE`** takes MySQL's mode names; `0`/`1` still work.
|
||||
- **`STORAGE` and `STORAGE_PATH` env are honoured** (1.x always used the `--storage` default and `/backup`). Unknown storage names are an error instead of a silent local backup.
|
||||
- **SFTP variables are renamed, and the old names are no longer read:** `SSH_HOST`/`SSH_HOST_NAME` → `SFTP_HOST`, `SSH_PORT` → `SFTP_PORT`, `SSH_USER` → `SFTP_USER`, `SSH_PASSWORD` → `SFTP_PASSWORD`, `SSH_IDENTIFY_FILE`/`SSH_IDENTITY_FILE` → `SFTP_IDENTITY_FILE`, `SSH_REMOTE_PATH` → `REMOTE_PATH`.
|
||||
- **SFTP host keys are verified by default.** Set `SFTP_KNOWN_HOSTS=/path/known_hosts`, or `SFTP_INSECURE_IGNORE_HOST_KEY=true` to keep the 1.x behaviour.
|
||||
- **`MAIL_SKIP_TLS` is read with its documented meaning** (`true` skips certificate verification). 1.x had it inverted, so `MAIL_SKIP_TLS=false` (the value in `.env.example`) turned verification *off*. Use `MAIL_INSECURE_SKIP_VERIFY` from now on.
|
||||
- **Retention only deletes indexed backups** (see the [configuration reference](../reference/)). 1.x deleted every file older than N days in the folder. Set `BACKUP_RETENTION_ADOPT_LEGACY=true` once to let retention handle 1.x backups, or `BACKUP_INDEX=false` for name-based retention.
|
||||
- **Encryption fails closed**: a `GPG_PUBLIC_KEY` that cannot be read is an error (1.x uploaded plaintext). The env var now wins over the auto-detected `/config/gnupg` key.
|
||||
- **Errors no longer kill the daemon**: a failed run is logged, notified and retried at the next schedule; other databases of the same run continue (`BACKUP_RESCUE_MODE` is no longer needed).
|
||||
- **Restore** works from every storage (1.x had no SFTP restore), is case-insensitive about the storage name, and no longer goes through `sh -c`.
|
||||
- **`migrate`** streams directly into the target (no intermediate file), creates the target database if needed, and `TARGET_DB_NAME` defaults to the source name; `-a` migrates every database.
|
||||
- **Notification templates** are built in; custom ones still go in `/config/templates` (`TEMPLATES_DIR`).
|
||||
- **Version** is injected with `-ldflags "-X main.Version=..."` (was `utils.Version`).
|
||||
|
||||
|
||||
## Checklist
|
||||
|
||||
1. **SFTP users:** mount a `known_hosts` file and set `SFTP_KNOWN_HOSTS`, or set `SFTP_INSECURE_IGNORE_HOST_KEY=true`.
|
||||
2. **Email notifications with `MAIL_SKIP_TLS=false`:** certificate verification is now on, as the name always promised. Remove the variable, or set `MAIL_INSECURE_SKIP_VERIFY=true` if your SMTP server has a self-signed certificate.
|
||||
3. **Retention:** set `BACKUP_RETENTION_ADOPT_LEGACY=true` for one run if retention should also delete the backups made by 1.x.
|
||||
4. **Quoted values in Compose files:** in the list syntax (`- AWS_DISABLE_SSL="false"`) the quotes are part of the value. v2 validates booleans and rejects `"false"`; write `- AWS_DISABLE_SSL=false`.
|
||||
5. **`--user` runs:** `/backup` and `/tmp/backup` are writable by any user (sticky, like `/tmp`); mounted volumes need the usual permissions.
|
||||
6. **Restoring 1.x backups:** nothing to do. `.sql`, `.sql.gz`, `.sql.gpg` and `.sql.gz.gpg` files from 1.x restore as before.
|
||||
|
||||
## New in v2
|
||||
|
||||
- MySQL servers are dumped with Oracle `mysqldump` 8.4, MariaDB servers with `mariadb-dump` 11.8 (flavour detected automatically).
|
||||
- `DB_URL` (`mysql://user:password@host:3306/db?ssl-mode=required`), TLS modes, client certificates.
|
||||
- age encryption, zstd compression, Google Cloud Storage, FTPS, extra storage copies.
|
||||
- `verify`, `list`, `delete`, `prune`, `reconcile`, `config`, `health` commands; backup index with sha256 per backup.
|
||||
- Slack, Discord, ntfy, webhook and healthcheck notifications; Prometheus metrics in daemon mode.
|
||||
- `migrate -a`, and migrate without an intermediate file.
|
||||
Reference in new issue
Block a user