jkaninda/mysql-bkup
1
1
Fork 0
mirror of https://github.com/jkaninda/mysql-bkup.git synced 2025-12-06 05:29:41 +01:00
Code Issues Actions Projects Releases Wiki Activity

doc: reviewed docs

Browse Source
This commit is contained in:
Jonas Kaninda
2025-01-13 14:56:08 +01:00
parent 0f28772659
commit 68322e6b9f
9 changed files with 653 additions and 394 deletions
Download Patch File Download Diff File Expand all files Collapse all files

144
docs/how-tos/backup-to-ssh.md
View File

@@ -1,91 +1,129 @@
---
title: Backup to SSH
title: Backup to SSH or SFTP
layout: default
parent: How Tos
nav_order: 3
---
# Backup to SSH remote server
# Backup to SFTP or SSH Remote Server
To store your backups on an `SFTP` or `SSH` remote server instead of the default storage, you can configure the backup process to use the `--storage ssh` or `--storage remote` option.
This section explains how to set up and configure SSH-based backups.
As described for s3 backup section, to change the storage of your backup and use SSH Remote server as storage. You need to add `--storage ssh` or `--storage remote`.
You need to add the full remote path by adding `--path /home/jkaninda/backups` flag or using `REMOTE_PATH` environment variable.
---
{: .note }
These environment variables are required for SSH backup `SSH_HOST`, `SSH_USER`, `SSH_REMOTE_PATH`, `SSH_IDENTIFY_FILE`, `SSH_PORT` or `SSH_PASSWORD` if you dont use a private key to access to your server.
Accessing the remote server using password is not recommended, use private key instead.
## Configuration Steps
```yml
1. **Specify the Storage Type**
Add the `--storage ssh` or `--storage remote` flag to your backup command.
2. **Set the Remote Path**
Define the full remote path where backups will be stored using the `--path` flag or the `REMOTE_PATH` environment variable.
Example: `--path /home/jkaninda/backups`.
3. **Required Environment Variables**
The following environment variables are mandatory for SSH-based backups:
- `SSH_HOST`: The hostname or IP address of the remote server.
- `SSH_USER`: The username for SSH authentication.
- `REMOTE_PATH`: The directory on the remote server where backups will be stored.
- `SSH_IDENTIFY_FILE`: The path to the private key file for SSH authentication.
- `SSH_PORT`: The SSH port (default is `22`).
- `SSH_PASSWORD`: (Optional) Use this only if you are not using a private key for authentication.
{: .note }
**Security Recommendation**: Using a private key (`SSH_IDENTIFY_FILE`) is strongly recommended over password-based authentication (`SSH_PASSWORD`) for better security.
---
## Example Configuration
Below is an example `docker-compose.yml` configuration for backing up to an SSH remote server:
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup --storage remote -d database
volumes:
- ./id_ed25519:/tmp/id_ed25519"
environment:
- DB_PORT=3306
- DB_HOST=mysql
#- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## SSH config
- SSH_HOST="hostname"
- SSH_PORT=22
- SSH_USER=user
- REMOTE_PATH=/home/jkaninda/backups
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
## We advise you to use a private jey instead of password
#- SSH_PASSWORD=password
mysql-bkup:
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup --storage remote -d database
volumes:
- ./id_ed25519:/tmp/id_ed25519
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## SSH Configuration
- SSH_HOST="hostname"
- SSH_PORT=22
- SSH_USER=user
- REMOTE_PATH=/home/jkaninda/backups
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
## Optional: Use password instead of private key (not recommended)
#- SSH_PASSWORD=password
# Ensure the mysql-bkup container is connected to the same network as your database
networks:
- web
# mysql-bkup container must be connected to the same network with your database
networks:
- web
networks:
web:
web:
```
---
### Recurring backups to SSH remote server
## Recurring Backups to SSH Remote Server
As explained above, you need just to add required environment variables and specify the storage type `--storage ssh`.
You can use `--cron-expression "* * * * *"` or `BACKUP_CRON_EXPRESSION=0 1 * * *` as described below.
To schedule recurring backups, you can use the `--cron-expression` flag or the `BACKUP_CRON_EXPRESSION` environment variable.
This allows you to define a cron schedule for automated backups.
```yml
### Example: Recurring Backup Configuration
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
mysql-bkup:
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup -d database --storage ssh --cron-expression "0 1 * * *"
command: backup -d database --storage ssh --cron-expression "@daily"
volumes:
- ./id_ed25519:/tmp/id_ed25519"
- ./id_ed25519:/tmp/id_ed25519
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_HOST=postgres
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## SSH config
## SSH Configuration
- SSH_HOST="hostname"
- SSH_PORT=22
- SSH_USER=user
- REMOTE_PATH=/home/jkaninda/backups
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
# - BACKUP_CRON_EXPRESSION=0 1 * * * # Optional
#Delete old backup created more than specified days ago
## Optional: Delete old backups after a specified number of days
#- BACKUP_RETENTION_DAYS=7
## We advise you to use a private jey instead of password
## Optional: Use password instead of private key (not recommended)
#- SSH_PASSWORD=password
# mysql-bkup container must be connected to the same network with your database
# Ensure the mysql-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
---
## Key Notes
- **Cron Expression**: Use the `--cron-expression` flag or `BACKUP_CRON_EXPRESSION` environment variable to define the backup schedule. For example, `0 1 * * *` runs the backup daily at 1:00 AM.
- **Backup Retention**: Optionally, use the `BACKUP_RETENTION_DAYS` environment variable to automatically delete backups older than a specified number of days.
- **Security**: Always prefer private key authentication (`SSH_IDENTIFY_FILE`) over password-based authentication (`SSH_PASSWORD`) for enhanced security.
---

363
docs/how-tos/deploy-on-kubernetes.md
View File

@@ -5,12 +5,17 @@ parent: How Tos
nav_order: 9
---
## Deploy on Kubernetes
# Deploy on Kubernetes
To deploy MySQL Backup on Kubernetes, you can use Job to backup or Restore your database.
For recurring backup you can use CronJob, you don't need to run it in scheduled mode. as described bellow.
To deploy MySQL Backup on Kubernetes, you can use a `Job` for one-time backups or restores, and a `CronJob` for recurring backups.
## Backup to S3 storage
Below are examples for different use cases.
---
## Backup Job to S3 Storage
This example demonstrates how to configure a Kubernetes `Job` to back up a MySQL database to an S3-compatible storage.
```yaml
apiVersion: batch/v1
@@ -21,50 +26,53 @@ spec:
template:
spec:
containers:
- name: mysql-bkup
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup --storage s3
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "dbname"
- name: DB_USERNAME
value: "username"
# Please use secret!
- name: DB_PASSWORD
value: ""
- name: AWS_S3_ENDPOINT
value: "https://s3.amazonaws.com"
- name: AWS_S3_BUCKET_NAME
value: "xxx"
- name: AWS_REGION
value: "us-west-2"
- name: AWS_ACCESS_KEY
value: "xxxx"
- name: AWS_SECRET_KEY
value: "xxxx"
- name: AWS_DISABLE_SSL
value: "false"
- name: AWS_FORCE_PATH_STYLE
value: "false"
- name: mysql-bkup
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup --storage s3
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: ""
- name: DB_USERNAME
value: ""
# Use Kubernetes Secrets for sensitive data like passwords
- name: DB_PASSWORD
value: ""
- name: AWS_S3_ENDPOINT
value: "https://s3.amazonaws.com"
- name: AWS_S3_BUCKET_NAME
value: "xxx"
- name: AWS_REGION
value: "us-west-2"
- name: AWS_ACCESS_KEY
value: "xxxx"
- name: AWS_SECRET_KEY
value: "xxxx"
- name: AWS_DISABLE_SSL
value: "false"
- name: AWS_FORCE_PATH_STYLE
value: "false"
restartPolicy: Never
```
## Backup Job to SSH remote server
---
## Backup Job to SSH Remote Server
This example demonstrates how to configure a Kubernetes `Job` to back up a MySQL database to an SSH remote server.
```yaml
apiVersion: batch/v1
@@ -77,15 +85,14 @@ spec:
spec:
containers:
- name: mysql-bkup
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup --storage ssh
- /bin/sh
- -c
- backup --storage ssh --disable-compression
resources:
limits:
memory: "128Mi"
@@ -98,8 +105,8 @@ spec:
- name: DB_NAME
value: "dbname"
- name: DB_USERNAME
value: "username"
# Please use secret!
value: "postgres"
# Use Kubernetes Secrets for sensitive data like passwords
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
@@ -112,14 +119,18 @@ spec:
value: "xxxx"
- name: SSH_REMOTE_PATH
value: "/home/toto/backup"
# Optional, required if you want to encrypt your backup
# Optional: Required if you want to encrypt your backup
- name: GPG_PASSPHRASE
value: "secure-passphrase"
value: "xxxx"
restartPolicy: Never
```
---
## Restore Job
This example demonstrates how to configure a Kubernetes `Job` to restore a MySQL database from a backup stored on an SSH remote server.
```yaml
apiVersion: batch/v1
kind: Job
@@ -131,48 +142,51 @@ spec:
spec:
containers:
- name: mysql-bkup
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup --storage ssh --file store_20231219_022941.sql.gz
- /bin/sh
- -c
- restore --storage ssh --file store_20231219_022941.sql.gz
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "dbname"
- name: DB_USERNAME
value: "username"
# Please use secret!
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "xxx"
- name: SSH_PORT
value: "22"
- name: SSH_USER
value: "xxx"
- name: SSH_PASSWORD
value: "xxxx"
- name: SSH_REMOTE_PATH
value: "/home/xxxx/backup"
# Optional, required if your backup was encrypted
#- name: GPG_PASSPHRASE
# value: "xxxx"
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "dbname"
- name: DB_USERNAME
value: "postgres"
# Use Kubernetes Secrets for sensitive data like passwords
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "xxx"
- name: SSH_PORT
value: "22"
- name: SSH_USER
value: "xxx"
- name: SSH_PASSWORD
value: "xxxx"
- name: SSH_REMOTE_PATH
value: "/home/toto/backup"
# Optional: Required if your backup was encrypted
#- name: GPG_PASSPHRASE
# value: "xxxx"
restartPolicy: Never
```
## Recurring backup
---
## Recurring Backup with CronJob
This example demonstrates how to configure a Kubernetes `CronJob` for recurring backups to an SSH remote server.
```yaml
apiVersion: batch/v1
@@ -187,51 +201,51 @@ spec:
spec:
containers:
- name: mysql-bkup
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- bkup
- backup
- --storage
- ssh
- --disable-compression
- backup --storage ssh --disable-compression
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "username"
- name: DB_USERNAME
value: "username"
# Please use secret!
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "xxx"
- name: SSH_PORT
value: "xxx"
- name: SSH_USER
value: "jkaninda"
- name: SSH_REMOTE_PATH
value: "/home/jkaninda/backup"
- name: SSH_PASSWORD
value: "password"
# Optional, required if you want to encrypt your backup
#- name: GPG_PASSPHRASE
# value: "xxx"
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "test"
- name: DB_USERNAME
value: "postgres"
# Use Kubernetes Secrets for sensitive data like passwords
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "192.168.1.16"
- name: SSH_PORT
value: "2222"
- name: SSH_USER
value: "jkaninda"
- name: SSH_REMOTE_PATH
value: "/config/backup"
- name: SSH_PASSWORD
value: "password"
# Optional: Required if you want to encrypt your backup
#- name: GPG_PASSPHRASE
# value: "xxx"
restartPolicy: Never
```
## Kubernetes Rootless
---
This image also supports Kubernetes security context, you can run it in Rootless environment.
It has been tested on Openshift, it works well.
## Kubernetes Rootless Deployment
This example demonstrates how to run the backup container in a rootless environment, suitable for platforms like OpenShift.
```yaml
apiVersion: batch/v1
@@ -249,53 +263,52 @@ spec:
runAsGroup: 3000
fsGroup: 2000
containers:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
- name: mysql-bkup
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- bkup
- backup
- --storage
- ssh
- --disable-compression
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "xxx"
- name: DB_USERNAME
value: "xxx"
# Please use secret!
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "xxx"
- name: SSH_PORT
value: "22"
- name: SSH_USER
value: "jkaninda"
- name: SSH_REMOTE_PATH
value: "/home/jkaninda/backup"
- name: SSH_PASSWORD
value: "password"
# Optional, required if you want to encrypt your backup
- name: mysql-bkup
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
- -c
- backup --storage ssh --disable-compression
resources:
limits:
memory: "128Mi"
cpu: "500m"
env:
- name: DB_PORT
value: "3306"
- name: DB_HOST
value: ""
- name: DB_NAME
value: "test"
- name: DB_USERNAME
value: "postgres"
# Use Kubernetes Secrets for sensitive data like passwords
- name: DB_PASSWORD
value: ""
- name: SSH_HOST_NAME
value: "192.168.1.16"
- name: SSH_PORT
value: "2222"
- name: SSH_USER
value: "jkaninda"
- name: SSH_REMOTE_PATH
value: "/config/backup"
- name: SSH_PASSWORD
value: "password"
# Optional: Required if you want to encrypt your backup
#- name: GPG_PASSPHRASE
# value: "xxx"
restartPolicy: OnFailure
```
## Migrate database
---
## Migrate Database
This example demonstrates how to configure a Kubernetes `Job` to migrate a MySQL database from one server to another.
```yaml
apiVersion: batch/v1
@@ -308,10 +321,9 @@ spec:
spec:
containers:
- name: mysql-bkup
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
command:
- /bin/sh
@@ -322,11 +334,11 @@ spec:
memory: "128Mi"
cpu: "500m"
env:
## Source Database
## Source Database
- name: DB_HOST
value: "mysql"
value: "postgres"
- name: DB_PORT
value: "3306"
value: "3306"
- name: DB_NAME
value: "dbname"
- name: DB_USERNAME
@@ -335,7 +347,7 @@ spec:
value: "password"
## Target Database
- name: TARGET_DB_HOST
value: "target-mysql"
value: "target-postgres"
- name: TARGET_DB_PORT
value: "3306"
- name: TARGET_DB_NAME
@@ -345,4 +357,13 @@ spec:
- name: TARGET_DB_PASSWORD
value: "password"
restartPolicy: Never
```
```
---
## Key Notes
- **Security**: Always use Kubernetes Secrets for sensitive data like passwords and access keys.
- **Resource Limits**: Adjust resource limits (`memory` and `cpu`) based on your workload requirements.
- **Cron Schedule**: Use standard cron expressions for scheduling recurring backups.
- **Rootless Deployment**: The image supports running in rootless environments, making it suitable for platforms like OpenShift.

101
docs/how-tos/encrypt-backup.md
View File

@@ -1,47 +1,38 @@
---
title: Encrypt backups
title: Encrypt backups using GPG
layout: default
parent: How Tos
nav_order: 8
---
# Encrypt backup
# Encrypt Backup
The image supports encrypting backups using one of two available methods: GPG with passphrase or GPG with a public key.
The image supports encrypting backups using GPG out of the box. In case a `GPG_PASSPHRASE` or `GPG_PUBLIC_KEY` environment variable is set, the backup archive will be encrypted using the given key and saved as a sql.gpg file instead or sql.gz.gpg.
The image supports encrypting backups using one of two methods: **GPG with a passphrase** or **GPG with a public key**. When a `GPG_PASSPHRASE` or `GPG_PUBLIC_KEY` environment variable is set, the backup archive will be encrypted and saved as a `.sql.gpg` or `.sql.gz.gpg` file.
{: .warning }
To restore an encrypted backup, you need to provide the same GPG passphrase used during backup process.
To restore an encrypted backup, you must provide the same GPG passphrase or private key used during the backup process.
- GPG home directory `/config/gnupg`
- Cipher algorithm `aes256`
---
{: .note }
The backup encrypted using `GPG passphrase` method can be restored automatically, no need to decrypt it before restoration.
Suppose you used a GPG public key during the backup process. In that case, you need to decrypt your backup before restoration because decryption using a `GPG private` key is not fully supported.
## Key Features
To decrypt manually, you need to install `gnupg`
- **Cipher Algorithm**: `aes256`
- **Automatic Restoration**: Backups encrypted with a GPG passphrase can be restored automatically without manual decryption.
- **Manual Decryption**: Backups encrypted with a GPG public key require manual decryption before restoration.
```shell
gpg --batch --passphrase "my-passphrase" \
--output database_20240730_044201.sql.gz \
--decrypt database_20240730_044201.sql.gz.gpg
```
Using your private key
---
```shell
gpg --output database_20240730_044201.sql.gz --decrypt database_20240730_044201.sql.gz.gpg
```
## Using GPG passphrase
## Using GPG Passphrase
```yml
To encrypt backups using a GPG passphrase, set the `GPG_PASSPHRASE` environment variable. The backup will be encrypted and can be restored automatically.
### Example Configuration
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup -d database
@@ -55,26 +46,34 @@ services:
- DB_PASSWORD=password
## Required to encrypt backup
- GPG_PASSPHRASE=my-secure-passphrase
# mysql-bkup container must be connected to the same network with your database
# Ensure the pg-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
---
## Using GPG Public Key
```yml
To encrypt backups using a GPG public key, set the `GPG_PUBLIC_KEY` environment variable to the path of your public key file. Backups encrypted with a public key require manual decryption before restoration.
### Example Configuration
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup -d database
volumes:
- ./backup:/backup
- ./public_key.asc:/config/public_key.asc
environment:
- DB_PORT=3306
- DB_HOST=mysql
@@ -83,9 +82,39 @@ services:
- DB_PASSWORD=password
## Required to encrypt backup
- GPG_PUBLIC_KEY=/config/public_key.asc
# mysql-bkup container must be connected to the same network with your database
# Ensure the pg-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
```
---
## Manual Decryption
If you encrypted your backup using a GPG public key, you must manually decrypt it before restoration. Use the `gnupg` tool for decryption.
### Decrypt Using a Passphrase
```bash
gpg --batch --passphrase "my-passphrase" \
--output database_20240730_044201.sql.gz \
--decrypt database_20240730_044201.sql.gz.gpg
```
### Decrypt Using a Private Key
```bash
gpg --output database_20240730_044201.sql.gz \
--decrypt database_20240730_044201.sql.gz.gpg
```
---
## Key Notes
- **Automatic Restoration**: Backups encrypted with a GPG passphrase can be restored directly without manual decryption.
- **Manual Decryption**: Backups encrypted with a GPG public key require manual decryption using the corresponding private key.
- **Security**: Always keep your GPG passphrase and private key secure. Use Kubernetes Secrets or other secure methods to manage sensitive data.

82
docs/how-tos/migrate.md
View File

@@ -5,76 +5,102 @@ parent: How Tos
nav_order: 10
---
# Migrate database
# Migrate Database
To migrate the database, you need to add `migrate` command.
To migrate a MySQL database from a source to a target database, you can use the `migrate` command. This feature simplifies the process by combining the backup and restore operations into a single step.
{: .note }
The Mysql backup has another great feature: migrating your database from a source database to a target.
As you know, to restore a database from a source to a target database, you need 2 operations: which is to start by backing up the source database and then restoring the source backed database to the target database.
Instead of proceeding like that, you can use the integrated feature `(migrate)`, which will help you migrate your database by doing only one operation.
The `migrate` command eliminates the need for separate backup and restore operations. It directly transfers data from the source database to the target database.
{: .warning }
The `migrate` operation is irreversible, please backup your target database before this action.
The `migrate` operation is **irreversible**. Always back up your target database before performing this action.
### Docker compose
```yml
---
## Configuration Steps
1. **Source Database**: Provide connection details for the source database.
2. **Target Database**: Provide connection details for the target database.
3. **Run the Migration**: Use the `migrate` command to initiate the migration.
---
## Example: Docker Compose Configuration
Below is an example `docker-compose.yml` configuration for migrating a database:
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysqlbkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: migrate
volumes:
- ./backup:/backup
environment:
## Source database
## Source Database
- DB_PORT=3306
- DB_HOST=mysql
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## Target database
- TARGET_DB_HOST=target-mysql
## Target Database
- TARGET_DB_HOST=target-postgres
- TARGET_DB_PORT=3306
- TARGET_DB_NAME=dbname
- TARGET_DB_USERNAME=username
- TARGET_DB_PASSWORD=password
# mysql-bkup container must be connected to the same network with your database
# Ensure the mysql-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
---
### Migrate database using Docker CLI
## Migrate Database Using Docker CLI
You can also run the migration directly using the Docker CLI. Below is an example:
```
## Source database
DB_HOST=mysql
### Environment Variables
Save your source and target database connection details in an environment file (e.g., `your-env`):
```bash
## Source Database
DB_HOST=postgres
DB_PORT=3306
DB_NAME=dbname
DB_USERNAME=username
DB_PASSWORD=password
## Taget database
TARGET_DB_HOST=target-mysql
## Target Database
TARGET_DB_HOST=target-postgres
TARGET_DB_PORT=3306
TARGET_DB_NAME=dbname
TARGET_DB_USERNAME=username
TARGET_DB_PASSWORD=password
```
```shell
docker run --rm --network your_network_name \
--env-file your-env
-v $PWD/backup:/backup/ \
jkaninda/mysql-bkup migrate
### Run the Migration
```bash
docker run --rm --network your_network_name \
--env-file your-env \
-v $PWD/backup:/backup/ \
jkaninda/pg-bkup migrate
```
---
## Key Notes
- **Irreversible Operation**: The `migrate` command directly transfers data from the source to the target database. Ensure you have a backup of the target database before proceeding.
- **Network Configuration**: Ensure the `mysql-bkup` container is connected to the same network as your source and target databases.

69
docs/how-tos/mutli-backup.md
View File

@@ -1,63 +1,96 @@
---
title: Run multiple backup schedules in the same container
title: Run multiple database backup schedules in the same container
layout: default
parent: How Tos
nav_order: 11
---
Multiple backup schedules with different configuration can be configured by mounting a configuration file into `/config/config.yaml` `/config/config.yml` or by defining an environment variable `BACKUP_CONFIG_FILE=/backup/config.yaml`.
## Configuration file
# Multiple Backup Schedules
You can configure multiple backup schedules with different configurations by using a configuration file.
This file can be mounted into the container at `/config/config.yaml`, `/config/config.yml`, or specified via the `BACKUP_CONFIG_FILE` environment variable.
---
## Configuration File
The configuration file allows you to define multiple databases and their respective backup settings.
Below is an example configuration file:
```yaml
#cronExpression: "@every 20m" //Optional for scheduled backups
cronExpression: ""
# Optional: Define a global cron expression for scheduled backups
# cronExpression: "@every 20m"
cronExpression: ""
databases:
- host: mysql1
port: 3306
name: database1
user: database1
password: password
path: /s3-path/database1 #For SSH or FTP you need to define the full path (/home/toto/backup/)
path: /s3-path/database1 # For SSH or FTP, define the full path (e.g., /home/toto/backup/)
- host: mysql2
port: 3306
name: lldap
user: lldap
password: password
path: /s3-path/lldap #For SSH or FTP you need to define the full path (/home/toto/backup/)
path: /s3-path/lldap # For SSH or FTP, define the full path (e.g., /home/toto/backup/)
- host: mysql3
port: 3306
name: keycloak
user: keycloak
password: password
path: /s3-path/keycloak #For SSH or FTP you need to define the full path (/home/toto/backup/)
path: /s3-path/keycloak # For SSH or FTP, define the full path (e.g., /home/toto/backup/)
- host: mysql4
port: 3306
name: joplin
user: joplin
password: password
path: /s3-path/joplin #For SSH or FTP you need to define the full path (/home/toto/backup/)
path: /s3-path/joplin # For SSH or FTP, define the full path (e.g., /home/toto/backup/)
```
## Docker compose file
---
## Docker Compose Configuration
To use the configuration file in a Docker Compose setup, mount the file and specify its path using the `BACKUP_CONFIG_FILE` environment variable.
### Example: Docker Compose File
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup
volumes:
- ./backup:/backup
- ./backup:/backup # Mount the backup directory
- ./config.yaml:/backup/config.yaml # Mount the configuration file
environment:
## Multi backup config file
## Specify the path to the configuration file
- BACKUP_CONFIG_FILE=/backup/config.yaml
# mysql-bkup container must be connected to the same network with your database
# Ensure the pg-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
```
---
## Key Notes
- **Global Cron Expression**: You can define a global `cronExpression` in the configuration file to schedule backups for all databases. If omitted, backups will run immediately.
- **Database-Specific Paths**: For SSH or FTP storage, ensure the `path` field contains the full remote path (e.g., `/home/toto/backup/`).
- **Environment Variables**: Use the `BACKUP_CONFIG_FILE` environment variable to specify the path to the configuration file.
- **Security**: Avoid hardcoding sensitive information like passwords in the configuration file. Use environment variables or secrets management tools instead.

115
docs/how-tos/receive-notification.md
View File

@@ -4,10 +4,20 @@ layout: default
parent: How Tos
nav_order: 12
---
Send Email or Telegram notifications on successfully or failed backup.
### Email
To send out email notifications on failed or successfully backup runs, provide SMTP credentials, a sender and a recipient:
# Receive Notifications
You can configure the system to send email or Telegram notifications when a backup succeeds or fails.
This section explains how to set up and customize notifications.
---
## Email Notifications
To send email notifications, provide SMTP credentials, a sender address, and recipient addresses. Notifications will be sent for both successful and failed backup runs.
### Example: Email Notification Configuration
```yaml
services:
@@ -23,25 +33,33 @@ services:
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
- MAIL_HOST=
## SMTP Configuration
- MAIL_HOST=smtp.example.com
- MAIL_PORT=587
- MAIL_USERNAME=
- MAIL_PASSWORD=!
- MAIL_USERNAME=your-email@example.com
- MAIL_PASSWORD=your-email-password
- MAIL_FROM=Backup Jobs <backup@example.com>
## Multiple recipients separated by a comma
- MAIL_TO=me@example.com,team@example.com,manager@example.com
- MAIL_SKIP_TLS=false
## Time format for notification
## Time format for notifications
- TIME_FORMAT=2006-01-02 at 15:04:05
## Backup reference, in case you want to identify every backup instance
## Backup reference (e.g., database/cluster name or server name)
- BACKUP_REFERENCE=database/Paris cluster
networks:
- web
networks:
web:
```
### Telegram
---
## Telegram Notifications
To send Telegram notifications, provide your bot token and chat ID. Notifications will be sent for both successful and failed backup runs.
### Example: Telegram Notification Configuration
```yaml
services:
@@ -57,41 +75,49 @@ services:
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## Telegram Configuration
- TG_TOKEN=[BOT ID]:[BOT TOKEN]
- TG_CHAT_ID=
## Time format for notification
- TG_CHAT_ID=your-chat-id
## Time format for notifications
- TIME_FORMAT=2006-01-02 at 15:04:05
## Backup reference, in case you want to identify every backup instance
## Backup reference (e.g., database/cluster name or server name)
- BACKUP_REFERENCE=database/Paris cluster
networks:
- web
networks:
web:
```
### Customize notifications
---
The title and body of the notifications can be tailored to your needs using Go templates.
Template sources must be mounted inside the container in /config/templates:
## Customize Notifications
- email.tmpl: Email notification template
- telegram.tmpl: Telegram notification template
- email-error.tmpl: Error notification template
- telegram-error.tmpl: Error notification template
You can customize the title and body of notifications using Go templates. Template files must be mounted inside the container at `/config/templates`. The following templates are supported:
### Data
- `email.tmpl`: Template for successful email notifications.
- `telegram.tmpl`: Template for successful Telegram notifications.
- `email-error.tmpl`: Template for failed email notifications.
- `telegram-error.tmpl`: Template for failed Telegram notifications.
Here is a list of all data passed to the template:
- `Database` : Database name
- `StartTime`: Backup start time process
- `EndTime`: Backup start time process
- `Storage`: Backup storage
- `BackupLocation`: Backup location
- `BackupSize`: Backup size
- `BackupReference`: Backup reference(eg: database/cluster name or server name)
### Template Data
> email.template:
The following data is passed to the templates:
- `Database`: Database name.
- `StartTime`: Backup start time.
- `EndTime`: Backup end time.
- `Storage`: Backup storage type (e.g., local, S3, SSH).
- `BackupLocation`: Backup file location.
- `BackupSize`: Backup file size in bytes.
- `BackupReference`: Backup reference (e.g., database/cluster name or server name).
- `Error`: Error message (only for error templates).
---
### Example Templates
#### `email.tmpl` (Successful Backup)
```html
<h2>Hi,</h2>
@@ -104,29 +130,29 @@ Here is a list of all data passed to the template:
<li>Backup Storage: {{.Storage}}</li>
<li>Backup Location: {{.BackupLocation}}</li>
<li>Backup Size: {{.BackupSize}} bytes</li>
<li>Backup Reference: {{.BackupReference}} </li>
<li>Backup Reference: {{.BackupReference}}</li>
</ul>
<p>Best regards,</p>
```
> telegram.template
#### `telegram.tmpl` (Successful Backup)
```html
✅ Database Backup Notification – {{.Database}}
✅ Database Backup Notification – {{.Database}}
Hi,
Backup of the {{.Database}} database has been successfully completed on {{.EndTime}}.
Backup Details:
- Database Name: {{.Database}}
- Backup Start Time: {{.StartTime}}
- Backup EndTime: {{.EndTime}}
- Backup End Time: {{.EndTime}}
- Backup Storage: {{.Storage}}
- Backup Location: {{.BackupLocation}}
- Backup Size: {{.BackupSize}} bytes
- Backup Reference: {{.BackupReference}}
```
> email-error.template
#### `email-error.tmpl` (Failed Backup)
```html
<!DOCTYPE html>
@@ -140,16 +166,15 @@ Backup Details:
<p>An error occurred during database backup.</p>
<h3>Failure Details:</h3>
<ul>
<li>Error Message: {{.Error}}</li>
<li>Date: {{.EndTime}}</li>
<li>Backup Reference: {{.BackupReference}} </li>
<li>Error Message: {{.Error}}</li>
<li>Date: {{.EndTime}}</li>
<li>Backup Reference: {{.BackupReference}}</li>
</ul>
</body>
</html>
```
> telegram-error.template
#### `telegram-error.tmpl` (Failed Backup)
```html
🔴 Urgent: Database Backup Failure Notification
@@ -159,4 +184,14 @@ Failure Details:
Error Message: {{.Error}}
Date: {{.EndTime}}
```
Backup Reference: {{.BackupReference}}
```
---
## Key Notes
- **SMTP Configuration**: Ensure your SMTP server supports TLS unless `MAIL_SKIP_TLS` is set to `true`.
- **Telegram Configuration**: Obtain your bot token and chat ID from Telegram.
- **Custom Templates**: Mount custom templates to `/config/templates` to override default notifications.
- **Time Format**: Use the `TIME_FORMAT` environment variable to customize the timestamp format in notifications.

60
docs/how-tos/restore-from-s3.md
View File

@@ -5,45 +5,71 @@ parent: How Tos
nav_order: 6
---
# Restore database from S3 storage
# Restore Database from S3 Storage
To restore the database, you need to add `restore` command and specify the file to restore by adding `--file store_20231219_022941.sql.gz`.
To restore a MySQL database from a backup stored in S3, use the `restore` command and specify the backup file with the `--file` flag. The system supports the following file formats:
{: .note }
It supports __.sql__,__.sql.gpg__ and __.sql.gz__,__.sql.gz.gpg__ compressed file.
- `.sql` (uncompressed SQL dump)
- `.sql.gz` (gzip-compressed SQL dump)
- `.sql.gpg` (GPG-encrypted SQL dump)
- `.sql.gz.gpg` (GPG-encrypted and gzip-compressed SQL dump)
### Restore
---
```yml
## Configuration Steps
1. **Specify the Backup File**: Use the `--file` flag to specify the backup file to restore.
2. **Set the Storage Type**: Add the `--storage s3` flag to indicate that the backup is stored in S3.
3. **Provide S3 Configuration**: Include the necessary AWS S3 credentials and configuration.
4. **Provide Database Credentials**: Ensure the correct database connection details are provided.
---
## Example: Restore from S3 Configuration
Below is an example `docker-compose.yml` configuration for restoring a database from S3 storage:
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: restore --storage s3 -d my-database -f store_20231219_022941.sql.gz --path /my-custom-path
volumes:
- ./backup:/backup
- ./backup:/backup # Mount the directory for local operations (if needed)
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## AWS configurations
## AWS S3 Configuration
- AWS_S3_ENDPOINT=https://s3.amazonaws.com
- AWS_S3_BUCKET_NAME=backup
- AWS_REGION="us-west-2"
- AWS_REGION=us-west-2
- AWS_ACCESS_KEY=xxxx
- AWS_SECRET_KEY=xxxxx
## In case you are using S3 alternative such as Minio and your Minio instance is not secured, you change it to true
- AWS_DISABLE_SSL="false"
- AWS_FORCE_PATH_STYLE="false"
# mysql-bkup container must be connected to the same network with your database
## Optional: Disable SSL for S3 alternatives like Minio
- AWS_DISABLE_SSL=false
## Optional: Enable path-style access for S3 alternatives like Minio
- AWS_FORCE_PATH_STYLE=false
# Ensure the pg-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
---
## Key Notes
- **Supported File Formats**: The restore process supports `.sql`, `.sql.gz`, `.sql.gpg`, and `.sql.gz.gpg` files.
- **S3 Path**: Use the `--path` flag to specify the folder within the S3 bucket where the backup file is located.
- **Encrypted Backups**: If the backup is encrypted with GPG, ensure the `GPG_PASSPHRASE` environment variable is set for automatic decryption.
- **S3 Alternatives**: For S3-compatible storage like Minio, set `AWS_DISABLE_SSL` and `AWS_FORCE_PATH_STYLE` as needed.
- **Network Configuration**: Ensure the `pg-bkup` container is connected to the same network as your database.

61
docs/how-tos/restore-from-ssh.md
View File

@@ -4,44 +4,71 @@ layout: default
parent: How Tos
nav_order: 7
---
# Restore database from SSH remote server
To restore the database from your remote server, you need to add `restore` command and specify the file to restore by adding `--file store_20231219_022941.sql.gz`.
# Restore Database from SSH Remote Server
{: .note }
It supports __.sql__,__.sql.gpg__ and __.sql.gz__,__.sql.gz.gpg__ compressed file.
To restore a MySQL database from a backup stored on an SSH remote server, use the `restore` command and specify the backup file with the `--file` flag. The system supports the following file formats:
### Restore
- `.sql` (uncompressed SQL dump)
- `.sql.gz` (gzip-compressed SQL dump)
- `.sql.gpg` (GPG-encrypted SQL dump)
- `.sql.gz.gpg` (GPG-encrypted and gzip-compressed SQL dump)
```yml
---
## Configuration Steps
1. **Specify the Backup File**: Use the `--file` flag to specify the backup file to restore.
2. **Set the Storage Type**: Add the `--storage ssh` flag to indicate that the backup is stored on an SSH remote server.
3. **Provide SSH Configuration**: Include the necessary SSH credentials and configuration.
4. **Provide Database Credentials**: Ensure the correct database connection details are provided.
---
## Example: Restore from SSH Remote Server Configuration
Below is an example `docker-compose.yml` configuration for restoring a database from an SSH remote server:
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: restore --storage ssh -d my-database -f store_20231219_022941.sql.gz --path /home/jkaninda/backups
volumes:
- ./backup:/backup
- ./backup:/backup # Mount the directory for local operations (if needed)
- ./id_ed25519:/tmp/id_ed25519 # Mount the SSH private key file
environment:
- DB_PORT=3306
- DB_HOST=postgres
- DB_HOST=mysql
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## SSH config
- SSH_HOST_NAME="hostname"
## SSH Configuration
- SSH_HOST_NAME=hostname
- SSH_PORT=22
- SSH_USER=user
- SSH_REMOTE_PATH=/home/jkaninda/backups
- SSH_IDENTIFY_FILE=/tmp/id_ed25519
## We advise you to use a private jey instead of password
## Optional: Use password instead of private key (not recommended)
#- SSH_PASSWORD=password
# mysql-bkup container must be connected to the same network with your database
# Ensure the mysql-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
```
---
## Key Notes
- **Supported File Formats**: The restore process supports `.sql`, `.sql.gz`, `.sql.gpg`, and `.sql.gz.gpg` files.
- **SSH Path**: Use the `--path` flag to specify the folder on the SSH remote server where the backup file is located.
- **Encrypted Backups**: If the backup is encrypted with GPG, ensure the `GPG_PASSPHRASE` environment variable is set for automatic decryption.
- **SSH Authentication**: Use a private key (`SSH_IDENTIFY_FILE`) for SSH authentication instead of a password for better security.
- **Network Configuration**: Ensure the `mysql-bkup` container is connected to the same network as your database.

52
docs/how-tos/restore.md
View File

@@ -5,36 +5,60 @@ parent: How Tos
nav_order: 5
---
# Restore database
To restore the database, you need to add `restore` command and specify the file to restore by adding `--file store_20231219_022941.sql.gz`.
# Restore Database
{: .note }
It supports __.sql__,__.sql.gpg__ and __.sql.gz__,__.sql.gz.gpg__ compressed file.
To restore a MySQL database, use the `restore` command and specify the backup file to restore with the `--file` flag.
### Restore
The system supports the following file formats:
```yml
- `.sql` (uncompressed SQL dump)
- `.sql.gz` (gzip-compressed SQL dump)
- `.sql.gpg` (GPG-encrypted SQL dump)
- `.sql.gz.gpg` (GPG-encrypted and gzip-compressed SQL dump)
---
## Configuration Steps
1. **Specify the Backup File**: Use the `--file` flag to specify the backup file to restore.
2. **Provide Database Credentials**: Ensure the correct database connection details are provided.
---
## Example: Restore Configuration
Below is an example `docker-compose.yml` configuration for restoring a database:
```yaml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
# In production, lock your image tag to a specific release version
# instead of using `latest`. Check https://github.com/jkaninda/mysql-bkup/releases
# for available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: restore -d database -f store_20231219_022941.sql.gz
volumes:
- ./backup:/backup
- ./backup:/backup # Mount the directory containing the backup file
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_HOST=postgres
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
# mysql-bkup container must be connected to the same network with your database
# Ensure the pg-bkup container is connected to the same network as your database
networks:
- web
networks:
web:
```
```
---
## Key Notes
- **Supported File Formats**: The restore process supports `.sql`, `.sql.gz`, `.sql.gpg`, and `.sql.gz.gpg` files.
- **Encrypted Backups**: If the backup is encrypted with GPG, ensure the `GPG_PASSPHRASE` environment variable is set for automatic decryption.
- **Network Configuration**: Ensure the `mysql-bkup` container is connected to the same network as your database.