jkaninda/mysql-bkup
1
1
Fork 0
mirror of https://github.com/jkaninda/mysql-bkup.git synced 2025-12-06 21:49:40 +01:00
Code Issues Actions Projects Releases Wiki Activity

Add SSH storage, add database backup encrypt and decrypt

Browse Source
This commit is contained in:
Jonas Kaninda
2024-08-03 16:03:17 +02:00
parent c8e68af09f
commit caeba955c5
46 changed files with 2618 additions and 726 deletions
Download Patch File Download Diff File Expand all files Collapse all files

391
README.md
View File

@@ -1,338 +1,95 @@
# MySQL Backup
MySQL Backup and Restoration tool. Backup database to AWS S3 storage or any S3 Alternatives for Object Storage.
mysql-bkup is a Docker container image that can be used to backup and restore Postgres database. It supports local storage, AWS S3 or any S3 Alternatives for Object Storage, and SSH compatible storage.
It also supports __encrypting__ your backups using GPG.
[![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/build.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/build.yml)
The [jkaninda/mysql-bkup](https://hub.docker.com/r/jkaninda/mysql-bkup) Docker image can be deployed on Docker, Docker Swarm and Kubernetes.
It handles __recurring__ backups of postgres database on Docker and can be deployed as __CronJob on Kubernetes__ using local, AWS S3 or SSH compatible storage.
It also supports __encrypting__ your backups using GPG.
[![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml)
[![Go Report](https://goreportcard.com/badge/github.com/jkaninda/mysql-bkup)](https://goreportcard.com/report/github.com/jkaninda/mysql-bkup)
![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/jkaninda/mysql-bkup?style=flat-square)
![Docker Pulls](https://img.shields.io/docker/pulls/jkaninda/mysql-bkup?style=flat-square)
<p align="center">
<a href="https://github.com/jkaninda/mysql-bkup">
<img src="https://www.mysql.com/common/logos/logo-mysql-170x115.png" alt="Logo">
</a>
</p>
> Runs on:
- Docker
- Kubernetes
> Links:
## Documentation is found at <https://jkaninda.github.io/mysql-bkup>
## Links:
- [Docker Hub](https://hub.docker.com/r/jkaninda/mysql-bkup)
- [Github](https://github.com/jkaninda/mysql-bkup)
## PostgreSQL solution :
- [PostgreSQL](https://github.com/jkaninda/pg-bkup)
## MySQL solution :
- [MySQL](https://github.com/jkaninda/mysql-bkup)
## Storage:
- local
- s3
- Object storage
- Local
- AWS S3 or any S3 Alternatives for Object Storage
- SSH
## Volumes:
## Quickstart
- /s3mnt => S3 mounting path
- /backup => local storage mounting path
### Simple backup using Docker CLI
## Usage
To run a one time backup, bind your local volume to `/backup` in the container and run the `mysql-bkup backup` command:
| Options | Shorts | Usage |
|-----------------------|--------|------------------------------------------------------------------------|
| mysql-bkup | bkup | CLI utility |
| backup | | Backup database operation |
| restore | | Restore database operation |
| history | | Show the history of backup |
| --storage | -s | Set storage. local or s3 (default: local) |
| --file | -f | Set file name for restoration |
| --path | | Set s3 path without file name. eg: /custom_path |
| --dbname | -d | Set database name |
| --port | -p | Set database port (default: 3306) |
| --mode | -m | Set execution mode. default or scheduled (default: default) |
| --disable-compression | | Disable database backup compression |
| --prune | | Delete old backup, default disabled |
| --keep-last | | Delete old backup created more than specified days ago, default 7 days |
| --period | | Set crontab period for scheduled mode only. (default: "0 1 * * *") |
| --timeout | -t | Set timeout (default: 60s) |
| --help | -h | Print this help message and exit |
| --version | -V | Print version information and exit |
## Environment variables
| Name | Requirement | Description |
|-------------|--------------------------------------------------|-----------------------------------------------------|
| DB_PORT | Optional, default 3306 | Database port number |
| DB_HOST | Required | Database host |
| DB_NAME | Optional if it was provided from the -d flag | Database name |
| DB_USERNAME | Required | Database user name |
| DB_PASSWORD | Required | Database password |
| ACCESS_KEY | Optional, required for S3 storage | AWS S3 Access Key |
| SECRET_KEY | Optional, required for S3 storage | AWS S3 Secret Key |
| BUCKET_NAME | Optional, required for S3 storage | AWS S3 Bucket Name |
| S3_ENDPOINT | Optional, required for S3 storage | AWS S3 Endpoint |
| FILE_NAME | Optional if it was provided from the --file flag | Database file to restore (extensions: .sql, .sql.gz |
## Note:
Creating a user for backup tasks who has read-only access is recommended!
> create read-only user
```sh
mysql -u root -p
```shell
docker run --rm --network your_network_name \
-v $PWD/backup:/backup/ \
-e "DB_HOST=dbhost" \
-e "DB_USERNAME=username" \
-e "DB_PASSWORD=password" \
jkaninda/mysql-bkup mysql-bkup backup -d database_name
```
```sql
CREATE USER read_only_user IDENTIFIED BY 'your_strong_password';
Alternatively, pass a `--env-file` in order to use a full config as described below.
```
```sql
GRANT SELECT, SHOW VIEW ON *.* TO read_only_user;
```
```sql
FLUSH PRIVILEGES;
```
### Simple backup in docker compose file
## Backup database :
Simple backup usage
```sh
mysql-bkup backup --dbname database_name
```
```sh
mysql-bkup backup -d database_name
```
### S3
```sh
mysql-bkup backup --storage s3 --dbname database_name
```
## Docker run:
```sh
docker run --rm --network your_network_name --name mysql-bkup -v $PWD/backup:/backup/ -e "DB_HOST=database_host_name" -e "DB_USERNAME=username" -e "DB_PASSWORD=password" jkaninda/mysql-bkup:latest mysql-bkup backup -d database_name
```
## Docker compose file:
```yaml
version: '3'
services:
mariadb:
container_name: mariadb
image: mariadb
environment:
MYSQL_DATABASE: mariadb
MYSQL_USER: mariadb
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command:
- /bin/sh
- -c
- mysql-bkup backup -d database_name
- mysql-bkup backup
volumes:
- ./backup:/backup
environment:
- DB_PORT=3306
- DB_HOST=mariadb
- DB_USERNAME=mariadb
- DB_PORT=5432
- DB_HOST=postgres
- DB_NAME=foo
- DB_USERNAME=bar
- DB_PASSWORD=password
# mysql-bkup container must be connected to the same network with your database
networks:
- web
networks:
web:
```
## Restore database :
## Deploy on Kubernetes
Simple database restore operation usage
For Kubernetes, you don't need to run it in scheduled mode. You can deploy it as CronJob.
```sh
mysql-bkup restore --dbname database_name --file database_20231217_115621.sql
```
```sh
mysql-bkup restore -f database_20231217_115621.sql
```
### S3
```sh
mysql-bkup restore --storage s3 --file database_20231217_115621.sql
```
## Docker run:
```sh
docker run --rm --network your_network_name --name mysql-bkup -v $PWD/backup:/backup/ -e "DB_HOST=database_host_name" -e "DB_USERNAME=username" -e "DB_PASSWORD=password" jkaninda/mysql-bkup mysql-bkup backup -d database_name -f db_20231219_022941.sql.gz
```
## Docker compose file:
```yaml
version: '3'
services:
mariadb:
container_name: mariadb
image: mariadb:latest
environment:
MYSQL_DATABASE: mariadb
MYSQL_USER: mariadb
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
mysql-bkup:
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command:
- /bin/sh
- -c
- mysql-bkup restore --file database_20231217_115621.sql --dbname database_name
volumes:
- ./backup:/backup
environment:
#- FILE_NAME=mariadb_20231217_040238.sql # Optional if file name is set from command
- DB_PORT=3306
- DB_HOST=mariadb
- DB_NAME=mariadb
- DB_USERNAME=mariadb
- DB_PASSWORD=password
```
## Run
```sh
docker-compose up -d
```
## Backup to S3
```sh
docker run --rm --privileged --device /dev/fuse --name mysql-bkup -e "DB_HOST=db_hostname" -e "DB_USERNAME=username" -e "DB_PASSWORD=password" -e "ACCESS_KEY=your_access_key" -e "SECRET_KEY=your_secret_key" -e "BUCKETNAME=your_bucket_name" -e "S3_ENDPOINT=https://s3.us-west-2.amazonaws.com" jkaninda/mysql-bkup mysql-bkup backup -s s3 -d database_name
```
> To change s3 backup path add this flag : --path /myPath . default path is /mysql_bkup
Simple S3 backup usage
```sh
bkup backup --storage s3 --dbname mydatabase
```
```yaml
version: '3'
services:
mysql-bkup:
image: jkaninda/mysql-bkup
container_name: mysql-bkup
privileged: true
devices:
- "/dev/fuse"
command:
- /bin/sh
- -c
- mysql-bkup restore --storage s3 -f database_20231217_115621.sql.gz
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_NAME=mariadb
- DB_USERNAME=mariadb
- DB_PASSWORD=password
- ACCESS_KEY=${ACCESS_KEY}
- SECRET_KEY=${SECRET_KEY}
- BUCKET_NAME=${BUCKET_NAME}
- S3_ENDPOINT=${S3_ENDPOINT}
```
## Run in Scheduled mode
This tool can be run as CronJob in Kubernetes for a regular backup which makes deployment on Kubernetes easy as Kubernetes has CronJob resources.
For Docker, you need to run it in scheduled mode by adding `--mode scheduled` flag and specify the periodical backup time by adding `--period "0 1 * * *"` flag.
Make an automated backup on Docker
## Syntax of crontab (field description)
The syntax is:
- 1: Minute (0-59)
- 2: Hours (0-23)
- 3: Day (0-31)
- 4: Month (0-12 [12 == December])
- 5: Day of the week(0-7 [7 or 0 == sunday])
Easy to remember format:
```conf
* * * * * command to be executed
```
```conf
- - - - -
| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)
```
> At every 30th minute
```conf
*/30 * * * *
```
> “At minute 0.” every hour
```conf
0 * * * *
```
> “At 01:00.” every day
```conf
0 1 * * *
```
## Example of scheduled mode
> Docker run :
```sh
docker run --rm --name mysql-bkup -v $BACKUP_DIR:/backup/ -e "DB_HOST=$DB_HOST" -e "DB_USERNAME=$DB_USERNAME" -e "DB_PASSWORD=$DB_PASSWORD" jkaninda/mysql-bkup mysql-bkup backup --dbname $DB_NAME --mode scheduled --period "0 1 * * *"
```
> With Docker compose
```yaml
version: "3"
services:
mysql-bkup:
image: jkaninda/mysql-bkup
container_name: mysql-bkup
privileged: true
devices:
- "/dev/fuse"
command:
- /bin/sh
- -c
- mysql-bkup backup --storage s3 --path /mys3_custome_path --dbname database_name --mode scheduled --period "*/30 * * * *"
environment:
- DB_PORT=3306
- DB_HOST=mysqlhost
- DB_USERNAME=userName
- DB_PASSWORD=${DB_PASSWORD}
- ACCESS_KEY=${ACCESS_KEY}
- SECRET_KEY=${SECRET_KEY}
- BUCKET_NAME=${BUCKET_NAME}
- S3_ENDPOINT=${S3_ENDPOINT}
```
## Kubernetes CronJob
For Kubernetes you don't need to run it in scheduled mode.
Simple Kubernetes CronJob usage:
### Simple Kubernetes CronJob usage:
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: mysql-bkup-job
name: bkup-job
spec:
schedule: "0 1 * * *"
jobTemplate:
@@ -342,15 +99,13 @@ spec:
containers:
- name: mysql-bkup
image: jkaninda/mysql-bkup
securityContext:
privileged: true
command:
- /bin/sh
- -c
- mysql-bkup backup -s s3 --path /custom_path
env:
- name: DB_PORT
value: "3306"
value: "5432"
- name: DB_HOST
value: ""
- name: DB_NAME
@@ -359,22 +114,48 @@ spec:
value: ""
# Please use secret!
- name: DB_PASSWORD
value: "password"
- name: ACCESS_KEY
value: ""
- name: SECRET_KEY
value: ""
- name: BUCKET_NAME
value: ""
- name: S3_ENDPOINT
value: "https://s3.us-west-2.amazonaws.com"
- name: AWS_S3_ENDPOINT
value: "https://s3.amazonaws.com"
- name: AWS_S3_BUCKET_NAME
value: "xxx"
- name: AWS_REGION
value: "us-west-2"
- name: AWS_ACCESS_KEY
value: "xxxx"
- name: AWS_SECRET_KEY
value: "xxxx"
- name: AWS_DISABLE_SSL
value: "false"
restartPolicy: Never
```
## Available image registries
## Contributing
This Docker image is published to both Docker Hub and the GitHub container registry.
Depending on your preferences and needs, you can reference both `jkaninda/mysql-bkup` as well as `ghcr.io/jkaninda/mysql-bkup`:
```
docker pull jkaninda/mysql-bkup:v1.0
docker pull ghcr.io/jkaninda/mysql-bkup:v1.0
```
Documentation references Docker Hub, but all examples will work using ghcr.io just as well.
## Supported Engines
This image is developed and tested against the Docker CE engine and Kubernetes exclusively.
While it may work against different implementations, there are no guarantees about support for non-Docker engines.
## References
We decided to publish this image as a simpler and more lightweight alternative because of the following requirements:
- The original image is based on `ubuntu` and requires additional tools, making it heavy.
- This image is written in Go.
- `arm64` and `arm/v7` architectures are supported.
- Docker in Swarm mode is supported.
- Kubernetes is supported.
Contributions are welcome! If you encounter any issues or have suggestions for improvements, please create an issue or submit a pull request.
Make sure to follow the existing coding style and provide tests for your changes.
## License