jkaninda/mysql-bkup
1
1
Fork 0
mirror of https://github.com/jkaninda/mysql-bkup.git synced 2025-12-06 21:49:40 +01:00
Code Issues Actions Projects Releases Wiki Activity

Merge pull request #146 from jkaninda/feature/azure-blob

Browse Source 
feat: add Azure Blob storage
This commit is contained in:
Jonas Kaninda
2024-12-06 18:29:34 +01:00
committed by GitHub
parent 4ecd96e75c c360441445
commit f53272ccf0
29 changed files with 1373 additions and 602 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
.golangci.yml Normal file
View File

@@ -0,0 +1,43 @@
run:
timeout: 5m
allow-parallel-runners: true
issues:
# don't skip warning about doc comments
# don't exclude the default set of lint
exclude-use-default: false
# restore some of the defaults
# (fill in the rest as needed)
exclude-rules:
- path: "internal/*"
linters:
- dupl
- lll
- goimports
linters:
disable-all: true
enable:
- dupl
- errcheck
- copyloopvar
- ginkgolinter
- goconst
- gocyclo
- gofmt
- gosimple
- govet
- ineffassign
- misspell
- nakedret
- prealloc
- revive
- staticcheck
- typecheck
- unconvert
- unparam
- unused
linters-settings:
revive:
rules:
- name: comment-spacings

3
Dockerfile
View File

@@ -1,5 +1,6 @@
FROM golang:1.23.3 AS build FROM golang:1.23.3 AS build
WORKDIR /app WORKDIR /app
ARG appVersion=""
# Copy the source code. # Copy the source code.
COPY . . COPY . .
@@ -7,7 +8,7 @@ COPY . .
RUN go mod download RUN go mod download
# Build # Build
RUN CGO_ENABLED=0 GOOS=linux go build -o /app/mysql-bkup RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-X 'github.com/jkaninda/pg-bkup/utils.Version=${appVersion}'" -o /app/mysql-bkup
FROM alpine:3.20.3 FROM alpine:3.20.3
ENV TZ=UTC ENV TZ=UTC

52
README.md
View File

@@ -1,14 +1,7 @@
# MySQL Backup # MYSQL-BKUP
MySQL Backup is a Docker container image that can be used to backup, restore and migrate MySQL database. It supports local storage, AWS S3 or any S3 Alternatives for Object Storage, FTP and SSH compatible storage.
It also supports __encrypting__ your backups using GPG.
The [jkaninda/mysql-bkup](https://hub.docker.com/r/jkaninda/mysql-bkup) Docker image can be deployed on Docker, Docker Swarm and Kubernetes.
It handles __recurring__ backups of MySQL or MariaDB database on Docker and can be deployed as __CronJob on Kubernetes__ using local, AWS S3, FTP or SSH compatible storage.
It also supports database __encryption__ using GPG.
Telegram and Email notifications on successful and failed backups.
**MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**.
It supports a variety of storage options and ensures data security through GPG encryption.
[![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml) [![Build](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml/badge.svg)](https://github.com/jkaninda/mysql-bkup/actions/workflows/release.yml)
[![Go Report](https://goreportcard.com/badge/github.com/jkaninda/mysql-bkup)](https://goreportcard.com/report/github.com/jkaninda/mysql-bkup) [![Go Report](https://goreportcard.com/badge/github.com/jkaninda/mysql-bkup)](https://goreportcard.com/report/github.com/jkaninda/mysql-bkup)
@@ -16,6 +9,37 @@ Telegram and Email notifications on successful and failed backups.
![Docker Pulls](https://img.shields.io/docker/pulls/jkaninda/mysql-bkup?style=flat-square) ![Docker Pulls](https://img.shields.io/docker/pulls/jkaninda/mysql-bkup?style=flat-square)
<a href="https://ko-fi.com/jkaninda"><img src="https://uploads-ssl.webflow.com/5c14e387dab576fe667689cf/5cbed8a4ae2b88347c06c923_BuyMeACoffee_blue.png" height="20" alt="buy ma a coffee"></a> <a href="https://ko-fi.com/jkaninda"><img src="https://uploads-ssl.webflow.com/5c14e387dab576fe667689cf/5cbed8a4ae2b88347c06c923_BuyMeACoffee_blue.png" height="20" alt="buy ma a coffee"></a>
## Features
- **Storage Options:**
- Local storage
- AWS S3 or any S3-compatible object storage
- FTP
- SSH-compatible storage
- Azure Blob storage
- **Data Security:**
- Backups can be encrypted using **GPG** to ensure confidentiality.
- **Deployment Flexibility:**
- Available as the [jkaninda/mysql-bkup](https://hub.docker.com/r/jkaninda/mysql-bkup) Docker image.
- Deployable on **Docker**, **Docker Swarm**, and **Kubernetes**.
- Supports recurring backups of PostgreSQL databases when deployed:
- On Docker for automated backup schedules.
- As a **Job** or **CronJob** on Kubernetes.
- **Notifications:**
- Get real-time updates on backup success or failure via:
- **Telegram**
- **Email**
## Use Cases
- **Automated Recurring Backups:** Schedule regular backups for PostgreSQL databases.
- **Cross-Environment Migration:** Easily migrate your PostgreSQL databases across different environments using supported storage options.
- **Secure Backup Management:** Protect your data with Gmysql encryption.
Successfully tested on: Successfully tested on:
- Docker - Docker
- Docker in Swarm mode - Docker in Swarm mode
@@ -91,7 +115,6 @@ services:
networks: networks:
web: web:
``` ```
### Docker recurring backup ### Docker recurring backup
```shell ```shell
@@ -163,16 +186,11 @@ docker pull ghcr.io/jkaninda/mysql-bkup
Documentation references Docker Hub, but all examples will work using ghcr.io just as well. Documentation references Docker Hub, but all examples will work using ghcr.io just as well.
## Supported Engines
This image is developed and tested against the Docker CE engine and Kubernetes exclusively.
While it may work against different implementations, there are no guarantees about support for non-Docker engines.
## References ## References
We decided to publish this image as a simpler and more lightweight alternative because of the following requirements: We decided to publish this image as a simpler and more lightweight alternative because of the following requirements:
- The original image is based on `alpine` and requires additional tools, making it heavy. - The original image is based on `Alpine` and requires additional tools, making it heavy.
- This image is written in Go. - This image is written in Go.
- `arm64` and `arm/v7` architectures are supported. - `arm64` and `arm/v7` architectures are supported.
- Docker in Swarm mode is supported. - Docker in Swarm mode is supported.

31
cmd/backup.go
View File

@@ -1,13 +1,32 @@
// Package cmd / // Package cmd /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package cmd package cmd
import ( import (
"github.com/jkaninda/mysql-bkup/internal" "github.com/jkaninda/mysql-bkup/internal"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -20,7 +39,7 @@ var BackupCmd = &cobra.Command{
if len(args) == 0 { if len(args) == 0 {
internal.StartBackup(cmd) internal.StartBackup(cmd)
} else { } else {
utils.Fatal(`"backup" accepts no argument %q`, args) logger.Fatal(`"backup" accepts no argument %q`, args)
} }
}, },
} }

32
cmd/migrate.go
View File

@@ -1,14 +1,32 @@
// Package cmd / // Package cmd /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package cmd package cmd
import ( import (
"github.com/jkaninda/mysql-bkup/internal" "github.com/jkaninda/mysql-bkup/internal"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -19,7 +37,7 @@ var MigrateCmd = &cobra.Command{
if len(args) == 0 { if len(args) == 0 {
internal.StartMigration(cmd) internal.StartMigration(cmd)
} else { } else {
utils.Fatal(`"migrate" accepts no argument %q`, args) logger.Fatal(`"migrate" accepts no argument %q`, args)
} }

26
cmd/restore.go
View File

@@ -1,7 +1,31 @@
package cmd package cmd
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
import ( import (
"github.com/jkaninda/mysql-bkup/internal" "github.com/jkaninda/mysql-bkup/internal"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -14,7 +38,7 @@ var RestoreCmd = &cobra.Command{
if len(args) == 0 { if len(args) == 0 {
internal.StartRestore(cmd) internal.StartRestore(cmd)
} else { } else {
utils.Fatal(`"restore" accepts no argument %q`, args) logger.Fatal(`"restore" accepts no argument %q`, args)
} }

28
cmd/root.go
View File

@@ -1,9 +1,27 @@
// Package cmd / // Package cmd /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package cmd package cmd
import ( import (

28
cmd/version.go
View File

@@ -1,9 +1,27 @@
// Package cmd / // Package cmd /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package cmd package cmd
import ( import (

48
docs/how-tos/azure-blob.md Normal file
View File

@@ -0,0 +1,48 @@
---
title: Azure Blob storage
layout: default
parent: How Tos
nav_order: 5
---
# Azure Blob storage
{: .note }
As described on local backup section, to change the storage of you backup and use Azure Blob as storage. You need to add `--storage azure` (-s azure).
You can also specify a folder where you want to save you data by adding `--path my-custom-path` flag.
## Backup to S3
```yml
services:
mysql-bkup:
# In production, it is advised to lock your image tag to a proper
# release version instead of using `latest`.
# Check https://github.com/jkaninda/mysql-bkup/releases
# for a list of available releases.
image: jkaninda/mysql-bkup
container_name: mysql-bkup
command: backup --storage s3 -d database --path /my-custom-path
environment:
- DB_PORT=3306
- DB_HOST=mysql
- DB_NAME=database
- DB_USERNAME=username
- DB_PASSWORD=password
## Azure Blob configurations
- AZURE_STORAGE_CONTAINER_NAME=backup-container
- AZURE_STORAGE_ACCOUNT_NAME=account-name
- AZURE_STORAGE_ACCOUNT_KEY=Ppby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
## In case you are using S3 alternative such as Minio and your Minio instance is not secured, you change it to true
- AWS_DISABLE_SSL="false"
- AWS_FORCE_PATH_STYLE=true # true for S3 alternative such as Minio
# mysql-bkup container must be connected to the same network with your database
networks:
- web
networks:
web:
```

35
docs/index.md
View File

@@ -6,23 +6,40 @@ nav_order: 1
# About mysql-bkup # About mysql-bkup
{:.no_toc} {:.no_toc}
MySQL Backup is a Docker container image that can be used to backup, restore and migrate MySQL database. It supports local storage, AWS S3 or any S3 Alternatives for Object Storage, FTP and SSH remote storage.
It also supports __encrypting__ your backups using GPG.
Telegram and Email notifications on successful and failed backups. **MYSQL-BKUP** is a Docker container image designed to **backup, restore, and migrate MySQL databases**.
It supports a variety of storage options and ensures data security through GPG encryption.
## Features
We are open to receiving stars, PRs, and issues! - **Storage Options:**
- Local storage
- AWS S3 or any S3-compatible object storage
- FTP
- SSH-compatible storage
- Azure Blob storage
- **Data Security:**
- Backups can be encrypted using **GPG** to ensure confidentiality.
{: .fs-6 .fw-300 } - **Deployment Flexibility:**
- Available as the [jkaninda/mysql-bkup](https://hub.docker.com/r/jkaninda/mysql-bkup) Docker image.
- Deployable on **Docker**, **Docker Swarm**, and **Kubernetes**.
- Supports recurring backups of PostgreSQL databases when deployed:
- On Docker for automated backup schedules.
- As a **Job** or **CronJob** on Kubernetes.
--- - **Notifications:**
- Get real-time updates on backup success or failure via:
- **Telegram**
- **Email**
The [jkaninda/mysql-bkup](https://hub.docker.com/r/jkaninda/mysql-bkup) Docker image can be deployed on Docker, Docker Swarm and Kubernetes. ## Use Cases
It handles __recurring__ backups of postgres database on Docker and can be deployed as __CronJob on Kubernetes__ using local, AWS S3 or SSH compatible storage.
- **Automated Recurring Backups:** Schedule regular backups for PostgreSQL databases.
- **Cross-Environment Migration:** Easily migrate your PostgreSQL databases across different environments using supported storage options.
- **Secure Backup Management:** Protect your data with Gmysql encryption.
It also supports database __encryption__ using GPG.
{: .note } {: .note }

8
go.mod
View File

@@ -1,19 +1,22 @@
module github.com/jkaninda/mysql-bkup module github.com/jkaninda/mysql-bkup
go 1.22.5 go 1.23.2
require github.com/spf13/pflag v1.0.5 // indirect require github.com/spf13/pflag v1.0.5 // indirect
require ( require (
github.com/go-mail/mail v2.3.1+incompatible github.com/go-mail/mail v2.3.1+incompatible
github.com/jkaninda/encryptor v0.0.0-20241013064832-ed4bd6a1b221 github.com/jkaninda/encryptor v0.0.0-20241013064832-ed4bd6a1b221
github.com/jkaninda/go-storage v0.1.1 github.com/jkaninda/go-storage v0.1.2
github.com/robfig/cron/v3 v3.0.1 github.com/robfig/cron/v3 v3.0.1
github.com/spf13/cobra v1.8.1 github.com/spf13/cobra v1.8.1
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )
require ( require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 // indirect
github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f // indirect github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f // indirect
github.com/ProtonMail/gopenpgp/v2 v2.7.5 // indirect github.com/ProtonMail/gopenpgp/v2 v2.7.5 // indirect
@@ -27,6 +30,7 @@ require (
github.com/jmespath/go-jmespath v0.4.0 // indirect github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
golang.org/x/crypto v0.28.0 // indirect golang.org/x/crypto v0.28.0 // indirect
golang.org/x/net v0.29.0 // indirect
golang.org/x/sys v0.26.0 // indirect golang.org/x/sys v0.26.0 // indirect
golang.org/x/text v0.19.0 // indirect golang.org/x/text v0.19.0 // indirect
gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect

12
go.sum
View File

@@ -1,3 +1,9 @@
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0 h1:JZg6HRh6W6U4OLl6lk7BZ7BLisIzM9dG1R50zUk9C/M=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0/go.mod h1:YL1xnZ6QejvQHWJrX/AvhFl4WW4rqHVoKspWNVwFk0M=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0 h1:mlmW46Q0B79I+Aj4azKC6xDMFN9a9SyZWESlGWYXbFs=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0/go.mod h1:PXe2h+LKcWTX9afWdZoHyODqR4fBa5boUM/8uJfZ0Jo=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 h1:KLq8BE0KwCL+mmXnjLWEAOYO+2l2AE4YMmqG1ZpZHBs= github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95 h1:KLq8BE0KwCL+mmXnjLWEAOYO+2l2AE4YMmqG1ZpZHBs=
github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0= github.com/ProtonMail/go-crypto v0.0.0-20230717121422-5aa5874ade95/go.mod h1:EjAoLdwvbIOoOQr3ihjnSoLZRtE8azugULFRteWMNc0=
github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f h1:tCbYj7/299ekTTXpdwKYF8eBlsYsDVoggDAuAjoK66k= github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f h1:tCbYj7/299ekTTXpdwKYF8eBlsYsDVoggDAuAjoK66k=
@@ -11,6 +17,8 @@ github.com/bramvdbogaerde/go-scp v1.5.0/go.mod h1:on2aH5AxaFb2G0N5Vsdy6B0Ml7k9Hu
github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0= github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/cloudflare/circl v1.3.3 h1:fE/Qz0QdIGqeWfnwq0RE0R7MI51s0M2E4Ga9kq5AEMs= github.com/cloudflare/circl v1.3.3 h1:fE/Qz0QdIGqeWfnwq0RE0R7MI51s0M2E4Ga9kq5AEMs=
github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA= github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be h1:J5BL2kskAlV9ckgEsNQXscjIaLiOYiZ75d4e94E6dcQ=
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be/go.mod h1:mk5IQ+Y0ZeO87b858TlA645sVcEcbiX6YqP98kt+7+w=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -28,6 +36,8 @@ github.com/jkaninda/encryptor v0.0.0-20241013064832-ed4bd6a1b221 h1:AwkCf7el1kze
github.com/jkaninda/encryptor v0.0.0-20241013064832-ed4bd6a1b221/go.mod h1:9F8ZJ+ZXE8DZBo77+aneGj8LMjrYXX6eFUCC/uqZOUo= github.com/jkaninda/encryptor v0.0.0-20241013064832-ed4bd6a1b221/go.mod h1:9F8ZJ+ZXE8DZBo77+aneGj8LMjrYXX6eFUCC/uqZOUo=
github.com/jkaninda/go-storage v0.1.1 h1:vjpdD/fh39S5HGyfHvLE5HGYOEPIukINlOX3OnM3GW4= github.com/jkaninda/go-storage v0.1.1 h1:vjpdD/fh39S5HGyfHvLE5HGYOEPIukINlOX3OnM3GW4=
github.com/jkaninda/go-storage v0.1.1/go.mod h1:7VK5gQISQaLxtLfBtc+een8spcgLVSBAKTRuyF1N81I= github.com/jkaninda/go-storage v0.1.1/go.mod h1:7VK5gQISQaLxtLfBtc+een8spcgLVSBAKTRuyF1N81I=
github.com/jkaninda/go-storage v0.1.2 h1:d7+TRPjmHXdSqO0wne3KAB8zt9ih8lf5D8aL4n7/Dds=
github.com/jkaninda/go-storage v0.1.2/go.mod h1:zVRnLprBk/9AUz2+za6Y03MgoNYrqKLy3edVtjqMaps=
github.com/jlaffaye/ftp v0.2.0 h1:lXNvW7cBu7R/68bknOX3MrRIIqZ61zELs1P2RAiA3lg= github.com/jlaffaye/ftp v0.2.0 h1:lXNvW7cBu7R/68bknOX3MrRIIqZ61zELs1P2RAiA3lg=
github.com/jlaffaye/ftp v0.2.0/go.mod h1:is2Ds5qkhceAPy2xD6RLI6hmp/qysSoymZ+Z2uTnspI= github.com/jlaffaye/ftp v0.2.0/go.mod h1:is2Ds5qkhceAPy2xD6RLI6hmp/qysSoymZ+Z2uTnspI=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
@@ -64,6 +74,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY= golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc= golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.29.0 h1:5ORfpBpCs4HzDYoodCDBbwHzdR5UrLBZ3sOnUJmFoHo=
golang.org/x/net v0.29.0/go.mod h1:gLkgy8jTGERgjzMic6DS9+SP0ajcu6Xu3Orq/SpETg0=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

122
internal/azure.go Normal file
View File

@@ -0,0 +1,122 @@
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal
import (
"fmt"
"github.com/jkaninda/go-storage/pkg/azure"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils"
"os"
"path/filepath"
"time"
)
func azureBackup(db *dbConfig, config *BackupConfig) {
logger.Info("Backup database to the remote FTP server")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
logger.Info("Uploading backup archive to Azure Blob storage ...")
logger.Info("Backup name is %s", finalFileName)
azureConfig := loadAzureConfig()
azureStorage, err := azure.NewStorage(azure.Config{
ContainerName: azureConfig.containerName,
AccountName: azureConfig.accountName,
AccountKey: azureConfig.accountKey,
RemotePath: config.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = azureStorage.Copy(finalFileName)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
logger.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
// Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
// Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error deleting file: %v", err)
}
if config.prune {
err := azureStorage.Prune(config.backupRetention)
if err != nil {
logger.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
logger.Info("Uploading backup archive to Azure Blob storage ... done ")
// Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
// Delete temp
deleteTemp()
logger.Info("Backup completed successfully")
}
func azureRestore(db *dbConfig, conf *RestoreConfig) {
logger.Info("Restore database from Azure Blob storage")
azureConfig := loadAzureConfig()
azureStorage, err := azure.NewStorage(azure.Config{
ContainerName: azureConfig.containerName,
AccountName: azureConfig.accountName,
AccountKey: azureConfig.accountKey,
RemotePath: conf.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = azureStorage.CopyFrom(conf.file)
if err != nil {
logger.Fatal("Error downloading backup file: %s", err)
}
RestoreDatabase(db, conf)
}

358
internal/backup.go
View File

@@ -1,18 +1,34 @@
// Package internal / // Package internal /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal package internal
import ( import (
"fmt" "fmt"
"github.com/jkaninda/encryptor" "github.com/jkaninda/encryptor"
"github.com/jkaninda/go-storage/pkg/ftp"
"github.com/jkaninda/go-storage/pkg/local" "github.com/jkaninda/go-storage/pkg/local"
"github.com/jkaninda/go-storage/pkg/s3" "github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/go-storage/pkg/ssh"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/robfig/cron/v3" "github.com/robfig/cron/v3"
"github.com/spf13/cobra" "github.com/spf13/cobra"
@@ -37,7 +53,7 @@ func StartBackup(cmd *cobra.Command) {
if utils.IsValidCronExpression(config.cronExpression) { if utils.IsValidCronExpression(config.cronExpression) {
scheduledMode(dbConf, config) scheduledMode(dbConf, config)
} else { } else {
utils.Fatal("Cron expression is not valid: %s", config.cronExpression) logger.Fatal("Cron expression is not valid: %s", config.cronExpression)
} }
} }
} else { } else {
@@ -48,22 +64,22 @@ func StartBackup(cmd *cobra.Command) {
// scheduledMode Runs backup in scheduled mode // scheduledMode Runs backup in scheduled mode
func scheduledMode(db *dbConfig, config *BackupConfig) { func scheduledMode(db *dbConfig, config *BackupConfig) {
utils.Info("Running in Scheduled mode") logger.Info("Running in Scheduled mode")
utils.Info("Backup cron expression: %s", config.cronExpression) logger.Info("Backup cron expression: %s", config.cronExpression)
utils.Info("The next scheduled time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat)) logger.Info("The next scheduled time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat))
utils.Info("Storage type %s ", config.storage) logger.Info("Storage type %s ", config.storage)
// Test backup // Test backup
utils.Info("Testing backup configurations...") logger.Info("Testing backup configurations...")
BackupTask(db, config) testDatabaseConnection(db)
utils.Info("Testing backup configurations...done") logger.Info("Testing backup configurations...done")
utils.Info("Creating backup job...") logger.Info("Creating backup job...")
// Create a new cron instance // Create a new cron instance
c := cron.New() c := cron.New()
_, err := c.AddFunc(config.cronExpression, func() { _, err := c.AddFunc(config.cronExpression, func() {
BackupTask(db, config) BackupTask(db, config)
utils.Info("Next backup time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat)) logger.Info("Next backup time is: %v", utils.CronNextTime(config.cronExpression).Format(timeFormat))
}) })
if err != nil { if err != nil {
@@ -71,8 +87,8 @@ func scheduledMode(db *dbConfig, config *BackupConfig) {
} }
// Start the cron scheduler // Start the cron scheduler
c.Start() c.Start()
utils.Info("Creating backup job...done") logger.Info("Creating backup job...done")
utils.Info("Backup job started") logger.Info("Backup job started")
defer c.Stop() defer c.Stop()
select {} select {}
} }
@@ -90,7 +106,7 @@ func multiBackupTask(databases []Database, bkConfig *BackupConfig) {
// BackupTask backups database // BackupTask backups database
func BackupTask(db *dbConfig, config *BackupConfig) { func BackupTask(db *dbConfig, config *BackupConfig) {
utils.Info("Starting backup task...") logger.Info("Starting backup task...")
// Generate file name // Generate file name
backupFileName := fmt.Sprintf("%s_%s.sql.gz", db.dbName, time.Now().Format("20060102_150405")) backupFileName := fmt.Sprintf("%s_%s.sql.gz", db.dbName, time.Now().Format("20060102_150405"))
if config.disableCompression { if config.disableCompression {
@@ -106,42 +122,49 @@ func BackupTask(db *dbConfig, config *BackupConfig) {
sshBackup(db, config) sshBackup(db, config)
case "ftp", "FTP": case "ftp", "FTP":
ftpBackup(db, config) ftpBackup(db, config)
case "azure":
azureBackup(db, config)
default: default:
localBackup(db, config) localBackup(db, config)
} }
} }
func startMultiBackup(bkConfig *BackupConfig, configFile string) { func startMultiBackup(bkConfig *BackupConfig, configFile string) {
utils.Info("Starting backup task...") logger.Info("Starting backup task...")
conf, err := readConf(configFile) conf, err := readConf(configFile)
if err != nil { if err != nil {
utils.Fatal("Error reading config file: %s", err) logger.Fatal("Error reading config file: %s", err)
} }
// Check if cronExpression is defined in config file // Check if cronExpression is defined in config file
if conf.CronExpression != "" { if conf.CronExpression != "" {
bkConfig.cronExpression = conf.CronExpression bkConfig.cronExpression = conf.CronExpression
} }
if len(conf.Databases) == 0 {
logger.Fatal("No databases found")
}
// Check if cronExpression is defined // Check if cronExpression is defined
if bkConfig.cronExpression == "" { if bkConfig.cronExpression == "" {
multiBackupTask(conf.Databases, bkConfig) multiBackupTask(conf.Databases, bkConfig)
} else { } else {
// Check if cronExpression is valid // Check if cronExpression is valid
if utils.IsValidCronExpression(bkConfig.cronExpression) { if utils.IsValidCronExpression(bkConfig.cronExpression) {
utils.Info("Running backup in Scheduled mode") logger.Info("Running backup in Scheduled mode")
utils.Info("Backup cron expression: %s", bkConfig.cronExpression) logger.Info("Backup cron expression: %s", bkConfig.cronExpression)
utils.Info("The next scheduled time is: %v", utils.CronNextTime(bkConfig.cronExpression).Format(timeFormat)) logger.Info("The next scheduled time is: %v", utils.CronNextTime(bkConfig.cronExpression).Format(timeFormat))
utils.Info("Storage type %s ", bkConfig.storage) logger.Info("Storage type %s ", bkConfig.storage)
// Test backup // Test backup
utils.Info("Testing backup configurations...") logger.Info("Testing backup configurations...")
multiBackupTask(conf.Databases, bkConfig) for _, db := range conf.Databases {
utils.Info("Testing backup configurations...done") testDatabaseConnection(getDatabase(db))
utils.Info("Creating backup job...") }
logger.Info("Testing backup configurations...done")
logger.Info("Creating backup job...")
// Create a new cron instance // Create a new cron instance
c := cron.New() c := cron.New()
_, err := c.AddFunc(bkConfig.cronExpression, func() { _, err := c.AddFunc(bkConfig.cronExpression, func() {
multiBackupTask(conf.Databases, bkConfig) multiBackupTask(conf.Databases, bkConfig)
utils.Info("Next backup time is: %v", utils.CronNextTime(bkConfig.cronExpression).Format(timeFormat)) logger.Info("Next backup time is: %v", utils.CronNextTime(bkConfig.cronExpression).Format(timeFormat))
}) })
if err != nil { if err != nil {
@@ -149,13 +172,13 @@ func startMultiBackup(bkConfig *BackupConfig, configFile string) {
} }
// Start the cron scheduler // Start the cron scheduler
c.Start() c.Start()
utils.Info("Creating backup job...done") logger.Info("Creating backup job...done")
utils.Info("Backup job started") logger.Info("Backup job started")
defer c.Stop() defer c.Stop()
select {} select {}
} else { } else {
utils.Fatal("Cron expression is not valid: %s", bkConfig.cronExpression) logger.Fatal("Cron expression is not valid: %s", bkConfig.cronExpression)
} }
} }
@@ -163,10 +186,9 @@ func startMultiBackup(bkConfig *BackupConfig, configFile string) {
// BackupDatabase backup database // BackupDatabase backup database
func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool) { func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool) {
storagePath = os.Getenv("STORAGE_PATH") storagePath = os.Getenv("STORAGE_PATH")
utils.Info("Starting database backup...") logger.Info("Starting database backup...")
err := os.Setenv("MYSQL_PWD", db.dbPassword) err := os.Setenv("MYSQL_PWD", db.dbPassword)
if err != nil { if err != nil {
@@ -174,7 +196,7 @@ func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool
} }
testDatabaseConnection(db) testDatabaseConnection(db)
// Backup Database database // Backup Database database
utils.Info("Backing up database...") logger.Info("Backing up database...")
// Verify is compression is disabled // Verify is compression is disabled
if disableCompression { if disableCompression {
@@ -187,21 +209,26 @@ func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool
) )
output, err := cmd.Output() output, err := cmd.Output()
if err != nil { if err != nil {
log.Fatal(err) logger.Fatal(err.Error())
} }
// save output // save output
file, err := os.Create(filepath.Join(tmpPath, backupFileName)) file, err := os.Create(filepath.Join(tmpPath, backupFileName))
if err != nil { if err != nil {
log.Fatal(err) logger.Fatal(err.Error())
} }
defer file.Close() defer func(file *os.File) {
err := file.Close()
if err != nil {
logger.Fatal(err.Error())
}
}(file)
_, err = file.Write(output) _, err = file.Write(output)
if err != nil { if err != nil {
log.Fatal(err) logger.Fatal(err.Error())
} }
utils.Info("Database has been backed up") logger.Info("Database has been backed up")
} else { } else {
// Execute mysqldump // Execute mysqldump
@@ -213,9 +240,9 @@ func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool
gzipCmd := exec.Command("gzip") gzipCmd := exec.Command("gzip")
gzipCmd.Stdin = stdout gzipCmd.Stdin = stdout
gzipCmd.Stdout, err = os.Create(filepath.Join(tmpPath, backupFileName)) gzipCmd.Stdout, err = os.Create(filepath.Join(tmpPath, backupFileName))
gzipCmd.Start() err = gzipCmd.Start()
if err != nil { if err != nil {
log.Fatal(err) return
} }
if err := cmd.Run(); err != nil { if err := cmd.Run(); err != nil {
log.Fatal(err) log.Fatal(err)
@@ -223,12 +250,12 @@ func BackupDatabase(db *dbConfig, backupFileName string, disableCompression bool
if err := gzipCmd.Wait(); err != nil { if err := gzipCmd.Wait(); err != nil {
log.Fatal(err) log.Fatal(err)
} }
utils.Info("Database has been backed up") logger.Info("Database has been backed up")
} }
} }
func localBackup(db *dbConfig, config *BackupConfig) { func localBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to local storage") logger.Info("Backup database to local storage")
startTime = time.Now().Format(utils.TimeFormat()) startTime = time.Now().Format(utils.TimeFormat())
BackupDatabase(db, config.backupFileName, disableCompression) BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName finalFileName := config.backupFileName
@@ -238,19 +265,19 @@ func localBackup(db *dbConfig, config *BackupConfig) {
} }
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName)) fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil { if err != nil {
utils.Error("Error: %s", err) logger.Error("Error: %s", err)
} }
backupSize = fileInfo.Size() backupSize = fileInfo.Size()
utils.Info("Backup name is %s", finalFileName) logger.Info("Backup name is %s", finalFileName)
localStorage := local.NewStorage(local.Config{ localStorage := local.NewStorage(local.Config{
LocalPath: tmpPath, LocalPath: tmpPath,
RemotePath: storagePath, RemotePath: storagePath,
}) })
err = localStorage.Copy(finalFileName) err = localStorage.Copy(finalFileName)
if err != nil { if err != nil {
utils.Fatal("Error copying backup file: %s", err) logger.Fatal("Error copying backup file: %s", err)
} }
utils.Info("Backup saved in %s", filepath.Join(storagePath, finalFileName)) logger.Info("Backup saved in %s", filepath.Join(storagePath, finalFileName))
// Send notification // Send notification
utils.NotifySuccess(&utils.NotificationData{ utils.NotifySuccess(&utils.NotificationData{
File: finalFileName, File: finalFileName,
@@ -265,249 +292,40 @@ func localBackup(db *dbConfig, config *BackupConfig) {
if config.prune { if config.prune {
err = localStorage.Prune(config.backupRetention) err = localStorage.Prune(config.backupRetention)
if err != nil { if err != nil {
utils.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err) logger.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
} }
} }
// Delete temp // Delete temp
deleteTemp() deleteTemp()
utils.Info("Backup completed successfully") logger.Info("Backup completed successfully")
}
func s3Backup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to s3 storage")
startTime = time.Now().Format(utils.TimeFormat())
//Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
utils.Info("Uploading backup archive to remote storage S3 ... ")
awsConfig := initAWSConfig()
if config.remotePath == "" {
config.remotePath = awsConfig.remotePath
}
utils.Info("Backup name is %s", finalFileName)
s3Storage, err := s3.NewStorage(s3.Config{
Endpoint: awsConfig.endpoint,
Bucket: awsConfig.bucket,
AccessKey: awsConfig.accessKey,
SecretKey: awsConfig.secretKey,
Region: awsConfig.region,
DisableSsl: awsConfig.disableSsl,
ForcePathStyle: awsConfig.forcePathStyle,
RemotePath: awsConfig.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating s3 storage: %s", err)
}
err = s3Storage.Copy(finalFileName)
if err != nil {
utils.Fatal("Error copying backup file: %s", err)
}
//Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
utils.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
//Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, config.backupFileName))
if err != nil {
fmt.Println("Error deleting file: ", err)
}
// Delete old backup
if config.prune {
err := s3Storage.Prune(config.backupRetention)
if err != nil {
utils.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
utils.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
utils.Info("Uploading backup archive to remote storage S3 ... done ")
//Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
//Delete temp
deleteTemp()
utils.Info("Backup completed successfully")
}
func sshBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to Remote server")
startTime = time.Now().Format(utils.TimeFormat())
//Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
utils.Info("Uploading backup archive to remote storage ... ")
utils.Info("Backup name is %s", finalFileName)
sshConfig, err := loadSSHConfig()
if err != nil {
utils.Fatal("Error loading ssh config: %s", err)
}
sshStorage, err := ssh.NewStorage(ssh.Config{
Host: sshConfig.hostName,
Port: sshConfig.port,
User: sshConfig.user,
Password: sshConfig.password,
IdentifyFile: sshConfig.identifyFile,
RemotePath: config.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating SSH storage: %s", err)
}
err = sshStorage.Copy(finalFileName)
if err != nil {
utils.Fatal("Error copying backup file: %s", err)
}
//Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
utils.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
utils.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
//Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, finalFileName))
if err != nil {
utils.Error("Error deleting file: %v", err)
}
if config.prune {
err := sshStorage.Prune(config.backupRetention)
if err != nil {
utils.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
utils.Info("Uploading backup archive to remote storage ... done ")
//Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
//Delete temp
deleteTemp()
utils.Info("Backup completed successfully")
}
func ftpBackup(db *dbConfig, config *BackupConfig) {
utils.Info("Backup database to the remote FTP server")
startTime = time.Now().Format(utils.TimeFormat())
//Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
utils.Info("Uploading backup archive to the remote FTP server ... ")
utils.Info("Backup name is %s", finalFileName)
ftpConfig := loadFtpConfig()
ftpStorage, err := ftp.NewStorage(ftp.Config{
Host: ftpConfig.host,
Port: ftpConfig.port,
User: ftpConfig.user,
Password: ftpConfig.password,
RemotePath: config.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating SSH storage: %s", err)
}
err = ftpStorage.Copy(finalFileName)
if err != nil {
utils.Fatal("Error copying backup file: %s", err)
}
utils.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
//Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
utils.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
//Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, finalFileName))
if err != nil {
utils.Error("Error deleting file: %v", err)
}
if config.prune {
err := ftpStorage.Prune(config.backupRetention)
if err != nil {
utils.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
utils.Info("Uploading backup archive to the remote FTP server ... done ")
//Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
//Delete temp
deleteTemp()
utils.Info("Backup completed successfully")
} }
func encryptBackup(config *BackupConfig) { func encryptBackup(config *BackupConfig) {
backupFile, err := os.ReadFile(filepath.Join(tmpPath, config.backupFileName)) backupFile, err := os.ReadFile(filepath.Join(tmpPath, config.backupFileName))
outputFile := fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension) outputFile := fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension)
if err != nil { if err != nil {
utils.Fatal("Error reading backup file: %s ", err) logger.Fatal("Error reading backup file: %s ", err)
} }
if config.usingKey { if config.usingKey {
utils.Info("Encrypting backup using public key...") logger.Info("Encrypting backup using public key...")
pubKey, err := os.ReadFile(config.publicKey) pubKey, err := os.ReadFile(config.publicKey)
if err != nil { if err != nil {
utils.Fatal("Error reading public key: %s ", err) logger.Fatal("Error reading public key: %s ", err)
} }
err = encryptor.EncryptWithPublicKey(backupFile, fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension), pubKey) err = encryptor.EncryptWithPublicKey(backupFile, fmt.Sprintf("%s.%s", filepath.Join(tmpPath, config.backupFileName), gpgExtension), pubKey)
if err != nil { if err != nil {
utils.Fatal("Error encrypting backup file: %v ", err) logger.Fatal("Error encrypting backup file: %v ", err)
} }
utils.Info("Encrypting backup using public key...done") logger.Info("Encrypting backup using public key...done")
} else if config.passphrase != "" { } else if config.passphrase != "" {
utils.Info("Encrypting backup using passphrase...") logger.Info("Encrypting backup using passphrase...")
err := encryptor.Encrypt(backupFile, outputFile, config.passphrase) err := encryptor.Encrypt(backupFile, outputFile, config.passphrase)
if err != nil { if err != nil {
utils.Fatal("error during encrypting backup %v", err) logger.Fatal("error during encrypting backup %v", err)
} }
utils.Info("Encrypting backup using passphrase...done") logger.Info("Encrypting backup using passphrase...done")
} }

67
internal/config.go
View File

@@ -1,13 +1,32 @@
// Package internal / // Package internal /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal package internal
import ( import (
"fmt" "fmt"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"os" "os"
@@ -65,6 +84,11 @@ type FTPConfig struct {
port string port string
remotePath string remotePath string
} }
type AzureConfig struct {
accountName string
accountKey string
containerName string
}
// SSHConfig holds the SSH connection details // SSHConfig holds the SSH connection details
type SSHConfig struct { type SSHConfig struct {
@@ -97,8 +121,8 @@ func initDbConfig(cmd *cobra.Command) *dbConfig {
err := utils.CheckEnvVars(dbHVars) err := utils.CheckEnvVars(dbHVars)
if err != nil { if err != nil {
utils.Error("Please make sure all required environment variables for database are set") logger.Error("Please make sure all required environment variables for database are set")
utils.Fatal("Error checking environment variables: %s", err) logger.Fatal("Error checking environment variables: %s", err)
} }
return &dConf return &dConf
} }
@@ -140,11 +164,26 @@ func loadFtpConfig() *FTPConfig {
fConfig.remotePath = os.Getenv("REMOTE_PATH") fConfig.remotePath = os.Getenv("REMOTE_PATH")
err := utils.CheckEnvVars(ftpVars) err := utils.CheckEnvVars(ftpVars)
if err != nil { if err != nil {
utils.Error("Please make sure all required environment variables for FTP are set") logger.Error("Please make sure all required environment variables for FTP are set")
utils.Fatal("Error missing environment variables: %s", err) logger.Fatal("Error missing environment variables: %s", err)
} }
return &fConfig return &fConfig
} }
func loadAzureConfig() *AzureConfig {
// Initialize data configs
aConfig := AzureConfig{}
aConfig.containerName = os.Getenv("AZURE_STORAGE_CONTAINER_NAME")
aConfig.accountName = os.Getenv("AZURE_STORAGE_ACCOUNT_NAME")
aConfig.accountKey = os.Getenv("AZURE_STORAGE_ACCOUNT_KEY")
err := utils.CheckEnvVars(azureVars)
if err != nil {
logger.Error("Please make sure all required environment variables for Azure Blob storage are set")
logger.Fatal("Error missing environment variables: %s", err)
}
return &aConfig
}
func initAWSConfig() *AWSConfig { func initAWSConfig() *AWSConfig {
// Initialize AWS configs // Initialize AWS configs
aConfig := AWSConfig{} aConfig := AWSConfig{}
@@ -167,8 +206,8 @@ func initAWSConfig() *AWSConfig {
aConfig.forcePathStyle = forcePathStyle aConfig.forcePathStyle = forcePathStyle
err = utils.CheckEnvVars(awsVars) err = utils.CheckEnvVars(awsVars)
if err != nil { if err != nil {
utils.Error("Please make sure all required environment variables for AWS S3 are set") logger.Error("Please make sure all required environment variables for AWS S3 are set")
utils.Fatal("Error checking environment variables: %s", err) logger.Fatal("Error checking environment variables: %s", err)
} }
return &aConfig return &aConfig
} }
@@ -258,15 +297,15 @@ func initRestoreConfig(cmd *cobra.Command) *RestoreConfig {
func initTargetDbConfig() *targetDbConfig { func initTargetDbConfig() *targetDbConfig {
tdbConfig := targetDbConfig{} tdbConfig := targetDbConfig{}
tdbConfig.targetDbHost = os.Getenv("TARGET_DB_HOST") tdbConfig.targetDbHost = os.Getenv("TARGET_DB_HOST")
tdbConfig.targetDbPort = utils.EnvWithDefault("TARGET_DB_PORT", "5432") tdbConfig.targetDbPort = utils.EnvWithDefault("TARGET_DB_PORT", "3306")
tdbConfig.targetDbName = os.Getenv("TARGET_DB_NAME") tdbConfig.targetDbName = os.Getenv("TARGET_DB_NAME")
tdbConfig.targetDbUserName = os.Getenv("TARGET_DB_USERNAME") tdbConfig.targetDbUserName = os.Getenv("TARGET_DB_USERNAME")
tdbConfig.targetDbPassword = os.Getenv("TARGET_DB_PASSWORD") tdbConfig.targetDbPassword = os.Getenv("TARGET_DB_PASSWORD")
err := utils.CheckEnvVars(tdbRVars) err := utils.CheckEnvVars(tdbRVars)
if err != nil { if err != nil {
utils.Error("Please make sure all required environment variables for the target database are set") logger.Error("Please make sure all required environment variables for the target database are set")
utils.Fatal("Error checking target database environment variables: %s", err) logger.Fatal("Error checking target database environment variables: %s", err)
} }
return &tdbConfig return &tdbConfig
} }

46
internal/helper.go
View File

@@ -1,14 +1,33 @@
// Package internal / // Package internal /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal package internal
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"os" "os"
@@ -18,13 +37,14 @@ import (
) )
func intro() { func intro() {
utils.Info("Starting MySQL Backup...") fmt.Println("Starting MySQL Backup...")
utils.Info("Copyright (c) 2024 Jonas Kaninda ") fmt.Printf("Version: %s\n", utils.Version)
fmt.Println("Copyright (c) 2024 Jonas Kaninda")
} }
// copyToTmp copy file to temporary directory // copyToTmp copy file to temporary directory
func deleteTemp() { func deleteTemp() {
utils.Info("Deleting %s ...", tmpPath) logger.Info("Deleting %s ...", tmpPath)
err := filepath.Walk(tmpPath, func(path string, info os.FileInfo, err error) error { err := filepath.Walk(tmpPath, func(path string, info os.FileInfo, err error) error {
if err != nil { if err != nil {
return err return err
@@ -40,9 +60,9 @@ func deleteTemp() {
return nil return nil
}) })
if err != nil { if err != nil {
utils.Error("Error deleting files: %v", err) logger.Error("Error deleting files: %v", err)
} else { } else {
utils.Info("Deleting %s ... done", tmpPath) logger.Info("Deleting %s ... done", tmpPath)
} }
} }
@@ -52,7 +72,7 @@ func testDatabaseConnection(db *dbConfig) {
if err != nil { if err != nil {
return return
} }
utils.Info("Connecting to %s database ...", db.dbName) logger.Info("Connecting to %s database ...", db.dbName)
cmd := exec.Command("mysql", "-h", db.dbHost, "-P", db.dbPort, "-u", db.dbUserName, db.dbName, "-e", "quit") cmd := exec.Command("mysql", "-h", db.dbHost, "-P", db.dbPort, "-u", db.dbUserName, db.dbName, "-e", "quit")
// Capture the output // Capture the output
var out bytes.Buffer var out bytes.Buffer
@@ -60,10 +80,10 @@ func testDatabaseConnection(db *dbConfig) {
cmd.Stderr = &out cmd.Stderr = &out
err = cmd.Run() err = cmd.Run()
if err != nil { if err != nil {
utils.Fatal("Error testing database connection: %v\nOutput: %s", err, out.String()) logger.Fatal("Error testing database connection: %v\nOutput: %s", err, out.String())
} }
utils.Info("Successfully connected to %s database", db.dbName) logger.Info("Successfully connected to %s database", db.dbName)
} }

38
internal/migrate.go
View File

@@ -1,21 +1,39 @@
// Package internal / // Package internal /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal package internal
import ( import (
"fmt" "fmt"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"time" "time"
) )
func StartMigration(cmd *cobra.Command) { func StartMigration(cmd *cobra.Command) {
intro() intro()
utils.Info("Starting database migration...") logger.Info("Starting database migration...")
// Get DB config // Get DB config
dbConf = initDbConfig(cmd) dbConf = initDbConfig(cmd)
targetDbConf = initTargetDbConfig() targetDbConf = initTargetDbConfig()
@@ -35,8 +53,8 @@ func StartMigration(cmd *cobra.Command) {
// Backup source Database // Backup source Database
BackupDatabase(dbConf, backupFileName, true) BackupDatabase(dbConf, backupFileName, true)
// Restore source database into target database // Restore source database into target database
utils.Info("Restoring [%s] database into [%s] database...", dbConf.dbName, targetDbConf.targetDbName) logger.Info("Restoring [%s] database into [%s] database...", dbConf.dbName, targetDbConf.targetDbName)
RestoreDatabase(&newDbConfig, conf) RestoreDatabase(&newDbConfig, conf)
utils.Info("[%s] database has been restored into [%s] database", dbConf.dbName, targetDbConf.targetDbName) logger.Info("[%s] database has been restored into [%s] database", dbConf.dbName, targetDbConf.targetDbName)
utils.Info("Database migration completed.") logger.Info("Database migration completed.")
} }

218
internal/remote.go Normal file
View File

@@ -0,0 +1,218 @@
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal
import (
"fmt"
"github.com/jkaninda/go-storage/pkg/ftp"
"github.com/jkaninda/go-storage/pkg/ssh"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils"
"os"
"path/filepath"
"time"
)
func sshBackup(db *dbConfig, config *BackupConfig) {
logger.Info("Backup database to Remote server")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
logger.Info("Uploading backup archive to remote storage ... ")
logger.Info("Backup name is %s", finalFileName)
sshConfig, err := loadSSHConfig()
if err != nil {
logger.Fatal("Error loading ssh config: %s", err)
}
sshStorage, err := ssh.NewStorage(ssh.Config{
Host: sshConfig.hostName,
Port: sshConfig.port,
User: sshConfig.user,
Password: sshConfig.password,
RemotePath: config.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = sshStorage.Copy(finalFileName)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
// Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
logger.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
// Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error deleting file: %v", err)
}
if config.prune {
err := sshStorage.Prune(config.backupRetention)
if err != nil {
logger.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
logger.Info("Uploading backup archive to remote storage ... done ")
// Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
// Delete temp
deleteTemp()
logger.Info("Backup completed successfully")
}
func ftpBackup(db *dbConfig, config *BackupConfig) {
logger.Info("Backup database to the remote FTP server")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
logger.Info("Uploading backup archive to the remote FTP server ... ")
logger.Info("Backup name is %s", finalFileName)
ftpConfig := loadFtpConfig()
ftpStorage, err := ftp.NewStorage(ftp.Config{
Host: ftpConfig.host,
Port: ftpConfig.port,
User: ftpConfig.user,
Password: ftpConfig.password,
RemotePath: config.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = ftpStorage.Copy(finalFileName)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
logger.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
// Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
// Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error deleting file: %v", err)
}
if config.prune {
err := ftpStorage.Prune(config.backupRetention)
if err != nil {
logger.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
logger.Info("Uploading backup archive to the remote FTP server ... done ")
// Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
// Delete temp
deleteTemp()
logger.Info("Backup completed successfully")
}
func remoteRestore(db *dbConfig, conf *RestoreConfig) {
logger.Info("Restore database from remote server")
sshConfig, err := loadSSHConfig()
if err != nil {
logger.Fatal("Error loading ssh config: %s", err)
}
sshStorage, err := ssh.NewStorage(ssh.Config{
Host: sshConfig.hostName,
Port: sshConfig.port,
User: sshConfig.user,
Password: sshConfig.password,
IdentifyFile: sshConfig.identifyFile,
RemotePath: conf.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = sshStorage.CopyFrom(conf.file)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
RestoreDatabase(db, conf)
}
func ftpRestore(db *dbConfig, conf *RestoreConfig) {
logger.Info("Restore database from FTP server")
ftpConfig := loadFtpConfig()
ftpStorage, err := ftp.NewStorage(ftp.Config{
Host: ftpConfig.host,
Port: ftpConfig.port,
User: ftpConfig.user,
Password: ftpConfig.password,
RemotePath: conf.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating SSH storage: %s", err)
}
err = ftpStorage.CopyFrom(conf.file)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
RestoreDatabase(db, conf)
}

156
internal/restore.go
View File

@@ -1,17 +1,33 @@
// Package internal / // Package internal /
/*****
@author Jonas Kaninda
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda
**/
package internal package internal
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
import ( import (
"github.com/jkaninda/encryptor" "github.com/jkaninda/encryptor"
"github.com/jkaninda/go-storage/pkg/ftp"
"github.com/jkaninda/go-storage/pkg/local" "github.com/jkaninda/go-storage/pkg/local"
"github.com/jkaninda/go-storage/pkg/s3" "github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/go-storage/pkg/ssh"
"github.com/jkaninda/mysql-bkup/utils" "github.com/jkaninda/mysql-bkup/utils"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"os" "os"
@@ -28,137 +44,69 @@ func StartRestore(cmd *cobra.Command) {
case "local": case "local":
localRestore(dbConf, restoreConf) localRestore(dbConf, restoreConf)
case "s3", "S3": case "s3", "S3":
restoreFromS3(dbConf, restoreConf) s3Restore(dbConf, restoreConf)
case "ssh", "SSH", "remote": case "ssh", "SSH", "remote":
restoreFromRemote(dbConf, restoreConf) remoteRestore(dbConf, restoreConf)
case "ftp", "FTP": case "ftp", "FTP":
restoreFromFTP(dbConf, restoreConf) ftpRestore(dbConf, restoreConf)
case "azure":
azureRestore(dbConf, restoreConf)
default: default:
localRestore(dbConf, restoreConf) localRestore(dbConf, restoreConf)
} }
} }
func localRestore(dbConf *dbConfig, restoreConf *RestoreConfig) { func localRestore(dbConf *dbConfig, restoreConf *RestoreConfig) {
utils.Info("Restore database from local") logger.Info("Restore database from local")
localStorage := local.NewStorage(local.Config{ localStorage := local.NewStorage(local.Config{
RemotePath: storagePath, RemotePath: storagePath,
LocalPath: tmpPath, LocalPath: tmpPath,
}) })
err := localStorage.CopyFrom(restoreConf.file) err := localStorage.CopyFrom(restoreConf.file)
if err != nil { if err != nil {
utils.Fatal("Error copying backup file: %s", err) logger.Fatal("Error copying backup file: %s", err)
} }
RestoreDatabase(dbConf, restoreConf) RestoreDatabase(dbConf, restoreConf)
} }
func restoreFromS3(db *dbConfig, conf *RestoreConfig) {
utils.Info("Restore database from s3")
awsConfig := initAWSConfig()
if conf.remotePath == "" {
conf.remotePath = awsConfig.remotePath
}
s3Storage, err := s3.NewStorage(s3.Config{
Endpoint: awsConfig.endpoint,
Bucket: awsConfig.bucket,
AccessKey: awsConfig.accessKey,
SecretKey: awsConfig.secretKey,
Region: awsConfig.region,
DisableSsl: awsConfig.disableSsl,
ForcePathStyle: awsConfig.forcePathStyle,
RemotePath: awsConfig.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating s3 storage: %s", err)
}
err = s3Storage.CopyFrom(conf.file)
if err != nil {
utils.Fatal("Error download file from S3 storage: %s", err)
}
RestoreDatabase(db, conf)
}
func restoreFromRemote(db *dbConfig, conf *RestoreConfig) {
utils.Info("Restore database from remote server")
sshConfig, err := loadSSHConfig()
if err != nil {
utils.Fatal("Error loading ssh config: %s", err)
}
sshStorage, err := ssh.NewStorage(ssh.Config{
Host: sshConfig.hostName,
Port: sshConfig.port,
User: sshConfig.user,
Password: sshConfig.password,
RemotePath: conf.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating SSH storage: %s", err)
}
err = sshStorage.CopyFrom(conf.file)
if err != nil {
utils.Fatal("Error copying backup file: %s", err)
}
RestoreDatabase(db, conf)
}
func restoreFromFTP(db *dbConfig, conf *RestoreConfig) {
utils.Info("Restore database from FTP server")
ftpConfig := loadFtpConfig()
ftpStorage, err := ftp.NewStorage(ftp.Config{
Host: ftpConfig.host,
Port: ftpConfig.port,
User: ftpConfig.user,
Password: ftpConfig.password,
RemotePath: conf.remotePath,
LocalPath: tmpPath,
})
if err != nil {
utils.Fatal("Error creating SSH storage: %s", err)
}
err = ftpStorage.CopyFrom(conf.file)
if err != nil {
utils.Fatal("Error copying backup file: %s", err)
}
RestoreDatabase(db, conf)
}
// RestoreDatabase restore database // RestoreDatabase restore database
func RestoreDatabase(db *dbConfig, conf *RestoreConfig) { func RestoreDatabase(db *dbConfig, conf *RestoreConfig) {
if conf.file == "" { if conf.file == "" {
utils.Fatal("Error, file required") logger.Fatal("Error, file required")
} }
extension := filepath.Ext(filepath.Join(tmpPath, conf.file)) extension := filepath.Ext(filepath.Join(tmpPath, conf.file))
rFile, err := os.ReadFile(filepath.Join(tmpPath, conf.file)) rFile, err := os.ReadFile(filepath.Join(tmpPath, conf.file))
outputFile := RemoveLastExtension(filepath.Join(tmpPath, conf.file)) outputFile := RemoveLastExtension(filepath.Join(tmpPath, conf.file))
if err != nil { if err != nil {
utils.Fatal("Error reading backup file: %s ", err) logger.Fatal("Error reading backup file: %s ", err)
} }
if extension == ".gpg" { if extension == ".gpg" {
if conf.usingKey { if conf.usingKey {
utils.Info("Decrypting backup using private key...") logger.Info("Decrypting backup using private key...")
utils.Warn("Backup decryption using a private key is not fully supported") logger.Warn("Backup decryption using a private key is not fully supported")
prKey, err := os.ReadFile(conf.privateKey) prKey, err := os.ReadFile(conf.privateKey)
if err != nil { if err != nil {
utils.Fatal("Error reading public key: %s ", err) logger.Fatal("Error reading public key: %s ", err)
} }
err = encryptor.DecryptWithPrivateKey(rFile, outputFile, prKey, conf.passphrase) err = encryptor.DecryptWithPrivateKey(rFile, outputFile, prKey, conf.passphrase)
if err != nil { if err != nil {
utils.Fatal("error during decrypting backup %v", err) logger.Fatal("error during decrypting backup %v", err)
} }
utils.Info("Decrypting backup using private key...done") logger.Info("Decrypting backup using private key...done")
} else { } else {
if conf.passphrase == "" { if conf.passphrase == "" {
utils.Error("Error, passphrase or private key required") logger.Error("Error, passphrase or private key required")
utils.Fatal("Your file seems to be a GPG file.\nYou need to provide GPG keys. GPG_PASSPHRASE or GPG_PRIVATE_KEY environment variable is required.") logger.Fatal("Your file seems to be a GPG file.\nYou need to provide GPG keys. GPG_PASSPHRASE or GPG_PRIVATE_KEY environment variable is required.")
} else { } else {
utils.Info("Decrypting backup using passphrase...") logger.Info("Decrypting backup using passphrase...")
// decryptWithGPG file // decryptWithGPG file
err := encryptor.Decrypt(rFile, outputFile, conf.passphrase) err := encryptor.Decrypt(rFile, outputFile, conf.passphrase)
if err != nil { if err != nil {
utils.Fatal("Error decrypting file %s %v", file, err) logger.Fatal("Error decrypting file %s %v", file, err)
} }
utils.Info("Decrypting backup using passphrase...done") logger.Info("Decrypting backup using passphrase...done")
// Update file name // Update file name
conf.file = RemoveLastExtension(file) conf.file = RemoveLastExtension(file)
} }
@@ -172,7 +120,7 @@ func RestoreDatabase(db *dbConfig, conf *RestoreConfig) {
return return
} }
testDatabaseConnection(db) testDatabaseConnection(db)
utils.Info("Restoring database...") logger.Info("Restoring database...")
extension := filepath.Ext(filepath.Join(tmpPath, conf.file)) extension := filepath.Ext(filepath.Join(tmpPath, conf.file))
// Restore from compressed file / .sql.gz // Restore from compressed file / .sql.gz
@@ -180,10 +128,10 @@ func RestoreDatabase(db *dbConfig, conf *RestoreConfig) {
str := "zcat " + filepath.Join(tmpPath, conf.file) + " | mysql -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName str := "zcat " + filepath.Join(tmpPath, conf.file) + " | mysql -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName
_, err := exec.Command("sh", "-c", str).Output() _, err := exec.Command("sh", "-c", str).Output()
if err != nil { if err != nil {
utils.Fatal("Error, in restoring the database %v", err) logger.Fatal("Error, in restoring the database %v", err)
} }
utils.Info("Restoring database... done") logger.Info("Restoring database... done")
utils.Info("Database has been restored") logger.Info("Database has been restored")
// Delete temp // Delete temp
deleteTemp() deleteTemp()
@@ -192,17 +140,17 @@ func RestoreDatabase(db *dbConfig, conf *RestoreConfig) {
str := "cat " + filepath.Join(tmpPath, conf.file) + " | mysql -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName str := "cat " + filepath.Join(tmpPath, conf.file) + " | mysql -h " + db.dbHost + " -P " + db.dbPort + " -u " + db.dbUserName + " " + db.dbName
_, err := exec.Command("sh", "-c", str).Output() _, err := exec.Command("sh", "-c", str).Output()
if err != nil { if err != nil {
utils.Fatal("Error in restoring the database %v", err) logger.Fatal("Error in restoring the database %v", err)
} }
utils.Info("Restoring database... done") logger.Info("Restoring database... done")
utils.Info("Database has been restored") logger.Info("Database has been restored")
// Delete temp // Delete temp
deleteTemp() deleteTemp()
} else { } else {
utils.Fatal("Unknown file extension %s", extension) logger.Fatal("Unknown file extension %s", extension)
} }
} else { } else {
utils.Fatal("File not found in %s", filepath.Join(tmpPath, conf.file)) logger.Fatal("File not found in %s", filepath.Join(tmpPath, conf.file))
} }
} }

135
internal/s3.go Normal file
View File

@@ -0,0 +1,135 @@
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal
import (
"fmt"
"github.com/jkaninda/go-storage/pkg/s3"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/jkaninda/mysql-bkup/utils"
"os"
"path/filepath"
"time"
)
func s3Backup(db *dbConfig, config *BackupConfig) {
logger.Info("Backup database to s3 storage")
startTime = time.Now().Format(utils.TimeFormat())
// Backup database
BackupDatabase(db, config.backupFileName, disableCompression)
finalFileName := config.backupFileName
if config.encryption {
encryptBackup(config)
finalFileName = fmt.Sprintf("%s.%s", config.backupFileName, "gpg")
}
logger.Info("Uploading backup archive to remote storage S3 ... ")
awsConfig := initAWSConfig()
if config.remotePath == "" {
config.remotePath = awsConfig.remotePath
}
logger.Info("Backup name is %s", finalFileName)
s3Storage, err := s3.NewStorage(s3.Config{
Endpoint: awsConfig.endpoint,
Bucket: awsConfig.bucket,
AccessKey: awsConfig.accessKey,
SecretKey: awsConfig.secretKey,
Region: awsConfig.region,
DisableSsl: awsConfig.disableSsl,
ForcePathStyle: awsConfig.forcePathStyle,
RemotePath: awsConfig.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating s3 storage: %s", err)
}
err = s3Storage.Copy(finalFileName)
if err != nil {
logger.Fatal("Error copying backup file: %s", err)
}
// Get backup info
fileInfo, err := os.Stat(filepath.Join(tmpPath, finalFileName))
if err != nil {
logger.Error("Error: %s", err)
}
backupSize = fileInfo.Size()
// Delete backup file from tmp folder
err = utils.DeleteFile(filepath.Join(tmpPath, config.backupFileName))
if err != nil {
fmt.Println("Error deleting file: ", err)
}
// Delete old backup
if config.prune {
err := s3Storage.Prune(config.backupRetention)
if err != nil {
logger.Fatal("Error deleting old backup from %s storage: %s ", config.storage, err)
}
}
logger.Info("Backup saved in %s", filepath.Join(config.remotePath, finalFileName))
logger.Info("Uploading backup archive to remote storage S3 ... done ")
// Send notification
utils.NotifySuccess(&utils.NotificationData{
File: finalFileName,
BackupSize: backupSize,
Database: db.dbName,
Storage: config.storage,
BackupLocation: filepath.Join(config.remotePath, finalFileName),
StartTime: startTime,
EndTime: time.Now().Format(utils.TimeFormat()),
})
// Delete temp
deleteTemp()
logger.Info("Backup completed successfully")
}
func s3Restore(db *dbConfig, conf *RestoreConfig) {
logger.Info("Restore database from s3")
awsConfig := initAWSConfig()
if conf.remotePath == "" {
conf.remotePath = awsConfig.remotePath
}
s3Storage, err := s3.NewStorage(s3.Config{
Endpoint: awsConfig.endpoint,
Bucket: awsConfig.bucket,
AccessKey: awsConfig.accessKey,
SecretKey: awsConfig.secretKey,
Region: awsConfig.region,
DisableSsl: awsConfig.disableSsl,
ForcePathStyle: awsConfig.forcePathStyle,
RemotePath: awsConfig.remotePath,
LocalPath: tmpPath,
})
if err != nil {
logger.Fatal("Error creating s3 storage: %s", err)
}
err = s3Storage.CopyFrom(conf.file)
if err != nil {
logger.Fatal("Error download file from S3 storage: %s", err)
}
RestoreDatabase(db, conf)
}

33
internal/var.go
View File

@@ -1,9 +1,27 @@
// Package internal / // Package internal /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package internal package internal
const tmpPath = "/tmp/backup" const tmpPath = "/tmp/backup"
@@ -56,6 +74,11 @@ var ftpVars = []string{
"FTP_PASSWORD", "FTP_PASSWORD",
"FTP_PORT", "FTP_PORT",
} }
var azureVars = []string{
"AZURE_STORAGE_CONTAINER_NAME",
"AZURE_STORAGE_ACCOUNT_NAME",
"AZURE_STORAGE_ACCOUNT_KEY",
}
// AwsVars Required environment variables for AWS S3 storage // AwsVars Required environment variables for AWS S3 storage
var awsVars = []string{ var awsVars = []string{

28
main.go
View File

@@ -1,9 +1,27 @@
// Package main / // Package main /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package main package main
import "github.com/jkaninda/mysql-bkup/cmd" import "github.com/jkaninda/mysql-bkup/cmd"

97
pkg/logger/logger.go Normal file
View File

@@ -0,0 +1,97 @@
package logger
import (
"fmt"
"log"
"os"
"runtime"
"strings"
)
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
// Info returns info log
func Info(msg string, args ...interface{}) {
log.SetOutput(getStd("/dev/stdout"))
logWithCaller("INFO", msg, args...)
}
// Warn returns warning log
func Warn(msg string, args ...interface{}) {
log.SetOutput(getStd("/dev/stdout"))
logWithCaller("WARN", msg, args...)
}
// Error logs error messages
func Error(msg string, args ...interface{}) {
log.SetOutput(getStd("/dev/stderr"))
logWithCaller("ERROR", msg, args...)
}
func Fatal(msg string, args ...interface{}) {
log.SetOutput(os.Stdout)
logWithCaller("ERROR", msg, args...)
os.Exit(1)
}
// Helper function to format and log messages with file and line number
func logWithCaller(level, msg string, args ...interface{}) {
// Format message if there are additional arguments
formattedMessage := msg
if len(args) > 0 {
formattedMessage = fmt.Sprintf(msg, args...)
}
// Get the caller's file and line number (skip 2 frames)
_, file, line, ok := runtime.Caller(2)
if !ok {
file = "unknown"
line = 0
}
// Log message with caller information if GOMA_LOG_LEVEL is trace
if strings.ToLower(level) != "off" {
if strings.ToLower(level) == traceLog {
log.Printf("%s: %s (File: %s, Line: %d)\n", level, formattedMessage, file, line)
} else {
log.Printf("%s: %s\n", level, formattedMessage)
}
}
}
func getStd(out string) *os.File {
switch out {
case "/dev/stdout":
return os.Stdout
case "/dev/stderr":
return os.Stderr
case "/dev/stdin":
return os.Stdin
default:
return os.Stdout
}
}

26
pkg/logger/var.go Normal file
View File

@@ -0,0 +1,26 @@
package logger
/*
MIT License
# Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
const traceLog = "trace"

23
utils/config.go
View File

@@ -1,5 +1,28 @@
package utils package utils
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
import "os" import "os"
type MailConfig struct { type MailConfig struct {

28
utils/constant.go
View File

@@ -1,9 +1,27 @@
// Package utils / // Package utils /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package utils package utils
const RestoreExample = "restore --dbname database --file db_20231219_022941.sql.gz\n" + const RestoreExample = "restore --dbname database --file db_20231219_022941.sql.gz\n" +

60
utils/logger.go
View File

@@ -1,60 +0,0 @@
// Package utils /
/*****
@author Jonas Kaninda
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda
**/
package utils
import (
"fmt"
"os"
"time"
)
func Info(msg string, args ...any) {
var currentTime = time.Now().Format("2006/01/02 15:04:05")
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
fmt.Printf("%s INFO: %s\n", currentTime, msg)
} else {
fmt.Printf("%s INFO: %s\n", currentTime, formattedMessage)
}
}
// Warn warning message
func Warn(msg string, args ...any) {
var currentTime = time.Now().Format("2006/01/02 15:04:05")
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
fmt.Printf("%s WARN: %s\n", currentTime, msg)
} else {
fmt.Printf("%s WARN: %s\n", currentTime, formattedMessage)
}
}
func Error(msg string, args ...any) {
var currentTime = time.Now().Format("2006/01/02 15:04:05")
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
fmt.Printf("%s ERROR: %s\n", currentTime, msg)
} else {
fmt.Printf("%s ERROR: %s\n", currentTime, formattedMessage)
}
}
// Fatal logs an error message and exits the program
func Fatal(msg string, args ...any) {
var currentTime = time.Now().Format("2006/01/02 15:04:05")
// Fatal logs an error message and exits the program.
formattedMessage := fmt.Sprintf(msg, args...)
if len(args) == 0 {
fmt.Printf("%s ERROR: %s\n", currentTime, msg)
NotifyError(msg)
} else {
fmt.Printf("%s ERROR: %s\n", currentTime, formattedMessage)
NotifyError(formattedMessage)
}
os.Exit(1)
}

59
utils/notification.go
View File

@@ -1,13 +1,38 @@
package utils package utils
/*
MIT License
Copyright (c) 2023 Jonas Kaninda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
import ( import (
"bytes" "bytes"
"crypto/tls" "crypto/tls"
"encoding/json" "encoding/json"
"fmt" "fmt"
"github.com/go-mail/mail" "github.com/go-mail/mail"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"html/template" "html/template"
"io/ioutil" "io"
"net/http" "net/http"
"os" "os"
"path/filepath" "path/filepath"
@@ -31,7 +56,7 @@ func parseTemplate[T any](data T, fileName string) (string, error) {
} }
func SendEmail(subject, body string) error { func SendEmail(subject, body string) error {
Info("Start sending email notification....") logger.Info("Start sending email notification....")
config := loadMailConfig() config := loadMailConfig()
emails := strings.Split(config.MailTo, ",") emails := strings.Split(config.MailTo, ",")
m := mail.NewMessage() m := mail.NewMessage()
@@ -43,16 +68,16 @@ func SendEmail(subject, body string) error {
d.TLSConfig = &tls.Config{InsecureSkipVerify: config.SkipTls} d.TLSConfig = &tls.Config{InsecureSkipVerify: config.SkipTls}
if err := d.DialAndSend(m); err != nil { if err := d.DialAndSend(m); err != nil {
Error("Error could not send email : %v", err) logger.Error("Error could not send email : %v", err)
return err return err
} }
Info("Email notification has been sent") logger.Info("Email notification has been sent")
return nil return nil
} }
func sendMessage(msg string) error { func sendMessage(msg string) error {
Info("Sending Telegram notification... ") logger.Info("Sending Telegram notification... ")
chatId := os.Getenv("TG_CHAT_ID") chatId := os.Getenv("TG_CHAT_ID")
body, _ := json.Marshal(map[string]string{ body, _ := json.Marshal(map[string]string{
"chat_id": chatId, "chat_id": chatId,
@@ -72,11 +97,11 @@ func sendMessage(msg string) error {
} }
code := response.StatusCode code := response.StatusCode
if code == 200 { if code == 200 {
Info("Telegram notification has been sent") logger.Info("Telegram notification has been sent")
return nil return nil
} else { } else {
body, _ := ioutil.ReadAll(response.Body) body, _ := io.ReadAll(response.Body)
Error("Error could not send message, error: %s", string(body)) logger.Error("Error could not send message, error: %s", string(body))
return fmt.Errorf("error could not send message %s", string(body)) return fmt.Errorf("error could not send message %s", string(body))
} }
@@ -101,11 +126,11 @@ func NotifySuccess(notificationData *NotificationData) {
if err == nil { if err == nil {
body, err := parseTemplate(*notificationData, "email.tmpl") body, err := parseTemplate(*notificationData, "email.tmpl")
if err != nil { if err != nil {
Error("Could not parse email template: %v", err) logger.Error("Could not parse email template: %v", err)
} }
err = SendEmail(fmt.Sprintf("✅ Database Backup Notification – %s", notificationData.Database), body) err = SendEmail(fmt.Sprintf("✅ Database Backup Notification – %s", notificationData.Database), body)
if err != nil { if err != nil {
Error("Could not send email: %v", err) logger.Error("Could not send email: %v", err)
} }
} }
// Telegram notification // Telegram notification
@@ -113,12 +138,12 @@ func NotifySuccess(notificationData *NotificationData) {
if err == nil { if err == nil {
message, err := parseTemplate(*notificationData, "telegram.tmpl") message, err := parseTemplate(*notificationData, "telegram.tmpl")
if err != nil { if err != nil {
Error("Could not parse telegram template: %v", err) logger.Error("Could not parse telegram template: %v", err)
} }
err = sendMessage(message) err = sendMessage(message)
if err != nil { if err != nil {
Error("Could not send Telegram message: %v", err) logger.Error("Could not send Telegram message: %v", err)
} }
} }
} }
@@ -145,11 +170,11 @@ func NotifyError(error string) {
BackupReference: os.Getenv("BACKUP_REFERENCE"), BackupReference: os.Getenv("BACKUP_REFERENCE"),
}, "email-error.tmpl") }, "email-error.tmpl")
if err != nil { if err != nil {
Error("Could not parse error template: %v", err) logger.Error("Could not parse error template: %v", err)
} }
err = SendEmail(fmt.Sprintf("🔴 Urgent: Database Backup Failure Notification"), body) err = SendEmail("🔴 Urgent: Database Backup Failure Notification", body)
if err != nil { if err != nil {
Error("Could not send email: %v", err) logger.Error("Could not send email: %v", err)
} }
} }
// Telegram notification // Telegram notification
@@ -161,13 +186,13 @@ func NotifyError(error string) {
BackupReference: os.Getenv("BACKUP_REFERENCE"), BackupReference: os.Getenv("BACKUP_REFERENCE"),
}, "telegram-error.tmpl") }, "telegram-error.tmpl")
if err != nil { if err != nil {
Error("Could not parse error template: %v", err) logger.Error("Could not parse error template: %v", err)
} }
err = sendMessage(message) err = sendMessage(message)
if err != nil { if err != nil {
Error("Could not send telegram message: %v", err) logger.Error("Could not send telegram message: %v", err)
} }
} }
} }

71
utils/utils.go
View File

@@ -1,13 +1,32 @@
// Package utils / // Package utils /
/***** /*
@author Jonas Kaninda MIT License
@license MIT License <https://opensource.org/licenses/MIT>
@Copyright © 2024 Jonas Kaninda Copyright (c) 2023 Jonas Kaninda
**/
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
package utils package utils
import ( import (
"fmt" "fmt"
"github.com/jkaninda/mysql-bkup/pkg/logger"
"github.com/robfig/cron/v3" "github.com/robfig/cron/v3"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"io" "io"
@@ -17,6 +36,8 @@ import (
"time" "time"
) )
var Version = "development"
// FileExists checks if the file does exist // FileExists checks if the file does exist
func FileExists(filename string) bool { func FileExists(filename string) bool {
info, err := os.Stat(filename) info, err := os.Stat(filename)
@@ -31,7 +52,13 @@ func WriteToFile(filePath, content string) error {
if err != nil { if err != nil {
return err return err
} }
defer file.Close() defer func(file *os.File) {
err := file.Close()
if err != nil {
return
}
}(file)
_, err = file.WriteString(content) _, err = file.WriteString(content)
return err return err
@@ -49,14 +76,25 @@ func CopyFile(src, dst string) error {
if err != nil { if err != nil {
return fmt.Errorf("failed to open source file: %v", err) return fmt.Errorf("failed to open source file: %v", err)
} }
defer sourceFile.Close() defer func(sourceFile *os.File) {
err := sourceFile.Close()
if err != nil {
return
}
}(sourceFile)
// Create the destination file // Create the destination file
destinationFile, err := os.Create(dst) destinationFile, err := os.Create(dst)
if err != nil { if err != nil {
return fmt.Errorf("failed to create destination file: %v", err) return fmt.Errorf("failed to create destination file: %v", err)
} }
defer destinationFile.Close() defer func(destinationFile *os.File) {
err := destinationFile.Close()
if err != nil {
return
}
}(destinationFile)
// Copy the content from source to destination // Copy the content from source to destination
_, err = io.Copy(destinationFile, sourceFile) _, err = io.Copy(destinationFile, sourceFile)
@@ -74,7 +112,7 @@ func CopyFile(src, dst string) error {
} }
func ChangePermission(filePath string, mod int) { func ChangePermission(filePath string, mod int) {
if err := os.Chmod(filePath, fs.FileMode(mod)); err != nil { if err := os.Chmod(filePath, fs.FileMode(mod)); err != nil {
Fatal("Error changing permissions of %s: %v\n", filePath, err) logger.Fatal("Error changing permissions of %s: %v\n", filePath, err)
} }
} }
@@ -83,7 +121,12 @@ func IsDirEmpty(name string) (bool, error) {
if err != nil { if err != nil {
return false, err return false, err
} }
defer f.Close() defer func(f *os.File) {
err := f.Close()
if err != nil {
return
}
}(f)
_, err = f.Readdirnames(1) _, err = f.Readdirnames(1)
if err == nil { if err == nil {
@@ -131,7 +174,7 @@ func GetEnvVariable(envName, oldEnvName string) string {
if err != nil { if err != nil {
return value return value
} }
Warn("%s is deprecated, please use %s instead! ", oldEnvName, envName) logger.Warn("%s is deprecated, please use %s instead! ", oldEnvName, envName)
} }
} }
return value return value
@@ -178,10 +221,11 @@ func GetIntEnv(envName string) int {
} }
ret, err := strconv.Atoi(val) ret, err := strconv.Atoi(val)
if err != nil { if err != nil {
Error("Error: %v", err) logger.Error("Error: %v", err)
} }
return ret return ret
} }
func EnvWithDefault(envName string, defaultValue string) string { func EnvWithDefault(envName string, defaultValue string) string {
value := os.Getenv(envName) value := os.Getenv(envName)
if value == "" { if value == "" {
@@ -202,13 +246,12 @@ func CronNextTime(cronExpr string) time.Time {
// Parse the cron expression // Parse the cron expression
schedule, err := cron.ParseStandard(cronExpr) schedule, err := cron.ParseStandard(cronExpr)
if err != nil { if err != nil {
Error("Error parsing cron expression: %s", err) logger.Error("Error parsing cron expression: %s", err)
return time.Time{} return time.Time{}
} }
// Get the current time // Get the current time
now := time.Now() now := time.Now()
// Get the next scheduled time // Get the next scheduled time
next := schedule.Next(now) next := schedule.Next(now)
//Info("The next scheduled time is: %v\n", next)
return next return next
} }