文档课题:在rhel 6.5将mysql备份压缩指令由gzip调整为pigz.
系统:rhel 6.5 64位
数据库:mysql 5.5.18
应用场景:mysql生产环境中TB级别的数据量在压缩备份时会消耗很多时间,当将压缩命令由gzip调整为pigz时可加快速度,此结论已在rhel 7.3验证成功.现笔者在rhel 6.5上进行相关测试.
1、原备份语句
[mysql@MySQL5518-Master ~]$ /usr/bin/innobackupex --defaults-file=/etc/my.cnf --socket=/tmp/mysql.sock --user=root --password=mysql_4U --port=3306 --slave-info --stream=tar --tmpdir=/mysql/tmpfile /mysql/bak/ 2>/mysql/log_bak/backup.log |gzip ->/mysql/bak/192.168.133.122_20230819_3306.tar.gz
2、pigz安装
2.1、理论知识
pigz是压缩命令gzip的并行版,默认线程数是cpu个数.centos 7可以用yum安装,centos 6需编译安装.
2.2、安装基础包
[root@MySQL5518-Master ~]# yum -y install zlib-devel gcc gcc-c++ make
[root@MySQL5518-Master ~]# rpm -q zlib-devel gcc gcc-c++ make | grep "not installed"
2.3、上传pigz包
sftp> lcd F:\package
sftp> cd /
sftp> put pigz-2.7.tar.gz
注意:pigz-2.7.tar.gz安装包勿上传到/root目录进行此后的处理,否则安装好pigz后mysql用户备份时会报如下错误.
[mysql@MySQL5518-Master ~]$ /usr/bin/innobackupex --defaults-file=/etc/my.cnf --socket=/tmp/mysql.sock --user=root --password=mysql_4U --port=3306 --slave-info --stream=tar --tmpdir=/mysql/tmpfile /mysql/bak/ 2>/mysql/log_bak/backup.log |pigz -k -p6 ->/mysql/bak/192.168.133.122_20230819_3306.tar.gz
-bash: pigz: command not found
2.4、解压编译
[root@MySQL5518-Master ~]# cd /
[root@MySQL5518-Master /]# tar xvf pigz-2.7.tar.gz
[root@MySQL5518-Master /]# cd pigz-2.7
[root@MySQL5518-Master pigz-2.7]# make
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o pigz.o pigz.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o yarn.o yarn.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o try.o try.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/deflate.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/blocksplitter.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/tree.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/lz77.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/cache.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/hash.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/util.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/squeeze.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/katajainen.c
gcc -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/symbols.c
gcc -o pigz pigz.o yarn.o try.o deflate.o blocksplitter.o tree.o lz77.o cache.o hash.o util.o squeeze.o katajainen.o symbols.o -lm -lpthread -lz
ln -f pigz unpigz
[root@MySQL5518-Master pigz-2.7]# echo $?
0
[root@MySQL5518-Master pigz-2.7]# echo $PATH
/usr/local/webserver/mysql/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
[root@MySQL5518-Master pigz-2.7]# ln -s /pigz-2.7/pigz /usr/local/bin
[root@MySQL5518-Master pigz-2.7]# ln -s /pigz-2.7/unpigz /usr/local/bin
[root@MySQL5518-Slave pigz-2.7]# chmod 777 -R /pigz-2.7
[root@MySQL5518-Slave pigz-2.7]# chown -R mysql:mysql /pigz-2.7
2.5、pigz帮助信息
[root@MySQL5518-Master pigz-2.7]# pigz
Usage: pigz [options] [files ...]
will compress files in place, adding the suffix '.gz'. If no files are
specified, stdin will be compressed to stdout. pigz does what gzip does,
but spreads the work over multiple processors and cores when compressing.
Options:
-0 to -9, -11 Compression level (level 11, zopfli, is much slower)
--fast, --best Compression levels 1 and 9 respectively
-A, --alias xxx Use xxx as the name for any --zip entry from stdin
-b, --blocksize mmm Set compression block size to mmmK (default 128K)
-c, --stdout Write all processed output to stdout (won't delete)
-C, --comment ccc Put comment ccc in the gzip or zip header
-d, --decompress Decompress the compressed input
-f, --force Force overwrite, compress .gz, links, and to terminal
-F --first Do iterations first, before block split for -11
-h, --help Display a help screen and quit
-H, --huffman Use only Huffman coding for compression
-i, --independent Compress blocks independently for damage recovery
-I, --iterations n Number of iterations for -11 optimization
-J, --maxsplits n Maximum number of split blocks for -11
-k, --keep Do not delete original file after processing
-K, --zip Compress to PKWare zip (.zip) single entry format
-l, --list List the contents of the compressed input
-L, --license Display the pigz license and quit
-m, --no-time Do not store or restore mod time
-M, --time Store or restore mod time
-n, --no-name Do not store or restore file name or mod time
-N, --name Store or restore file name and mod time
-O --oneblock Do not split into smaller blocks for -11
-p, --processes n Allow up to n compression threads (default is the
number of online processors, or 8 if unknown)
-q, --quiet Print no messages, even on error
-r, --recursive Process the contents of all subdirectories
-R, --rsyncable Input-determined block locations for rsync
-S, --suffix .sss Use suffix .sss instead of .gz (for compression)
-t, --test Test the integrity of the compressed input
-U, --rle Use run-length encoding for compression
-v, --verbose Provide more verbose output
-V --version Show the version of pigz
-Y --synchronous Force output file write to permanent storage
-z, --zlib Compress to zlib (.zz) instead of gzip format
-- All arguments after "--" are treated as files
3、备份验证
[root@MySQL5518-Master local]# /usr/bin/innobackupex --defaults-file=/etc/my.cnf --socket=/tmp/mysql.sock --user=root --password=mysql_4U --port=3306 --slave-info --stream=tar --tmpdir=/mysql/tmpfile /mysql/bak/ 2>/mysql/log_bak/backup.log |pigz -k -p6 ->/mysql/bak/192.168.133.122_20230819_3306.tar.gz
说明:该备份语句成功执行.
参考网址:https://blog.csdn.net/zhaoyq008/article/details/103662355
在rhel 6.5将mysql备份压缩指令由gzip调整为pigz
原创
©著作权归作者所有:来自51CTO博客作者Liujun_Deng的原创作品,请联系作者获取转载授权,否则将追究法律责任
提问和评论都可以,用心的回复会被更多人看到
评论
发布评论
相关文章
-
pigz快速压缩
官网:http://zlib.net/pigz/简单的说,Pigz就是支持并行压缩的gzip。Pigz默认用当前逻辑cpu个数来
pigz linux 压缩文件 官网 -
Linux中pigz与gzip工具压缩速度对比测试
Linux中pigz与gzip工具压缩速度对比测试
mysql linux pigz gzip -
gzip压缩
gzip压缩
压缩 gzip