centos离线安装python3/pip3和项目所需第三方模块

  • 写在前面
  • 一、安装python3和pip3
  • （一）下载python3和依赖包
  • （二）离线服务器安装python3
  • 二、安装项目所需模块
  • （一）在联网机部署项目所需的python3开发环境
  • （二）将联网机的python3开发环境打包
  • （三）在离线服务器部署项目所需python3开发环境


写在前面

本人也是由于项目开发需要离线安装python3，在网上也查询了许多资料，特写此博客记录离线安装python3的过程，希望对大家能有帮助，主要还是怕自己以后又要部署环境又忘记了。相关博客链接在文中已经写明了。

一、安装python3和pip3

这一部分主要参考了这些博文：
linux全离线安装python环境Centos7下安装Python3.7CentOS7下安装Python3及Pip3并保留Python2

（一）下载python3和依赖包

1.通过修改联网机yum配置，在使用yum install xxx 的时候直接将这些依赖保存在指定目录中，而不是直接删除，便于后面将这些依赖包拷贝到离线服务器中。具体操作如下：

vim /etc/yum.conf
cachedir=/var/cache/yum # yum install xxx中xxx依赖的存储位置
keepcache=0	# 改为keepcache=1这样yum install xxx下载的rpm包就不会删除

2.yum install xxx 下载python3的依赖

yum install zlib-devel bzip2-devel openssl-devel ncurses-devel  epel-release gcc gcc-c++ xz-devel readline-devel gdbm-devel sqlite-devel tk-devel db4-devel libpcap-devel libffi-devel

3.下载python3
在python官网下载所需的python3，或者使用wget命令下载

wget --no-check-certificate https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz

（二）离线服务器安装python3

1.在联网机中将Python-3.7.2.tgz和/var/cache/yum目录下的所有rpm包都拷贝至离线服务器。
2.安装python3的编译环境

rpm -Uvh ./*.rpm --nodeps --force

3.安装python3

tar -zxvf Python-3.7.2.tgz	# 解压python3安装包
cd Python-3.7.2 	# 进入python3安装包目录
./configure --prefix=/usr/local/python3	# 将python3安装在这个目录
make && make install	# 编译和安装

4.创建软连接

ln -s /usr/local/python3/bin/python3 /usr/bin/python3	# 创建python3软连接
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3	# 创建pip3的软连接

至此python3/pip3的安装已经完成，接下来就是利用pip3来安装项目开发中所需的python3模块了。

二、安装项目所需模块

这一部分主要参考这些博文：
python3.X 使用pip命令离线打包及安装断网环境下利用pip安装Python离线安装包sasl/saslwrapper.h:22:23: fatal error: sasl/sasl.h: No such file or directoryPython安装第三方库，出现‘PyThreadState’ has no member named ‘exc_type’解决方法

（一）在联网机部署项目所需的python3开发环境

1.pip3 install 各种装模块就行了，要是报错只要用yum install 下载相应的依赖就行，只要知道yum install 安装的依赖文件现在存在/var/cache/yum，到时候记得拷贝的离线服务器就行。
下面是我在安装时遇到的一些问题，例如我项目中的sasl，因为缺少某些依赖会报错，部分报错内容如下：

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Isasl -I/usr/local/bin/python3/include/python3.7m -c sasl/saslwrapper.cpp -o build/temp.linux-x86_64-3.7/sasl/saslwrapper.o
    In file included from sasl/saslwrapper.cpp:254:0:
    sasl/saslwrapper.h:22:23: fatal error: sasl/sasl.h: No such file or directory
     #include <sasl/sasl.h>
                           ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

stackoverflow中的这个问题sasl/saslwrapper.h:22:23: fatal error: sasl/sasl.h: No such file or directory与我遇到的一致，这是由于缺少cyrus-sasl-devel依赖，所以使用yum install cyrus-sasl-devel安装依赖即可。

yum install cyrus-sasl-devel	# 这个仍会下载到/var/cache/yum/中，记得将后来补充的依赖也要拷贝到离线服务器！

安装thriftpy时又报错，部分信息如下：

thriftpy/transport/cybase.c: In function '__Pyx__GetException':
    thriftpy/transport/cybase.c:3127:22: error: 'PyThreadState' has no member named 'exc_type'
         tmp_type = tstate->exc_type;
                          ^
    thriftpy/transport/cybase.c:3128:23: error: 'PyThreadState' has no member named 'exc_value'
         tmp_value = tstate->exc_value;
                           ^
    thriftpy/transport/cybase.c:3129:20: error: 'PyThreadState' has no member named 'exc_traceback'
         tmp_tb = tstate->exc_traceback;
                        ^
    thriftpy/transport/cybase.c:3130:11: error: 'PyThreadState' has no member named 'exc_type'
         tstate->exc_type = local_type;
               ^
    thriftpy/transport/cybase.c:3131:11: error: 'PyThreadState' has no member named 'exc_value'
         tstate->exc_value = local_value;
               ^
    thriftpy/transport/cybase.c:3132:11: error: 'PyThreadState' has no member named 'exc_traceback'
         tstate->exc_traceback = local_tb;
               ^
    thriftpy/transport/cybase.c: In function '__Pyx__ExceptionSwap':
    thriftpy/transport/cybase.c:3154:22: error: 'PyThreadState' has no member named 'exc_type'
         tmp_type = tstate->exc_type;
                          ^
    thriftpy/transport/cybase.c:3155:23: error: 'PyThreadState' has no member named 'exc_value'
         tmp_value = tstate->exc_value;
                           ^
    thriftpy/transport/cybase.c:3156:20: error: 'PyThreadState' has no member named 'exc_traceback'
         tmp_tb = tstate->exc_traceback;
                        ^
    thriftpy/transport/cybase.c:3157:11: error: 'PyThreadState' has no member named 'exc_type'
         tstate->exc_type = *type;
               ^
    thriftpy/transport/cybase.c:3158:11: error: 'PyThreadState' has no member named 'exc_value'
         tstate->exc_value = *value;
               ^
    thriftpy/transport/cybase.c:3159:11: error: 'PyThreadState' has no member named 'exc_traceback'
         tstate->exc_traceback = *tb;
               ^
    thriftpy/transport/cybase.c: In function '__Pyx__ExceptionSave':
    thriftpy/transport/cybase.c:3178:19: error: 'PyThreadState' has no member named 'exc_type'
         *type = tstate->exc_type;
                       ^
    thriftpy/transport/cybase.c:3179:20: error: 'PyThreadState' has no member named 'exc_value'
         *value = tstate->exc_value;
                        ^
    thriftpy/transport/cybase.c:3180:17: error: 'PyThreadState' has no member named 'exc_traceback'
         *tb = tstate->exc_traceback;
                     ^
    thriftpy/transport/cybase.c: In function '__Pyx__ExceptionReset':
    thriftpy/transport/cybase.c:3187:22: error: 'PyThreadState' has no member named 'exc_type'
         tmp_type = tstate->exc_type;
                          ^
    thriftpy/transport/cybase.c:3188:23: error: 'PyThreadState' has no member named 'exc_value'
         tmp_value = tstate->exc_value;
                           ^
    thriftpy/transport/cybase.c:3189:20: error: 'PyThreadState' has no member named 'exc_traceback'
         tmp_tb = tstate->exc_traceback;
                        ^
    thriftpy/transport/cybase.c:3190:11: error: 'PyThreadState' has no member named 'exc_type'
         tstate->exc_type = type;
               ^
    thriftpy/transport/cybase.c:3191:11: error: 'PyThreadState' has no member named 'exc_value'
         tstate->exc_value = value;
               ^
    thriftpy/transport/cybase.c:3192:11: error: 'PyThreadState' has no member named 'exc_traceback'
         tstate->exc_traceback = tb;
               ^
    error: command 'gcc' failed with exit status 1

Python安装第三方库，出现‘PyThreadState’ has no member named ‘exc_type’解决方法这个博客解决了我的问题，这是因为缺少cython，看清楚是cython不是cpython，我就看错了。。。

pip3 install cython

2.将已部署好的python3开发环境中的模块导出到requirement.txt中。

在所有的环境都部署好之后，将联网机中的python3模块导入到requirement.txt中。

pip3 freeze >requirements.txt # 查询安装模块

requirement.txt中的部分内容如下所示：

asn1crypto==0.24.0
cffi==1.11.5
cryptography==2.3.1
cython==0.29.2
dateutils==0.6.6
Django==1.11.16
django-cors-headers==2.4.0
django-crontab==0.7.1
djangorestframework==3.8.2
gunicorn==19.9.0
idna==2.7
numpy==1.15.1
pandas==0.23.4
pycparser==2.19
pytz==2018.7
six==1.11.0
SQLAlchemy==1.2.11
elasticsearch==6.3.1
thrift==0.11.0
thrift-sasl==0.2.1
thriftpy==0.3.9

（二）将联网机的python3开发环境打包

1.下载单个python3模块。

pip3 download xxx  -d /tmp/packages/

2.根据联网机已部署好的开发环境来下载所需的python3模块。

pip3 download -r requirements.txt -d /tmp/packages/   # 推荐使用
#或者：
pip3 install --download /tmp/packages -r requirements.txt

然后将requirement.txt和/tmp/packages/下的这些模块都拷贝到离线服务器。

（三）在离线服务器部署项目所需python3开发环境

1.离线服务器安装单个模块。

pip3 install xxx.tar.gz   # 从A机器拷贝的文件
#或者：
tar -xvf xxx.tar.gz
cd xxx
python3 setup.py install

2.离线服务器批量安装模块，/tmp/packages是在离线服务器中这些模块的存储位置，根据实际情况修改。

pip3 install --no-index --find-links=/tmp/packages  -r requirments.txt # 拷贝过来的文件

over。