当同事告诉你这台机器上装了docker并且机器上已经有你想要的docker镜像时,你就可以直接拿来用了。你可以查看下镜像docker images,看下哪个是你需要的镜像。你也可以用docker ps看下是否有你需要的docker容器,如果有并且在征得容器主人的同意后便可以直接拿来用。当然,最好是你根据别人的镜像用 docker run新开启一个docker容器,而不是在别人的容器上搞动作。
dokcer服务操作:
启动docker服务:service docker start
查看docker运行状态:service docker status
关闭docker进程:service docker stop
重启docker服务:service docker restart
docker镜像操作:
查看镜像:docker images
镜像保存: docker save python_pytorch -o /usr/xxx/data/python_pytorch.tar
镜像载入:docker load < python_pytorch.tar
删除镜像:docker rmi 镜像ID
修改镜像后提交新镜像:docker commit -m "add gensim to pythonpytorch" 容器名或容器ID python_pytorch:v1.0
重命名镜像:docker tag 镜像ID 镜像名:版本号
docker容器操作:
查看容器:docker ps (-a 表示包括未运行的容器)
新创建容器:docker run -it -d -p 10.11.123.45:8080:80 --name pytorch_model python_pytorch:v1.0 /bin/bash (-d为后台运行,如果当前窗口进入容器需要去掉,-p 后面为端口号映射,即将本地的端口号映射到docker容器,后续在docker中部署模型等服务时就可以在本地的映射端口查看输出结果了。多个端口映射需要加多个-p,服务器ip一般可略去,比如 -p 8001:8001 -p 8002:8002 -p 8003:8003)
从服务器本地拷贝文件到容器里面:docker cp /usr/xxx/data/entity_file.csv 容器ID:/usr/xxx/data/(从容器拷贝文件到服务器本地即反向操作即可)
从容器中退出:exit(不会关闭容器)
停止容器:docker stop 容器名或容器ID
启动容器:docker start 容器名或容器ID
在现有已启动容器中运行命令:docker exec -it 容器名或容器ID /bin/bash
杀掉容器进程:docker kill 容器ID
删除容器:docker rm 容器ID
删除运行中的容器:docker rm -f 容器ID
导出容器快照到本地:docker export 容器名或容器ID > pytorch_model.tar
导入容器快照作为镜像:cat pytorchmodel.tar | docker import - pytorch_model0928:v1.0
重命名容器:docker rename pytorch_model(旧容器名) entity_linking_model(新容器名)
查看docker容器日志:docker logs 容器ID (排查bug时常用)
查看docker镜像历史构建信息:docker history 镜像ID
通过dockerfile构建镜像并压缩镜像:docker build --squash -t bert:v1.0 . (需要设置"experimental"=True)
如果需要在docker中使用gpu加速模型,则需要安装NVIDIA docker版本,并且将相关命令开头的docker全部变更为nvidia-docker
dockerfile:centos7+cuda+anaconda+bert_pytorch
# 基础镜像
FROM nvidia/cuda:10.0-cudnn7-devel-centos7
#维护者信息
LABEL maintainer "SnailDM"
# 安装bzip2
RUN yum -y install bzip2
# 安装Anaconda
COPY ./Anaconda3-2020.02-Linux-x86_64.sh /tmp/Anaconda3-2020.02-Linux-x86_64.sh
COPY ./torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl
COPY ./pytorch_pretrained_bert-0.6.2-py3-none-any.whl /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl
COPY ./Flask-1.1.1-py2.py3-none-any.whl /tmp/Flask-1.1.1-py2.py3-none-any.whl
WORKDIR /tmp
RUN sh -c '/bin/echo -e "nyesnnyes" | sh Anaconda3-2020.02-Linux-x86_64.sh'
#设置环境变量
ENV PATH /root/anaconda3/bin:$PATH
RUN pip install threadpool &&
pip install /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl &&
pip install /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl &&
pip install /tmp/Flask-1.1.1-py2.py3-none-any.whl &&
rm -rf /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl &&
rm -rf /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl &&
rm -rf /tmp/Flask-1.1.1-py2.py3-none-any.whl &&
rm -rf /tmp/Anaconda3-2020.02-Linux-x86_64.sh
# 设置软连接
RUN rm -rf /usr/bin/python && ln -s /root/anaconda3/bin/python /usr/bin/python
dockerfile:centos7+cuda+python+bert_pytorch
FROM nvidia/cuda:10.0-cudnn7-devel-centos7
MAINTAINER SnailDM # 指定作者信息
COPY ./torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl
COPY ./pytorch_pretrained_bert-0.6.2-py3-none-any.whl /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl
COPY ./Flask-1.1.1-py2.py3-none-any.whl /tmp/Flask-1.1.1-py2.py3-none-any.whl
COPY ./numpy-1.15.0-cp37-cp37m-manylinux1_x86_64.whl /tmp/numpy-1.15.0-cp37-cp37m-manylinux1_x86_64.whl
COPY ./pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl /tmp/pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl
COPY ./Python-3.7.0.tgz /tmp/Python-3.7.0.tgz
RUN set -ex
# 预安装所需组件
&& yum install -y wget tar libffi-devel zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make initscripts
&& tar -zxvf /tmp/Python-3.7.0.tgz
&& cd /Python-3.7.0
&& ./configure prefix=/usr/local/python3
&& make
&& make install
&& make clean
&& yum install -y epel-release
&& yum install -y python-pip
# 设置默认为python3
RUN set -ex
# 备份旧版本python
&& mv /usr/bin/python /usr/bin/python27
&& mv /usr/bin/pip /usr/bin/pip-python27
# 配置默认为python3
&& ln -s /usr/local/python3/bin/python3.7 /usr/bin/python
&& ln -s /usr/local/python3/bin/pip3 /usr/bin/pip
# 修复因修改python版本导致yum失效问题
RUN set -ex
&& sed -i "s#/usr/bin/python#/usr/bin/python27#" /usr/bin/yum
&& sed -i "s#/usr/bin/python#/usr/bin/python27#" /usr/libexec/urlgrabber-ext-down
&& yum install -y deltarpm
# 支持中文
RUN yum install kde-l10n-Chinese -y
RUN localedef -c -f UTF-8 -i zh_CN zh_CN.utf8
RUN pip install --upgrade pip &&
pip install threadpool &&
pip install /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl &&
pip install /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl &&
pip install /tmp/Flask-1.1.1-py2.py3-none-any.whl &&
pip install /tmp/numpy-1.15.0-cp37-cp37m-manylinux1_x86_64.whl &&
pip install /tmp/pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl &&
rm -rf /tmp/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl &&
rm -rf /tmp/pytorch_pretrained_bert-0.6.2-py3-none-any.whl &&
rm -rf /tmp/Flask-1.1.1-py2.py3-none-any.whl &&
rm -rf /tmp/numpy-1.15.0-cp37-cp37m-manylinux1_x86_64.whl &&
rm -rf /tmp/pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl &&
rm -rf /tmp/Python-3.7.0*
ENV LC_ALL zh_CN.UTF-8