问题描述:
最近因为机房停电,机器上面的所有服务有出现重启,包括jenkins服务。jenkins在重启之后出现了不能访问,界面报错信息:
报错说启动了多个实例,点击下面的**“Ignore this problem and keep using Jenkins anyway”**按钮,可以恢复到原来正常的界面,但是这个报错,不久又会出现。
原因分析
因为jenkins是放在tomcat下面的,查看tomcat进程,发现确实只有一个,端口也没有什么异常:
[root@master-2 bin]# ps -ef|grep tomcat
root 21780 1 43 15:15 pts/2 00:00:12 /usr/local/jdk/bin/java -Djava.util.logging.config.file=/usr/local/apache-tomcat-8.0.48/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dignore.endorsed.dirs= -classpath /usr/local/apache-tomcat-8.0.48/bin/bootstrap.jar:/usr/local/apache-tomcat-8.0.48/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat-8.0.48 -Dcatalina.home=/usr/local/apache-tomcat-8.0.48 -Djava.io.tmpdir=/usr/local/apache-tomcat-8.0.48/temp org.apache.catalina.startup.Bootstrap start
root 12245 7771 0 15:16 pts/2 00:00:00 grep --color=auto tomcat
[root@master-2 bin]#
[root@master-2 bin]#
[root@master-2 bin]#
[root@master-2 bin]#
[root@master-2 bin]# netstat -tlunp|grep 28080
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 28080/fdfs_storaged
tcp6 6 0 :::28080 :::* LISTEN 12168/java
[root@master-2 bin]#
根据https://stackoverflow.com/questions/44378841/jenkins-detecting-more-than-one-instance的提示: • Check if another java process is running which runs jenkins e.g. in unix using top **• If you're using tomcat, search for a second instance of tomcat having the jenkins.war exploded somewhere ** • Could be helpful to just have one version of java installed
查看jenkins,怀疑可能是哪里启动了多个jenkins进程,发现有两个一模一样的jenkins进程,只是进程号不一样:
[root@master-2 home]# ps -ef|grep jenkins
samba 10835 10820 0 May18 ? 00:00:05 /bin/tini -- /usr/local/bin/jenkins.sh
samba 10854 10835 0 May18 ? 00:03:53 java -Duser.home=/var/jenkins_home -jar /usr/share/jenkins/jenkins.war
root 30593 21780 0 10:34 pts/2 00:00:00 sh -c { while [ -d '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7' -a \! -f '/root/.jenkins/workspins-result.txt' ]; do touch '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/jenkins-log.txt'; sleep 3; done } & jsc=durable-95de6aeeb4c40d371cff43dc '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/script.sh' > '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/jenkins-log.txt' 2>&1; echo $?tmp/durable-9c600ea7/jenkins-result.txt'; wait #这个进程后面的21780看,应该是之前tomcat正常启动的进程
*root 30596 30593 0 10:34 pts/2 00:00:00 sh -c { while [ -d '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7' -a \! -f '/root/.jenkins/workspins-result.txt' ]; do touch '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/jenkins-log.txt'; sleep 3; done } & jsc=durable-95de6aeeb4c40d371cff43dc '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/script.sh' > '/root/.jenkins/workspace/locman@tmp/durable-9c600ea7/jenkins-log.txt' 2>&1; echo $?tmp/durable-9c600ea7/jenkins-result.txt'; wait #这个进程有点异常
root 30597 30593 0 10:34 pts/2 00:00:00 /bin/sh -xe /root/.jenkins/workspace/locman@tmp/durable-9c600ea7/script.sh
root 30600 30597 44 10:34 pts/2 00:00:13 /usr/local/jdk/bin/java -classpath /var/jenkins_home/apache-maven-3.5.2/boot/plexus-classworlds-2.5.2.jare/apache-maven-3.5.2/bin/m2.conf -Dmaven.home=/var/jenkins_home/apache-maven-3.5.2 -Dlibrary.jansi.path=/var/jenkins_home/apache-maven-3.5.2/lib/jansi-natory=/home/.jenkins/workspace/locman/locman-service-controller org.codehaus.plexus.classworlds.launcher.Launcher package -DskipTests
root 30702 30600 0 10:35 pts/2 00:00:00 /bin/sh -c cd /home/.jenkins/workspace/locman/locman-service-controller && /usr/local/jdk/jre/../bin/java.javac.JavacCompiler8324825440823874050arguments
root 31210 7771 0 10:35 pts/2 00:00:00 grep --color=auto jenkins
[root@master-2 home]#
原因说明: 因为jenkins是放在tomcat下面的,将jenkins.war解压到了/usr/local/apache-tomcat-8.0.48/webapps/ROOT目录下面,但是还放了一个jenkins.war在webapps下面忘记删掉。而且并没有在server.xml里面配置<Context>标签,因此在启动tomcat的时候,就会启动部署tomcat/webapp下的所有项目(或者说文件夹:ROOT,PLAYER,项目1,项目2...),也就一起启动了webapps下面的jenkins.war和ROOT目录下面的jenkins包内容。所以就启动了两个jenkins实例。
jenkins.war放在webapps目录下面和解压到ROOT目录下面的区别: ROOT目录下面默认是tomcat的管理程序,但是如果你把自己的jenkins发布到root下面的话,你可以不通过项目名直接访问你的项目,直接使用通过http://localhost:8080直接访问你的项目(下图端口号用的28080):
如果放在webapps下面,访问的时候,就需要通过项目名称来访问http://localhost:8080/jenkins/(下图端口号用的28080):
解决方法:
1、杀掉所有的jenkins和tomcat进程,然后将webapps下面的jenkins.war改个名字或者删掉 2、在tomcat的conf目录下server.xml里面的</Host>前面加一行<Context path="" docBase="ROOT" debug="0" reloadable="true"/> 再重启tomcat
[root@master-2 webapps]# mv jenkins.war jenkins.war.bak
[root@master-2 bin]#
[root@master-2 bin]# ./startup.sh
[root@master-2 webapps]# ps -ef|grep jenkins
root 3611 7771 0 10:54 pts/2 00:00:00 grep --color=auto jenkins
samba 10835 10820 0 May18 ? 00:00:05 /bin/tini -- /usr/local/bin/jenkins.sh
samba 10854 10835 0 May18 ? 00:03:54 java -Duser.home=/var/jenkins_home -jar /usr/share/jenkins/jenkins.war
[root@master-2 webapps]# ps -ef|grep tomcat
root 1238 1 19 10:41 pts/2 00:02:33 /usr/local/jdk/bin/java -Djava.util.logging.config.file=/usr/local/apache-tomcat-8.0.48/conf/logging.proprg.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dignore.endorsed.dicat-8.0.48/bin/bootstrap.jar:/usr/local/apache-tomcat-8.0.48/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/apache-tomcat-8.0.48 -Dcatalina.home=/usr/locpdir=/usr/local/apache-tomcat-8.0.48/temp org.apache.catalina.startup.Bootstrap start
root 3632 7771 0 10:54 pts/2 00:00:00 grep --color=auto tomcat
然后暂时没有出现这个错误了。
备注:webapps下面的jenkins.war一直都存在,在异常停电重启之前,没有发现报这个错。目前暂时还没出现上面的报错了,问题是否解决还需后续继续观察