Hadoop的负载均衡 httpd负载均衡

转载

mob64ca140b0bc8 2024-01-10 19:09:46

文章标签 Hadoop的负载均衡负载均衡 tomcat apache html 文章分类 Hadoop 大数据

1、环境
redhat6(5)
apache2.2.*
tomcat6
2、安装apache
A、首先卸载红帽自带httpd服务
a、执行命令查询是否安装httpd
rpm -qa|grep httpd
b、执行命令删除已安装的包
rpm -e 安装包名称
如果有依赖，使用--nodeps参数 rpm -e 安装包名称 --nodeps
c、删除/etc/httpd目录，该目录有自带配置文件，会有影响
B、编译apache源文件，使用的插件（或者说模块）有：proxy、proxy-ajp、proxy-connect、proxy-http、proxy-ftp、proxy-balancer
编译步骤：
a、解压缩文件到目录 unzip file.zip
b、设置权限为777 chmod -R 777 fileAddr
c、进入目录，首先执行配置命令：
./configure --prefix=/usr/local/httpd --enable-so --enable-mods-shared='proxy proxy-connect proxy-ajp proxy-http proxy-ftp proxy-balancer'

说明：prexif为执行安装时的安装目录；
enable是启用哪些模块等；proxy*等是指定加载启用的模块，也可以使用参数 most（all）如：--enable-mod-shared=most
如没有特殊情况，一般可能会出现缺少依赖编译环境的错误，比如缺少gcc包，可以安装后继续编译（最好使用yum进行安装 yum install gcc）
C、执行make
D、如果没有错误，执行make install，进行安装，如无错误（exit code 0）安装成功。
E、启动服务 httpd\bin 下执行 ./httpd -k start
F、输入地址：http：\\ip，如没问题将会出现：It works！
服务配置完毕

3、安装配置tomcat
*以下配置为2台机器
A、解压缩tomcat压缩文件到目录，设置权限（简便起见，可设置777）
B、修改server.xml配置文件
a、修改端口
如无系统占用，可以使用默认http、ajp端口，否则需要进行修改（尤其是单台机器配置多个节点时）
b、修改cluster、engine配置
engine使用注释的配置即可，修改jvmRoute，配置为你想要的名称（同一集群不能重复）
cluster配置如下

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"   channelSendOptions="8">   

         <Manager className="org.apache.catalina.ha.session.DeltaManager"  expireSessionsOnShutdown="false"   notifyListenersOnReplication="true"/>    

         <Channel className="org.apache.catalina.tribes.group.GroupChannel">    

                 <Membership className="org.apache.catalina.tribes.membership.McastService"  

                 <!-- 同一集群使用同一个广播地址和端口 --> 

                 address="228.0.0.4" port="45564" frequency="500"        dropTime="3000"/>  

                  

                 <!-- 每个节点必须不同，ajp以此进行通信 port="4000"  --> 

                 <Receiver  className="org.apache.catalina.tribes.transport.nio.NioReceiver"    

                         address="auto" 

                         port="4001"  

                         autoBind="100" 

                         selectorTimeout="5000" 

                         axThreads="6"/>  



                 <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">  

                         <Transport  className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>   

                 </Sender>    

                 <Interceptor  className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>    

                 <Interceptor  className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>    

         </Channel> 

         <!-- 此filter可进行配置 --> 

         <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"    filter=""/>    

         <Valve  className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>   

         <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"   tempDir="/tmp/war-temp/" deployDir="/tmp/war-deploy/" watchDir="/tmp/war-listen/" watchEnabled="false"/>    

         <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>   

         <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>  

 </Cluster> 



                 完整server.xml配置文件内容如下： 

                  

         <?xml version='1.0' encoding='utf-8'?> 

 <!-- 

   Licensed to the Apache Software Foundation (ASF) under one or more 

   contributor license agreements.  See the NOTICE file distributed with 

   this work for additional information regarding copyright ownership. 

   The ASF licenses this file to You under the Apache License, Version 2.0 

   (the "License"); you may not use this file except in compliance with 

   the License.  You may obtain a copy of the License at 



       http://www.apache.org/licenses/LICENSE-2.0 


   Unless required by applicable law or agreed to in writing, software 

   distributed under the License is distributed on an "AS IS" BASIS, 

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 

   See the License for the specific language governing permissions and 

   limitations under the License. 

 --> 

 <!-- Note:  A "Server" is not itself a "Container", so you may not 

      define subcomponents such as "Valves" at this level. 

      Documentation at /docs/config/server.html 

  --> 

 <Server port="8005" shutdown="SHUTDOWN"> 



   <!--APR library loader. Documentation at /docs/apr.html --> 

   <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" /> 

   <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html --> 

   <Listener className="org.apache.catalina.core.JasperListener" /> 

   <!-- JMX Support for the Tomcat server. Documentation at /docs/non-existent.html --> 

   <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" /> 

   <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" /> 



   <!-- Global JNDI resources 

        Documentation at /docs/jndi-resources-howto.html 

   --> 

   <GlobalNamingResources> 

     <!-- Editable user database that can also be used by 

          UserDatabaseRealm to authenticate users 

     --> 

     <Resource name="UserDatabase" auth="Container" 

               type="org.apache.catalina.UserDatabase" 

               description="User database that can be updated and saved" 

               factory="org.apache.catalina.users.MemoryUserDatabaseFactory" 

               pathname="conf/tomcat-users.xml" /> 

              

             

   </GlobalNamingResources> 



   <!-- A "Service" is a collection of one or more "Connectors" that share 

        a single "Container" Note:  A "Service" is not itself a "Container",  

        so you may not define subcomponents such as "Valves" at this level. 

        Documentation at /docs/config/service.html 

    --> 

   <Service name="Catalina"> 

    

     <!--The connectors can use a shared executor, you can define one or more named thread pools--> 

     <!-- 

     <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"  

         maxThreads="150" minSpareThreads="4"/> 

     --> 

      

      

     <!-- A "Connector" represents an endpoint by which requests are received 

          and responses are returned. Documentation at : 

          Java HTTP Connector: /docs/config/http.html (blocking & non-blocking) 

          Java AJP  Connector: /docs/config/ajp.html 

          APR (HTTP/AJP) Connector: /docs/apr.html 

          Define a non-SSL HTTP/1.1 Connector on port 8080 

     --> 

     <Connector port="8080" protocol="HTTP/1.1"  

                connectionTimeout="20000"  

                redirectPort="8443" URIEncoding="GBK"/> 

     <!-- A "Connector" using the shared thread pool--> 

     <!-- 

     <Connector executor="tomcatThreadPool" 

                port="8080" protocol="HTTP/1.1"  

                connectionTimeout="20000"  

                redirectPort="8443" /> 

     -->            

     <!-- Define a SSL HTTP/1.1 Connector on port 8443 

          This connector uses the JSSE configuration, when using APR, the  

          connector should be using the OpenSSL style configuration 

          described in the APR documentation --> 

     <!-- 

     <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true" 

                maxThreads="150" scheme="https" secure="true" 

                clientAuth="false" sslProtocol="TLS" /> 

     --> 



     <!-- Define an AJP 1.3 Connector on port 8009 --> 

     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /> 





     <!-- An Engine represents the entry point (within Catalina) that processes 

          every request.  The Engine implementation for Tomcat stand alone 

          analyzes the HTTP headers included with the request, and passes them 

          on to the appropriate Host (virtual host). 

          Documentation at /docs/config/engine.html --> 



     <!-- You should set jvmRoute to support load-balancing via AJP ie : 

     <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">          

     -->  

         <!-- 每个节点对应配置文件 jvmRoute="tomcat1" --> 

     <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1" > 

                  



 <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"   channelSendOptions="8">   

         <Manager className="org.apache.catalina.ha.session.DeltaManager"  expireSessionsOnShutdown="false"   notifyListenersOnReplication="true"/>    

         <Channel className="org.apache.catalina.tribes.group.GroupChannel">    

                 <Membership className="org.apache.catalina.tribes.membership.McastService" address="228.0.0.4" port="45564" frequency="500"        dropTime="3000"/>  

                  

                 <!-- 每个节点必须不同 port="4000"  --> 

                 <Receiver  className="org.apache.catalina.tribes.transport.nio.NioReceiver"    

                         address="auto" 

                         port="4001"  

                         autoBind="100" 

                         selectorTimeout="5000" 

                         axThreads="6"/>  



                 <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">  

                         <Transport  className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>   

                 </Sender>    

                 <Interceptor  className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>    

                 <Interceptor  className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>    

         </Channel> 

         <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"    filter=""/>    

         <Valve  className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>   

         <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"   tempDir="/tmp/war-temp/" deployDir="/tmp/war-deploy/" watchDir="/tmp/war-listen/" watchEnabled="false"/>    

         <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>   

         <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>  

 </Cluster> 





       <!--For clustering, please take a look at documentation at: 

           /docs/cluster-howto.html  (simple how to) 

           /docs/config/cluster.html (reference documentation) --> 

       <!-- 

       <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/> 

       -->         



       <!-- The request dumper valve dumps useful debugging information about 

            the request and response data received and sent by Tomcat. 

            Documentation at: /docs/config/valve.html --> 

       <!-- 

       <Valve className="org.apache.catalina.valves.RequestDumperValve"/> 

       --> 



       <!-- This Realm uses the UserDatabase configured in the global JNDI 

            resources under the key "UserDatabase".  Any edits 

            that are performed against this UserDatabase are immediately 

            available for use by the Realm.  --> 

       <Realm className="org.apache.catalina.realm.UserDatabaseRealm" 

              resourceName="UserDatabase"/> 



       <!-- Define the default virtual host 

            Note: XML Schema validation will not work with Xerces 2.2. 

        --> 

       <Host name="localhost"  appBase="webapps" 

             unpackWARs="true" autoDeploy="true" 

             xmlValidation="false" xmlNamespaceAware="false"> 



         <!-- SingleSignOn valve, share authentication between web applications 

              Documentation at: /docs/config/valve.html --> 

         <!-- 

         <Valve className="org.apache.catalina.authenticator.SingleSignOn" /> 

         --> 



         <!-- Access log processes all example. 

              Documentation at: /docs/config/valve.html --> 

         <!-- 

         <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"   

                prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/> 

      

         --> 

          

 </Host> 

     </Engine> 

   </Service> 

 </Server> 



   注： 

     启动失败可能原因： 

1、端口号占用 

修改端口号 

2、没有主机名  

vi /etc/networks 

在里面加 hostname=ffzhang 

vi /etc/hosts 

在第一行 第二行 中都加 ffzhang 

重启 

     启动成功后仅自己可以访问，则是因为防火墙没有关闭 

执行：   service iptables stop 即可 

 



 4、apache负载均衡设置 

         A、httpd.conf 

                 a、 listen 

                         此参数可以直接设置监听端口，也可以使用IP：PORT的形式，如果服务器仅有一个ip，仅设置监听端口即可，如果多个ip，可以 

                         使用另外一种方式         

                         此参数可以配置多个，就是说，可以监听多个地址，监听多个端口（对应单服务器绑定多个ip、设置多个集群的情况下使用） 

                 系统默认80端口 

                 b、ServerName 

                         可设置域名，或者机器名称，如有没有可以设置为ip 

                 c、ServerAdmin 如有系统问题，发送邮箱 

                 d、虚拟目录设置 

                         两种方式：一是单独文件设置（有点麻烦），二是在httpd.conf文件结尾添加（如仅配置一个集群，此方式较好） 

                         *按第二种配置 

                         在httpd.conf结尾添加以下配置 

                          

                         NameVirtualHost 10.150.0.158:80 #虚拟主机，以此判断集群设置 

 # 

 # VirtualHost example: 

 # Almost any Apache directive may go into a VirtualHost container. 

 # The first VirtualHost section is used for all requests that do not 

 # match a ServerName or ServerAlias in any <VirtualHost> block. 

 # 

 <VirtualHost 10.150.0.158:80>    #对应上面的虚拟主机          

         ServerName 10.150.0.158 #主机ip 

         ServerAlias 10.150.0.158 #域名或者机器名 

         #集群配置 

         ProxyPass / balancer://myclus/ stickysession=jsessionid nofailover=On 

         #ProxyPass 满足条件的通过 

         #balancer 集群设置 

         #myclus 集群名称，以/结尾 

         #stickysession=jsessionid nofailover=On 

         #反向代理的集群，即上面配置的内容 

         ProxyPassReverse / balancer://myclus/ 

         #日志 

         ErrorLog "logs/lbtest-error.log" 

         CustomLog "logs/lbtest-access.log" common 

 </VirtualHost> 



 #对应以上虚拟主机设置 



 ProxyRequests Off #关闭正向代理（启用反向代理 

 #集群节点设置 

 <proxy balancer://myclus> 

         #此端口为ajp设置端口，非http端口 

         #loadfactor 分配权值，按访问次数比例分配 

         #route tomcat中engine节点设置的名称 

         BalancerMember ajp://10.150.0.126:8009/ loadfactor=1 route=jvm1 

         BalancerMember ajp://10.150.0.125:8009/ loadfactor=2 route=jvm2         

 </proxy> 



 ## 

         B、httpd-vhost.conf 

         上述采用第一种配置时，需配置此项，配置内容同上；需要将httpd.conf中关于httpd-xhost注释去掉即可，ProxyRequests一段需要添加到文件结尾 

5、测试


第一次发，还求大神指导。

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。