socket通信之listen函数

listen函数原型如下:

#include <sys/types.h>
#include <sys/socket.h>

int listen(int sockfd, int backlog);

第一个参数sockfd为创建socket返回的文件描述符。

第二个参数backlog为建立好连接处于ESTABLISHED状态的队列的长度。

backlog的最大值128(linux原文描述如下):

If  the  backlog argument is greater than the value in /proc/sys/net/core/somaxconn, then it is silently truncated to that value; the default value in this file is 128.  In kernels before 2.4.25, this limit was a hard coded value, SOMAXCONN, with the value 128.

下面附上TCP三次握手的执行过程图:

socket通信之listen函数_linux

TCP中有如下两个队列:

  1. SYN队列(半连接队列):当服务器端收到客户端的SYN报文时,会响应SYN/ACK报文,然后连接就会进入SYN RECEIVED状态,处于SYN RECEIVED状态的连接被添加到SYN队列,并且当它们的状态改变为ESTABLISHED时,即当接收到3次握手中的ACK分组时,将它们移动到accept队列。SYN队列的大小由内核参数/proc/sys/net/ipv4/tcp_max_syn_backlog设置。
  2. accept队列(完全连接队列):accept队列存放的是已经完成TCP三次握手的连接,而accept系统调用只是简单地从accept队列中取出连接而已,并不是调用accept函数才会完成TCP三次握手,accept队列的大小可以通过listen函数的第二个参数控制。

实验

下面我们用实验验证下backlog这个参数的含义:

服务器端代码如下:

#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <string.h>
#include <stdio.h>

int main() {
    
    // create socket
    int listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if(-1 == listenfd) {
        printf("create socket error");
        return -1;
    }
    
    // bind port 
    struct sockaddr_in bindaddr;
    bindaddr.sin_family = AF_INET;
    bindaddr.sin_addr.s_addr = htonl(INADDR_ANY);
    bindaddr.sin_port = htons(3000);
    if(-1 == bind(listenfd, (struct sockaddr *)&bindaddr, sizeof(bindaddr))) {
        printf("bind error");
        return -1;
    }
    
    // start listen
    if (listen(listenfd, 2) == -1) {
        printf("listem error");
        return -1;
    }
    
    while(1){}
    return 0;
}

客户端代码如下:

#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <string.h>
#include <stdio.h>
#include <pthread.h>

#define PORT 3000
#define thread_num 10

struct sockaddr_in serv_addr;

void *func() {
    int conn_fd;
    conn_fd = socket(AF_INET, SOCK_STREAM, 0);
    printf("conn_fd : %d\n", conn_fd);

    if( connect(conn_fd, (struct sockaddr *)&serv_addr, sizeof(struct sockaddr_in)) == -1) {
        printf("connect error\n");
    }

    while(1) {}
}

int main(int argc,char *argv[]) {
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(PORT);
	serv_addr.sin_addr.s_addr = inet_addr("10.0.4.129");;

    pthread_t pid[thread_num];
    int i;
    for(i = 0 ; i < thread_num; ++i) {
        pthread_create(&pid[i], NULL, &func, NULL);
    }

    for(i = 0 ; i < thread_num; ++i) {
        pthread_join(pid[i], NULL);
    }

    return 0;
}

由于建立连接和关闭连接的速度非常快,肉眼无法观察到,所以需要将变化的内容存到文本中,然后通过观察文本中的内容进行分析:

# watch -n 1 "netstat -anotp|grep 3000 >> a.txt"

启动服务器端,一开始没有连接进来:

tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      65667/./server.out   off (0.00/0/0)

然后启动客户端,用10个线程去连接服务器,因此服务器上有10条连接,其中只有三个连接是ESTABLISHED(backlog+1):

tcp        3      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      65667/./server.out   keepalive (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57492        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57484        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57488        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57478        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57482        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57480        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57486        SYN_RECV    -                    on (0.87/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57476        ESTABLISHED -                    off (0.00/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57474        ESTABLISHED -                    off (0.00/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57490        ESTABLISHED -                    off (0.00/0/0)

过了一会,连接超时会自动关闭:

tcp        3      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      65667/./server.out   off (0.00/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57476        ESTABLISHED -                    off (0.00/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57474        ESTABLISHED -                    off (0.00/0/0)
tcp        0      0 10.0.4.129:3000         10.0.4.120:57490        ESTABLISHED -                    off (0.00/0/0)

探究Netty中SO_BACKLOG与listen函数中backlog的关系

package com.morris.netty.basic;

import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;

public class NettyServer {

    public static final int PORT = 8899;

    public static void main(String[] args) throws InterruptedException {

        EventLoopGroup bossGroup = new NioEventLoopGroup(); // 负责接收客户端连接
        EventLoopGroup workerGroup = new NioEventLoopGroup(3); // 负责读取和发送数据

        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG, 8)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch) {
                            ch.pipeline().addLast(new SimpleChannelInboundHandler() {
                                @Override
                                protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {
                                    ByteBuf receiveByteBuf = (ByteBuf) msg;
                                    byte[] bytes = new byte[receiveByteBuf.readableBytes()];
                                    receiveByteBuf.readBytes(bytes);

                                    System.out.println("receive from client: " + new String(bytes));

                                    ByteBuf sendByteBuf = Unpooled.copiedBuffer("hello client".getBytes());
                                    ctx.writeAndFlush(sendByteBuf);
                                    ctx.close();
                                }
                            });
                        }
                    });

            // 启动 server.
            ChannelFuture f = b.bind(PORT).sync();

            System.out.println("server is start on port: " + PORT);

            // 等待socket关闭
            f.channel().closeFuture().sync();
        } finally {
            workerGroup.shutdownGracefully();
            bossGroup.shutdownGracefully();
        }
    }

}

使用strace命令根据程序所产生的系统调用:

# javac -cp ".:/root/java/netty/libs/dom4j-1.6.1.jar:/root/java/netty/libs/hamcrest-core-1.3.jar:/root/java/netty/libs/javassist-3.18.1-GA.jar:/root/java/netty/libs/jboss-marshalling-2.0.6.Final.jar:/root/java/netty/libs/jboss-marshalling-serial-2.0.6.Final.jar:/root/java/netty/libs/jdom-1.1.3.jar:/root/java/netty/libs/jibx-extras-1.3.1.jar:/root/java/netty/libs/jibx-run-1.3.1.jar:/root/java/netty/libs/joda-time-2.9.5.jar:/root/java/netty/libs/json-simple-1.1.1.jar:/root/java/netty/libs/junit-4.13.jar:/root/java/netty/libs/log4j-1.2.17.jar:/root/java/netty/libs/lombok-1.18.2.jar:/root/java/netty/libs/msgpack-0.6.12.jar:/root/java/netty/libs/netty-all-4.1.51.Final.jar:/root/java/netty/libs/protobuf-java-3.6.1.jar:/root/java/netty/libs/slf4j-api-1.7.30.jar:/root/java/netty/libs/slf4j-log4j12-1.7.30.jar:/root/java/netty/libs/xml-apis-1.0.b2.jar:/root/java/netty/libs/xpp3-1.1.3.4.O.jar" NettyServer.java 
# strace -ff -o out java -cp ".:/root/java/netty/libs/dom4j-1.6.1.jar:/root/java/netty/libs/hamcrest-core-1.3.jar:/root/java/netty/libs/javassist-3.18.1-GA.jar:/root/java/netty/libs/jboss-marshalling-2.0.6.Final.jar:/root/java/netty/libs/jboss-marshalling-serial-2.0.6.Final.jar:/root/java/netty/libs/jdom-1.1.3.jar:/root/java/netty/libs/jibx-extras-1.3.1.jar:/root/java/netty/libs/jibx-run-1.3.1.jar:/root/java/netty/libs/joda-time-2.9.5.jar:/root/java/netty/libs/json-simple-1.1.1.jar:/root/java/netty/libs/junit-4.13.jar:/root/java/netty/libs/log4j-1.2.17.jar:/root/java/netty/libs/lombok-1.18.2.jar:/root/java/netty/libs/msgpack-0.6.12.jar:/root/java/netty/libs/netty-all-4.1.51.Final.jar:/root/java/netty/libs/protobuf-java-3.6.1.jar:/root/java/netty/libs/slf4j-api-1.7.30.jar:/root/java/netty/libs/slf4j-log4j12-1.7.30.jar:/root/java/netty/libs/xml-apis-1.0.b2.jar:/root/java/netty/libs/xpp3-1.1.3.4.O.jar" NettyServer

在out.69240文件中有下面这么两行:

bind(45, {sa_family=AF_INET6, sin6_port=htons(8899), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0
listen(45, 8)

可见在Netty中设置的ChannelOption.SO_BACKLOG其实就是listen的第二个参数backlog。

修改内核参数

永久修改:直接修改配置文件/etc/sysctl.conf

临时修改:使用sysctl命令,系统重启或者网络重启(systemctl retart network)了,设置就会失效。

sysctl的使用:

  • -a:显示所有的系统参数。
  • -w:临时修改某个指定参数的值,如net.ipv4.tcp_syncookies = 1

例如修改TCP接收缓冲区的大小:

在netty中是这么设置的:

.option(ChannelOption.SO_SNDBUF, 1024)

实际上底层的系统调用为:

setsockopt(45, SOL_SOCKET, SO_RCVBUF, [1024], 4) = 0

当然也可以全局配置整个操作系统中所有TCP的接收缓冲区的大小:

# sysctl -r net.ipv4.tcp_rmem=1024 4096 65535

查看操作系统默认的TCP的接收缓冲区的大小:

# cat /proc/sys/net/ipv4/tcp_rmem
4096	87380	6291456