server端是使用了Reactor模式对nio进行了一些封装,Reactor模式网上有很多资料,不赘述,了解了这个模式开始看源码

netty的版本是4.0.21.Final


<dependency>
	<groupId>io.netty</groupId>
	<artifactId>netty-all</artifactId>
	<version>4.0.21.Final</version>
</dependency>





netty端server端的比较经典的代码如下:

(代码一)

public void start(){
        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap sbs = new ServerBootstrap();
            sbs.group(bossGroup,workerGroup);
            sbs.channel(NioServerSocketChannel.class);
            sbs.localAddress(new InetSocketAddress(port));
            sbs.childHandler(new ChannelInitializer<SocketChannel>() {
                        
                        protected void initChannel(SocketChannel ch) throws Exception {
                            ch.pipeline().addLast("decoder", new StringDecoder());
                            ch.pipeline().addLast("encoder", new StringEncoder());
                            ch.pipeline().addLast(new HelloWorldServerHandler());
                        };
                        
                    }).option(ChannelOption.SO_BACKLOG, 128)   
                    .childOption(ChannelOption.SO_KEEPALIVE, true);
             // 绑定端口,开始接收进来的连接
             ChannelFuture future = sbs.bind(port).sync();  
             
             System.out.println("Server start listen at " + port );
             future.channel().closeFuture().sync();
        } catch (Exception e) {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }



step1:

这是一段普通不能再普通的代码了,先看EventLoopGroup这个类,这个类是EventLoop的一个集合,我们先简单的看下EventLoopGroup的一个具体实现NioEventLoopGroup


public NioEventLoopGroup() {
        this(0);
}

public NioEventLoopGroup(int nThreads) {
        this(nThreads, null);
}

public NioEventLoopGroup(int nThreads, ThreadFactory threadFactory) {
        this(nThreads, threadFactory, SelectorProvider.provider());
}

public NioEventLoopGroup(
            int nThreads, ThreadFactory threadFactory, final SelectorProvider selectorProvider) {
        super(nThreads, threadFactory, selectorProvider);
}



NioEventLoopGroup构造函数最后调用了它父类MultithreadEventLoopGroup的构造函数:

(代码二)

protected MultithreadEventLoopGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
        super(nThreads == 0? DEFAULT_EVENT_LOOP_THREADS : nThreads, threadFactory, args);
}

其中DEFAULT_EVENT_LOOP_THREADS设置了这个group中的个数,如果我们没有默认传递参数,系统默认会是:

DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt(
                "io.netty.eventLoopThreads", Runtime.getRuntime().availableProcessors() * 2));

一般是你CPU *2的个数,在Reactor模式中,mainReactor角色一般只需要一个线程就搞定了,subReactor角色就是那个苦逼的worker了,一般boss(mainReactor)一个就够了,subReactor就是需要多个了,所以在【代码一】中:

EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();



看名字就知道哪个EventLoopGroup对应哪一个Reactor模型中的角色了


回到上面一个话题中,我们接着【代码二】看,它接着调用它的父类MultithreadEventExecutorGroup的构造函数

protected MultithreadEventExecutorGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
        if (nThreads <= 0) {
            throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
        }

        if (threadFactory == null) {
            threadFactory = newDefaultThreadFactory();
        }

        children = new SingleThreadEventExecutor[nThreads];
        if (isPowerOfTwo(children.length)) {
            chooser = new PowerOfTwoEventExecutorChooser();
        } else {
            chooser = new GenericEventExecutorChooser();
        }

        for (int i = 0; i < nThreads; i ++) {
            boolean success = false;
            try {
                children[i] = newChild(threadFactory, args);
                success = true;
            } catch (Exception e) {
                // TODO: Think about if this is a good exception type
                throw new IllegalStateException("failed to create a child event loop", e);
            } finally {
                if (!success) {
                    for (int j = 0; j < i; j ++) {
                        children[j].shutdownGracefully();
                    }

                    for (int j = 0; j < i; j ++) {
                        EventExecutor e = children[j];
                        try {
                            while (!e.isTerminated()) {
                                e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
                            }
                        } catch (InterruptedException interrupted) {
                            Thread.currentThread().interrupt();
                            break;
                        }
                    }
                }
            }
        }

这是本质的方法,所有的工作基本上都在这里做了,因为前面我们很懒惰,啥都没有做,因为我们构造NioEventLoopGroup的时候,没有传递任何参数,netty给了一些默认的实现,首先newDefaultThreadFactory用来穿件一个默认的ThreadFactory,然后初始化了一个叫做children的实例变量,该children的定义是这样的

private final EventExecutor[] children;

EventExecutor是一个Executor,也就是一个线程执行器,换而言之,children = new SingleThreadEventExecutor[nThreads]这行代码对children进行了初始化动作,nThreads表示个数,如果我们CPU是4核的话,4*2表示初始化8个SingleThreadEventExecutor线程执行器,SingleThreadEventExecutor其实就是Reactor模型中真正工作的worker了,我们先简单的看下SingleThreadEventExecutor源码,下面列出2段SingleThreadEventExecutor的代码片段

SingleThreadEventExecutor片段一:

private final EventExecutorGroup parent;
private final Queue<Runnable> taskQueue;
final Queue<ScheduledFutureTask<?>> delayedTaskQueue = new PriorityQueue<ScheduledFutureTask<?>>();

private final Thread thread;
private final Semaphore threadLock = new Semaphore(0);
private final Set<Runnable> shutdownHooks = new LinkedHashSet<Runnable>();
private final boolean addTaskWakesUp;

首先先看taskQueue这是一个Runnable的任务队列,还有一个delayedTaskQueue延迟队列,最后每一个SingleThreadEventExecutor都绑定了唯一的Thread,用这个thread去执行这些taskQueue和delayedTaskQueue中的任务

SingleThreadEventExecutor片段二:

thread = threadFactory.newThread(new Runnable() {
            @Override
            public void run() {
                boolean success = false;
                updateLastExecutionTime();
                try {
                    SingleThreadEventExecutor.this.run();
                    success = true;
                } catch (Throwable t) {
                    logger.warn("Unexpected exception from an event executor: ", t);
                }

这个thread最后回去调用SingleThreadEventExecutor的run方法,这个后面再一起分析


回到上文children的初始化的代码中来,初始化好之后,开始为一个child赋值:

children[i] = newChild(threadFactory, args);

看newChild方法:

@Override
    protected EventExecutor newChild(
            ThreadFactory threadFactory, Object... args) throws Exception {
        return new NioEventLoop(this, threadFactory, (SelectorProvider) args[0]);
}

NioEventLoop(NioEventLoopGroup parent, ThreadFactory threadFactory, SelectorProvider selectorProvider) {
        super(parent, threadFactory, false);
        if (selectorProvider == null) {
            throw new NullPointerException("selectorProvider");
        }
        provider = selectorProvider;
        selector = openSelector();
}

protected SingleThreadEventLoop(EventLoopGroup parent, ThreadFactory threadFactory, boolean addTaskWakesUp) {
        super(parent, threadFactory, addTaskWakesUp);
 }

可以看出最后还是调用的SingleThreadEventExecutor的构造函数,期间做了selector的处理,我们知道NIO是基于事件驱动的,所以netty也是离不开selector,所以每一个NioEventLoop(上文描述的SingleThreadEventLoop的子类)都需要绑定一个selector,这样在netty初始化channel的时候,只需要将channel绑定selector就可以了,关于openSelector这个方法,netty也进行了大量的优化,关于openselector这个方法,大家可以参考http://budairenqin.iteye.com/blog/2215896,我这个就不重复介绍了


好了,到此为止NioEventLoopGroup的源码就已经分析完了,此时NioEventLoopGroup中的每一个NioEventLoop中绑定的thread还没有start,我们接着看


step2:

回到代码一:

ServerBootstrap sbs = new ServerBootstrap();
sbs.group(bossGroup,workerGroup);

group方法是将我们刚刚初始化好的bossGroup,workerGroup分别进行赋值(当然需要进行赋值绑定啦,否则实例化他们干啥用,实例化就是为了用,多说两句,不忘初心,bossGroup和workerGroup这两个的使命是什么,重述一遍,bossGroup和workerGroup中维护的都是维护了两个队列,一个thread,boss的thread线程是用来接收链接的然后转发给worker处理,worker列队就是一个个任务,等待worker的thread做处理,但是这两个group中的thread都没有start呢,并且selector都没有注册,哈哈,不忘初心,做人也是一样)


废话不多说,workerGroup赋值给ServerBootstrap实例的childGroup,bossGroup赋值给AbstractBootstrap的group,具体代码如下:

ServerBootstrap.java

private volatile EventLoopGroup childGroup;

public ServerBootstrap group(EventLoopGroup parentGroup, EventLoopGroup childGroup) {
        super.group(parentGroup);
        if (childGroup == null) {
            throw new NullPointerException("childGroup");
        }
        if (this.childGroup != null) {
            throw new IllegalStateException("childGroup set already");
        }
        this.childGroup = childGroup;
        return this;
}

AbstractBootstrap.java

private volatile EventLoopGroup group;

AbstractBootstrap(AbstractBootstrap<B, C> bootstrap) {
        group = bootstrap.group;
        channelFactory = bootstrap.channelFactory;
        handler = bootstrap.handler;
        localAddress = bootstrap.localAddress;
        synchronized (bootstrap.options) {
            options.putAll(bootstrap.options);
        }
        synchronized (bootstrap.attrs) {
            attrs.putAll(bootstrap.attrs);
        }
}

接着看代码一:

sbs.channel(NioServerSocketChannel.class);

这个channel方法则是规定了这个ServerBootstrap的channel是啥,这里面维护了一个叫做channelFactory的东西,顾名思义,这是一个channel的工厂,我们传递的是NioServerSocketChannel这个类,那么这个channelFactory工厂创建出来的channel就是NioServerSocketChannel



去掉一些handler类,我们看:

ChannelFuture future = sbs.bind(port).sync();

我们看bind方法

public ChannelFuture bind(int inetPort) {
        return bind(new InetSocketAddress(inetPort));
}

public ChannelFuture bind(SocketAddress localAddress) {
        validate();
        if (localAddress == null) {
            throw new NullPointerException("localAddress");
        }
        return doBind(localAddress);
}

private ChannelFuture doBind(final SocketAddress localAddress) {
        final ChannelFuture regFuture = initAndRegister();
        final Channel channel = regFuture.channel();
        if (regFuture.cause() != null) {
            return regFuture;
        }

        final ChannelPromise promise;
        if (regFuture.isDone()) {
            promise = channel.newPromise();
            doBind0(regFuture, channel, localAddress, promise);
        } else {
            // Registration future is almost always fulfilled already, but just in case it's not.
            promise = new PendingRegistrationPromise(channel);
            regFuture.addListener(new ChannelFutureListener() {
                @Override
                public void operationComplete(ChannelFuture future) throws Exception {
                    doBind0(regFuture, channel, localAddress, promise);
                }
            });
        }

        return promise;
}



最后调用了AbstractBootstrap(这个我们刚才做过处理,把bossGroup赋值了AbstractBootstrap的group,也就是说这个AbstractBootstrap获取到一个thread执行器了)

言归正传,我们看doBind方法:

(代码三)

final ChannelFuture regFuture = initAndRegister();



final ChannelFuture initAndRegister() {
        final Channel channel = channelFactory().newChannel();
        try {
            init(channel);
        } catch (Throwable t) {
            channel.unsafe().closeForcibly();
            return channel.newFailedFuture(t);
        }

        ChannelFuture regFuture = group().register(channel);
        if (regFuture.cause() != null) {
            if (channel.isRegistered()) {
                channel.close();
            } else {
                channel.unsafe().closeForcibly();
            }
        }

        return regFuture;
    }

好了,我们看到了channelFactory了,那么它new出来的channel当然就是NioServerSocketChannel

(代码四)

@Override
        public T newChannel() {
            try {
                return clazz.newInstance();
            } catch (Throwable t) {
                throw new ChannelException("Unable to create Channel from class " + clazz, t);
            }
        }

这边要提及的是,我们创建NioServerSocketChannel的实例的时候,会触发它的父类AbstractChannel的构造函数:

protected AbstractChannel(Channel parent) {
        this.parent = parent;
        unsafe = newUnsafe();
        pipeline = new DefaultChannelPipeline(this);
}

unsafe = newUnsafe()这个是我们需要关注的,unsafe看起来就很熟悉,不过此unsafe并不是juc包那个CAS的unsafe,我们看看它的抽象:

源码位于io.netty.channel.Channel的Unsafe

interface Unsafe {
        /**
         * Return the {@link SocketAddress} to which is bound local or
         * {@code null} if none.
         */
        SocketAddress localAddress();

        /**
         * Return the {@link SocketAddress} to which is bound remote or
         * {@code null} if none is bound yet.
         */
        SocketAddress remoteAddress();

        /**
         * Register the {@link Channel} of the {@link ChannelPromise} with the {@link EventLoop} and notify
         * the {@link ChannelFuture} once the registration was complete.
         */
        void register(EventLoop eventLoop, ChannelPromise promise);



我列出部分代码的意思就是这个unsafe其实是束缚于Channel的,但也提供了channel的一些方法,比如register(EventLoop eventLoop, ChannelPromise promise)这个方法,将channel注册到eventLoop上去,这是很关键的,我们再回归到AbstractChannel的newUnsafe方法上来,追踪源代码最终发现:

@Override
    protected AbstractNioUnsafe newUnsafe() {
        return new NioMessageUnsafe();
}



好了,最终NioServerSocketChannel这个channel对应的Unsafe是NioMessageUnsafe,关于AbstractChannel的newUnsafe这只是一个插曲(下文会用到),我们还是回到【代码三】

final Channel channel = channelFactory().newChannel();

init(channel);

init的方法,很重要,这里面规范了channel的一些属性,我们看看位于ServerBootstrap中的init方法

@Override
    void init(Channel channel) throws Exception {
        final Map<ChannelOption<?>, Object> options = options();
        synchronized (options) {
            channel.config().setOptions(options);
        }

        final Map<AttributeKey<?>, Object> attrs = attrs();
        synchronized (attrs) {
            for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) {
                @SuppressWarnings("unchecked")
                AttributeKey<Object> key = (AttributeKey<Object>) e.getKey();
                channel.attr(key).set(e.getValue());
            }
        }

        ChannelPipeline p = channel.pipeline();
        if (handler() != null) {
            p.addLast(handler());
        }

        final EventLoopGroup currentChildGroup = childGroup;
        final ChannelHandler currentChildHandler = childHandler;
        final Entry<ChannelOption<?>, Object>[] currentChildOptions;
        final Entry<AttributeKey<?>, Object>[] currentChildAttrs;
        synchronized (childOptions) {
            currentChildOptions = childOptions.entrySet().toArray(newOptionArray(childOptions.size()));
        }
        synchronized (childAttrs) {
            currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(childAttrs.size()));
        }

        p.addLast(new ChannelInitializer<Channel>() {
            @Override
            public void initChannel(Channel ch) throws Exception {
                ch.pipeline().addLast(new ServerBootstrapAcceptor(
                        currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
            }
        });
    }

因为channel这边有很多option,另外我们看到了childGroup(我们刚才赋值的workerGroup),childHandler也就是handlers,也就是channel配置了options,获取到线程执行器了,最后我们需要关注的就是channel中的ChannelPipeline中增加了一个ServerBootstrapAcceptor(后面再分析,这边就是说明一下,在channelPipeLine中有了ServerBootstrapAcceptor)


回到initAndRegister这个方法中来,init完了channel,最最关键的一步就是

ChannelFuture regFuture = group().register(channel);

还是不忘初心,在NIO的编程中有一个很重要的步骤,我们可以将这个步骤写出来:

SocketChannel channel = SocketChannel.open();
channel.configureBlocking(false);
InetSocketAddress s = new InetSocketAddress("localhost", 2000);
channel.connect(s);

Selector selector = Selector.open();
channel.register(selector, SelectionKey.OP_CONNECT | SelectionKey.OP_READ);

这是一段很经典的纯的Java的NIO的的client代码。


这边我们看channel.register这行代码,我们需要将channel注册到selector上,并且传递自己感兴趣的事件参数,这样当有selector上有事件发生的是,就会通知注册的channel触发对应的事件


我们在分析netty的时候,我们知道在分析NioEventLoopGroup源码的时候,我们知道每一个NioEventLoopGroup都维护了一个selector,再后文中我们也分析了一个channel的创建过程,但是还没有进行selector和selector进行register


这边的group().register(channel)最后调用的是:

channel.unsafe().register(this, promise);

回到一圈,我们有看到了unsafe,这个unsafe上文已经分析过了,是NioMessageUnsafe,register是它父类AbstractUnsafe完成的动作

我们也注意下register的入参,this指代的是SingleThreadEventLoop这个实例(其实我要的不是这个实例,而是这个实例中的selector(位于NioEventLoop.java中))


最后的注册动作是在AbstractNioChannel.java中完成的

@Override
    protected void doRegister() throws Exception {
        boolean selected = false;
        for (;;) {
            try {
                selectionKey = javaChannel().register(eventLoop().selector, 0, this);
                return;
            } catch (CancelledKeyException e) {
                if (!selected) {
                    // Force the Selector to select now as the "canceled" SelectionKey may still be
                    // cached and not removed because no Select.select(..) operation was called yet.
                    eventLoop().selectNow();
                    selected = true;
                } else {
                    // We forced a select operation on the selector before but the SelectionKey is still cached
                    // for whatever reason. JDK bug ?
                    throw e;
                }
            }
        }
    }

这边我们要注意evetLoop就是我们刚才说的this,也就是SingleThreadEventLoop抽象类的具体实现,这边获取到了selector,javaChannel()这个方法就是

protected SelectableChannel javaChannel() {
        return ch;
    }
private final SelectableChannel ch;

估计这边大家要问这个ch是什么时候赋值的,请看【代码四】

在newInstant的时候创建NioServerSocketChannel实例的时候,进行了赋值,这边ch其实就是NioServerSocketChannel

好了,到此为止,我们基本分析了Netty的server端的启动过程,你也许会问server的thread貌似还没有start吧,其实不然,你可以仔细观察一下register0这个方法,是这么调用的:

try {
                    eventLoop.execute(new OneTimeTask() {
                        @Override
                        public void run() {
                            register0(promise);
                        }
                    });
                } catch (Throwable t) {
                    logger.warn(
                            "Force-closing a channel whose registration task was not accepted by an event loop: {}",
                            AbstractChannel.this, t);
                    closeForcibly();
                    closeFuture.setClosed();
                    safeSetFailure(promise, t);
}

这个eventLoop其实就是AbstractBootstrap的group,像注册这种动作也是一种事件,所以需要Bootstrap的自带的thread去做这个动作,最终会调用SingleThreadEventExecutor的execute的方法:

@Override
    public void execute(Runnable task) {
        if (task == null) {
            throw new NullPointerException("task");
        }

        boolean inEventLoop = inEventLoop();
        if (inEventLoop) {
            addTask(task);
        } else {
            startThread();
            addTask(task);
            if (isShutdown() && removeTask(task)) {
                reject();
            }
        }

        if (!addTaskWakesUp && wakesUpForTask(task)) {
            wakeup(inEventLoop);
        }
    }



最后会调用startThread方法,归于自然,归于本心,调用SingleThreadEventExecutor的

private void startThread() {
        if (STATE_UPDATER.get(this) == ST_NOT_STARTED) {
            if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
                delayedTaskQueue.add(new ScheduledFutureTask<Void>(
                        this, delayedTaskQueue, Executors.<Void>callable(new PurgeTask(), null),
                        ScheduledFutureTask.deadlineNanos(SCHEDULE_PURGE_INTERVAL), -SCHEDULE_PURGE_INTERVAL));
                thread.start();
            }
        }
    }



thrad的start调用的方法就是我们刚才片段二种的

SingleThreadEventExecutor.this.run();




好了,先分析这么多吧,我们进行总结一下,分析太多,我自己也记不过来

1)Netty的server端代码一开始初始化了两个EventLoopGroup,其实就是初始化EventLoop,每一个EventLoop的具体实现就是维护了一个任务队列,一个延迟任务队列,一个thread,并且每一个EventLoop都有一个属于自己的Executor执行器,这样做的好处就是每一个唯一的thread去不停的循环调用,去执行任务队列和延迟任务队列中的task,没有了上下文的切换们还要记得每一个EventLoop还初始化了一个selector,关于selector的创建,netty做了很大的优化,使其与CPU更加亲和

(中间还忘记分析了,CPU是2的倍数的时候,Netty的优化,大家可以自己看下)

2)关于serverBootstrap的初始化,主要就是做了channel的创建,channel的执行器的绑定,option属性的配置,绑定端口,这些配置好了之后就是channel和selector的绑定,绑定的时候,顺带启动一些AbstractBootstrap中的thread,让其进行无限循环中

3)关于细节

①serverBootstrap中在channelPipeline中偷偷地加了一个ServerBootstrapAcceptor

②serverBootstrap中的boss线程对应的unsafe对象是NioMessageUnsafe实例