gRPC 服务端流式RPC基于python的应用

应用场景： python服务端 java客户端 通过grpc进行通信

java客户端
  2. 在与Sources同级的文件夹下新建proto文件夹，在proto文件夹中新建 msg.proto

    3. grpc流量控制策略_python

  3. msg.proto内容如下
// @1 使用proto3语法
syntax = "proto3";
// @2 生成多个类（一个类方便管理）
option java_multiple_files = false;
// @3 生成java类所在包
option java_package = "com.example.spbclient.proto";
// @4 生成外层类类名
option java_outer_classname = "MsgProto";

// @6 .proto包名(逻辑包名)
package msg;

// @7 定义服务，用于描述要生成的API接口，类似于Java的业务逻辑接口类
service MsgService {
  // imgIdentify 方法名 ImgRequest 传入参数  ImgResponse 返回响应

  //注意：这里是returns 不是return
  // Unary RPC（单项RPC）
  rpc GetMsg (MsgRequest) returns (MsgResponse) {}

  // A server-to-client streaming RPC.（服务端流式RPC）
  rpc GetMsgByServerStream(MsgRequest) returns (stream MsgResponse) {}

  // A client-to-server streaming RPC. （客户端流式RPC）
  rpc GetMsgByClientStream(stream MsgRequest) returns (MsgResponse) {}

  // A Bidirectional streaming RPC.（双向流式RPC）
  rpc GetMsgByBidirectionalStream(stream MsgRequest) returns (stream MsgResponse) {}
}
//定义请求数据结构
// string 数据类型
// calName 参数名称
// 1 序号、索引值（表示第一个参数，防止传参顺序错乱），一旦开始就不能改变
// int32 int
// int64 long
// 不可以使用 19000-19999 保留字
// TODO 大小写问题
message MsgRequest {
  string name = 1;
}

message MsgResponse {
  string msg = 1;
}
  1. pom.xml文件增加依赖和插件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http:///POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http:///POM/4.0.0 https:///xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.1.1</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>grpc-demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>grpc-demo</name>
    <description>grpc-demo</description>
    <properties>
        <java.version>17</java.version>

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <grpc.version>1.56.0</grpc.version><!-- CURRENT_GRPC_VERSION -->
        <protobuf.version>3.22.3</protobuf.version>
        <protoc.version>3.22.3</protoc.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>io.grpc</groupId>
                <artifactId>grpc-bom</artifactId>
                <version>${grpc.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!--grpc所需的依赖-->
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-netty-shaded</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-protobuf</artifactId>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-services</artifactId>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-stub</artifactId>
        </dependency>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java-util</artifactId>
            <version>${protobuf.version}</version>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.10.1</version> <!-- prevent downgrade via protobuf-java-util -->
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>31.1-jre</version> <!-- prevent downgrade of version in protobuf-java-util -->
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>annotations-api</artifactId>
            <version>6.0.53</version>
            <scope>provided</scope> <!-- not needed at runtime -->
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-testing</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>
<!--        <dependency>-->
<!--            <groupId>org.mockito</groupId>-->
<!--            <artifactId>mockito-core</artifactId>-->
<!--            <version>3.4.0</version>-->
<!--            <scope>test</scope>-->
<!--        </dependency>-->
        <dependency>
            <groupId>net.devh</groupId>
            <artifactId>grpc-client-spring-boot-starter</artifactId>
            <version>2.12.0.RELEASE</version>
        </dependency>
    </dependencies>

    <build>
        <extensions>
            <extension>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.7.1</version>
            </extension>
        </extensions>
        <plugins>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}
                    </protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}
                    </pluginArtifact>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>1.4.1</version>
                <executions>
                    <execution>
                        <id>enforce</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <requireUpperBoundDeps/>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

这里grpc1.56.0版本用到的mockito-core和springboot里引用的mockito-core发生了版本冲突，直接使用springboot引用的版本，目前没发现任何问题

  1. 直接编译maven项目，或者使用protobuf插件都可以生成grpc的代码
  2. 执行完成可以在target/generated-sources中看到生成的文件，主要grpc-java和java都需要是源文件类型，不是的话自己设置一下
  3. java作为客户端的代码
package com.example.grpc.demo.client;


import com.example.spbclient.proto.MsgProto;
import com.example.spbclient.proto.MsgServiceGrpc;
import io.grpc.Channel;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.stub.StreamObserver;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;

import java.util.Iterator;
import java.util.concurrent.TimeUnit;

@Slf4j
public class MsgClient {
    private final MsgServiceGrpc.MsgServiceBlockingStub msgServiceBlockingStub;
    private final MsgServiceGrpc.MsgServiceStub msgServiceStub;


    public MsgClient(Channel channel) {
        msgServiceBlockingStub = MsgServiceGrpc.newBlockingStub(channel);
        msgServiceStub = MsgServiceGrpc.newStub(channel);
    }

    /**
     * 给服务端发送请求
     *
     * @param name
     */
    public void getMsg(String name) {
        log.info("给服务端发送消息： " + name + "...");
        MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name).build();
        MsgProto.MsgResponse response = msgServiceBlockingStub.getMsg(request);
        log.info("收到服务端返回的消息: " + response.getMsg());
    }

    /**
     * 给服务端发送请求 服务端流式RPC
     *
     * @param name
     */
    public void getMsgByServerStream(String name) {
        log.info("给服务端发送消息： " + name + "...");
        MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name).build();

        Iterator<MsgProto.MsgResponse> msgResponseIterator;
        try {
            msgResponseIterator = msgServiceBlockingStub.getMsgByServerStream(request);
            for (int i = 1; msgResponseIterator.hasNext(); i++) {
                MsgProto.MsgResponse msgResponse = msgResponseIterator.next();
                log.info("第{}次收到服务端返回消息：{}", i, msgResponse.getMsg());
            }
        } catch (StatusRuntimeException e) {
            log.warn("RPC failed: {}", e.getStatus());
        }
    }

    /**
     * 给服务端发送请求 客户端流式RPC
     *
     * @param name
     */
    @SneakyThrows
    public void getMsgByClientStream(String name) {
        System.out.println("-----------------------------");
        System.out.println("流式请求-响应，getMsgByClientStream");
        StreamObserver<MsgProto.MsgResponse> responseObserver = new StreamObserver<MsgProto.MsgResponse>() {
            @Override
            public void onNext(MsgProto.MsgResponse msgResponse) {
                log.warn("getMsgByClientStream onNext:{}", msgResponse.getMsg());
            }

            @Override
            public void onError(Throwable t) {
                log.error("getMsgByClientStream onError: {}", t.toString());
            }

            @Override
            public void onCompleted() {
                log.warn("getMsgByClientStream onCompleted");
            }
        };

        StreamObserver<MsgProto.MsgRequest> requestObserver = msgServiceStub.getMsgByClientStream(responseObserver);

        try {
            //循环发送100次消息
            for (int i = 0; i < 5; i++) {
                MsgProto.MsgRequest request = MsgProto.MsgRequest.newBuilder().setName(name + i).build();
                log.info(String.format("客户端发送%s", request.getName()));
                requestObserver.onNext(request);
                // Sleep for a bit before sending the next one.
                Thread.sleep(1000);
            }
        } catch (RuntimeException e) {
            // Cancel RPC
            requestObserver.onError(e);
            throw e;
        }
        requestObserver.onCompleted();
        try {
            Thread.sleep(50000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("客户端流式请求发送完毕");
    }


    /**
     * 双向流
     *
     * @param name
     */
    public void getMsgByBidirectionalStream(String name) {

        System.out.println("-----------------------------");
        System.out.println("流式请求-流式响应，调用BiTalk");
        StreamObserver<MsgProto.MsgRequest> requestStreamObserver = msgServiceStub.getMsgByBidirectionalStream(new StreamObserver<MsgProto.MsgResponse>() {
            @Override
            public void onNext(MsgProto.MsgResponse response) {
                System.out.println("收到服务端返回的数据" + response.getMsg());
            }

            @Override
            public void onError(Throwable throwable) {
                System.out.println(throwable.getMessage());
            }

            @Override
            public void onCompleted() {
                System.out.println("onComplated");
            }
        });

        for (int i = 0; i < 10; i++) {
            requestStreamObserver.onNext(MsgProto.MsgRequest.newBuilder().setName(name + i).build());
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }


        try {
            Thread.sleep(50000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }


    /**
     * 测试
     *
     * @param args
     * @throws InterruptedException
     */
    public static void main(String[] args) throws InterruptedException {
        String user = "Tom";
        String target = "localhost:50051";

        ManagedChannel channel = ManagedChannelBuilder.forTarget(target)
                .usePlaintext()
                .build();

        try {
            MsgClient client = new MsgClient(channel);
            //
//            client.getMsg(user);
            client.getMsgByServerStream(user);
//            client.getMsgByClientStream(user);
//            client.getMsgByBidirectionalStream(user);
        } finally {
            channel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
        }
    }


}
python服务端
  1. 新建python项目，项目中新建prtot包，把之前写好的msg.proto拷贝到包路径下，然后在该包路径下执行一下python命令，生成代码
pip install grpcio
pip install grpcio-tools googleapis-common-protos
python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. msg.proto
  1. 执行完成后我们可以看到proto文件夹下多了两个文件，msg_pb2.py和msg_pb2_grpc.py。
  2. 在proto包下的init.py文件中添加如下代码（否则无法引入proto包下的文件）
import sys
# 将__init__.py文件所在目录加入Python搜索目录中，否则会提示找不到myModule1
sys.path.append(__file__[:-12])
  1. 在项目目录下新建main_server.py文件
import grpc
import proto.msg_pb2 as msg_pb2
import proto.msg_pb2_grpc as msg_pb2_grpc

#  并发
from concurrent import futures
import time

# Unary RPC（单项RPC）

_ONE_DAY_IN_SECONDS = 60 * 60 * 24
# service 实现GetMsg方法 MsgServicer继承MsgServiceServicer
class MsgServicer(msg_pb2_grpc.MsgServiceServicer):

    def GetMsg(self, request, context):
        print("收到客户端请求: %s" % request.name)
        return msg_pb2.MsgResponse(msg='Hello, %s!' % request.name)

    def GetMsgByServerStream(self, request, context):
        print("流式服务器收到客户端请求: %s" % request.name)
        while True:
            yield msg_pb2.MsgResponse(msg='Hello, %s!' % request.name)
            time.sleep(1)

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    msg_pb2_grpc.add_MsgServiceServicer_to_server(MsgServicer(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    try:
        while True:
            time.sleep(_ONE_DAY_IN_SECONDS)
    except KeyboardInterrupt:
        server.stop(0)


if __name__ == '__main__':
    serve()
  1. 启动python服务端，再启动java客户端即可看到 流式服务端的效果

总结：grpc流式服务端，实际就是客户端发送个消息，表示你可以给我发消息了，然后服务端就开始一直给客户端返回消息。