Exposing gRPC services as JSON over Http

Exposing gRPC services as JSON over Http

While GJEX supports writing gRPC services, there can be cases where the gRPC service needs to be exposed as a JSON over Http API. Few examples can be.

• Support for testing services using JSON/Http1.x
• Using existing JSON testing tools like Postman to validate the APIs
• Existing integration testing frameworks may support only JSON/Http interfaces
• Slow moving clients of services get extra time for adoption
• Can be used by clients built on language platforms that do not support gRPC.

gRPC-JSON Transcoder

gRPC-JSON Transcoder is an out of the box solution that can expose a gRPC service as a RESTful JSON API.

How it works

Envoy proxy acts as a reverse proxy for the gRPC service. It can be configured as a filter in Envoy proxy which allows a RESTful JSON API client to send requests to Envoy over HTTP and get proxied to a gRPC service.

Steps to achieve this

Complete working example of a sample UserService is available : grpc-jexpress-template.

Step 1: The HTTP mapping for the gRPC service has to be defined by custom options.

service UserService {
    rpc GetUser (GetRequest) returns (GetResponse) {
        option (google.api.http) = {
           get: "/v1/userservice/{id}"
       };
    }

    rpc CreateUser (CreateRequest) returns (CreateResponse) {
        option (google.api.http) = {
           post: "/v1/userservice"
           body: "*"
       };
    }
}

Here

         option (google.api.http) = {
            get: "/v1/userservice/{id}"
        };

and

     option (google.api.http) = {
           post: "/v1/userservice"
           body: "*"
       };

is the extra information added. Here get or post specifies the HTTP request method. The value of get or post corresponds to the request URL. Inside the URL there is a path variable called id. This path variable is automatically mapped to a field with the same name in the input operation. In this example it will be GetRequest.id.

Step 2: Generate a proto descriptor set using protoc command. If get protoc (if not already available), download stable protobuf compiler. https://github.com/protocolbuffers/protobuf/releases?after=v3.6.0 We have tested with protoc-3.5.1-osx-x86_64.zip. Optionally we have added in path so that this command is available everywhere.

    protoc -I.  -I${proto_dependencies} -Isrc/main/proto --include_imports  --include_source_info  --descriptor_set_out=sample_proto_descriptor_set.pb  src/main/proto/userservice.proto

Step 3: Run Envoy proxy pointing to the proto descriptor set using Docker. To install docker please follow this link.

    sudo docker run -it --rm --name envoy -p 51051:51051 -p 9901:9901 -v "$(pwd)/sample_service_definition.pb:/tmp/sample_proto_descriptor_set.pb:ro" -v "$(pwd)/envoy.yml:/etc/envoy/envoy.yaml:ro" envoyproxy/envoy

Sample envoy.yml

admin:
  access_log_path: /tmp/admin-access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
  - name: listener1
    address:
      socket_address: { address: 0.0.0.0, port_value: 51051 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          stat_prefix: grpc_json
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" , grpc: {}}
                route: { cluster: grpc, timeout: { seconds: 60 } }
          http_filters:
          - name: envoy.grpc_json_transcoder
            config:
              proto_descriptor: "/tmp/sample_proto_descriptor_set.pb"
              services: ["service.UserService"]
              print_options:
                add_whitespace: true
                always_print_primitive_fields: true
                always_print_enums_as_ints: false
                preserve_proto_field_names: false
          - name: envoy.router

  clusters:
  - name: grpc
    connect_timeout: 1.25s
    type: logical_dns
    lb_policy: round_robin
    dns_lookup_family: V4_ONLY
    http2_protocol_options: {}
    hosts:
    - socket_address:
        address: docker.for.mac.localhost
        port_value: 50051

Step 4: Testing the REST API

curl -XPOST "http://localhost:51051/v1/userservice" -H 'Content-Type: application/json' -d  '{ "userName": "Foo"}'

curl "http://localhost:51051/v1/userservice/1" -H 'Content-Type: application/json'