Microservices based webapplication in golang using grpc.

GRPC is “A high performance, open source, general RPC framework that puts mobile and HTTP/2 first” , i was pretty impressed with its architecture after going thorough the announcement on feb 26th 2015. I was keen on microservices and
wanted to try out this framework for my webapplication. GRPC is after all an RPC ,meaning you can invoke methods remotely if you follow the framework. To be more clear to begginers , you can invoke a method running in a different process.

I initially thought that i can host httpservices as a grpc server but unfortunately got to know that browser to grpc server is not yet there ,i posted few questions in grpc-io chat rooms & forums. But it doesnt stop you from using GRPC , hence
this article…

Microservices has become the buzzword since last year and many architects are exploring the benefits that they can get by employing it. But you have to be carefull choosing services that can be exposed as microservices, try to group them as much possible and reduce number of processes on the system.
I felt that go is really nice and offers better flexibility as the process size appears to be lesser than other languages. I see atleast the overhead less. This artcile is not about why microservice & how should you choose your services ? rather
i wanted to put how you can write microservice based webapplication in golang using grpc.
In this artcile i have choosen two services 1) Catalog service 2) Recommendation service as microservices. The webapplication is built using go’s “net” package .

The design is like this… Webapplicaiton in golang , two microservices in the backend. I remember reading a microservices article on infoq with similar services.

Webserver (main.go): You can create a webserver in golang with very few lines… Below is the code..

http.HandleFunc(“/catalog”, GetProductCatalog)
http.Handle(“/”, http.FileServer(http.Dir(“public”)))
http.ListenAndServe(“:8080”, nil)

The http listenandserve binds the port and listens for http traffic. The handlefunc takes url or path as parameter and invokes the function when such path is hit. I have put static resources in the public directory , so this pathis looked up for static resources.
Protocol buffers: Protocol buffers is the binary serialization protocol for easier and efficient transfer of data between processes. proto3 is the version used in grpc .Three protocol buffer files were created and put in the directory “proto”.
The corresponding go code is generated by running the command “protoc -I ../protos ../protos/*.proto –go_out=plugins=grpc:basicwebapp”. I have checked in the generated files as well for easier understanding.

CatalogServer/CatalogServiceServer : Sorry for the naming convention here if its confusing . The go process is CatalogServicesServer and it serves “catalog services” . You should be able to navigate to folder “CatalogServer” in the source code at github
run “go build CatalogServiceServer.go” in the commandline . If you are successful then catalogserviceserver.exe gets generated. run it to start the catalog server . The primary purpose of this microservice is to deliver catalog of products.
I hard coded the response to a static list of products, you may wish to connect to a database and retrive them.

The rpc method for the catalogserver is define in the “Catalogservice.proto” file . You can find it in the “proto” directory.

service CatalogService {

rpc GetProductCatalog(CatalogRequest) returns (CatalogResponse) {}

}

message Category {

string categoryName=1;
}

message CatalogResponse {

repeated Product products =1;

}

message CatalogRequest {

Category category=1;
}

As you can see the method “GetProductCatalog” is the rpc method . This method is implemented in the catalogserviceserver file. The method takes a catalogrequest and gives back catalogresponse.

recommendationserver/RecommendationServiceServer: This is the second microservice that i used. Its primary purpose is to take a product as input and send back the recommendations as response. The protocol buffer file that declares
the rpc method and serialization format is inside “proto” folder… below is the gist of it..

service RecommendationService {

rpc GetRecommendations(Product) returns (RecommendationResponse);
}
message RecommendationResponse{

message Recommendation{

int32 rating =1;

int32 productid=2;
}
repeated Recommendation result=1;

}

Complile the recommendation server and run it before running the webserver..

Communication between grpc servers and webserver :

1. Run main.go

2. Type localhost:8080 in the browser.. you should be able to see html page with submit button like below.

3. By clicking on the submit button the form is submitted to the server . The target path is “catalog” so it will be handled by the getproductcatalog callback function. Find the below code in the function which is responsible for calling microservice..

conn, err := grpc.Dial(catalogServerAddress, opts…) //DIaling in to the catalogServerAddress “port is hardcoded at 50052”

if err !=nil {
log.Fatal(“fail to dial %v”, err)
}

defer conn.Close()

client := pb.NewCatalogServiceClient(conn) // Create new Catalogserviceclient stub.
stream, err := client.GetProductCatalog(context.Background(), &pb.CatalogRequest{&pb.Category{CategoryName:”Computer”}}) // Invoke the remote method “GetProductCatalog” and get the response into stream variable..

Source code : https://github.com/devicharan/basicwebapp

Whats out of scope of this article : Service discovery : The servers are known in the article and hence hardcoded. But in real world you need a proper server discovery with loadbalancing, zookeeper or consul could be the right choices.

protoc grpc golang

How to generate code from proto files..

1. I had visualstudio 2010 with me and straight away i was not able to compile and generate protoc.exe. It threw lot of LNK errors. So i did install service pack of VS2010 to get rid of them.

2. Rebuilding the generated code as specified in https://github.com/grpc/grpc-common/tree/master/go

The commands that were given there were not clear, you have to do few things..

a.  go get google.golang.org/grpc

b.  Grab the code from the repository and install the proto package.

go get -u github.com/golang/protobuf/{proto,protoc-gen-go}   do get both of them..

c.  set PATH=%PATH%;C:\code\go\bin

d. go to protoc-gen-go folder and build it.

e. Also set protoc.exe in the patch. (“the one you have built using visual studio).

f. Now you can generate code for proto files by typing ..

protoc –go_out=. helloworld.proto

Update : You can get protoc.exe from https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-2 as well.