gRPC Meets .NET SDK And Visual Studio: Automatic Codegen On Build

Posted on Tuesday, December 18, 2018 by Kirill 'kkm' Katsnelson

As part of Microsoft’s move towards its cross-platform .NET offering, they have greatly simplified the project file format, and allowed a tight integration of third-party code generators with .NET projects. We are listening, and now proud to introduce integrated compilation of Protocol Buffer and gRPC service .proto files in .NET C# projects starting with the version 1.17 of the Grpc.Tools NuGet package, now available from Nuget.org.

You no longer need to use hand-written scripts to generate code from .proto files: The .NET build magic handles this for you. The integrated tools locate the proto compiler and gRPC plugin, standard Protocol Buffer imports, and track dependencies before invoking the code generators, so that the generated C# source files are never out of date, at the same time keeping regeneration to the minimum required. In essence, .proto files are treated as first-class sources in a .NET C# project.

A Walkthrough

In this blog post, we’ll walk through the simplest and probably the most common scenario of creating a library from .proto files using the cross-platform dotnet command. We will implement essentially a clone of the Greeter library, shared by client and server projects in the C# Helloworld example directory .

Create a new project

Let’s start by creating a new library project.

~/work$ dotnet new classlib -o MyGreeter
The template "Class library" was created successfully.

~/work$ cd MyGreeter
~/work/MyGreeter$ ls -lF
total 12
-rw-rw-r-- 1 kkm kkm   86 Nov  9 16:10 Class1.cs
-rw-rw-r-- 1 kkm kkm  145 Nov  9 16:10 MyGreeter.csproj
drwxrwxr-x 2 kkm kkm 4096 Nov  9 16:10 obj/

Observe that the dotnet new command has created the file Class1.cs that we won’t need, so remove it. Also, we need some .proto files to compile. For this exercise, we’ll copy an example file examples/protos/helloworld.proto from the gRPC distribution.

~/work/MyGreeter$ rm Class1.cs
~/work/MyGreeter$ wget -q https://raw.githubusercontent.com/grpc/grpc/master/examples/protos/helloworld.proto

(on Windows, use del Class1.cs, and, if you do not have the wget command, just open the above URL and use a Save As… command from your Web browser).

Next, add required NuGet packages to the project:

~/work/MyGreeter$ dotnet add package Grpc
info : PackageReference for package 'Grpc' version '1.17.0' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.
~/work/MyGreeter$ dotnet add package Grpc.Tools
info : PackageReference for package 'Grpc.Tools' version '1.17.0' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.
~/work/MyGreeter$ dotnet add package Google.Protobuf
info : PackageReference for package 'Google.Protobuf' version '3.6.1' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.

Add .proto files to the project

Next comes an important part. First of all, by default, a .csproj project file automatically finds all .cs files in its directory, although Microsoft now recommends suppressing this globbing behavior, so we too decided against globbing .proto files. Thus the .proto files must be added to the project explicitly.

Second of all, it is important to add a property PrivateAssets="All" to the Grpc.Tools package reference, so that it will not be needlessly fetched by the consumers of your new library. This makes sense, as the package only contains compilers, code generators and import files, which are not needed outside of the project where the .proto files have been compiled. While not strictly required in this simple walkthrough, it must be your standard practice to do that always.

So edit the file MyGreeter.csproj to add the helloworld.proto so that it will be compiled, and the PrivateAssets property to the Grpc.Tools package reference. Your resulting project file should now look like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Google.Protobuf" Version="3.6.1" />
    <PackageReference Include="Grpc" Version="1.17.0" />

    <!-- The Grpc.Tools package generates C# sources from .proto files during
         project build, but is not needed by projects using the built library.
         It's IMPORTANT to add the 'PrivateAssets="All"' to this reference: -->
    <PackageReference Include="Grpc.Tools" Version="1.17.0" PrivateAssets="All" />

    <!-- Explicitly include our helloworld.proto file by adding this line: -->
    <Protobuf Include="helloworld.proto" />
  </ItemGroup>

</Project>

Build it!

At this point you can build the project with the dotnet build command to compile the .proto file and the library assembly. For this walkthrough, we’ll add a logging switch -v:n to the command, so we can see that the command to compile the helloworld.proto file was in fact run. You may find it a good idea to always do that the very first time you compile a project!

Note that many output lines are omitted below, as the build output is quite verbose.

~/work/MyGreeter$ dotnet build -v:n

Build started 11/9/18 5:33:44 PM.
  1:7>Project "/home/kkm/work/MyGreeter/MyGreeter.csproj" on node 1 (Build target(s)).
   1>_Protobuf_CoreCompile:
      /home/kkm/.nuget/packages/grpc.tools/1.17.0/tools/linux_x64/protoc
        --csharp_out=obj/Debug/netstandard2.0
        --plugin=protoc-gen-grpc=/home/kkm/.nuget/packages/grpc.tools/1.17.0/tools/linux_x64/grpc_csharp_plugin
        --grpc_out=obj/Debug/netstandard2.0 --proto_path=/home/kkm/.nuget/packages/grpc.tools/1.17.0/build/native/include
        --proto_path=. --dependency_out=obj/Debug/netstandard2.0/da39a3ee5e6b4b0d_helloworld.protodep helloworld.proto
     CoreCompile:

        [ ... skipping long output ... ]

       MyGreeter -> /home/kkm/work/MyGreeter/bin/Debug/netstandard2.0/MyGreeter.dll

Build succeeded.

If at this point you invoke the dotnet build -v:n command again, protoc would not be invoked, and no C# sources would be compiled. But if you change the helloworld.proto source, then its outputs will be regenerated and then recompiled by the C# compiler during the build. This is a regular dependency tracking behavior that you expect from modifying any source file.

Of course, you can also add .cs files to the same project: It is a regular C# project building a .NET library, after all. This is done in our RouteGuide example.

Where are the generated files?

You may wonder where the proto compiler and gRPC plugin output C# files are. By default, they are placed in the same directory as other generated files, such as objects (termed the “intermediate output” directory in the .NET build parlance), under the obj/ directory. This is a regular practice of .NET builds, so that autogenerated files do not clutter the working directory or accidentally placed under source control. Otherwise, they are accessible to the tools like the debugger. You can see other autogenerated sources in that directory, too:

~/work/MyGreeter$ find obj -name '*.cs'
obj/Debug/netstandard2.0/MyGreeter.AssemblyInfo.cs
obj/Debug/netstandard2.0/Helloworld.cs
obj/Debug/netstandard2.0/HelloworldGrpc.cs

(use dir /s obj\*.cs if you are following this walkthrough from a Windows command prompt).

There Is More To It

While the simplest default behavior is adequate in many cases, there are many ways to fine-tune your .proto compilation process in a large project. We encourage you to read the documentation file BUILD-INTEGRATION.md for available options if you find that the default arrangement does not suit your workflow. The package also extends the Visual Studio’s Properties window, so you may set some options per file in the Visual Studio interface.

“Classic” .csproj projects and Mono are also supported.

Share Your Experience

As with any initial release of a complex feature, we are thrilled to receive your feedback. Did something not work as expected? Do you have a scenario that is not easy to cover with the new tools? Do you have an idea how to improve the workflow in general? Please read the documentation carefully, and then open an issue in the gRPC code repository on GitHub. Your feedback is important to determine the future direction for our build integration work!