Protobuf and gRPC rules for Bazel

Bazel rules for building Protocol Buffers ± gRPC code and libraries from proto_library targets

• 2019/10/10: Bazel 1.0.0 has been released but is not yet supported fully by rules_proto_grpc. Please follow the Bazel 1.0.0 Support issue if you are waiting for support, which will be released in version 1.0.0 of rules_proto_grpc. If you are seeing issues with Bazel 1.0.0 that are not mentioned or linked in that issue, please add a comment or open a new issue.

• Overview
• Installation
• Rules
  • Android
  • Closure
  • C++
  • C#
  • D
  • Go
  • Go (gogoprotobuf)
  • grpc-gateway
  • gRPC-Web
  • grpc.js
  • Java
  • Node.js
  • Objective-C
  • PHP
  • Python
  • Ruby
  • Rust
  • Scala
  • Swift
• Example Usage
• Migration
• Developers
  • Code Layout
  • Rule Generation
  • How-it-works
  • Developing Custom Plugins
• License
• Contributing

These rules provide Protocol Buffers (Protobuf) and gRPC rules for a range of languages and services.

Each supported language ({lang} below) is generally split into four rule flavours:

• {lang}_proto_compile: Provides generated files from the Protobuf protoc plugin for the language. e.g for C++ this provides the generated *.pb.cc and *.pb.h files.

• {lang}_proto_library: Provides a language-specific library from the generated Protobuf protoc plugin outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc and *pb.h files, with the Protobuf library linked. For languages that do not have a 'library' concept, this rule may not exist.

• {lang}_grpc_compile: Provides generated files from both the Protobuf and gRPC protoc plugins for the language. e.g for C++ this provides the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files.

• {lang}_proto_library: Provides a language-specific library from the generated Protobuf and gRPC protoc plugins outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files, with the Protobuf and gRPC libraries linked. For languages that do not have a 'library' concept, this rule may not exist.

Therefore, if you are solely interested in the generated source code artifacts, use the {lang}_{proto|grpc}_compile rules. Otherwise, if you want a ready-to-go library, use the {lang}_{proto|grpc}_library rules.

These rules are derived from the excellent stackb/rules_proto and add aspect-based compilation to all languages, allowing for all proto_library options to be expressed in user code.

Add rules_proto_grpc your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_proto_grpc",
    urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/0.2.0.tar.gz"],
    sha256 = "1e08cd6c61f893417b14930ca342950f5f22f71f929a38a8c4bbfeae2a80d03e",
    strip_prefix = "rules_proto_grpc-0.2.0",
)

load("@rules_proto_grpc//:repositories.bzl", "rules_proto_grpc_toolchains")
rules_proto_grpc_toolchains()

It is recommended that you use the tagged releases for stable rules. Master is intended to be 'ready-to-use', but may be unstable at certain periods. To be notified of new releases, you can use GitHub's 'Watch Releases Only' on the repository.

Important: You will also need to follow instructions in the language-specific README.md for additional workspace dependencies that may be required.

These steps walk through the steps to go from a raw .proto file to a C++ library:

Step 1: Write a Protocol Buffer file (example: thing.proto):

syntax = "proto3";

package example;

import "google/protobuf/any.proto";

message Thing {
    string name = 1;
    google.protobuf.Any payload = 2;
}

Step 2: Write a BAZEL.build file with a native proto_library rule:

proto_library(
    name = "thing_proto",
    srcs = ["thing.proto"],
    deps = ["@com_google_protobuf//:any_proto"],
)

In this example we have a dependency on a well-known type any.proto, hence the proto_library to proto_library dependency ("@com_google_protobuf//:any_proto")

Step 3: Add a cpp_proto_compile rule (substitute cpp_ for the language of your choice).

NOTE: In this example thing.proto does not include service definitions (gRPC). For protos with services, use the cpp_grpc_compile rule instead.

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_compile")

cpp_proto_compile(
    name = "cpp_thing_proto",
    deps = [":thing_proto"],
)

But wait, before we can build this, we need to load the dependencies necessary for this rule (from cpp/README.md):

Step 4: Load the workspace macro corresponding to the build rule.

# WORKSPACE
load("@rules_proto_grpc//cpp:repositories.bzl", "cpp_repos")

cpp_repos()

We're now ready to build the rule:

Step 5: Build it!

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

If we were only interested in the generated file artifacts, the cpp_grpc_compile rule would be fine. However, for convenience we'd rather have the outputs compiled into an *.so file. To do that, let's change the rule from cpp_proto_compile to cpp_proto_library:

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_library")

cpp_proto_library(
    name = "cpp_thing_proto",
    deps = [":thing_proto"],
)

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-bin/example/proto/libcpp_thing_proto.a
  bazel-bin/example/proto/libcpp_thing_proto.so  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

This way, we can use //example/proto:cpp_thing_proto as a dependency of any other cc_library or cc_binary rule as per normal.

NOTE: the cpp_proto_library implicitly calls cpp_proto_compile, and we can access that rule by adding _pb at the end of the rule name, like bazel build //example/proto:cpp_thing_proto_pb.

For users migrating from the stackb/rules_proto rules, please see the help at MIGRATION.md

Each language {lang} has a top-level subdirectory that contains:

1. {lang}/README.md: Generated documentation for the language rules.

2. {lang}/repositories.bzl: Macro functions that declare repository rule dependencies for that language.

3. {lang}/{rule}.bzl: Rule implementations of the form {lang}_{kind}_{type}, where kind is one of proto|grpc and type is one of compile|library.

4. {lang}/BUILD.bazel: proto_plugin() declarations for the available plugins for the language.

5. example/{lang}/{rule}/: Generated WORKSPACE and BUILD.bazel demonstrating standalone usage of the rules.

6. {lang}/example/routeguide/: Example routeguide example implementation, if possible.

The repository root directory contains the base rule defintions:

• plugin.bzl: A build rule that defines the name, tool binary, and options for a particular proto plugin.

• aspect.bzl: Contains the implementation of the compilation aspect. This is shared by all rules and is the heart of rules_proto_grpc; it calls protoc with a given list of plugins and generates output files.

Additional protoc plugins and their rules are scoped to the github repository name where the plugin resides.

To help maintain consistency of the rule implementations and documentation, all of the rule implementations are generated by the tool //tools/rulegen. Changes in the main README.md should be placed in tools/rulegen/README.header.md or tools/rulegen/README.footer.md. Changes to generated rules should be put in the source files (example: tools/rulegen/java.go).

Briefly, here's how the rules work:

1. Using the proto_library graph, an aspect walks through the ProtoInfo providers on the deps attribute to {lang}_{proto|grpc}_compile. This finds all the directly and transitively required proto files, along with their options.

2. At each node visited by the aspect, protoc is invoked with the relevant plugins and options to generate the desired outputs. The aspect uses only the generated proto descriptors from the ProtoInfo providers.

3. Once the aspect stage is complete, all generated outputs are optionally gathered into a final output tree.

4. For {lang}_{proto|grpc}_library rules, the generated outputs are then aggregated into a language-specific library. e.g a .so file for C++.

Generally, follow the pattern seen in the multiple language examples in this repository. The basic idea is:

1. Load the plugin rule: load("@rules_proto_grpc//:plugin.bzl", "proto_plugin").
2. Define the rule, giving it a name, options (not mandatory), tool, and outputs.
3. tool is a label that refers to the binary executable for the plugin itself.
4. Choose your output type (pick one!):
   • outputs: a list of strings patterns that predicts the pattern of files generated by the plugin. For plugins that produce one output file per input proto file
   • out: the name of a single output file generated by the plugin.
   • output_directory: Set to true if your plugin generates files in a non-predictable way. e.g. if the output paths depend on the service names.
5. Create a compilation rule and aspect using the following template:

load("@rules_proto_grpc//:plugin.bzl", "ProtoPluginInfo")
load(
    "@rules_proto_grpc//:aspect.bzl",
    "ProtoLibraryAspectNodeInfo",
    "proto_compile_aspect_attrs",
    "proto_compile_aspect_impl",
    "proto_compile_attrs",
    "proto_compile_impl",
)

# Create aspect
example_aspect = aspect(
    implementation = proto_compile_aspect_impl,
    provides = [ProtoLibraryAspectNodeInfo],
    attr_aspects = ["deps"],
    attrs = dict(
        proto_compile_aspect_attrs,
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
        _prefix = attr.string(
            doc = "String used to disambiguate aspects when generating outputs",
            default = "example_aspect",
        )
    ),
    toolchains = ["@rules_proto_grpc//protobuf:toolchain_type"],
)

# Create compile rule to apply aspect
_rule = rule(
    implementation = proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        deps = attr.label_list(
            mandatory = True,
            providers = [ProtoInfo, ProtoLibraryAspectNodeInfo],
            aspects = [example_aspect],
        ),
    ),
)

# Create macro for converting attrs and passing to compile
def example_compile(**kwargs):
    _rule(
        verbose_string = "{}".format(kwargs.get("verbose", 0)),
        merge_directories = True,
        **{k: v for k, v in kwargs.items() if k != "merge_directories"}
    )

This project is derived from stackb/rules_proto under the Apache 2.0 license and this project therefore maintains the terms of that license. An overview of the changes can be found at MIGRATION.md.

Contributions are very welcome. Please see CONTRIBUTING.md for further details.