Why bundle prebuild protoc? I'd rather use the "system" one.

[oakad]

Many projects rely on protobuf and repeated protoc recompilation is an unfortunate nuisance, no discussion here.

However:

1. Most builds these days are containerized, or, at least, happen in controlled environments. All serious projects have a bunch of written rules, on what tools have to be installed on the developer's machine, where, what version, etc, etc.
2. Simply having some random version of protoc bundled with Bazel is not solving any practical problems. I, for one, have custom protoc plugins building as part of my app projects, I don't want bazel to chose the protoc version for me. On a side note, I don't think bazelisk/.bazelversion is such a good idea either; dependency hell is not fun as it is, even without having to think about which exact version the build tool is.
3. A convenient repository rule to point bazel at "system" protoc will solve it, because I know what version of protoc I want to use and can ensure that build environment has it. Something nice like protosdk.host(), not the general toolchain mess which may or may not work and otherwise looks verbose and confusing.

[sluongng]

The problem with system one is that each host can have a different protoc version.
Since the hash of the protoc binary is part of the cache key calculation, that means you will have different cache keys on different machines, thus reducing the hermeticity/reproducibility of the build.

Most builds these days are containerized, or, at least, happen in controlled environments

This is not a safe assumption to make. Many people prefer to run builds from their laptops.

The download of the protoc binary here is the "controlled environments" in such a case: instead of getting the protoc binary through a big tarball of a container image layer, you get only the files you need through fine-grained downloads and checksum verification.

---

Worth noting that while using a prebuilt protoc or building protoc from source is the recommended approach, they are not the only approach. You can write your own protoc toolchain, which points to your system protoc if needed.

[oakad]

It should not be an "assumption", it should be a choice, offered to user. Most important Bazel rules_* offer that choice. rules_proto can too.

[sluongng]

1. rules_proto is archived. https://github.com/bazelbuild/rules_proto Or are you talking about something else?

2. It is a choice. As I mentioned, the proto compiler is defined as a toolchain, and you can register your own system toolchain based on what is provided here https://github.com/protocolbuffers/protobuf/blob/main/bazel/toolchains/proto_toolchain.bzl.

Nobody knows how your system is configured, so it makes sense for you to bring your own based on your system configuration. What is missing here in your opinion?

[oakad]

1. As of right now my projects still use rules_proto and I have no idea what to use instead. Was there any sort of publication regarding the replacement? I am reasonably interested in Bazel projects and I had seen none.
2. Toolchains are different enough between bazel rulesets. There should be an example somewhere.

For example, rules_go ask as to do this:

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk", dev_dependency
= True)
go_sdk.host()

rules_python asks us to do this (quite different):

python_local_runtime_repo = use_repo_rule(
    "@rules_python//python/local_toolchains:repos.bzl",
    "local_runtime_repo",
)

python_local_runtime_toolchains_repo = use_repo_rule(
    "@rules_python//python/local_toolchains:repos.bzl",
    "local_runtime_toolchains_repo",
)
    
python_local_runtime_repo(
    name = "python3_local",
    interpreter_path = "python3",
    on_failure = "fail",
    dev_dependency = True
)
    
python_local_runtime_toolchains_repo(
    name = "python3_toolchains",
    runtimes = ["python3_local"],
    dev_dependency = True
)

register_toolchains("@python3_toolchains//:all", dev_dependency = True)

How do we go about protoc?

[oakad]

In fact, even the main bazel docs still suggest using rules_proto. And some important rulesets, like rules_go, for example, still import it and not the protobuf repo one. Deprecation of widely used rulesets should be handled differently, I'd say. :-)

https://bazel.build/versions/9.0.0/reference/be/protocol-buffer#proto_lang_toolchain

[thii]

Not sure if it still works today but there is a --proto_compiler flag that you can point to your preferred protoc (has to be a target label).

[oakad]

The flag is definitely still there, but it has to point at a label, not some executable. So, there needs to be a protoc binary within the source code (not a fan of committing binaries into rcs) or a special rule of some sort, providing that label, and a specialized .bazelrc on top. Hardly a sound solution.

[thii]

Hmm, I just checked but the flag doesn't seem to work anymore, as of Bazel 9. Looks like it requires a toolchain now. This works for me:

# MODULE.bazel
new_local_repository = use_repo_rule("@bazel_tools//tools/build_defs/repo:local.bzl", "new_local_repository")
                                                                                                                      
new_local_repository(
    name = "protoc",
    path = "/usr/bin",
    build_file_content = """exports_files(["protoc"])""",
)
register_toolchains("//:protoc_toolchain")

# BUILD
load("@com_google_protobuf//bazel/toolchains:proto_toolchain.bzl", "proto_toolchain")

proto_toolchain(
    name = "protoc_toolchain",
    proto_compiler = "@protoc",
)

[oakad]

Thank you for the suggestion. I was lamenting a lack of good toolchain example in the other subthread. :-)

[shameelvk9-png]

#28951