prysm-pulse/tools/cross-toolchain
Preston Van Loon 93c11e0e53
Update rules_go (#7202)
* Update rules_go

* go 1.15

* try with v0.24.2

* Update Mac OS X SDK

* gaz

* update SDK in toolchain config

* -I flag

* another -I flag

* Update rules_go, gazelle, bazel version

* regen, update rules_docker

* Revert "another -I flag"

This reverts commit 9255133d99bee1e94560a64316a4a7539a363935.

* Revert "-I flag"

This reverts commit 2954a41d76adc81ccb6281af243f4be1f79152e9.

* giving up

* Use OS X 10.12

* Use OS X 10.12

* Revert "Use OS X 10.12"

This reverts commit 4f60d5cb807b2ccccd484a85a6a7477a5fd13e28.

* Revert "Use OS X 10.12"

This reverts commit a79177fab7182ad792d1cee7fb61214ef8bd73ab.

* osx toolchain tweaks necessary to work with 10.15 mac sdk

* Update docker image, regen

* gaz

* test using custom image

* Revert "test using custom image"

This reverts commit 95b8666810a254f16061c99e2ea0ae4d020b2f2d.

* explicit go version

* Clean up docker image rules with new definitions. gazelle

* please the linter

* Update protobuf compiler to 3.13.0, run gazelle

* Update gazelle to fix empty build files. https://github.com/bazelbuild/bazel-gazelle/pull/926

* update skylib

* fix herumi fuzz build

* remove comment from tools/cross-toolchain/regenerate.sh

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: SuburbanDad <gts.mobile@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 03:01:56 +00:00
..
configs Update rules_go (#7202) 2020-11-10 03:01:56 +00:00
BUILD.bazel First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
cc_toolchain_config_linux_arm64.bzl.tpl First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
cc_toolchain_config_osx.bzl.tpl Update rules_go (#7202) 2020-11-10 03:01:56 +00:00
cc_toolchain_config_windows.bzl.tpl First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
cc_toolchain.BUILD.bazel.tpl First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
Dockerfile Update rules_go (#7202) 2020-11-10 03:01:56 +00:00
empty.bzl First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
install_clang_cross10.sh First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
install_osxcross.sh Update rules_go (#7202) 2020-11-10 03:01:56 +00:00
prysm_toolchains.bzl First cut at multi-arch cross compiling toolchain (#4945) 2020-03-05 16:59:56 +00:00
rbe_toolchains_config.bzl Update rules_go (#7202) 2020-11-10 03:01:56 +00:00
README.md Amend Cross Compile README (#7615) 2020-10-22 21:41:46 +00:00
regenerate.sh Update rules_go (#7202) 2020-11-10 03:01:56 +00:00

Multiarch Cross Compiling Toolchain

Toolchain suite

This package declares a c++ toolchain suite with cross compilers for targeting four platforms:

  • linux_amd64
  • linux_arm64
  • osx_amd64
  • windows_amd64

This toolchain suite describes cross compile configuration with a Dockerfile with the appropriate host dependencies. These toolchains can be used locally (see caveats), Remote Build Execution (RBE), and in a docker sandbox (like RBE, but local).

Cross compile target support

target linux_amd64 linux_arm64 osx_amd64 windows_amd64
//beacon-chain ✔️ docker-sandbox and RBE, libkafka supported locally only ✔️ docker-sandbox and RBE, no libkafka support ✔️ docker-sandbox, no libkafka support ✔️ docker-sandbox, no libkafka support
//validator ✔️ docker-sandbox and RBE ✔️ docker-sandbox and RBE ✔️ docker-sandbox ✔️

The configurations above are enforced via pull request presubmit checks.

Bazel config flag values

Use these values with --config=<flag>, multiple times if more than one value is defined in the table. Example: bazel build //beacon-chain --config=windows_amd64_docker to build windows binary in a docker sandbox.

Config linux_amd64 linux_arm64 osx_amd64 windows_amd64
Local run linux_amd64 linux_arm64 osx_amd64 windows_amd64
Docker sandbox linux_amd64_docker linux_arm64_docker osx_amd64_docker windows_amd64_docker
RBE (See Caveats) linux_amd64 and remote linux_arm64 and remote osx_amd64 and remote windows_amd64 and remote

Caveats

There are a few caveats to each of these strategies.

  • Local runs require clang compiler and the appropriate cross compilers installed. These runs should only be considered for a power user or user with specific build requirements. See the Dockerfile setup scripts to understand what dependencies must be installed and where.
  • Docker sandbox is slow. Like really slow! The purpose of the docker sandbox is to test RBE builds without deploying a full RBE system. Each build action is executed in its own container. Given the large number of small targets in this project, the overhead of creating docker containers makes this strategy the slowest of all, but requires zero additional setup.
  • Remote Build Execution is by far the fastest, if you have a RBE backend available. This is another advanced use case which will require two config flags above as well as additional flags to specify the --remote_executor. Some of these flags are present in the project .bazelrc with example values, but commented out.
  • Building with libkafka (--define kafka_enabled=true) is not supported with docker-sandbox or RBE at this time. Likely due to missing cmake dependencies in the builder image or lack of configuration via toolchains.