49a0d3caf0
* Fix a few deps to work with go.mod, check in generated files * Update Gossipsub to 1.1 (#5998) * update libs * add new validators * add new deps * new set of deps * tls * further fix gossip update * get everything to build * clean up * gaz * fix build * fix all tests * add deps to images * imports Co-authored-by: rauljordan <raul@prysmaticlabs.com> * Beacon chain builds with go build * fix bazel * fix dep * lint * Add github action for testing go * on PR for any branch * fix libp2p test failure * Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test * Revert "Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test" This reverts commit 43676894ab01f03fe90a9b8ee3ecfbc2ec1ec4e4. * Compute and set proposer index instead of hard code * Add back go mod/sum, fix deps * go build ./... * Temporarily skip two tests * Fix kafka confluent patch * Fix kafka confluent patch * fix kafka build * fix kafka * Add info in DEPENDENCIES. Added a stub link for Why Bazel? until https://github.com/prysmaticlabs/documentation/issues/138 * Update fuzz ssz files as well * Update fuzz ssz files as well * getting closer * rollback rules_go and gazelle * fix gogo protobuf * install librdkafka-dev as part of github actions * Update kafka to a recent version where librkafkfa is not required for go modules * clarify comment * fix kafka build * disable go tests * comment * Fix geth dependencies for end to end * rename word * lint * fix docker Co-authored-by: Nishant Das <nishdas93@gmail.com> Co-authored-by: rauljordan <raul@prysmaticlabs.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com>
3.1 KiB
Dependency Managagement in Prysm
Prysm is go project with many complicated dependencies, including some c++ based libraries. There are two parts to Prysm's dependency management. Go modules and bazel managed dependencies. Be sure to read Why Bazel? to fully understand the reasoning behind an additional layer of build tooling via Bazel rather than a pure "go build" project.
Go Module support
The Prysm project officially supports go modules with some caveats.
Caveat 1: Some c++ libraries are precompiled archives
Given some of Prysm's c++ dependencies have very complicated project structures which make building difficult or impossible with "go build" alone. Additionally, building c++ dependencies with certain compilers, like clang / LLVM, offer a significant performance improvement. To get around this issue, c++ dependencies have been precompiled as linkable archives. While there isn't necessarily anything bad about precompiled archives, these files are a "blackbox" which a 3rd party author could have compiled anything for this archive and detecting undesired behavior would be nearly impossible. If your risk tolerance is low, always compile everything from source yourself, including complicated c++ dependencies.
Recommendation: Use go build only for local development and use bazel build for production.
Caveat 2: Generated gRPC protobuf libraries
One key advantage of Bazel over vanilla go build is that Bazel automatically (re)builds generated
pb.go files at build time when file changes are present in any protobuf definition file or after
any updates to the protobuf compiler or other relevant dependencies. Vanilla go users should run
the following scripts often to ensure their generated files are up to date. Further more, Prysm
generates SSZ marshal related code based on defined data structures. These generated files must
also be updated and checked in as frequently.
./scripts/update-go-pbs.sh
./scripts/update-go-ssz.sh
Recommendation: Use go build only for local development and use bazel build for production.
Caveat 3: Compile-time optimizations
When Prysmatic Labs builds production binaries, they use the "release" configuration of bazel to compile with several compiler optimizations and recommended production build configurations. Additionally, the release build properly stamps the built binaries to include helpful metadata about how and when the binary was built.
Recommendation: Use go build only for local development and use bazel build for production.
bazel build //beacon-chain --config=release
Adding / updating dependencies
- Add your dependency as you would with go modules. I.e.
go get ... - Run
gazelle update-repos -from_file=go.modto update the bazel managed dependencies.
Example:
go get github.com/prysmaticlabs/example@v1.2.3
bazel run //:gazelle -- update-repos -from_file=go.mod
The deps.bzl file should have been updated with the dependency and any transitive dependencies.
Do NOT add new go_repository to the WORKSPACE file. All dependencies should live in deps.bzl.