Commit Graph

407 Commits

Author SHA1 Message Date
Radosław Kapka
0b261cba5e
Unify log fields (#13654)
* unify fields

* fix tests
2024-02-22 22:40:36 +00:00
terence
5a66807989
Update to V5 (#13622)
* First take at updating everything to v5

* Patch gRPC gateway to use prysm v5

Fix patch

* Update go ssz

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-15 05:46:47 +00:00
Thabokani
692ebd313f
Fix typos in doc (#13583)
Signed-off-by: Thabokani <149070269+Thabokani@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-02-06 10:18:21 +00:00
Preston Van Loon
8839015312
docker: Add coreutils to docker images (#13564)
* Add coreutils to docker images

* add coreutils dependencies

* Add a prysmaticlabs.com/uploads backup of the deb files

* Run gazelle and fix issues

* Remove broken tar, change http_archive deps to debian_archive, remove http mirrors in favor of snapshot

* Add comments about which deps are required by other deps
2024-02-03 19:21:21 +00:00
Nishant Das
5a2453ac9c
Add Debug State Transition Method (#13495)
* add it

* lint
2024-01-22 14:46:20 +00:00
Nishant Das
fc05e306dd
Allow Pcli to Run State Transitions Easily (#13484)
* add all this in

* gaz

* add flag
2024-01-18 14:44:06 +00:00
Preston Van Loon
b585ff77f5
Fix port logging in bootnode (#13457) 2024-01-15 04:38:22 +00:00
Preston Van Loon
3a2b486bde
Bazel 7.0.0 (#13321) 2024-01-10 15:34:11 +00:00
Preston Van Loon
d70f477b1e
Fix docker image version strings in CI (#13356) 2023-12-15 19:15:51 +00:00
Preston Van Loon
09f3df309d
Remove rules_docker, make multiarch images canonical (#13324)
* Remove rules_docker

* Update base image
2023-12-13 23:31:58 +00:00
Preston Van Loon
f537a98fcd
Add staticchecks to bazel builds (#13298)
* Update staticcheck to latest

* Add static checks while ignoring for third party / external stuff

* Added a hack to keep go mod happy.

* disable SA2002

* Pin go mod tidy checker image to golang:1.20-alpine
2023-12-08 05:42:55 +00:00
kasey
737e0e0d3a
Use functional options for --blob-retention-epochs (#13283)
* blob retention period functional opts

* missed unstaged change

* missed other init after cleardb

* fix ineffassign

* fix dup import

* config failsafe for tests

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-06 20:20:34 +00:00
Preston Van Loon
bba8dd6f1e
bazel: Run buildifier, general cleanup (#13193)
* Run buildifier

* Other BUILD.bazel cleanup, rm swagger stuff.

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-16 18:41:37 +00:00
Preston Van Loon
82640b3d88
Enable profile guided optimization for beacon-chain (#13035)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 05:52:36 +00:00
terencechain
9387a36b66
Refactor Exported Names to Follow Golang Best Practices (#13075)
* Fix exported names that start with a package name

* A few more renames

* Fix exported names that start with a package name

* A few more renames

* Radek's feedback

* Fix conflict

* fix keymanager test

* Fix comments

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 16:45:33 +00:00
Preston Van Loon
0bd232667b
Fix image deps (#13022) 2023-10-09 17:21:49 -05:00
Preston Van Loon
a7361cd5ab
Docker: Add "manual" build tag to reduce build times in CI (#12967)
* Add manual tags to oci images. This reduces build times for build and test in CI.

* Ran buildifier on tools dir
2023-09-28 17:49:26 +00:00
Preston Van Loon
02ee6897bb
Multiarch docker containers (#12428)
* Add bazel-zig-cc for a hermetic cc toolchain

* gazelle

* Remove llvm

* remove wl

* Add new URLs for renamed repo

* gazelle

* Update to v2.0.0-rc1

* bump to rc2

* Proof of concept multi-arch containers for beacon-chain

* TODO and gaz

* Refactor to starlark macro. Use a version of bash that actually works

* progress

* gaz

* multirun to use multiple repositories, but doesn't work with tag stamping

* Revert "multirun to use multiple repositories, but doesn't work with tag stamping"

This reverts commit 93afa76f65daeba9426c3d714dabf94410fa3a0c.

* Add targets for all supported docker images and temporarily set the repository to prysm-dev for testing

* use a temporary fix to see if it works on buildkite

* Revert "use a temporary fix to see if it works on buildkite"

This reverts commit ddc79283caa6b39c8aff4c62772728832d0dfe64.

* testing a cURL fix

* try fix with my fix

* Revert "try fix with my fix"

This reverts commit bb7521bf4712abb6779bbf3132e64911b751db65.

* Revert "testing a cURL fix"

This reverts commit 8a4782110f8853cb7278baf507621bd6684572b1.

* try tip of main branch for rules_oci

* update to 1.2.0

* Update rules_oci to v1.3.0

* Update rule_oci to 1.3.4

* Disable experimental_remote_downloader

* Remove extra zig bazelrc

* Move image deps to its own file

* PR feedback

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-28 15:24:16 +00:00
Sammy Rosso
d55757500f
Integrate EIP-4881 Deposit Tree Into Prysm via a Feature Flag (#11942)
* Initial spec rewrite

* Finish adding merkle tree implementation

* Last bits

* Move reverse function

* Add comments

* Add deposit tree snapshot

* Add deposit tree

* Add comments + cleanup

* Fixes

* Add missing errors

* Small fixes

* Add unhandled error

* Cleanup

* Fix unsafe file.Close

* Add missing comments

* Small fixes

* Address some of deepSource' compaints

* Add depositCount check

* Add finalizedDeposit check

* Replace pointer magic with copy()

* Add test for slice reversal

* add back bytes method

* Add package level description

* Remove zerohash gen and add additional checks

* Add additional comments

* Small lint fixes

* Forgot an error

* Small fixes

* Move Uint64ToBytesLittleEndian32 + test

* Fix uint subtraction issue

* Move mixInLength below error handling

* Fix

* Fix deposit root

* integrate 4881

* edits

* added in deposit tree fetcher

* add file

* Add remaining fetcher functions

* Add new file for inserter functions

* Fixes and additional funcs

* Cleanup

* Add

* Graph

* pushed up edits

* fix up

* Updates

* Add EIP4881 toggle flag

* Add interfaces

* Fix tests

* More changes

* Fix

* Remove generated graph

* Fix spacing

* Changes

* Fixes

* Changes

* Test Fix

* gaz

* Fix a couple tests

* Fix last tests

* define protos

* proto methods

* pushed

* regen

* Add proto funcs

* builds

* pushin up

* Fix and cleanup

* Fix spectest

* General cleanup

* add 4881 to e2e

* Remove debug statements + remove test skip

* Implement first set of  missing methods

* Replace Zerohashes + cleanup

* gazelle

* fmt

* Put back defensive check

* Add error logs

* InsertFinalizedDeposits: return an error

* Remove logging

* Radek' Review

* Lint fixes

* build

* Remove cancel

* Update beacon-chain/deterministic-genesis/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/cache/depositsnapshot/deposit_inserter.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Cleanup

* Fix panic when DepositSnapshot is nil on init

* Gofmt

* Fix RootEquivalence test

* Gofmt

* Add missing comments

* Nishant' review

* Add Insert benchmarks

* fix up copy method

* Fix deep copy

* Fix conflicts

* Return error

* Fix linter issues

* add in migration logic

* Cleanup + tests

* fix

* Fix incorrect index in test

* Fix linter

* Gofmt

* fix it

* fixes for off by 1

* gaz

* fix cast

* fix it

* remove ErrZeroIndex

* Fix merkle_tree_test

* add fallback

* add fix for insertion bug

* add many fixes

* fix empty snapshot

* clean up

* use feature

* remove check

* fix failing tests

* skip it

* fix test

* fix test again

* fix for the last time

* Apply suggestions from code review

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix it

* remove cancel

* fix for voting

* addressing more comments

* fix err

* potuz's review

* one more test

* fix bad test

* make 4881 part of dev mode

* add workaround for new trie

* comment

* preston's review

* james's review

* add comment

* james review

* preston's review

* remove skipped test

* gaz

---------

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-07 03:19:32 +00:00
Nishant Das
945c76132c
Add Allocation Data To Benchmark (#12641)
* add more data

* terence's review
2023-07-20 08:49:17 +00:00
terencechain
145a485b75
fix(pcli): use state trie for HTR duration (#12629) 2023-07-15 20:39:58 -07:00
terencechain
7e474b7a30
fix: benchmark deserialize without clone and init trie (#12626)
* fix: benchmark deserialize without clone

* remove tree initialization

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-07-15 12:35:58 +00:00
Potuz
cd8847c53b
Add deserialization time in pcli benchmark (#12620) 2023-07-13 12:19:10 +00:00
Preston Van Loon
a19044051f
Add hermetic_cc_toolchain for a hermetic cc toolchain (#12135)
* Add bazel-zig-cc for a hermetic cc toolchain

* gazelle

* Remove llvm

* remove wl

* Add new URLs for renamed repo

* gazelle

* Update to v2.0.0-rc1

* bump to rc2

* Some PR feedback

* use v2.0.0 from rc2

* Disable hermetic builds for mac and windows.

* bump bazel version, add darwin hack

* fix

* Add the no-op emtpy cc toolchain code

* typo and additional copy

* update protobuf and fix vaticle warning

* Revert "update protobuf and fix vaticle warning"

This reverts commit 7bb4b6b564d2d588567d08dfdd3b9837c26d3c05.

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-06-26 14:31:40 +00:00
Nishant Das
194b3b1c5e
Ensure File Does Not Exist (#12536)
* error out

* gaz

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-06-15 21:41:46 +00:00
james-prysm
d15122fae2
Reopenning fix for keystore field name change to align with EIP2335 (#12530)
* adding changes

* fixing deepsource
2023-06-14 15:48:30 -05:00
Potuz
826267310e
benchmark cold hashing of capella beaconstates (#12516)
* benchmark cold hashing of capella beaconstates

* use since
2023-06-12 16:54:43 +00:00
Nishant Das
33cf52831c
Update Libp2p to v0.27.5 (#12486)
* add deps

* update to v0.27.5 and handle panic
2023-06-06 08:41:15 +08:00
Nishant Das
09e22538f9
Support Capella Blocks for Tool (#12402)
* fix it

* fix it
2023-05-15 17:59:02 +08:00
Raul Jordan
e5c9387cd9
Update Github Actions Go Version (#12391)
* update github actions

* use quotes or it is go 1.2 

lol

* Update gosec

* Update gosec

* Update go lint

* fix gosec violations

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-05-12 15:51:20 +00:00
Nick Sullivan
5c00fcb84f
Fix numerous spelling error and typos in the log messages, comments, and documentation. (#12385)
* Minor typos and spelling fixes (comments, logs, & docs only, no code changes)

* Fix seplling in log message

* Additional spelling tweaks based on review from @prestonvanloon
2023-05-11 20:45:43 +00:00
Justin Traglia
4c916403e9
Enable dupword linter & fix findings (#12271)
* Enable dupword linter & fix findings

* Correct an incorrect fix

* Add nolint comment

* Add another nolint comment

* Revert unintended change to bazel version

---------

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-04-18 22:01:27 +00:00
Justin Traglia
8428a79971
Enable whitespace linter & fix findings (#12273)
* Enable whitespace linter & fix findings

* Fix new finding

* fix new violation

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-04-18 18:58:27 +00:00
terencechain
d17996f8b0
Update to V4 🚀 (#12134)
* Update V3 from V4

* Fix build v3 -> v4

* Update ssz

* Update beacon_chain.pb.go

* Fix formatter import

* Update update-mockgen.sh comment to v4

* Fix conflicts. Pass build and tests

* Fix test
2023-03-17 18:52:56 +00:00
Preston Van Loon
f6eb42b761
Update bazel to 6.1.0 (#12121)
* Update references for cc toolchain after removal of @bazel_tools//cpp/cc_toolchain_config.bzl in 1727361563

* Update to bazel 6.1.0

* Update cross-toolchain configs
2023-03-13 23:17:26 +00:00
Preston Van Loon
39fe29d8f4
Replace @bazel pkg_tar rule with canonical @rules_pkg pkg_tar (#12120)
* Replace @bazel pkg_tar rule with canonical @rules_pkg pkg_tar

* gazelle on workspace
2023-03-13 20:17:12 +00:00
terencechain
00b9e484e5
Cleanups from eip4844 branch (#12109)
* Minor cleanups from eip4844 branch

* Add back handler

* Fix gazelle

* More fixes

* More fixes

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-03-10 16:46:27 +00:00
Preston Van Loon
4356cbc352
Update cross compile toolchains (#12069)
* Regenerate cross-toolchain configs

* Remove some extra whitespaces

* Run gazelle and add that note to the README

* Format numbered lists better in markdown

* gcloud docker command is deprecated, just use docker

* Add comment about docker credentials for gcr.io

* Update dockerfile, some remote executor config work

* gazelle

* Remove commented lines

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-03-07 20:09:46 +00:00
Preston Van Loon
91fee5db17
Update distroless base images (#12061)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-02-28 22:50:37 +00:00
terencechain
d4f3257f60
Rename block interfaces (#11975) 2023-02-09 10:23:32 +01:00
Radosław Kapka
77d3ccb9ad
Clean up state types (#11916)
* Clean up state types

* rename package
2023-01-26 14:40:12 +00:00
terencechain
7fa3ebfaa8
Add historical summaries to state and processing (#11842)
* Add historical summaries to state and processing

* Process historical roots test

* Passing spec tests

* Fix shas and tests

* Fix mainnet spectest sha

* Fix tests

* Update beacon-chain/core/epoch/epoch_processing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/core/epoch/epoch_processing_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update proto/prysm/v1alpha1/beacon_state.proto

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/state/state-native/hasher.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Radek's feedback

* Getters error

* Dont return

* Fix else

* Fix tests

* Fix test

* Rm white space

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-01-11 22:20:15 +00:00
Preston Van Loon
4b033f4cc7
Update go to 1.19.3 (#11630)
* Update go to 1.19.3

* Update other items to 1.19

* Update golangci-lint to latest release

* Run gofmt -s with go1.19

* Huge gofmt changes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-11-18 19:12:19 +00:00
Preston Van Loon
55690de685
bazel: Check in 5.3.0 cross compile clang toolchain (#11577)
* Check in 5.3.0 generated clang cross-toolchain

* Remove tools/cross-toolchain/configs/clang/bazel_5.0.0

* Add manual tags and run gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-27 02:48:23 +00:00
Radosław Kapka
b7a878d011
Resolve remaining native state tasks (#11561)
* remove ToProto and ToProtoUnsafe wrappers

* TestAppendBeyondIndicesLimit

* change type of genesisValidatorsRoot

* fuzz tests

* check type assertion
2022-10-19 10:37:45 -04:00
Preston Van Loon
6bea17cb54
Update libp2p to support go 1.19 (#11309)
* Update libp2p to support go 1.19

* gaz

* go mod tidy

* Only update the minimum deps

* go mod tidy

* revert .bazelrc

* Update go-libp2p to v0.22.0 and update import paths (#11440)

* Fix import paths

* Fix go-libp2p-peerstore import

* Bazel updates

* fix

* revert some comments changes

* revert some comment stuff

* fix dependency issues

* vendor problematic library

* use your brain

* remove

* tests

Co-authored-by: Marco Munizaga <marco@marcopolo.io>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-10-07 15:24:51 +08:00
Radosław Kapka
73443208a1
Remove proto state (#11445)
* Remove native state flag and use native state in spectests

* remove feature from tests

* use e2e config in slasher simulator

* use params.BeaconConfig in testutil

* use correct function

* use minimal config in go_test

* fix TestListValidators

* parameterize sync committee bits and aggregation bits

* Fix TestServer_ListIndexedAttestations_GenesisEpoch

(cherry picked from commit 254ab623dde08ae8886b152facdbbd8889ed79db)

* fix more tests

* fix even more

* moreeee

* aaaand more

* one more fix

* one more

* simplify TestGetAltairDuties_UnknownPubkey

* comment out problematic test

* one more fix

* one more

* aaaand one more

* another

* use fieldparams in HydrateBlindedBeaconBlockBodyBellatrix

* create new package for mainnet tests

* TestServer_GetBellatrixBeaconBlock

* change slashed validator index

* clear cache in reward_test.go

* deprecate flag

* create bazel mainnet target

* move attester mainnet test to mainnet target

* "fix" proposer tests

* use minimal config in TestServer_circuitBreakBuilder

* fix TestProposer_ProposeBlock_OK

* more fixes in validator package

* more fixes

* more fixes

* test code

* move TestProposer_GetBeaconBlock_BellatrixEpoch to minimal

* finally

* remove proposer_bellatrix_mainnet_test.go

* fix TestServer_GetBellatrixBeaconBlock_HappyCase

* fix TestServer_GetBellatrixBeaconBlock_BuilderCase

* Preston needs to fix this!

* Revert "Preston needs to fix this!"

This reverts commit b03d97a16e3080e254c7b19d7f193d3c600ca869.

* remove proto state tests

* fix migration tests

* static analysis fix

* review

* remove proto state

* swap state in tests

* fix BUILD file in /proto/testing

* remove metrics test with nil state
2022-09-16 18:17:46 -04:00
Radosław Kapka
4d90afe944
Fixed size arrays in block interfaces and structs (#11375)
* Fixed size arrays in block fields

* test fix

* fmt

* fix fetcher test

* fix fuzz tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 14:30:16 +00:00
Håvard Anda Estensen
9dee22f7ab
Pre-allocate slices (#11317) 2022-08-26 13:49:50 +02:00
Raul Jordan
20ed47a107
Add a Generate Genesis State Command to Prysmctl (#11259)
* genesis tool

* done with tool

* delete old tool

* no fatal

* fix up

* Update cmd/prysmctl/testnet/generate_genesis.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek feedback

* more feedback

* required

* fix up

* gaz

* radek feedback

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-08-19 09:26:25 +00:00