Commit Graph

8492 Commits

Author SHA1 Message Date
Potuz
96df81d5c5
Hook to slot stream instead of block stream on the VC (#13327)
* Hook to slot stream instead of block stream on the VC

* Implement StreamSlots in the BN

* mock update

* fix tests

* don't return from stream

* Terence's review

* deepsource second complain

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2023-12-13 23:13:56 +00:00
terence
c47c52152b
Enhance Pruning Logs (#13331)
* Log prunning info

* Added start log

* Log prunning info

* Added start log
2023-12-13 19:24:47 +00:00
james-prysm
4cbe144a6c
CLI: fixing account import ux bugs (#13328)
* fixing account import checking wallet twice, and adding sub folder search with a depth of 2

* removing uneeded check

* fixing unit test

* adding reset cache to fix potential flake

* improving test based on feedback
2023-12-13 17:11:32 +00:00
Justin Traglia
52b9b65adb
Add sanity checks for bundle from builder (#13319)
* Add sanity checks for bundle from builder

* Add more checks to BlobsBundle.ToProto()

* Fix minor typo

* Fix tests & add new ones

* Add tests for ToProto

* Add "not" to error message

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-13 15:54:00 +00:00
Radosław Kapka
ea59b1ec71
Increase buffer of events channel (#13329) 2023-12-13 15:37:45 +00:00
Radosław Kapka
175c484c44
Uncomment e2e flakiness (#13326) 2023-12-13 12:50:13 +00:00
Nishant Das
8aaab86987
fix it (#13325) 2023-12-13 11:01:01 +00:00
Preston Van Loon
381116a3e8
Fix missing testnet versions. Issue #13288 (#13323) 2023-12-12 21:44:14 +00:00
Sammy Rosso
3d61fd0436
Blob filesystem add pruning during blob write (#13275)
* Add prune during write

* Fix merge errors

* Add test

* Add test timeout

* Gaz

* Check prune at midpoint

* Fix slot number

* More checks
2023-12-12 21:27:15 +00:00
james-prysm
b19d24c581
Remove signed block requirement from no-verify functions (#13314)
* removing fake wrappers

* fixing conficts and missed tests

* fixing more conflicts

* addressing missed unit test

* fixing nogo error

* fixing more unit tests

* fixing more tests
2023-12-12 20:18:40 +00:00
Radosław Kapka
8387088a52
Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-12 19:46:19 +00:00
Potuz
ce7452c97a
update spectests to 1.4.0-beta.5 (#13318)
* update spectests to 1.4.0-beta.5

* add spec config
2023-12-12 18:27:48 +00:00
james-prysm
5e56b5fdd7
Beacon APIs: re enabling blob events (#13315)
* re enabling blob events

* terence's comments

* Update beacon-chain/rpc/eth/events/events_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-12 15:58:11 +00:00
Nishant Das
bfaba378f6
activate deneb (#13311) 2023-12-12 08:04:55 +00:00
Sammy Rosso
3bd116db16
Blob filesystem add pruning at startup (#13253)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Add pruning at startup

* Add nil check

* Fix merge errors

* Fix test

* Fix test

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-11 17:08:52 +00:00
terence
7d2ddaee43
Test improvement TestValidateVoluntaryExit_ValidExit (#13313) 2023-12-11 08:01:56 +00:00
terence
122a7782ff
Initialize blob storage for initial sync service (#13312) 2023-12-11 07:52:07 +00:00
terence
9b1b6f9be6
Use verified blob for gossip checks (#13294)
* Use blob verifier for gossip rules

* Fixing tests

* Fix lint

* Mocks

* Trying Kasey's rec

* mock verifier init workaround

* Add more tests

* Reset deneb epoch for exit test

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-11 00:37:45 +00:00
Delweng
0eb08a4f96
beacon-chain/rpc: use BalanceAtIndex instead of Balances to reduce memory copy (#13279)
* beacon-chain/rpc: use BalanceAtIndex instead of Balances

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: stream use BalanceAtIndex is sufficient

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: fix commit review

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/rpc: http2 -> httputil

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-10 00:04:45 +00:00
Radosław Kapka
bdfa06ed65
Simplify post-evaluation in Beacon API evaluator (#13309)
* Simplify post-evaluation in Beacon API evaluator

* revert slottime

* remove unused error
2023-12-09 07:54:31 +00:00
Manu NALEPA
a94f2b93e3
filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
* `filterAndCacheActiveKeys`: Add test cases

- Validator is in unknown status (to be fitered out)
- Validator is in pending status,
   with activation period > current period (to be filtered out)
- Validator is in pending status,
   with activation period == current period (to be kept)

* `filterAndCacheActiveKeys`: Keep exiting keys

Initially:
-------
If a validator is in exiting (so, with status==EXITING != ACTIVE) state,
it will be filtered out by the `filterAndCacheActiveKeys` function.
The validator won't be registered to the beacon node.

If this exiting validator has to propose a block:
- the block will be proposed using local block building only.
- the fee recipient will be the one set in the beacon node.

(Additionally, if the beacon node Prysm without any
fee recipient defined at the beacon node level, the fee recipient
will default on the `0x00000...` burn address.)

This commit modifies the `filterAndCacheActiveKeys` function
by stopping filtering out exiting validators.
2023-12-09 07:53:08 +00:00
Radosław Kapka
4c47756aed
HTTP endpoints cleanup (#13251)
* remove validation package

* structs cleanup

* merge with apimiddleware removal

* more validation and Bls capitalization

* builder test fix

* use strconv for uint->str conversions

* use DecodeHexWithLength

* use exact param names

* rename http package to httputil

* change conversions to fmt.Sprintf

* handle query paramsd and route variables

* spans and receiver name

* split structs, move bytes helper

* missing ok check

* fix reference to indexed failure

* errors fixup

* add godoc to helper

* fix BLS casing and chainhead ref

* review

* fix import in tests

* gzl
2023-12-08 20:37:20 +00:00
Nishant Das
440841d565
only run it in the middle of an epoch (#13303) 2023-12-08 15:14:01 +00:00
Preston Van Loon
ff99616833
Fix staticcheck violations (#13301)
* Fix violations of sa2002

* Fix violations of sa4005

* Fix violations of sa4010

* Fix violations for sa4023

* Comment on commented static checks
2023-12-08 13:07:52 +00:00
Preston Van Loon
f537a98fcd
Add staticchecks to bazel builds (#13298)
* Update staticcheck to latest

* Add static checks while ignoring for third party / external stuff

* Added a hack to keep go mod happy.

* disable SA2002

* Pin go mod tidy checker image to golang:1.20-alpine
2023-12-08 05:42:55 +00:00
Radosław Kapka
cee38660c7
Gracefully handle unknown validator index in the REST VC (#13296)
* Gracefully handle unknown validaor index in the REST VC

* add apostrophes
2023-12-08 04:30:50 +00:00
james-prysm
481d77bfde
APIs: reusing grpc cors middleware for rest (#13284)
* reusing grpc cors middleware for rest

* addressing radek's comments

* Update api/server/middleware.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing to recommended name

* fixing naming

* fixing rename on test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-08 04:24:18 +00:00
Nishant Das
590317553c
Support New Subnet Backbone (#13179)
* add in changes

* fix it up

* fix test

* gaz

* lint

* add back

* fix tests

* fix it

* fix tests

* add lib

* fix it
2023-12-08 04:07:48 +00:00
terence
68b7d1009e
Update README.md (#13302) 2023-12-08 04:07:10 +00:00
james-prysm
b5b8825cc8
Beacon API: fix get blob returns 500 instead of empty (#13297)
* fix blob api, should return empty if no indicies were found

* fixing small bug with slice
2023-12-07 22:33:26 +00:00
Justin Traglia
382b8b23c2
Ensure partial blob is deleted if there's an error (#13292)
* Ensure partial blob is deleted if there's an error

* Add debug log if file is removed
2023-12-07 20:52:16 +00:00
kasey
40a3ebab91
initialize sig cache for verification.Initializer (#13295)
* initialize sig cache for verification.Initializer

* gaz

* lint

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 20:14:01 +00:00
james-prysm
83af9a5694
Beacon API: update Deneb endpoints after removing blob signing (#13235)
* making needed changes to beacon API based on removal of blobsidecar from block contents

* fixing tests and reverting some changes to be addressed later

* fixing generated code from protos

* gaz

* fixing get blob handler and adding blob storage to the blob service

* updating unit tests

* WIP

* wip tests

* got tests passing but needs cleanup

* removing gomod and gosum changes

* fixing more tests

* fixing more tests

* fixing more tests

* gaz

* moving some proto types around

* removing unneeded unit test

* fixing proposer paths

* adding more tests

* fixing more tests

* improving more unit tests

* updating one blob only unit test

* changing arguments of buildBlobSidecar

* reverting a change based on feedback

* terence's review items

* fixing test based on new develop changes

* radek's comments

* addressed more comments from radek

* adding in blobs to test data

* fixing casing in test

* removing extra line

* fixing issue from bad merge

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* moving core getblob business logic to blocker based on radek's comment

* fixing mock blocker

* gaz

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-07 17:37:11 +00:00
Nishant Das
6a45323ab7
only run metrics for canonical blocks (#13289) 2023-12-07 11:03:23 +00:00
kasey
4008ea736f
Verify roblobs (#13245)
* scaffolding for verification package

* WIP blob verification methods

* lock wrapper for safer forkchoice sharing

* more solid cache and verification designs; adding tests

* more test coverage, adding missing cache files

* clearer func name

* remove forkchoice borrower (it's in another PR)

* revert temporary interface experiment

* lint

* nishant feedback

* add comments with spec text to all verifications

* some comments on public methods

* invert confusing verification name

* deep source

* remove cache from ProposerCache + gaz

* more consistently early return on error paths

* messed up the test with the wrong config value

* terence naming feedback

* tests on BeginsAt

* lint

* deep source...

* name errors after failure, not expectation

* deep sooource

* check len()==0 instead of nil so empty lists work

* update test for EIP-7044

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 02:36:25 +00:00
Justin Traglia
4e4fb9ad52
Split blob pruning into two funcs (#13285) 2023-12-06 23:39:02 +00:00
kasey
737e0e0d3a
Use functional options for --blob-retention-epochs (#13283)
* blob retention period functional opts

* missed unstaged change

* missed other init after cleardb

* fix ineffassign

* fix dup import

* config failsafe for tests

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-06 20:20:34 +00:00
Potuz
604c82626f
Allow requests for old target roots (#13281) 2023-12-06 17:11:34 +00:00
Nishant Das
e1a3852f08
push up the defaults (#13278) 2023-12-06 16:46:46 +00:00
Preston Van Loon
a40cc40edf
CI: Add merge queue events trigger for github workflows (#13282) 2023-12-06 16:13:13 +00:00
james-prysm
0e3c1d42f6
Beacon API: routes unit test (#13276)
* refactoring to add a routes unit test for rest handlers

* gaz

* updating names for functions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 15:17:13 +00:00
Radosław Kapka
f41e603e5a
Simplify Beacon API evaluator (#13265)
* Simplify Beacon API evaluator

* remove comment

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 14:54:33 +00:00
Radosław Kapka
28c3330375
Don't fetch duties for unknown keys (#13269)
* Don't fetch duties for unknown keys

* test
2023-12-06 14:18:45 +00:00
Nishant Das
cb465086e8
Fix Optimistic Sync Evaluator (#13262)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-06 13:41:47 +00:00
Nishant Das
cdd7958739
Fix Domain Data Caching (#13263)
* fix domain data caching

* fix locking of domain data

* preston's review
2023-12-06 12:04:19 +00:00
Nishant Das
97a522827b
Bump Up Gossip Queue Size (#13277) 2023-12-06 09:49:26 +00:00
Preston Van Loon
b84b795f23
Relax file permissions check on existing directories (#13274) 2023-12-05 19:39:45 -06:00
Sammy Rosso
f40b8583f7
Blob filesystem: prune blobs (#13147)
* Add Save blob and tests

* Remove locks

* Remove test cleanup

* Fix go mod

* Cleanup

* Add checksum

* Add file hashing to fileutil

* Move test

* Check data when exists

* Add one more test

* Rename

* Gaz

* Add packaged level comment

* Fix block proposals in the REST validator client (#13116)

* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* fix head slot in log (#13139)

* zig: Update zig to recent main branch commit (#13142)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix Pending Queue Deadline Bug (#13145)

* rearrange deadline

* naming

* Add pruning

* Gaz

* Gaz

* Update pruning

* Cleanup

* Making a mess

* Benchmarking

* Forgot to add the file + fixes

* Fixes

* Pruning from DB fixed

* Add prune by file data

* Fix pruning

* Prune fixes

* Cleanup db blockRoot filter

* Handle file close error

* Fix deletion

* Change read at + remove retentionEpich from bs

* Gaz

* Seperate logic + add detailed comments

* Add tests

* Add retention slot when creating blobStorage

* Fix tests

* Gaz

* Fix testonly import

* Fix linter errors

* Fix retentionSlot calculation

* Move + use MaxEpochsToPersistBlobs

* Remove unused ctx

* Prestons suggestion

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Rename

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-05 21:07:34 +00:00
Preston Van Loon
a6b6a938de
blobstorage: Improve mkdirall error (#13271) 2023-12-05 19:57:08 +00:00
Brandon Liu
c78d698d89
Add --jwt-id flag (#13218)
* add jwt-id flag

* optimize unit test for jwt-id

* Add jwt-id to help text

* gofmt

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-05 19:02:25 +00:00