terence
c3dbfa66d0
Change blob latency metrics to ms ( #13481 )
2024-01-17 23:28:42 +00:00
terence
6ddafe1159
Delete invalid blob at block processing ( #13456 )
...
* Delete invalid blob at block processing
* Fix test
2024-01-12 08:09:45 +00:00
kasey
5cea6bebb8
minimize syscalls in pruning routine ( #13425 )
...
* minimize syscalls in pruning routine
* terence feedback
* Update beacon-chain/db/filesystem/pruner.go
Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
* pr feedback
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-01-08 22:31:16 +00:00
terence
ff06e08274
Prune dangling blob ( #13424 )
...
* Prune dangling blob
* Fix test
* Kasey's feedback
* Preston's feedback
* Use warning, fix test
2024-01-05 22:29:57 +00:00
terence
3c1c0b3c00
Update blob pruning log ( #13417 )
2024-01-04 18:02:19 +00:00
terence
9709412511
Use Afero Walk for Pruning Blob ( #13410 )
...
* Use Afero walk
* Return err
* Use wrap
* More err to the end
* Fix loop
2024-01-04 16:41:00 +00:00
terence
bd4a520013
Initialize blob storage without pruning ( #13412 )
2024-01-04 05:56:38 +00:00
Sammy Rosso
a0ff1351a0
Fix batch pruning errors ( #13355 )
...
* Add compareAndSwap
* Update lastPrunedEpoch before prune
* Fix and test
* Remove debug log
* Kasey' review
* Fix tests
* Address Kasey's comments
* Fix prune before slot
* Rename
* Fix bad test
---------
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2024-01-03 20:52:07 +00:00
Sammy Rosso
c6801df05a
Fix total pruned metric + add to logging ( #13367 )
2023-12-19 16:15:01 +00:00
Preston Van Loon
844b2c6602
Add error wrapping to blob initialization errors ( #13366 )
2023-12-19 14:55:26 +00:00
Sammy Rosso
374d77f437
Blob filesystem metrics ( #13316 )
...
* Add metrics
* Replace counter with gauge
* Preston's comments
* Remove hardcoded number
* Count blob files
* Fix count order
* Fixes
* Cleanup
* Add blob bucket
* Update beacon-chain/node/node.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Rename
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-18 18:24:07 +00:00
Nishant Das
b45a6664be
Enable Deneb For E2E Scenario Tests ( #13317 )
...
* fix all cases
* update web3signer
* current progress
* fix it finally
* push it back to capella
* remove hard-coded forks
* fix failing tests
* gaz
* fix dumb bug
* fix bad test setup
* change back
2023-12-16 11:37:44 +00:00
Justin Traglia
1962cca69e
Fix error string generation for missing commitments ( #13338 )
2023-12-15 04:03:45 +00:00
terence
53bc96844e
Move pruning log to after retention check ( #13348 )
2023-12-15 00:49:29 +00:00
terence
c47c52152b
Enhance Pruning Logs ( #13331 )
...
* Log prunning info
* Added start log
* Log prunning info
* Added start log
2023-12-13 19:24:47 +00:00
Sammy Rosso
3d61fd0436
Blob filesystem add pruning during blob write ( #13275 )
...
* Add prune during write
* Fix merge errors
* Add test
* Add test timeout
* Gaz
* Check prune at midpoint
* Fix slot number
* More checks
2023-12-12 21:27:15 +00:00
Justin Traglia
382b8b23c2
Ensure partial blob is deleted if there's an error ( #13292 )
...
* Ensure partial blob is deleted if there's an error
* Add debug log if file is removed
2023-12-07 20:52:16 +00:00
Justin Traglia
4e4fb9ad52
Split blob pruning into two funcs ( #13285 )
2023-12-06 23:39:02 +00:00
kasey
737e0e0d3a
Use functional options for --blob-retention-epochs ( #13283 )
...
* blob retention period functional opts
* missed unstaged change
* missed other init after cleardb
* fix ineffassign
* fix dup import
* config failsafe for tests
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-06 20:20:34 +00:00
Preston Van Loon
b84b795f23
Relax file permissions check on existing directories ( #13274 )
2023-12-05 19:39:45 -06:00
Sammy Rosso
f40b8583f7
Blob filesystem: prune blobs ( #13147 )
...
* Add Save blob and tests
* Remove locks
* Remove test cleanup
* Fix go mod
* Cleanup
* Add checksum
* Add file hashing to fileutil
* Move test
* Check data when exists
* Add one more test
* Rename
* Gaz
* Add packaged level comment
* Fix block proposals in the REST validator client (#13116 )
* Fix block proposals in the REST validator client
* fix graffiti test
* return empty graffiti
* fallback to old endpoints
* logs
* handle 404
* everything passes
* review from James
* log undecoded value
* test fixes and additions
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* fix head slot in log (#13139 )
* zig: Update zig to recent main branch commit (#13142 )
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix Pending Queue Deadline Bug (#13145 )
* rearrange deadline
* naming
* Add pruning
* Gaz
* Gaz
* Update pruning
* Cleanup
* Making a mess
* Benchmarking
* Forgot to add the file + fixes
* Fixes
* Pruning from DB fixed
* Add prune by file data
* Fix pruning
* Prune fixes
* Cleanup db blockRoot filter
* Handle file close error
* Fix deletion
* Change read at + remove retentionEpich from bs
* Gaz
* Seperate logic + add detailed comments
* Add tests
* Add retention slot when creating blobStorage
* Fix tests
* Gaz
* Fix testonly import
* Fix linter errors
* Fix retentionSlot calculation
* Move + use MaxEpochsToPersistBlobs
* Remove unused ctx
* Prestons suggestion
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Rename
---------
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-12-05 21:07:34 +00:00
Preston Van Loon
a6b6a938de
blobstorage: Improve mkdirall error ( #13271 )
2023-12-05 19:57:08 +00:00
Sammy Rosso
7a70305935
Blob filesystem: delete blobs ( #13233 )
...
* Add deletion
* Gaz
* Return on removal
* Test cleanup
* Simply blob deletion
* Add test case to prove that deleting a root that doesn't exist will not return an error
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-12-01 19:03:47 +00:00
kasey
1123df7432
Verified roblobs ( #13190 )
...
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-21 18:44:38 +00:00
Sammy Rosso
0f65e51d1e
Blob filesystem: Save Blobs ( #13129 )
...
* Add Save blob and tests
* Remove locks
* Remove test cleanup
* Fix go mod
* Cleanup
* Add checksum
* Add file hashing to fileutil
* Move test
* Check data when exists
* Add one more test
* Rename
* Gaz
* Add packaged level comment
* Save full sidecar + reviews
* Use path builder in test
* Use other BlobSidecar
* Cleanup
* Fix gosec
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-03 16:24:30 +00:00
