leonardchinonso
3b87f8c726
Fix disabled value for --prune.* flags ( #4165 )
...
* Changes:
* Proceeded to check if other h, r, t or c flags for 'before' and 'older' are set before returning from the default state of --prune
Concern:
* Does the 'disabled' value for '--prune=' need to be updated if the '--prune.*' flags are set but the '--prune=' flag is not?
* Modify mode.Initialised only when it is changed
* Updated layout for --prune=*
* save
* save
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-05-17 09:31:14 +07:00
Alex Sharov
e1dec529d4
Generic sort of slices (no allocs, inlinable) ( #4161 )
...
* save
* save
* save
* save
* save
2022-05-16 08:24:12 +01:00
ledgerwatch
24244a09c7
Fix interaction with beacon node ( #4157 )
...
* Fix interaction with beacon node
* Fix logging
* Fix logging and add comments
* Fix logging
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-15 23:32:38 +01:00
ledgerwatch
7125593cf5
Fix nil deference in ProcessHeadersPOS ( #4155 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-05-15 17:09:27 +01:00
Enrique Jose Avila Asapche
a9ce803c67
No hack skip cycle ( #4147 )
...
* added check method to hd
* goroutine to check if we have headers
* added initial cycle to Ethereum object
* stage loop will only run ones
* got rid of anchor print
* header stage skip cycle message
* increased original time by a 2 seconds
* for inside of stageloop
* simplified
* little fixes
* got rid of skipCycle
* simplified
* hmm
2022-05-15 15:11:18 +01:00
ledgerwatch
0b6532fec1
Fix race condition in ProcessHeadersPOS ( #4151 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-15 08:54:36 +01:00
ledgerwatch
e02c77bf5a
Further simplify header download, req/resp metrics ( #4121 )
...
* Further simplify header download, req/resp metrics
* Not break the loop
* Move anchors correctly
* Log anchor state
* Print minNum
* Print hash
* Always add links
* Remove obsolete test
* cleanup
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alex Sharp <alexsharp@alexs-macbook-pro.home>
2022-05-13 11:18:30 +01:00
Alex Sharov
91e43e52a6
Snapsync for ropsten ( #4143 )
...
* save
* save
2022-05-13 14:58:03 +07:00
Alex Sharov
4e0f3c8c4c
Sentry: batch events ( #4140 )
2022-05-13 11:00:44 +07:00
Alex Sharov
1b82acd7ff
it's ok to not have snapshots dir ( #4135 )
...
* save
* save
* save
2022-05-12 15:14:16 +07:00
Alex Sharov
763563ec77
RPCDaemon: open snapshots on startup (because now snapshots dir is atomic), even if no Erigon available ( #4110 )
...
* save
* save
* save
* save
* save
* save
2022-05-10 09:54:44 +01:00
ledgerwatch
8001cf2ad6
Simplify header downloader ( #4104 )
...
* Simplify header downloader
* Remove VerifyQueue
* Fix
* More fixes
* Fix
* Break out of the loop
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-10 08:06:16 +01:00
Alex Sharov
2c26583f6f
Rename sentry.ControlServerImpl to sentry.MultyClient and sentry.SentryServerImpl to sentry.GrpcServer #444
2022-05-10 12:17:44 +07:00
Alex Sharov
3e4fb5dd49
up linter version ( #4108 )
...
* save
* save
* save
2022-05-10 10:14:09 +07:00
Alex Sharov
4ba1f03efe
Snapshots: atomic dir, step 3 ( #4103 )
2022-05-10 09:29:44 +07:00
Alex Sharov
32d8ab34d9
atomic snapshot dir, step 2 ( #4093 )
...
* atomic snapshot dir
* atomic snapshot dir
2022-05-07 14:53:21 +07:00
ledgerwatch
ba987cf262
Remove preverified hashes ( #4088 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-06 15:15:01 +01:00
ledgerwatch
77441916fd
Update header_algos.go ( #4078 )
2022-05-04 19:41:02 +01:00
Andrew Ashikhmin
4529cc308d
Small performance optimization (hash already calculated) ( #4053 )
2022-05-04 10:58:26 +02:00
Giulio rebuffo
6927ffc54b
penalize naughty peers on PoS ( #4060 )
...
* penalize naughty peers
* lint
2022-05-04 10:39:34 +02:00
ledgerwatch
f634314226
Update header_algos.go
2022-05-04 08:33:42 +01:00
ledgerwatch
7c2a7150bb
Reverse relaxation of header downloader ( #4064 )
...
* Undo headers relaxation change
* Undo headers relaxation change
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-05-04 07:46:52 +01:00
Alex Sharov
f590bf682f
RPCDaemon - open snapshots only when they are ready (and indices ready).
2022-05-04 12:55:58 +07:00
battlmonstr
c9a1c93b44
tests: split slow integration tests ( #4048 )
2022-05-04 10:35:59 +07:00
ledgerwatch
f297a28e6d
More relax inclusion of headers in the downloader ( #4050 )
...
* More relax inclusion of headers in the downloader
* Fix
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-03 12:05:54 +01:00
Giulio rebuffo
af5bdceb14
added ovveride merge fork block ( #4027 )
2022-04-29 18:27:17 +02:00
Andrew Ashikhmin
f9858ff9f2
Add override.terminaltotaldifficulty flag ( #4018 )
...
* cmd/utils: initialize f.Value before setting variable
* override.terminaltotaldifficulty flag
* Add OverrideTerminalTotalDifficulty to default_flags
2022-04-28 19:01:51 +02:00
Alex Sharov
eda385eb28
Handle system-txn in block_reader ( #4012 )
...
* reduce downloader deps
* reduce downloader deps
* save
* reduce downloader deps
2022-04-28 13:08:27 +07:00
Alex Sharov
2655d57288
reduce downloader deps ( #4010 )
...
* reduce downloader deps
* reduce downloader deps
2022-04-28 12:13:30 +07:00
Alex Sharov
8b8c442dc6
Open shorter logs #400
2022-04-28 10:11:07 +07:00
Alex Sharov
ea8a347ece
Downloader: calc stat inside, add --torrent.download.slots and limit downloads inside ( #3986 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-04-28 09:10:00 +07:00
Andrew Ashikhmin
6c9d515968
No NewBlock gossip after Merge ( #3995 )
2022-04-27 14:03:06 +02:00
Andrew Ashikhmin
4d2db4467e
FullSync instead of FastSync ( #3980 )
2022-04-26 17:19:24 +02:00
ledgerwatch
b5a2a62c05
Potential fix for verification ( #3962 )
...
* Potential fix for verification
* multi verify
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-04-26 15:36:48 +01:00
Alex Sharov
3906d7e51f
Snapshots: allow stage_headers --unwind behind available snapshots ( #3971 )
...
* save
* save
* save
2022-04-26 12:04:38 +07:00
ledgerwatch
538543ad17
Experiment files 1 by 1 ( #3959 )
...
* Experiment files 1 by 1
* Remove check
* sort preverified snapshots
* docs: docker permissions
* sort preverified snapshots
* sort preverified snapshots
* sort preverified snapshots
* sort preverified snapshots
* sort preverified snapshots
* sort preverified snapshots
* save
* Fix speed log, remove file name
* Move timer out of the loop
* Calculate total size of downloaded files
* Fixes
* Fix
* Fix
* Fix
* Move downloadData
* Fix
* Revert "Fix"
This reverts commit 038e02b8a4d23cd32ddb111e9f325fc4ce1bbe2b.
* Revert "Move downloadData"
This reverts commit 8130a4d9bdc0705082eb7fe94e2261c9313f8482.
* Revert "Fix"
This reverts commit 1dca25bd68772bc42ac710c24698c8670f9f6b86.
* Revert "Fix"
This reverts commit ee5a1e82abd47bef4f9d8f0f68b8497476d29c0b.
* Revert "Fix"
This reverts commit 8af7be71d4685e0d6115fef91ed2f304695e1df9.
* Revert "Fixes"
This reverts commit 50509af81f3721cca957cd15d0286e8f30e5097b.
* Revert "Calculate total size of downloaded files"
This reverts commit 64a26df54f6226d739c8a5b57b32ad5af07d3061.
* Remove progress
* Remove progress
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-25 19:32:27 +01:00
battlmonstr
07367c2312
sentry: refactor flags, add maxpeers. ( #3961 )
2022-04-25 21:47:25 +07:00
Alex Sharov
b4774a831e
Snapshots: support empty buf case ( #3957 )
2022-04-25 10:24:37 +07:00
battlmonstr
3ad25c929c
Sentry GRPC: rename Peers to PeerEvents ( #3944 )
...
* Sentry GRPC: rename Peers to PeerEvents
see https://github.com/ledgerwatch/interfaces/pull/101
* Update to erigon-lib main
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-23 17:52:22 +01:00
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os ( #3946 )
2022-04-23 15:43:00 +01:00
Giulio rebuffo
c5b1836481
Added Ethstats service ( #3931 )
...
* somewhat there but not yet
* lol
* more efficient ethstats
* lint
* not die on no wifi
2022-04-22 07:23:52 +01:00
ledgerwatch
3ec9d270a2
Update flags.go ( #3927 )
2022-04-21 16:18:49 +01:00
Alex Sharov
58c86d550d
Snapshots: Max index parallelism to 4 ( #3925 )
...
* save
* save
* Readme update: built-in rpc daemon and syncmode=snap
* save
* save
* save
2022-04-21 17:49:42 +07:00
Alex Sharov
6aa4999729
Parallel indexing ( #3921 )
2022-04-21 10:34:10 +07:00
alex.sharov
25ab64558d
save
2022-04-20 16:00:04 +07:00
ledgerwatch
5b9f973a56
Update preverified hashes ( #3918 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-19 17:17:26 +01:00
alex.sharov
cbd1116cf3
save
2022-04-19 17:40:09 +07:00
Alex Sharov
8d81c3b0f8
Rwdir fix2 ( #3915 )
2022-04-19 08:55:46 +07:00
ledgerwatch
7080c6fab9
Fix help and README.md about pruning options ( #3914 )
...
* Update README.md
* Update flags.go
* Update README.md
2022-04-18 21:50:21 +01:00
ledgerwatch
748f70e7be
Fixes in header downloader ( #3913 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-18 18:01:20 +01:00
Alex Sharov
e1d7cc3d3f
Enable: syncmode=snap by default (goerli, mainnet, bsc) ( #3910 )
2022-04-18 14:40:40 +07:00
alex.sharov
d10d194984
save
2022-04-18 11:27:36 +07:00
Alex Sharov
9305efeaa7
Snapshots: use kv.ReadAhead helper ( #3909 )
2022-04-18 10:48:09 +07:00
ledgerwatch
b45707ca06
Update header_algos.go ( #3907 )
2022-04-17 07:45:56 +01:00
Alex Sharov
4419c23866
Compress: reduce maxlen to 512 ( #3891 )
2022-04-17 07:59:38 +07:00
ledgerwatch
ba689258fa
Update header_algos.go ( #3900 )
2022-04-16 14:48:45 +01:00
Alex Sharov
777412e0bb
Snapshots: notify before remove ( #3885 )
...
* goerli
* goerli
2022-04-14 14:56:08 +07:00
alex.sharov
d439b76a1f
goerli
2022-04-14 11:12:15 +07:00
ledgerwatch
78a58047dc
Fix in recompression ( #3884 )
...
* Fix in recompression
* Fix error handling
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-14 11:07:17 +07:00
ledgerwatch
156da607e7
More compact representation of huffman trees in the seg files ( #3875 )
...
* More compact representation of huffman trees in the seg files
* Update to latest erigon-lib
* recompress
* recompress
* recompress
* uncompress: buf fix
* Update to latest erigon-lib
* Revert default sync mode
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-13 16:29:44 +01:00
alex.sharov
482e06ead3
uncompress: buf fix
2022-04-13 16:35:21 +07:00
alex.sharov
594477ab23
uncompress: buf fix
2022-04-13 12:45:22 +07:00
Andrew Ashikhmin
b41fe19ecb
Memorize bad PoS headers ( #3879 )
...
* Memorize bad PoS headers
* More places to ReportBadHeaderPoS
2022-04-12 17:05:23 +02:00
alex.sharov
6cbbfb46eb
recompress
2022-04-12 14:53:21 +07:00
Alex Sharov
0842e2bd12
RetireBlocks: preserve genesis ( #3869 )
...
* save
* save
* save
2022-04-11 13:16:45 +07:00
Alex Sharov
54ec6df6e1
ErigonLib: move types to own package ( #3865 )
2022-04-11 10:05:07 +07:00
ledgerwatch
f56d4c5881
Switch peerId from 256 to 512 bit (as in stable) ( #3862 )
...
* Switch peerId from 256 to 512 bit (as in stable)
* go mod tidy
* Fix some tests
* Fixed
* Fixes
* Fix tests
* Update to erigon-lib main
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-10 08:01:25 +01:00
Alex Sharov
9b47daee44
Default syncmode=snap for Mainnet, Bsc, Goerli ( #3861 )
...
* change syncmode default to snap sync
* save
2022-04-10 07:20:20 +01:00
Alex Sharov
b6334870e8
Snapshots: recompress.sh ( #3845 )
...
* save
* save
2022-04-07 13:26:14 +07:00
Alex Sharov
7d86283910
Sapshots: idx merge ( #3841 )
...
* return err on interruption
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-04-07 11:27:57 +07:00
Alex Sharov
bc368098c8
Torrent: more cli params ( #3838 )
2022-04-06 21:06:58 +07:00
alex.sharov
5c5bcfe0fe
bsc snapshot
2022-04-06 15:44:12 +07:00
alex.sharov
d223580d10
goerli snapshot
2022-04-06 09:49:58 +07:00
ledgerwatch
d604b5e107
Update preverified hashes and skip analysis ( #3831 )
...
* Update skip_analysis
* Preverified hashes
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-05 21:49:45 +01:00
Alex Sharov
80bd44fce5
Snapsthos: build indices on retire blocks ( #3823 )
...
* grpc up
* grpc up
* grpc up
2022-04-05 16:22:11 +07:00
Alex Sharov
4cdc38ca58
reopen after merge ( #3822 )
2022-04-05 09:18:39 +07:00
TBC Dev
23e1bc9771
Fix block hash announces ( #3819 )
2022-04-05 08:06:12 +07:00
TBC Dev
057298f71b
Fix broken Goerli sync ( #3817 )
...
Previous preverified hashes were not from Goerli
2022-04-05 08:05:28 +07:00
alex.sharov
4fb2a14756
add readahead to segment uncompress pipe
2022-04-04 21:07:02 +07:00
Giulio rebuffo
1b0d7c80bf
Better batches for Execution (Hashmaps based instead of BTree based) ( #3814 )
2022-04-04 14:11:59 +07:00
alex.sharov
32467007c1
snapshots: auto fix
2022-04-02 13:28:18 +07:00
alex.sharov
38ad751fa0
goerli snapshot
2022-04-02 11:26:39 +07:00
Alex Sharov
68153ff6e6
Relax segments expectation ( #3812 )
...
* skip empty seg
* skip empty seg
* skip empty seg
2022-04-02 10:47:01 +07:00
Alex Sharov
d1218181b0
Snapshots: geth compatibility, use --syncmode=snap flag ( #3808 )
2022-04-01 15:00:50 +07:00
Alex Sharov
8b03582558
compressor: log lvl #408
...
* save
* save
2022-04-01 10:44:30 +07:00
Andrew Ashikhmin
6ff6c39d13
Prefer forkchoice head as latest block in RPC ( #3806 )
...
* LastForkchoice accessors
* Update erigon-lib
* Prefer forkchoice head as latest block in RPC
* Fix typo
2022-04-01 08:33:50 +07:00
Alex Sharov
804f322e91
fix nil pointer in fetch.go ( #3801 )
...
* fix nil pointer in fetch.go
* fix nil pointer in fetch.go
2022-03-31 15:13:20 +07:00
Alex Sharov
328c7b28a6
compress uncompress cli methods ( #3798 )
2022-03-31 14:10:40 +07:00
alex.sharov
4519e10e39
save
2022-03-31 09:44:12 +07:00
Alex Sharov
84cff259b5
snapshots: prune txlookup ( #3794 )
2022-03-30 11:50:16 +07:00
Andrew Ashikhmin
816ad30eff
Small Merge-related clarifications & improvements ( #3785 )
...
* Hd.Fetching -> FetchingNew
* ReadHeadHeaderHash makes more sense in verifyAndSaveNewPoSHeader
* Potentially more correct LatestValidHash in case of payload with invalid transactions
2022-03-29 11:47:07 +02:00
ledgerwatch
eb07ad89dd
Update Skip analysis and preverified hashes ( #3777 )
...
* Update skip analysis
* Add preverified hashes for mainnet and ropsten
* preverified hashes and bootnode for sepolia
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-27 09:42:33 +01:00
Alex Sharov
89d4477df3
Snapshots: nil indices on p2p fix ( #3772 )
2022-03-27 10:07:58 +07:00
Giulio rebuffo
6a345847d8
[Critical] POS Panda Banner Added ( #3758 )
...
* pandaaa
* better
2022-03-24 07:35:58 +00:00
Alex Sharov
e0dfb8fb22
Snapshots: handle well gaps in idx files ( #3762 )
...
* handle well gaps in idx files
* handle well gaps in idx files
2022-03-24 14:19:14 +07:00
Alex Sharov
a357421719
Snapshots: notify rpcdaemon about new snapshot ( #3755 )
2022-03-24 09:25:38 +07:00
Giulio rebuffo
f4bf94f78a
Fixed Erigon README for Authentication API and jwt.hex generates in Datadir ( #3757 )
...
* added datadir jwt
* fixed readme
2022-03-24 09:10:29 +07:00
Andrew Ashikhmin
74a7d7c75a
Refactor PoS downloader ( #3717 )
...
* Remove blockHeight arg from FeedHeaderPoW
* Revert "Remove blockHeight arg from FeedHeaderPoW"
This reverts commit 9bf92921db11cd4e13386fcee29f30241d070fc6.
* Move PayloadMessage & ForkChoiceMessage to package engineapi
* RequestList instead of newPayloadCh & forkChoiceCh
* Introduce request status
* Send reply only when request status is New
* Move BeaconRequestList to HeaderDownload
* Don't SetFetching when PoS (Fetching means handling newBlockHashes)
* Merge Syncing & Synced into DataWasMissing
* Introduce SyncStatus
* onlyNew arg in WaitForRequest
* Move waitingForBeaconChain into RequestList
* Interrupt instead of skipCycleHack
* Introduce timeout for PoS headers
* Split downloadMissingPoSHeaders
* Move StartPoSDownloader into HeaderDownload
* Move Stopping interrupt to StartPoSDownloader
* Move stopping PayloadStatus back to HeadersPOS
* cleanUpPoSDownload
* Post-merge fix
* TestBogusForkchoice
* TestPoSDownloader
* requestStatus in attemptPoSDownload
* Broadcast in SetStatus
* Cosmetics
* attemptPoSDownload -> schedulePoSDownload
* Demote some logs to Trace
2022-03-22 17:49:12 +01:00
Alex Sharov
36b9cc2982
Snapshots: initial sync fix ( #3750 )
2022-03-22 10:33:15 +07:00
Alex Sharov
4cf4c1bac1
Snapshots: better support of p2p ( #3749 )
2022-03-21 20:36:03 +07:00
alex.sharov
752a52c4af
fix test
2022-03-21 19:33:26 +07:00
Alex Sharov
9100e0dc3c
Snapshots: fix for fresh sync ( #3748 )
2022-03-21 14:07:39 +07:00
Alex Sharov
9c49142e6b
Snapshots: script to commit hashes ( #3746 )
2022-03-21 13:40:35 +07:00
Alex Sharov
586ad297e6
Snapshots: script to auto-commit torrent hashes ( #3745 )
2022-03-21 11:22:37 +07:00
Alex Sharov
7799d145ca
Snapshots: fix partial .idx detection ( #3744 )
...
* save
* save
* save
* save
2022-03-21 10:28:33 +07:00
Alex Sharov
5f4bb98264
Snapshots: do indexing for older ranges if need ( #3733 )
...
* save
* save
* save
* save
2022-03-21 10:28:24 +07:00
alex.sharov
1e78e509ac
save
2022-03-21 09:10:52 +07:00
Enrique Jose Avila Asapche
726557fe72
Expose jwt-secret flag in erigon ( #3741 )
2022-03-21 08:33:10 +07:00
ledgerwatch
d5fb8f7d24
[erigon2] Separate handover timing ( #3736 )
...
* Separate handover timing
* Update
* Corrected handover time calculation
* Not use compression when aggregate
* Update to latest erigon-lib
* Update to erigon-lib main
* Update
* Disable reproducible builds
* Restore github actions
* Restore github actions
* Revert "linter to support go 1.18 (#3739 )"
This reverts commit 1fd434d3d1
2022-03-19 16:15:12 +00:00
ledgerwatch
4f7f1554c5
Update helper.go ( #3737 )
2022-03-19 09:36:35 +07:00
EXEC
a6ff889db9
Fix erigon version miss git commit ( #3738 )
2022-03-19 09:36:00 +07:00
Alex Sharov
a04fa4838d
Snapshots: corner cases when .seg exists and .idx doesn't ( #3731 )
2022-03-18 13:36:59 +07:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files ( #3724 )
2022-03-18 11:12:18 +07:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 ( #3705 )
...
* added support of Clique consensus for block processing for erigon2
* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
d5dbad4f27
save (#3715co
2022-03-16 15:35:36 +07:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default ( #3707 )
2022-03-16 09:57:48 +07:00
ledgerwatch
10aee02e2e
Update skip analysis and preverified hashes ( #3700 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-15 14:45:36 +00:00
alex.sharov
6823dc21c4
goerli snapshots
2022-03-13 10:16:19 +07:00
Alex Sharov
f03d08c5ce
Snapshots: retire testing tool ( #3684 )
2022-03-12 17:26:11 +07:00
Alex Sharov
aed679c8a0
snapshots: enum file types ( #3683 )
2022-03-12 15:27:55 +07:00
alex.sharov
82ac1b9649
goerli snapshots
2022-03-12 10:39:34 +07:00
Alex Sharov
b003d07839
Add system-txs to begin end of block ( #3654 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* up torrent
* up torrent
* up torrent
* up torrent
* up torrent
* up torrent
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-03-10 07:48:58 +00:00
Vox
865999abb6
Fix dev chain listen port not set ( #3671 )
...
* Fix forced --port=0 and warn user of hard-coded flags
* Fix typo
* Check for port explicitly set, be verbose about it
Co-authored-by: 0xVox <ohnoyoudont@nottoday.com>
2022-03-10 12:13:55 +07:00
Alex Sharov
0b2ce6f845
rpcdaemon: limit amount of read transactions ( #3637 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* a
* a
* a
* a
* a
* a
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* fefefe
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-03-08 10:43:14 +07:00
Alex Sharov
f5be8919ef
snapshots: merge segments strategy ( #3645 )
...
* save
* save
* save
2022-03-04 14:16:16 +07:00
Andrew Ashikhmin
4607179849
Handle a corner case in forkchoiceUpdated ( #3640 )
...
* Fix typo
* fix typos
* Handle corner case when unwinding to geneis
* No-op ForkChoice
2022-03-03 16:09:03 +01:00
battlmonstr
47df98a499
Use "err" key for logging errors. ( #3632 )
...
log.Warn/Error uses "err" key to log errors in most places.
This renames "error" to "err" in some places to adhere to this convention.
2022-03-01 15:40:51 +00:00
Enrique Jose Avila Asapche
a65f12b248
State override support ( #3628 )
...
* added stateOverride type
* solved import cycle
* refactoring
* imported wrong package
* fixed Call arguments
* typo
* override for traceCall
2022-03-01 15:40:24 +00:00
Alex Sharov
f8c1fb9563
snapshots: speedup slow test ( #3623 )
2022-02-25 11:42:54 +07:00
Alex Sharov
96e0691d4a
snapshots: delete .idx after merge ( #3619 )
2022-02-25 11:41:49 +07:00
battlmonstr
65ce332e8b
Fix lint: remove unused findAndMergeBlockSegments log level parameter. ( #3613 )
...
Fixes lint errors:
turbo/snapshotsync/block_snapshots_test.go:77:69: not enough arguments in call to findAndMergeBlockSegments (typecheck)
_, err := findAndMergeBlockSegments(context.Background(), s, dir, 1)
turbo/snapshotsync/block_snapshots_test.go:88:68: not enough arguments in call to findAndMergeBlockSegments (typecheck)
_, err = findAndMergeBlockSegments(context.Background(), s, dir, 1)
2022-02-24 20:15:33 +07:00
Alex Sharov
ac15d2b922
snapshots: better logging ( #3611 )
2022-02-24 16:28:46 +07:00
ledgerwatch
67b9e5988c
Add preverified hashes for mainnet and ropsten ( #3609 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-24 08:37:20 +00:00
Alex Sharov
454045185a
snapshots: fix indexing of recently merged range ( #3607 )
2022-02-24 14:19:01 +07:00
Alex Sharov
809cfdcf19
snapshots: indexing of recently merged range ( #3606 )
...
* snapshots: fix indexing of recently merged range
* snapshots: fix indexing of recently merged range
* snapshots: fix indexing of recently merged range
* snapshots: fix indexing of recently merged range
* snapshots: fix indexing of recently merged range
2022-02-24 14:15:03 +07:00
Alex Sharov
b94bc84a3b
snapshots: command to debug retire blocks ( #3603 )
...
* snapshots: command_to_debug_retire
* snapshots: command_to_debug_retire
2022-02-24 11:30:34 +07:00
ledgerwatch
17b2a9ba93
Bor fixes ( #3553 )
...
* Integration to work with bor
* Turn off validator set check
* disable verifySeal, add skeleton of postExec stage
* Pass around syscall
* Print
* Print more
* Default heimdall values for integration
* restore contract
* Print
* Print
* Print
* Print
* Print
* Print
* Print
* Print
* Fix nonce of system contract
* Remove prints
* Revert some more printing
* More fixes
* Print log
* Fix transfer log
* More printing
* More printing
* Print
* Print
* Print
* Print
* Print
* Print
* Print
* Fix validaor reward
* Remove printing
* Remove more prints
* Less printing
* Fetch validators from heimdall
* Remove syscall from Seal and CalcDifficulty
* Remove syscall from Prepare
* Print
* Remove DNS discovery
* Print apply snapshot
* Print
* Chunk up snapshot generation
* Chunk up snapshot generation
* Better logs when snapshotting
* Handle parents
* Prevent shadowing of snap
* Fix heimdall fetch
* Logging fixes
* Save generated snapshots
* Add header
* Less logging
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-24 00:03:10 +00:00
ledgerwatch
ee488020dd
Auto detect latest block for optimal use of plain state and state cache ( #3598 )
...
* Auto detect latest block for optimal use of plain state and state cache
* Fix lint
* Fix test
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 23:42:14 +00:00
ledgerwatch
7397af57d4
Optimise eth_getStorageAt for current state ( #3580 )
...
* Optimise eth_getStorageAt for current state
* Fix
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 09:44:55 +00:00
alex.sharov
258e8d5506
snapshots: fix test
2022-02-23 13:18:00 +07:00
Alex Sharov
0a09187492
Snapshots: more filename fixes ( #3583 )
2022-02-23 13:05:44 +07:00
Alex Sharov
81d51e1589
snapshots: better files sort and filter ( #3582 )
2022-02-23 10:54:35 +07:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. ( #3576 )
...
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Giulio rebuffo
b49381821f
Fixed get-payload and Hanging ForkChoice ( #3575 )
...
* done
* removed sleep
* f
* errors.Is(err, io.EOF) instead of err == io.EOF
Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
2022-02-22 14:47:23 +01:00
Alex Sharov
ab01d64916
fix close files error on win ( #3572 )
2022-02-22 16:24:33 +07:00
Alex Sharov
b08cb44353
embed rpcdaemon: enable it only by --http flag ( #3574 )
2022-02-22 16:23:54 +07:00
Alex Sharov
fc278327b9
"erigon snapshots recompress" add test ( #3573 )
2022-02-22 16:02:09 +07:00
Alex Sharov
dbd8a93fa8
"erigon snapshots recompress" to apply new compression rules without db access ( #3571 )
...
* mainnet
* save
* save
* save
* save
2022-02-22 15:35:04 +07:00
alex.sharov
f624d1cc22
save
2022-02-22 15:33:07 +07:00
alex.sharov
1b67c64ca3
update sapshot hashes
2022-02-22 15:02:07 +07:00
alex.sharov
d3c8634df8
fix windows tests
2022-02-22 14:44:57 +07:00
Alex Sharov
5cd561bd74
block snapshots merge ( #3558 )
2022-02-22 08:55:24 +07:00
Andrew Ashikhmin
980276c322
Fix coinbase in stage_mining_exec ( #3564 )
...
* Simplify TerminalBlockHash & TerminalBlockNumber
* Simplify FeedHeaderPoS
* More accurate comment
* Relax TerminalBlockHash check
* Extract the sleep hack into stageLoopIsBusy
* Several short naps instead of a single longish sleep
* Fix coinbase in stage_mining_exec
* More orthodox loop
* typo in comment
2022-02-21 18:05:39 +01:00
Alex Sharov
bed56c6363
Embed rpcdaemon - more cli flags ( #3556 )
...
* more rpc emed flags
* more rpc emed flags
* more rpc emed flags
2022-02-21 09:32:42 +07:00
Enrique Jose Avila Asapche
e348b3ea93
deleted kovan ( #3546 )
2022-02-19 08:15:44 +00:00
Giulio rebuffo
5db869236a
Fix Reorg Indexing on Proof-of-Stake ( #3538 )
...
* little fix
* feed pos
2022-02-18 17:03:47 +01:00
Alex Sharov
226982d293
Use class dir.Rw - to separate Ro and Rw access to snapshotDir ( #3534 )
...
* save
* save
* save
* save
* save
* save
* save
2022-02-18 09:24:17 +07:00
Andrew Ashikhmin
0701c83222
Fix stall on unwind in handleForkChoice ( #3536 )
...
* ReadHeader instead of ReadHeaderByHash
* Add Engine API to rpcdaemon README
* Fix nil pointer dereference in HandlePendingBlock
* Fix latestValidHash JSON
* Delete GetPayloadBodiesV1
* Break infinte unwind loop in handleForkChoice
* More logging
2022-02-17 18:30:57 +01:00
ledgerwatch
f1aea7707f
Update skip analysis and preverified headers ( #3527 )
...
* Updated skip analysis for mainnet
* Preverified hashes for mainnet
* Add Sepolia and Ropsten
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 18:41:49 +00:00
Alex Sharov
237a7d24d0
Embed RPCDaemon ( #3522 )
...
* save
* save
* save
* save
* save
2022-02-16 17:38:54 +00:00
Alex Sharov
7960d0a472
Snapshot: link.header is nil sometime ( #3520 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
* Add another check
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 12:55:43 +00:00
Andrew Ashikhmin
7bad3a9c59
state.PlainState is now at the beginning of blockNbr, not the end ( #3517 )
...
* state.PlainState is now at the beginning of blockNbr, not the end (fix discprepancy with GetAsOf)
* Fix tests
2022-02-16 08:38:12 +00:00
alex.sharov
1589a8d3d6
snapshots: header stage fixes
2022-02-16 09:30:57 +07:00
Alex Sharov
e652b5fa87
snapshots: header stage fixes ( #3518 )
2022-02-16 08:38:54 +07:00
alex.sharov
83c99d66da
torrent flags - fix parsing logic
2022-02-14 14:14:37 +07:00
alex.sharov
f23fdf51c1
snapshots: add torrent cli flags
2022-02-14 10:34:14 +07:00
ledgerwatch
d4e623b9e5
[header downloader] introduce queues instead of insertList ( #3489 )
...
* First change for header queues
* Fix
* Fix import
* trigger verification when highestInDb changes
* Print hash, fix MarkAllPreverified
* Fix test
* Cleanup
* More cleanup
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-13 11:27:53 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) ( #3493 )
2022-02-12 20:33:09 +07:00
Andrew Ashikhmin
fe9f841172
Add Beacon payloads to body downloader prefetch ( #3482 )
...
* Block instead of PayloadMessage
* tx.MarshalBinary instead of rlp.Encode in StartProposer
* Add Beacon payloads to body downloader prefetch
* Restore PayloadMessage
2022-02-11 13:24:29 +01:00
ledgerwatch
a5bd91e2e1
Update skip analysis and preverified hashes ( #3471 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-10 14:10:10 +00:00
Andrew Ashikhmin
d6525b342d
Update instead of SaveStageProgress for stages.Header ( #3456 )
...
* Update instead of SaveStageProgress for stages.Header
* Call WriteHeadHeaderHash/stage.Update after the loop
2022-02-10 13:18:51 +00:00
ledgerwatch
2e0602b8dd
Revert "Inverted lowestNum and highestNum in header downloader ( #3301 )" ( #3468 )
...
This reverts commit 6704c48ed7
2022-02-10 12:23:40 +00:00
Alex Sharov
2e17b8f515
snapshot: parallel transaction index, recsplit: reset offset collector, etl: faster flush and load ( #3465 )
2022-02-10 14:40:29 +07:00
Jared Doro
0223489183
[WIP] Adding Support for Sepolia ( #3462 )
...
* [WIP] Adding Sepolia
* Fix formatting
* Added Timestamp to Sepolia genesis
* update config
* fixed formating
2022-02-10 07:27:36 +00:00
Alex Sharov
e695a6e58b
state and newHead: 1 goroutine per subscriber ( #3452 )
2022-02-10 08:25:58 +07:00
Andrew Ashikhmin
8466cb1b7b
New Engine API semantics (Kiln v1) ( #3340 )
...
* Disable PoS sync temporarily
* Resore PoS sync
* Handle Ctrl^C in HeadersPOS
* Consistent naming
* Extract verifyAndSavePoSHeader & downloadMissingPoSHeaders
* Preparation for EngineForkChoiceUpdated re-orgs
* Extract ForkingPoint
* Comments
* New proto for Engine API
* EngineExecutePayload -> EngineNewPayload
* Return INVALID_BLOCK_HASH if block hash is invalid
* Return EngineStatus_ACCEPTED for side chain blocks
* Update erigon-lib (PR 268)
* Fix payload2Hash
* reverseDownloadCh -> beaconPayloadCh
* Update erigon-lib
* Engine API updated
* ExecutionStatus -> PayloadStatus
* Introduce forkChoiceCh
* Mock ForkChoiceMessage/PayloadStatus
* Add ValidationError to PayloadStatus
* Small clean-ups
* Add INVALID_TERMINAL_BLOCK to EngineStatus
* Add a comment
* Extract handleNewPayload & handleForkChoice
* Partially implement handleForkChoice
* Update erigon-lib
* short vs long re-org
* Move header insertion out of downloadMissingPoSHeaders
* Update erigon-lib
* Refactor ProcessSegmentPOS
* Fix imports
* Wire downloadMissingPoSHeaders into handleForkChoice
* evictOldPendingPayloads
* nolint:unused for assertSegment
* Try nolint instead of nolint:unused
* Small comment improvements
* HeadHeaderHash/StageProgress in handleForkChoice
* TODO: bodyDownloader.AddToPrefetch(block)
* Review call suggestions
* Don't use ReadHeaderNumber in ProcessSegmentPOS
* Don't leave ethbackend waiting when server is stopping
* Update erigon-lib
* More explicit signature of downloadMissingPoSHeaders
2022-02-09 08:33:22 +01:00
alex.sharov
ab623773c6
Revert "save"
...
This reverts commit 104ed39729
2022-02-09 14:25:33 +07:00
alex.sharov
104ed39729
save
2022-02-09 14:11:52 +07:00
Alex Sharov
1d6f8a04ac
Embed downloader ( #3450 )
...
* save
* save
* save
* save
* save
* save
2022-02-09 13:22:43 +07:00
alex.sharov
d8af3953c2
snapshot: mainnet hashes
2022-02-09 09:43:09 +07:00
alex.sharov
e82e19c4a4
snapshot: mainnet hashes
2022-02-09 08:15:49 +07:00
Alex Sharov
98ca4bd460
snapshots: funcs to fast decompress all segments ( #3445 )
2022-02-08 08:58:56 +07:00
Enrique Jose Avila Asapche
35fcd3a829
Merging Turbo bor into devel ( #3372 )
...
* implemented bor consensus
* add bor flags to default
* change bucket into snapshot to clique
* enable stateSync
* bypass reciept checks
* fix receipt calculation and bor logs
* fix: contract call wrt bor
* Update mumbai config
* Add: bor-mainnet flag and config
* Add bor consensus to integration
* use header coinbase in block context
* london fork mumbai changes
* fix genesis error
* Jaipur fork for mumbai
* add sysCall to verifyHeader
* added bor related rpc method implementation
* added bor specific rpc extensions
* fixes in snapshot implementation, major refactor for bor rpc
* modify consensus specific db path for bor
* fix: remove parallel compute for get root hash rpc method
* Added bor-receipt flow
* Use turbo-bor-lib and bor tables
* Use bor table in RPC snapshot
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* update rpc readme
* link rpc docs in readme
* Update Readme
* Update Readme
* move erigon namespace rpc methods to eth
* rm: erigon namespace
* rm: erigon namespace, update list of available rpc methods, add example
* fix: binary name in rpc readme
* fix: max db size
* Add london to bor-mainnet
* updated node.go
* add system req to readme
* golang version fix readme
* added networknames in correct place
* nil
* ran gofmt
* erigon
* fixed fake.go
* dont need turbor-lib
* old readme
* fixing readme
* half
* other half
* changed return
* fixing return
* fixed return
* fixed flags
* gofmt
* merge with devel
* latest erigon-lib
* fixed context.coinbase
* took out syscall
* fixed params in hash
* bor type now is consensus.Engine
* parlia is consensus.Engine
* missing arg and repeated importation
* repeated importation
* fixed eth_receipts.go
* deleted duplicate issuance
* part of consensus.Engine type
* added eth_api issuance
* networkname
* added erigon_system file
* fork struct taken out
* added erigon block
* getLogByHash for erigonImpl
* gofmt
* fixed lint
* ops
* gofmt
* gofmt
* added APIImple functions
* fixed clique test
* took out print
* fixed state added balance
* fixed README
* fixed rpcDaemon README
* fixed integration README
* updated blockchain.go
* lint
* added bor back into blockchain.go
* took out comment
* lint
* updated daemon
* updated wtb
* removed duplicate
* removed VerifyHeaders
* prevent use of wrong Transfer
* fixed state_processor.go
* fixed state_transition.go
* fixed headers
* returning err
* error handling in bor read tx look up
* put for txLookUp
* dealing with error
* lint
* traces
* more traces
* fixed receipt in execution
* getTrasanction receipt for bor or others
* nil
* lint
* ops
* deleted syscall
* took out else
* Merge branch 'devel
* tests syscalls
* changed borReceipt to receipt
* reset header algos
* arguments fix
* took out prefixes
* lint
* erigon-named
* borReceiptKey = blocknumber
* reverts e3b60c2e159d03efcb855f7ab3da5a098dd60c33.
* correct hashing tx
* dont need it here
* lint
* added txlookup for bor
* change to uint256
* outputs for isBor
* wrapper
* added isBor and isParlia
* isBor
* fixed BorTransfer
* not readBody
* correct prefix
* added blockNum
* added readStorageBody
* readStorageBody
* lint
* got rid of unnecessary bor_receipt func
* onlny if bor
* use clone
* append
* writeToSlice
* added isBor flag
* fixed writeToSlice
* normal sorting
* lint
* Reset erigon-snapshots
* Move bor prefix into if
Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Uttam Singh <uttamkhanduja@yahoo.in>
Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-07 21:30:46 +00:00
alex.sharov
ad866187e2
save
2022-02-07 18:33:45 +07:00
Alex Sharov
69ae7dcad9
prepare to add downloader inside erigon ( #3441 )
...
* save
* save
* save
* save
2022-02-07 16:54:20 +07:00
Alex Sharov
8fa4d759f8
Snapshots: Add verify command, fix mainnet hashes ( #3439 )
2022-02-07 12:07:46 +07:00
Alex Sharov
bbf1b5b4bd
Torrent less goroutines ( #3423 )
2022-02-04 12:42:55 +07:00
ledgerwatch
00c5c7d589
Update preverified hashes for mainnet and ropsten ( #3415 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 10:11:59 +00:00
Alex Sharov
15a57373af
remove debug line ( #3406 )
2022-02-02 15:25:56 +07:00
Alex Sharov
ba35b31f51
Snapshot: fix empty sender ( #3405 )
...
* save
* save
* save
* save
2022-02-02 15:24:06 +07:00
Alex Sharov
5567eebb23
Decompressor.WithReadAhead ( #3395 )
...
* readahead
* readahead
2022-02-01 11:19:16 +07:00
Alex Sharov
d5cbbf5a65
Fix panic in devel ( #3386 )
2022-01-31 14:04:16 +07:00
Alex Sharov
94930c5fe0
snapshots: bsc ( #3382 )
2022-01-31 12:39:56 +07:00
Alex Sharov
7b77585195
Downloader: supress some errors ( #3381 )
2022-01-31 12:28:59 +07:00
Alex Sharov
a46a15a5e5
goerli snapshot hashes ( #3365 )
2022-01-27 23:19:41 +07:00
Alexey Sharp
91f694ab3b
Add ropsten preverified hashes
2022-01-27 11:18:17 +00:00
Giulio rebuffo
9ca0a721af
Cumulative index stage ( #3314 )
...
* prog
* chain traffic
* added log estimation
* fixed err
* unwind
2022-01-27 10:49:03 +00:00
Alexey Sharp
8bf28dec45
Update skip_analysis and preverified hashes for mainnet
2022-01-27 10:29:47 +00:00
ledgerwatch
46a69c1f89
Fix sending transactions via txpool ( #3350 )
...
* Fix sending transactions via txpool
* Update to latest erigon-lib
* Update to latest erigon-lib
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-26 16:44:21 +00:00
Alex Sharov
bb1d19060f
Add downloader to docker-compose ( #3348 )
2022-01-26 22:26:33 +07:00
