Commit Graph

743 Commits

Author SHA1 Message Date
Giulio rebuffo
f4bf94f78a
Fixed Erigon README for Authentication API and jwt.hex generates in Datadir (#3757)
* added datadir jwt

* fixed readme
2022-03-24 09:10:29 +07:00
Andrew Ashikhmin
74a7d7c75a
Refactor PoS downloader (#3717)
* Remove blockHeight arg from FeedHeaderPoW

* Revert "Remove blockHeight arg from FeedHeaderPoW"

This reverts commit 9bf92921db11cd4e13386fcee29f30241d070fc6.

* Move PayloadMessage & ForkChoiceMessage to package engineapi

* RequestList instead of newPayloadCh & forkChoiceCh

* Introduce request status

* Send reply only when request status is New

* Move BeaconRequestList to HeaderDownload

* Don't SetFetching when PoS (Fetching means handling newBlockHashes)

* Merge Syncing & Synced into DataWasMissing

* Introduce SyncStatus

* onlyNew arg in WaitForRequest

* Move waitingForBeaconChain into RequestList

* Interrupt instead of skipCycleHack

* Introduce timeout for PoS headers

* Split downloadMissingPoSHeaders

* Move StartPoSDownloader into HeaderDownload

* Move Stopping interrupt to StartPoSDownloader

* Move stopping PayloadStatus back to HeadersPOS

* cleanUpPoSDownload

* Post-merge fix

* TestBogusForkchoice

* TestPoSDownloader

* requestStatus in attemptPoSDownload

* Broadcast in SetStatus

* Cosmetics

* attemptPoSDownload -> schedulePoSDownload

* Demote some logs to Trace
2022-03-22 17:49:12 +01:00
Alex Sharov
36b9cc2982
Snapshots: initial sync fix (#3750) 2022-03-22 10:33:15 +07:00
Alex Sharov
4cf4c1bac1
Snapshots: better support of p2p (#3749) 2022-03-21 20:36:03 +07:00
alex.sharov
752a52c4af fix test 2022-03-21 19:33:26 +07:00
Alex Sharov
9100e0dc3c
Snapshots: fix for fresh sync (#3748) 2022-03-21 14:07:39 +07:00
Alex Sharov
9c49142e6b
Snapshots: script to commit hashes (#3746) 2022-03-21 13:40:35 +07:00
Alex Sharov
586ad297e6
Snapshots: script to auto-commit torrent hashes (#3745) 2022-03-21 11:22:37 +07:00
Alex Sharov
7799d145ca
Snapshots: fix partial .idx detection (#3744)
* save

* save

* save

* save
2022-03-21 10:28:33 +07:00
Alex Sharov
5f4bb98264
Snapshots: do indexing for older ranges if need (#3733)
* save

* save

* save

* save
2022-03-21 10:28:24 +07:00
alex.sharov
1e78e509ac save 2022-03-21 09:10:52 +07:00
Enrique Jose Avila Asapche
726557fe72
Expose jwt-secret flag in erigon (#3741) 2022-03-21 08:33:10 +07:00
ledgerwatch
d5fb8f7d24
[erigon2] Separate handover timing (#3736)
* Separate handover timing

* Update

* Corrected handover time calculation

* Not use compression when aggregate

* Update to latest erigon-lib

* Update to erigon-lib main

* Update

* Disable reproducible builds

* Restore github actions

* Restore github actions

* Revert "linter to support go 1.18 (#3739)"

This reverts commit 1fd434d3d1.

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-19 16:15:12 +00:00
ledgerwatch
4f7f1554c5
Update helper.go (#3737) 2022-03-19 09:36:35 +07:00
EXEC
a6ff889db9
Fix erigon version miss git commit (#3738) 2022-03-19 09:36:00 +07:00
Alex Sharov
a04fa4838d
Snapshots: corner cases when .seg exists and .idx doesn't (#3731) 2022-03-18 13:36:59 +07:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files (#3724) 2022-03-18 11:12:18 +07:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 (#3705)
* added support of Clique consensus for block processing for erigon2

* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
d5dbad4f27
save (#3715co 2022-03-16 15:35:36 +07:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default (#3707) 2022-03-16 09:57:48 +07:00
ledgerwatch
10aee02e2e
Update skip analysis and preverified hashes (#3700)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-15 14:45:36 +00:00
alex.sharov
6823dc21c4 goerli snapshots 2022-03-13 10:16:19 +07:00
Alex Sharov
f03d08c5ce
Snapshots: retire testing tool (#3684) 2022-03-12 17:26:11 +07:00
Alex Sharov
aed679c8a0
snapshots: enum file types (#3683) 2022-03-12 15:27:55 +07:00
alex.sharov
82ac1b9649 goerli snapshots 2022-03-12 10:39:34 +07:00
Alex Sharov
b003d07839
Add system-txs to begin end of block (#3654)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-03-10 07:48:58 +00:00
Vox
865999abb6
Fix dev chain listen port not set (#3671)
* Fix forced --port=0 and warn user of hard-coded flags

* Fix typo

* Check for port explicitly set, be verbose about it

Co-authored-by: 0xVox <ohnoyoudont@nottoday.com>
2022-03-10 12:13:55 +07:00
Alex Sharov
0b2ce6f845
rpcdaemon: limit amount of read transactions (#3637)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* a

* a

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-03-08 10:43:14 +07:00
Alex Sharov
f5be8919ef
snapshots: merge segments strategy (#3645)
* save

* save

* save
2022-03-04 14:16:16 +07:00
Andrew Ashikhmin
4607179849
Handle a corner case in forkchoiceUpdated (#3640)
* Fix typo

* fix typos

* Handle corner case when unwinding to geneis

* No-op ForkChoice
2022-03-03 16:09:03 +01:00
battlmonstr
47df98a499
Use "err" key for logging errors. (#3632)
log.Warn/Error uses "err" key to log errors in most places.
This renames "error" to "err" in some places to adhere to this convention.
2022-03-01 15:40:51 +00:00
Enrique Jose Avila Asapche
a65f12b248
State override support (#3628)
* added stateOverride type

* solved import cycle

* refactoring

* imported wrong package

* fixed Call arguments

* typo

* override for traceCall
2022-03-01 15:40:24 +00:00
Alex Sharov
f8c1fb9563
snapshots: speedup slow test (#3623) 2022-02-25 11:42:54 +07:00
Alex Sharov
96e0691d4a
snapshots: delete .idx after merge (#3619) 2022-02-25 11:41:49 +07:00
battlmonstr
65ce332e8b
Fix lint: remove unused findAndMergeBlockSegments log level parameter. (#3613)
Fixes lint errors:
turbo/snapshotsync/block_snapshots_test.go:77:69: not enough arguments in call to findAndMergeBlockSegments (typecheck)
	_, err := findAndMergeBlockSegments(context.Background(), s, dir, 1)
turbo/snapshotsync/block_snapshots_test.go:88:68: not enough arguments in call to findAndMergeBlockSegments (typecheck)
	_, err = findAndMergeBlockSegments(context.Background(), s, dir, 1)
2022-02-24 20:15:33 +07:00
Alex Sharov
ac15d2b922
snapshots: better logging (#3611) 2022-02-24 16:28:46 +07:00
ledgerwatch
67b9e5988c
Add preverified hashes for mainnet and ropsten (#3609)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-24 08:37:20 +00:00
Alex Sharov
454045185a
snapshots: fix indexing of recently merged range (#3607) 2022-02-24 14:19:01 +07:00
Alex Sharov
809cfdcf19
snapshots: indexing of recently merged range (#3606)
* snapshots: fix indexing of recently merged range

* snapshots: fix indexing of recently merged range

* snapshots: fix indexing of recently merged range

* snapshots: fix indexing of recently merged range

* snapshots: fix indexing of recently merged range
2022-02-24 14:15:03 +07:00
Alex Sharov
b94bc84a3b
snapshots: command to debug retire blocks (#3603)
* snapshots: command_to_debug_retire

* snapshots: command_to_debug_retire
2022-02-24 11:30:34 +07:00
ledgerwatch
17b2a9ba93
Bor fixes (#3553)
* Integration to work with bor

* Turn off validator set check

* disable verifySeal, add skeleton of postExec stage

* Pass around syscall

* Print

* Print more

* Default heimdall values for integration

* restore contract

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Fix nonce of system contract

* Remove prints

* Revert some more printing

* More fixes

* Print log

* Fix transfer log

* More printing

* More printing

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Fix validaor reward

* Remove printing

* Remove more prints

* Less printing

* Fetch validators from heimdall

* Remove syscall from Seal and CalcDifficulty

* Remove syscall from Prepare

* Print

* Remove DNS discovery

* Print apply snapshot

* Print

* Chunk up snapshot generation

* Chunk up snapshot generation

* Better logs when snapshotting

* Handle parents

* Prevent shadowing of snap

* Fix heimdall fetch

* Logging fixes

* Save generated snapshots

* Add header

* Less logging

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-24 00:03:10 +00:00
ledgerwatch
ee488020dd
Auto detect latest block for optimal use of plain state and state cache (#3598)
* Auto detect latest block for optimal use of plain state and state cache

* Fix lint

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 23:42:14 +00:00
ledgerwatch
7397af57d4
Optimise eth_getStorageAt for current state (#3580)
* Optimise eth_getStorageAt for current state

* Fix

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 09:44:55 +00:00
alex.sharov
258e8d5506 snapshots: fix test 2022-02-23 13:18:00 +07:00
Alex Sharov
0a09187492
Snapshots: more filename fixes (#3583) 2022-02-23 13:05:44 +07:00
Alex Sharov
81d51e1589
snapshots: better files sort and filter (#3582) 2022-02-23 10:54:35 +07:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. (#3576)
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Giulio rebuffo
b49381821f
Fixed get-payload and Hanging ForkChoice (#3575)
* done

* removed sleep

* f

* errors.Is(err, io.EOF) instead of err == io.EOF

Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
2022-02-22 14:47:23 +01:00
Alex Sharov
ab01d64916
fix close files error on win (#3572) 2022-02-22 16:24:33 +07:00
Alex Sharov
b08cb44353
embed rpcdaemon: enable it only by --http flag (#3574) 2022-02-22 16:23:54 +07:00
Alex Sharov
fc278327b9
"erigon snapshots recompress" add test (#3573) 2022-02-22 16:02:09 +07:00
Alex Sharov
dbd8a93fa8
"erigon snapshots recompress" to apply new compression rules without db access (#3571)
* mainnet

* save

* save

* save

* save
2022-02-22 15:35:04 +07:00
alex.sharov
f624d1cc22 save 2022-02-22 15:33:07 +07:00
alex.sharov
1b67c64ca3 update sapshot hashes 2022-02-22 15:02:07 +07:00
alex.sharov
d3c8634df8 fix windows tests 2022-02-22 14:44:57 +07:00
Alex Sharov
5cd561bd74
block snapshots merge (#3558) 2022-02-22 08:55:24 +07:00
Andrew Ashikhmin
980276c322
Fix coinbase in stage_mining_exec (#3564)
* Simplify TerminalBlockHash & TerminalBlockNumber

* Simplify FeedHeaderPoS

* More accurate comment

* Relax TerminalBlockHash check

* Extract the sleep hack into stageLoopIsBusy

* Several short naps instead of a single longish sleep

* Fix coinbase in stage_mining_exec

* More orthodox loop

* typo in comment
2022-02-21 18:05:39 +01:00
Alex Sharov
bed56c6363
Embed rpcdaemon - more cli flags (#3556)
* more rpc emed flags

* more rpc emed flags

* more rpc emed flags
2022-02-21 09:32:42 +07:00
Enrique Jose Avila Asapche
e348b3ea93
deleted kovan (#3546) 2022-02-19 08:15:44 +00:00
Giulio rebuffo
5db869236a
Fix Reorg Indexing on Proof-of-Stake (#3538)
* little fix

* feed pos
2022-02-18 17:03:47 +01:00
Alex Sharov
226982d293
Use class dir.Rw - to separate Ro and Rw access to snapshotDir (#3534)
* save

* save

* save

* save

* save

* save

* save
2022-02-18 09:24:17 +07:00
Andrew Ashikhmin
0701c83222
Fix stall on unwind in handleForkChoice (#3536)
* ReadHeader instead of ReadHeaderByHash

* Add Engine API to rpcdaemon README

* Fix nil pointer dereference in HandlePendingBlock

* Fix latestValidHash JSON

* Delete GetPayloadBodiesV1

* Break infinte unwind loop in handleForkChoice

* More logging
2022-02-17 18:30:57 +01:00
ledgerwatch
f1aea7707f
Update skip analysis and preverified headers (#3527)
* Updated skip analysis for mainnet

* Preverified hashes for mainnet

* Add Sepolia and Ropsten

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 18:41:49 +00:00
Alex Sharov
237a7d24d0
Embed RPCDaemon (#3522)
* save

* save

* save

* save

* save
2022-02-16 17:38:54 +00:00
Alex Sharov
7960d0a472
Snapshot: link.header is nil sometime (#3520)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* Add another check

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 12:55:43 +00:00
Andrew Ashikhmin
7bad3a9c59
state.PlainState is now at the beginning of blockNbr, not the end (#3517)
* state.PlainState is now at the beginning of blockNbr, not the end (fix discprepancy with GetAsOf)

* Fix tests
2022-02-16 08:38:12 +00:00
alex.sharov
1589a8d3d6 snapshots: header stage fixes 2022-02-16 09:30:57 +07:00
Alex Sharov
e652b5fa87
snapshots: header stage fixes (#3518) 2022-02-16 08:38:54 +07:00
alex.sharov
83c99d66da torrent flags - fix parsing logic 2022-02-14 14:14:37 +07:00
alex.sharov
f23fdf51c1 snapshots: add torrent cli flags 2022-02-14 10:34:14 +07:00
ledgerwatch
d4e623b9e5
[header downloader] introduce queues instead of insertList (#3489)
* First change for header queues

* Fix

* Fix import

* trigger verification when highestInDb changes

* Print hash, fix MarkAllPreverified

* Fix test

* Cleanup

* More cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-13 11:27:53 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
Andrew Ashikhmin
fe9f841172
Add Beacon payloads to body downloader prefetch (#3482)
* Block instead of PayloadMessage

* tx.MarshalBinary instead of rlp.Encode in StartProposer

* Add Beacon payloads to body downloader prefetch

* Restore PayloadMessage
2022-02-11 13:24:29 +01:00
ledgerwatch
a5bd91e2e1
Update skip analysis and preverified hashes (#3471)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-10 14:10:10 +00:00
Andrew Ashikhmin
d6525b342d
Update instead of SaveStageProgress for stages.Header (#3456)
* Update instead of SaveStageProgress for stages.Header

* Call WriteHeadHeaderHash/stage.Update after the loop
2022-02-10 13:18:51 +00:00
ledgerwatch
2e0602b8dd
Revert "Inverted lowestNum and highestNum in header downloader (#3301)" (#3468)
This reverts commit 6704c48ed7.

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-10 12:23:40 +00:00
Alex Sharov
2e17b8f515
snapshot: parallel transaction index, recsplit: reset offset collector, etl: faster flush and load (#3465) 2022-02-10 14:40:29 +07:00
Jared Doro
0223489183
[WIP] Adding Support for Sepolia (#3462)
* [WIP] Adding Sepolia

* Fix formatting

* Added Timestamp to Sepolia genesis

* update config

* fixed formating
2022-02-10 07:27:36 +00:00
Alex Sharov
e695a6e58b
state and newHead: 1 goroutine per subscriber (#3452) 2022-02-10 08:25:58 +07:00
Andrew Ashikhmin
8466cb1b7b
New Engine API semantics (Kiln v1) (#3340)
* Disable PoS sync temporarily

* Resore PoS sync

* Handle Ctrl^C in HeadersPOS

* Consistent naming

* Extract verifyAndSavePoSHeader & downloadMissingPoSHeaders

* Preparation for EngineForkChoiceUpdated re-orgs

* Extract ForkingPoint

* Comments

* New proto for Engine API

* EngineExecutePayload -> EngineNewPayload

* Return INVALID_BLOCK_HASH if block hash is invalid

* Return EngineStatus_ACCEPTED for side chain blocks

* Update erigon-lib (PR 268)

* Fix payload2Hash

* reverseDownloadCh -> beaconPayloadCh

* Update erigon-lib

* Engine API updated

* ExecutionStatus -> PayloadStatus

* Introduce forkChoiceCh

* Mock ForkChoiceMessage/PayloadStatus

* Add ValidationError to PayloadStatus

* Small clean-ups

* Add INVALID_TERMINAL_BLOCK to EngineStatus

* Add a comment

* Extract handleNewPayload & handleForkChoice

* Partially implement handleForkChoice

* Update erigon-lib

* short vs long re-org

* Move header insertion out of downloadMissingPoSHeaders

* Update erigon-lib

* Refactor ProcessSegmentPOS

* Fix imports

* Wire downloadMissingPoSHeaders into handleForkChoice

* evictOldPendingPayloads

* nolint:unused for assertSegment

* Try nolint instead of nolint:unused

* Small comment improvements

* HeadHeaderHash/StageProgress in handleForkChoice

* TODO: bodyDownloader.AddToPrefetch(block)

* Review call suggestions

* Don't use ReadHeaderNumber in ProcessSegmentPOS

* Don't leave ethbackend waiting when server is stopping

* Update erigon-lib

* More explicit signature of downloadMissingPoSHeaders
2022-02-09 08:33:22 +01:00
alex.sharov
ab623773c6 Revert "save"
This reverts commit 104ed39729.
2022-02-09 14:25:33 +07:00
alex.sharov
104ed39729 save 2022-02-09 14:11:52 +07:00
Alex Sharov
1d6f8a04ac
Embed downloader (#3450)
* save

* save

* save

* save

* save

* save
2022-02-09 13:22:43 +07:00
alex.sharov
d8af3953c2 snapshot: mainnet hashes 2022-02-09 09:43:09 +07:00
alex.sharov
e82e19c4a4 snapshot: mainnet hashes 2022-02-09 08:15:49 +07:00
Alex Sharov
98ca4bd460
snapshots: funcs to fast decompress all segments (#3445) 2022-02-08 08:58:56 +07:00
Enrique Jose Avila Asapche
35fcd3a829
Merging Turbo bor into devel (#3372)
* implemented bor consensus

* add bor flags to default

* change bucket into snapshot to clique

* enable stateSync

* bypass reciept checks

* fix receipt calculation and bor logs

* fix: contract call wrt bor

* Update mumbai config

* Add: bor-mainnet flag and config

* Add bor consensus to integration

* use header coinbase in block context

* london fork mumbai changes

* fix genesis error

* Jaipur fork for mumbai

* add sysCall to verifyHeader

* added bor related rpc method implementation

* added bor specific rpc extensions

* fixes in snapshot implementation, major refactor for bor rpc

* modify consensus specific db path for bor

* fix: remove parallel compute for get root hash rpc method

* Added bor-receipt flow

* Use turbo-bor-lib and bor tables

* Use bor table in RPC snapshot

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* update rpc readme

* link rpc docs in readme

* Update Readme

* Update Readme

* move erigon namespace rpc methods to eth

* rm: erigon namespace

* rm: erigon namespace, update list of available rpc methods, add example

* fix: binary name in rpc readme

* fix: max db size

* Add london to bor-mainnet

* updated node.go

* add system req to readme

* golang version fix readme

* added networknames in correct place

* nil

* ran gofmt

* erigon

* fixed fake.go

* dont need turbor-lib

* old readme

* fixing readme

* half

* other half

* changed return

* fixing return

* fixed return

* fixed flags

* gofmt

* merge with devel

* latest erigon-lib

* fixed context.coinbase

* took out syscall

* fixed params in hash

* bor type now is consensus.Engine

* parlia is consensus.Engine

* missing arg and repeated importation

* repeated importation

* fixed eth_receipts.go

* deleted duplicate issuance

* part of consensus.Engine type

* added eth_api issuance

* networkname

* added erigon_system file

* fork struct taken out

* added erigon block

* getLogByHash for erigonImpl

* gofmt

* fixed lint

* ops

* gofmt

* gofmt

* added APIImple functions

* fixed clique test

* took out print

* fixed state added balance

* fixed README

* fixed rpcDaemon README

* fixed integration README

* updated blockchain.go

* lint

* added bor back into blockchain.go

* took out comment

* lint

* updated daemon

* updated wtb

* removed duplicate

* removed VerifyHeaders

* prevent use of wrong Transfer

* fixed state_processor.go

* fixed state_transition.go

* fixed headers

* returning err

* error handling in bor read tx look up

* put for txLookUp

* dealing with error

* lint

* traces

* more traces

* fixed receipt in execution

* getTrasanction receipt for bor or others

* nil

* lint

* ops

* deleted syscall

* took out else

* Merge branch 'devel

* tests syscalls

* changed borReceipt to receipt

* reset header algos

* arguments fix

* took out prefixes

* lint

* erigon-named

* borReceiptKey = blocknumber

* reverts e3b60c2e159d03efcb855f7ab3da5a098dd60c33.

* correct hashing tx

* dont need it here

* lint

* added txlookup for bor

* change to uint256

* outputs for isBor

* wrapper

* added isBor and isParlia

* isBor

* fixed BorTransfer

* not readBody

* correct prefix

* added blockNum

* added readStorageBody

* readStorageBody

* lint

* got rid of unnecessary bor_receipt func

* onlny if bor

* use clone

* append

* writeToSlice

* added isBor flag

* fixed writeToSlice

* normal sorting

* lint

* Reset erigon-snapshots

* Move bor prefix into if

Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Uttam Singh <uttamkhanduja@yahoo.in>
Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-07 21:30:46 +00:00
alex.sharov
ad866187e2 save 2022-02-07 18:33:45 +07:00
Alex Sharov
69ae7dcad9
prepare to add downloader inside erigon (#3441)
* save

* save

* save

* save
2022-02-07 16:54:20 +07:00
Alex Sharov
8fa4d759f8
Snapshots: Add verify command, fix mainnet hashes (#3439) 2022-02-07 12:07:46 +07:00
Alex Sharov
bbf1b5b4bd
Torrent less goroutines (#3423) 2022-02-04 12:42:55 +07:00
ledgerwatch
00c5c7d589
Update preverified hashes for mainnet and ropsten (#3415)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 10:11:59 +00:00
Alex Sharov
15a57373af
remove debug line (#3406) 2022-02-02 15:25:56 +07:00
Alex Sharov
ba35b31f51
Snapshot: fix empty sender (#3405)
* save

* save

* save

* save
2022-02-02 15:24:06 +07:00
Alex Sharov
5567eebb23
Decompressor.WithReadAhead (#3395)
* readahead

* readahead
2022-02-01 11:19:16 +07:00
Alex Sharov
d5cbbf5a65
Fix panic in devel (#3386) 2022-01-31 14:04:16 +07:00
Alex Sharov
94930c5fe0
snapshots: bsc (#3382) 2022-01-31 12:39:56 +07:00
Alex Sharov
7b77585195
Downloader: supress some errors (#3381) 2022-01-31 12:28:59 +07:00
Alex Sharov
a46a15a5e5
goerli snapshot hashes (#3365) 2022-01-27 23:19:41 +07:00
Alexey Sharp
91f694ab3b Add ropsten preverified hashes 2022-01-27 11:18:17 +00:00
Giulio rebuffo
9ca0a721af
Cumulative index stage (#3314)
* prog

* chain traffic

* added log estimation

* fixed err

* unwind
2022-01-27 10:49:03 +00:00
Alexey Sharp
8bf28dec45 Update skip_analysis and preverified hashes for mainnet 2022-01-27 10:29:47 +00:00
ledgerwatch
46a69c1f89
Fix sending transactions via txpool (#3350)
* Fix sending transactions via txpool

* Update to latest erigon-lib

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-26 16:44:21 +00:00
Alex Sharov
bb1d19060f
Add downloader to docker-compose (#3348) 2022-01-26 22:26:33 +07:00
Giulio rebuffo
a75c625d3e
Watchtheburn flag for enabling watch the burn stage (#3325)
* now watch-the-burn needs to be enabled

* refine

* ops

* readme.md

* ops
2022-01-25 17:36:21 +00:00
ledgerwatch
0bb6478d3d
Add blockGasLimit to the state stream (#3321)
* Add blockGasLimit to the state stream

* Send block gas limit

* Switch to new erigon-lib

* Update

* Update txpool - safe operations

* Update erigon-lib

* Txpool working

* Update

* Update

* Update

* Fix lint

* fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-25 10:49:28 +00:00
ledgerwatch
079a177614
[erigon2] Computing data dependencies between transactions (#3290)
* Use ibs per transaction instead of per block

* Remove copy functions, introduce balance increase map

* Fix test

* Fix lint

* Introduce balanceIncrease journal action

* Introduce balanceIncrease journal action

* Simplifications

* Fix

* Reset

* Simplify

* Fix

* Fixes

* Fix for reading nil accounts

* Fix edge cases with reverts

* Clean up test output

* Fix transfers

* Handle special case of ripemd

* Try with fixed compress, only commit every commfreq blocks

* Updates

* Updates

* Update to fixed erigon-lib

* Update to erigon-lib

* Remove stop

* Update to latest erigon-lib

* fixup

* Cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-24 23:12:25 +00:00
Alex Sharov
9e8543d948
Snapshots: fill kv.HeaderNumber table (#3328) 2022-01-24 13:47:05 +07:00
Alex Sharov
2a3f87b5c8
Mdbx page size (#3323) 2022-01-22 12:53:25 +07:00
Alex Sharov
304b40dd0a
Snapshot: prohibit to change cli flags (#3311) 2022-01-22 11:18:24 +07:00
Enrique Jose Avila Asapche
ffedd6ac3f
Enabled proposer by default (#3319) 2022-01-21 22:50:34 +03:00
alex.sharov
2468d61daa mainnet 2022-01-20 17:39:19 +07:00
Somu Bhargava
c620b1e7c9
Add Erigon Import feature for Hive integration purpose (#3298) 2022-01-20 07:34:50 +00:00
Alex Sharov
6fd002eb5a
Snapshot: move old blocks to snapshots (non-async version) (#3309)
* save

* save
2022-01-20 12:01:02 +07:00
Alex Sharov
70607a849f
DumpBlocks method (#3308)
* save

* save

* save

* save

* save

* save

* save

* save
2022-01-20 10:44:45 +07:00
ledgerwatch
6704c48ed7
Inverted lowestNum and highestNum in header downloader (#3301)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-19 16:25:53 +00:00
ledgerwatch
8287b3eda1
skip analysis and preverified hashes update (#3300)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-19 16:24:46 +00:00
Alex Sharov
cfff241d1d
Snapshots: prepare to re-use some funcs (#3296)
* save

* save
2022-01-19 13:30:32 +07:00
Alex Sharov
1e3228124a
Move cli root context to erigon-lib (#3294)
* save

* save

* save
2022-01-19 10:49:07 +07:00
alex.sharov
9303b28ebc mainnet 2022-01-19 10:02:32 +07:00
Alex Sharov
9188705849
Switch to parallel (#3285)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-01-18 15:01:36 +07:00
alex.sharov
032b7e2dec goerli new hashes 2022-01-17 19:14:45 +07:00
Alex Sharov
f9983edc76
Snapshots: ParallelCompressor class, DecompressedFile class (#3265) 2022-01-17 16:57:57 +07:00
Alex Sharov
359af9e49d
Snapshots: files list (#3264) 2022-01-15 11:23:37 +07:00
Dmitry Savonin
a49d409457
Full BSC support with validator mode (#3233)
* migrated consensus and chain config files for bsc support

* migrated more files from bsc

* fixed consensus crashing

* updated erigon lib for parlia snapshot prefix

* added staticpeers for bsc

* [+] added system contracts
[*] fixed bug with loading snapshot
[+] enabled gas bailout
[+] added fix to prevent syncing more than 1000 headers (for testing only)
[*] fixed bug with crashing sender recover sometimes

* migrated system contract calls

* [*] fixed bug with returning mutable balance object
[+] migrated lightclient contracts from bsc
[*] fixed parlia consensus config param

* [*] fixed tendermint deps

* [+] added some logs

* [+] enabled bsc forks
[*] fixed syscalls from coinbase
[*] more logging

* Fix call sys contract gas calculation

* [*] fixed executing system transactions

* [*] enabled receipt hash, gas and bloom filter checks

* [-] removed some logging scripts
[*] set header checkpoint to 10 million blocks (for testing forks)

* [*] fixed bug with commiting dirty inter block state state after system transaction execution
[-] removed some extra logs and comments

* [+] added chapel and rialto testnet support

* [*] fixed chapel allocs

* [-] removed 6 mil block limit for headers sync

* Fix hardforks on chapel and other testnets

* [*] fixed header sync issue after merge

* [*] tiny code cleanup

* [-] removed some comments

* [*] increased mdbx map size to 4 TB

* [*] increased max chaindata size to 6 tb

* [*] bring more compatibility with origin erigon and some code cleanup

* [+] added support of validator mode for BSC chain

* [*] enable private key load for bsc, rialto and chapel chains

* [*] fixed running BSC validator node

* Fix the branch list

* [*] tiny fixes for linter

* [*] formatted imports for core and parlia packages

* [*] fixed import rules in other files

* Revert "[*] formatted imports for core and parlia packages"

This reverts commit c764b58b34fedc2b14d69458583ba0dad114f227.

* [*] changed import rules in more packages

* [*] fixed type mismatch in hack command

* [*] fixed crash on new epoch, enabled bootstrap flags

* [*] fixed linter errors

* [*] fixed missing err check for syscalls

* [*] now BSC implementation is fully compatible with erigon original sources

* Revert "Add chain config and CLI changes for Binance Smart Chain support (#3131)"

This reverts commit 3d048b7f1a.

* Revert "Add Parlia consensus engine for Binance Smart Chain support (#3086)"

This reverts commit ee99f17fbe.

* [*] fixed several issues after merge

* [*] fixed integration compilation

* Revert "Fix the branch list"

This reverts commit 8150ca57e5f2707a84a9f6a1c5b809b7cc84547b.

* [-] removed receipt repair migration

* [*] fixed parlia fork numbers output

* [*] bring more devel compatibility, fixed bsc address list for access list calculation

* [*] fixed bug with commiting state transition for bad blocks in BSC

* [*] fixed bsc changes apply for integration command and updated config print for parlia

* [*] fixed bug with applying bsc forks for chapel and rialto testnet chains
[*] let's use finalize and assemble for mining to  let consensus know for what it's finalizing block

* Fix compilation errors in hack.go

* Fix lint

* reset changes in erigon-snapshots to devel

* Remove unrelated changes

* Fix embed

* Remove more unrelated changes

* Remove more unrelated changes

* Restore clique and aura miner config

* Refactor interfaces not to use slice pointers

* Refactor parlia functions to return tx and receipt instead of dealing with slices

* Fix for header panic

* Fix lint, restore system contract addresses

* Remove more unrelated changes, unify GatherForks

Co-authored-by: Dmitry Ivanov <convexman18@gmail.com>
Co-authored-by: j75689 <j75689@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 19:06:35 +00:00
ledgerwatch
ea6dbd49e8
[erigon2] Running with commitment (#3203)
* [erigon2] Running with commitment

* [erigon2] Running with commitment

* [erigon2] Running with commitment

* go mod tidy

* [erigon2] Running with commitment

* More

* Debug

* fix

* Fix

* state root command

* More fixes

* Fix

* Progress to 164735

* Only trace when failing

* fix for firstInsert

* Over block 1.36m

* Update

* fix to deleteAccount

* Fixes for plainKeys

* Undo printing

* No changeset generation by default

* Print block number on interrupt, fix lint

* Fix lint

* Open history DB as read only

* Print error

* Open non read only

* Readonly again

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 16:02:44 +00:00
Alex Sharov
b86f883f73
Snapshots: watch the burn stage #3259 2022-01-14 14:55:31 +07:00
ledgerwatch
c0db9295b8
Skip analysis and preverified hashe update (#3244)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-12 10:36:12 +00:00
Alex Sharov
5b07f7c09f
Decompressor: fast Count() method (#3225) 2022-01-09 17:32:55 +07:00
Alex Sharov
d5fab965f4
pprof flag enable (#3224) 2022-01-09 15:30:26 +07:00
Alex Sharov
2bcc40fd3f
Snapshots: create .dat in tmpdir (#3222) 2022-01-09 14:43:58 +07:00
Alex Sharov
b4ab063be8
Snapshot: indexing to print logs with clear progress (#3221) 2022-01-09 12:06:00 +07:00
Andrew Ashikhmin
c2d0f0082a
Report Valid ExecutionStatus only after all stages (#3212)
* Stricter transition check

* EthBackendServer.numberSent was unused

* Take account of PoS blocks in TopSeenHeight

* Report Valid ExecutionStatus only after all stages, not straight in headers

* Print TTD on startup

* No need to check parent.Hash() again

* core: change baseFee into baseFeePerGas in genesis json

* fix typo

Co-authored-by: Martin Holst Swende <martin@swende.se>
2022-01-07 16:54:26 +01:00
Alex Sharov
4ef541f5dc
Snapshots: tx lookup in RPC from snapshots (#3214) 2022-01-07 20:52:38 +07:00
Alex Sharov
13d9e71ac7
Snapshots: txnHash2BlockNum idx (#3213) 2022-01-06 18:22:59 +07:00
Alex Sharov
eed7475af0
Snapshot: move parallel compression to erigon-lib (#3211) 2022-01-06 14:13:09 +07:00
Alex Sharov
1b856b5237
Snapshot: mainnet hashes and fixes (#3210) 2022-01-06 11:33:01 +07:00
alex.sharov
1ae4c72af8 mainnet hashes 2022-01-06 09:33:26 +07:00
Alex Sharov
65188ce440
Snapshot: add hash first byte to headers.seg, serve p2p blocks from snapshots (#3198) 2022-01-05 17:14:37 +07:00
Andrew Ashikhmin
d14c223881
Simplify miner gas limit post EIP-1559. Default to 30M (#3205) 2022-01-05 16:36:24 +07:00
Giulio rebuffo
195eb9abaa
Added mining for POS (#3187)
* added comunication with channels

* added final mining

* removed trash

* bug fixing

* async calls

* one thread

* tests

* better comments

* no lock

* better placing of skipCycleHack

* removed long timer

* moved transitioned block processing

* better naming

* disabled updates on blocks

* sync.Cond

* 2 sync.Cond

* better locking

* Use single sync.Cond. Proposer shutdown

Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
2022-01-04 18:37:36 +01:00
Alex Sharov
156287a982
Sanity check p2p block (#3197) 2022-01-04 15:46:22 +07:00
Alex Sharov
1bfc2ffde4
Snapshots: add --to parameter to "erigon snapshots create" (#3194)
* save

* save
2022-01-03 11:39:20 +07:00
Alex Sharov
6904e4c3c4
Snapshot: describe idx schema (#3191) 2021-12-31 20:42:56 +07:00
alex.sharov
e29be219a7 improve downloader readme 2021-12-31 11:53:15 +07:00
Giulio rebuffo
864b744ed2
Support for syncing on mergemock (#3174)
* block proposing

* standard finalized

* mergemock execution

* private chain can now be ran yay

* perfectioned

* polished

* more polishing

* resize PR

* resize PR

* resize PR

* simplifications

* fixed tests

* better syncronous communication

* better syncronous once again

* clean

* Re-enabled headers verification

* mining finish

* mining finish

* cleaned hash computation

* fixed evm bug

* go.mod

* Update .gitignore

* Update .gitignore

* Update .gitignore

* removed new line from .gitignore

* added go.mod and go.sum

* feeRecipient into preset

* useExternal

* todo

* fixed comment for forkchoiceUpdateV1

* smaller

* smaller

* Revert changes to miner frequency

* Restore useExternalTx

* Fix headerLoadFunc

* do not reset payloadId

* rename

* extra is empty

Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
2021-12-29 17:25:13 +01:00
bgelb
2d9fe6cd9a
fix debug_storageRangeAt (#3148) 2021-12-29 10:36:41 +07:00
alex.sharov
1fb53eaf39 remove some unused flags 2021-12-28 10:37:37 +07:00
Alex Sharov
5cac9fbff5
Move snapshot creation from hack.go to "erigon snapshot create" cli sub-commmand (#3177) 2021-12-27 14:59:29 +07:00
Andrew Ashikhmin
445ac05220
Verify PoS headers before inserting them into DB (#3151)
* Remove TotalDifficulty stage

* Introduce FeedHeaderFuncType

* Some TODOs

* Don't gossip PoS headers

* FeedHeaderPoS

* Call header.Hash once

* Use FeedHeaderPoS for to write PoS header

* Support errors in EngineExecutePayloadV1

* useExternalTx boilerplate

* Call VerifyHeader on PoS header

* latestValidHash must be nil when status is SYNCING

* fixCanonicalChain instead of canonicalHeadersCollector

* engine.VerifyHeader + FeedHeaderPoS as headerLoadFunc

* fix daemon

Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
2021-12-23 15:06:10 +01:00