Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
Moves bodies request logic to pre OOM changes. No logging or checking
for timeouts and moving forwards with every request made. Tested on a
local node that went through the DB migration on this branch and worked
fine, caught back up to the tip. Tested on POW initial sync and looked
to be working fine writing bodies, although didn't have time to see this
run all the way through the merge.
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)
so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Test GnosisGenesisStateRoot
* Delete obsolete allocations
* SysCallContract shouldn't increase nonce of SystemAddress
* Max gas limit in SysCallContract
* Restore error swallowing for Bor
* Fail newPayload if execution is unsuccessful
* fix linter
* fix message
* no self-hosted
* try force ci
* back to unbuntu
* Remove unnecessary unwinds
* Fix canonicalising
* Quiet logs for in-memory exec
* Remove experimental overlay flag
* Fix sync test
* Fix test
* Fix lint
* Print hash in the end of the cycle
* Not notify headers if list is empty
* Supress timings and tables logs when no change in progress
* Reduce logging
* fix
* better log for pos download
* Better newPayload and forkChoice messages
* Fix lint
* simplify
* No duplication of download messages
* Reverse log for extension
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
* OOM fix for bodies stage
change misleading bodies stage log messages
remove duplicate func to create key from block number
ensure body isn't nil when reading from bucket in bodies stage
remove unused blocknum variable from body requests
moving to using RLP encoding for stage bodies storage
encode bodies to RLP for stage bodies storage
better error handling in stage bodies
fixup body algos after rebase
* fixes for body stage oom after rebase
* Fixes for trace_block
* Print
* Print
* extra bodies
* extra bodies
* Add canonical check
* First fix for bodies mismatch
* More cleanup
* Advance progress
* Reset snapshots in integration
* Cleanup
* Way to reset snapshots stage
* Not reset sequence if not needed
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* added a way to get latest executed block post POS
* added erigon_ExecutedBlockNumber into readme
* optional rpc.BlockNumber
* better message
* updated readme
* added snapshots into stages list && deleted snapshots from defaul unwind
* added verkle trie
* using the same sentries client hd
* need to safe the stage progress of headers as well
* Test case that reproduces wrong trie root in Hive
* Do not hardcode roots
* Fix IH when state contains addresses smaller than the first key in AccTrie
* printing
* logging
* spam off
* ops
* clean up
* deleted all bor related stuff in snapshots
* adding bor tx to BorTxLookUp
* added readBorTxLookUp
* getting bor tx from bor lookup
* comments
* deleted bor related stuff
* deleting bor tx
* writing borTxLookUp in stage_txlookup
* using bytes
* little fixes
* updated erigon-lib
* using biEndian
* clean up
* setting bytes for big int
* log
* logging
* more logs
* delete logs
* added tree key functions for verkle tries
* added tree key functions for verkle tries
* added tree key functions for verkle tries
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
* General cleanup and fixes
* Cleaning part 2
* cleanup part 3
* Refactored staterunner to have a handler function
* Fixed linting
* Cleanup part 4
* Changed the initialization of the require package and updated .golangci.yml
* Fixed lint errors from golangci-lint
* deleted state_recon_1.go
* fixed build error
* Added ReadHeaderTimeouts for httpServer
* made changes to golangci
* Reset golangci.yml
* getBlockTransactionCountByHash using snapshots
* got body from block reader
* requesting block body with transactions
* tests
* added tx amount into blocreader body
* using txAmount from body
* README: Move port 8551 from RPC to erigon ports
* Some renaming for consistency
* Remove authrpc.* flags from rpcdaemon
* docker-compose: move --authrpc.jwtsecret to erigon
* minor typo
* Update README re. Engine API
* Remove obsolete code
* Don't apply --rpc.accessList to Engine API listener
* Simplify startAuthenticatedRpcServer
* Rename engine.* cmd flags to authrpc.* for consistency with geth
* More renamings
* Introduce --authrpc.vhosts flag
* added flag
* working flag ovewrite for http
* setting up all flags with yaml file
* comment
* ops
* set flags in the cli trump yaml config file
* changed to config.yaml.file
* ops
* better naming
* accepting toml or yaml files
* fixed flag
* fixed messages
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* if PoS download is fast enough, determine VALID or INVALID
* more acceptable absolute value
* solved comments
* remove useless
* moved check in stage_headers
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
* Consensus tests update 11
* Add GrayGlacier fork
* Wire currentRandom from tests into PREVRANDAO
* Serenity engine
* Support insertion of PoS blocks in MockSentry
* Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()
* Revert "Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()"
This reverts commit 903fca572be03c7de33318ce177a03a4be34927b.
* Post-merge fix
* Don't wait for Beacon Chain in tests
* Skip powToPosBlockRejection transition test
* ForkChoice in insertPoSBlocks
* Add withPosDownloader arg to MockWithEverything in order to fix TestPoSDownloader
* avoid constantly triggering stageloop when using Engine API
* fix lint + test
* fixed comments
* ops
* little fixes here and there
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
* added proper cleanup when we get notified of new height
* added extra cleanup
* removed bad if condition
* fixed hive tests
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
* added merge range into segments
* got rid of missing snapshot errors
* reusing RequestSnapshotDownload
* sleep out of download
* ops
* warning if we are missing snapshots
* refactored request download
* keeping track of missing snapshots
* using slice mergeRange
* request snapshots on reopen
* passing arguments
* passed in var
* Revert "passed in var"
This reverts commit 90478978dfa9f2a6dd5b1b051fc1d3f9e5f7a9c5.
* Revert "passing arguments"
This reverts commit 1e39c4152003796f6ff0bcfc188512d4a43bd18d.
* Revert "request snapshots on reopen"
This reverts commit d40212b973bc15db2b25cc1b0abb22051a3debb1.
* added downloadRequest
;
* downloading missing headers at start up
* there shouldnt be an error anymore
* not using nil;
;
* Add borTx to GetBlockByHash; ensure borTxs have hashes; don't try to derive sender for borTxs
* Surface borReceipt logs in eth_getLogs
* Check for existence of borReceipt before synthesizing a borTx
* Add eth/67
* Listen to eth/66 on a separate port
* Fix compilation error
* Fix cfg66.ListenAddr
* Update erigon ports in README
* Expose port 30304 in docker
* P2pProtocolVersionFlag instead of second sentry
* Remove "66 by default" from usage
* Small comment
* evm t8n tool to use ExecuteBlockEphemerally api (#4512)
* fix to set V, R, S in legacy transaction
* fix to dump post-execution alloc for evm t8n
* close tx in evm t8n
* populate current difficulty and gas used in output result
- update the ExecutionResult to include corresponding info (like
Difficulty/GasUsed)
* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally
* using ExecutionResult in ExecuteBlockEphemerally
* bypass validations and integrate with EphemeralExecResult
* fixing output of 'evm t8n'
- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
alloc.txt (for testdata=1)
* get ExecuteBlockEphemerally to accept getTracer lambda
* fix build failure
* test cases for evm t8n
* more test cases for evm t8n
* fix stateRoot computation in evm t8n
* remove reward argument, as EBE itself takes care of it
* final cleanups for migration to using ExecuteBlockEphemerally
* change EBEforBSC to match EBE
* fix linter issues
* manually revert an unwanted diff
* avoid calculating ReceiptHash twice
* linter check
* minor correction
* remove unnecessary logic in EBEforBsc
* fix integration tests
* fix build
* Ensure fake Bor txs + receipts are returned from all relevant RPC methods
* Add rest of bor implementation for eth_getBlockByNumber
* Use TxLookup index to find Bor txs
* Fix txHash on emitted borTxs and borReceipts
* Fix checks given that borTxs get registered in TxLookup; remove useless ref indirections
* Small optimisation for eth_getBlockByNumber
* Option to not retrieve transactions
* fixes
* Check hash
* Fixes
* Avoid shadowing of err in BlockWithSenders
* Fix test
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)"
This reverts commit db93d2ea37.
* Fix compilation
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* reconstitution
* Add history access without state function
* More on state reconstitution
* More on state recon
* More on state recon
* More
* More
* support dao fork
* More on state reconstitution
* Update to erigon-lib
* More
* Added genesis block and filling with history
* update
* Genesis works
* Start on parallel
* Preparation for parallel reconstitution, stats for EfSearch
* continue with parallel work
* Fix history reader
* Remove time measurements
* Fixes
* Fixes and UX improvements
* Fixes
* More tracing
* More fixes
* More fixes
* Fix code size
* Update to latest erigon-lib
* Fix for dao fork
* Remove hacks
* Update to erigon-lib, fix lint
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* fix to set V, R, S in legacy transaction
* fix to dump post-execution alloc for evm t8n
* close tx in evm t8n
* populate current difficulty and gas used in output result
- update the ExecutionResult to include corresponding info (like
Difficulty/GasUsed)
* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally
* using ExecutionResult in ExecuteBlockEphemerally
* bypass validations and integrate with EphemeralExecResult
* fixing output of 'evm t8n'
- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
alloc.txt (for testdata=1)
* get ExecuteBlockEphemerally to accept getTracer lambda
* fix build failure
* test cases for evm t8n
* more test cases for evm t8n
* fix stateRoot computation in evm t8n
* remove reward argument, as EBE itself takes care of it
* final cleanups for migration to using ExecuteBlockEphemerally
* change EBEforBSC to match EBE
* fix linter issues
* manually revert an unwanted diff
* avoid calculating ReceiptHash twice
* linter check
* minor correction
* remove unnecessary logic in EBEforBsc
* pendingPayloadStatus -> pendingPayloadHash
* Ensure that Engine API response is sent after tx.Commit() even when useExternalTx
* ProcessEngineApiResponse -> SendEngineApiResponse
* PayloadResponse -> PayloadStatus
* Remove an unused interrupt value
* Remove sendErrResponse from safeAndFinalizedBlocksAreCanonical
* Clean up err handling
* startHandlingForkChoice returns response
* handleNewPayload returns response
* Commit transaction before responding on Engine API
* small fix
* Reply with SYNCING before long unwind
* schedulePoSDownload doesn't require requestStatus anymore
* No reorgs when fork choice head points to a canonical header
* Return immediately
* Fix sentry
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Fix a typo
* BlockBuilder dummy
* BlockProposerParametersPOS -> BlockBuilderParameters
* Pass tx to MiningStep
* BlockBuilderFunc
* Interrupt in MiningExec Stage
* Draft implementation of BlockBuilder
* Fail back to empty header
* Add a comment
* cosmetic change
* Cosmetic change again
* It's not safe to pass transactions between goroutines
* save
* save
* save
* save
* save
* save
* Update reset_blocks.go
* Not to remove too many tx lookup files
* Fix truncate blocks and add reset txlookup
* Fix bodies
* Fix nil pointer
Co-authored-by: ledgerwatch <akhounov@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* For for overwritten anchor
* Switch to time types, log retry time in diagnostics
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
