Commit Graph

3349 Commits

Author SHA1 Message Date
alex.sharov
d488482c89 save 2022-02-13 08:53:10 +07:00
ledgerwatch
ef6013faff
Automation for gathering erigon2 history data (#3501)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 22:16:06 +00:00
Alex Sharov
1ba95c1087
atomic close 2 channels (#3498) 2022-02-12 17:09:17 +00:00
ledgerwatch
2074b98568
Tools for gathering data on erigon2 history and bitmaps (#3488)
* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 15:48:38 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
Alex Sharov
479c7f3d9d
Embed rpcdaemon: prepared direct clients (#3492) 2022-02-12 19:47:19 +07:00
leonardchinonso
aca755ab43
Fix/rpcdaemon ws upgrade (#3490)
* Added a method to `cmd/rpcdaemon/cli/config.go` to check header of incoming http request for a ws upgrade request

* Added the testing of the 'GET /' request for ws to the devnet tool

* Fixed lint errors
2022-02-12 12:40:19 +01:00
Alex Sharov
8d7eca012a
user-friendly pagesize flag (#3484) 2022-02-12 15:23:06 +07:00
ledgerwatch
eb6a168603
[erigon2] Chain history and bitmap indices (#3437)
* First update

* Update to latest erigon-lib

* Fix lint

* Update

* Update

* Update

* Add history2 command

* Updates

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-12 00:06:57 +00:00
Alex Sharov
a1318f869a
make services net-independent (#3481) 2022-02-11 17:11:59 +07:00
alex.sharov
49efd8c051 downloader: fix Ctrl+C 2022-02-11 14:59:51 +07:00
alex.sharov
8068a1bed2 save 2022-02-11 14:57:59 +07:00
Alex Sharov
dcabe33f33
Downloader: don't setup erigon's config if --dowloader.api.addr is set (#3480) 2022-02-11 14:39:28 +07:00
Alex Sharov
f66f937d4d
downloader: print logs on validity check (#3479) 2022-02-11 12:47:29 +07:00
Alex Sharov
3d3315132c
downloader switch back to bolt backend (#3478) 2022-02-11 12:44:18 +07:00
alex.sharov
92fc5ae3de Revert "downloader: enable logs"
This reverts commit db97199d5b.
2022-02-11 11:21:42 +07:00
alex.sharov
db97199d5b downloader: enable logs 2022-02-11 11:17:42 +07:00
leonardchinonso
9dbedb2e1d
Chore/improve devnettest (#3476)
* Increased hardcoded gas price from 50000

* Added functionality to search for last propagated transaction in mined blocks after sending a transaction.

* Added functionality to search for last propagated transaction in mined blocks after sending a transaction.
2022-02-11 10:23:40 +07:00
Alex Sharov
79de194edf
snapshot: less logs (#3467) 2022-02-10 16:16:55 +07:00
Alex Sharov
ccf434f270
GetTransactionByHash: fallback to txpool (#3463)
* GetTransactionByHash: fallback to txpool

* GetTransactionByHash: fallback to txpool
2022-02-10 08:36:24 +00:00
Alex Sharov
2e17b8f515
snapshot: parallel transaction index, recsplit: reset offset collector, etl: faster flush and load (#3465) 2022-02-10 14:40:29 +07:00
Jared Doro
0223489183
[WIP] Adding Support for Sepolia (#3462)
* [WIP] Adding Sepolia

* Fix formatting

* Added Timestamp to Sepolia genesis

* update config

* fixed formating
2022-02-10 07:27:36 +00:00
Alex Sharov
7346f0b56d
Update readme.md 2022-02-10 08:33:05 +07:00
Alex Sharov
e695a6e58b
state and newHead: 1 goroutine per subscriber (#3452) 2022-02-10 08:25:58 +07:00
Andrew Ashikhmin
8466cb1b7b
New Engine API semantics (Kiln v1) (#3340)
* Disable PoS sync temporarily

* Resore PoS sync

* Handle Ctrl^C in HeadersPOS

* Consistent naming

* Extract verifyAndSavePoSHeader & downloadMissingPoSHeaders

* Preparation for EngineForkChoiceUpdated re-orgs

* Extract ForkingPoint

* Comments

* New proto for Engine API

* EngineExecutePayload -> EngineNewPayload

* Return INVALID_BLOCK_HASH if block hash is invalid

* Return EngineStatus_ACCEPTED for side chain blocks

* Update erigon-lib (PR 268)

* Fix payload2Hash

* reverseDownloadCh -> beaconPayloadCh

* Update erigon-lib

* Engine API updated

* ExecutionStatus -> PayloadStatus

* Introduce forkChoiceCh

* Mock ForkChoiceMessage/PayloadStatus

* Add ValidationError to PayloadStatus

* Small clean-ups

* Add INVALID_TERMINAL_BLOCK to EngineStatus

* Add a comment

* Extract handleNewPayload & handleForkChoice

* Partially implement handleForkChoice

* Update erigon-lib

* short vs long re-org

* Move header insertion out of downloadMissingPoSHeaders

* Update erigon-lib

* Refactor ProcessSegmentPOS

* Fix imports

* Wire downloadMissingPoSHeaders into handleForkChoice

* evictOldPendingPayloads

* nolint:unused for assertSegment

* Try nolint instead of nolint:unused

* Small comment improvements

* HeadHeaderHash/StageProgress in handleForkChoice

* TODO: bodyDownloader.AddToPrefetch(block)

* Review call suggestions

* Don't use ReadHeaderNumber in ProcessSegmentPOS

* Don't leave ethbackend waiting when server is stopping

* Update erigon-lib

* More explicit signature of downloadMissingPoSHeaders
2022-02-09 08:33:22 +01:00
Alex Sharov
1d6f8a04ac
Embed downloader (#3450)
* save

* save

* save

* save

* save

* save
2022-02-09 13:22:43 +07:00
alex.sharov
863db147cf bor: ensure db exists 2022-02-09 09:31:12 +07:00
Alex Sharov
e5081c78b9
bor: ensure db exists (#3449) 2022-02-09 09:28:06 +07:00
primal_concrete_sledge
60f4b3a46d
fix/add_4th_parameter_to_parity_list (#3443)
* fix/add_4th_parameter_to_parity_list

* Change blocknum type from interface{} to rpc.BlockNumberOrHash

* Change latest block check
2022-02-08 13:02:18 +00:00
Enrique Jose Avila Asapche
35fcd3a829
Merging Turbo bor into devel (#3372)
* implemented bor consensus

* add bor flags to default

* change bucket into snapshot to clique

* enable stateSync

* bypass reciept checks

* fix receipt calculation and bor logs

* fix: contract call wrt bor

* Update mumbai config

* Add: bor-mainnet flag and config

* Add bor consensus to integration

* use header coinbase in block context

* london fork mumbai changes

* fix genesis error

* Jaipur fork for mumbai

* add sysCall to verifyHeader

* added bor related rpc method implementation

* added bor specific rpc extensions

* fixes in snapshot implementation, major refactor for bor rpc

* modify consensus specific db path for bor

* fix: remove parallel compute for get root hash rpc method

* Added bor-receipt flow

* Use turbo-bor-lib and bor tables

* Use bor table in RPC snapshot

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* update rpc readme

* link rpc docs in readme

* Update Readme

* Update Readme

* move erigon namespace rpc methods to eth

* rm: erigon namespace

* rm: erigon namespace, update list of available rpc methods, add example

* fix: binary name in rpc readme

* fix: max db size

* Add london to bor-mainnet

* updated node.go

* add system req to readme

* golang version fix readme

* added networknames in correct place

* nil

* ran gofmt

* erigon

* fixed fake.go

* dont need turbor-lib

* old readme

* fixing readme

* half

* other half

* changed return

* fixing return

* fixed return

* fixed flags

* gofmt

* merge with devel

* latest erigon-lib

* fixed context.coinbase

* took out syscall

* fixed params in hash

* bor type now is consensus.Engine

* parlia is consensus.Engine

* missing arg and repeated importation

* repeated importation

* fixed eth_receipts.go

* deleted duplicate issuance

* part of consensus.Engine type

* added eth_api issuance

* networkname

* added erigon_system file

* fork struct taken out

* added erigon block

* getLogByHash for erigonImpl

* gofmt

* fixed lint

* ops

* gofmt

* gofmt

* added APIImple functions

* fixed clique test

* took out print

* fixed state added balance

* fixed README

* fixed rpcDaemon README

* fixed integration README

* updated blockchain.go

* lint

* added bor back into blockchain.go

* took out comment

* lint

* updated daemon

* updated wtb

* removed duplicate

* removed VerifyHeaders

* prevent use of wrong Transfer

* fixed state_processor.go

* fixed state_transition.go

* fixed headers

* returning err

* error handling in bor read tx look up

* put for txLookUp

* dealing with error

* lint

* traces

* more traces

* fixed receipt in execution

* getTrasanction receipt for bor or others

* nil

* lint

* ops

* deleted syscall

* took out else

* Merge branch 'devel

* tests syscalls

* changed borReceipt to receipt

* reset header algos

* arguments fix

* took out prefixes

* lint

* erigon-named

* borReceiptKey = blocknumber

* reverts e3b60c2e159d03efcb855f7ab3da5a098dd60c33.

* correct hashing tx

* dont need it here

* lint

* added txlookup for bor

* change to uint256

* outputs for isBor

* wrapper

* added isBor and isParlia

* isBor

* fixed BorTransfer

* not readBody

* correct prefix

* added blockNum

* added readStorageBody

* readStorageBody

* lint

* got rid of unnecessary bor_receipt func

* onlny if bor

* use clone

* append

* writeToSlice

* added isBor flag

* fixed writeToSlice

* normal sorting

* lint

* Reset erigon-snapshots

* Move bor prefix into if

Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Uttam Singh <uttamkhanduja@yahoo.in>
Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-07 21:30:46 +00:00
Alex Sharov
69ae7dcad9
prepare to add downloader inside erigon (#3441)
* save

* save

* save

* save
2022-02-07 16:54:20 +07:00
alex.sharov
c421df7953 don't drop on ctx cancel 2022-02-07 15:23:07 +07:00
alex.sharov
c461eec22e don't drop on ctx cancel 2022-02-07 15:10:55 +07:00
Alex Sharov
4540cdd04e
don't drop on ctx cancel (#3440) 2022-02-07 14:58:46 +07:00
alex.sharov
5f4c284eab remove debug line 2022-02-07 12:13:55 +07:00
Alex Sharov
8fa4d759f8
Snapshots: Add verify command, fix mainnet hashes (#3439) 2022-02-07 12:07:46 +07:00
Alex Sharov
25a94ceb66
Downloader: print banned ips (#3430)
* save

* save

* save

* save

* save
2022-02-05 10:40:27 +07:00
alex.sharov
5a748687f8 save 2022-02-05 08:46:00 +07:00
Alex Sharov
ef668bee47
Open downloader: add % progress to logs (#3428) 2022-02-04 16:52:46 +07:00
Alex Sharov
29a4adfbaa
ParallelCompressor: Remove intermediate ETL collectors (#3427)
* save

* save

* save
2022-02-04 16:48:16 +07:00
Alex Sharov
b8c75e6b0b
save (#3426) 2022-02-04 13:10:21 +07:00
Alex Sharov
01380316a5
close nodes db on server start error (#3424) 2022-02-04 13:04:37 +07:00
Alex Sharov
bbf1b5b4bd
Torrent less goroutines (#3423) 2022-02-04 12:42:55 +07:00
ledgerwatch
093bfbad39
[erigon2] Optimisations (#3412)
* Update

* [erigon2] optimisations

* Update to latest erigon-lib

* Update to latest erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 09:36:27 +00:00
Igor Mandrigin
ff4fa5efd3
fix compatibility of getting tx by number (#3409)
* fix compatibility of getting tx by number

`eth_getTransactionByBlockHashAndIndex` and `eth_getTransactionByBlockNumberAndIndex` should return `null` if the index provided is out of bound (checked with Infura and Cloudflare ETH gateway).

* small fixup (formatting)
2022-02-02 14:25:16 +00:00
Alex Sharov
68f1b52b1f
Sentry: 1 goroutine per peer (for outbound requests) (#3403) 2022-02-02 14:59:00 +07:00
alex.sharov
7e93a08155 Revert "save"
This reverts commit 5ad09bfef3.
2022-02-02 09:58:34 +07:00
alex.sharov
5ad09bfef3 save 2022-02-02 09:58:16 +07:00
Alex Sharov
1214831c6e
downloader reduce amount of trackers to 10 (#3394) 2022-02-01 10:41:10 +07:00
Alex Sharov
0f44a545e6
reduce client limit (#3393) 2022-02-01 10:06:31 +07:00