Commit Graph

3570 Commits

Author SHA1 Message Date
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os (#3946) 2022-04-23 15:43:00 +01:00
ledgerwatch
961f4f5888
Introduce unlimited download rate (#3945)
* Introduce unlimited download rate

* More generous burst

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-22 17:59:49 +01:00
Alex Sharov
62c377521b
Torrent conns print (#3943)
* save

* save
2022-04-22 18:24:21 +07:00
battlmonstr
92db317e06
Observer - P2P network crawler (#3928)
Observer crawls the Ethereum network and collects information about the nodes.
2022-04-22 12:02:12 +01:00
Alex Sharov
7a226ef94c
Downloader: re-use flags defaults (#3941) 2022-04-22 17:30:48 +07:00
Krishna Upadhyaya
4a8eb34d1e
Use heimdall url in integration bor consensus (#3940) 2022-04-22 15:41:49 +07:00
Giulio rebuffo
c5b1836481
Added Ethstats service (#3931)
* somewhat there but not yet

* lol

* more efficient ethstats

* lint

* not die on no wifi
2022-04-22 07:23:52 +01:00
Chase Wright
dc5125eec5
Fix some cli flag descriptions (#3933)
* Fix some cli flag descriptions

* add node about verbosity
2022-04-22 09:26:59 +07:00
Alex Sharov
1d1282f3c0
Update readme.md 2022-04-21 14:42:07 +07:00
Alex Sharov
9051d95836
Downloader tables, mdbx-based torrent pieces completion store (#3923)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-04-21 10:34:31 +07:00
Alex Sharov
6aa4999729
Parallel indexing (#3921) 2022-04-21 10:34:10 +07:00
Alex Sharov
89c3049244
limit amount of sentries in broadcasts (#3920)
* save

* save
2022-04-20 22:00:58 +07:00
Alex Sharov
8d81c3b0f8
Rwdir fix2 (#3915) 2022-04-19 08:55:46 +07:00
ledgerwatch
7080c6fab9
Fix help and README.md about pruning options (#3914)
* Update README.md

* Update flags.go

* Update README.md
2022-04-18 21:50:21 +01:00
Alex Sharov
9305efeaa7
Snapshots: use kv.ReadAhead helper (#3909) 2022-04-18 10:48:09 +07:00
Alex Sharov
0a5ce152a4
save (#3902) 2022-04-17 09:22:00 +01:00
Alex Sharov
f2a9763a6d
integration: senders check that snapshots are not nil #3905 2022-04-17 09:52:05 +07:00
Alex Sharov
9d2455318a
do panic - to exit with correct code #3886 2022-04-15 08:30:24 +07:00
Andrew Ashikhmin
7b8d828aa1
RPC: return latest available block rather than forkchoice head (#3888) 2022-04-14 18:05:59 +02:00
ledgerwatch
156da607e7
More compact representation of huffman trees in the seg files (#3875)
* More compact representation of huffman trees in the seg files

* Update to latest erigon-lib

* recompress

* recompress

* recompress

* uncompress: buf fix

* Update to latest erigon-lib

* Revert default sync mode

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-13 16:29:44 +01:00
Alex Sharov
652825b629
recompress (#3880) 2022-04-13 13:00:05 +01:00
alex.sharov
1501b47a45 recompress 2022-04-12 22:22:13 +07:00
alex.sharov
bba0c686e9 recompress 2022-04-12 15:03:44 +07:00
alex.sharov
0b63264531 recompress 2022-04-12 15:01:25 +07:00
alex.sharov
35674e81c0 recompress 2022-04-12 14:51:45 +07:00
Alex Sharov
ff44db147e
Torrent: Increase peers amount to improve Swarm-ing, can do it now because have ratelimiter #3877 2022-04-12 09:09:09 +07:00
Alex Sharov
4be4e5b99a
Torrent: enable rate limiter (#3872)
* save

* save

* save

* save
2022-04-11 18:45:28 +07:00
Alex Sharov
b329d675ec
Torrent best trackers (#3870)
* best trackers

* save
2022-04-11 14:01:12 +07:00
Alex Sharov
54ec6df6e1
ErigonLib: move types to own package (#3865) 2022-04-11 10:05:07 +07:00
ledgerwatch
f56d4c5881
Switch peerId from 256 to 512 bit (as in stable) (#3862)
* Switch peerId from 256 to 512 bit (as in stable)

* go mod tidy

* Fix some tests

* Fixed

* Fixes

* Fix tests

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-10 08:01:25 +01:00
Alex Sharov
9b47daee44
Default syncmode=snap for Mainnet, Bsc, Goerli (#3861)
* change syncmode default to snap sync

* save
2022-04-10 07:20:20 +01:00
alex.sharov
b921813c7d snapshots: recompress by 2 erigon binaries 2022-04-08 11:02:44 +07:00
Andrew Ashikhmin
4531461d66
ExchangeTransitionConfiguration: allow non-zero TerminalBlockNumber (#3846) 2022-04-08 10:27:56 +07:00
alex.sharov
44a217892b test of getPayload 2022-04-08 08:47:40 +07:00
Andrew Ashikhmin
c2f1a6082f
Fix runPeer lint (#3847) 2022-04-07 13:47:28 +02:00
Alex Sharov
b6334870e8
Snapshots: recompress.sh (#3845)
* save

* save
2022-04-07 13:26:14 +07:00
Giulio rebuffo
396bd03021
Fixed JWT path when running the rpcdaemon (#3840) 2022-04-07 08:53:34 +07:00
Alex Sharov
bc368098c8
Torrent: more cli params (#3838) 2022-04-06 21:06:58 +07:00
Alex Sharov
3d52a22f9f
Torrent: uses our logger (#3837) 2022-04-06 18:33:48 +07:00
alex.sharov
fee018ae1e trackerslist up 2022-04-06 16:19:59 +07:00
Alex Sharov
2efc723dbd
Torrent: enable ws (#3836) 2022-04-06 16:15:37 +07:00
Alex Sharov
6f7a9dbace
Snapshots: nat support (#3834) 2022-04-06 15:25:32 +07:00
Alex Sharov
80bd44fce5
Snapsthos: build indices on retire blocks (#3823)
* grpc up

* grpc up

* grpc up
2022-04-05 16:22:11 +07:00
TBC Dev
bf06095811
hack: Allow blockTotal <= 0 as offset from head (#3818) 2022-04-05 08:07:07 +07:00
Alex Sharov
3030526afb
Delete blocks on integration stage_header --unwind (#3813)
* fix for non-can block

* fix for non-can block
2022-04-04 14:12:14 +07:00
alex.sharov
c4879dd4b8 snapshots: auto fix 2022-04-02 13:20:26 +07:00
alex.sharov
10f195578b fix linter 2022-04-01 15:11:02 +07:00
Alex Sharov
d1218181b0
Snapshots: geth compatibility, use --syncmode=snap flag (#3808) 2022-04-01 15:00:50 +07:00
Alex Sharov
8b03582558
compressor: log lvl #408
* save

* save
2022-04-01 10:44:30 +07:00
Andrew Ashikhmin
6ff6c39d13
Prefer forkchoice head as latest block in RPC (#3806)
* LastForkchoice accessors

* Update erigon-lib

* Prefer forkchoice head as latest block in RPC

* Fix typo
2022-04-01 08:33:50 +07:00
leonardchinonso
fbb62c7a35
Chore/test log subscription (#3786)
* saving

* Implemented and tested subscription of logs

* Fixed lint errors

* fixed compilation error

* Removed print statements across code

* made review changes

* Validated hex addresses
2022-03-31 13:40:09 +01:00
battlmonstr
930d662f21
Refactor crypto.FromECDSAPub to MarshalPubkey/Std (#3797)
Most places that used this method were cutting off the 1st byte.
Refactor this idea to a common place.

* better naming: MarshalPubkey matches existing UnmarshalPubkey
* "Std" suffix for the ANSI standard encoding without cut off
* docs
2022-03-31 11:06:20 +07:00
Alex Sharov
b7afdf4a83
Single root ctx (#3796) 2022-03-30 16:51:43 +07:00
Alex Sharov
27f1bdbd51
cloud: commit (#3793) 2022-03-30 09:55:22 +07:00
Alex Sharov
2e49137a23
cloud: commit (#3792) 2022-03-30 09:54:07 +07:00
Igor Mandrigin
7f1771516c
Fix 'all defaults' case for eth_estimateGas (#3790)
* Fix 'all defaults' case for eth_estimateGas

* fix tests

Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-03-30 08:04:51 +07:00
Andrew Ashikhmin
816ad30eff
Small Merge-related clarifications & improvements (#3785)
* Hd.Fetching -> FetchingNew

* ReadHeadHeaderHash makes more sense in verifyAndSaveNewPoSHeader

* Potentially more correct LatestValidHash in case of payload with invalid transactions
2022-03-29 11:47:07 +02:00
Alex Sharov
477d04f669
Integration: update StageFinish too #3782 2022-03-29 13:45:53 +07:00
Alex Sharov
f5a8230016
Snapshots: write hashes to file only if amount of them growth (#3784) 2022-03-28 14:44:11 +07:00
Alex Sharov
89d4477df3
Snapshots: nil indices on p2p fix (#3772) 2022-03-27 10:07:58 +07:00
ledgerwatch
9ea6398524
Fixes to subscribe logs (#3769)
* Fixes to subscribe logs

* Add criteria to logs subscription

* Skeleton of RPC daemon event log distribution

* Simplify

* Send aggregated filter to Erigon

* Change API

* Print

* Fixes

* Fix topics filtering

* Fill txHash and blockHash

* Timing logs, fill tx index

* Print

* More print

* Print

* Asynchronous sending of log events to RPC daemon

* Remove prints

* Only extract logs if there are subscribers

* Check empty when RPC daemon is removed

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-26 18:21:31 +00:00
Marin Ivanov
1075bcacae
rpcdaemon: Tidy up eth_getLogs (#3765) 2022-03-25 11:17:23 +07:00
Alex Sharov
4da85fe94e
integration_unwind_sender (#3767) 2022-03-25 11:00:52 +07:00
Alex Sharov
8754415359
fix integration unwind nil pointer (#3763) 2022-03-24 15:13:56 +07:00
Alex Sharov
a21f7561ab
Snapshots: gen .torrent file only for big segments #3761 (#3761) 2022-03-24 12:26:22 +07:00
Alex Sharov
85e47d83d2
Snapshots: remove "experimental" prefix from cli flag (#3760) 2022-03-24 12:17:31 +07:00
Alex Sharov
a357421719
Snapshots: notify rpcdaemon about new snapshot (#3755) 2022-03-24 09:25:38 +07:00
Giulio rebuffo
f4bf94f78a
Fixed Erigon README for Authentication API and jwt.hex generates in Datadir (#3757)
* added datadir jwt

* fixed readme
2022-03-24 09:10:29 +07:00
Giulio rebuffo
904674e1a1
added Websocket support and authentication to Engine API (#3752)
* added ws support and auth

* fixed lint
2022-03-23 17:12:19 +01:00
ledgerwatch
f8668da905
[erigon2] Bring back MDBX as storage for recent data (#3756)
* fixes

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-23 16:00:06 +00:00
leonardchinonso
ae1655caef
Feature/automate devnet tool nonce (#3754)
* Added eth_getTransactionCount to the devnet tool

* Fixed lint errors

* Fixed lint

* Fixed compilation error
2022-03-23 14:26:33 +00:00
Giulio rebuffo
b7fb85df6a
demoted logs (#3753) 2022-03-23 10:28:32 +01:00
primal_concrete_sledge
fed19d5591
feat/rpcadaemon_logs_sub (#3751) 2022-03-23 07:35:04 +00:00
Andrew Ashikhmin
74a7d7c75a
Refactor PoS downloader (#3717)
* Remove blockHeight arg from FeedHeaderPoW

* Revert "Remove blockHeight arg from FeedHeaderPoW"

This reverts commit 9bf92921db11cd4e13386fcee29f30241d070fc6.

* Move PayloadMessage & ForkChoiceMessage to package engineapi

* RequestList instead of newPayloadCh & forkChoiceCh

* Introduce request status

* Send reply only when request status is New

* Move BeaconRequestList to HeaderDownload

* Don't SetFetching when PoS (Fetching means handling newBlockHashes)

* Merge Syncing & Synced into DataWasMissing

* Introduce SyncStatus

* onlyNew arg in WaitForRequest

* Move waitingForBeaconChain into RequestList

* Interrupt instead of skipCycleHack

* Introduce timeout for PoS headers

* Split downloadMissingPoSHeaders

* Move StartPoSDownloader into HeaderDownload

* Move Stopping interrupt to StartPoSDownloader

* Move stopping PayloadStatus back to HeadersPOS

* cleanUpPoSDownload

* Post-merge fix

* TestBogusForkchoice

* TestPoSDownloader

* requestStatus in attemptPoSDownload

* Broadcast in SetStatus

* Cosmetics

* attemptPoSDownload -> schedulePoSDownload

* Demote some logs to Trace
2022-03-22 17:49:12 +01:00
Alex Sharov
4cf4c1bac1
Snapshots: better support of p2p (#3749) 2022-03-21 20:36:03 +07:00
alex.sharov
a9475d9693 trackerslist 2022-03-21 14:49:59 +07:00
Alex Sharov
9c49142e6b
Snapshots: script to commit hashes (#3746) 2022-03-21 13:40:35 +07:00
Alex Sharov
586ad297e6
Snapshots: script to auto-commit torrent hashes (#3745) 2022-03-21 11:22:37 +07:00
alex.sharov
a8c13ebd4f docs about faster rsync 2022-03-21 09:39:48 +07:00
Enrique Jose Avila Asapche
726557fe72
Expose jwt-secret flag in erigon (#3741) 2022-03-21 08:33:10 +07:00
Enrique Jose Avila Asapche
022e0dd6b2
Added kiln devnet (#3742)
* kiln-devnet flag

* kiln-devnet genesis

* bootnodes

* added chain name

* lint

* allocs
2022-03-19 17:04:48 +00:00
ledgerwatch
d5fb8f7d24
[erigon2] Separate handover timing (#3736)
* Separate handover timing

* Update

* Corrected handover time calculation

* Not use compression when aggregate

* Update to latest erigon-lib

* Update to erigon-lib main

* Update

* Disable reproducible builds

* Restore github actions

* Restore github actions

* Revert "linter to support go 1.18 (#3739)"

This reverts commit 1fd434d3d1.

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-19 16:15:12 +00:00
leonardchinonso
d0322ab840
Added eth_getTransactionCount to the devnet tool (#3734)
* Added eth_getTransactionCount to the devnet tool

* Fixed lint errors
2022-03-18 11:57:23 +00:00
ledgerwatch
c47cf1fd49
[erigon2] Optimise Huffman decoding (#3727)
* Update to erigon-lib

* Fix for block number

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-18 09:55:38 +00:00
Alex Sharov
7bf87e9597
Snapshots: grpc event (#3732) 2022-03-18 15:06:23 +07:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files (#3724) 2022-03-18 11:12:18 +07:00
leonardchinonso
d074a403c0
improved error handling and segmented packages and functions (#3728) 2022-03-17 20:16:02 +00:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 (#3705)
* added support of Clique consensus for block processing for erigon2

* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
426d972356
Integration: fix nil snapshots case (#3723)
* save

* save
2022-03-17 08:57:03 +07:00
Enrique Jose Avila Asapche
bb711676a4
enabling apis for merge if chainConfig has TTD (#3721) 2022-03-17 08:28:17 +07:00
leonardchinonso
2e81cdbfd9
Feature/add getlogs to devnet tool (#3722)
* adding eth_getLogs functionality for contract events for the devnet tool

* Made changes to fix double hashing

* Fixed lint errors

* Changed strings.Replace to strings.ReplaceAll for replacing strings

* Cleaned up print statements across files

* Fixed logs not being populated in result typee.go

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-16 22:21:05 +00:00
ledgerwatch
52852c0850
rpcdaemon: fix TxContext in traceBlock (#3716) (#3718)
Previously `txCtx` is not updated for every tx, which
leads to wrong tracing results.

Co-authored-by: can <can@canx.me>
2022-03-16 16:21:40 +00:00
Alex Sharov
d5dbad4f27
save (#3715co 2022-03-16 15:35:36 +07:00
Andrew Ashikhmin
2785a57d1c
More Merge logging (#3712)
* More Merge logging

* Even more logging

* Marshall incoming payload json

* Fix json formatting

* Revert json marshalling

* Some extra logs
2022-03-16 06:55:21 +01:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default (#3707) 2022-03-16 09:57:48 +07:00
mars
a0765a8bc4
No need to panic (#3694) 2022-03-15 07:47:23 +00:00
leonardchinonso
4ee0ae1610
adding eth_getLogs functionality for contract events for the devnet tool (#3680)
* adding eth_getLogs functionality for contract events for the devnet tool

* Made changes to fix double hashing

* Fixed lint errors

* Changed strings.Replace to strings.ReplaceAll for replacing strings

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-14 13:59:51 +00:00
Giulio rebuffo
e20506e932
fixed JWT authentication port (#3689)
* fixed JWT

* jwt.hex

* ops
2022-03-14 14:47:26 +01:00
Giulio rebuffo
b6f707d811
fixed bootstrap nodes bug (#3688)
* fix transition warning

* fixed commit genesis

* fixed bootstrap nodes bug

* Update genesis.go
2022-03-14 09:47:56 +01:00
ledgerwatch
8b00636e21
[erigon2] history indices based on Elias Fano (#3679)
* [erigon2] history indices based on Elias Fano

* Tidy

* Update

* tx lookup to support blocTo in integration

* Update to fix

* Update

* Fix

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-13 23:11:49 +00:00