Commit Graph

129 Commits

Author SHA1 Message Date
Andrew Ashikhmin
4931f113c2
Add override.shanghaiTime flag (#6414)
and remove `--override.terminaltotaldifficulty` &
`--override.mergeNetsplitBlock`
2022-12-23 17:06:56 +01:00
Alex Sharov
863e1ed07b
e3: handle "kill -9" during merge, auto-hide history .ef file if not corresponding .v file (#6304) 2022-12-15 13:49:06 +07:00
Andrew Ashikhmin
1e206e375b
Set default for externalcl based on whether embedded CL is supported (#6294)
Embedded CL is not supported for Gnosis Chain, so it makes sense to set
`externalcl` to true by default for it.

Also, this PR sets `terminalTotalDifficultyPassed` for Gnosis Chain &
Chiado (see https://docs.gnosischain.com/updates/20221210-merge).
2022-12-13 14:51:13 +01:00
Andrew Ashikhmin
ecefc53dd0
Remove Ropsten & Fermion (#6262)
See https://blog.ethereum.org/2022/11/30/ropsten-shutdown-announcement
2022-12-10 22:55:31 +00:00
Alex Sharov
9bcd7fe6c3
e3: reduce state write lock (#6096) 2022-11-21 10:38:22 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
Philippe Schommers
806117fbab
feat: add chiado config (#6058)
Hey guys, I'm trying to add the Chiado network ([Gnosis'
testnet](https://docs.gnosischain.com/about/networks/chiado)) now that
Gnosis can be synced fully with Erigon, so we can test it on the testnet
as well.

This is mostly inspired from
cd5ef32f37.

Probably missing:
- [ ] The right consensus config (currently only a copy of Gnosis)
- [ ] Fixes to the chainspec?
- [ ] Presumably something in `cl/clparams/config.go`

Current state:
```
$ ./build/bin/erigon --chain=chiado --log.console.verbosity=debug
WARN[11-16|11:52:28.188] no log dir set, console logging only 
WARN[11-16|11:52:28.193] no log dir set, console logging only 
INFO[11-16|11:52:28.193] Build info                               git_branch=feat/chiado git_tag=v2021.10.03-2291-g17fae73f8 git_commit=17fae73f8af5348ba7c04684f2a2978daf81b67e
INFO[11-16|11:52:28.193] Starting Erigon on                       devnet=chiado
INFO[11-16|11:52:28.194] Maximum peer count                       ETH=100 total=100
INFO[11-16|11:52:28.194] starting HTTP APIs                       APIs=eth,erigon,engine
INFO[11-16|11:52:28.194] torrent verbosity                        level=WRN
INFO[11-16|11:52:30.300] Set global gas cap                       cap=50000000
INFO[11-16|11:52:30.302] Opening Database                         label=chaindata path=/home/filoozom/.local/share/erigon/chiado/chaindata
INFO[11-16|11:52:30.310] Initialised chain configuration          config="{ChainID: 10200, Homestead: 0, DAO: <nil>, DAO Support: false, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 0, Muir Glacier: <nil>, Berlin: 0, London: 0, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: <nil>, Merge Netsplit: <nil>, Shanghai: <nil>, Cancun: <nil>, Engine: aura}" genesis=0xf463abeb7ee27fa62be3ac36a264e8174ee3458da451e6403df47618fd2cf415
WARN[11-16|11:52:30.311] Incorrect snapshot enablement            got=true change_to=false
INFO[11-16|11:52:30.311] Effective                                prune_flags= snapshot_flags= history.v3=false
INFO[11-16|11:52:30.312] Initialising Ethereum protocol           network=10200
INFO[11-16|11:52:30.329] Starting private RPC server              on=127.0.0.1:9090
INFO[11-16|11:52:30.329] new subscription to logs established 
INFO[11-16|11:52:30.329] rpc filters: subscribing to Erigon events 
DBUG[11-16|11:52:30.330] Establishing event subscription channel with the RPC daemon ... 
INFO[11-16|11:52:30.330] New txs subscriber joined 
INFO[11-16|11:52:30.330] new subscription to newHeaders established 
INFO[11-16|11:52:30.330] Reading JWT secret                       path=/home/filoozom/.local/share/erigon/chiado/jwt.hex
INFO[11-16|11:52:30.331] HTTP endpoint opened for Engine API      url=localhost:8551 ws=true ws.compression=true
INFO[11-16|11:52:30.332] HTTP endpoint opened                     url=localhost:8545 ws=false ws.compression=true grpc=false
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=tcp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=udp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.341] [1/16 Snapshots] DONE                    in=134.7µs
INFO[11-16|11:52:30.341] [txpool] Started 
INFO[11-16|11:52:30.341] [2/16 Headers] Waiting for headers...    from=0
DBUG[11-16|11:52:30.341] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=udp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.344] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.346] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.347] Started P2P networking                   version=67 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30304 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:30.351] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.351] Started P2P networking                   version=66 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30303 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:31.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:52:32.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
INFO[11-16|11:37:00.062] [p2p] GoodPeers                          eth66=0 eth67=0
INFO[11-16|11:37:00.077] [txpool] stat                            block=0 pending=0 baseFee=0 queued=0 alloc=42.7MB sys=79.5MB
INFO[11-16|11:37:00.089] [2/16 Headers] No block headers to write in this log period block number=0
INFO[11-16|11:37:00.089] Req/resp stats                           req=0 reqMin=0 reqMax=0 skel=0 skelMin=0 skelMax=0 resp=0 respMin=0 respMax=0 dups=0
DBUG[11-16|11:37:00.089] [Downloader] Queue sizes                 anchors=0 links=0 persisted=1
DBUG[11-16|11:37:00.089] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:37:06.095] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
```

I guess it comes down to:
```
--- FAIL: TestDefaultBSCGenesisBlock (0.34s)
    genesis_test.go:27: 
                Error Trace:    /home/filoozom/projects/erigon/core/genesis_test.go:27
                                                        /home/filoozom/projects/erigon/core/genesis_test.go:30
                Error:          Not equal: 
                                expected: []byte{0xe8, 0x72, 0x46, 0x2c, 0x6b, 0xd5, 0xf4, 0x2, 0xec, 0x81, 0xde, 0x7c, 0x5b, 0xd2, 0x82, 0x3e, 0x13, 0x7c, 0x66, 0x6b, 0x78, 0xe8, 0x2b, 0x7e, 0xb0, 0xbe, 0x95, 0xaf, 0x5e, 0xce, 0xa1, 0x8d}
                                actual  : []byte{0xad, 0xa4, 0x4f, 0xd8, 0xd2, 0xec, 0xab, 0x8b, 0x8, 0xf2, 0x56, 0xaf, 0x7, 0xad, 0x3e, 0x77, 0x7f, 0x17, 0xfb, 0x43, 0x4f, 0x8f, 0x8e, 0x67, 0x8b, 0x31, 0x2f, 0x57, 0x62, 0x12, 0xba, 0x9a}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,4 +1,4 @@
                                 ([]uint8) (len=32) {
                                - 00000000  e8 72 46 2c 6b d5 f4 02  ec 81 de 7c 5b d2 82 3e  |.rF,k......|[..>|
                                - 00000010  13 7c 66 6b 78 e8 2b 7e  b0 be 95 af 5e ce a1 8d  |.|fkx.+~....^...|
                                + 00000000  ad a4 4f d8 d2 ec ab 8b  08 f2 56 af 07 ad 3e 77  |..O.......V...>w|
                                + 00000010  7f 17 fb 43 4f 8f 8e 67  8b 31 2f 57 62 12 ba 9a  |...CO..g.1/Wb...|
                                 }
                Test:           TestDefaultBSCGenesisBlock
                Messages:       chiado
FAIL
FAIL    github.com/ledgerwatch/erigon/core      0.823s
```

----------

Turns out that the `code` in Erigon's chainspec is not the same as
`constructor` in Nethermind for example.
Comparison for Sokol:
- Erigon:
https://github.com/ledgerwatch/erigon/blob/devel/core/allocs/sokol.json#L20-L34
- Nethermind:
https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Chains/sokol.json#L248-L252
2022-11-18 12:54:18 +01:00
Giulio rebuffo
ae01026ca9
Customizable networking for Erigon Lightclient (#6032) 2022-11-11 23:22:47 +01:00
ledgerwatch
5fc2ec4b52
Enable snapshot creation for Gnosis Chain (#6030)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-11 15:38:59 +00:00
Alex Sharov
1e5a513657
e3: configurable recon workers (#5983) 2022-11-07 11:42:44 +07:00
Alex Sharov
f3d8533879
e3: cheaper recon estimate (#5958) 2022-11-04 13:21:44 +07:00
hexoscott
c611c7489f
reduce estimated RAM from 80% to 50% (#5930) 2022-11-01 21:54:39 +07:00
Alex Sharov
8fe89f4e5b
e3: less small merges, parallel merge (#5895) 2022-10-29 13:21:06 +07:00
Alex Sharov
a1e21260ad
e3: reduce reconstitute ram estimation (#5848) 2022-10-23 18:49:57 +07:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram (#5784) 2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing (#5782) 2022-10-18 15:29:59 +07:00
Alex Sharov
3aa97c57c1
fix ram estimation for indexing (#5773) 2022-10-18 10:19:58 +07:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl (#5742) 2022-10-15 08:20:58 +07:00
Giulio rebuffo
2fd96a0b1c
[Experimental] GRPC Prototype with --experimental.lightclient to have Erigon listen on ConsenSUS gossip. (#5590)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-06 11:01:56 +02:00
Alex Sharov
c48d51a582
snapshot index workers amount - based on available RAM and CPU (#5524) 2022-10-05 17:07:01 +07:00
ledgerwatch
49aeb53988
Fail newPayload if execution is unsuccessful (#5553)
* Fail newPayload if execution is unsuccessful

* fix linter

* fix message

* no self-hosted

* try force ci

* back to unbuntu

* Remove unnecessary unwinds

* Fix canonicalising

* Quiet logs for in-memory exec

* Remove experimental overlay flag

* Fix sync test

* Fix test

* Fix lint

* Print hash in the end of the cycle

* Not notify headers if list is empty

* Supress timings and tables logs when no change in progress

* Reduce logging

* fix

* better log for pos download

* Better newPayload and forkChoice messages

* Fix lint

* simplify

* No duplication of download messages

* Reverse log for extension

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-09-29 06:38:13 +01:00
Alex Sharov
cb60382e6d
erigon3: rename "history.v2" to "history.v3" to avoid naming miss-match with "erigon3" (#5519) 2022-09-26 10:54:42 +07:00
Alex Sharov
c1bb4bc734
exec22: stagedsync test v1 (#5425)
* save

* save
2022-09-19 12:17:36 +07:00
Alex Sharov
8813ea87fd
Erigon22: add state reconstitution to stage_exec (#5321) 2022-09-09 14:40:25 +07:00
Alex Sharov
46ef7e5305
erigon22: incremental hash state stage (#5102) 2022-08-19 09:35:43 +07:00
Alex Sharov
e27f669a3f
erigon2.2: save stage progress to chainDB, can run "integration stage_exec" and "state erigon22" on same datadir (#5033) 2022-08-13 11:17:24 +07:00
Alex Sharov
4594ce5ef7
erigon22: history.v2 flag, align rpcdaemon22 (#5016)
* save

* save

* save

* save

* save
2022-08-12 16:13:14 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Alex Sharov
2415fecb26
--downloader.verfiy flag to verify once on startup (#4597)
* save

* save

* save

* save

* save

* save
2022-07-01 16:52:43 +06:00
Alex Sharov
afd07e5dee
--no-downloader flag support (#4545) 2022-06-26 17:13:32 +06:00
Giulio rebuffo
7a2b575e56
Added memory execution (#4446)
* add

* added in memory execution draft

* func to state

* added functionality

* backend function for memory execution

* simplified stage state

* haltable stage execution

* added mod sum

* hd

* sl

* LOL

* LOL again

* need trace

* more logs

* added cleanup on fcu

* fcu

* mod

* bunch of prtln

* feed

* ops

* headers notify

* revert

* slightly more commented

* head

* nil block retire

* comments

* prevent clean

* corrected occasional panics

* fixed lint
2022-06-19 20:45:36 +01:00
Andrew Ashikhmin
644d25d25d
MergeForkBlock -> MergeNetsplitBlock (#4407) 2022-06-08 12:41:50 +02:00
Alex Sharov
a53642b4bf
datadir.Dirs configuration object to group dir config (#4387) 2022-06-07 10:24:50 +07:00
Krishna Upadhyaya
0dae3ad69c
add bor-mainnet to chain snapshots (#4369) 2022-06-05 14:45:45 +05:30
Enrique Jose Avila Asapche
9f1cc40c89
changed syncmode flag to snapshots flag (#4353)
* changed syncmode flag to snapshots flag

* using boolT

* Remove syncmode concept

* Add mumbai

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 07:33:55 +01:00
Enrique Jose Avila Asapche
bc21ab9d97
syncmode changes (#4334)
* log syncmode

* little comment

* ops

* calling log in correct place

* got rid of error instead we warn

* no return
2022-06-02 14:15:00 +01:00
Alex Sharov
955c669d21
Group few sync configs (#4303)
* save

* save
2022-05-30 17:08:49 +07:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00
Andrew Ashikhmin
16950ebc2c
Fix OverrideTerminalTotalDifficulty when datadir is not empty (#4280) 2022-05-26 21:21:51 +02:00
Alex Sharov
66248c4bfb
Torrent: verify all files (#4270) 2022-05-26 12:27:44 +07:00
Alex Sharov
c2e0956e3e
return err on invalid syncmode (#4219)
* save

* save

* save
2022-05-20 16:04:38 +07:00
Alex Sharov
4e7807251a
Ensure snapshotDir not created by accident (#4164)
* save

* save

* save
2022-05-17 09:40:45 +07:00
Alex Sharov
91e43e52a6
Snapsync for ropsten (#4143)
* save

* save
2022-05-13 14:58:03 +07:00
Alex Sharov
32d8ab34d9
atomic snapshot dir, step 2 (#4093)
* atomic snapshot dir

* atomic snapshot dir
2022-05-07 14:53:21 +07:00
Alex Sharov
831da9ba41
reduce downloader deps (#4011) 2022-04-28 12:42:14 +07:00
Andrew Ashikhmin
4d2db4467e
FullSync instead of FastSync (#3980) 2022-04-26 17:19:24 +02:00
Alex Sharov
61515ed21e
Bsc: enable syncmode=snap by default #3973 2022-04-26 13:06:18 +07:00
Alex Sharov
874cd5dca0
bsc: disable snap sync (#3955) 2022-04-25 08:52:25 +07:00
Giulio rebuffo
c5b1836481
Added Ethstats service (#3931)
* somewhat there but not yet

* lol

* more efficient ethstats

* lint

* not die on no wifi
2022-04-22 07:23:52 +01:00