Commit Graph

162 Commits

Author SHA1 Message Date
alex.sharov
6f503165cd Revert "reduce amount of index workers (#8192)"
This reverts commit e9a8e669e6.
2023-09-14 12:25:01 +07:00
Alex Sharov
e9a8e669e6
reduce amount of index workers (#8192)
https://github.com/ledgerwatch/erigon/issues/8189
2023-09-14 12:21:56 +07:00
Somnath Banerjee
a699f64761
Txpool upgrades for EIP-4844 Blob Transactions (#8004)
See https://github.com/ledgerwatch/erigon-lib/pull/1075
2023-09-11 09:38:58 +07:00
alex.sharov
1160df12a3 save 2023-09-06 14:44:51 +07:00
Alex Sharov
d1d348211f
add flag --force.partial.commit: to workaround problem "start from backup takes long time and can't save partial progress" (#8090) 2023-08-30 08:49:16 +07:00
Andrew Ashikhmin
63628618eb
Small fixes to Holesky config (#8092)
Small fixes to PR #8064:

- Don't set cancunTime for Holesky – otherwise we'll have to overwrite
it later using the `--override.cancun` flag
- We don't have block snapshots for Holesky yet
- Add Holesky to `isChainIDPoS`

See https://github.com/eth-clients/holesky/pull/68
2023-08-29 16:29:14 +02:00
racytech
f477348a50
Support for --chain=holesky (#8064) 2023-08-26 16:31:29 +06:00
battlmonstr
32ca0e5ab1
sync: revert flawed dropUselessPeers logic and alleviate its issues (#8062)
The current logic is flawed, because it drops all peers that are less
synced.
It is valid to return empty responses by the eth spec.
A proper logic should penalize from the context of the sync process,
where enough "reputation" data is collected about a peer.

In order to be able to connect to erigon 2.48 peers that have
--sentry.drop-useless-peers enabled,
this adds a check to not reply with an empty headers list.
If we reply with an empty list, we're going to be considered useless and
kicked.
Once enough of erigon nodes are updated in the network past this commit,
this check should be removed,
because it is totally acceptable to return an empty list by the eth
spec.
2023-08-25 11:42:54 +02:00
Giulio rebuffo
0e4e36b142
Replaced old version of Engine API with newer version (#7972) 2023-08-05 23:33:10 +02:00
Andrew Ashikhmin
d014da4dc0
Replace --override.shanghaiTime flag with --override.cancun (#7964)
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/1067
2023-08-03 16:05:35 +02:00
Giulio rebuffo
39853432be
added --experimental.modular (#7925) 2023-07-24 18:04:36 +02:00
Alex Sharov
e5023775aa
Enforce blockReader interface (#7737)
- breaks dependency from staged_sync to package with block_reader
implementation
- breaks dependency from snap_sync to package with block_reader
implementation
- breaks dependency from mining to txpool implementation
2023-06-15 13:11:51 +07:00
Alex Sharov
63c92010cd
remove txsV3 cli flag (#7644) 2023-06-03 15:54:27 +07:00
Alex Sharov
63afe65686
Use BlockReader in ForkValidator, CliqueAPI (#7562) 2023-05-23 14:49:17 +07:00
alex.sharov
1247c2adbb e3: fix agg step size 2023-04-23 13:33:32 +07:00
Alex Sharov
74e08e197d
e3: remove files when no readers (#7370) 2023-04-23 11:55:16 +07:00
Giulio rebuffo
2ce5e761bb
added --internalcl flag to Erigon (#7349) 2023-04-19 14:37:35 +02:00
Andrew Ashikhmin
96bb5d544b
Remove Parlia (#7306)
Pre-requisite: https://github.com/ledgerwatch/erigon-lib/pull/969
2023-04-14 06:24:10 +00:00
Alex Sharov
625b0c5b15
take in account memlimit of cgroups (#7204)
for cgroups v2 support see:
https://github.com/shirou/gopsutil/issues/1416
2023-04-04 03:29:56 +00:00
ledgerwatch
7258a2b872
Remove BSC support in Erigon (step 1) (#7246)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-04-03 14:46:21 +00:00
Alex Sharov
592ec1ee0d
e4: step 1 to run tests (#7209) 2023-03-29 14:01:27 +00:00
Alex Sharov
417a437584
Break dependency of ethcfg package to core/consensus/etc... move genesis struct to 'types' package (#7206) 2023-03-29 07:27:06 +00:00
alex.sharov
afe0e385d2 e4: add flag in tests 2023-03-29 11:51:05 +07:00
alex.sharov
f69f7095bb e3: exec to use half of CPU's by default 2023-03-22 10:17:03 +07:00
hexoscott
77065ee0f3
remove PeerUseless calls (#7032) 2023-03-06 18:59:14 +00:00
nanevardanyan
ab6239b30f
WIP: cmd, turbo, core, eth: TransactionsV3 flag and persist in new table (#6754) 2023-02-24 18:49:25 +00:00
Alex Sharov
ab8e59ff0e
Sepolia: enable blocks snapshots by default (#6847) 2023-02-12 04:14:13 +00:00
Anshal Shukla
a4f8175136
Bor file structure updated (#6167)
Updated bor consensus folder structure
2023-01-31 14:00:57 +05:30
Alex Sharov
bb2bec2439
sepolia block snaspshots (#6626) 2023-01-25 09:55:26 +07:00
Andrew Ashikhmin
37741b74eb
Remove erigon_issuance & erigon_watchTheBurn (#6685)
Some issuance happens on the CL side now, so removing support for these
now inaccurate proprietary methods to make our life easier.
2023-01-24 22:54:08 +01:00
ledgerwatch
cfb0d99cfa
Improvements for body downloader (#6589)
1. Replacing temporary MBDX table with limited-size btree
2. Always scan block numbers from the start to prioritise low-number
blocks
3. Other fixes and simplifications

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-16 22:09:28 +00:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
Alex Sharov
e88a5aa2e8
HashState: parallel promote cleanly (#6512) 2023-01-09 11:01:21 +07:00
Andrew Ashikhmin
4931f113c2
Add override.shanghaiTime flag (#6414)
and remove `--override.terminaltotaldifficulty` &
`--override.mergeNetsplitBlock`
2022-12-23 17:06:56 +01:00
Alex Sharov
863e1ed07b
e3: handle "kill -9" during merge, auto-hide history .ef file if not corresponding .v file (#6304) 2022-12-15 13:49:06 +07:00
Andrew Ashikhmin
1e206e375b
Set default for externalcl based on whether embedded CL is supported (#6294)
Embedded CL is not supported for Gnosis Chain, so it makes sense to set
`externalcl` to true by default for it.

Also, this PR sets `terminalTotalDifficultyPassed` for Gnosis Chain &
Chiado (see https://docs.gnosischain.com/updates/20221210-merge).
2022-12-13 14:51:13 +01:00
Andrew Ashikhmin
ecefc53dd0
Remove Ropsten & Fermion (#6262)
See https://blog.ethereum.org/2022/11/30/ropsten-shutdown-announcement
2022-12-10 22:55:31 +00:00
Alex Sharov
9bcd7fe6c3
e3: reduce state write lock (#6096) 2022-11-21 10:38:22 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
Philippe Schommers
806117fbab
feat: add chiado config (#6058)
Hey guys, I'm trying to add the Chiado network ([Gnosis'
testnet](https://docs.gnosischain.com/about/networks/chiado)) now that
Gnosis can be synced fully with Erigon, so we can test it on the testnet
as well.

This is mostly inspired from
cd5ef32f37.

Probably missing:
- [ ] The right consensus config (currently only a copy of Gnosis)
- [ ] Fixes to the chainspec?
- [ ] Presumably something in `cl/clparams/config.go`

Current state:
```
$ ./build/bin/erigon --chain=chiado --log.console.verbosity=debug
WARN[11-16|11:52:28.188] no log dir set, console logging only 
WARN[11-16|11:52:28.193] no log dir set, console logging only 
INFO[11-16|11:52:28.193] Build info                               git_branch=feat/chiado git_tag=v2021.10.03-2291-g17fae73f8 git_commit=17fae73f8af5348ba7c04684f2a2978daf81b67e
INFO[11-16|11:52:28.193] Starting Erigon on                       devnet=chiado
INFO[11-16|11:52:28.194] Maximum peer count                       ETH=100 total=100
INFO[11-16|11:52:28.194] starting HTTP APIs                       APIs=eth,erigon,engine
INFO[11-16|11:52:28.194] torrent verbosity                        level=WRN
INFO[11-16|11:52:30.300] Set global gas cap                       cap=50000000
INFO[11-16|11:52:30.302] Opening Database                         label=chaindata path=/home/filoozom/.local/share/erigon/chiado/chaindata
INFO[11-16|11:52:30.310] Initialised chain configuration          config="{ChainID: 10200, Homestead: 0, DAO: <nil>, DAO Support: false, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 0, Muir Glacier: <nil>, Berlin: 0, London: 0, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: <nil>, Merge Netsplit: <nil>, Shanghai: <nil>, Cancun: <nil>, Engine: aura}" genesis=0xf463abeb7ee27fa62be3ac36a264e8174ee3458da451e6403df47618fd2cf415
WARN[11-16|11:52:30.311] Incorrect snapshot enablement            got=true change_to=false
INFO[11-16|11:52:30.311] Effective                                prune_flags= snapshot_flags= history.v3=false
INFO[11-16|11:52:30.312] Initialising Ethereum protocol           network=10200
INFO[11-16|11:52:30.329] Starting private RPC server              on=127.0.0.1:9090
INFO[11-16|11:52:30.329] new subscription to logs established 
INFO[11-16|11:52:30.329] rpc filters: subscribing to Erigon events 
DBUG[11-16|11:52:30.330] Establishing event subscription channel with the RPC daemon ... 
INFO[11-16|11:52:30.330] New txs subscriber joined 
INFO[11-16|11:52:30.330] new subscription to newHeaders established 
INFO[11-16|11:52:30.330] Reading JWT secret                       path=/home/filoozom/.local/share/erigon/chiado/jwt.hex
INFO[11-16|11:52:30.331] HTTP endpoint opened for Engine API      url=localhost:8551 ws=true ws.compression=true
INFO[11-16|11:52:30.332] HTTP endpoint opened                     url=localhost:8545 ws=false ws.compression=true grpc=false
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=tcp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=udp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.341] [1/16 Snapshots] DONE                    in=134.7µs
INFO[11-16|11:52:30.341] [txpool] Started 
INFO[11-16|11:52:30.341] [2/16 Headers] Waiting for headers...    from=0
DBUG[11-16|11:52:30.341] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=udp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.344] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.346] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.347] Started P2P networking                   version=67 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30304 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:30.351] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.351] Started P2P networking                   version=66 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30303 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:31.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:52:32.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
INFO[11-16|11:37:00.062] [p2p] GoodPeers                          eth66=0 eth67=0
INFO[11-16|11:37:00.077] [txpool] stat                            block=0 pending=0 baseFee=0 queued=0 alloc=42.7MB sys=79.5MB
INFO[11-16|11:37:00.089] [2/16 Headers] No block headers to write in this log period block number=0
INFO[11-16|11:37:00.089] Req/resp stats                           req=0 reqMin=0 reqMax=0 skel=0 skelMin=0 skelMax=0 resp=0 respMin=0 respMax=0 dups=0
DBUG[11-16|11:37:00.089] [Downloader] Queue sizes                 anchors=0 links=0 persisted=1
DBUG[11-16|11:37:00.089] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:37:06.095] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
```

I guess it comes down to:
```
--- FAIL: TestDefaultBSCGenesisBlock (0.34s)
    genesis_test.go:27: 
                Error Trace:    /home/filoozom/projects/erigon/core/genesis_test.go:27
                                                        /home/filoozom/projects/erigon/core/genesis_test.go:30
                Error:          Not equal: 
                                expected: []byte{0xe8, 0x72, 0x46, 0x2c, 0x6b, 0xd5, 0xf4, 0x2, 0xec, 0x81, 0xde, 0x7c, 0x5b, 0xd2, 0x82, 0x3e, 0x13, 0x7c, 0x66, 0x6b, 0x78, 0xe8, 0x2b, 0x7e, 0xb0, 0xbe, 0x95, 0xaf, 0x5e, 0xce, 0xa1, 0x8d}
                                actual  : []byte{0xad, 0xa4, 0x4f, 0xd8, 0xd2, 0xec, 0xab, 0x8b, 0x8, 0xf2, 0x56, 0xaf, 0x7, 0xad, 0x3e, 0x77, 0x7f, 0x17, 0xfb, 0x43, 0x4f, 0x8f, 0x8e, 0x67, 0x8b, 0x31, 0x2f, 0x57, 0x62, 0x12, 0xba, 0x9a}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,4 +1,4 @@
                                 ([]uint8) (len=32) {
                                - 00000000  e8 72 46 2c 6b d5 f4 02  ec 81 de 7c 5b d2 82 3e  |.rF,k......|[..>|
                                - 00000010  13 7c 66 6b 78 e8 2b 7e  b0 be 95 af 5e ce a1 8d  |.|fkx.+~....^...|
                                + 00000000  ad a4 4f d8 d2 ec ab 8b  08 f2 56 af 07 ad 3e 77  |..O.......V...>w|
                                + 00000010  7f 17 fb 43 4f 8f 8e 67  8b 31 2f 57 62 12 ba 9a  |...CO..g.1/Wb...|
                                 }
                Test:           TestDefaultBSCGenesisBlock
                Messages:       chiado
FAIL
FAIL    github.com/ledgerwatch/erigon/core      0.823s
```

----------

Turns out that the `code` in Erigon's chainspec is not the same as
`constructor` in Nethermind for example.
Comparison for Sokol:
- Erigon:
https://github.com/ledgerwatch/erigon/blob/devel/core/allocs/sokol.json#L20-L34
- Nethermind:
https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Chains/sokol.json#L248-L252
2022-11-18 12:54:18 +01:00
Giulio rebuffo
ae01026ca9
Customizable networking for Erigon Lightclient (#6032) 2022-11-11 23:22:47 +01:00
ledgerwatch
5fc2ec4b52
Enable snapshot creation for Gnosis Chain (#6030)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-11 15:38:59 +00:00
Alex Sharov
1e5a513657
e3: configurable recon workers (#5983) 2022-11-07 11:42:44 +07:00
Alex Sharov
f3d8533879
e3: cheaper recon estimate (#5958) 2022-11-04 13:21:44 +07:00
hexoscott
c611c7489f
reduce estimated RAM from 80% to 50% (#5930) 2022-11-01 21:54:39 +07:00
Alex Sharov
8fe89f4e5b
e3: less small merges, parallel merge (#5895) 2022-10-29 13:21:06 +07:00
Alex Sharov
a1e21260ad
e3: reduce reconstitute ram estimation (#5848) 2022-10-23 18:49:57 +07:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram (#5784) 2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing (#5782) 2022-10-18 15:29:59 +07:00