Philippe Schommers
806117fbab
feat: add chiado config ( #6058 )
...
Hey guys, I'm trying to add the Chiado network ([Gnosis'
testnet](https://docs.gnosischain.com/about/networks/chiado )) now that
Gnosis can be synced fully with Erigon, so we can test it on the testnet
as well.
This is mostly inspired from
cd5ef32f37https://github.com/ledgerwatch/erigon/blob/devel/core/allocs/sokol.json#L20-L34
- Nethermind:
https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Chains/sokol.json#L248-L252
2022-11-18 12:54:18 +01:00
lupin012
21d24a20a0
fix on number of samples for blocks ( #5826 )
...
eth_gasPrice(): A maximum of 3 samples should be taken per block (see
sample const)
The getBlockPrices() function takes 3 samples on the first block and one
on the others.
In the check s.Len() >= limit s.Len() after first block will contain 3
samples and so the loop will be executed only one time for each block
NOT three times
2022-11-18 03:14:49 +03:00
Manav Darji
1da4d3abbf
eth, consensus/bor: fixes and improvements related to mining ( #6051 )
...
In context of https://github.com/ledgerwatch/erigon/issues/5694 , this PR
adds some fixes and improvement in the mining flow. Also, a relevant
change in txpool (present in erigon-lib) is made here:
https://github.com/ledgerwatch/erigon-lib/pull/737
#### Changes in triggering mining in `startMining()`
The mining module didn't honour the block time as a simple 3 second
timer and a notifier from txpool was used to trigger mining. This would
cause inconsistencies, at least with the bor consensus. Hence, a geth
like approach is used instead for simplicity. A new head channel
subscription is added in the `startMining()` loop which would notify the
addition of new block. Hence, this would make sure that the block time
is being honoured. Moreover, the fixed 3 second timer is replaced by the
`miner.recommit` value set using flags.
#### Changes in the arrangement of calls made post mining
When all the mining stages are completed, erigon writes all the data in
a cache. It then processes the block through all the stages as it would
process a block received from P2P. In this case, some of the stages
aren't really required. Like the block header and body download stage is
not required as the block was mined locally. Even execution stage is not
required as it already went through it in the mining stages.
Now, we encountered an issue where the chain was halted and kept mining
the same block again and again (liveness issue). The root cause is
because of an error in a stage of it's parent block. This stage turns
out to be the 4th stage which is "Block body download" stage. This stage
tries to download the block body from peers using the headers. As, we
mined this block locally we don't really need to download anything (or
process anything again). Hence, it reaches out to the cache which we
store for the block body.
Interestingly that cache turned out to be empty for some blocks. This
was because post mining, before adding block header and body to a cache,
we call the broadcast method which starts the staged sync. So,
technically it’s a bit uncertain at any stage if the block header and
body has been written or not.(see
[this](https://github.com/ledgerwatch/erigon/blob/devel/eth/backend.go#L553-L572 )).
To achieve complete certainty, we rearranged the calls with the write to
cache being called first and broadcast next. This pretty much solves the
issue as now we’re sure that we’d always have a block body in the cache
when we reach the body download stage.
#### Misc changes
This PR also adds some logs in bor consensus.
2022-11-18 02:39:16 +03:00
Alex Sharov
874d96fb47
e3: Non parallel exec (with exec in RO goroutine) ( #6073 )
2022-11-17 20:52:12 +03:00
Alex Sharov
c50d4e6c0b
e3: panic logging ( #6072 )
2022-11-17 16:17:02 +03:00
Alex Sharov
d5c23c4df4
e3: enable test ( #6071 )
2022-11-17 16:06:57 +03:00
Alex Sharov
93926646f8
e3: prepare downloader for extraction ( #6061 )
2022-11-16 15:48:23 +03:00
Alex Sharov
7d034d0c79
e3: fix stuck exec after recon ( #6059 )
2022-11-16 14:22:49 +03:00
Giulio rebuffo
3a2d3be080
Added persistence in LightClient ( #6056 )
2022-11-15 20:39:18 +01:00
Alex Sharov
242b5a76df
e3: fix doneCount variable ( #6053 )
2022-11-15 18:03:10 +03:00
Andrew Ashikhmin
14c0643476
Enable both eth/66 and eth/67 by default ( #6048 )
2022-11-15 10:41:56 +01:00
Giulio rebuffo
ae01026ca9
Customizable networking for Erigon Lightclient ( #6032 )
2022-11-11 23:22:47 +01:00
ledgerwatch
5fc2ec4b52
Enable snapshot creation for Gnosis Chain ( #6030 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-11 15:38:59 +00:00
Alex Sharov
90952c4110
e3: wal ram configure ( #6023 )
2022-11-11 11:28:22 +07:00
Giulio rebuffo
f8916e9226
Made Lightclient default ( #5813 )
...
lightclient cl is default, for external cl, run `--externalcl`
2022-11-10 17:06:04 +00:00
hexoscott
475fa08fdf
eth_gasPrice performance tweaks ( #6016 )
...
Code news up an oracle for every call so existing cache checks always
came back as 0. Moved cache up a layer and pass in via the new
`gasprice.Cache` interface. Looked at putting the oracle instance onto
the ethApi itself to re-use it that way, but the backend transaction
made it a little hard work as we can't re-use that. This seemed cleaner
but happy to take feedback.
Locally takes me from ±2.5k rps to ±43k rps so quite a difference there.
(k6 with 1000 virtual users)
2022-11-10 15:36:04 +00:00
Alex Sharov
6b0f7fa913
e3: recon reduce ram ( #6004 )
2022-11-09 14:32:44 +07:00
Alex Sharov
57db64b692
e3: fix coinbase 2 ( #6006 )
2022-11-09 13:53:07 +07:00
Alex Sharov
906953b246
e3: fix coinbase ( #6005 )
2022-11-09 13:48:23 +07:00
hexoscott
636586c1b5
cache state check ( #5844 )
...
draft for now to get some early feedback on approach
2022-11-07 13:04:31 +00:00
Alex Sharov
160407b36a
e3: recon metrics ( #5984 )
2022-11-07 12:02:46 +07:00
Alex Sharov
1e5a513657
e3: configurable recon workers ( #5983 )
2022-11-07 11:42:44 +07:00
Alex Sharov
92a1aab1f1
e3: accumulator nil on unwind ( #5981 )
2022-11-07 10:14:35 +07:00
Enrique Jose Avila Asapche
da455c8f48
got rid of effective engine ( #5975 )
...
It is not being used and it hasn't been used for a while #5847
2022-11-07 08:55:03 +07:00
Alex Sharov
766d0fd5dd
e3: more workers ( #5972 )
2022-11-06 12:40:41 +07:00
Alex Sharov
4954b0be1c
e3: more logs ( #5971 )
2022-11-06 11:54:29 +07:00
alex.sharov
949ec358fa
save
2022-11-06 11:52:25 +07:00
Alex Sharov
e7c53ac9b7
e3: fix sequential commit place ( #5969 )
2022-11-06 10:27:52 +07:00
Alex Sharov
bb1811a76e
e3: increase workers ( #5966 )
2022-11-05 16:31:10 +07:00
alex.sharov
1a6bc51f18
save
2022-11-05 16:29:13 +07:00
ledgerwatch
5deaa9cec1
Fix panic in txpool (due to modificatin with read lock) and snapshot … ( #5931 )
...
…access (due to inconsistent indices)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-04 12:43:59 +00:00
Alex Sharov
b1fc489024
e3: fix tests ( #5961 )
2022-11-04 16:14:58 +07:00
Alex Sharov
d4fc7c7ef0
e3: reduce write-lock of rws ( #5959 )
2022-11-04 15:09:29 +07:00
Alex Sharov
f3d8533879
e3: cheaper recon estimate ( #5958 )
2022-11-04 13:21:44 +07:00
Alex Sharov
bc2a873c9d
e3: disable txlookup stage ( #5957 )
2022-11-04 10:51:09 +07:00
Alex Sharov
655e32a199
e3: commit between blocks ( #5955 )
2022-11-04 09:50:35 +07:00
Alex Sharov
7516c33527
e3: idx size metric ( #5954 )
2022-11-04 09:31:30 +07:00
Alex Sharov
d4c406d3f0
e3: aggressive prune on seq exec ( #5953 )
2022-11-04 09:26:34 +07:00
Alex Sharov
d418069553
e3: fix retainList on unwind ( #5947 )
2022-11-03 15:12:53 +07:00
Alex Sharov
32eaa82fba
e3: prune debug log ( #5944 )
2022-11-03 11:15:03 +07:00
Giulio rebuffo
5c3245d4e0
Added Checkpoint sync for CL full client ( #5934 )
2022-11-02 13:12:44 +01:00
Alex Sharov
7208b0dc60
remove fmt ( #5937 )
2022-11-02 10:08:25 +07:00
Alex Sharov
96deee29bb
e3: build files sequentially, flush indices in-advance before commit ( #5935 )
2022-11-02 10:04:12 +07:00
hexoscott
c611c7489f
reduce estimated RAM from 80% to 50% ( #5930 )
2022-11-01 21:54:39 +07:00
Alex Sharov
58c8baf2ef
e3: parallel exec - spend time for prune instead of flush when much data in db ( #5925 )
2022-11-01 14:17:25 +07:00
Alex Sharov
e490d2d0c7
e3: background merge ( #5921 )
2022-11-01 09:55:38 +07:00
Andrew Ashikhmin
9ffc457cbb
EIP-3860: Limit and meter initcode ( #5892 )
...
[EIP-3860](https://eips.ethereum.org/EIPS/eip-3860 ) is
[included](https://github.com/ethereum/execution-specs/pull/633 ) into
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md ).
TODO: similar changes to `txpool`.
Co-authored-by: Andrei Maiboroda <andrei@ethereum.org>
2022-10-31 13:40:41 +01:00
Alex Sharov
4ead27fad9
e3: add "make test3" to CI ( #5910 )
2022-10-31 14:25:52 +07:00
Alex Sharov
381bea5b92
e3: eth_traceTransaction implementation ( #5909 )
2022-10-31 12:31:38 +07:00
Giulio rebuffo
adf5fc7b82
refactored files regarding lightclient ( #5904 )
2022-10-29 21:51:32 +02:00
Alex Sharov
8c36c3fe99
e3: recon progress metric ( #5902 )
2022-10-29 21:51:01 +07:00
Alex Sharov
fde630f76b
e3: wal preallocate historyKey #713 ( #5899 )
2022-10-29 19:18:35 +07:00
Alex Sharov
8fe89f4e5b
e3: less small merges, parallel merge ( #5895 )
2022-10-29 13:21:06 +07:00
Alex Sharov
d8d4a0a240
e3: wait group for apply loop ( #5894 )
2022-10-29 11:48:53 +07:00
Andrew Ashikhmin
dd43d486f3
Enable London in AllEthashProtocolChanges ( #5891 )
...
`AllEthashProtocolChanges` should contain all the EIPs accepted by the
core developers into the Ethash consensus, including the
[London](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md )
protocol upgrade.
2022-10-28 17:19:18 +02:00
Alex Sharov
0074ad0e48
e3: parallel exec, rotate and partial-flush of indices wal - while main thread is idle ( #5889 )
...
While main thread is idle (apply is on RoTx now):
- periodically still hisotry/index etl collectors and load them to db
- prune old history/index in db
Also:
- fixed RAM estimation calculation of State22
2022-10-28 21:13:53 +07:00
Alex Sharov
8d1ed547b0
E3: parallel exec, apply on roTx ( #5879 )
2022-10-28 08:47:45 +07:00
Andrew Ashikhmin
d9ff645fbf
Make lint happy ( #5874 )
...
Fixes
```
eth/stagedsync/exec3.go:885:6: ineffectual assignment to txs (ineffassign)
b, txs = nil, nil
^
```
2022-10-26 14:41:56 +02:00
Andrew Ashikhmin
6a5968e7f0
AuRa service transactions are free ( #5873 )
...
and the don't pay baseFee after EIP-1559.
See https://openethereum.github.io/Permissioning.html#gas-price
2022-10-26 13:03:47 +02:00
Alex Sharov
7056ef07be
don't stop stageLoop if error happen during background blocks snapshot creation ( #5868 )
2022-10-26 09:43:10 +07:00
Alex Sharov
2d7cd0fae2
e3: better prune logs, apply thread can work without rwTx ( #5860 )
2022-10-25 16:08:01 +07:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live ( #5857 )
2022-10-25 09:58:25 +07:00
Alex Sharov
592ad57921
add DECOMPRESS_CONDENSITY env variable ( #5853 )
2022-10-24 10:00:55 +07:00
Alex Sharov
4ca4aa7144
e3: bsc recon leack fix ( #5852 )
2022-10-24 09:14:34 +07:00
Alex Sharov
a1cd46c5be
e3: recon metrics ( #5850 )
2022-10-23 19:31:24 +07:00
Alex Sharov
cd4ce6291f
don't accumulate state changes for jumps > 1K blocks ( #5845 )
2022-10-23 18:50:11 +07:00
Alex Sharov
a1e21260ad
e3: reduce reconstitute ram estimation ( #5848 )
2022-10-23 18:49:57 +07:00
Alex Sharov
96233cfa7f
e3: metrics ( #5846 )
2022-10-23 17:05:26 +07:00
Krishna Upadhyaya
aea019a3ff
Fix BorTxLookup ( #5800 )
2022-10-22 17:37:01 +01:00
Alex Sharov
d6de913a34
e3: fix races ( #5822 )
2022-10-21 21:02:18 +07:00
Andrew Ashikhmin
64a3156112
Support BaseFee in AuRa headers ( #5820 )
...
BaseFee is required in AuRa headers when
[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559 ) is activated.
Also:
- Basic AuRa header verification
- Extract some common RLP methods
- Tiny log clean-up
2022-10-21 12:43:44 +02:00
Alex Sharov
86551e82f4
e3: parallel build missed indices ( #5817 )
2022-10-21 13:45:20 +07:00
Giulio rebuffo
889164e847
Strenghtened safety system for LC ( #5811 )
2022-10-20 16:07:17 +02:00
Alex Sharov
0db6b9b44b
e3: move skip analysis out of the loop ( #5803 )
2022-10-20 10:39:40 +07:00
Alex Sharov
c1313a1669
e3: Close recon db earlier to free ram ( #5794 )
2022-10-19 16:45:55 +07:00
Alex Sharov
bab0aaa24d
e3: close recon db earlier ( #5793 )
2022-10-19 16:44:06 +07:00
Alex Sharov
965619ff83
e3: fix parallel exec deadlock ( #5790 )
2022-10-19 12:25:57 +07:00
Giulio rebuffo
39dd749db0
Added verbosity flag to lightclient ( #5785 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-18 12:46:48 +02:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram ( #5784 )
2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing ( #5782 )
2022-10-18 15:29:59 +07:00
Alex Sharov
b909bfa581
e3 tests ( #5774 )
2022-10-18 11:08:14 +07:00
gfx
d1e9149c93
add more sync stages to metrics ( #5740 )
...
hi!
this pr adds more sync stages to the prometheus metrics. node now shows
```
sync{stage="account_history_index"} 15742138
sync{stage="block_hashes"} 15742138
sync{stage="bodies"} 15742138
sync{stage="call_traces"} 15742138
sync{stage="cumulative_index"} 15742138
sync{stage="execution"} 15742138
sync{stage="finish"} 15742138
sync{stage="hash_state"} 15742138
sync{stage="headers"} 15742138
sync{stage="intermediate_hashes"} 15742138
sync{stage="log_index"} 15742138
sync{stage="senders"} 15742138
sync{stage="snapshots"} 15742138
sync{stage="storage_history_index"} 15742138
sync{stage="translation"} 0
sync{stage="tx_lookup"} 15742138
sync{stage="verkle_trie"} 0
sync{stage="watch_the_burn"} 0
```
Co-authored-by: a <a@a.a>
2022-10-18 10:22:57 +07:00
Alex Sharov
3aa97c57c1
fix ram estimation for indexing ( #5773 )
2022-10-18 10:19:58 +07:00
juroberttyb
64fe6708fa
Correct failed link in README.md ( #5762 )
...
[RPC calls] link path (../../cmd/rpcdaemon/Readme.md) corrected to
(../../cmd/rpcdaemon/README.md)
2022-10-18 09:23:44 +07:00
hexoscott
adadef62c8
remove estimated duration from stage execution ( #5768 )
2022-10-18 09:22:44 +07:00
J1ang
2177c8faa5
fix: issue #5723 type conversion bug ( #5763 )
...
fix:
fix for issue #5723 type conversion bug by adding check. `expValue`
should be `uint64[]`.
2022-10-17 11:25:17 +01:00
Giulio rebuffo
ab3a0b0fbd
Switched to validating lightclient ( #5761 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-16 00:07:27 +02:00
Giulio rebuffo
d075618ecc
Importing of ETH2 blocks from Lightclient ( #5760 )
2022-10-15 23:01:52 +02:00
Giulio rebuffo
6f1c2be4e5
Implemented clause 3 of phase 4 ( #5758 )
2022-10-15 19:03:31 +02:00
ledgerwatch
90d26a3f46
Fix lint ( #5756 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 10:34:12 +01:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl ( #5742 )
2022-10-15 08:20:58 +07:00
ledgerwatch
525a3a7cbb
Fix for empty blocks being proposed ( #5749 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 00:12:38 +01:00
Giulio rebuffo
497d414b36
Added Lightclient update base validation without BLS ( #5746 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-14 15:24:44 +02:00
hexoscott
e50110562a
revert execution batch size logic to size on disk ( #5732 )
...
Revert to older batch size logic to keep memory usage down during
execution phase and closer to the --batchSize flag size.
Spotted a "leak" in key generation as well. The unsafe pointer keeps the
byte slice around for as long as the batch is not committed, takes up a
fair chunk of memory surprisingly doing that so removed the unsafe
pointer usage giving the GC some chance to clean up along the way.
Moved the batch rollback into a defer func call rather than allowing
them to stack in the for loop. If this isn't going to work just let me
know and can change it back.
2022-10-13 09:10:26 +01:00
Alex Sharov
d2ef27d746
e3: reconstitution last step improve ( #5733 )
2022-10-13 14:33:39 +07:00
Alex Sharov
6cc10a27d3
e3: recondb no meminit ( #5730 )
2022-10-13 12:24:53 +07:00
Alex Sharov
1774abd6cd
e3: don't create own context, we are enough enough safe for kill ( #5728 )
2022-10-13 11:30:54 +07:00
Alex Sharov
b12b0d4627
e3: history bsc and mainnet snapshots ( #5725 )
2022-10-13 09:46:32 +07:00
Alex Sharov
d2436ab344
e3: optimize stateWriter.Put ( #5724 )
2022-10-13 09:37:14 +07:00
Alex Sharov
a9853d2ef8
Recon parallel: split ReconState to 2 objects to avoid lock contention between .Done() and .Get(), less ram in beginning ( #5713 )
2022-10-12 17:23:10 +07:00
ledgerwatch
750d23427d
Covalenthq move status execution flag into vm config ( #5710 )
...
Co-authored-by: Sudeep <sudeepdino008@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-12 09:27:48 +01:00
Alex Sharov
4795217e4e
e3: prevent files ranges overlap ( #5684 )
2022-10-12 10:18:55 +07:00
hexoscott
ab0ac1bfd4
wait for at most 500ms when receiving the GetPayload message from CL ( #5701 )
...
Seems simple enough, don't stop on interrupt but start a timer instead
and check on each loop if we should stop now or allow the block to
finish on its own
2022-10-11 16:51:53 +01:00
Andrew Ashikhmin
7286a0fef7
Create in-memory MDBX inside dirs.Tmp ( #5702 )
...
Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676 .
2022-10-11 16:49:38 +01:00
ledgerwatch
a170d730c0
Fixing issues related to sync delays ( #5689 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-11 08:04:40 +01:00
Alex Sharov
a2e51a2469
e3: prune limited amount before commit #675 ( #5693 )
2022-10-11 11:25:13 +07:00
Giulio rebuffo
a8338e4dd2
Infra refactoring ( #5673 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-09 13:41:15 +02:00
Giulio rebuffo
e36b1206f7
Resolve panic on ctx.done when using --experimental.lightclient ( #5670 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-08 15:07:04 +02:00
ledgerwatch
d4a5ddc7ee
erigon3 - fix state reconstitution ( #5666 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-07 21:40:58 +01:00
hexoscott
c829c9f4f3
back to existing request bodies logic ( #5650 )
...
Moves bodies request logic to pre OOM changes. No logging or checking
for timeouts and moving forwards with every request made. Tested on a
local node that went through the DB migration on this branch and worked
fine, caught back up to the tip. Tested on POW initial sync and looked
to be working fine writing bodies, although didn't have time to see this
run all the way through the merge.
2022-10-06 20:27:06 +01:00
Giulio rebuffo
ebe63cfa30
Added tests for config functions and further refactoring ( #5646 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-06 18:23:46 +02:00
Giulio rebuffo
2fd96a0b1c
[Experimental] GRPC Prototype with --experimental.lightclient to have Erigon listen on ConsenSUS gossip. ( #5590 )
...
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-06 11:01:56 +02:00
Alex Sharov
ae449eba4a
E3 exec atomics ( #5641 )
2022-10-06 13:11:11 +07:00
Alex Sharov
ca9aa4723c
Compress params change ( #5631 )
...
as of https://github.com/ledgerwatch/erigon-lib/pull/651
2022-10-05 17:54:54 +07:00
Alex Sharov
c48d51a582
snapshot index workers amount - based on available RAM and CPU ( #5524 )
2022-10-05 17:07:01 +07:00
Alex Sharov
3777b613f9
erigon3: bigger outCh and log it's len/cap ( #5612 )
2022-10-05 15:37:41 +07:00
ledgerwatch
94f4ea805d
Fixing hive SideChain reorg test ( #5620 )
...
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)
so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-10-05 05:42:38 +01:00
ledgerwatch
92bc23bd45
Return correct LastValidHash ( #5616 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-04 13:38:47 +01:00
ledgerwatch
fd52a788b7
Fix creation of block snapshots ( #5579 )
...
* Print snapshot prune
* More print
* Print
* Print
* Print
* Move snapshots stage forward
* Cleanup
* Fix tests
* Print
* Too much logging
* Remove print
* Log, check
* Revert
* No panic, print
* Fix tx numbering
* Harder condition to start retiring blocks
* Disable Pow verification after TTD is reached
* Fix POW verifying
* Print
* Prints
* Fix?
* cleanup
* Add migrations and hack
* More diagnostics
* More print
* Reset sequence only once
* Fix migration
* Remove print reset
* Fix lint
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-10-04 11:14:18 +01:00
Alex Sharov
39383d936d
erigon3: move tx.AsMessage() out of critical path ( #5611 )
...
* save
* save
2022-10-04 10:56:41 +01:00
hexoscott
b581bf30c0
Reintroduce oom fix ( #5604 )
...
* bring back oom fix for bodies with some new tweaks
* use in memory body cache when processing stages in memory
2022-10-04 10:44:23 +01:00
Alex Sharov
a03c43fe3c
erigon3: allow commit in parallel exec #5614
2022-10-04 13:51:30 +07:00
Alex Sharov
ede306baa5
erigon3: no overlap parallel tx
2022-10-04 13:40:27 +07:00
Enrique Jose Avila Asapche
fce9a665bb
warning of missing block and returning nil if no err ( #5606 )
...
* warning of missing block and returning nil if no err
* deleted message
2022-10-04 09:00:27 +07:00
Alex Sharov
2b4046f372
erigon3 recon fix overlap txs #5609
2022-10-04 08:59:35 +07:00
Alex Sharov
20f50da285
r.cache = prevView.cache.Clone()
2022-10-03 09:50:17 +07:00
Alex Sharov
fba6efb1c8
erigon3: a bit simplify reconstitution #5599
2022-10-02 20:37:28 +07:00
Alex Sharov
fda2748447
erigon3: no overlap tx, better defer rollback ( #5598 )
2022-10-02 19:54:26 +07:00
Alex Sharov
c38a9dcfab
Erigon3: don't commit in the middle of block execution ( #5596 )
2022-10-02 16:26:56 +07:00
Alex Sharov
fa16d5e855
erigon3: step toward background snapshots build ( #5588 )
2022-10-02 10:21:17 +07:00
Alex Sharov
1b17a3226a
agg print stats at startup ( #5586 )
2022-10-01 09:29:28 +07:00
ledgerwatch
7018c5e746
Ergonomic fixes for POS ( #5574 )
...
* Various POS fixes
* More quiet
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-29 22:17:16 +01:00
ledgerwatch
49aeb53988
Fail newPayload if execution is unsuccessful ( #5553 )
...
* Fail newPayload if execution is unsuccessful
* fix linter
* fix message
* no self-hosted
* try force ci
* back to unbuntu
* Remove unnecessary unwinds
* Fix canonicalising
* Quiet logs for in-memory exec
* Remove experimental overlay flag
* Fix sync test
* Fix test
* Fix lint
* Print hash in the end of the cycle
* Not notify headers if list is empty
* Supress timings and tables logs when no change in progress
* Reduce logging
* fix
* better log for pos download
* Better newPayload and forkChoice messages
* Fix lint
* simplify
* No duplication of download messages
* Reverse log for extension
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-09-29 06:38:13 +01:00
Alex Sharov
f4a7d2cf6d
erigon3: build .vi after download ( #5568 )
2022-09-29 12:14:51 +07:00
Alex Sharov
e4ad73bcb4
erigon3: agg to use workers for compress and merge ( #5561 )
2022-09-28 13:05:26 +07:00
ledgerwatch
11893429f5
cleanup bloombits and filters ( #5551 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-27 17:19:59 +01:00
Enrique Jose Avila Asapche
a968c7ed5b
cleaned up storage and retrieval of state sync receipts ( #5379 )
...
* cleaned up storage and retrieval of state sync receipts
* added ctx to the error and got rid of unused var
* missing one
* small change
* log error
* ops
* read header number
* checking if status is succesful
* nil ptr err
2022-09-27 11:39:58 +01:00
Alex Sharov
fb44051a82
erigon3: move temporary db creation inside Recon22 ( #5533 )
2022-09-27 10:41:30 +07:00
Max Revitt
b18e9beefc
feat(log): log message when no blocks written ( #5526 )
...
both in headers and bodies stages this adds a separate message
showing that in a given log period no headers/blocks were processed
2022-09-26 17:46:54 +01:00
ledgerwatch
c094a2b960
Revert "OOM fix for bodies stage ( #5253 )" ( #5525 )
...
This reverts commit 7fc7b91270
2022-09-26 15:27:38 +01:00
Alex Sharov
da7ca3cacf
erigon3: build .efi after download #654 ( #5523 )
2022-09-26 15:51:39 +07:00
Alex Sharov
cb60382e6d
erigon3: rename "history.v2" to "history.v3" to avoid naming miss-match with "erigon3" ( #5519 )
2022-09-26 10:54:42 +07:00
Alex Sharov
cadc9dfe74
erigon3: fix conflict with innner_tx in stage_bodies #5518
2022-09-26 10:40:23 +07:00
Alex Sharov
72a4225478
erigon22: reconstitute optimize #5505 ( #5505 )
2022-09-26 09:13:44 +07:00
Alex Sharov
fdb83de6f5
erigon22: authapi to use aggregator22 ( #5506 )
...
* save
* save
2022-09-26 09:09:30 +07:00
ledgerwatch
8b65d45499
Fix state reconstitution after change of txNums semantics ( #5484 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-23 08:46:25 +07:00
Giulio rebuffo
11db9f4ea4
Added Verkle tree based stagedsync as a prototype (Activates if CancunBlock == 0) ( #5483 )
...
* added prototype of verkle tree stages in Erigon, activation set at Cancun
* ops
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-09-22 23:29:44 +02:00
Andrew Ashikhmin
ebaf9ede82
Fix AuRa genesis block reading ( #5479 )
...
* Fix Gnosis genesis block reading
* Proper HeaderSeal fix
2022-09-22 17:38:05 +02:00
Andrew Ashikhmin
5880e5e4bb
Block builder: bump maxTransactions to 1000 ( #5475 )
2022-09-22 13:46:19 +02:00
Alex Sharov
fb0f811f19
erigon22: "max txNum of a block N" semantic #5465
2022-09-22 15:38:16 +07:00
hexoscott
7fc7b91270
OOM fix for bodies stage ( #5253 )
...
* OOM fix for bodies stage
change misleading bodies stage log messages
remove duplicate func to create key from block number
ensure body isn't nil when reading from bucket in bodies stage
remove unused blocknum variable from body requests
moving to using RLP encoding for stage bodies storage
encode bodies to RLP for stage bodies storage
better error handling in stage bodies
fixup body algos after rebase
* fixes for body stage oom after rebase
2022-09-22 12:27:42 +07:00
hexoscott
705673c336
increased logging around POS mining ( #5442 )
2022-09-20 13:15:56 +01:00
Alex Sharov
5ccc039497
Test e22 incarnation ( #5436 )
2022-09-20 11:10:31 +07:00
Alex Sharov
db622cba4a
remove async snapshots interator ( #5429 )
2022-09-20 09:36:48 +07:00
Andrew Ashikhmin
05df5a4739
Revert "Docker pos fixup attempt1 ( #5401 )" ( #5421 )
...
This reverts commit 43c848ceb9
2022-09-19 16:13:20 +02:00
Alex Sharov
c1bb4bc734
exec22: stagedsync test v1 ( #5425 )
...
* save
* save
2022-09-19 12:17:36 +07:00
Alex Sharov
9fb8a190bc
erigon22: folder snapshots/history ( #5351 )
2022-09-18 17:41:01 +07:00
Enrique Jose Avila Asapche
7b1bf44ce6
dont check for block reward or uncles if we are tracing POS ( #5410 )
...
* dont check for block reward or uncles if we are tracing POS
* comparing chainConfig to header difficulty
* fixed test
* difficulty == 0 then we are POS
2022-09-17 22:04:37 +01:00
Enrique Jose Avila Asapche
b8a8ea6ace
Log stage finish ( #5409 )
...
* logging latest block at stage finish
* better logging debug
2022-09-17 12:54:18 +01:00
ledgerwatch
43f592ed0e
Fixes for trace_block ( #5402 )
...
* Fixes for trace_block
* Print
* Print
* extra bodies
* extra bodies
* Add canonical check
* First fix for bodies mismatch
* More cleanup
* Advance progress
* Reset snapshots in integration
* Cleanup
* Way to reset snapshots stage
* Not reset sequence if not needed
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-17 12:53:27 +01:00
Giulio rebuffo
43c848ceb9
Docker pos fixup attempt1 ( #5401 )
...
* maybe fix
* maybe fix
* fix test sentry_mock
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-09-17 11:49:13 +02:00
Andrew Ashikhmin
a8a104c35c
Merge & Downloader logging improvements ( #5398 )
...
* Prefix downloader logs with [Downloader]
* Better check for syncing
* Better handle nil dereference
2022-09-16 15:15:48 +02:00
Andrew Ashikhmin
4ef182e6e0
Tiny log improvement ( #5394 )
2022-09-16 12:19:44 +02:00
ledgerwatch
caeba44728
[erigon22] fix state reconstitution (testing on BSC) ( #5391 )
...
* Fix NPE in Parlia integration
* Fix state reconstitution for integration
* Update to latest erigon-lib
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-16 10:41:01 +01:00
Andrew Ashikhmin
e5afa7292e
Potential improvement to CL timeouts ( #5378 )
...
* Strengthen early return when PoS syncing
* Be less aggressive about new payloads
* Less scary log message
* Clean up LVH in EngineStatus_SYNCING
* Slightly more correct message
* Try to fix Hive tests
2022-09-16 10:59:46 +02:00
Alex Sharov
b8e81f9b40
erigon22: save correct parallel progress #5388
2022-09-16 11:52:02 +07:00
Enrique Jose Avila Asapche
390108f352
pruning bor receipts with receipts ( #5371 )
...
* pruning bor receipts with receipts
* no need for rawdb.DeleteBorReceipt
2022-09-16 09:39:48 +07:00
Enrique Jose Avila Asapche
fc766c65f3
added downloading to avoid user panic ( #5376 )
2022-09-15 16:57:37 +02:00
Andrew Ashikhmin
3012bf6d3c
Log more info on missing parent ( #5375 )
2022-09-15 15:56:27 +02:00
Andrew Ashikhmin
079b0d9074
Demote execution unwind message to log.Debug ( #5356 )
2022-09-13 20:36:32 +01:00
Enrique Jose Avila Asapche
0a7a45169a
Short-circuiting ( #5342 )
...
* getting latest executed no forkchoice
* short circuting
* Revert "getting latest executed no forkchoice"
This reverts commit d3061b75bd7fd441880912b38033fd60b43e9029.
* naming
* updated erigon-lib
2022-09-13 08:53:43 +07:00
Enrique Jose Avila Asapche
bbea37a454
updated tracer js ( #5347 )
...
* updated tracer js
* using SUICIDE
* fixed test
2022-09-13 08:51:07 +07:00
Enrique Jose Avila Asapche
473ca51830
Adding back stage snapshots ( #5331 )
...
* added snapshots into stages list && deleted snapshots from defaul unwind
* added verkle trie
* using the same sentries client hd
* need to safe the stage progress of headers as well
2022-09-12 10:26:00 +07:00
Alex Sharov
78ab986d50
erigon22: fix cli param parse #5337
2022-09-12 10:24:23 +07:00
Enrique Jose Avila Asapche
8632befccc
separated bor block execution ( #5330 )
2022-09-11 09:36:23 +07:00
Giulio rebuffo
99fbbb545b
Added complete verkle trie regeneration ( #5310 )
...
* verkle generation
* better UX
* q
* better verkle stuff
* for
* for o
* lint
* direct vtree
* addee verkle trees incremental promotion
* lint
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-09-10 20:00:03 +02:00
Leonard Chinonso
7495986515
Reverting Adding snapshots as a separate stage commit ( #5329 )
...
* reverting Adding snapshots as a separate stage commit
* more fixes to merge conflicts
2022-09-10 19:05:18 +02:00
Alex Sharov
d1ddb5ee82
erigon22: fix txn overlap ( #5327 )
...
* save
* save
2022-09-10 11:26:58 +07:00
Enrique Jose Avila Asapche
6d1d3a9f47
moved retiring of blocks from senders into snapshot stage ( #5290 )
2022-09-09 21:48:16 +07:00
Alex Sharov
8813ea87fd
Erigon22: add state reconstitution to stage_exec ( #5321 )
2022-09-09 14:40:25 +07:00
Alex Sharov
caa23b9532
erigon22: simplify interhashes stage ( #5315 )
2022-09-08 14:02:50 +07:00
Andrew Ashikhmin
299adf4d3e
New transactions should prompt mining instead of postponing it ( #5309 )
2022-09-07 22:17:14 +07:00
Alex Sharov
fceb272c6b
Erigon22: use history iterator ( #5303 )
2022-09-07 15:58:41 +07:00
Andrew Ashikhmin
270fb4cde9
Fix IH when state contains addresses < 1st key in AccTrie ( #5296 )
...
* Test case that reproduces wrong trie root in Hive
* Do not hardcode roots
* Fix IH when state contains addresses smaller than the first key in AccTrie
2022-09-07 10:34:49 +02:00
Max Revitt
488121f669
fix(headers): signal cleanly shutdown headers and allow exit ( #5286 )
2022-09-06 08:02:24 +01:00
Alex Sharov
f8c20afe8d
erigon22: remove hardcoded workers amount ( #5288 )
2022-09-06 13:54:16 +07:00
Enrique Jose Avila Asapche
036d6f3e1d
Cursor err ( #5283 )
...
* deleting BorTxLookUp separetly
* logs
* another place
* typo
* full separation of borTxnLookup and using next
* not checking if we have tx
2022-09-06 08:45:24 +07:00
Leonard Chinonso
0543ba6120
Adding a new stage for snapshots download and creation ( #5261 )
...
* Adding a new stage for snapshots download and creation
* More changes made
* finished moving things into snapshot stage
* ops
* another merge problem
Co-authored-by: Enrique Jose Avila Asapche <eavilaasapche@gmail.com>
2022-09-06 08:44:22 +07:00
Alex Sharov
25e615a878
erigon22: historyReader22 and more tests ( #5281 )
...
* save
* save
2022-09-05 21:31:00 +07:00
Enrique Jose Avila Asapche
6ce38853a2
getting rid of breaking line ( #5246 )
2022-09-05 10:42:32 +07:00
Alex Sharov
7874921e73
erigon22: correct incarnation ( #5275 )
2022-09-05 10:31:17 +07:00
Enrique Jose Avila Asapche
8fc7c78620
Bor txlookup ( #5260 )
...
* printing
* logging
* spam off
* ops
* clean up
* deleted all bor related stuff in snapshots
* adding bor tx to BorTxLookUp
* added readBorTxLookUp
* getting bor tx from bor lookup
* comments
* deleted bor related stuff
* deleting bor tx
* writing borTxLookUp in stage_txlookup
* using bytes
* little fixes
* updated erigon-lib
* using biEndian
* clean up
* setting bytes for big int
* log
* logging
* more logs
* delete logs
2022-09-02 17:41:58 +03:00
Enrique Jose Avila Asapche
68cc6e4612
taking bad transaction out the slice ( #5265 )
2022-09-02 15:26:38 +02:00
ledgerwatch
693017c554
Cleanup Tevm experimental code ( #5259 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-01 19:49:29 +01:00
ledgerwatch
73e2d1146e
Cleanup interfaces ( #5254 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-01 16:44:37 +01:00
Alex Sharov
35687c0e2c
erigon22: TestRecreateAndRewind ( #5241 )
2022-08-31 13:34:16 +07:00
Nathan (Blaise) Bruer
002aba4686
Add STOP_AFTER_STAGE environ ( #5207 )
...
Adds ability to stop after a stage instead of just before a stage.
This will allow users to use something like:
`STOP_AFTER_STAGE=Finish erigon`
And erigon will stop after a full cycle of syncing.
2022-08-31 07:46:27 +07:00
Enrique Jose Avila Asapche
12331e018a
Fix bor txlookup ( #5170 )
2022-08-30 17:23:16 +03:00
Andrew Ashikhmin
07f6bea8e3
Remove unused function parameter ( #5232 )
2022-08-30 12:07:32 +02:00
fynn.z
9c713d8168
Fixed mining system contract txn lost ( #5212 )
...
* fix mining exec error
* fix invalid memory
* remove logs
Co-authored-by: Jeff Rossiter <jeffrey.rossiter@me.com>
2022-08-30 09:47:47 +07:00
