mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-23 04:03:49 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
17,514 Commits 5 Branches 14 Tags 294 MiB
e6e07fec72
Commit Graph

2862 Commits

Author SHA1 Message Date
Alex Sharov
8c36c3fe99
e3: recon progress metric (#5902) 2022-10-29 21:51:01 +07:00
Alex Sharov
fde630f76b
e3: wal preallocate historyKey #713 (#5899) 2022-10-29 19:18:35 +07:00
Alex Sharov
8fe89f4e5b
e3: less small merges, parallel merge (#5895) 2022-10-29 13:21:06 +07:00
Alex Sharov
d8d4a0a240
e3: wait group for apply loop (#5894) 2022-10-29 11:48:53 +07:00
Andrew Ashikhmin
dd43d486f3
Enable London in AllEthashProtocolChanges (#5891) 
`AllEthashProtocolChanges` should contain all the EIPs accepted by the
core developers into the Ethash consensus, including the
[London](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md)
protocol upgrade.
 2022-10-28 17:19:18 +02:00
Alex Sharov
0074ad0e48
e3: parallel exec, rotate and partial-flush of indices wal - while main thread is idle (#5889) 
While main thread is idle (apply is on RoTx now):
- periodically still hisotry/index etl collectors and load them to db 
- prune old history/index in db

Also:
- fixed RAM estimation calculation of State22
 2022-10-28 21:13:53 +07:00
Alex Sharov
8d1ed547b0
E3: parallel exec, apply on roTx (#5879) 2022-10-28 08:47:45 +07:00
Andrew Ashikhmin
d9ff645fbf
Make lint happy (#5874) 
Fixes
```
eth/stagedsync/exec3.go:885:6: ineffectual assignment to txs (ineffassign)
        b, txs = nil, nil
           ^
```
 2022-10-26 14:41:56 +02:00
Andrew Ashikhmin
6a5968e7f0
AuRa service transactions are free (#5873) 
and the don't pay baseFee after EIP-1559.

See https://openethereum.github.io/Permissioning.html#gas-price
 2022-10-26 13:03:47 +02:00
Alex Sharov
7056ef07be
don't stop stageLoop if error happen during background blocks snapshot creation (#5868) 2022-10-26 09:43:10 +07:00
Alex Sharov
2d7cd0fae2
e3: better prune logs, apply thread can work without rwTx (#5860) 2022-10-25 16:08:01 +07:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
592ad57921
add DECOMPRESS_CONDENSITY env variable (#5853) 2022-10-24 10:00:55 +07:00
Alex Sharov
4ca4aa7144
e3: bsc recon leack fix (#5852) 2022-10-24 09:14:34 +07:00
Alex Sharov
a1cd46c5be
e3: recon metrics (#5850) 2022-10-23 19:31:24 +07:00
Alex Sharov
cd4ce6291f
don't accumulate state changes for jumps > 1K blocks (#5845) 2022-10-23 18:50:11 +07:00
Alex Sharov
a1e21260ad
e3: reduce reconstitute ram estimation (#5848) 2022-10-23 18:49:57 +07:00
Alex Sharov
96233cfa7f
e3: metrics (#5846) 2022-10-23 17:05:26 +07:00
Krishna Upadhyaya
aea019a3ff
Fix BorTxLookup (#5800) 2022-10-22 17:37:01 +01:00
Alex Sharov
d6de913a34
e3: fix races (#5822) 2022-10-21 21:02:18 +07:00
Andrew Ashikhmin
64a3156112
Support BaseFee in AuRa headers (#5820) 
BaseFee is required in AuRa headers when
[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559) is activated.

Also:
- Basic AuRa header verification
- Extract some common RLP methods
- Tiny log clean-up
 2022-10-21 12:43:44 +02:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Giulio rebuffo
889164e847
Strenghtened safety system for LC (#5811) 2022-10-20 16:07:17 +02:00
Alex Sharov
0db6b9b44b
e3: move skip analysis out of the loop (#5803) 2022-10-20 10:39:40 +07:00
Alex Sharov
c1313a1669
e3: Close recon db earlier to free ram (#5794) 2022-10-19 16:45:55 +07:00
Alex Sharov
bab0aaa24d
e3: close recon db earlier (#5793) 2022-10-19 16:44:06 +07:00
Alex Sharov
965619ff83
e3: fix parallel exec deadlock (#5790) 2022-10-19 12:25:57 +07:00
Giulio rebuffo
39dd749db0
Added verbosity flag to lightclient (#5785) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-18 12:46:48 +02:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram (#5784) 2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing (#5782) 2022-10-18 15:29:59 +07:00
Alex Sharov
b909bfa581
e3 tests (#5774) 2022-10-18 11:08:14 +07:00
gfx
d1e9149c93
add more sync stages to metrics (#5740) 
hi! 

this pr adds more sync stages to the prometheus metrics. node now shows 

```
sync{stage="account_history_index"} 15742138
sync{stage="block_hashes"} 15742138
sync{stage="bodies"} 15742138
sync{stage="call_traces"} 15742138
sync{stage="cumulative_index"} 15742138
sync{stage="execution"} 15742138
sync{stage="finish"} 15742138
sync{stage="hash_state"} 15742138
sync{stage="headers"} 15742138
sync{stage="intermediate_hashes"} 15742138
sync{stage="log_index"} 15742138
sync{stage="senders"} 15742138
sync{stage="snapshots"} 15742138
sync{stage="storage_history_index"} 15742138
sync{stage="translation"} 0
sync{stage="tx_lookup"} 15742138
sync{stage="verkle_trie"} 0
sync{stage="watch_the_burn"} 0
```

Co-authored-by: a <a@a.a>
 2022-10-18 10:22:57 +07:00
Alex Sharov
3aa97c57c1
fix ram estimation for indexing (#5773) 2022-10-18 10:19:58 +07:00
juroberttyb
64fe6708fa
Correct failed link in README.md (#5762) 
[RPC calls] link path (../../cmd/rpcdaemon/Readme.md) corrected to
(../../cmd/rpcdaemon/README.md)
 2022-10-18 09:23:44 +07:00
hexoscott
adadef62c8
remove estimated duration from stage execution (#5768) 2022-10-18 09:22:44 +07:00
J1ang
2177c8faa5
fix: issue #5723 type conversion bug (#5763) 
fix:
fix for issue #5723 type conversion bug by adding check. `expValue`
should be `uint64[]`.
 2022-10-17 11:25:17 +01:00
Giulio rebuffo
ab3a0b0fbd
Switched to validating lightclient (#5761) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-16 00:07:27 +02:00
Giulio rebuffo
d075618ecc
Importing of ETH2 blocks from Lightclient (#5760) 2022-10-15 23:01:52 +02:00
Giulio rebuffo
6f1c2be4e5
Implemented clause 3 of phase 4 (#5758) 2022-10-15 19:03:31 +02:00
ledgerwatch
90d26a3f46
Fix lint (#5756) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-15 10:34:12 +01:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl (#5742) 2022-10-15 08:20:58 +07:00
ledgerwatch
525a3a7cbb
Fix for empty blocks being proposed (#5749) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-15 00:12:38 +01:00
Giulio rebuffo
497d414b36
Added Lightclient update base validation without BLS (#5746) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-14 15:24:44 +02:00
hexoscott
e50110562a
revert execution batch size logic to size on disk (#5732) 
Revert to older batch size logic to keep memory usage down during
execution phase and closer to the --batchSize flag size.

Spotted a "leak" in key generation as well. The unsafe pointer keeps the
byte slice around for as long as the batch is not committed, takes up a
fair chunk of memory surprisingly doing that so removed the unsafe
pointer usage giving the GC some chance to clean up along the way.

Moved the batch rollback into a defer func call rather than allowing
them to stack in the for loop. If this isn't going to work just let me
know and can change it back.
 2022-10-13 09:10:26 +01:00
Alex Sharov
d2ef27d746
e3: reconstitution last step improve (#5733) 2022-10-13 14:33:39 +07:00
Alex Sharov
6cc10a27d3
e3: recondb no meminit (#5730) 2022-10-13 12:24:53 +07:00
Alex Sharov
1774abd6cd
e3: don't create own context, we are enough enough safe for kill (#5728) 2022-10-13 11:30:54 +07:00
Alex Sharov
b12b0d4627
e3: history bsc and mainnet snapshots (#5725) 2022-10-13 09:46:32 +07:00
Alex Sharov
d2436ab344
e3: optimize stateWriter.Put (#5724) 2022-10-13 09:37:14 +07:00
Alex Sharov
a9853d2ef8
Recon parallel: split ReconState to 2 objects to avoid lock contention between .Done() and .Get(), less ram in beginning (#5713) 2022-10-12 17:23:10 +07:00
ledgerwatch
750d23427d
Covalenthq move status execution flag into vm config (#5710) 
Co-authored-by: Sudeep <sudeepdino008@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-12 09:27:48 +01:00
Alex Sharov
4795217e4e
e3: prevent files ranges overlap (#5684) 2022-10-12 10:18:55 +07:00
hexoscott
ab0ac1bfd4
wait for at most 500ms when receiving the GetPayload message from CL (#5701) 
Seems simple enough, don't stop on interrupt but start a timer instead
and check on each loop if we should stop now or allow the block to
finish on its own
 2022-10-11 16:51:53 +01:00
Andrew Ashikhmin
7286a0fef7
Create in-memory MDBX inside dirs.Tmp (#5702) 
Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.

Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
 2022-10-11 16:49:38 +01:00
ledgerwatch
a170d730c0
Fixing issues related to sync delays (#5689) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-11 08:04:40 +01:00
Alex Sharov
a2e51a2469
e3: prune limited amount before commit #675 (#5693) 2022-10-11 11:25:13 +07:00
Giulio rebuffo
a8338e4dd2
Infra refactoring (#5673) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-09 13:41:15 +02:00
Giulio rebuffo
e36b1206f7
Resolve panic on ctx.done when using --experimental.lightclient (#5670) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-08 15:07:04 +02:00
ledgerwatch
d4a5ddc7ee
erigon3 - fix state reconstitution (#5666) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-07 21:40:58 +01:00
hexoscott
c829c9f4f3
back to existing request bodies logic (#5650) 
Moves bodies request logic to pre OOM changes. No logging or checking
for timeouts and moving forwards with every request made. Tested on a
local node that went through the DB migration on this branch and worked
fine, caught back up to the tip. Tested on POW initial sync and looked
to be working fine writing bodies, although didn't have time to see this
run all the way through the merge.
 2022-10-06 20:27:06 +01:00
Giulio rebuffo
ebe63cfa30
Added tests for config functions and further refactoring (#5646) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-06 18:23:46 +02:00
Giulio rebuffo
2fd96a0b1c
[Experimental] GRPC Prototype with --experimental.lightclient to have Erigon listen on ConsenSUS gossip. (#5590) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-10-06 11:01:56 +02:00
Alex Sharov
ae449eba4a
E3 exec atomics (#5641) 2022-10-06 13:11:11 +07:00
Alex Sharov
ca9aa4723c
Compress params change (#5631) 
as of https://github.com/ledgerwatch/erigon-lib/pull/651
 2022-10-05 17:54:54 +07:00
Alex Sharov
c48d51a582
snapshot index workers amount - based on available RAM and CPU (#5524) 2022-10-05 17:07:01 +07:00
Alex Sharov
3777b613f9
erigon3: bigger outCh and log it's len/cap (#5612) 2022-10-05 15:37:41 +07:00
ledgerwatch
94f4ea805d
Fixing hive SideChain reorg test (#5620) 
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)

so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-10-05 05:42:38 +01:00
ledgerwatch
92bc23bd45
Return correct LastValidHash (#5616) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-10-04 13:38:47 +01:00
ledgerwatch
fd52a788b7
Fix creation of block snapshots (#5579) 
* Print snapshot prune

* More print

* Print

* Print

* Print

* Move snapshots stage forward

* Cleanup

* Fix tests

* Print

* Too much logging

* Remove print

* Log, check

* Revert

* No panic, print

* Fix tx numbering

* Harder condition to start retiring blocks

* Disable Pow verification after TTD is reached

* Fix POW verifying

* Print

* Prints

* Fix?

* cleanup

* Add migrations and hack

* More diagnostics

* More print

* Reset sequence only once

* Fix migration

* Remove print reset

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-10-04 11:14:18 +01:00
Alex Sharov
39383d936d
erigon3: move tx.AsMessage() out of critical path (#5611) 
* save

* save
 2022-10-04 10:56:41 +01:00
hexoscott
b581bf30c0
Reintroduce oom fix (#5604) 
* bring back oom fix for bodies with some new tweaks

* use in memory body cache when processing stages in memory
 2022-10-04 10:44:23 +01:00
Alex Sharov
a03c43fe3c
erigon3: allow commit in parallel exec #5614 2022-10-04 13:51:30 +07:00
Alex Sharov
ede306baa5
erigon3: no overlap parallel tx 2022-10-04 13:40:27 +07:00
Enrique Jose Avila Asapche
fce9a665bb
warning of missing block and returning nil if no err (#5606) 
* warning of missing block and returning nil if no err

* deleted message
 2022-10-04 09:00:27 +07:00
Alex Sharov
2b4046f372
erigon3 recon fix overlap txs #5609 2022-10-04 08:59:35 +07:00
Alex Sharov
20f50da285
r.cache = prevView.cache.Clone() 2022-10-03 09:50:17 +07:00
Alex Sharov
fba6efb1c8
erigon3: a bit simplify reconstitution #5599 2022-10-02 20:37:28 +07:00
Alex Sharov
fda2748447
erigon3: no overlap tx, better defer rollback (#5598) 2022-10-02 19:54:26 +07:00
Alex Sharov
c38a9dcfab
Erigon3: don't commit in the middle of block execution (#5596) 2022-10-02 16:26:56 +07:00
Alex Sharov
fa16d5e855
erigon3: step toward background snapshots build (#5588) 2022-10-02 10:21:17 +07:00
Alex Sharov
1b17a3226a
agg print stats at startup (#5586) 2022-10-01 09:29:28 +07:00
ledgerwatch
7018c5e746
Ergonomic fixes for POS (#5574) 
* Various POS fixes

* More quiet

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-29 22:17:16 +01:00
ledgerwatch
49aeb53988
Fail newPayload if execution is unsuccessful (#5553) 
* Fail newPayload if execution is unsuccessful

* fix linter

* fix message

* no self-hosted

* try force ci

* back to unbuntu

* Remove unnecessary unwinds

* Fix canonicalising

* Quiet logs for in-memory exec

* Remove experimental overlay flag

* Fix sync test

* Fix test

* Fix lint

* Print hash in the end of the cycle

* Not notify headers if list is empty

* Supress timings and tables logs when no change in progress

* Reduce logging

* fix

* better log for pos download

* Better newPayload and forkChoice messages

* Fix lint

* simplify

* No duplication of download messages

* Reverse log for extension

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
 2022-09-29 06:38:13 +01:00
Alex Sharov
f4a7d2cf6d
erigon3: build .vi after download (#5568) 2022-09-29 12:14:51 +07:00
Alex Sharov
e4ad73bcb4
erigon3: agg to use workers for compress and merge (#5561) 2022-09-28 13:05:26 +07:00
ledgerwatch
11893429f5
cleanup bloombits and filters (#5551) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-27 17:19:59 +01:00
Enrique Jose Avila Asapche
a968c7ed5b
cleaned up storage and retrieval of state sync receipts (#5379) 
* cleaned up storage and retrieval of state sync receipts

* added ctx to the error and got rid of unused var

* missing one

* small change

* log error

* ops

* read header number

* checking if status is succesful

* nil ptr err
 2022-09-27 11:39:58 +01:00
Alex Sharov
fb44051a82
erigon3: move temporary db creation inside Recon22 (#5533) 2022-09-27 10:41:30 +07:00
Max Revitt
b18e9beefc
feat(log): log message when no blocks written (#5526) 
both in headers and bodies stages this adds a separate message
showing that in a given log period no headers/blocks were processed
 2022-09-26 17:46:54 +01:00
ledgerwatch
c094a2b960
Revert "OOM fix for bodies stage (#5253)" (#5525) 
This reverts commit 7fc7b91270.

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-26 15:27:38 +01:00
Alex Sharov
da7ca3cacf
erigon3: build .efi after download #654 (#5523) 2022-09-26 15:51:39 +07:00
Alex Sharov
cb60382e6d
erigon3: rename "history.v2" to "history.v3" to avoid naming miss-match with "erigon3" (#5519) 2022-09-26 10:54:42 +07:00
Alex Sharov
cadc9dfe74
erigon3: fix conflict with innner_tx in stage_bodies #5518 2022-09-26 10:40:23 +07:00
Alex Sharov
72a4225478
erigon22: reconstitute optimize #5505 (#5505) 2022-09-26 09:13:44 +07:00
Alex Sharov
fdb83de6f5
erigon22: authapi to use aggregator22 (#5506) 
* save

* save
 2022-09-26 09:09:30 +07:00
ledgerwatch
8b65d45499
Fix state reconstitution after change of txNums semantics (#5484) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-23 08:46:25 +07:00
Giulio rebuffo
11db9f4ea4
Added Verkle tree based stagedsync as a prototype (Activates if CancunBlock == 0) (#5483) 
* added prototype of verkle tree stages in Erigon, activation set at Cancun

* ops

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-09-22 23:29:44 +02:00
Andrew Ashikhmin
ebaf9ede82
Fix AuRa genesis block reading (#5479) 
* Fix Gnosis genesis block reading

* Proper HeaderSeal fix
 2022-09-22 17:38:05 +02:00
Andrew Ashikhmin
5880e5e4bb
Block builder: bump maxTransactions to 1000 (#5475) 2022-09-22 13:46:19 +02:00
Alex Sharov
fb0f811f19
erigon22: "max txNum of a block N" semantic #5465 2022-09-22 15:38:16 +07:00
hexoscott
7fc7b91270
OOM fix for bodies stage (#5253) 
* OOM fix for bodies stage


change misleading bodies stage log messages


remove duplicate func to create key from block number


ensure body isn't nil when reading from bucket in bodies stage


remove unused blocknum variable from body requests


moving to using RLP encoding for stage bodies storage


encode bodies to RLP for stage bodies storage


better error handling in stage bodies


fixup body algos after rebase

* fixes for body stage oom after rebase
 2022-09-22 12:27:42 +07:00
hexoscott
705673c336
increased logging around POS mining (#5442) 2022-09-20 13:15:56 +01:00
Alex Sharov
5ccc039497
Test e22 incarnation (#5436) 2022-09-20 11:10:31 +07:00
Alex Sharov
db622cba4a
remove async snapshots interator (#5429) 2022-09-20 09:36:48 +07:00
Andrew Ashikhmin
05df5a4739
Revert "Docker pos fixup attempt1 (#5401)" (#5421) 
This reverts commit 43c848ceb9.
 2022-09-19 16:13:20 +02:00
Alex Sharov
c1bb4bc734
exec22: stagedsync test v1 (#5425) 
* save

* save
 2022-09-19 12:17:36 +07:00
Alex Sharov
9fb8a190bc
erigon22: folder snapshots/history (#5351) 2022-09-18 17:41:01 +07:00
Enrique Jose Avila Asapche
7b1bf44ce6
dont check for block reward or uncles if we are tracing POS (#5410) 
* dont check for block reward or uncles if we are tracing POS

* comparing chainConfig to header difficulty

* fixed test

* difficulty == 0 then we are POS
 2022-09-17 22:04:37 +01:00
Enrique Jose Avila Asapche
b8a8ea6ace
Log stage finish (#5409) 
* logging latest block at stage finish

* better logging debug
 2022-09-17 12:54:18 +01:00
ledgerwatch
43f592ed0e
Fixes for trace_block (#5402) 
* Fixes for trace_block

* Print

* Print

* extra bodies

* extra bodies

* Add canonical check

* First fix for bodies mismatch

* More cleanup

* Advance progress

* Reset snapshots in integration

* Cleanup

* Way to reset snapshots stage

* Not reset sequence if not needed

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-17 12:53:27 +01:00
Giulio rebuffo
43c848ceb9
Docker pos fixup attempt1 (#5401) 
* maybe fix

* maybe fix

* fix test sentry_mock

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
 2022-09-17 11:49:13 +02:00
Andrew Ashikhmin
a8a104c35c
Merge & Downloader logging improvements (#5398) 
* Prefix downloader logs with [Downloader]

* Better check for syncing

* Better handle nil dereference
 2022-09-16 15:15:48 +02:00
Andrew Ashikhmin
4ef182e6e0
Tiny log improvement (#5394) 2022-09-16 12:19:44 +02:00
ledgerwatch
caeba44728
[erigon22] fix state reconstitution (testing on BSC) (#5391) 
* Fix NPE in Parlia integration

* Fix state reconstitution for integration

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-16 10:41:01 +01:00
Andrew Ashikhmin
e5afa7292e
Potential improvement to CL timeouts (#5378) 
* Strengthen early return when PoS syncing

* Be less aggressive about new payloads

* Less scary log message

* Clean up LVH in EngineStatus_SYNCING

* Slightly more correct message

* Try to fix Hive tests
 2022-09-16 10:59:46 +02:00
Alex Sharov
b8e81f9b40
erigon22: save correct parallel progress #5388 2022-09-16 11:52:02 +07:00
Enrique Jose Avila Asapche
390108f352
pruning bor receipts with receipts (#5371) 
* pruning bor receipts with receipts

* no need for rawdb.DeleteBorReceipt
 2022-09-16 09:39:48 +07:00
Enrique Jose Avila Asapche
fc766c65f3
added downloading to avoid user panic (#5376) 2022-09-15 16:57:37 +02:00
Andrew Ashikhmin
3012bf6d3c
Log more info on missing parent (#5375) 2022-09-15 15:56:27 +02:00
Andrew Ashikhmin
079b0d9074
Demote execution unwind message to log.Debug (#5356) 2022-09-13 20:36:32 +01:00
Enrique Jose Avila Asapche
0a7a45169a
Short-circuiting (#5342) 
* getting latest executed no forkchoice

* short circuting

* Revert "getting latest executed no forkchoice"

This reverts commit d3061b75bd7fd441880912b38033fd60b43e9029.

* naming

* updated erigon-lib
 2022-09-13 08:53:43 +07:00
Enrique Jose Avila Asapche
bbea37a454
updated tracer js (#5347) 
* updated tracer js

* using SUICIDE

* fixed test
 2022-09-13 08:51:07 +07:00
Enrique Jose Avila Asapche
473ca51830
Adding back stage snapshots (#5331) 
* added snapshots into stages list && deleted snapshots from defaul unwind

* added verkle trie

* using the same sentries client hd

* need to safe the stage progress of headers as well
 2022-09-12 10:26:00 +07:00
Alex Sharov
78ab986d50
erigon22: fix cli param parse #5337 2022-09-12 10:24:23 +07:00
Enrique Jose Avila Asapche
8632befccc
separated bor block execution (#5330) 2022-09-11 09:36:23 +07:00
Giulio rebuffo
99fbbb545b
Added complete verkle trie regeneration (#5310) 
* verkle generation

* better UX

* q

* better verkle stuff

* for

* for o

* lint

* direct vtree

* addee verkle trees incremental promotion

* lint

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-09-10 20:00:03 +02:00
Leonard Chinonso
7495986515
Reverting Adding snapshots as a separate stage commit (#5329) 
* reverting Adding snapshots as a separate stage commit

* more fixes to merge conflicts
 2022-09-10 19:05:18 +02:00
Alex Sharov
d1ddb5ee82
erigon22: fix txn overlap (#5327) 
* save

* save
 2022-09-10 11:26:58 +07:00
Enrique Jose Avila Asapche
6d1d3a9f47
moved retiring of blocks from senders into snapshot stage (#5290) 2022-09-09 21:48:16 +07:00
Alex Sharov
8813ea87fd
Erigon22: add state reconstitution to stage_exec (#5321) 2022-09-09 14:40:25 +07:00
Alex Sharov
caa23b9532
erigon22: simplify interhashes stage (#5315) 2022-09-08 14:02:50 +07:00
Andrew Ashikhmin
299adf4d3e
New transactions should prompt mining instead of postponing it (#5309) 2022-09-07 22:17:14 +07:00
Alex Sharov
fceb272c6b
Erigon22: use history iterator (#5303) 2022-09-07 15:58:41 +07:00
Andrew Ashikhmin
270fb4cde9
Fix IH when state contains addresses < 1st key in AccTrie (#5296) 
* Test case that reproduces wrong trie root in Hive

* Do not hardcode roots

* Fix IH when state contains addresses smaller than the first key in AccTrie
 2022-09-07 10:34:49 +02:00
Max Revitt
488121f669
fix(headers): signal cleanly shutdown headers and allow exit (#5286) 2022-09-06 08:02:24 +01:00
Alex Sharov
f8c20afe8d
erigon22: remove hardcoded workers amount (#5288) 2022-09-06 13:54:16 +07:00
Enrique Jose Avila Asapche
036d6f3e1d
Cursor err (#5283) 
* deleting BorTxLookUp separetly

* logs

* another place

* typo

* full separation of borTxnLookup and using next

* not checking if we have tx
 2022-09-06 08:45:24 +07:00
Leonard Chinonso
0543ba6120
Adding a new stage for snapshots download and creation (#5261) 
* Adding a new stage for snapshots download and creation

* More changes made

* finished moving things into snapshot stage

* ops

* another merge problem

Co-authored-by: Enrique Jose Avila Asapche <eavilaasapche@gmail.com>
 2022-09-06 08:44:22 +07:00
Alex Sharov
25e615a878
erigon22: historyReader22 and more tests (#5281) 
* save

* save
 2022-09-05 21:31:00 +07:00
Enrique Jose Avila Asapche
6ce38853a2
getting rid of breaking line (#5246) 2022-09-05 10:42:32 +07:00
Alex Sharov
7874921e73
erigon22: correct incarnation (#5275) 2022-09-05 10:31:17 +07:00
Enrique Jose Avila Asapche
8fc7c78620
Bor txlookup (#5260) 
* printing

* logging

* spam off

* ops

* clean up

* deleted all bor related stuff in snapshots

* adding bor tx to BorTxLookUp

* added readBorTxLookUp

* getting bor tx from bor lookup

* comments

* deleted bor related stuff

* deleting bor tx

* writing borTxLookUp in stage_txlookup

* using bytes

* little fixes

* updated erigon-lib

* using biEndian

* clean up

* setting bytes for big int

* log

* logging

* more logs

* delete logs
 2022-09-02 17:41:58 +03:00
Enrique Jose Avila Asapche
68cc6e4612
taking bad transaction out the slice (#5265) 2022-09-02 15:26:38 +02:00
ledgerwatch
693017c554
Cleanup Tevm experimental code (#5259) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-01 19:49:29 +01:00
ledgerwatch
73e2d1146e
Cleanup interfaces (#5254) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-09-01 16:44:37 +01:00
Alex Sharov
35687c0e2c
erigon22: TestRecreateAndRewind (#5241) 2022-08-31 13:34:16 +07:00
Nathan (Blaise) Bruer
002aba4686
Add STOP_AFTER_STAGE environ (#5207) 
Adds ability to stop after a stage instead of just before a stage.
This will allow users to use something like:
`STOP_AFTER_STAGE=Finish erigon`
And erigon will stop after a full cycle of syncing.
 2022-08-31 07:46:27 +07:00
Enrique Jose Avila Asapche
12331e018a
Fix bor txlookup (#5170) 2022-08-30 17:23:16 +03:00
Andrew Ashikhmin
07f6bea8e3
Remove unused function parameter (#5232) 2022-08-30 12:07:32 +02:00
fynn.z
9c713d8168
Fixed mining system contract txn lost (#5212) 
* fix mining exec error

* fix invalid memory

* remove logs

Co-authored-by: Jeff Rossiter <jeffrey.rossiter@me.com>
 2022-08-30 09:47:47 +07:00
Andrew Ashikhmin
7e0f6e0864
Small logging improvements (#5217) 
* More logging to debug invalid blocks

* typo

* More logging consistency

* Print mismatched receipt hash

* Fix logging

* Improve log consistency

* Don't ignore errors in txn.Sender

* Log txn sender as well

* Log txn receipt as well

* Revert log level to debug
 2022-08-29 15:26:36 +02:00
Andrew Ashikhmin
94d14dd248
BaseFee is nil prior to London hard fork (#5213) 2022-08-29 15:12:31 +02:00
Alex Sharov
01f2965d8e
erigon22: MakeCanonical must update txNums (#5211) 2022-08-29 11:44:31 +07:00
Giulio rebuffo
2c43ffe910
Fixed Invalid block production with invalid txpool txs. (#5205) 
* updated mining

* print writer

* stageloop

* removed logs

* mining exec

* preemptive stuff

* create and execution

* demote to Debug

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-27 12:18:33 +02:00
Alex Sharov
a0af80014c
erigon22: rename historyReader22 to historyReader23 (#5202) 2022-08-27 15:21:02 +07:00
Alex Sharov
d33096eaa7
erigon22: tests support (#5196) 2022-08-26 16:54:54 +07:00
Alex Sharov
def681e3db
ctrl+c execution #5194 2022-08-26 16:32:16 +07:00
Andrew Ashikhmin
62bb2c47f7
Disable PoW sync for networks that have already passed through the Merge (#5172) 
* Disable PoW sync for networks that have already passed through the Merge

* Fix FirstPoSHeight
 2022-08-26 09:16:39 +02:00
Håvard Anda Estensen
7c15ed59e4
Enable prealloc linter (#5177) 
* Enable prealloc linter

* Set inital slice len to 0
 2022-08-26 10:04:36 +07:00
Alex Sharov
6371cd7b5b
erigon22: disable logIndex, historyIndex, callTracesIndex stages in --history.v2=true mode 2022-08-25 17:17:20 +07:00
Alex Sharov
69f3e1e99a
erigon22: step toward /tests 2022-08-25 15:32:05 +07:00
Alex Sharov
fb2294d124
Erigon22: basic txNum forward/unwind (#5176) 2022-08-25 12:24:01 +07:00
Alex Sharov
a6d92130a6
snapshots: check gap between snapshots and db at Erigon startup (after download confirmation from Downloader) #5174 2022-08-25 09:08:31 +07:00
Andrew Ashikhmin
e40170ebd1
Correct classification of errors in addTransactionsToMiningBlock (#5171) 2022-08-24 16:44:38 +02:00
Andrew Ashikhmin
b3d99f8e8c
Small cleanups (#5162) 2022-08-24 11:35:39 +02:00
Alex Sharov
07a9dc9db0
up metrics version (#5160) 
* save

* save
 2022-08-24 11:49:58 +07:00
Alex Sharov
16d02c8cc9
erigon22: unwind trie support (#5159) 2022-08-24 11:37:58 +07:00
Enrique Jose Avila Asapche
6b08174f30
predict download time left (#5152) 
* predict download time left

* ops
 2022-08-24 10:23:46 +07:00
Alex Sharov
69fbfc9bef
Snapshots: Erigon update list in db only after Downloader confirmation, rpcdaemon read list from db (#5150) 2022-08-23 16:28:07 +07:00
Andrew Ashikhmin
41c7be5e85
Mining: restore gasPool for invalid transactions (#5140) 2022-08-23 13:12:12 +07:00
Alex Sharov
316ed48427
erigon22: incremental state root calcc (#5132) 2022-08-22 15:52:23 +07:00
Giulio rebuffo
1303023cb8
Optimized backend pre-checks and fixed FCU deadlock (#5124) 
* Optimized backend pre-checks and fixed FCU deadlock

* fixed tests

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-20 16:27:51 +02:00
Enrique Jose Avila Asapche
d794420db8
nil ptr (#5119) 2022-08-19 14:51:14 +07:00
fynn.z
a4fb414345
fix channel hang when close in-flight sealing task (#5118) 2022-08-19 14:50:39 +07:00
Alex Sharov
c1214e4ac5
erigon22: incremental stage trie (#5116) 2022-08-19 13:23:23 +07:00
Alex Sharov
30324fbb81
erigon22: get acc index from aggregator22 (#5114) 2022-08-19 11:30:59 +07:00
Alex Sharov
46ef7e5305
erigon22: incremental hash state stage (#5102) 2022-08-19 09:35:43 +07:00
fynn.z
82fb73a545
Fix in-flight sealing task not abort when next sealing task coming (#5103) 
* Fix in-flight sealing task not abort when next sealing task coming, may cause lightclient state verify failed.

* fix integretion err
 2022-08-18 18:35:30 +07:00
Alex Sharov
3f20363da4
erigon22: fix storage cleanup on unwind (#5088) 2022-08-17 16:24:11 +07:00
fynn.z
597d1e5a64
fix panic by parlia consensus (#5078) 2022-08-16 19:16:16 +07:00
Enrique Jose Avila Asapche
5c05ebcbeb
No refetching Header (#5067) 
* no re-fetching of header

* checking for empty header
 2022-08-16 18:10:49 +07:00
Alex Sharov
8083aaa013
erigon22: save correct stage progress (#5074) 2022-08-16 15:58:35 +07:00
Håvard Anda Estensen
13322893cb
Enable gosimple linter (#5071) 
* Run gofmt

* Simplify slice making

* Use simple channel to receive instead of select with single case

* Enable gosimple linter
 2022-08-16 13:02:13 +07:00
Enrique Jose Avila Asapche
6a61a68ae4
Fix trie hash nil ptr (#5055) 
* reading headers from snapshots

* ops

* verifying snapshots if missing header

* no report if missing header

* Revert "no report if missing header"

This reverts commit b644c0ffc82ab47c567fc0c079bb7c5c32e28ca8.

* Revert "verifying snapshots if missing header"

This reverts commit ad1a9c0c4a12dd1773e05fca1a6a5e58e33592b5.

* warning if header not found
 2022-08-15 23:32:00 +07:00
Reinhard Schu
c30ee205f8
Update README.md (#5063) 
correct stage numbers in heading
 2022-08-15 18:47:29 +07:00
Alex Sharov
6df6640868
erigon22: prune - check key existance (#5059) 2022-08-15 14:36:57 +07:00
Alex Sharov
1a1b606146
erigon22: unwind code (#5058) 2022-08-15 10:27:32 +07:00
Reinhard Schu
7009a2cbb2
Update README.md (#5052) 
Update the description of the stages to align with current alpha version
 2022-08-14 20:32:37 +07:00
Alex Sharov
6495ec6d28
erigon22: working PlainState unwind (#5051) 2022-08-14 18:06:32 +07:00
Alex Sharov
a8cbf6e476
Open erigon_watchTheBurn: could not find block header (#5050) 2022-08-14 17:52:42 +07:00
Giulio rebuffo
6713abb420
fixed exec22 panic (#5045) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-13 18:54:32 +01:00
Alex Sharov
52fd0d0e8b
Aggregator22.Unwind() (#5039) 
* save

* save
 2022-08-13 18:51:25 +07:00
Alex Sharov
64bc837ccf
erigon22: integration, to support parallel execution (#5037) 
* save

* save
 2022-08-13 16:53:10 +07:00
Alex Sharov
de3a602a24
erigon22: bsc fixes 2022-08-13 16:09:52 +07:00
Alex Sharov
4f007ae43e
erigon22: use chainDB instead of db22 for everything (#5034) 2022-08-13 12:08:15 +07:00
Alex Sharov
e27f669a3f
erigon2.2: save stage progress to chainDB, can run "integration stage_exec" and "state erigon22" on same datadir (#5033) 2022-08-13 11:17:24 +07:00
Giulio rebuffo
8ade569cc5
Save PoW blocks for processing received through P2P (#5015) 
* save terminal blocks

* ops

* save less PoW blocks

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-12 21:54:20 +02:00
Andrew Ashikhmin
0e55660f11
Don't log "Wrote block ..." unless we're actually writing something (#5027) 2022-08-12 22:40:27 +07:00
Alex Sharov
811f531561
erigon22: support of integration stage_exec (#5029) 2022-08-12 22:39:33 +07:00
Alex Sharov
25d85d88c5
state erigon22: use right db (#5028) 2022-08-12 21:45:09 +07:00
Alex Sharov
4594ce5ef7
erigon22: history.v2 flag, align rpcdaemon22 (#5016) 
* save

* save

* save

* save

* save
 2022-08-12 16:13:14 +07:00
Giulio rebuffo
348be13508
fixed gossip post TTD (#5012) 
* gossip all gossip

* gossip all gossip

* fixed tests

* fixed gossip post TTD

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-11 19:55:59 +02:00
Enrique Jose Avila Asapche
30df8a9765
Solving log index out of bound error (#5009) 2022-08-11 19:08:20 +03:00
Alex Sharov
a7d07ad04d
erigon22 add to exec stage (#5008) 
* save

* save

* save
 2022-08-11 21:06:41 +07:00
Alex Sharov
e786cbac24
E22 step2 (#5002) 2022-08-11 11:19:59 +07:00
Alex Sharov
f83032533f
Revert "move exec22 to package, call it from stage_exec (#5000)" (#5001) 
This reverts commit efa6dfd8ce.
 2022-08-11 11:18:23 +07:00
Alex Sharov
efa6dfd8ce
move exec22 to package, call it from stage_exec (#5000) 2022-08-11 11:17:03 +07:00
Giulio rebuffo
b09de9e0d5
[P2P] gossip all gossip (#4995) 
* gossip all gossip

* gossip all gossip

* fixed tests

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-08-10 22:08:56 +02:00
Enrique Jose Avila Asapche
816ecc56db
Get logs fix (#4992) 
* getting header number from snapshots

* Revert "getting header number from snapshots"

This reverts commit b2a899ae6b1a18d5eb769522c30caa3aebd060e7.

* using latest executed block number && snapshots

* printing bimaps

* log info

* more logs

* more logs

* logs

* ops

* nil ptr

* log topic

* more print

* ops

* storing bor logs

* got rid of the logs

* clearing buf
 2022-08-10 19:03:22 +03:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Enrique Jose Avila Asapche
832d9fd3ec
notifying header if cannonical (#4930) 
* notifying header if cannonical

* checking if header is cannonical at notification

* getting hash from rlp

* adding to headerRlp if cannonical

* declaration
 2022-08-10 11:22:11 +07:00
Alex Sharov
1b20322b60
snap: no tmp subfolder (#4940) 
* save

* save

* save

* save

* save
 2022-08-08 09:24:39 +07:00
Leonard Chinonso
43f5ab09f8
Fixed lint errors for gosec linter (#4933) 
* Fixed lint errors for gosec linter

* Changed uint64 to int64 in common/GetRandInt

* Changed naming of function
 2022-08-04 14:59:40 +01:00
Leonard Chinonso
592fe32217
Refactoring To Increase Code Quality (#4842) 
* General cleanup and fixes

* Cleaning part 2

* cleanup part 3

* Refactored staterunner to have a handler function

* Fixed linting

* Cleanup part 4

* Changed the initialization of the require package and updated .golangci.yml

* Fixed lint errors from golangci-lint

* deleted state_recon_1.go

* fixed build error

* Added ReadHeaderTimeouts for httpServer

* made changes to golangci

* Reset golangci.yml
 2022-08-04 20:23:00 +07:00
ledgerwatch
e56491c601
Erigon22 prototype without parallel execution, fix for BSC upgrade contract (#4884) 
* No parallelisation in erigon22 prototype

* Remove paralell

* tx/s

* commit

* Fix overlap

* workers argument

* Print

* Print

* Stable sort

* Print

* Init

* Remove print

* Remove print

* Remove print

* No sorting

* Cleanup

* Fix hangs

* Fix reconstitution for contract upgrades

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-08-04 13:00:46 +01:00
Enrique Jose Avila Asapche
a0da31bb1d
Transaction count fix (#4888) 
* getBlockTransactionCountByHash using snapshots

* got body from block reader

* requesting block body with transactions

* tests

* added tx amount into blocreader body

* using txAmount from body
 2022-08-04 18:49:53 +07:00
Andrew Ashikhmin
4fb27b60f9
Clean up BodyDownload.VerifyUncles (#4928) 2022-08-04 11:47:01 +02:00
Leonard Chinonso
ec67e80a8a
Fixed lint errors for gocritic linter (#4904) 2022-08-03 00:35:22 +07:00
ledgerwatch
2b517b7630
Erigon22 prototype to support BSC (#4867) 
* Reduce workers and queue size

* Add traces and event logs

* Add chain flag

* Print

* Open consensus db as read only

* Use only one engine instance

* chainDb

* Display palia error

* Not to panic

* Not to exit

* Add syscall

* Print

* Pointer to TxTask

* Not register senders

* Print

* Relax check

* Skip system transactions

* increase workerCount again

* Remove print

* Change recon

* Fix types

* Skip system txs

* Print

* No rules for genesis commit

* Print genesis

* Print|

* Print

* Print

* Update

* Fix

* Remove print

* No print

* Support contract upgrades

* Use the same dif

* Print create2

* Fix

* Fix

* Print

* Hack iterate

* Print empty vals

* Fix hack

* Code hashes

* Code hashes

* Print

* No panic

* No panic

* Fix?

* another incarnation fix

* Remove print

* Remove print

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-07-31 21:54:23 +01:00
Giulio rebuffo
3a972cc074
Removed the use of memory mutation for side-fork unwinds. (#4857) 
* done

* updated sum and mod

* error handling

* error handling

* comments

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-31 15:56:08 +02:00
nanevardanyan
de0471aace
cmd: separate auth rpc server from regular (#4822) 
* WIP: cmd: separate auth rpc server from regular

* WIP: cmd: updates after code review

* WIP: cmd: eth and engine should be predefined values for authAPI

* cmd: http enabled flag should not affect admin

* cmd: eliminate engine checks in case of non-auth

* cmd: remove engine from http.api options
 2022-07-31 12:16:19 +02:00
Andrew Ashikhmin
5dc8baee7a
Improve error handling for the non-delayed PoS downloader (#4864) 
* PoSDownloaderTip is set to headerHash, so it should be used in IsBadHeaderPoS

* Don't wait for downloader if can't schedulePoSDownload
 2022-07-31 11:58:26 +02:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777) 
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
 2022-07-28 16:57:38 +07:00
Andrew Ashikhmin
1ffa984457
Don't wait for CL in initialCycle (#4841) 2022-07-27 13:52:50 +02:00
Andrew Ashikhmin
c591266914
Small PoS clean-ups (#4834) 
* Amend comment

* More watchful sleep

* improve comment

* Cosmetics for lock/unlock

* statusRef -> statusDeref

* Comment on obsure logic
 2022-07-27 12:14:35 +02:00
Andrew Ashikhmin
b42b4e2aa2
More timeout prevention (#4839) 
* More timeout prevention

* Fix Hive tests
 2022-07-27 11:28:00 +02:00
Giulio rebuffo
039b661803
fixed timeout on CL (#4836) 
* fixed timeout

* send response immediately wen dealing with SYNCING

* send response immediately wen dealing with SYNCING

* correct lvh for ACCEPTED

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-27 09:16:26 +02:00
Giulio rebuffo
101e181c52
if PoS download block is within reach, determine VALID or INVALID (#4812) 
* if PoS download is fast enough, determine VALID or INVALID

* more acceptable absolute value

* solved comments

* remove useless

* moved check in stage_headers

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-26 15:24:02 +02:00
Andrew Ashikhmin
c1f848746d
Update consensus tests to v11 (#4724) 
* Consensus tests update 11

* Add GrayGlacier fork

* Wire currentRandom from tests into PREVRANDAO

* Serenity engine

* Support insertion of PoS blocks in MockSentry

* Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()

* Revert "Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()"

This reverts commit 903fca572be03c7de33318ce177a03a4be34927b.

* Post-merge fix

* Don't wait for Beacon Chain in tests

* Skip powToPosBlockRejection transition test

* ForkChoice in insertPoSBlocks

* Add withPosDownloader arg to MockWithEverything in order to fix TestPoSDownloader
 2022-07-26 09:35:38 +02:00
Alex Sharov
b6ea28ea80
kv.Del() remove second parameter (#4832) 
* save

* save

* save

* save

* save

* save

* save
 2022-07-26 12:47:05 +07:00
Alex Sharov
60f1adddcd
implement server kv.Snapsthos() method (#4831) 
* save

* save
 2022-07-26 10:44:47 +07:00
Alex Sharov
5ee60dba3b
fix sort func (#4828) 2022-07-26 09:46:11 +07:00
Andrew Ashikhmin
a4d8031f5d
Fix RemoveContents for non-existing dir (#4823) 2022-07-25 15:58:40 +02:00
Alex Sharov
9e371fef5c
remove only etl-tmp content, but not dir itself #4816 2022-07-25 11:31:57 +07:00
Giulio rebuffo
1cb6be02a5
Avoid constantly triggering stageloop when using Engine API (#4797) 
* avoid constantly triggering stageloop when using Engine API

* fix lint + test

* fixed comments

* ops

* little fixes here and there

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-23 18:57:23 +02:00
Alex Sharov
68e35417fc
RetireBlocks: encapsulate delete logic 2022-07-23 11:09:16 +07:00
Giulio rebuffo
46a8c531ce
Optimized PoS header downloader (#4775) 
* optimized PoS header downloader

* removed println

* comments

* ops

* Restore schedulePoSDownload params + simplify

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
 2022-07-22 11:07:58 +02:00
Giulio rebuffo
1d378b6618
Filter out bad tx with wrong chain id during block building phase. (#4783) 
* filter out bad tx with wrong chain id

* report bad txs

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-22 09:18:19 +02:00
Alex Sharov
66758c7960
RetireBlocks: less arguments (#4785) 
* save

* save
 2022-07-22 13:44:42 +07:00
Andrew Ashikhmin
6060b87840
Fix binary vs raw confusion for PoS transaction (#4781) 
* Replace PayloadMessage with Block

* RawTransactions -> BinaryTransactions for clarity

* add a log warning
 2022-07-21 19:40:00 +02:00
Giulio rebuffo
73b028a5fd better payload cleanup (#4772) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-20 18:16:42 +02:00
Alex Sharov
5805d963ea
erigon-snapshot: convert from git sumbodule to golang package (#4760) 2022-07-20 09:34:12 +07:00
Giulio rebuffo
d4f865d725
Added proper cleanup when we get notified of new height (#4753) 
* added proper cleanup when we get notified of new height

* added extra cleanup

* removed bad if condition

* fixed hive tests

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-19 22:31:15 +02:00
Enrique Jose Avila Asapche
e768227d38
Merge range (#4749) 
* added merge range into segments

* got rid of missing snapshot errors

* reusing RequestSnapshotDownload

* sleep out of download

* ops

* warning if we are missing snapshots
 2022-07-19 19:27:54 +07:00
michaelscheung
c7da7a6d90
Support block parameter for integration stage_log_index (#4740) 
* Support block parameter for integration stage_log_index

* Add logPrefix

* Skip stage_log_index if endBlock < startBlock

Co-authored-by: michaelscheung <michael.szeyu.cheung@gmail.com>
 2022-07-19 10:48:22 +07:00
nanevardanyan
59dda48567
eth: replace maps with etl.Collectors (#4707) 
* WIP: eth: replace maps with etl.Collectors

* WIP: eth: replace maps with etl.Collectors in pruneOldLogChunks

* WIP: eth: use appendBuffer to avoid duplicates

* WIP: eth: replace with oldestEntrySortableBuffer
 2022-07-19 08:11:37 +07:00
Enrique Jose Avila Asapche
8dceb6fe82
Auto download snapshots (#4729) 
* refactored request download

* keeping track of missing snapshots

* using slice mergeRange

* request snapshots on reopen

* passing arguments

* passed in var

* Revert "passed in var"

This reverts commit 90478978dfa9f2a6dd5b1b051fc1d3f9e5f7a9c5.

* Revert "passing arguments"

This reverts commit 1e39c4152003796f6ff0bcfc188512d4a43bd18d.

* Revert "request snapshots on reopen"

This reverts commit d40212b973bc15db2b25cc1b0abb22051a3debb1.

* added downloadRequest
;

* downloading missing headers at start up

* there shouldnt be an error anymore

* not using nil;
;
 2022-07-18 17:42:20 +03:00
Giulio rebuffo
fb9f193349
fixed Two Block PoW Re-org to Higher-Height Chain (#4730) 
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
 2022-07-18 03:04:02 +02:00
Giulio rebuffo
8a754cd252
Added PoS download validation when applicable (#4728) 
* added incomplete version of PoS download validation

* fixed stuff
 2022-07-17 02:02:53 +02:00
Enrique Jose Avila Asapche
793ffcce3b
separated interrupt logic (#4714) 2022-07-14 17:22:46 +03:00