Commit Graph

2651 Commits

Author SHA1 Message Date
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
592ad57921
add DECOMPRESS_CONDENSITY env variable (#5853) 2022-10-24 10:00:55 +07:00
Alex Sharov
4ca4aa7144
e3: bsc recon leack fix (#5852) 2022-10-24 09:14:34 +07:00
Alex Sharov
a1cd46c5be
e3: recon metrics (#5850) 2022-10-23 19:31:24 +07:00
Alex Sharov
cd4ce6291f
don't accumulate state changes for jumps > 1K blocks (#5845) 2022-10-23 18:50:11 +07:00
Alex Sharov
a1e21260ad
e3: reduce reconstitute ram estimation (#5848) 2022-10-23 18:49:57 +07:00
Alex Sharov
96233cfa7f
e3: metrics (#5846) 2022-10-23 17:05:26 +07:00
Krishna Upadhyaya
aea019a3ff
Fix BorTxLookup (#5800) 2022-10-22 17:37:01 +01:00
Alex Sharov
d6de913a34
e3: fix races (#5822) 2022-10-21 21:02:18 +07:00
Andrew Ashikhmin
64a3156112
Support BaseFee in AuRa headers (#5820)
BaseFee is required in AuRa headers when
[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559) is activated.

Also:
- Basic AuRa header verification
- Extract some common RLP methods
- Tiny log clean-up
2022-10-21 12:43:44 +02:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Giulio rebuffo
889164e847
Strenghtened safety system for LC (#5811) 2022-10-20 16:07:17 +02:00
Alex Sharov
0db6b9b44b
e3: move skip analysis out of the loop (#5803) 2022-10-20 10:39:40 +07:00
Alex Sharov
c1313a1669
e3: Close recon db earlier to free ram (#5794) 2022-10-19 16:45:55 +07:00
Alex Sharov
bab0aaa24d
e3: close recon db earlier (#5793) 2022-10-19 16:44:06 +07:00
Alex Sharov
965619ff83
e3: fix parallel exec deadlock (#5790) 2022-10-19 12:25:57 +07:00
Giulio rebuffo
39dd749db0
Added verbosity flag to lightclient (#5785)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-18 12:46:48 +02:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram (#5784) 2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing (#5782) 2022-10-18 15:29:59 +07:00
Alex Sharov
b909bfa581
e3 tests (#5774) 2022-10-18 11:08:14 +07:00
gfx
d1e9149c93
add more sync stages to metrics (#5740)
hi! 

this pr adds more sync stages to the prometheus metrics. node now shows 

```
sync{stage="account_history_index"} 15742138
sync{stage="block_hashes"} 15742138
sync{stage="bodies"} 15742138
sync{stage="call_traces"} 15742138
sync{stage="cumulative_index"} 15742138
sync{stage="execution"} 15742138
sync{stage="finish"} 15742138
sync{stage="hash_state"} 15742138
sync{stage="headers"} 15742138
sync{stage="intermediate_hashes"} 15742138
sync{stage="log_index"} 15742138
sync{stage="senders"} 15742138
sync{stage="snapshots"} 15742138
sync{stage="storage_history_index"} 15742138
sync{stage="translation"} 0
sync{stage="tx_lookup"} 15742138
sync{stage="verkle_trie"} 0
sync{stage="watch_the_burn"} 0
```

Co-authored-by: a <a@a.a>
2022-10-18 10:22:57 +07:00
Alex Sharov
3aa97c57c1
fix ram estimation for indexing (#5773) 2022-10-18 10:19:58 +07:00
juroberttyb
64fe6708fa
Correct failed link in README.md (#5762)
[RPC calls] link path (../../cmd/rpcdaemon/Readme.md) corrected to
(../../cmd/rpcdaemon/README.md)
2022-10-18 09:23:44 +07:00
hexoscott
adadef62c8
remove estimated duration from stage execution (#5768) 2022-10-18 09:22:44 +07:00
J1ang
2177c8faa5
fix: issue #5723 type conversion bug (#5763)
fix:
fix for issue #5723 type conversion bug by adding check. `expValue`
should be `uint64[]`.
2022-10-17 11:25:17 +01:00
Giulio rebuffo
ab3a0b0fbd
Switched to validating lightclient (#5761)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-16 00:07:27 +02:00
Giulio rebuffo
d075618ecc
Importing of ETH2 blocks from Lightclient (#5760) 2022-10-15 23:01:52 +02:00
Giulio rebuffo
6f1c2be4e5
Implemented clause 3 of phase 4 (#5758) 2022-10-15 19:03:31 +02:00
ledgerwatch
90d26a3f46
Fix lint (#5756)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 10:34:12 +01:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl (#5742) 2022-10-15 08:20:58 +07:00
ledgerwatch
525a3a7cbb
Fix for empty blocks being proposed (#5749)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 00:12:38 +01:00
Giulio rebuffo
497d414b36
Added Lightclient update base validation without BLS (#5746)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-14 15:24:44 +02:00
hexoscott
e50110562a
revert execution batch size logic to size on disk (#5732)
Revert to older batch size logic to keep memory usage down during
execution phase and closer to the --batchSize flag size.

Spotted a "leak" in key generation as well. The unsafe pointer keeps the
byte slice around for as long as the batch is not committed, takes up a
fair chunk of memory surprisingly doing that so removed the unsafe
pointer usage giving the GC some chance to clean up along the way.

Moved the batch rollback into a defer func call rather than allowing
them to stack in the for loop. If this isn't going to work just let me
know and can change it back.
2022-10-13 09:10:26 +01:00
Alex Sharov
d2ef27d746
e3: reconstitution last step improve (#5733) 2022-10-13 14:33:39 +07:00
Alex Sharov
6cc10a27d3
e3: recondb no meminit (#5730) 2022-10-13 12:24:53 +07:00
Alex Sharov
1774abd6cd
e3: don't create own context, we are enough enough safe for kill (#5728) 2022-10-13 11:30:54 +07:00
Alex Sharov
b12b0d4627
e3: history bsc and mainnet snapshots (#5725) 2022-10-13 09:46:32 +07:00
Alex Sharov
d2436ab344
e3: optimize stateWriter.Put (#5724) 2022-10-13 09:37:14 +07:00
Alex Sharov
a9853d2ef8
Recon parallel: split ReconState to 2 objects to avoid lock contention between .Done() and .Get(), less ram in beginning (#5713) 2022-10-12 17:23:10 +07:00
ledgerwatch
750d23427d
Covalenthq move status execution flag into vm config (#5710)
Co-authored-by: Sudeep <sudeepdino008@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-12 09:27:48 +01:00
Alex Sharov
4795217e4e
e3: prevent files ranges overlap (#5684) 2022-10-12 10:18:55 +07:00
hexoscott
ab0ac1bfd4
wait for at most 500ms when receiving the GetPayload message from CL (#5701)
Seems simple enough, don't stop on interrupt but start a timer instead
and check on each loop if we should stop now or allow the block to
finish on its own
2022-10-11 16:51:53 +01:00
Andrew Ashikhmin
7286a0fef7
Create in-memory MDBX inside dirs.Tmp (#5702)
Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.

Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
2022-10-11 16:49:38 +01:00
ledgerwatch
a170d730c0
Fixing issues related to sync delays (#5689)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-11 08:04:40 +01:00
Alex Sharov
a2e51a2469
e3: prune limited amount before commit #675 (#5693) 2022-10-11 11:25:13 +07:00
Giulio rebuffo
a8338e4dd2
Infra refactoring (#5673)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-09 13:41:15 +02:00
Giulio rebuffo
e36b1206f7
Resolve panic on ctx.done when using --experimental.lightclient (#5670)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-08 15:07:04 +02:00
ledgerwatch
d4a5ddc7ee
erigon3 - fix state reconstitution (#5666)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-07 21:40:58 +01:00
hexoscott
c829c9f4f3
back to existing request bodies logic (#5650)
Moves bodies request logic to pre OOM changes. No logging or checking
for timeouts and moving forwards with every request made. Tested on a
local node that went through the DB migration on this branch and worked
fine, caught back up to the tip. Tested on POW initial sync and looked
to be working fine writing bodies, although didn't have time to see this
run all the way through the merge.
2022-10-06 20:27:06 +01:00
Giulio rebuffo
ebe63cfa30
Added tests for config functions and further refactoring (#5646)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-06 18:23:46 +02:00