Commit Graph

1094 Commits

Author SHA1 Message Date
Alex Sharov
aaec53857f
e3: last reader to close/remove merged files (marked as canDelete) inside tx.Rollback() (#6738) 2023-02-01 15:44:13 +07:00
Alex Sharov
dfca60c054
e3: indices wal - to reuse etl collector (#6757) 2023-02-01 10:02:29 +07:00
Andrew Ashikhmin
004c5afdfa
Avoid unnecessary unwind+rewind (#6712)
On
[withdrawal-mainnet-shadowfork-1](https://withdrawal-mainnet-shadowfork-1.ethpandaops.io/)
erigon was unnecessarily re-executing blocks after 16m (snapshot)
multiple times. That was likely due to CL issuing `forkchoiceUpdated`
pointing to an old block for some reason. This PR introduces a
protection against such inefficiency.
2023-01-30 15:42:23 +01:00
Alex Sharov
b99e4abb3e
move math big constants to erigon-lib (#6719) 2023-01-27 11:39:34 +07:00
Alex Sharov
d908f57528
Senders: nil check (#6717) 2023-01-27 10:19:33 +07:00
Alex Sharov
82c478a419
e3: getLogs on iterators (#6683) 2023-01-25 16:29:41 +07:00
Andrew Ashikhmin
37741b74eb
Remove erigon_issuance & erigon_watchTheBurn (#6685)
Some issuance happens on the CL side now, so removing support for these
now inaccurate proprietary methods to make our life easier.
2023-01-24 22:54:08 +01:00
ledgerwatch
5e205f61c0
More tweaks for BSC headers (#6662)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-22 14:19:16 +00:00
Alex Sharov
4fbbdf9186
e3: move txnum to erigon-lib (#6663) 2023-01-22 19:39:33 +07:00
ledgerwatch
706a999178
Improvements for the BSC stuck header sync (#6653)
There are 3 changes:
1. Replace `anchorQueue` with `anchorTree` to be able to always walk the
anchors in the order of increasing blockHeights (not possible with the
queue) to prioritise making progress on the lowest block heights
2. Not increment `nextRetryTime` if the request was not sent
3. Reduce the strides in skeleton from `8*192` to `192` to reduce
reliance of the long series of requests to make progress

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-01-22 12:28:17 +00:00
ledgerwatch
2c3299d073
Better logging for body writing (#6634)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-20 00:04:47 +00:00
ledgerwatch
f2111b4132
Another fix for body download (#6633)
When body gets evicted from the cache, its corresponding entry in
`requestMap` is also removed but was never re-instated when re-request
happens

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-19 23:05:38 +00:00
ledgerwatch
44b834e77a
Further fixes for body downloader (#6610)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-17 22:03:28 +00:00
ledgerwatch
19ce8d2a8f
stage_headers: use block time instead of "topSeenHeight" to determine when it is in sync (#6602)
This is more reliable criterion because it is harder to spoof
continuously by malicious peers

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-17 17:28:02 +00:00
ledgerwatch
0bea0437bf
Deal with the situation when body cache is too small (#6601)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-17 12:52:07 +00:00
Alex Sharov
5ad4eb936a
split backend.go:new to 2 funcs (#6595)
Split backend.go:New method to 2. 1st will create db and basic
configuration. 2nd will create instances of stagedsync, txpool, etc...
based on this configuration.
2023-01-17 13:20:31 +07:00
Alex Sharov
97a769cfc1
e3: increase some defaults (#6592) 2023-01-17 09:14:17 +07:00
ledgerwatch
cfb0d99cfa
Improvements for body downloader (#6589)
1. Replacing temporary MBDX table with limited-size btree
2. Always scan block numbers from the start to prioritise low-number
blocks
3. Other fixes and simplifications

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-16 22:09:28 +00:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
Andrew Ashikhmin
45d34ff038
Quiet virtual stage bodies in PoS validator (#6576)
In the vein of PR #5574.

Logs before:
```
[INFO] [01-13|12:37:02.940] [2/16 Headers] Handling new payload      height=25889655 hash=0xe41526b51c04f1a2547d356d4009ff43b9f2e
b8691fce5f0cdc2a9e763b23828
[INFO] [01-13|12:37:02.940] [2/7 Bodies] Writing block bodies        block_num=25889655 remaining=0 alloc=108.4MB sys=222.5MB
[INFO] [01-13|12:37:02.942] [2/16 Headers] Waiting for Consensus Layer... 
```

Logs after:
```
[INFO] [01-13|12:50:55.227] [2/16 Headers] Handling new payload      height=25891426 hash=0x2b890eb0ae6e3ce32fe6cad02d1defebbea71ec27d8c02c42233f369a27f40ad
[INFO] [01-13|12:50:55.237] [2/16 Headers] Waiting for Consensus Layer... 
```
2023-01-13 13:25:13 +00:00
Seonggwon Yoon
798d1b8a2d
Link ETL Repository (#6573)
Reconnect broken link for etl
2023-01-13 17:45:32 +07:00
Alex Sharov
731569bbb7
e3: invalid txnum table (#6540) 2023-01-12 09:31:41 +07:00
Alex Sharov
f96ba2631a
simplify mdbx_to_mdbx (#6561) 2023-01-12 09:13:58 +07:00
hexoscott
6aae186fac
wait on error from call to parallelWarmup (#6558) 2023-01-12 07:59:43 +07:00
alex.sharov
8be6e1b2cd STOP_AFTER_RECONSTITUTE=true 2023-01-11 10:16:21 +07:00
alex.sharov
89f8a43509 parallel warmup to exit early if nil 2023-01-10 09:57:52 +07:00
Alex Sharov
7c2ed8e01f
mdbx: v0.12.3 (#6521) 2023-01-10 09:15:18 +07:00
Alex Sharov
2621ef45f6
e3: fix trace json (#6535) 2023-01-09 17:33:18 +07:00
ledgerwatch
7243db8dbf
Simplify code around topSeenHeight (#6529)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-09 06:42:26 +00:00
Alex Sharov
e88a5aa2e8
HashState: parallel promote cleanly (#6512) 2023-01-09 11:01:21 +07:00
Alex Sharov
720b5f77d6
e3: bsc oom fix (#6526) 2023-01-08 15:07:34 +07:00
Alex Sharov
08904da83e
e3: fix couple deadlocks in parallel exec (#6525) 2023-01-08 10:00:41 +07:00
Alex Sharov
3c7e6b114e
e3: locality index (#6513) 2023-01-07 12:31:00 +07:00
ledgerwatch
2941e754e9
Fix hive tests and reorganise the fix for body download problem (#6515)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-06 12:43:46 +00:00
hexoscott
1a09dcbdb3
use set rather than slice for tracking yielded when mining (#6488)
make use of sets when tracking yielded transactions during mining
2023-01-02 13:00:08 +00:00
Alex Sharov
11637a5c7d
e3: force merge snapshots before reconst (#6484) 2023-01-02 10:10:48 +07:00
Alex Sharov
4a9c871628
a bit rename 22 to v3 (#6476) 2022-12-30 21:53:42 +07:00
Alex Sharov
b0af04f2df
bsc: incrementally calc and store all old snapshots. slower initial stage_snapshots, but faster restart during initial sync (#6457) 2022-12-30 12:44:54 +07:00
Giulio rebuffo
4932aa5264
added indexes to beacon blocks (#6460) 2022-12-29 15:15:25 +01:00
Alex Sharov
e7104c9a68
e3: support withdrawals (#6467) 2022-12-29 19:57:58 +07:00
alex.sharov
82e7893837 e3: remove debug line 2022-12-29 18:52:33 +07:00
alex.sharov
ed94d60671 e3: disable some checks for genesis, to fix tests 2022-12-29 16:04:56 +07:00
Alex Sharov
c04668d0ff
e3: agg cancel background jobs (#6463) 2022-12-29 15:04:07 +07:00
ledgerwatch
387b4d7f35
Native tracers step 10 - remove duplicate SelfDestruct capture, clean up (#6440)
Moved some of the tracers to `eth/tracers/logger` to make it more
similar to go-ethereum.

Removed Erigon-specific `Capture-` functions

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-26 04:56:39 +00:00
Alex Sharov
8161e17694
e3: fix non-parallel nil (#6438) 2022-12-25 19:48:44 +07:00
Alex Sharov
4e26f53a75
e3: return error from inline-apply (#6427) 2022-12-24 17:30:26 +07:00
Alex Sharov
2fa3f69a1f
stage_snapshots: clean table before initial write (#6431) 2022-12-24 16:15:24 +07:00
Alex Sharov
ade933be6b
kv_remote: server to support thread-safe multi-streams per 1 tx (#6402) 2022-12-24 13:11:15 +07:00
ledgerwatch
a49e287618
Native tracers step 6: Rename Tracer to EVMLogger, reduce args CaptureExit (#6410)
Getting closer to Geth's tracer interfaces

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-23 05:43:08 +00:00
Alex Sharov
cce0f251b4
e3: kv/temporal prototype 2 (#6377) 2022-12-20 09:28:10 +07:00