Commit Graph

354 Commits

Author SHA1 Message Date
Shane Bammel
e1a28e26c4 Add network config for Testnet-V4 2023-06-27 19:11:26 -05:00
Shane Bammel
cab77c932e Add v3 suffix to pulsechain-testnet 2023-06-27 19:11:26 -05:00
bretep
c8c9b1f523 Switch testnet snapshots to PulseChain from Mainnet 2023-06-27 19:11:26 -05:00
bretep
b79df0254f Initial PulseChain integration
Sign block using correct chainId

Add IsPulseChain to config

Add support for pulsechain blocks and snapshots

Add support for chain config now located in erigon-lib

Fix decoding hex to int

Ensure pulsechain config

Add support for PulseChain snapshots
2023-06-27 17:50:22 -05:00
ledgerwatch
f8a5c8c6ce
Fix DumpTxs (#7782) 2023-06-22 12:08:16 +01:00
Alex Sharov
a77edd7a32
faster opening of snapshots and indices (#7757)
- mostly by MADV_SEQUENTIAL during opening 
- see https://github.com/ledgerwatch/erigon-lib/pull/1025
2023-06-19 11:34:01 +07:00
Alex Sharov
d3c3be9c91
e2: optimize tests speed (#7738) 2023-06-15 16:09:11 +07:00
Alex Sharov
e5023775aa
Enforce blockReader interface (#7737)
- breaks dependency from staged_sync to package with block_reader
implementation
- breaks dependency from snap_sync to package with block_reader
implementation
- breaks dependency from mining to txpool implementation
2023-06-15 13:11:51 +07:00
Alex Sharov
afa4f53ae2
e3: use deserializeV3 for history (#7691) 2023-06-09 10:53:04 +07:00
Alex Sharov
e14664d53b
Store Canonical/NonCanonical/Bad blocks/senders/txs in same tables. Make re-org cheaper (#7648)
- allow store non-canonical blocks/senders
- optimize re-org: don't update/delete most of data
- allow mark chain as `Bad` - will be not visible by eth_getBlockByHash,
but can read if have hash+num
2023-06-05 12:36:24 +07:00
Alex Sharov
895e7dd96a
cleanup (#7655) 2023-06-04 10:37:20 +07:00
Alex Sharov
63c92010cd
remove txsV3 cli flag (#7644) 2023-06-03 15:54:27 +07:00
Alex Sharov
ad72b7178e
prune speedup. stage_senders: don't re-calc existing senders (#7643)
- stage_senders: don't re-calc existing senders
- stage_tx_lookup: prune less blocks per iteration - because
random-deletes are expensive. pruning must not slow-down sync.
- prune data even if --snap.stop is set
- "prune as-much-as-possible at startup" is not very good idea: at
initialCycle machine can be cold and prune will cause big downtime, no
reason to produce much freelist in 1 tx. People may also restart erigon
- because of some bug - and it will cause unexpected downtime (usually
Erigon startup very fast). So, I just remove all `initialSync`-related
logic in pruning.
- fix lost metrics about disk write byte/sec
2023-06-03 12:30:53 +07:00
Alex Sharov
d40317c905
preparation for --txs.v3, step 2 (#7636) 2023-06-03 07:20:22 +07:00
Alex Sharov
6439bdd835
txsv3: release small parts (#7632) 2023-06-02 11:38:20 +07:00
Alex Sharov
436656a40f
snapshots: DumpBodies - break dependency on body.BaseTxNum value in db (#7607)
it's step towards saving canonical and non-canonical bodies in same
table (and txs also in same own table). to reduce write amplification
(cheaper re-orgs)

PR change: reading BaseTxNum from existing snapshots instead of DB 
DB will store in field body.BaseTxNum - non-canonical TxnID  
Snapshots will store only canonical TxNum in field body.BaseTxNum
2023-06-02 11:01:57 +07:00
Alex Sharov
5c7da48331
blockReader: fix blockNum check logic (#7615) 2023-06-01 15:04:21 +07:00
Alex Sharov
3cba3908dd
use BlockReader.CanonicalHash method instead of rawdb (#7604) 2023-05-31 13:41:10 +07:00
Alex Sharov
279e1bec33
use blockReader as service-provider of RoSnapshots (#7584) 2023-05-26 17:12:33 +07:00
Alex Sharov
cfafa24481
tests for blocks snapshots creation data producer (#7579) 2023-05-25 14:58:31 +07:00
Alex Sharov
63afe65686
Use BlockReader in ForkValidator, CliqueAPI (#7562) 2023-05-23 14:49:17 +07:00
Alex Sharov
4d0dee6fb0
Introduce BlockWriter object (txsV3 step 0) (#7559) 2023-05-22 15:49:21 +07:00
Alex Sharov
2865b85888
move e2 snapshots management closer to e3: step 1 (#7543)
- always RLock all snapshots - to guarantee consistency
- introduce class View (analog of RoTx and MakeContext)
- move read methods to View object
- View object will be managed by temporal_tx

---------

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-22 10:09:46 +07:00
ledgerwatch
b382f96f6c
Introduce logger into etl (#7537)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-18 21:20:07 +01:00
Alex Sharov
b3aca15ff8
e2: ReadAhead of blocks, senders accounts, code (#7501)
It improves performance of initial sync (stage exec) by 5-20% when
blocks snapshots are mounted to high-latency drive and when chaindata is
on high-latency drive. And improving cold-start performance.

Current implementation using 2 goroutines for ReadAhead. It also
producing more garbage, can improve it later (here are dashboard with
impact).
```
mainnet2-1: with ReadAhead
mainnet2-3: no ReadAhead
```

<img width="949" alt="Screenshot 2023-05-12 at 09 24 31"
src="https://github.com/ledgerwatch/erigon/assets/46885206/b90b1fa8-9099-48ff-95b3-86e864a36d46">

<img width="845" alt="Screenshot 2023-05-12 at 09 24 13"
src="https://github.com/ledgerwatch/erigon/assets/46885206/39d90c0c-a9d5-4735-8c03-da1455b147aa">
2023-05-15 15:08:35 +07:00
racytech
42e8db3958
eip-4844: modified DecodeTransaction version 1 (#7442)
Blob transactions are SSZ encoded, so it had to be added to decoding.
There are 2 encoding forms: `network` and `minimal` (usual). Network
encoded blob transactions include "wrapper data" which are `kzgs`,
`blobs` and `proofs`, and decoded by `DecodeWrappedTransaction`. For
previous types of transactions the network encoding is no different.
Execution-payloads / blocks use the minimal encoding of transactions. In
the transaction-pool and local transaction-journal the network encoding
is used.

Concerns: 
1. Possible performance reduction caused by these changes, not sure if
streams are better then slices. Go slices in this modifications are
read-only, so they should be referred to the same underlying array and
passed by a reference.
2. If `DecodeWrappedTransaction` and `DecodeTransaction` will create
confusion and should be merged into one function.
2023-05-09 18:44:53 +01:00
Andrew Ashikhmin
3f60a902d4
Fix nil withdrawals in eth_getBlockByHash (#7332) 2023-04-18 11:23:22 +00:00
Alex Sharov
64e7e788b5
e3: print background indexing logs in e2 style (summary of all files progress in 1 line) (#7256) 2023-04-05 04:49:02 +00:00
ledgerwatch
7258a2b872
Remove BSC support in Erigon (step 1) (#7246)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-04-03 14:46:21 +00:00
Alex Sharov
f541f616e6
e3: one more reconst deadlock fix (#7207) 2023-03-29 07:32:31 +00:00
Alex Sharov
9001f668a7
go 1.19 atomics (#7164) 2023-03-23 05:11:28 +00:00
Alex Sharov
4ab63db6e2
nil ptr in delete ancient (#7081) 2023-03-11 05:46:32 +00:00
Alex Sharov
d4e25daf22
etl: distinct empty values from nil (#7039)
Reverts ledgerwatch/erigon#7038
2023-03-07 04:19:00 +00:00
Alex Sharov
7206e482da
Revert "etl: distinct empty values from nil" (#7038)
Reverts ledgerwatch/erigon#6934
2023-03-07 03:22:23 +00:00
Alex Sharov
c5acfd0503
etl: distinct empty values from nil (#6934) 2023-03-07 02:33:21 +00:00
nanevardanyan
ab6239b30f
WIP: cmd, turbo, core, eth: TransactionsV3 flag and persist in new table (#6754) 2023-02-24 18:49:25 +00:00
Jochen Müller
4106a899d5
Chiado snapshots (#6886)
No change to `go.mod` since `erigon-snapshot` has already been updated
in `devel`.

I'll keep my Chiado node up to serve the snapshots for a while, until
there are more erigon nodes in the network.

Resolves #6307
2023-02-16 12:05:33 +00:00
Alex Sharov
999899d66d
e3: read files list from db (#6833) 2023-02-13 05:17:01 +00:00
Alex Sharov
60a4b8bc7b
Snapshots Indexing: avoid loop with semaphore locking before logging (#6845) 2023-02-12 02:49:20 +00:00
Alex Sharov
c31824ee17
less logs (#6813) 2023-02-09 13:27:45 +07:00
Alex Sharov
145f689e78
less logs about compression (#6812) 2023-02-09 12:28:09 +07:00
Alex Sharov
bb2bec2439
sepolia block snaspshots (#6626) 2023-01-25 09:55:26 +07:00
alex.sharov
03144279ef fix lint 2023-01-22 20:47:36 +07:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
alex.sharov
abff625dcb fix test 2023-01-11 12:04:22 +07:00
ledgerwatch
e8287918ba
Gnosis block snapshots (#6516)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-01-07 12:07:36 +00:00
Alex Sharov
4e26f53a75
e3: return error from inline-apply (#6427) 2022-12-24 17:30:26 +07:00
Alex Sharov
ade933be6b
kv_remote: server to support thread-safe multi-streams per 1 tx (#6402) 2022-12-24 13:11:15 +07:00
Alex Sharov
599695d001
RemoteBlockReader.TxnByIdxInBlock() (#6404) 2022-12-22 12:54:07 +07:00
Alex Sharov
dfa6505f93
e3: kv/temporal prototype (#6367) 2022-12-19 15:38:54 +07:00