Commit Graph

43 Commits

Author SHA1 Message Date
Alex Sharov
a53642b4bf
datadir.Dirs configuration object to group dir config () 2022-06-07 10:24:50 +07:00
Alex Sharov
f838268e44
[wip] Snapshots: more runtime invariants check () 2022-06-03 18:50:01 +07:00
ledgerwatch
bcbb8bec65
Try to prevent double block retire ()
* Try to prevent double block retire

* Remove print

* More careful setting of segmentsMax

* Print snapshots max

* Clean lists

* Print

* Use Reopen

* Fix

* Remove prints

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-02 16:58:38 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag () 2022-05-30 09:59:38 +07:00
Alex Sharov
66248c4bfb
Torrent: verify all files () 2022-05-26 12:27:44 +07:00
Alex Sharov
b1b5a319e1
It's safe now to open snapshots at app start () 2022-05-20 14:46:04 +07:00
Alex Sharov
4e7807251a
Ensure snapshotDir not created by accident ()
* save

* save

* save
2022-05-17 09:40:45 +07:00
Alex Sharov
4ba1f03efe
Snapshots: atomic dir, step 3 () 2022-05-10 09:29:44 +07:00
Alex Sharov
32d8ab34d9
atomic snapshot dir, step 2 ()
* atomic snapshot dir

* atomic snapshot dir
2022-05-07 14:53:21 +07:00
Alex Sharov
2655d57288
reduce downloader deps ()
* reduce downloader deps

* reduce downloader deps
2022-04-28 12:13:30 +07:00
Alex Sharov
58c86d550d
Snapshots: Max index parallelism to 4 ()
* save

* save

* Readme update: built-in rpc daemon and syncmode=snap

* save

* save

* save
2022-04-21 17:49:42 +07:00
Alex Sharov
6aa4999729
Parallel indexing () 2022-04-21 10:34:10 +07:00
Alex Sharov
8d81c3b0f8
Rwdir fix2 () 2022-04-19 08:55:46 +07:00
Alex Sharov
e1d7cc3d3f
Enable: syncmode=snap by default (goerli, mainnet, bsc) () 2022-04-18 14:40:40 +07:00
ledgerwatch
78a58047dc
Fix in recompression ()
* Fix in recompression

* Fix error handling

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-14 11:07:17 +07:00
ledgerwatch
156da607e7
More compact representation of huffman trees in the seg files ()
* More compact representation of huffman trees in the seg files

* Update to latest erigon-lib

* recompress

* recompress

* recompress

* uncompress: buf fix

* Update to latest erigon-lib

* Revert default sync mode

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-13 16:29:44 +01:00
alex.sharov
482e06ead3 uncompress: buf fix 2022-04-13 16:35:21 +07:00
alex.sharov
594477ab23 uncompress: buf fix 2022-04-13 12:45:22 +07:00
alex.sharov
6cbbfb46eb recompress 2022-04-12 14:53:21 +07:00
Alex Sharov
b6334870e8
Snapshots: recompress.sh ()
* save

* save
2022-04-07 13:26:14 +07:00
Alex Sharov
80bd44fce5
Snapsthos: build indices on retire blocks ()
* grpc up

* grpc up

* grpc up
2022-04-05 16:22:11 +07:00
alex.sharov
4fb2a14756 add readahead to segment uncompress pipe 2022-04-04 21:07:02 +07:00
alex.sharov
32467007c1 snapshots: auto fix 2022-04-02 13:28:18 +07:00
Alex Sharov
8b03582558
compressor: log lvl
* save

* save
2022-04-01 10:44:30 +07:00
Alex Sharov
328c7b28a6
compress uncompress cli methods () 2022-03-31 14:10:40 +07:00
Alex Sharov
89d4477df3
Snapshots: nil indices on p2p fix () 2022-03-27 10:07:58 +07:00
Alex Sharov
a357421719
Snapshots: notify rpcdaemon about new snapshot () 2022-03-24 09:25:38 +07:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files () 2022-03-18 11:12:18 +07:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default () 2022-03-16 09:57:48 +07:00
Alex Sharov
f03d08c5ce
Snapshots: retire testing tool () 2022-03-12 17:26:11 +07:00
Alex Sharov
b003d07839
Add system-txs to begin end of block ()
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-03-10 07:48:58 +00:00
Alex Sharov
ac15d2b922
snapshots: better logging () 2022-02-24 16:28:46 +07:00
Alex Sharov
b94bc84a3b
snapshots: command to debug retire blocks ()
* snapshots: command_to_debug_retire

* snapshots: command_to_debug_retire
2022-02-24 11:30:34 +07:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. ()
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Alex Sharov
fc278327b9
"erigon snapshots recompress" add test () 2022-02-22 16:02:09 +07:00
Alex Sharov
dbd8a93fa8
"erigon snapshots recompress" to apply new compression rules without db access ()
* mainnet

* save

* save

* save

* save
2022-02-22 15:35:04 +07:00
Alex Sharov
5cd561bd74
block snapshots merge () 2022-02-22 08:55:24 +07:00
Alex Sharov
226982d293
Use class dir.Rw - to separate Ro and Rw access to snapshotDir ()
* save

* save

* save

* save

* save

* save

* save
2022-02-18 09:24:17 +07:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) () 2022-02-12 20:33:09 +07:00
Alex Sharov
2e17b8f515
snapshot: parallel transaction index, recsplit: reset offset collector, etl: faster flush and load () 2022-02-10 14:40:29 +07:00
Alex Sharov
bbf1b5b4bd
Torrent less goroutines () 2022-02-04 12:42:55 +07:00
Alex Sharov
304b40dd0a
Snapshot: prohibit to change cli flags () 2022-01-22 11:18:24 +07:00
Somu Bhargava
c620b1e7c9
Add Erigon Import feature for Hive integration purpose () 2022-01-20 07:34:50 +00:00