Commit Graph

89 Commits

Author SHA1 Message Date
Alex Sharov
82c478a419
e3: getLogs on iterators (#6683) 2023-01-25 16:29:41 +07:00
Alex Sharov
4fbbdf9186
e3: move txnum to erigon-lib (#6663) 2023-01-22 19:39:33 +07:00
Alex Sharov
17e0afbe44
"erigon snapshots retire" save progress (#6486) 2023-01-02 12:38:03 +07:00
Alex Sharov
94fd30c5eb
"erigon snapshots retire" to save progress (#6485) 2023-01-02 12:26:56 +07:00
Alex Sharov
c04668d0ff
e3: agg cancel background jobs (#6463) 2022-12-29 15:04:07 +07:00
Alex Sharov
05f715ca51
E3: chain rules fix (#6448) 2022-12-27 13:14:24 +07:00
Alex Sharov
dfa6505f93
e3: kv/temporal prototype (#6367) 2022-12-19 15:38:54 +07:00
Alex Sharov
863e1ed07b
e3: handle "kill -9" during merge, auto-hide history .ef file if not corresponding .v file (#6304) 2022-12-15 13:49:06 +07:00
alex.sharov
62125a0dc1 save 2022-12-14 21:41:54 +07:00
Alex Sharov
99a3ab88f2
e3: cli command to retire/prune history (#6300) 2022-12-14 10:01:11 +07:00
alex.sharov
b9e54fa2e1 save 2022-12-14 09:53:38 +07:00
Alex Sharov
ce9203d80d
e3: more aggressive prune #780 (#6299) 2022-12-14 09:50:48 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
Alex Sharov
93926646f8
e3: prepare downloader for extraction (#6061) 2022-11-16 15:48:23 +03:00
Andrew Ashikhmin
32629bdce4
Upgrade urfave/cli to v2 (#6047)
See also
https://github.com/urfave/cli/blob/main/docs/migrate-v1-to-v2.md
2022-11-14 17:33:57 +01:00
hexoscott
636586c1b5
cache state check (#5844)
draft for now to get some early feedback on approach
2022-11-07 13:04:31 +00:00
Alex Sharov
1e5a513657
e3: configurable recon workers (#5983) 2022-11-07 11:42:44 +07:00
Alex Sharov
e490d2d0c7
e3: background merge (#5921) 2022-11-01 09:55:38 +07:00
Alex Sharov
ba02673e4d
e3: try move old data from db as early as possible #712 (#5897) 2022-10-29 14:58:34 +07:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Alex Sharov
606ce5c99a
add logging cli flags to various cmd (#5816) 2022-10-21 12:36:17 +07:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl (#5742) 2022-10-15 08:20:58 +07:00
Alex Sharov
b12b0d4627
e3: history bsc and mainnet snapshots (#5725) 2022-10-13 09:46:32 +07:00
Alex Sharov
a2e51a2469
e3: prune limited amount before commit #675 (#5693) 2022-10-11 11:25:13 +07:00
Alex Sharov
6a7f8fb2a6
erigon3: cli command to force merge snapshots (#5680) 2022-10-10 09:47:01 +07:00
Alex Sharov
ca9aa4723c
Compress params change (#5631)
as of https://github.com/ledgerwatch/erigon-lib/pull/651
2022-10-05 17:54:54 +07:00
Alex Sharov
c48d51a582
snapshot index workers amount - based on available RAM and CPU (#5524) 2022-10-05 17:07:01 +07:00
Alex Sharov
fa16d5e855
erigon3: step toward background snapshots build (#5588) 2022-10-02 10:21:17 +07:00
Alex Sharov
f4a7d2cf6d
erigon3: build .vi after download (#5568) 2022-09-29 12:14:51 +07:00
Alex Sharov
9cac57bcc9
erigon3: set seedable files to 32 steps #5534 2022-09-27 10:51:14 +07:00
Alex Sharov
da7ca3cacf
erigon3: build .efi after download #654 (#5523) 2022-09-26 15:51:39 +07:00
Alex Sharov
ef9e296efd
erigon method to test decompression speed (#5464) 2022-09-22 14:11:39 +07:00
Alex Sharov
50811f9cd6
"erigon snapshots ram" command (#5449)
* save

* save

* save
2022-09-21 09:18:37 +07:00
Alex Sharov
9fb8a190bc
erigon22: folder snapshots/history (#5351) 2022-09-18 17:41:01 +07:00
Enrique Jose Avila Asapche
473ca51830
Adding back stage snapshots (#5331)
* added snapshots into stages list && deleted snapshots from defaul unwind

* added verkle trie

* using the same sentries client hd

* need to safe the stage progress of headers as well
2022-09-12 10:26:00 +07:00
Leonard Chinonso
7495986515
Reverting Adding snapshots as a separate stage commit (#5329)
* reverting Adding snapshots as a separate stage commit

* more fixes to merge conflicts
2022-09-10 19:05:18 +02:00
Enrique Jose Avila Asapche
6d1d3a9f47
moved retiring of blocks from senders into snapshot stage (#5290) 2022-09-09 21:48:16 +07:00
Alex Sharov
caa23b9532
erigon22: simplify interhashes stage (#5315) 2022-09-08 14:02:50 +07:00
Enrique Jose Avila Asapche
8fc7c78620
Bor txlookup (#5260)
* printing

* logging

* spam off

* ops

* clean up

* deleted all bor related stuff in snapshots

* adding bor tx to BorTxLookUp

* added readBorTxLookUp

* getting bor tx from bor lookup

* comments

* deleted bor related stuff

* deleting bor tx

* writing borTxLookUp in stage_txlookup

* using bytes

* little fixes

* updated erigon-lib

* using biEndian

* clean up

* setting bytes for big int

* log

* logging

* more logs

* delete logs
2022-09-02 17:41:58 +03:00
Enrique Jose Avila Asapche
12331e018a
Fix bor txlookup (#5170) 2022-08-30 17:23:16 +03:00
Alex Sharov
69fbfc9bef
Snapshots: Erigon update list in db only after Downloader confirmation, rpcdaemon read list from db (#5150) 2022-08-23 16:28:07 +07:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-07-28 16:57:38 +07:00
Alex Sharov
68e35417fc
RetireBlocks: encapsulate delete logic 2022-07-23 11:09:16 +07:00
Alex Sharov
cd8b10f57e
snapshot merger: smaller interface (#4786)
* save

* save

* save
2022-07-22 16:54:05 +07:00
Alex Sharov
66758c7960
RetireBlocks: less arguments (#4785)
* save

* save
2022-07-22 13:44:42 +07:00
Alex Sharov
a53642b4bf
datadir.Dirs configuration object to group dir config (#4387) 2022-06-07 10:24:50 +07:00
Alex Sharov
f838268e44
[wip] Snapshots: more runtime invariants check (#4350) 2022-06-03 18:50:01 +07:00
ledgerwatch
bcbb8bec65
Try to prevent double block retire (#4330)
* Try to prevent double block retire

* Remove print

* More careful setting of segmentsMax

* Print snapshots max

* Clean lists

* Print

* Use Reopen

* Fix

* Remove prints

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-02 16:58:38 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00