Commit Graph

20 Commits

Author SHA1 Message Date
Igor Mandrigin
adf52465e3
move ./trie to ./turbo/trie (#1114)
Useful for minimizing merge conflicts when rebasing new geth
2020-09-14 11:33:39 +01:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support (#1080)
* etl.Loader - allow use of custom comparator

* log timing

* try now

* try now

* more performance

* etl.Loader - allow use of custom comparator

* working version

* simplify IH cursor

* clean

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* clean

* add only unwind support

* squash

* squash

* clean

* fix test

* clean

* clean

* clean
2020-09-10 13:35:58 +01:00
Alex Sharov
da31494cc3
transactional migrations (#1079) 2020-09-08 20:39:43 +01:00
Igor Mandrigin
2a6478d799
turbo-api: create a second executable file with a custom stage example (#1055) 2020-09-05 18:07:27 +02:00
Alex Sharov
bf596c26f3
[merge after release] dupsort of plain state (#913)
* dupsort of plain state

* rebase master
2020-08-15 08:11:40 +01:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object (#914) 2020-08-14 13:41:18 +07:00
Alex Sharov
760fe0f0b6
DupSort of hash state (#896) 2020-08-12 09:57:55 +07:00
Alex Sharov
088bb58023
Migrations validation: duplicate names and commit call (#902) 2020-08-11 18:23:41 +07:00
Giulio rebuffo
057894994d
add blockhashes stage (#862)
* add blockhashes stage

* unwindOrder specified

* fixed typo

* added common.copybytes

* fixed little typo

* better progress

* better progress

* refactoring with etl

* added blockhashes to disabled stages in cmd/integration

* added migraations

* fixed go.mod

* better migrations

* rename unwind

* simplified migrations

* added onloadcommit
2020-08-10 18:46:06 +01:00
Alex Sharov
5c75003456
fix migration idempotency (#898) 2020-08-10 16:16:14 +07:00
Alex Sharov
115cc7d736
use transactional putter (#893) 2020-08-10 14:28:38 +07:00
Alex Sharov
d458c4cc1e
Migrations: use stage name as db key (#868) 2020-08-05 17:13:35 +07:00
ledgerwatch
0514501937
Revert "Use stage name as db key (#858)" (#867)
This reverts commit 7290bf519e.
2020-08-04 11:05:27 +01:00
Alex Sharov
7290bf519e
Use stage name as db key (#858) 2020-08-04 16:25:28 +07:00
ledgerwatch
f06db2f37b
Stages 6 and 7 for generating history indices (#569)
* save state

* add current index feature

* fix test

* remove logs

* Only execute 1000 blocks

* Reset history index

* Correct action

* Increase batch size

* Increase chunk size, print memory stats

* Fix linter

* Remove unused from

* Split into 2 staged

* Use storage history gen

* remove log

* Not to run tx_cacher in staged mode

* Not to recover during stage 2

* Not to recover during stage 2

* Remove counter

Co-authored-by: b00ris <b00ris@mail.ru>
2020-05-23 10:19:56 +01:00
b00ris
cc137af1ce
[WIP] New storage changeset encoding (#493)
* new storage changeset encoding

* fix lint
2020-04-28 08:41:55 +01:00
Alex Sharov
535d73be5f
Run CI on bolt with prefix_compression_without_allocation (fixes) (#380) 2020-04-11 20:28:15 +07:00
Alex Sharov
965d1101f4
Intermediate hash phase 4 (#396)
* another way to check if account has storage

* cleanup

* v0 of walk by db version

* save progress, to switch to another task. Put tombstone is still not correct.

* place tombstone only if exists something to hide

* db-based implementation

* db-based implementation

* db-based implementation

* fix prop check

* improve prop check logic

* Need custom logic to skip subtree for account and storage buckets because storage bucket has incarnation in key

* rebase to master

* remove all tombstones when account deleted

* remove all tombstones when account deleted

* added db integrity check

* don't rely on account.Root because it valid only about last incarnation

* remove all tombstones when account deleted

* dial with incarnation in MultiWalk2

* dial with incarnation in MultiWalk2

* when fixedbytes=40 resolver did compare cacheKey with storageKey without removing incarnation

* rebase to master

* rebase to master

Co-authored-by: alex.sharov <alex.sharov@lazada.com>
2020-03-20 11:30:14 +00:00
b00ris
179acf936b
fix log (#386) 2020-02-29 20:37:53 +00:00
b00ris
83e65fdd81 migrations
fix build

add storage mode check

add intermitiate migration state&snapshot migrations
2020-02-26 00:25:30 +03:00