Commit Graph

22 Commits

Author SHA1 Message Date
Alex Sharov
ce96cf75b2
Intermediate hash phase 3 (#377)
* #remove debug prints

* remove storage-mode="i"

* minnet re-execute hack with checkpoints

* minnet re-execute hack with checkpoints

* rollback to master setup

* mainnet re-exec hack

* rollback some changes

* v0 of "push down" functionality

* move all logic to own functions

* handle case when re-created account already has some storage

* clear path for storage

* try to rely on tree structure (but maybe need to rely on DB because can be intra-block re-creations of account)

* fix some bugs with indexes, moving to tests

* tests added

* make linter happy

* make linter happy

* simplify logic

* adjust comparison of keys with and without incarnation

* test for keyIsBefore

* test for keyIsBefore

* better nibbles alignment

* better nibbles alignment

* cleanup

* continue work on tests

* simplify test

* check tombstone existence before pushing it down.

* put tombstone only when account deleted, not created

* put tombstone only when account has storage

* make linter happy

* test for storage resolver

* make fixedbytes work without incarnation

* fix panic on short keys

* use special comparison only when working with keys from cache

* add blockNr for better tracing

* fix: incorrect tombstone check

* fix: incorrect tombstone check

* trigger ci

* hack for problem block

* more test-cases

* add test case for too long keys

* speedup cached resolver by removing bucket creation transaction

* remove parent type check in pruning, remove unused copy from mutation.put

* dump resolving info on fail

* dump resolving info on fail

* set tombstone everytime for now to check if it will help

* on unload: check parent type, not type of node

* fix wrong order of checking node type

* fix wrong order of checking node type

* rebase to new master

* make linter happy

* rebase to new master

* place tombstone only if acc has storage

* rebase master

* rebase master

* rebase master

* rebase master

Co-authored-by: alex.sharov <alex.sharov@lazada.com>
2020-03-11 10:31:49 +00:00
b00ris
83e65fdd81 migrations
fix build

add storage mode check

add intermitiate migration state&snapshot migrations
2020-02-26 00:25:30 +03:00
b00ris
2b74b43678 History index encoding
save state

save state

remove old history index

fix lint

compress index
2020-02-24 23:29:46 +03:00
b00ris
958ace4088 split storage and account encoding for changeset 2020-02-24 22:36:16 +03:00
b00ris
e69f53db8d Merge branch 'master' of github.com:ledgerwatch/turbo-geth into split_cs_buckets_squash 2020-02-24 21:47:26 +03:00
b00ris
aedecfc820 split changeset bucket
fmt

badger changes

badget puts

fix shadowing

fix mutation test. restore DecodeChangeSet

fix pruner test
2020-02-24 20:56:32 +03:00
Alex Sharov
fdbba5202b
Trie: store self-destructed accounts (#355)
* squash commits

* enable storage cache

* make linter happy

* fix subtree prefix len check

* save cahnges to test master

* remove restriction on prefix len

* fix comparison of last bits

* fix wrong alignment

* remove debug prints

* commit current state

* commit current state

* avoid changing state of resolver from multiwalk

* remove debug code

* remove debug code

* remove debug code

* remove unnecessary copy

* make code more readable

* reduce rebuildHashes initial resolution

* fix test after rebase to master

* make code more readable

* improve pruner

* pruner add IntermediateCache bucket

* fix panic in Walk on short keys

* reduce allocations for storage keys decompression by increasing default buffer size

* re-run CI

* fix iterator behaviour

* rename cache to hash for unification

* re-run ci

* avoid using underlying DB

* hash all subtree nodes before unload

* fix getNode method

* need to check node type, not parent - before put to hashBucket

* return back parent type check, doesn't work without it.

* don't recalculate hash again

* move unloadFunc from trie to pruner

* rename bucket to shorter name

* rename bucket to shorter name

* clean

* rebase to master
2020-02-12 13:52:59 +00:00
ledgerwatch
0cd4f65ffe
Copy values from the ChangeSet bucket during Rewind (#348)
* Fix rewinding

* Fix rewinding
2020-02-01 16:24:38 +00:00
Alex Sharov
ff23980dc6
Intermediate hash phase 2 (#341)
* add env INTERMEDIATE_TRIE_CACHE
* try to use assert.New() pattern
* Fix "maligned" linter warnings to reduce space consumption of structs:

core/types/accounts/account.go:18:14: struct of size 136 bytes could be of size 128 bytes (maligned)
type Account struct {
--
trie/node.go:44:10: struct of size 80 bytes could be of size 72 bytes (maligned)
	duoNode struct {
--
trie/resolve_set.go:28:17: struct of size 56 bytes could be of size 48 bytes (maligned)
type ResolveSet struct {
--
trie/resolver.go:34:15: struct of size 88 bytes could be of size 72 bytes (maligned)
type Resolver struct {
--
trie/visual.go:32:17: struct of size 104 bytes could be of size 96 bytes (maligned)
type VisualOpts struct {
2020-01-31 14:31:50 +07:00
Alex Sharov
0dd4386a37
Intermediate trie hashes phase 1 (#334)
* add intermediate trie hashes bucket
* update docs
* use version of CompressNibbles with buf pool (less GC) but without io.ByteWriter (slow)
2020-01-31 11:11:20 +07:00
Giulio rebuffo
32375df7af made changeSet an helper type (#314)
* made changeSet an helper type

* added comment

* removed dbutils.Add

* remove common.CopyBytes from walk

* fixes
2020-01-15 20:51:10 +00:00
Alex Sharov
fc7291ec34
Use local db in analytics (#308)
* Use local db in analytics (#308)
* pool of encoders
* incr and decr funcs
2020-01-15 19:47:13 +07:00
b00ris
bcee4845dc
Thin history (#272)
* save state

* remove repair

* save state

* remove emptydb check

* save state

* add walkAsOf test

* add WalkAsOf and MultiWalkAsOf tests

* deployed contracts counter

* reference counter for contract code

* drop storage root&contract hash for changesets

* start incarnation is 1(save state)

* fix ReorgOverSelfDestruct test

* hack fix TestReorgOverSelfDestruct

* test benchmark

* cleanup

* remove useless debug

* remove print trie

* return remove subtrie call  to updateTrieRoot

* save state

* add mutation test

* remove useless test

* fix

* added mutation commit test

* rename experiment to thin history

* thin history mutation commit test

* fix ethdb tests

* getAsOf test

* add test&fix history index

* fix test

* make test for index search

* compute trie root incarnation fix

* tests fixes

* done job in case of panic

* fix lint

* fix&test for bad incarnation

* fix initial incarnation for genesis

* fix lint

* fix changeset test

* fix storage ranges test

* fix lint

* move set incarnation to create contract

* add comment

Co-authored-by: ledgerwatch <akhounov@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
2019-12-20 15:25:40 +03:00
alex.sharov
187670fcda more tests 2019-12-09 21:34:47 +07:00
alex.sharov
432c2a5b26 fix linter warning 2019-12-09 21:34:47 +07:00
alex.sharov
04a6517c5b added gasLimits to report object 2019-12-09 21:34:47 +07:00
Andrew Ashikhmin
8e5e5d881f
Issue 123 Make encoding of ChangeSet ordered by keys and binary-searchable (#207) 2019-12-03 12:13:12 +01:00
andrew
8cb8544c45 Merge branch 'master' into badger2 2019-11-11 17:12:28 +01:00
b00ris
6e880cc945
suffix to changeset rename (#152) 2019-11-07 18:51:25 +03:00
andrew
9be775c8bd Port my work from badger after the master rebase 2019-11-05 13:28:36 +01:00
b00ris
9eb7d8b1c2 Suffix reorg (#113) 2019-11-04 14:15:26 +01:00
Alexey Akhunov
fe01bccbb8 Apply Turbo-Geth modifications to go-ethereum codebase 2019-11-01 21:52:03 +01:00