Alex Sharov
5ef9a3feb9
Trie: use APPEND when re-generating trie. Also including db-migration for PR#1535 ( #1549 )
2021-03-12 16:26:39 +07:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record ( #1500 )
...
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
2021-02-21 18:41:59 +00:00
Artem Vorotnikov
8c0ecaeea9
Remove StageData ( #1419 )
2020-12-21 11:01:37 +00:00
Alex Sharov
76f1b05cb2
Move tmpdir definition to app-start, move migrations folder inside tmpdir ( #1282 )
...
* extract tmpdir to app-level-code
* extract tmpdir to app-level-code
* save progresss
2020-10-23 12:18:45 +01:00
ledgerwatch
88acfc55ca
Vary batch size for execution stage ( #1274 )
...
* Vary batch size for execution stage
* Fix in logging
* Improve log of timings
* Log improvements
* Log improvements
* Fix tests
* More log improvments, fix tests
* Fix tests
* Better logging for Bodies and Interhash
* Fix tests
* Minor fixes
* Fix test
2020-10-21 18:01:40 +01:00
Alex Sharov
44bbe97a1f
Migration idempotency: handle corner-cases of CriticalCollector ( #1257 )
...
* save progress of db migration
* save progress of db migration
* delete
* delete
* delete
* delete
* delete
* print full key if it has 8 leading zeroes
* print full key if it has 8 leading zeroes
* print full key if it has 8 leading zeroes
2020-10-19 20:20:18 +01:00
ledgerwatch
d2e5cc5cb5
Better migration - receipts_cbor_encode not to increase size of the database ( #1250 )
...
* Add print_migration command to integration
* Not run migrations for print commands
* Reverse migration
* Fix lint
* Add datadir to run_migrations
* Add lmdb.mapSize to integration
* Make lmdb.mapSize effective
* Fix
* receipt sizes
* fix print
* Switch to another version of lmdb
* Collector from existing files
* Not load
* Commit after clearing
* Add methods for clear/drop buckets by multiple transactions
* Fix to lmdb
* Not remove temp files
* Not load
* Re-enable loading
* Prints
* Try to fix refill
* Migration in 3 steps
* tidy mod
* Fix lint
* Fix tests
* Migrations for CBOR receipts in 3 txs
* Fix lint
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-10-17 14:00:12 +01:00
Alex Sharov
809d79b15f
Switch to cbor ( #1172 )
...
* switch receipts to cbor
* switch receipts to cbor
* rpcdaemon to cache chainconfig
* rpcdaemon to cache chainconfig
* rpcdaemon to cache chainconfig
* rpcdaemon to cache chainconfig
2020-10-02 13:51:20 +01:00
Alex Sharov
48cf5fb43e
History-style sharding for bitmap indices ( #1159 )
2020-10-02 10:54:11 +07:00
Alex Sharov
dae72a8589
(related to lmdb assert) Avoid deleteCurrent on cursor which used for iterations ( #1151 )
...
* save progress
* save progress
* avoid delete current
* undupsort_ih
2020-09-29 10:28:49 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs ( #1124 )
...
* save progress
* try now
* don't create bloom inside rlpDecode
* don't create bloom inside ApplyTransaction
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* rename method
* print timings
* print timings
* print timings
* sort before flush
* fix err lint
* clean
* move tests to transactions
* compressed version
* up bound
* up bound
* more tests
* more tests
* more tests
* more tests
* better removal
* clean
* better performance of get/put methods
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* optimize rpcdaemon
* fix test
* fix rpcdaemon
* fix test
* simplify
* simplify
* fix nil pointer
* clean
* revert some changes
* add some logs
* clean
* try without optimize
* clean
* clean
* clean
* clean
* try
* move log_index to own stage
* move log_index to own stage
* integration add log_index stage
* integration add log_index stage
* clean
* clean
* print timing
* remove duplicates at unwind
* extract truncateBitmaps func
* try detect
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* add blackList of topics
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* sharding 1
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 3
* sharded 3
* sharded 3
* speedup things by putCurrent and putReserve
* clean
* optimize trim
* clean
* remove blacklist
* add more info to err
* ?
* clean
* clean
* clean
* clean
* clean
* working version
* switch to cgo version of roaring bitmaps
* clean
* clean
* clean
* clean
* more docs
* clean
* clean
* fix logs bloom field
* Fix debug_getModifiedAccountsByNumber
* Try to fix crash
* fix problem with "absent block"
* fix problem with "absent block"
* remove optimize method call
* remove roaring iterator
* fix problem with rebuild indicess
* remove debug prints
* tests for eth_getLogs involving topics
* add tests for new stage, speparate topics into 2 buckets
* version up
* remove debug logs
* remove debug logs
* remove bloom filter implementation
* Optimisation
* Optimisatin not required, make rpctest lenient to geth errors
* Lenient to geth failures
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-28 18:18:36 +01:00
Igor Mandrigin
adf52465e3
move ./trie to ./turbo/trie ( #1114 )
...
Useful for minimizing merge conflicts when rebasing new geth
2020-09-14 11:33:39 +01:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support ( #1080 )
...
* etl.Loader - allow use of custom comparator
* log timing
* try now
* try now
* more performance
* etl.Loader - allow use of custom comparator
* working version
* simplify IH cursor
* clean
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* clean
* add only unwind support
* squash
* squash
* clean
* fix test
* clean
* clean
* clean
2020-09-10 13:35:58 +01:00
Alex Sharov
da31494cc3
transactional migrations ( #1079 )
2020-09-08 20:39:43 +01:00
Alex Sharov
bf596c26f3
[merge after release] dupsort of plain state ( #913 )
...
* dupsort of plain state
* rebase master
2020-08-15 08:11:40 +01:00
