Alex Sharov
5ce73d438a
fix integration ( #999 )
2020-08-29 15:27:20 +07:00
ledgerwatch
335aedc7e3
Cut down the stateGrowth generation code ( #973 )
...
* Cut down the stateGrowth generation
* Fix NPE
* Fix for lastTimestamp
* Adjust to the current state
* Potential fix
* Fix for shorter keys in the state
* Extract code
* Iterate over code
2020-08-28 10:31:37 +01:00
Alex Sharov
d73b036cff
python_bublic_print_cli ( #994 )
2020-08-28 14:19:34 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount ( #975 )
...
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize
* remove bolt_db.go file
* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync ( #970 )
...
* Log in GetAsOf
* Default plain state
* trace
* More test fixes
* Fix lint
* Fix
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix for events
* Repair events
* Fix lint
* Fix lint
* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Giulio rebuffo
0ffe32ae16
State growth restore ( #951 )
...
* new state growth
* lint
2020-08-21 07:32:11 +01:00
Alex Sharov
f827f86c11
Rpcdaemon as lib 2 ( #943 )
2020-08-20 10:52:27 +07:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction ( #928 )
...
* drop Bucket object from KV abstraction
* add some non-public methods
* rebase master
* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
b2e35eb63f
rename "remote2" to "remote" ( #932 )
2020-08-17 07:47:06 +01:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object ( #914 )
2020-08-14 13:41:18 +07:00
Alex Sharov
3aa5526d5a
fix miss-type ( #916 )
2020-08-13 12:31:17 +07:00
Alex Sharov
9047f4fea0
python: don't hardcode buckets list ( #915 )
2020-08-13 11:40:30 +07:00
Alex Sharov
760fe0f0b6
DupSort of hash state ( #896 )
2020-08-12 09:57:55 +07:00
ledgerwatch
6f88e4058f
Fix ( #908 )
2020-08-11 20:48:37 +01:00
Alex Sharov
d9d9e14f45
change bucket type to string ( #894 )
2020-08-11 06:55:32 +07:00
Alex Sharov
81c487803e
add python docs ( #900 )
2020-08-10 16:36:46 +07:00
Igor Mandrigin
8600616c3d
geth 1.9.18 post-rebase fixups
2020-08-08 17:33:35 +02:00
Alex Sharov
d458c4cc1e
Migrations: use stage name as db key ( #868 )
2020-08-05 17:13:35 +07:00
ledgerwatch
0514501937
Revert "Use stage name as db key ( #858 )" ( #867 )
...
This reverts commit 7290bf519e
2020-08-04 11:05:27 +01:00
Alex Sharov
7290bf519e
Use stage name as db key ( #858 )
2020-08-04 16:25:28 +07:00
Alex Sharov
ecc94a63f0
DupSort for Plain and Hashed state buckets (behind feature flags) ( #854 )
...
* return error from rawdb
* squash
* v14
* improve performance of put
* clean
* clean
* hide feature behind ENV variable
* cleanup
* cleanup
* disable ipc and make Readme less confusing (people thought points are depend on each-other)
* fix test
* cleanup
* cleanup
2020-08-02 12:53:08 +01:00
Igor Mandrigin
3aedcbcd7d
Use datadir for temp files when generating indexes
2020-08-02 12:32:41 +02:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation ( #851 )
...
* Jumpdest skipping optimisation
* Fix formatting
* Move skipAnalysis into vmConfig, introduce tracing ability
* Improve detection logging
* Added release instructions
* Fix lint
2020-08-01 17:56:57 +01:00
Alex Sharov
c4a1ffa4f0
Grpc-based remoteDb ( #788 )
2020-07-27 19:15:48 +07:00
Alex Sharov
95365c8f37
remove ReadLastBlockNumber method = read execution stage status instead ( #780 )
...
* remove ReadLastBlockNumber method = read execution stage status instead
* make linter happy
2020-07-24 14:46:49 +01:00
Alex Sharov
a7e5c4b453
Remove remote_chain package, fix rpc methods: eth_blockNumber and eth_getBlockByNumber ( #769 )
2020-07-21 15:19:04 +07:00
Alex Sharov
1faa9dbcd4
add senders bucket to py ( #763 )
2020-07-20 16:11:47 +07:00
ledgerwatch
b765ff0e33
cmd/tester initiates staged sync, and unwind (but not reorg) ( #758 )
...
* First push
* Remove batch from Blockchain
* Fix compile error in cmd/integration
* Fix compile errors in cmd/state
* Fix compile errors
* Save changes
* Fix core tests (skip tx tests)
* More test fixes
* Fix compile error
* Fix formatting
* Fix formatting
* Fix lint
* Fixes
* Skip some miner tests
* Fix tests
* Cleanup
2020-07-19 09:11:53 +01:00
b00ris
12aa55bcbb
Remove incarnation inversion ( #755 )
...
* remove incarnation inversion
* fix lint
* remove incarnation inversion
* remove comment
* remove inversion from new code
* revert for unwind
2020-07-17 15:18:31 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly ( #748 )
...
* close goruotines
* fix using defer
* close pm managers a bit later
* fmt
* finalizer for in-mem db
* after cr
* revert finalizer
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
3359ba7a04
Make cmd/tester initiate staged sync, remove some pools and caches ( #746 )
...
* no pools
* Remove excessive logging
* Revert
* Initial commit
* Cleanup
* Fix some compile errors
* Fix core tests
* Fix more tests
* Fix eth/downloader
* go mod tidy
* Fix core/vm
* Fix formatting
* Fix formatting
* Fix lint
* Fix lint
* Switch tests to StagedSync
* Cleanup
* Reuse cache in stage4
* Fix
* Fix formatting
* Try to fix test
2020-07-15 07:15:48 +01:00
b00ris
eca0f233ec
[WIP] WalkAsOf ( #724 )
...
* save state
* walkAsOf by cs test passed
* cs search for plain state
* save state
* fix accounts tests
* refactor walkAsOf account tests
* fix storage test
* refactor walkAsOf storage tests
* fix lint
* fix test
* save state
* save state
* add test with fixed bits
* fmt
* add stages check
* fix lint
* fix lint
* remove obsoleted methods
2020-07-09 07:13:45 +01:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes ( #716 )
...
* Fixing history generation
* Try fixes
* Reset from 0
* Cleanup
2020-07-06 07:34:24 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets ( #698 )
...
* Debugging changesets
* Kinda works
* Fix compile error
* Fix formatting
* Fix lint
* Duplicate entries kludge
* Fix compile error
* Cleanup
2020-07-01 15:56:56 +01:00
ledgerwatch
ed866e6934
non-concurrent ETL, debug_traceTransaction in rcpdaemon ( #692 )
...
* Fixing history index
* Remove chunk generation, fix formatting
* Fix compile error, clean up hack.go
* Fix output tests
* Fix index generator test
* Fixed checkChangeSets
* Fix linter
2020-06-28 07:10:27 +01:00
Igor Mandrigin
69a20de5cc
Fix broken export file usage. ( #680 )
2020-06-20 21:07:39 +01:00
Alex Sharov
57777e7c60
Prepare codebase for future default DB change ( #670 )
...
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods
* choose db based on file suffix
* implement db.id method
* implement db.id method
* use ethdb.NewDatabase method
* use ethb.MustOpen method
* cleanup
* support TEST_DB env flag
* create db path automatically needed
* bolt - don't change prefix on happy path
2020-06-16 14:36:16 +01:00
Igor Mandrigin
93f6c4c01a
post-rebase fixups
2020-06-15 19:38:54 +03:00
Alex Sharov
15096f273d
Remove ctx from Open. Stop goroutines on Close. ( #650 )
...
* remove ctx from MustOpen
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from remote open (we have DialTimeout field to manage connection timeouts)
* enable RawReads and add native implementation of Get/Has methods
2020-06-12 10:31:21 +01:00
Alex Sharov
9961fef39e
experiment with rewriting gas_limits to python+lmdb (just to see how it can be) ( #654 )
2020-06-12 09:25:40 +01:00
ledgerwatch
853db762e2
Make GetAsOf work with partially missing history index ( #648 )
...
* Debug infos
* bucketStats lmdb by default
* Use bools instead of strings
* Fix linters
* Fix linter
* Finally fix
* Fix compile error
* Fix test
* Fix formatting
* Fix shadowing
* Cleanup
* cleanup
2020-06-11 07:19:15 +01:00
b00ris
b4ba764fb1
[WIP] TxLookup stage ( #646 )
...
* save state
* txlookup full results
* save state
* save state
* remove experiments
* some fix&lint
* add end key to txLookup and index generation
* change log message
* change log
* fix lint
* lint
* fix test
2020-06-10 23:07:14 +03:00
Andrew Ashikhmin
cf7b9a75a7
Issue 635: Stage4 to optionally generate persisted receipts ( #644 )
...
* Euphemerally -> Ephemerally
* Move StorageMode to ethdb and pass it to PrepareStagedSync
* linter
* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go
* Optionally write receipts in the execute stage
* memory profiler
* linter
* proper linter fix
* linter
* typo
* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
2020-06-09 14:11:09 +01:00
Alex Sharov
9f11dabede
Enable golint back but with conditions ( #641 )
...
* enable_golint_with_conditions
* enable_golint_with_conditions
2020-06-09 12:00:58 +01:00
ledgerwatch
462edc2345
CheckChangeSets to work with plain state and changeSets, optionally generate receipts ( #639 )
...
* Start from 9m7
* Regenerate IH + receipts
* Only stats for iH bucket
* Persist receipts
* Go all in
* Start from block 10m
* Convert DbState to use plain state
* Fix findHistory
* Hard-code export
* More fixes
* Fix test
* Fix formatting
* Introduce PlainDbState
* Actually return PlainDbState
* Fix formatting
* Fix name style lint
* Fix linters
* Fix history_test
* Fix blockchain_test
* Fix compile error
* Bucket stats from all buckets
2020-06-09 06:52:38 +01:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap ( #626 )
...
* optimize codeBitmap
* add lru
* remove custom lru
* jumpDests as explocit parameter
* lint
* lint
* linters
* default jumpDestsCache
* remove common.rand
2020-06-06 21:49:06 +01:00
Evgeny Danilenko
8d7e603f07
stack pool ( #627 )
2020-06-05 17:45:56 +03:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger ( #624 )
...
* lmdb tests
* trigger ci
* fix tests
* disable parallelism
* disable parallelism
* cleanup resources
* cleanup resources
* reduce concurency
* try run tests on bolt
* try run tests on bolt
* fix downloader test
* run bolt tests
* rely on interface instead of exact instance
* Rename AbstractKV to KV
* don't use separator for badger
* don't initialize badger cursor - because it not used here
* fix linter
* try reduce badger compactors
* compat with master
* try lmdb
* try lmdb
* try lmdb
* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option
* allow to close closed db
* release
* release
* ideal batch size for badger
* ideal batch size for badger
2020-06-05 10:25:33 +01:00
Andrew Ashikhmin
33c28f7fac
[GC] uint256 rather than big.Int in Transaction ( #614 )
...
* uint256 in rlp
* uint256 rather than big.Int in Transation
* linters
* more linters
* still linters
* Reduce garbage in writeUint256
* Experiment with GC in writeByteArray
2020-06-04 08:43:08 +01:00
ledgerwatch
7ab10c85af
Separate Collector pattern out of ETL, hash collector for rebuilding Intermediate Hashes ( #597 )
...
* Introduce hashCollector
* Add HashCollector to SubTrieLoader
* Fix linter
* Reset hashed state
* Not to regenerate the hashed state
* Not to delete state
* Fix linter
* Print expected hash in the beginning
* Simplify
* Remove initialTrie
* Use etl to buffer intermediate hashes
* Copy values, not insert empty key
* Compress instead of decompress
* Enhance file buffer logging, fix linter
* Fix compile errors
* Fix log
* Fix logging
* Exclude zero key again
* Add rewind
* Restrict timestamps
* Fix
* Fix formatting
* Incorporate separation
* Extract identityLoadFunction
* Fix formatting
2020-05-31 13:23:34 +01:00
