Horacio Mijail Antón Quiles
f8477e29ab
Move batch management into the writeReceipts case ( #1321 )
2020-10-30 08:41:35 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx ( #1271 )
...
* squash
* add --database flag to integration
* clean
* split to 2 buckets
* split to 2 buckets
* split to 2 buckets
* split to 2 buckets
* split to 2 buckets
* save progress
* save progress
* improve test
* improve test
* save progress
* change app logic
* change app logic
* return err from rawdb package
* don't clean automatically
* don't clean automatically
* clean
* clean
* clean
* don't rely on `make clean`
* improve cbor code
* clean
* clean
* clean
* fix tests
* rebase master
* stop on error: headers stage
* make TxDb walk and multiwalk safe
* Fix panics
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-25 08:38:55 +00:00
ledgerwatch
b747ab5324
[WIP] CallTraces index ( #1157 )
...
* Initial commit for CallTraces index
* Fix compilation
* fix lint, add comment
* Fix integration
* Add Close function to ethdb.Cursor, fix some compile errors
* Try to stop cursor leak in Get
* Fix compile errors in RPC daemon
* Fix compile errors
* fixing another way
* Some fixes
* More fixes
* More fixes
* More fixes
* Fixes to core/state
* Fix lint
* Fix lint
* Fixes
* Stage caching for call trace stage
* Add mem stats
* Try to stop the leak
* Turn off debug
* Chunks for 10k blocks
* Print
* Revert "Print"
This reverts commit 5ffada4828d61e00e5dad1ca12c98258dfbbad00.
* Revert "Chunks for 10k blocks"
This reverts commit cfb9d498e782e5583d41c30abf0e2137da27383e.
* Trying to fix the leak
* Don't compute receipts in re-tracing
* Not compose block
* Print speed, fix receipts, bigger caches
* Fix lint
* Utilise changeset info
* Counters
* Use NoReceipts and ReadOnly
* ReadOnly is incompatible with caching
* Skip test leaking transactions
* Fix block test
* Change disable message for call-traces stage
* Use block option for call traces integration
* Fix retracing due to incarnation
2020-10-12 09:39:04 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs ( #1124 )
...
* save progress
* try now
* don't create bloom inside rlpDecode
* don't create bloom inside ApplyTransaction
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* rename method
* print timings
* print timings
* print timings
* sort before flush
* fix err lint
* clean
* move tests to transactions
* compressed version
* up bound
* up bound
* more tests
* more tests
* more tests
* more tests
* better removal
* clean
* better performance of get/put methods
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* optimize rpcdaemon
* fix test
* fix rpcdaemon
* fix test
* simplify
* simplify
* fix nil pointer
* clean
* revert some changes
* add some logs
* clean
* try without optimize
* clean
* clean
* clean
* clean
* try
* move log_index to own stage
* move log_index to own stage
* integration add log_index stage
* integration add log_index stage
* clean
* clean
* print timing
* remove duplicates at unwind
* extract truncateBitmaps func
* try detect
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* add blackList of topics
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* sharding 1
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 2
* sharded 3
* sharded 3
* sharded 3
* speedup things by putCurrent and putReserve
* clean
* optimize trim
* clean
* remove blacklist
* add more info to err
* ?
* clean
* clean
* clean
* clean
* clean
* working version
* switch to cgo version of roaring bitmaps
* clean
* clean
* clean
* clean
* more docs
* clean
* clean
* fix logs bloom field
* Fix debug_getModifiedAccountsByNumber
* Try to fix crash
* fix problem with "absent block"
* fix problem with "absent block"
* remove optimize method call
* remove roaring iterator
* fix problem with rebuild indicess
* remove debug prints
* tests for eth_getLogs involving topics
* add tests for new stage, speparate topics into 2 buckets
* version up
* remove debug logs
* remove debug logs
* remove bloom filter implementation
* Optimisation
* Optimisatin not required, make rpctest lenient to geth errors
* Lenient to geth failures
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-28 18:18:36 +01:00
Horacio Mijail Antón Quiles
252863fedf
Use the passed VMConfig ( #1141 )
...
* Use the passed VMConfig
* Explicitly pass the vmConfig into runBlock
2020-09-27 21:31:49 +01:00
Alex Sharov
5ce73d438a
fix integration ( #999 )
2020-08-29 15:27:20 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount ( #975 )
...
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize
* remove bolt_db.go file
* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation ( #851 )
...
* Jumpdest skipping optimisation
* Fix formatting
* Move skipAnalysis into vmConfig, introduce tracing ability
* Improve detection logging
* Added release instructions
* Fix lint
2020-08-01 17:56:57 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly ( #748 )
...
* close goruotines
* fix using defer
* close pm managers a bit later
* fmt
* finalizer for in-mem db
* after cr
* revert finalizer
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes ( #716 )
...
* Fixing history generation
* Try fixes
* Reset from 0
* Cleanup
2020-07-06 07:34:24 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets ( #698 )
...
* Debugging changesets
* Kinda works
* Fix compile error
* Fix formatting
* Fix lint
* Duplicate entries kludge
* Fix compile error
* Cleanup
2020-07-01 15:56:56 +01:00
ledgerwatch
ed866e6934
non-concurrent ETL, debug_traceTransaction in rcpdaemon ( #692 )
...
* Fixing history index
* Remove chunk generation, fix formatting
* Fix compile error, clean up hack.go
* Fix output tests
* Fix index generator test
* Fixed checkChangeSets
* Fix linter
2020-06-28 07:10:27 +01:00
Alex Sharov
57777e7c60
Prepare codebase for future default DB change ( #670 )
...
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods
* choose db based on file suffix
* implement db.id method
* implement db.id method
* use ethdb.NewDatabase method
* use ethb.MustOpen method
* cleanup
* support TEST_DB env flag
* create db path automatically needed
* bolt - don't change prefix on happy path
2020-06-16 14:36:16 +01:00
ledgerwatch
853db762e2
Make GetAsOf work with partially missing history index ( #648 )
...
* Debug infos
* bucketStats lmdb by default
* Use bools instead of strings
* Fix linters
* Fix linter
* Finally fix
* Fix compile error
* Fix test
* Fix formatting
* Fix shadowing
* Cleanup
* cleanup
2020-06-11 07:19:15 +01:00
ledgerwatch
462edc2345
CheckChangeSets to work with plain state and changeSets, optionally generate receipts ( #639 )
...
* Start from 9m7
* Regenerate IH + receipts
* Only stats for iH bucket
* Persist receipts
* Go all in
* Start from block 10m
* Convert DbState to use plain state
* Fix findHistory
* Hard-code export
* More fixes
* Fix test
* Fix formatting
* Introduce PlainDbState
* Actually return PlainDbState
* Fix formatting
* Fix name style lint
* Fix linters
* Fix history_test
* Fix blockchain_test
* Fix compile error
* Bucket stats from all buckets
2020-06-09 06:52:38 +01:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap ( #626 )
...
* optimize codeBitmap
* add lru
* remove custom lru
* jumpDests as explocit parameter
* lint
* lint
* linters
* default jumpDestsCache
* remove common.rand
2020-06-06 21:49:06 +01:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger ( #624 )
...
* lmdb tests
* trigger ci
* fix tests
* disable parallelism
* disable parallelism
* cleanup resources
* cleanup resources
* reduce concurency
* try run tests on bolt
* try run tests on bolt
* fix downloader test
* run bolt tests
* rely on interface instead of exact instance
* Rename AbstractKV to KV
* don't use separator for badger
* don't initialize badger cursor - because it not used here
* fix linter
* try reduce badger compactors
* compat with master
* try lmdb
* try lmdb
* try lmdb
* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option
* allow to close closed db
* release
* release
* ideal batch size for badger
* ideal batch size for badger
2020-06-05 10:25:33 +01:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb ( #584 )
...
* Not hash, keep the files
* Calculate savings
* Fix
* Fix
* Fix
* Fix
* RestAPI to support local boltdb
* Not error on read-only db
* Changes so far
* Continue
* More
* Roll back a bit
* Restore newline
* something compiles
* Fix restapi
* Fix block number
* Fix reads
* Use plain writer
* Maps for storage reads and writes
* Clean up coersions
* Fix accounts/abi/bind
* Fix tests
* More fixes
* more fixes
* More fixes
* Fixes
* Fixed core/state
* Fixed eth tests
* Move code, fix linter
* Fix test
* Fix linter
* Fix linter
* Fix linter, badger_db to support AbstractKV
* Increase IdealBatchSize for badger
* Fix linter
* Fix linter
2020-05-27 17:24:34 +01:00
Igor Mandrigin
0dae3ade0d
post-rebase fixups
2020-05-20 15:26:22 +03:00
Igor Mandrigin
acbf8bcbdf
Enable thin history by default ( #456 )
...
* Enable thin history by default
* remove thin history flags
* remove from tests
* fix test compile error
* test fixups
* fix more stuff
2020-04-15 12:33:22 +03:00
ledgerwatch
ad64ae2085
[THIN_HISTORY] Decouple changesets and history from ethdb.Database (mutation/bolt_db/badger_db) ( #432 )
...
* Prepare for refactoring
* Decoupling
* Further cleanup of mutation
* Further cleanup of mutation
* Remove some tests (temporarily)
* Fix linter
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Recover mutation_test cases
* Fix lint
* Fix WalkAsOf test
* Fix lint
* Fix TODO, linter
* Fix lint
* Fix lint
* Fix tests for THIN_HISTORY
* Fix lint
* Fix lint
* Reduce visibility of DecodeStorage
* Revert "Reduce visibility of DecodeStorage"
This reverts commit 2e11e16ea095b75a62cdddb77b9477eabc259e67.
2020-04-09 18:23:29 +01:00
Alex Sharov
c8cbdb8c47
improve changeSetChecks to accept genesis.json ( #415 )
2020-03-30 19:56:53 +01:00
Andrew Ashikhmin
82b80c2ebe
Thin history: Fix an error in mutation.getNoLock ( #409 )
...
* CheckChangeSets and thin history
* small code de-duplication
* small code clean-up
* Fix an error in mutation.getNoLock
* CheckChangeSets: truly make historyfile = chaindata by default
2020-03-26 21:52:22 +00:00
ledgerwatch
6dcb8ade46
Convenience changes to checkChangeSets and stateless ( #406 )
...
* Allow separation of files
* Try next block
* Enable history
* use logging
* No check option
* Option to write history
2020-03-25 20:18:46 +00:00
Andrew Ashikhmin
aeed1657c7
Issue #340 : Re-execute all historical transaction in read-only mode and check ChangeSets ( #388 )
...
* Clean up code duplication between IntraBlockState's FinalizeTx & CommitBlock
* checkChangeSets command
* linter
* First attempt at checking account changes
* Reuse runBlock in CheckChangeSets
* linter
* linter
* Optionally include no-changes in the ChangeSets
* linter
* Detect storage changes for account change sets
* Fix post-merge compilation errors
* Use database format compatible with !debug.IsThinHistory()
* PrintChangedAccounts in ChangeSetWriter
* Avoid out-of-bounds access
* Storage changes
* hack FirstContractIncarnation
* Call ChangeSetWriter only once per block
2020-03-11 16:54:09 +01:00
