Commit Graph

44 Commits

Author SHA1 Message Date
Alex Sharov
fd9c7c3b2d
kill bolt (#1085)
* kill bolt

* kill bolt

* fix test

* clean
2020-09-09 11:09:55 +01:00
ledgerwatch
335aedc7e3
Cut down the stateGrowth generation code (#973)
* Cut down the stateGrowth generation

* Fix NPE

* Fix for lastTimestamp

* Adjust to the current state

* Potential fix

* Fix for shorter keys in the state

* Extract code

* Iterate over code
2020-08-28 10:31:37 +01:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975)
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Giulio rebuffo
0ffe32ae16
State growth restore (#951)
* new state growth

* lint
2020-08-21 07:32:11 +01:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928)
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object (#914) 2020-08-14 13:41:18 +07:00
Alex Sharov
d9d9e14f45
change bucket type to string (#894) 2020-08-11 06:55:32 +07:00
Igor Mandrigin
8600616c3d geth 1.9.18 post-rebase fixups 2020-08-08 17:33:35 +02:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation (#851)
* Jumpdest skipping optimisation

* Fix formatting

* Move skipAnalysis into vmConfig, introduce tracing ability

* Improve detection logging

* Added release instructions

* Fix lint
2020-08-01 17:56:57 +01:00
Alex Sharov
95365c8f37
remove ReadLastBlockNumber method = read execution stage status instead (#780)
* remove ReadLastBlockNumber method = read execution stage status instead

* make linter happy
2020-07-24 14:46:49 +01:00
Alex Sharov
a7e5c4b453
Remove remote_chain package, fix rpc methods: eth_blockNumber and eth_getBlockByNumber (#769) 2020-07-21 15:19:04 +07:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly (#748)
* close goruotines

* fix using defer

* close pm managers a bit later

* fmt

* finalizer for in-mem db

* after cr

* revert finalizer

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
Alex Sharov
57777e7c60
Prepare codebase for future default DB change (#670)
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* choose db based on file suffix

* implement db.id method

* implement db.id method

* use ethdb.NewDatabase method

* use ethb.MustOpen method

* cleanup

* support TEST_DB env flag

* create db path automatically needed

* bolt - don't change prefix on happy path
2020-06-16 14:36:16 +01:00
Igor Mandrigin
93f6c4c01a post-rebase fixups 2020-06-15 19:38:54 +03:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap (#626)
* optimize codeBitmap

* add lru

* remove custom lru

* jumpDests as explocit parameter

* lint

* lint

* linters

* default jumpDestsCache

* remove common.rand
2020-06-06 21:49:06 +01:00
Evgeny Danilenko
8d7e603f07
stack pool (#627) 2020-06-05 17:45:56 +03:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger (#624)
* lmdb tests

* trigger ci

* fix tests

* disable parallelism

* disable parallelism

* cleanup resources

* cleanup resources

* reduce concurency

* try run tests on bolt

* try run tests on bolt

* fix downloader test

* run bolt tests

* rely on interface instead of exact instance

* Rename AbstractKV to KV

* don't use separator for badger

* don't initialize badger cursor - because it not used here

* fix linter

* try reduce badger compactors

* compat with master

* try lmdb

* try lmdb

* try lmdb

* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option

* allow to close closed db

* release

* release

* ideal batch size for badger

* ideal batch size for badger
2020-06-05 10:25:33 +01:00
Alex Sharov
916a1f8b83
[wip] Lmdb: AbstractKV and DB interfaces (#589)
* resetIH from scratch if needed

* lmdb

* add AbstractKV to loader, added new Object accessor around AbstractKV

* add lmdb cli flag

* add requirement of k!=nil on error in docs

* add Size method for compatibility

* read after put tests

* fix multiput nils

* simplify loops

* increase mmap size

* better error messages

* better error messages

* fix tests

* better error messages

* cleanup

* avoid bolt usage in test

* move hardcoded bucket name to dbutils

* register more buckets

* register more buckets

* fix test
2020-05-30 09:12:21 +01:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb (#584)
* Not hash, keep the files

* Calculate savings

* Fix

* Fix

* Fix

* Fix

* RestAPI to support local boltdb

* Not error on read-only db

* Changes so far

* Continue

* More

* Roll back a bit

* Restore newline

* something compiles

* Fix restapi

* Fix block number

* Fix reads

* Use plain writer

* Maps for storage reads and writes

* Clean up coersions

* Fix accounts/abi/bind

* Fix tests

* More fixes

* more fixes

* More fixes

* Fixes

* Fixed core/state

* Fixed eth tests

* Move code, fix linter

* Fix test

* Fix linter

* Fix linter

* Fix linter, badger_db to support AbstractKV

* Increase IdealBatchSize for badger

* Fix linter

* Fix linter
2020-05-27 17:24:34 +01:00
Andrew Ashikhmin
5bbdb6a4b9
Use uint256 for account balance (#580)
* Use uint256 for account balance

* linter
2020-05-26 17:53:50 +01:00
Igor Mandrigin
0dae3ade0d post-rebase fixups 2020-05-20 15:26:22 +03:00
Giulio rebuffo
6e2fc0d936
[WIP] 459 merge storage account (#466)
* bt commit

* merged buckets

* adjusted cmd/

* fixed sync
2020-04-19 20:51:32 +01:00
Alex Sharov
9324d83cb2
ethdb.KV interface - pase 2 (#420)
* - handle cursor.Prefix on server
- move state reports to KV interface

* add CmdCursorSeekKey

* tests for abstract_kv

* avoid reading configs of databases

* avoid reading configs of databases

* make linter happy

* make linter happy

* cleanup

* port badger features from original implementation

* try to fix test

* try to fix test

* .Close() don't return error anymore - defer friendly

* try to enable badger now

* try to enable badger now

* badger can't run on CI yet

* badger can't run on CI yet

* re-run ci

* skip ctx cancelation for badger
2020-04-04 08:18:10 +01:00
Alex Sharov
c5ffc971c5
[WIP] Badger v2 (#378)
* badger v2 investigation

* buf pool - use native New method and avoid double checks

* db.Open prototype

* db.Tx/Bucket/Cursor prototypes

* Chained config

* Item concept added

* save changes to test on master

* make hack resumable

* Design document v0

* Cursor concept

* less brackets syntax of cursor builder

* benchmarks

* cleanup fs

* test for context cancelations

* test for context cancelations

* test for cursor.Prefix option

* add ForEachKey method

* add ForEachKey method

* add naming explanation

* experiment of non-pointers cursor/bucket

* .Bucket() and .Cursor() doesn't returns error

* .Bucket() and .Cursor() doesn't returns error

* .Bucket() and .Cursor() doesn't returns error

* remove CursorOpts concept

* more test-cases

* simplify open api

* Tx, Bucket, Cursor - now are interfaces

* Tx, Bucket, Cursor - now are interfaces

* switch to interfaces

* rebase master

Co-authored-by: alex.sharov <alex.sharov@lazada.com>
2020-03-11 11:02:37 +00:00
Alex Sharov
fc7291ec34
Use local db in analytics (#308)
* Use local db in analytics (#308)
* pool of encoders
* incr and decr funcs
2020-01-15 19:47:13 +07:00
Alex Sharov
438476d1ba
Serialization bench (#288)
* bench shows that need to pass pointer to []byte when use encoder.Encode()
2020-01-08 13:09:07 +07:00
Alex Sharov
33308dd5c2
State analytic less memory and checkpoints (#299)
* added prefix tree to analyses to reduce memory usage
* make new partition every day
* merge concepts of reporter and snapshot
* tests for .FirstKey() and .NextKey()
2020-01-07 09:27:19 +07:00
Alex Sharov
85009dabe6
return error from .First() and .Next() methods of RemoteDb (#292)
* return error from .First() and .Next() methods of RemoteDb
* re-run CI
2020-01-06 18:38:21 +07:00
Alex Sharov
805f9aa501
benchmark rpc daemon
* tracing, write bench1() result into files

* add context to computeTxEnv, remote_db cursor lazy cache allocation,

* more results

* added

* added

* added

* more info

* make linter happy

* exit from tracing gorutines

* remove unused param

* re-run CI

* re-run CI

* remove concept txHandle

* batch cursor in reports

* remove debug output

* fix shadow variables and unhandled errors
2019-12-22 21:10:46 +06:00
alex.sharov
e6b33b376f make ci happy 2019-12-16 20:55:10 +06:00
alex.sharov
88b05bd162 make linter happy 2019-12-16 20:54:30 +06:00
alex.sharov
71dc098968 include to report` only blocks on chain with highest difficulty 2019-12-09 21:34:47 +07:00
alex.sharov
04a6517c5b added gasLimits to report object 2019-12-09 21:34:47 +07:00
alex.sharov
7a4023b4eb added gasLimits to report object 2019-12-09 21:34:47 +07:00
alex.sharov
005d715545 move gasLimits to remote db 2019-12-09 21:34:47 +07:00
alex.sharov
bb2f7080ce Moved connection to .View method. Not very optimal, but it allow cancel connection from client side or configure timeout, also in future it will allow us make connection pool there (maybe don't need). 2019-12-06 08:12:02 +07:00
alex.sharov
8b69c54936 move context to from .NewDb to .View method 2019-12-06 08:12:02 +07:00
alex.sharov
344b8029cc handle interruption signals, added context for client and server. Only problem that it's context to DB, but not to each server call - it means you can't use HTTP Request context here (or create new connection on each request). 2019-12-06 08:12:02 +07:00
alex.sharov
49a614eae8 Bug fixes:
- rollback must catch local variable, if you write value to lastError variable
- then you can't do "return" - because it closing connection and client can't send CmdLastError
- somewhere error values was ommited in loging
2019-12-06 08:12:02 +07:00
alex.sharov
75e0b1bc85 revert changed bucket name 2019-12-06 08:12:02 +07:00
alex.sharov
1ffbb97752 removed unnecessary allocations, add tcp context 2019-12-06 08:12:02 +07:00
alex.sharov
d8f93ae187 Batch cursor.First and cursor.Next for remote db. 2019-12-06 08:12:02 +07:00
Igor Mandrigin
ac085db07d
Use 'cobra' package to make a more modular CLI for stateless ethereum prototype. (#200) 2019-11-25 14:46:36 +01:00