Commit Graph

51 Commits

Author SHA1 Message Date
ledgerwatch
767a374c2e
Tx pool use triedbstate, fixes to GetNodeData and move it to experimental (#364)
* Use TrieDbState for tx pool

* Not initialise tx pool until state is loaded

* Add preimage

* Fix account

* Print codehash

* Print correct code hash

* Print incarnatin

* Print incarnatin

* Use proper incarnation

* Print dbValue

* Actually fix

* Actually fix

* Fix verifySnapshot

* readAccount to get code hash

* Next incarnation

* Print addrHashes with 0 incarnations

* Print storage history

* Print storage history

* Print storage history

* Print storage history

* Print all storage history

* print change set keys

* print change set keys

* print change set keys

* print change set keys

* Not print codebucket info

* Fixes

* Fix for incarnation

* Fix for storage history bucket

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Fix embedded nodes

* Hasher

* Fix

* Test fixes

* Add experimental debug flag

* Fix tx_pool_test

* Disable GetNodeData test unless in experiment

* Fix more tests

* Fix lint and revert some changes

* Fix lint

* Fix lint
2020-02-10 17:05:32 +00:00
Igor Mandrigin
f4d0747949
stateless: flush the partial witnesses DB for every saved witness (#354)
* stateless: flush the partial witnesses DB for every saved witness.

it seems to fix to the out of memory error I had on blocks over 2.5M;

* fix linters

* simplify
2020-02-06 13:58:58 +03:00
ledgerwatch
dca36e8b29
Restore the functionality CalcTrieRoots (compute trie root without modifying the trie) (#327)
* Trace first block

* Fixes for CalcTrieRoots

* Timings of the CalcTrieRoot

* Fix lint

* Add memory profiling

* Reduce allocations in StreamHash

* Fix

* Fix

* Fix

* Optimise allocations

* Reuse streams

* Fixes

* Fix

* Unit test fix

* Fix lint

* Reuse hashbuilder

* No loop

* Reuse resolver

* Fixed tests

* Fix test

* Fix test

* Fix test

* Fix witness threshold

* Optimise allocations in RLP transform

* Optimise allocations in RLP transform

* Optimise branchHash

* 100 times again

* Replace ToStream with Iterator

* StreamMergeIterator

* No streams

* Amplification

* Minimise the use of hashOnly

* 100 times

* Reduce stack operations

* Reduce appends

* More optimisations

* More optimisations

* More optimisations

* local hex

* Small fix

* Remove test

* Fix lint

* Fix lint

* Fix lint

* Add test for empty

* Fix lint

* More tests

* Fix lint

* Add measurement of stateless exec
2020-01-30 13:16:12 +00:00
Igor Mandrigin
1115b23832
Stateless Resolver: store and use witnesses for resolved subtries (#326) 2020-01-24 11:58:01 +01:00
ledgerwatch
4f5c858f00
Collect data about Keccak256 invocations for evaluation of STARK proof sizes and performance (#315)
* Gathering start data on witnesses

* Fix number parsing

* Proper filename, actually outputting

* Correct block

* Visualise states

* Code and value fix

* Visualise code hash and storage hash

* Current block number

* Print bucket

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix typo

* Fixes after merging master

* Fix lint

Co-authored-by: b00ris <b00ris@mail.ru>
2020-01-15 17:33:36 +00:00
b00ris
df8798b94b
Thin history for storage (#324)
* save state

* save state

* fmt

* add check to test

* move incarnation length to common

* remove line

* fix lint
2020-01-15 17:55:43 +03:00
Igor Mandrigin
b7710ff7fa
New Witness Format (and remove the rest of the tapes) (#317) 2020-01-15 14:56:50 +01:00
Alex Sharov
fc7291ec34
Use local db in analytics (#308)
* Use local db in analytics (#308)
* pool of encoders
* incr and decr funcs
2020-01-15 19:47:13 +07:00
Alex Sharov
438476d1ba
Serialization bench (#288)
* bench shows that need to pass pointer to []byte when use encoder.Encode()
2020-01-08 13:09:07 +07:00
Alex Sharov
33308dd5c2
State analytic less memory and checkpoints (#299)
* added prefix tree to analyses to reduce memory usage
* make new partition every day
* merge concepts of reporter and snapshot
* tests for .FirstKey() and .NextKey()
2020-01-07 09:27:19 +07:00
Alex Sharov
85009dabe6
return error from .First() and .Next() methods of RemoteDb (#292)
* return error from .First() and .Next() methods of RemoteDb
* re-run CI
2020-01-06 18:38:21 +07:00
Alex Sharov
805f9aa501
benchmark rpc daemon
* tracing, write bench1() result into files

* add context to computeTxEnv, remote_db cursor lazy cache allocation,

* more results

* added

* added

* added

* more info

* make linter happy

* exit from tracing gorutines

* remove unused param

* re-run CI

* re-run CI

* remove concept txHandle

* batch cursor in reports

* remove debug output

* fix shadow variables and unhandled errors
2019-12-22 21:10:46 +06:00
alex.sharov
e6b33b376f make ci happy 2019-12-16 20:55:10 +06:00
alex.sharov
88b05bd162 make linter happy 2019-12-16 20:54:30 +06:00
alex.sharov
71dc098968 include to report` only blocks on chain with highest difficulty 2019-12-09 21:34:47 +07:00
alex.sharov
04a6517c5b added gasLimits to report object 2019-12-09 21:34:47 +07:00
alex.sharov
7a4023b4eb added gasLimits to report object 2019-12-09 21:34:47 +07:00
alex.sharov
005d715545 move gasLimits to remote db 2019-12-09 21:34:47 +07:00
alex.sharov
bb2f7080ce Moved connection to .View method. Not very optimal, but it allow cancel connection from client side or configure timeout, also in future it will allow us make connection pool there (maybe don't need). 2019-12-06 08:12:02 +07:00
alex.sharov
8b69c54936 move context to from .NewDb to .View method 2019-12-06 08:12:02 +07:00
alex.sharov
ada5d88be8 fix linter warnings 2019-12-06 08:12:02 +07:00
alex.sharov
344b8029cc handle interruption signals, added context for client and server. Only problem that it's context to DB, but not to each server call - it means you can't use HTTP Request context here (or create new connection on each request). 2019-12-06 08:12:02 +07:00
alex.sharov
49a614eae8 Bug fixes:
- rollback must catch local variable, if you write value to lastError variable
- then you can't do "return" - because it closing connection and client can't send CmdLastError
- somewhere error values was ommited in loging
2019-12-06 08:12:02 +07:00
alex.sharov
75e0b1bc85 revert changed bucket name 2019-12-06 08:12:02 +07:00
alex.sharov
1ffbb97752 removed unnecessary allocations, add tcp context 2019-12-06 08:12:02 +07:00
alex.sharov
d8f93ae187 Batch cursor.First and cursor.Next for remote db. 2019-12-06 08:12:02 +07:00
Igor Mandrigin
28d9b7c812
Use binary tries for the stateless action as an option. (#218) 2019-12-04 13:48:38 +01:00
ledgerwatch
8cb96471a3
Skeleton of RPC daemon, for 1 operation eth_blockNumber (#209)
* Finish CmdSeek and add CmdNext

* Add remoteDb listener and RPC daemon CLI

* Fix test

* Fix CLI

* Fix lint

* Fix unreachable code

* Fix lint

* First working RPC command eth_blockNumber

* Fix lint

* Fix lint

* Fix memprofile/cpuprofile confusion

* Add comment
2019-12-02 13:47:00 +00:00
Igor Mandrigin
ebae2fe281
Add --noverify flag to /cmd/state to skip roots check. (#205) 2019-11-27 14:52:22 +01:00
Igor Mandrigin
4fb54a7b82
Write proper stats to the CSV file; restore charting capabilities. (#201) 2019-11-26 14:29:41 +01:00
Igor Mandrigin
ac085db07d
Use 'cobra' package to make a more modular CLI for stateless ethereum prototype. (#200) 2019-11-25 14:46:36 +01:00
ledgerwatch
b2ca635d86
Debug tool for recursively comparing state (in DB or in a file) with geth archive node (#191)
* Fetching results of eth_getProof

* Dump 5 levels of the trie in a file for repeated runs

* Drill down to 6 levels of the trie

* Fix lint

* Fix lint

* Fix lint

* verifySnapshot to check accounts with emptyRoot

* Descend into short nodes

* Latest tool fixes

* Fix lint

* Fix state properly working
2019-11-25 13:36:21 +00:00
ledgerwatch
244d70fb9c
Further fixes for the no-mod-root (#186)
* Further fixes

* Repace 1000 with a symbol
2019-11-21 15:56:39 +00:00
Igor Mandrigin
aa71b298cc
Write tape stats to a csv file. (#185) 2019-11-21 15:09:21 +01:00
Igor Mandrigin
f97578bb04
Use ethdb.Database interface to generate snapshots. (#182) 2019-11-21 14:57:44 +01:00
Igor Mandrigin
f4cda8ba4c
Fix snapshot saving and add an interval setting. (#181) 2019-11-21 14:36:24 +01:00
ledgerwatch
c187d80152
Preparations for computing root without modifying the trie (#172)
Preparations for computing root without modifying the trie
2019-11-15 22:48:49 +00:00
ledgerwatch
f70252e121
Merge pull request #167 from ledgerwatch/statelesss_resolver
Prine tries (in memory) in the same cycle as db commits
2019-11-15 16:02:51 +00:00
Alexey Akhunov
540eb04273 Prine tries (in memory) in the same cycle as db commits, commit before closing db 2019-11-13 02:26:07 +00:00
Evgeny Danienko
5b4a371489 fmt 2019-11-07 19:01:58 +03:00
b00ris
6e880cc945
suffix to changeset rename (#152) 2019-11-07 18:51:25 +03:00
ledgerwatch
c28f4b203c
Merge pull request #153 from ledgerwatch/stateless_proto
Control the size of the trie in stateless prototype, save state snapshot after errors
2019-11-07 12:30:32 +00:00
Alexey Akhunov
71f3e7f3c4 Fix lint 2019-11-07 11:17:29 +00:00
Alexey Akhunov
a0aa6bac03 Fix lint 2019-11-07 11:14:30 +00:00
Alexey Akhunov
9cfceb90cd Save state snapshot after errors 2019-11-07 11:10:15 +00:00
Alexey Akhunov
49a478876e Control the size of the trie in stateless prototype 2019-11-07 10:36:27 +00:00
Giulio
138241bd2c linting 2019-11-05 22:51:11 +01:00
ledgerwatch
9258f510f2 Merge pull request #132 from ledgerwatch/state_growth_stats
Fix the data collection and charts for state growth vs block gas limit
2019-11-04 14:15:26 +01:00
ledgerwatch
f760001638 Merge pull request #112 from ledgerwatch/block_witness 2019-11-04 14:15:26 +01:00
ledgerwatch
f0dd8f407e Merge pull request #125 from ledgerwatch/broken_snapshot
Not to wrap storage values into RLP during reorgs
2019-11-04 14:15:26 +01:00