Commit Graph

290 Commits

Author SHA1 Message Date
b00ris
83e65fdd81 migrations
fix build

add storage mode check

add intermitiate migration state&snapshot migrations
2020-02-26 00:25:30 +03:00
b00ris
2b74b43678 History index encoding
save state

save state

remove old history index

fix lint

compress index
2020-02-24 23:29:46 +03:00
b00ris
958ace4088 split storage and account encoding for changeset 2020-02-24 22:36:16 +03:00
b00ris
e69f53db8d Merge branch 'master' of github.com:ledgerwatch/turbo-geth into split_cs_buckets_squash 2020-02-24 21:47:26 +03:00
b00ris
aedecfc820 split changeset bucket
fmt

badger changes

badget puts

fix shadowing

fix mutation test. restore DecodeChangeSet

fix pruner test
2020-02-24 20:56:32 +03:00
Andrew Ashikhmin
b541bff1eb
Enable GetNodeData by default (#368)
* Add a missing IsGetNodeData check

* Enable GetNodeData by default
2020-02-16 16:50:24 +00:00
Alex Sharov
fdbba5202b
Trie: store self-destructed accounts (#355)
* squash commits

* enable storage cache

* make linter happy

* fix subtree prefix len check

* save cahnges to test master

* remove restriction on prefix len

* fix comparison of last bits

* fix wrong alignment

* remove debug prints

* commit current state

* commit current state

* avoid changing state of resolver from multiwalk

* remove debug code

* remove debug code

* remove debug code

* remove unnecessary copy

* make code more readable

* reduce rebuildHashes initial resolution

* fix test after rebase to master

* make code more readable

* improve pruner

* pruner add IntermediateCache bucket

* fix panic in Walk on short keys

* reduce allocations for storage keys decompression by increasing default buffer size

* re-run CI

* fix iterator behaviour

* rename cache to hash for unification

* re-run ci

* avoid using underlying DB

* hash all subtree nodes before unload

* fix getNode method

* need to check node type, not parent - before put to hashBucket

* return back parent type check, doesn't work without it.

* don't recalculate hash again

* move unloadFunc from trie to pruner

* rename bucket to shorter name

* rename bucket to shorter name

* clean

* rebase to master
2020-02-12 13:52:59 +00:00
Andrew Ashikhmin
05a3483cb1
Fix hash map leaks (#367)
* Enable testGetNodeData by default

* Crude TestHashMapLeak

* nodeFlag ->nodeRef

* linter

* accountNode shouln't be in hashMap since only branch and short nodes are the standart ones

* Finalize hash eviction logic

* small changes to TestHashMapLeak

* Fix for the incorrect hash

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2020-02-12 11:34:44 +00:00
ledgerwatch
767a374c2e
Tx pool use triedbstate, fixes to GetNodeData and move it to experimental (#364)
* Use TrieDbState for tx pool

* Not initialise tx pool until state is loaded

* Add preimage

* Fix account

* Print codehash

* Print correct code hash

* Print incarnatin

* Print incarnatin

* Use proper incarnation

* Print dbValue

* Actually fix

* Actually fix

* Fix verifySnapshot

* readAccount to get code hash

* Next incarnation

* Print addrHashes with 0 incarnations

* Print storage history

* Print storage history

* Print storage history

* Print storage history

* Print all storage history

* print change set keys

* print change set keys

* print change set keys

* print change set keys

* Not print codebucket info

* Fixes

* Fix for incarnation

* Fix for storage history bucket

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Fix embedded nodes

* Hasher

* Fix

* Test fixes

* Add experimental debug flag

* Fix tx_pool_test

* Disable GetNodeData test unless in experiment

* Fix more tests

* Fix lint and revert some changes

* Fix lint

* Fix lint
2020-02-10 17:05:32 +00:00
Evgeny Danilenko
17a4a56634
Refactor mining. Remove a few mining goroutines (#338)
* add context

* extract chain events

* run commit in goroutines

* mine only on canonical

* typo

* linters

* fmt

* mark unused methods

* restore stress test

* test single miner

* remove unsafe Trie storage

* remove locks from miner

* restore interrupt

* remove result goroutine

* remove unconfirmedBlocks

* cherry-pick 04a1d475ff1a36ad8f92fec80385df18c52bdc1f

* extract uncles

* one miner succeeded

* restore context cancel

* cleanup

* skip an unstable test

* remove pending state

* use context instead of interrupt func

* calculate sealHash only once

* comment out unstable test

* after merge

* fix after merge

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2020-02-03 15:02:26 +03:00
ledgerwatch
0cd4f65ffe
Copy values from the ChangeSet bucket during Rewind (#348)
* Fix rewinding

* Fix rewinding
2020-02-01 16:24:38 +00:00
Alex Sharov
ff23980dc6
Intermediate hash phase 2 (#341)
* add env INTERMEDIATE_TRIE_CACHE
* try to use assert.New() pattern
* Fix "maligned" linter warnings to reduce space consumption of structs:

core/types/accounts/account.go:18:14: struct of size 136 bytes could be of size 128 bytes (maligned)
type Account struct {
--
trie/node.go:44:10: struct of size 80 bytes could be of size 72 bytes (maligned)
	duoNode struct {
--
trie/resolve_set.go:28:17: struct of size 56 bytes could be of size 48 bytes (maligned)
type ResolveSet struct {
--
trie/resolver.go:34:15: struct of size 88 bytes could be of size 72 bytes (maligned)
type Resolver struct {
--
trie/visual.go:32:17: struct of size 104 bytes could be of size 96 bytes (maligned)
type VisualOpts struct {
2020-01-31 14:31:50 +07:00
Alex Sharov
0dd4386a37
Intermediate trie hashes phase 1 (#334)
* add intermediate trie hashes bucket
* update docs
* use version of CompressNibbles with buf pool (less GC) but without io.ByteWriter (slow)
2020-01-31 11:11:20 +07:00
Giulio rebuffo
32375df7af made changeSet an helper type (#314)
* made changeSet an helper type

* added comment

* removed dbutils.Add

* remove common.CopyBytes from walk

* fixes
2020-01-15 20:51:10 +00:00
b00ris
df8798b94b
Thin history for storage (#324)
* save state

* save state

* fmt

* add check to test

* move incarnation length to common

* remove line

* fix lint
2020-01-15 17:55:43 +03:00
Alex Sharov
fc7291ec34
Use local db in analytics (#308)
* Use local db in analytics (#308)
* pool of encoders
* incr and decr funcs
2020-01-15 19:47:13 +07:00
Alex Sharov
33308dd5c2
State analytic less memory and checkpoints (#299)
* added prefix tree to analyses to reduce memory usage
* make new partition every day
* merge concepts of reporter and snapshot
* tests for .FirstKey() and .NextKey()
2020-01-07 09:27:19 +07:00
b00ris
bcee4845dc
Thin history (#272)
* save state

* remove repair

* save state

* remove emptydb check

* save state

* add walkAsOf test

* add WalkAsOf and MultiWalkAsOf tests

* deployed contracts counter

* reference counter for contract code

* drop storage root&contract hash for changesets

* start incarnation is 1(save state)

* fix ReorgOverSelfDestruct test

* hack fix TestReorgOverSelfDestruct

* test benchmark

* cleanup

* remove useless debug

* remove print trie

* return remove subtrie call  to updateTrieRoot

* save state

* add mutation test

* remove useless test

* fix

* added mutation commit test

* rename experiment to thin history

* thin history mutation commit test

* fix ethdb tests

* getAsOf test

* add test&fix history index

* fix test

* make test for index search

* compute trie root incarnation fix

* tests fixes

* done job in case of panic

* fix lint

* fix&test for bad incarnation

* fix initial incarnation for genesis

* fix lint

* fix changeset test

* fix storage ranges test

* fix lint

* move set incarnation to create contract

* add comment

Co-authored-by: ledgerwatch <akhounov@gmail.com>
Co-authored-by: Evgeny Danilenko <6655321@bk.ru>
2019-12-20 15:25:40 +03:00
alex.sharov
187670fcda more tests 2019-12-09 21:34:47 +07:00
alex.sharov
432c2a5b26 fix linter warning 2019-12-09 21:34:47 +07:00
alex.sharov
04a6517c5b added gasLimits to report object 2019-12-09 21:34:47 +07:00
Felix Lange
bbea24b5f2 common: improve GraphQL error messages (#20354) 2019-12-03 13:50:58 +01:00
Felix Lange
9f9126f9e8 common/hexutil: improve GraphQL error messages (#20353) 2019-12-03 13:50:16 +01:00
Andrew Ashikhmin
8e5e5d881f
Issue 123 Make encoding of ChangeSet ordered by keys and binary-searchable (#207) 2019-12-03 12:13:12 +01:00
Evgeny Danilenko
992e34745c
Replace red black trees (#184)
* use map instead of rb tree

* GetModifiedAccounts

* introduce tuples

* linters

* linters

* init puts

* init maps

* remove GetSortedKeys

* fix string bucket case

* use append in tuples

* fix tuples

* fix tests

* all tests are green

* fmt

* fmt

* rename tuple to tuples
2019-11-21 18:38:00 +00:00
ledgerwatch
c187d80152
Preparations for computing root without modifying the trie (#172)
Preparations for computing root without modifying the trie
2019-11-15 22:48:49 +00:00
andrew
8cb8544c45 Merge branch 'master' into badger2 2019-11-11 17:12:28 +01:00
b00ris
6e880cc945
suffix to changeset rename (#152) 2019-11-07 18:51:25 +03:00
andrew
9be775c8bd Port my work from badger after the master rebase 2019-11-05 13:28:36 +01:00
b00ris
9eb7d8b1c2 Suffix reorg (#113) 2019-11-04 14:15:26 +01:00
Alexey Akhunov
fe01bccbb8 Apply Turbo-Geth modifications to go-ethereum codebase 2019-11-01 21:52:03 +01:00
Felix Lange
b1c3010bf2
common/mclock: clean up AfterFunc support (#20054)
This change adds tests for the virtual clock and aligns the interface
with the time package by renaming Cancel to Stop. It also removes the
binary search from Stop because it complicates the code unnecessarily.
2019-09-16 11:16:30 +02:00
Péter Szilágyi
305ed955db
Merge pull request #20038 from holiman/minor_encodingfix
core/state: optimize some internals during encoding
2019-09-10 17:12:06 +03:00
Martin Holst Swende
72045dff4f
core/state: optimize some internals during encoding 2019-09-10 15:15:34 +02:00
Péter Szilágyi
49b86a2859
common, graphql: fix hash/address decoding + UI content type 2019-09-10 12:20:36 +03:00
lmittmann
1cd5bf080e common: unify hex prefix check code (#19937) 2019-08-22 11:45:07 +02:00
Felix Lange
9bad7fa717
common/compiler: fix lint issue (#19967) 2019-08-15 12:12:56 +02:00
shiqinfeng1
260b177fe3 common/compiler: support relative import paths (#17374) 2019-08-15 10:33:06 +02:00
Felföldi Zsolt
a7de796840
les: implement new client pool (#19745) 2019-08-03 14:36:10 +02:00
Péter Szilágyi
1a83114c74
all: update author list and licenses 2019-07-22 12:17:27 +03:00
David Chase
31a1f164d9 common/bitutil: use result of TestBytes to prevent dead code elimination (#19846)
Gollvm has very aggressive dead code elimination that completely
removes one of these two benchmarks.  To prevent this, use the
result of the benchmark (a boolean), and to be "fair", make the
transformation to both benchmarks.

To be reliably assured of not removing the code, "use" means
assigning to an exported global.  Non-exported globals and
//go:noinline functions are possibly subject to this optimization.
2019-07-17 10:23:43 +02:00
gary rong
22060611fb cmd/abigen: refactor command line interface (#19797)
* cmd, common: refactor abigen command line interface

* cmd/abigen: address comment
2019-07-08 14:59:07 +02:00
Guillaume Ballet
6bf5555c4f
accounts/abi/bind: Accept function ptr parameter (#19755)
* accounts/abi/bind: Accept function ptr parameter

They are translated as [24]byte

* Add Java template version

* accounts/abi/bind: fix merge issue

* Fix CI
2019-07-02 09:52:58 +02:00
Martin Holst Swende
17381ecc66 core/signer, clef: improve ui-test flow, fix errors in uint handling (#19584)
* core/signer, clef: improve ui-test flow, fix errors in uint handling for eip-712

* core/signer: add fuzzer testcases + crashfixes

* signer: address review concerns, check sign in integer parsing
2019-06-03 16:56:05 +02:00
Péter Szilágyi
fc85777a21
core: concurrent database reinit from freezer dump
* core: reinit chain from freezer in batches

* core/rawdb: concurrent database reinit from freezer dump

* core/rawdb: reinit from freezer in sequential order
2019-05-27 15:48:30 +03:00
gary rong
37d280da41
core, cmd, vendor: fixes and database inspection tool (#15)
* core, eth: some fixes for freezer

* vendor, core/rawdb, cmd/geth: add db inspector

* core, cmd/utils: check ancient store path forceily

* cmd/geth, common, core/rawdb: a few fixes

* cmd/geth: support windows file rename and fix rename error

* core: support ancient plugin

* core, cmd: streaming file copy

* cmd, consensus, core, tests: keep genesis in leveldb

* core: write txlookup during ancient init

* core: bump database version
2019-05-16 10:39:34 +03:00
Kushagra Sharma
fb458280d1 Modified Abigen to Support Vyper (#19120) 2019-03-18 13:29:26 +01:00
Martin Holst Swende
7504dbd6eb core/vm: 64 bit memory and gas calculations (#19210)
* core/vm: remove function call for stack validation from evm runloop

* core/vm: separate gas  calc into static + dynamic

* core/vm: optimize push1

* core/vm: reuse pooled bigints for ADDRESS, ORIGIN and CALLER

* core/vm: use generic error message for jump/jumpi, to avoid string interpolation

* testdata: fix tests for new error message

* core/vm: use 64-bit memory calculations

* core/vm: fix error in memory calculation

* core/vm: address review concerns

* core/vm: avoid unnecessary use of big.Int:BitLen()
2019-03-12 11:40:05 +02:00
Péter Szilágyi
2b6158a51e
common/fdlimit: fix macos file descriptors for Go 1.12 2019-02-27 14:21:02 +02:00
Enrique Fynn
628a0bde3f common/fdlimit: Fix compilation error in freebsd, Raise() returns uint64 (#19141)
cast limit.Cur (int64) to uint64, fix test and compilation error
2019-02-21 11:16:51 +02:00