Commit Graph

1701 Commits

Author SHA1 Message Date
ledgerwatch
5904332035
hack --action resetState to create both hashed and plain genesis, clean all buckets (#563)
* Reset plain genesis

* Change to optimised freelist implementation

* Fix linter

* Revert "Change to optimised freelist implementation"

This reverts commit 65e28f1737801178e95585381a789785504ec318.

* Noerrcheck
2020-05-21 06:18:25 +01:00
Igor Mandrigin
12a6b2f9f6
Use upstream EVMC (#561)
* Use upstream EVMC

* update to 7.3.0
2020-05-20 19:35:32 +01:00
Andrew Ashikhmin
9c47df0a2c
Use math.bits in common math Safe* for performance (#556)
* Use math.bits in comoon math Safe* for performance (the compiler optimizes away  math.bits)

* Prettier code in opExp
2020-05-20 14:31:23 +02:00
Andrew Ashikhmin
ff23652d50
GC: reducing copying in stateObject.updateTrie (#557)
* It's unnecessary to copy key in stateObject.updateTrie

* Move value copying from stateObject.updateTrie down to TrieStateWriter
2020-05-20 14:31:03 +02:00
Igor Mandrigin
0dae3ade0d post-rebase fixups 2020-05-20 15:26:22 +03:00
gary rong
1aa6ead45f core: fix missing receipt on Clique crashes (#21045)
* core: fix missing receipt

* core: address comment
# Conflicts:
#	core/blockchain.go
2020-05-20 15:26:22 +03:00
ucwong
0d2edd5653 core/rawdb: remove unused math (#21065) 2020-05-20 15:26:22 +03:00
ucwong
1ed0a13d69 core/rawdb : log format fix for Unindexing transaction (#21064)
* core/rawdb : log format fix for Unindexing transaction

* core/rawdb: tiny fixup

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2020-05-20 15:26:22 +03:00
Martin Holst Swende
42a23703f2 core: fixup blockchain tests (#21062)
core: fixup blockchain tests
# Conflicts:
#	core/blockchain_test.go
2020-05-20 15:26:22 +03:00
Martin Holst Swende
f66c118cad cmd, core, eth: background transaction indexing (#20302)
* cmd, core, eth: init tx lookup in background

* core/rawdb: tiny log fixes to make it clearer what's happening

* core, eth: fix rebase errors

* core/rawdb: make reindexing less generic, but more optimal

* rlp: implement rlp list iterator

* core/rawdb: new implementation of tx indexing/unindex using generic tx iterator and hashing rlp-data

* core/rawdb, cmd/utils: fix review concerns

* cmd/utils: fix merge issue

* core/rawdb: add some log formatting polishes

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	consensus/clique/snapshot_test.go
#	core/bench_test.go
#	core/block_validator_test.go
#	core/blockchain.go
#	core/blockchain_test.go
#	core/chain_makers_test.go
#	core/dao_test.go
#	core/rawdb/accessors_indexes.go
#	core/rawdb/schema.go
#	eth/config.go
#	eth/helper_test.go
#	eth/sync.go
#	light/odr_test.go
#	light/trie_test.go
#	light/txpool_test.go
#	miner/worker_test.go
#	tests/block_test_util.go
2020-05-20 15:26:22 +03:00
Boqin Qin
b42bb4db08 core: avoid double-lock in tx_pool_test (#20984) 2020-05-20 15:26:22 +03:00
Igor Mandrigin
46f9df7821 accounts/abi: move U256Bytes to common/math (#21020)
# Conflicts:
#	accounts/abi/abi_test.go
#	accounts/abi/numbers.go
#	signer/core/signed_data.go
2020-05-20 15:26:22 +03:00
Felix Lange
d5469b3b3c core: improve TestLogRebirth (#20961)
This is a resubmit of #20668 which rewrites the problematic test
without any additional goroutines. It also documents the test better.

The purpose of this test is checking whether log events are sent
correctly when importing blocks. The test was written at a time when
blockchain events were delivered asynchronously, making the check hard
to pull off. Now that core.BlockChain delivers events synchronously
during the call to InsertChain, the test can be simplified.

Co-authored-by: BurtonQin <bobbqqin@gmail.com>
# Conflicts:
#	core/blockchain_test.go
2020-05-20 15:26:22 +03:00
Boqin Qin
9258f47717 core: add check in AddChildIndexer to avoid double lock (#20982)
This fixes a theoretical double lock condition which could occur in

    indexer.AddChildIndexer(indexer)

Nobody would ever do that though.

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-05-20 15:26:22 +03:00
icodezjb
83857d4f4a core: mirror full node reorg logic in light client too (#20931)
* core: fix the condition of reorg

* core: fix nitpick to only retrieve head once

* core: don't reorg if received chain is longer at same diff

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	core/headerchain.go
2020-05-20 15:26:22 +03:00
gary rong
ad9e4241b1 all: seperate consensus error and evm internal error (#20830)
* all: seperate consensus error and evm internal error

There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.

This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.

And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails

* all: polish

* accounts/abi/bind/backends: add tests

* accounts/abi/bind/backends, internal: cleanup error message

* all: address comments

* core: fix lint

* accounts, core, eth, internal: address comments

* accounts, internal: resolve revert reason if possible

* accounts, internal: address comments
# Conflicts:
#	accounts/abi/abi.go
#	accounts/abi/bind/backends/simulated.go
#	cmd/geth/retesteth.go
#	core/state/snapshot/difflayer_test.go
#	core/state/snapshot/disklayer_test.go
#	core/state/snapshot/iterator_test.go
#	core/state_processor.go
#	core/state_transition.go
#	core/vm/evm.go
#	core/vm/instructions.go
#	core/vm/jump_table.go
#	eth/api_tracer.go
#	internal/ethapi/api.go
#	les/odr_test.go
#	light/odr_test.go
#	tests/state_test_util.go
2020-05-20 15:26:22 +03:00
rene
aa2f2b742a cmd, core: remove override muir glacier and override istanbul (#20942)
# Conflicts:
#	core/genesis.go
#	eth/backend.go
#	les/client.go
2020-05-20 15:26:22 +03:00
Andrew Ashikhmin
5bc45f2426
Use uint256 in EVM implementation (kudos to Pawel Bylica) (#551)
* core/vm: improve jumpdest lookup

* Use uint256 in EVM implementation (kudos to Pawel Bylica)

* Safety precaution in opMulmod

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-05-18 08:10:59 +01:00
Alex Sharov
b3a5c62147
sort buckets then sort keys inside buckets, then send data to MultiPut (#553) 2020-05-17 05:46:30 +01:00
Alex Sharov
02dd405e93
MGR Schedule v0 (#550)
* MGR Make Schedule v0
2020-05-15 15:58:36 +07:00
Igor Mandrigin
fd77eaf86a
Staged Sync: Execution phase should use "plain state" (#548)
* introduce PlainStateReader with fallbacks

* no 10.000 changes in tests

* even less iterations

* remove even more iterations

* add `go run ./cmd/geth --syncmode staged --plainstate` flag

* fix serialization calls

* make a more sensible file default

doesn’t affect anything, because this flag is always overriden when parsing CLI. but still.
2020-05-15 08:52:45 +01:00
ledgerwatch
729303eb6b
Start transforming SubTrieLoader into iterator/stream (#547)
* Splitting up into interations

* Some fixes

* Fixes

* Fix linter

* Fix linter

* Improvements after review

* Add more diagnostics

* Fix and better printing
2020-05-14 18:12:33 +01:00
ledgerwatch
9f3f768ab4
Rename Resolver => SubTrieLoader, ResolveSet => RetainList (#544)
* Rename Resolver => SubTrieLoader, ResolveSet => RetainList

* Fix linter

* Fix linter

* Fix linter

* Fix linter
2020-05-12 15:24:43 +01:00
ledgerwatch
82a67f4d48
eth_getProof step2 - Decouple Resolver from Trie, remove ResolveRequest (#543)
* Removing PrepareResolveParams and NeedResolution

* Remove ResolveRequest

* Decouple resolver from trie

* Remove hooks from resolver

* Fix util compilation

* More test fixes

* Compiled tests in trie

* Fix trie tests

* Fix linter

* Fix linter

* Fix linter

* Fix linter
2020-05-12 08:22:45 +01:00
ledgerwatch
41b5fce078
resolver: merge HashBuilder-s and ResolveSet-s, decouple from ResolveRequest-s (#540)
* merge hbStorage and hb in resolver

* Fix prepare

* Remove tracing by default

* cleanup

* Fix linter

* Not to include incarnation into the ResolveRequest

* Fix linter

* Fix linter

* Fix test

* Fix linter

* Revert

* Fix for infinite loop

* Fix linter

* Collapse rss into rs

* remove reliance of requests in resolver

* Remove currentReq

* Remove fields from ResolveRequest

* Fix linter
2020-05-11 05:46:07 +01:00
Giulio rebuffo
89a4f8ddff
Fix --rinkeby and --goerli (#539)
* fixed goerli and rinkeby testnets

* merge
2020-05-09 20:29:00 +01:00
ledgerwatch
5b2a11137b
[WIP] Fix block tests that were failing due to CalcTrieRoots (#536)
* Fix block tests, add support for code nodes in the stream

* Fix typo

* Fix test

* Comment cleanup
2020-05-09 05:44:56 +01:00
ledgerwatch
b4d2e57428
Introduce incarnationMap in both DbStateReader and DbStateWriter (#534)
* Add incarnationMap into db_state_reader

* Fix test

* Add incarnationMap back to the DbStateWriter
2020-05-08 05:52:55 +01:00
b00ris
2a6f12520c
remove pruning option. Inverse enabling pruning logic. (#532) 2020-05-07 14:31:14 +01:00
ledgerwatch
2fd8f1704c
Restore EVMC integration for staged sync (accidentally broken), resetState (#529)
* Add resetState

* resetState not to disturb header chain

* Fix linter

* Fix linter

* Restore evmc integration
2020-05-07 06:59:00 +01:00
ledgerwatch
f47d661c34
Proper unwinding in the staged sync (#521)
* introduce fork

* Fix unwind test

* Unwind now really works in the test

* Fix linter

* Fix linter

* Fix linter

* Not to handle block and block hash announced in staged sync
2020-05-06 18:42:10 +01:00
b00ris
abe54cd7a2
Remove EIP2027 (#512)
* save state

* fmt

* save state

* save state

* fix resolve test

* fix lint

* fix test
2020-05-06 09:59:50 +01:00
Alex Sharov
56022d8d85
- parent node type - doesn't matter for IH - just check that node itself is fullNode or duoNode is enough (#518)
- in unloading - 64 nibbles - it's hex of a.storage, but not a - then 64 nibbles hex must be stored in IH with incarnation
- EmptyRoot can't come to `WillUnloadBranchNode` method - because if account has no storage - then accountNode will have no branches in a.storage, a.storage will equal to `hashNode(EmptyRoot)` - and we don't unload hashNodes (only fullNode/duoNode).
2020-05-06 09:58:37 +01:00
Andrew Ashikhmin
39a37f2dd9
Don't check root in accountsEqual, that's covered by storageChanged anyway. (Fixing checkChangeSets for contract 0xe2942418b2d87a1d88cce66d0b11c7ef16a72971 in block 1467584.) (#519) 2020-05-06 09:58:12 +01:00
Alex Sharov
2686bf2f3b
Remove topLevels param of resolver (#515)
* remove topLevels param of resolver

* revert to master version of ih
2020-05-05 21:08:00 +01:00
Igor Mandrigin
cea453be74
Support EVMC (EVMone) (#509)
* add stuff

* vendor evmc and patch import paths

* some fixups

* compiles!

* fixups for evm initialization

* less chatty logs

* fix wrong import in test

* better (time based) logging

* remove unnecessary printf

* evmc as a module

* glue evmc code

* use a proper fork for evmc

* post-rebase fixups

* fix linterz
2020-05-04 17:13:51 +01:00
b00ris
d0af8a2139
Blocks compression (#510)
* block compression

* rerun lint

* fix lint
2020-05-04 06:55:51 +01:00
ledgerwatch
5d6fb16a97
[WIP] Rewinding for the staged sync, test for the staged sync (#508)
* Initial rework

* Fix lint

* boilerplate for the test

* Fix linter

* More to the test

* fix linter

* Save progress

* Test works

* Fix linter

* More on test

* Fix test, linter

* Fix lint

* Fix test
2020-05-03 13:39:50 +01:00
Alex Sharov
af4a99b977
Merge account and storage resolvers (#504)
* add_incarnation_to_acc_root_in_ih

* merge cached resolver into stateful resolver

* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash

* remove blockNR and bucket params from walker

* fix out of range panic

* calc acc.Root on the fly

* remove fieldSet field from resolver, make logic of root - lazy

* remove 2 parameters

* working version of forward-only walk over Acc and Storage

* improve test

* rebase master

* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go

* skip old incarnations

* don't rebuild when 0 requests

* fix tests

* start from account key when need resolve storage

* Error: stateless prototype faced hashNode when extracting witness

* Statless works: copy touches

* Remove getAccRoot function

* Remove "isAccount" parameter from resolver signature

* Fix: use correct storageResolveSet in finaliseStorageRoot

* Fix: when startKey changed - reset storage buffers also

* Fix: if account incarnation=0 - set EmptyRoot

* Fix: remove account roots by default from IntermediateHash bucket

* Fix: skip abandoned storage - which appeared just after startKey

* Fix: did reset acc key incorrectly

* Fix: clean previous key if receive IH

* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)

* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests

* Remove accNode from IH cycle

* Fix flags

* Fix: reset succStorage also

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: use rssStorage to HashOnly check

* Fix: remove termination symbol from resolveRequest

* cleanup

* Fix: skip abandoned storage after IH

* Debug

This reverts commit 9c5eb69465f25607d546b03359b2cbcb1bd46689.

* Fix linters

* add_incarnation_to_acc_root_in_ih

* merge cached resolver into stateful resolver

* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash

* remove blockNR and bucket params from walker

* fix out of range panic

* calc acc.Root on the fly

* remove fieldSet field from resolver, make logic of root - lazy

* remove 2 parameters

* working version of forward-only walk over Acc and Storage

* improve test

* rebase master

* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go

* skip old incarnations

* don't rebuild when 0 requests

* fix tests

* start from account key when need resolve storage

* Error: stateless prototype faced hashNode when extracting witness

* Statless works: copy touches

* Remove getAccRoot function

* Remove "isAccount" parameter from resolver signature

* Fix: use correct storageResolveSet in finaliseStorageRoot

* Fix: when startKey changed - reset storage buffers also

* Fix: if account incarnation=0 - set EmptyRoot

* Fix: remove account roots by default from IntermediateHash bucket

* Fix: skip abandoned storage - which appeared just after startKey

* Fix: did reset acc key incorrectly

* Fix: clean previous key if receive IH

* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)

* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests

* Remove accNode from IH cycle

* Fix flags

* Fix: reset succStorage also

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: use rssStorage to HashOnly check

* Fix: remove termination symbol from resolveRequest

* cleanup

* Fix: skip abandoned storage after IH

* remove inc

* remove inc from rss

* tr.succStorage.Reset()

* remove inc from rss

* Remove hard-coding

* succ.Reset

* Enable CalcTrieRoots

* Proper dumping of the trie

* Debug

* Fix for CalcTrieRoot

* Fix another inteference bug

* Temp

* Fix test

* Cleanup

* remove STORE_ACCOUNT_ROOT=true flag

* Fix linter

* Fix linter

* Disable getnodedata by default

* Fix test

* Fix test

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-05-02 19:00:57 +01:00
Andrew Ashikhmin
7b40cbb6fa
Incarnation should be read by StateReader, not StateWriter (#506)
* GetCurrentAccountIncarnation

* Incarnation should be read by StateReader, not StateWriter

* Use GetHistoricalAccountIncarnation in DbState

* RemoteReader ReadAccountIncarnation

* Handle the case where a contract has self-destructed, then Eth sent to it, then it got recreated again
2020-05-02 19:00:42 +01:00
ledgerwatch
d795bd5a90
Shortcut for 2^e (#505) 2020-05-02 15:42:37 +01:00
ledgerwatch
4991ce6858
Small fixes for download and execute sync stages (#503)
* Small fixes

* Disable tests that require double validation
2020-04-30 17:19:24 +01:00
Igor Mandrigin
30d41380ee
Parallel signature verification (#499)
* create context

* parallel recovery

* recovery should use the parallel contexts

* linters

* linters

* remove a leftover

* fix refactoring leftover
2020-04-29 14:17:28 +03:00
Alex Sharov
8ae055d04c
up syntax of solidity pragma (#498) 2020-04-29 11:51:07 +01:00
ledgerwatch
051323edbb
Tinkering with storage changesets (#497)
* Adjustments

* fix tests

* Fix tests

* Widen most numbers to 32 bits
2020-04-28 16:36:00 +01:00
b00ris
cc137af1ce
[WIP] New storage changeset encoding (#493)
* new storage changeset encoding

* fix lint
2020-04-28 08:41:55 +01:00
ledgerwatch
7211b0f261
Add senders recovery stage to staged sync (#494)
* Maintenance commit

* Fixes

* Fixes

* Fix linters

* Fix linter

* Fix linter
2020-04-28 06:36:51 +01:00
ledgerwatch
a09547e822
Truncate history index after reorgs (#492)
* Refactor RewindData

* Add cpu profile

* Index truncation

* Fix compile

* Fix linter

* Fix linter

* Fix tests

* Create test

* Remove Remove

* Clarifying comment
2020-04-28 06:36:33 +01:00
ledgerwatch
ccc0c4eeb8
Fix receipts (#488) 2020-04-27 07:42:21 +01:00
ledgerwatch
e1cda82eb8
Fix canonical hashes in staged bodies sync (#487)
* Fix canonical hashes

Remove changes in hack.go

* incarnations and db_state_reader

* Fix compilation

* Fix linter

* Test fixes
2020-04-26 22:58:26 +01:00