Commit Graph

469 Commits

Author SHA1 Message Date
Pau
4be8cfe42c accounts/keystore: fix typo in error message (#21200) 2020-08-07 11:13:12 +02:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation (#851)
* Jumpdest skipping optimisation

* Fix formatting

* Move skipAnalysis into vmConfig, introduce tracing ability

* Improve detection logging

* Added release instructions

* Fix lint
2020-08-01 17:56:57 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly (#748)
* close goruotines

* fix using defer

* close pm managers a bit later

* fmt

* finalizer for in-mem db

* after cr

* revert finalizer

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
665b1d88b9
Speed up GenerateChain by using intermediate hashes (#736)
* GenerateChain to support intermediate hashes

* Fix formatting

* Changeset stats

* Fix formatting
2020-07-10 22:37:34 +01:00
ledgerwatch
887fc4f99c
GenerateChain without using TrieDbState (#726)
* Different chain generation

* More fixes

* Fix TestSideLogRebirth

* Fix the rest of core tests

* Fix accounts/abi/bind

* Fix compile errors for cmd/pics

* Fix consensus/clique tests

* More fixes

* Fix core/state tests

* Fix eth tests

* Fix eth/downloader tests

* Fix more tests

* Fix compile errors

* Some more fixes

* Save

* Exists

* Fix last tests

* Clean up

* Fix formatting

* Fix linter

* Fix formatting

* Fix linter

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint
2020-07-09 07:15:28 +01:00
ledgerwatch
4c456a649f
Separating HashState and IntermediateHashes stages again, fixing errors (#688)
* TestWatchNoDir not to be parallel

* Print ks and inc

* Print addrHash

* Change the buffer

* Print loading

* Skip

* More logging

* Error out earlier

* Handle empty codes

* Handle empty codes

* Remove logging

* Compare states

* Not do stage5

* compareBucket

* Preimage

* Clearer errors

* No need to clean up contract code

* Restore stage6

* Printing

* Skip the skipping

* Print all

* Change buffer type

* Add limit to stage5

* Always fail

* Remove exessive logging

* Restore buffer type

* Revert

* Print when exception

* Reenable skipping

* Skip storage items for deleted contracts

* not shortcut

* Remove removeAccount

* Re-enable state hashing

* Default to plain state

* Disable hashing state

* Reenable reset5

* Print unfurl list

* Enable removingAccount

* No printing

* Reenable stage5 commit

* Swap order of stages

* Prevent backwards promotion, reset tx lookup

* reset finish

* Introduce storage item replacement

* See if unwind works

* Restore removingAccount

* Don't do removeAccount for unwinding

* Possible fix

* Proper(er) fix

* Don't exclude unwinding

* Remove unwinding flag

* Fix formatting

* Fix lint

* Not to ignore blocks if they cause reorg

* Fix test, separate stages again

* Fix TestUnwind

* Fix stage

* Swap unwinding

* Revert to unwinding flag

* Print unfurl list

* Print

* Print inside receive

* Print after

* No printing

* Cleanup

* Not use blockCache when doing GetBlock
2020-06-26 07:46:05 +01:00
Igor Mandrigin
93f6c4c01a post-rebase fixups 2020-06-15 19:38:54 +03:00
Marius van der Wijden
2d592517b9 internal/ethapi: return revert reason for eth_call (#21083)
* internal/ethapi: return revert reason for eth_call

* internal/ethapi: moved revert reason logic to doCall

* accounts/abi/bind/backends: added revert reason logic to simulated backend

* internal/ethapi: fixed linting error

* internal/ethapi: check if require reason can be unpacked

* internal/ethapi: better error logic

* internal/ethapi: simplify logic

* internal/ethapi: return vmError()

* internal/ethapi: move handling of revert out of docall

* graphql: removed revert logic until spec change

* rpc: internal/ethapi: added custom error types

* graphql: use returndata instead of return

Return() checks if there is an error. If an error is found, we return nil.
For most use cases it can be beneficial to return the output even if there
was an error. This code should be changed anyway once the spec supports
error reasons in graphql responses

* accounts/abi/bind/backends: added tests for revert reason

* internal/ethapi: add errorCode to revert error

* internal/ethapi: add errorCode of 3 to revertError

* internal/ethapi: unified estimateGasErrors, simplified logic

* internal/ethapi: unified handling of errors in DoEstimateGas

* rpc: print error data field

* accounts/abi/bind/backends: unify simulatedBackend and RPC

* internal/ethapi: added binary data to revertError data

* internal/ethapi: refactored unpacking logic into newRevertError

* accounts/abi/bind/backends: fix EstimateGas

* accounts, console, internal, rpc: minor error interface cleanups

* Revert "accounts, console, internal, rpc: minor error interface cleanups"

This reverts commit 2d3ef53c5304e429a04983210a417c1f4e0dafb7.

* re-apply the good parts of 2d3ef53c53

* rpc: add test for returning server error data from client

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	accounts/abi/bind/backends/simulated_test.go
2020-06-15 19:38:13 +03:00
Péter Szilágyi
53f6635660 accounts/keystore, cmd/faucet: return old account to allow unlock
# Conflicts:
#	cmd/faucet/faucet.go
2020-06-15 19:38:13 +03:00
Péter Szilágyi
95a3878474 acounts/keystore, cmd/faucet: fix faucet double import, fix twitter url
# Conflicts:
#	cmd/faucet/faucet.go
2020-06-15 19:38:13 +03:00
Martin Michlmayr
732a20d468 all: fix typos in comments (#21118)
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	cmd/puppeth/module_dashboard.go
#	core/blockchain_test.go
#	core/rawdb/chain_iterator.go
#	core/state/snapshot/difflayer.go
#	core/state/snapshot/iterator.go
#	core/state/snapshot/iterator_fast.go
#	core/state/snapshot/snapshot.go
#	core/state/snapshot/wipe.go
#	core/tx_pool.go
#	les/clientpool.go
#	les/peer.go
#	mobile/doc.go
#	trie/committer.go
#	trie/database.go
2020-06-15 19:38:13 +03:00
Boqin Qin
defe185e77 accounts: add blockByNumberNoLock() to avoid double-lock (#20983)
* abi/bind/backends: testcase for double-lock

* accounts: add blockByNumberNoLock to avoid double-lock

* backend/simulated: use stateroot, not blockhash for retrieveing state

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
2020-06-15 19:38:12 +03:00
Marius van der Wijden
88e3533dd5 accounts/abi: simplify reflection logic (#21058)
* accounts/abi: simplified reflection logic

* accounts/abi: simplified reflection logic

* accounts/abi: removed unpack

* accounts/abi: removed comments

* accounts/abi: removed uneccessary complications

* accounts/abi: minor changes in error messages

* accounts/abi: removed unnused code

* accounts/abi: fixed indexed argument unpacking

* accounts/abi: removed superfluous test cases

This commit removes two test cases. The first one is trivially invalid as we have the same
test cases as passing in packing_test.go L375. The second one passes now,
because we don't need the mapArgNamesToStructFields in unpack_atomic anymore.
Checking for purely underscored arg names generally should not be something we do
as the abi/contract is generally out of the control of the user.

* accounts/abi: removed comments, debug println

* accounts/abi: added commented out code

* accounts/abi: addressed comments

* accounts/abi: remove unnecessary dst.CanSet check

* accounts/abi: added dst.CanSet checks
# Conflicts:
#	accounts/abi/reflect.go
2020-06-15 19:38:12 +03:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap (#626)
* optimize codeBitmap

* add lru

* remove custom lru

* jumpDests as explocit parameter

* lint

* lint

* linters

* default jumpDestsCache

* remove common.rand
2020-06-06 21:49:06 +01:00
Evgeny Danilenko
199ede36a2
Prevent slice make and copy in Pushes (#625)
* prevent slice make and copy in Pushes

* do not pad if the same length

* after CR

* linters

* linters

* linters
2020-06-05 19:46:34 +03:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger (#624)
* lmdb tests

* trigger ci

* fix tests

* disable parallelism

* disable parallelism

* cleanup resources

* cleanup resources

* reduce concurency

* try run tests on bolt

* try run tests on bolt

* fix downloader test

* run bolt tests

* rely on interface instead of exact instance

* Rename AbstractKV to KV

* don't use separator for badger

* don't initialize badger cursor - because it not used here

* fix linter

* try reduce badger compactors

* compat with master

* try lmdb

* try lmdb

* try lmdb

* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option

* allow to close closed db

* release

* release

* ideal batch size for badger

* ideal batch size for badger
2020-06-05 10:25:33 +01:00
Andrew Ashikhmin
33c28f7fac
[GC] uint256 rather than big.Int in Transaction (#614)
* uint256 in rlp

* uint256 rather than big.Int in Transation

* linters

* more linters

* still linters

* Reduce garbage in writeUint256

* Experiment with GC in writeByteArray
2020-06-04 08:43:08 +01:00
Alex Sharov
ff1cde5259
simulated backend: replace .MemCopy by .NewBatch+.Commit (#613)
* simulated backend - replace .MemCopy by .NewBatch and native transactions
close database which createtd inside simulated backend
2020-06-04 11:59:16 +07:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb (#584)
* Not hash, keep the files

* Calculate savings

* Fix

* Fix

* Fix

* Fix

* RestAPI to support local boltdb

* Not error on read-only db

* Changes so far

* Continue

* More

* Roll back a bit

* Restore newline

* something compiles

* Fix restapi

* Fix block number

* Fix reads

* Use plain writer

* Maps for storage reads and writes

* Clean up coersions

* Fix accounts/abi/bind

* Fix tests

* More fixes

* more fixes

* More fixes

* Fixes

* Fixed core/state

* Fixed eth tests

* Move code, fix linter

* Fix test

* Fix linter

* Fix linter

* Fix linter, badger_db to support AbstractKV

* Increase IdealBatchSize for badger

* Fix linter

* Fix linter
2020-05-27 17:24:34 +01:00
Andrew Ashikhmin
5bbdb6a4b9
Use uint256 for account balance (#580)
* Use uint256 for account balance

* linter
2020-05-26 17:53:50 +01:00
Andrew Ashikhmin
b16e560529
Use uint256.Int rather than common.Hash for storage values to reduce memory allocation in opSload & opSstore (#575)
* Produce less garbage in GetState

* Still playing with mem allocation in GetCommittedState

* Pass key by pointer in GetState as well

* linter

* Avoid a memory allocation in opSload

* Use uint256.Int rather than common.Hash for storage values to reduce memory allocation in opSload & opSstore

* linter

* linters

* small clean up
2020-05-25 12:12:25 +01:00
Andrew Ashikhmin
df28575420
Use pointers to hashes in Get(Commited)State to reduce memory allocation (#573)
* Produce less garbage in GetState

* Still playing with mem allocation in GetCommittedState

* Pass key by pointer in GetState as well

* linter

* Avoid a memory allocation in opSload
2020-05-24 18:43:54 +02:00
Igor Mandrigin
0dae3ade0d post-rebase fixups 2020-05-20 15:26:22 +03:00
Marius van der Wijden
c9fb0f9a3f accounts/abi: allow overloaded argument names (#21060)
* accounts/abi: allow overloaded argument names

In solidity it is possible to create the following contract:
```
contract Overloader {
    struct F { uint _f; uint __f; uint f; }
    function f(F memory f) public {}
}
```
This however resulted in a panic in the abi package.

* accounts/abi fixed error handling
2020-05-20 15:26:22 +03:00
Marius van der Wijden
fc91ce52eb accounts/abi: accounts/abi/bind: Move topics to abi package (#21057)
* accounts/abi/bind: added test cases for waitDeployed

* accounts/abi/bind: added test case for boundContract

* accounts/abi/bind: removed unnecessary resolve methods

* accounts/abi: moved topics from /bind to /abi

* accounts/abi/bind: cleaned up format... functions

* accounts/abi: improved log message

* accounts/abi: added type tests

* accounts/abi/bind: remove superfluous template methods
# Conflicts:
#	accounts/abi/bind/bind.go
#	accounts/abi/topics.go
#	accounts/abi/topics_test.go
2020-05-20 15:26:22 +03:00
Martin Holst Swende
f66c118cad cmd, core, eth: background transaction indexing (#20302)
* cmd, core, eth: init tx lookup in background

* core/rawdb: tiny log fixes to make it clearer what's happening

* core, eth: fix rebase errors

* core/rawdb: make reindexing less generic, but more optimal

* rlp: implement rlp list iterator

* core/rawdb: new implementation of tx indexing/unindex using generic tx iterator and hashing rlp-data

* core/rawdb, cmd/utils: fix review concerns

* cmd/utils: fix merge issue

* core/rawdb: add some log formatting polishes

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	consensus/clique/snapshot_test.go
#	core/bench_test.go
#	core/block_validator_test.go
#	core/blockchain.go
#	core/blockchain_test.go
#	core/chain_makers_test.go
#	core/dao_test.go
#	core/rawdb/accessors_indexes.go
#	core/rawdb/schema.go
#	eth/config.go
#	eth/helper_test.go
#	eth/sync.go
#	light/odr_test.go
#	light/trie_test.go
#	light/txpool_test.go
#	miner/worker_test.go
#	tests/block_test_util.go
2020-05-20 15:26:22 +03:00
gary rong
5f7d454115 accounts/abi/bind/backend, internal/ethapi: recap gas limit with balance (#21043)
* accounts/abi/bind/backend, internal/ethapi: recap gas limit with balance

* accounts, internal: address comment and fix lint

* accounts, internal: extend log message

* tiny nits to format hexutil.Big and nil properly

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
2020-05-20 15:26:22 +03:00
Marius van der Wijden
a222f5f6cb accounts/abi/bind: add void if no return args specified (#21002)
* accounts/abi/bind: add void if no return args specified

Currently the java generator generates invalid input on pure/view functions
that have no return type. e.g. `function f(uint u) view public {}`
This is not a problem in practice as people rarely ever write functions like this.

* accounts/abi/bind: use elseif instead of nested if
2020-05-20 15:26:22 +03:00
Marius van der Wijden
ffc0f9e85b account/abi: remove superfluous type checking (#21022)
* accounts/abi: added getType func to Type struct

* accounts/abi: fixed tuple unpack

* accounts/abi: removed type.Type

* accounts/abi: added comment

* accounts/abi: removed unused types

* accounts/abi: removed superfluous declarations

* accounts/abi: typo
# Conflicts:
#	accounts/abi/numbers.go
2020-05-20 15:26:22 +03:00
Igor Mandrigin
46f9df7821 accounts/abi: move U256Bytes to common/math (#21020)
# Conflicts:
#	accounts/abi/abi_test.go
#	accounts/abi/numbers.go
#	signer/core/signed_data.go
2020-05-20 15:26:22 +03:00
Marius van der Wijden
7441eac281 accounts/abi: removed Kind from Type struct (#21009)
* accounts/abi: removed Kind from Type struct

* accounts/abi: removed unused code
2020-05-20 15:26:22 +03:00
Martin Holst Swende
b3a6c8eadd accounts/external: convert signature v value to 0/1 (#20997)
This fixes an issue with clef, which already transforms the signature
to use the legacy 27/28 encoding.

Fixes #20994
2020-05-20 15:26:22 +03:00
Martin Holst Swende
28b2a2a78b accounts/external: fill account-cache if that hasn't already been done, fixes #20995 (#20998) 2020-05-20 15:26:22 +03:00
Marius van der Wijden
3c2b9248ed accounts/abi: added abi test cases, minor bug fixes (#20903)
* accounts/abi: added documentation

* accounts/abi: reduced usage of arguments.LengthNonIndexed

* accounts/abi: simplified reflection logic

* accounts/abi: moved testjson data into global declaration

* accounts/abi: removed duplicate test cases

* accounts/abi: reworked abi tests

* accounts/abi: added more tests for abi packing

* accounts/abi/bind: refactored base tests

* accounts/abi: run pack tests as subtests

* accounts/abi: removed duplicate tests

* accounts/abi: removed unnused arguments.LengthNonIndexed

Due to refactors to the code, we do not need the arguments.LengthNonIndexed function anymore.
You can still get the length by calling len(arguments.NonIndexed())

* accounts/abi: added type test

* accounts/abi: modified unpack test to pack test

* accounts/abi: length check on arrayTy

* accounts/abi: test invalid abi

* accounts/abi: fixed rebase error

* accounts/abi: fixed rebase errors

* accounts/abi: removed unused definition

* accounts/abi: merged packing/unpacking tests

* accounts/abi: fixed [][][32]bytes encoding

* accounts/abi: added tuple test cases

* accounts/abi: renamed getMockLog -> newMockLog

* accounts/abi: removed duplicate test

* accounts/abi: bools -> booleans
# Conflicts:
#	accounts/abi/unpack_test.go
2020-05-20 15:26:22 +03:00
Marius van der Wijden
a7e120dc4c accounts/keystore: fix double import race (#20915)
* accounts/keystore: fix race in Import/ImportECDSA

* accounts/keystore: added import/export tests

* cmd/geth: improved TestAccountImport test

* accounts/keystore: added import/export tests

* accounts/keystore: fixed naming

* accounts/keystore: fixed typo

* accounts/keystore: use mutex instead of rwmutex

* accounts: use errors instead of fmt
# Conflicts:
#	accounts/keystore/keystore_test.go
#	cmd/geth/accountcmd_test.go
2020-05-20 15:26:22 +03:00
gary rong
ad9e4241b1 all: seperate consensus error and evm internal error (#20830)
* all: seperate consensus error and evm internal error

There are actually two types of error will be returned when
a tranaction/message call is executed: (a) consensus error
(b) evm internal error. The former should be converted to
a consensus issue, e.g. The sender doesn't enough asset to
purchase the gas it specifies. The latter is allowed since
evm itself is a blackbox and internal error is allowed to happen.

This PR emphasizes the difference by introducing a executionResult
structure. The evm error is embedded inside. So if any error
returned, it indicates consensus issue happens.

And also this PR improve the `EstimateGas` API to return the concrete
revert reason if the transaction always fails

* all: polish

* accounts/abi/bind/backends: add tests

* accounts/abi/bind/backends, internal: cleanup error message

* all: address comments

* core: fix lint

* accounts, core, eth, internal: address comments

* accounts, internal: resolve revert reason if possible

* accounts, internal: address comments
# Conflicts:
#	accounts/abi/abi.go
#	accounts/abi/bind/backends/simulated.go
#	cmd/geth/retesteth.go
#	core/state/snapshot/difflayer_test.go
#	core/state/snapshot/disklayer_test.go
#	core/state/snapshot/iterator_test.go
#	core/state_processor.go
#	core/state_transition.go
#	core/vm/evm.go
#	core/vm/instructions.go
#	core/vm/jump_table.go
#	eth/api_tracer.go
#	internal/ethapi/api.go
#	les/odr_test.go
#	light/odr_test.go
#	tests/state_test_util.go
2020-05-20 15:26:22 +03:00
Marius van der Wijden
b3ce15932b accounts/abi: Prevent recalculation of internal fields (#20895)
* accounts/abi: prevent recalculation of ID, Sig and String

* accounts/abi: fixed unpacking of no values

* accounts/abi: multiple fixes to arguments

* accounts/abi: refactored methodName and eventName

This commit moves the complicated logic of how we assign method names
and event names if they already exist into their own functions for
better readability.

* accounts/abi: prevent recalculation of internal

In this commit, I changed the way we calculate the string
representations, sig representations and the id's of methods. Before
that these fields would be recalculated everytime someone called .Sig()
.String() or .ID() on a method or an event.

Additionally this commit fixes issue #20856 as we assign names to inputs
with no name (input with name "" becomes "arg0")

* accounts/abi: added unnamed event params test

* accounts/abi: fixed rebasing errors in method sig

* accounts/abi: fixed rebasing errors in method sig

* accounts/abi: addressed comments

* accounts/abi: added FunctionType enumeration

* accounts/abi/bind: added test for unnamed arguments

* accounts/abi: improved readability in NewMethod, nitpicks

* accounts/abi: method/eventName -> overloadedMethodName
# Conflicts:
#	accounts/abi/abi.go
#	accounts/abi/abi_test.go
2020-05-20 15:26:22 +03:00
Igor Mandrigin
8c3d19fd4c
geth 1.9.13 (#469)
* core: initial version of state snapshots

* core/state: lazy sorting, snapshot invalidation

* core/state/snapshot: extract and split cap method, cover corners

* snapshot: iteration and buffering optimizations

* core/state/snapshot: unlink snapshots from blocks, quad->linear cleanup

* 123

* core/rawdb, core/state/snapshot: runtime snapshot generation

* core/state/snapshot: fix difflayer origin-initalization after flatten

* add "to merge"

* core/state/snapshot: implement snapshot layer iteration

* core/state/snapshot: node behavioural difference on bloom content

* core: journal the snapshot inside leveldb, not a flat file

* core/state/snapshot: bloom, metrics and prefetcher fixes

* core/state/snapshot: move iterator out into its own files

* core/state/snapshot: implement iterator priority for fast direct data lookup

* core/state/snapshot: full featured account iteration

* core/state/snapshot: faster account iteration, CLI integration

* core: fix broken tests due to API changes + linter

* core/state: fix an account resurrection issue

* core/tests: test for destroy+recreate contract with storage

* squashme

* core/state/snapshot, tests: sync snap gen + snaps in consensus tests

* core/state: extend snapshotter to handle account resurrections

* core/state: fix account root hash update point

* core/state: fix resurrection state clearing and access

* core/state/snapshot: handle deleted accounts in fast iterator

* core: more blockchain tests

* core/state/snapshot: fix various iteration issues due to destruct set

* core: fix two snapshot iterator flaws, decollide snap storage prefix

* core/state/snapshot/iterator: fix two disk iterator flaws

* core/rawdb: change SnapshotStoragePrefix to avoid prefix collision with preimagePrefix

* params: begin v1.9.13 release cycle

* cmd/checkpoint-admin: add some documentation (#20697)

* go.mod: update duktape to fix sprintf warnings (#20777)

This revision of go-duktype fixes the following warning

```
duk_logging.c: In function ‘duk__logger_prototype_log_shared’:
duk_logging.c:184:64: warning: ‘Z’ directive writing 1 byte into a region of size between 0 and 9 [-Wformat-overflow=]
  184 |  sprintf((char *) date_buf, "%04d-%02d-%02dT%02d:%02d:%02d.%03dZ",
      |                                                                ^
In file included from /usr/include/stdio.h:867,
                 from duk_logging.c:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 25 and 85 bytes into a destination of size 32
   36 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   37 |       __bos (__s), __fmt, __va_arg_pack ());
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* core/rawdb: fix freezer table test error check

Fixes: Condition is always 'false' because 'err' is always 'nil'

* core/rawdb: improve table database (#20703)

This PR fixes issues in TableDatabase.

TableDatabase is a wrapper of underlying ethdb.Database with an additional prefix.
The prefix is applied to all entries it maintains. However when we try to retrieve entries
from it we don't handle the key properly. In theory the prefix should be truncated and
only user key is returned. But we don't do it in some cases, e.g. the iterator and batch
replayer created from it. So this PR is the fix to these issues.

* eth: when triggering a sync, check the head header TD, not block

* internal/web3ext: fix clique console apis to work on missing arguments

* rpc: dont log an error if user configures --rpcapi=rpc... (#20776)

This just prevents a false negative ERROR warning when, for some unknown
reason, a user attempts to turn on the module rpc even though it's already going
to be on.

* node, cmd/clef: report actual port used for http rpc (#20789)

* internal/ethapi: don't set sender-balance to maxuint, fixes #16999 (#20783)

Prior to this change, eth_call changed the balance of the sender account in the
EVM environment to 2^256 wei to cover the gas cost of the call execution.
We've had this behavior for a long time even though it's super confusing.

This commit sets the default call gasprice to zero instead of updating the balance,
which is better because it makes eth_call semantics less surprising. Removing
the built-in balance assignment also makes balance overrides work as expected.

* metrics: disable CPU stats (gosigar) on iOS

* cmd/devp2p: tweak DNS TTLs (#20801)

* cmd/devp2p: tweak DNS TTLs

* cmd/devp2p: bump treeNodeTTL to four weeks

* cmd/devp2p: lower route53 change limit again (#20819)

* cmd/devp2p: be very correct about route53 change splitting (#20820)

Turns out the way RDATA limits work is documented after all,
I just didn't search right. The trick to make it work is to
count UPSERTs twice.

This also adds an additional check to ensure TTL changes are
applied on existing records.

* graphql, node, rpc: fix typos in comments (#20824)

* eth: improve shutdown synchronization (#20695)

* eth: improve shutdown synchronization

Most goroutines started by eth.Ethereum didn't have any shutdown sync at
all, which lead to weird error messages when quitting the client.

This change improves the clean shutdown path by stopping all internal
components in dependency order and waiting for them to actually be
stopped before shutdown is considered done. In particular, we now stop
everything related to peers before stopping 'resident' parts such as
core.BlockChain.

* eth: rewrite sync controller

* eth: remove sync start debug message

* eth: notify chainSyncer about new peers after handshake

* eth: move downloader.Cancel call into chainSyncer

* eth: make post-sync block broadcast synchronous

* eth: add comments

* core: change blockchain stop message

* eth: change closeBloomHandler channel type

* eth/filters: fix typo on unindexedLogs function's comment (#20827)

* core: bump txpool tx max size to 128KB

* snapshotter/tests: verify snapdb post-state against trie (#20812)

* core/state/snapshot: basic trie-to-hash implementation

* tests: validate snapshot after test

* core/state/snapshot: fix review concerns

* cmd, consensus: add option to disable mmap for DAG caches/datasets (#20484)

* cmd, consensus: add option to disable mmap for DAG caches/datasets

* consensus: add benchmarks for mmap with/with lock

* cmd/clef: add newaccount command (#20782)

* cmd/clef: add newaccount command

* cmd/clef: document clef_New, update API versioning

* Update cmd/clef/intapi_changelog.md

Co-Authored-By: ligi <ligi@ligi.de>

* Update signer/core/uiapi.go

Co-Authored-By: ligi <ligi@ligi.de>

Co-authored-by: ligi <ligi@ligi.de>

* eth: add debug_accountRange API (#19645)

This new API allows reading accounts and their content by address range.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>

* travis: allow cocoapods deploy to fail (#20833)

* metrics: improve TestTimerFunc (#20818)

The test failed due to what appears to be fluctuations in time.Sleep, which is
not the actual method under test. This change modifies it so we compare the
metered Max to the actual time instead of the desired time.

* README: update private network genesis spec with istanbul (#20841)

* add istanbul and muirGlacier to genesis states in README

* remove muirGlacier, relocate istanbul

* cmd/evm: Rework execution stats (#20792)

- Dump stats also for --bench flag.
- From memory stats only show number and size of allocations. This is what `test -bench` shows. I doubt others like number of GC runs are any useful, but can be added if requested.
- Now the mem stats are for single execution in case of --bench.

* cmd/devp2p, cmd/wnode, whisper: add missing calls to Timer.Stop (#20843)

* p2p/server: add UDP port mapping goroutine to wait group (#20846)

* accounts/abi faster unpacking of int256 (#20850)

* p2p/discv5: add missing Timer.Stop calls (#20853)

* miner/worker: add missing timer.Stop call (#20857)

* cmd/geth: fix bad genesis test (#20860)

* eth/filters: add missing Ticker.Stop call (#20862)

* eth/fetcher: add missing timer.Stop calls (#20861)

* event: add missing timer.Stop call in TestFeed (#20868)

* metrics: add missing calls to Ticker.Stop in tests (#20866)

* ethstats: add missing Ticker.Stop call (#20867)

* p2p/discv5, p2p/testing: add missing Timer.Stop calls in tests (#20869)

* core: add missing Timer.Stop call in TestLogReorgs (#20870)

* rpc: add missing timer.Stop calls in websocket tests (#20863)

* crypto/ecies: improve concatKDF (#20836)

This removes a bunch of weird code around the counter overflow check in
concatKDF and makes it actually work for different hash output sizes.

The overflow check worked as follows: concatKDF applies the hash function N
times, where N is roundup(kdLen, hashsize) / hashsize. N should not
overflow 32 bits because that would lead to a repetition in the KDF output.

A couple issues with the overflow check:

- It used the hash.BlockSize, which is wrong because the
  block size is about the input of the hash function. Luckily, all standard
  hash functions have a block size that's greater than the output size, so
  concatKDF didn't crash, it just generated too much key material.
- The check used big.Int to compare against 2^32-1.
- The calculation could still overflow before reaching the check.

The new code in concatKDF doesn't check for overflow. Instead, there is a
new check on ECIESParams which ensures that params.KeyLen is < 512. This
removes any possibility of overflow.

There are a couple of miscellaneous improvements bundled in with this
change:

- The key buffer is pre-allocated instead of appending the hash output
  to an initially empty slice.
- The code that uses concatKDF to derive keys is now shared between Encrypt
  and Decrypt.
- There was a redundant invocation of IsOnCurve in Decrypt. This is now removed
  because elliptic.Unmarshal already checks whether the input is a valid curve
  point since Go 1.5.

Co-authored-by: Felix Lange <fjl@twurst.com>

* rpc: metrics for JSON-RPC method calls (#20847)

This adds a couple of metrics for tracking the timing
and frequency of method calls:

- rpc/requests gauge counts all requests
- rpc/success gauge counts requests which return err == nil
- rpc/failure gauge counts requests which return err != nil
- rpc/duration/all timer tracks timing of all requests
- rpc/duration/<method>/<success/failure> tracks per-method timing

* mobile: use bind.NewKeyedTransactor instead of duplicating (#20888)

It's better to reuse the existing code to create a keyed transactor
than to rewrite the logic again.

* internal/ethapi: add CallArgs.ToMessage method (#20854)

ToMessage is used to convert between ethapi.CallArgs and types.Message.
It reduces the length of the DoCall method by about half by abstracting out
the conversion between the CallArgs and the Message. This should improve the
code's maintainability and reusability.

* eth, les: fix flaky tests (#20897)

* les: fix flaky test

* eth: fix flaky test

* cmd/geth: enable metrics for geth import command (#20738)

* cmd/geth: enable metrics for geth import command

* cmd/geth: enable metrics-flags for import command

* core/vm: use a callcontext struct (#20761)

* core/vm: use a callcontext struct

* core/vm: fix tests

* core/vm/runtime: benchmark

* core/vm: make intpool push inlineable, unexpose callcontext

* docs/audits: add discv5 protocol audits from LA and C53 (#20898)

* .github: change gitter reference to discord link in issue template (#20896)

* couple of fixes to docs in clef (#20900)

* p2p/discover: add initial discovery v5 implementation (#20750)This adds an implementation of the current discovery v5 spec.There is full integration with cmd/devp2p and enode.Iterator in thisversion. In theory we could enable the new protocol as a replacement ofdiscovery v4 at any time. In practice, there will likely be a few morechanges to the spec and implementation before this can happen.

* build: upgrade to golangci-lint 1.24.0 (#20901)

* accounts/scwallet: remove unnecessary uses of fmt.Sprintf

* cmd/puppeth: remove unnecessary uses of fmt.Sprintf

* p2p/discv5: remove unnecessary use of fmt.Sprintf

* whisper/mailserver: remove unnecessary uses of fmt.Sprintf

* core: goimports -w tx_pool_test.go

* eth/downloader: goimports -w downloader_test.go

* build: upgrade to golangci-lint 1.24.0

* accounts/abi/bind: Refactored topics  (#20851)

* accounts/abi/bind: refactored topics

* accounts/abi/bind: use store function to remove code duplication

* accounts/abi/bind: removed unused type defs

* accounts/abi/bind: error on tuples in topics

* Cosmetic changes to restart travis build

Co-authored-by: Guillaume Ballet <gballet@gmail.com>

* node: allow websocket and HTTP on the same port (#20810)

This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546.

    geth --rpc --rpcport 8545 --ws --wsport 8545

This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.

* crypto: improve error messages in LoadECDSA (#20718)

This improves error messages when the file is too short or too long.
Also rewrite the test for SaveECDSA because LoadECDSA has its own
test now.

Co-authored-by: Felix Lange <fjl@twurst.com>

* changed date of rpcstack.go since new file (#20904)

* accounts/abi/bind: fixed erroneous filtering of negative ints (#20865)

* accounts/abi/bind: fixed erroneous packing of negative ints

* accounts/abi/bind: added test cases for negative ints in topics

* accounts/abi/bind: fixed genIntType for go 1.12

* accounts/abi: minor  nitpick

* cmd: deprecate --testnet, use named networks instead (#20852)

* cmd/utils: make goerli the default testnet

* cmd/geth: explicitly rename testnet to ropsten

* core: explicitly rename testnet to ropsten

* params: explicitly rename testnet to ropsten

* cmd: explicitly rename testnet to ropsten

* miner: explicitly rename testnet to ropsten

* mobile: allow for returning the goerli spec

* tests: explicitly rename testnet to ropsten

* docs: update readme to reflect changes to the default testnet

* mobile: allow for configuring goerli and rinkeby nodes

* cmd/geth: revert --testnet back to ropsten and mark as legacy

* cmd/util: mark --testnet flag as deprecated

* docs: update readme to properly reflect the 3 testnets

* cmd/utils: add an explicit deprecation warning on startup

* cmd/utils: swap goerli and ropsten in usage

* cmd/geth: swap goerli and ropsten in usage

* cmd/geth: if running a known preset, log it for convenience

* docs: improve readme on usage of ropsten's testnet datadir

* cmd/utils: check if legacy `testnet` datadir exists for ropsten

* cmd/geth: check for legacy testnet path in console command

* cmd/geth: use switch statement for complex conditions in main

* cmd/geth: move known preset log statement to the very top

* cmd/utils: create new ropsten configurations in the ropsten datadir

* cmd/utils: makedatadir should check for existing testnet dir

* cmd/geth: add legacy testnet flag to the copy db command

* cmd/geth: add legacy testnet flag to the inspect command

* les, les/lespay/client: add service value statistics and API (#20837)

This PR adds service value measurement statistics to the light client. It
also adds a private API that makes these statistics accessible. A follow-up
PR will add the new server pool which uses these statistics to select
servers with good performance.

This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* README: update min go version to 1.13 (#20911)

* travis, appveyor, build, Dockerfile: bump Go to 1.14.2 (#20913)

* travis, appveyor, build, Dockerfile: bump Go to 1.14.2

* travis, appveyor: force GO111MODULE=on for every build

* core/rawdb: fix data race between Retrieve and Close (#20919)

* core/rawdb: fixed data race between retrieve and close

closes https://github.com/ethereum/go-ethereum/issues/20420

* core/rawdb: use non-atomic load while holding mutex

* all: simplify and fix database iteration with prefix/start (#20808)

* core/state/snapshot: start fixing disk iterator seek

* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start

* les, core/state/snapshot: iterator fixes

* all: remove two iterator methods

* all: rename Iteratee.NewIteratorWith -> NewIterator

* ethdb: fix review concerns

* params: update CHTs for the 1.9.13 release

* params: release Geth v1.9.13

* added some missing files

* post-rebase fixups

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: gary rong <garyrong0905@gmail.com>
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Co-authored-by: meowsbits <45600330+meowsbits@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>
Co-authored-by: Ha ĐANG <dvietha@gmail.com>
Co-authored-by: Hanjiang Yu <42531996+de1acr0ix@users.noreply.github.com>
Co-authored-by: ligi <ligi@ligi.de>
Co-authored-by: Wenbiao Zheng <delweng@gmail.com>
Co-authored-by: Adam Schmideg <adamschmideg@users.noreply.github.com>
Co-authored-by: Jeff Wentworth <jeff@curvegrid.com>
Co-authored-by: Paweł Bylica <chfast@gmail.com>
Co-authored-by: ucwong <ucwong@126.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Luke Champine <luke.champine@gmail.com>
Co-authored-by: Boqin Qin <Bobbqqin@gmail.com>
Co-authored-by: William Morriss <wjmelements@gmail.com>
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
Co-authored-by: Raw Pong Ghmoa <58883403+q9f@users.noreply.github.com>
Co-authored-by: Felföldi Zsolt <zsfelfoldi@gmail.com>
2020-04-19 18:31:47 +01:00
Andrew Ashikhmin
435d56e2fc
Timestamps must be sorted prior to appending into thin history index (#411)
* Timestamps must be sorted prior to appending into thin history index

* Skip unstable test
2020-03-27 16:46:34 +01:00
Igor Mandrigin
5ff3948129
Remove error param from NewTrieDbState (#408) 2020-03-26 16:21:15 +03:00
Andrew Ashikhmin
abadbdfb80
[Issue 340] Preserve the original when a contract is self-destructed and then its address is touched in the same block (e.g. # 156634) (#397)
* root was unused in BlockChain.StateAt

* TestDoubleAccountRemoval

* Preserve the original when a contract is self-destructed and then its address is touched in the same block (e.g. #
156634)
2020-03-20 10:08:13 +00:00
gary rong
11b53b12bc les, miner, accounts/abi/bind: fix load-sensitive unit tests (#20698) 2020-03-16 16:40:19 +03:00
Igor Mandrigin
edcd16c7ab post-merge fixups 2020-02-27 17:22:25 +03:00
Boqin Qin
1d0b41b6b0 accounts: add walletsNoLock to avoid double read lock (#20655) 2020-02-27 17:21:20 +03:00
Marius van der Wijden
a8b4a7cd49 accounts/abi/bind/backends: add support for historical state (#20644) 2020-02-27 17:21:20 +03:00
Evgeny Danilenko
17a4a56634
Refactor mining. Remove a few mining goroutines (#338)
* add context

* extract chain events

* run commit in goroutines

* mine only on canonical

* typo

* linters

* fmt

* mark unused methods

* restore stress test

* test single miner

* remove unsafe Trie storage

* remove locks from miner

* restore interrupt

* remove result goroutine

* remove unconfirmedBlocks

* cherry-pick 04a1d475ff1a36ad8f92fec80385df18c52bdc1f

* extract uncles

* one miner succeeded

* restore context cancel

* cleanup

* skip an unstable test

* remove pending state

* use context instead of interrupt func

* calculate sealHash only once

* comment out unstable test

* after merge

* fix after merge

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2020-02-03 15:02:26 +03:00
Igor Mandrigin
785d0eafd8 post-rebase fixups v1.9.10 2020-01-30 13:36:30 +02:00
Sylvain Laurent
9b3fc4282b accounts/abi: fix method constant flag for solidity 6.0 (#20482) 2020-01-30 13:36:30 +02:00
Ilan Gitter
97cf3f15fc accounts/abi/backends/simulated: add more API methods (#5) (#20208)
* Add more functionality to the sim (#5)

* backends: implement more of ethclient in sim

* backends: add BlockByNumber to simulated backend

* backends: make simulated progress function agree with syncprogress interface for client

* backends: add more tests

* backends: add more comments

* backends: fix sim for index in tx and add tests

* backends: add lock back to estimategas

* backends: goimports

* backends: go ci lint

* Add more functionality to the sim (#5)

* backends: implement more of ethclient in sim

* backends: add BlockByNumber to simulated backend

* backends: make simulated progress function agree with syncprogress interface for client

* backends: add more tests

* backends: add more comments

* backends: fix sim for index in tx and add tests

* backends: add lock back to estimategas

* backends: goimports

* backends: go ci lint

* assert errs
2020-01-30 13:36:30 +02:00
Jeff Wentworth
278982f1e8 [#20266] Fix bugs decoding integers and fixed bytes in indexed event fields (#20269)
* fix parseTopics() and add tests

* remove printf

* add ParseTopicsIntoMap() tests

* fix FixedBytesTy

* fix int and fixed bytes

* golint topics_test.go
2020-01-30 13:36:30 +02:00