Commit Graph

13291 Commits

Author SHA1 Message Date
Péter Szilágyi
40a366086e core/state: convert prefetcher to concurrent per-trie loader
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	core/blockchain.go
#	core/state/state_object.go
#	core/state/state_test.go
#	core/state/statedb.go
#	core/state/statedb_test.go
#	eth/api_tracer.go
#	miner/worker.go
2021-03-09 12:30:09 +01:00
Martin Holst Swende
6c3d78d0d9 core: implement background trie prefetcher
Squashed from the following commits:

core/state: lazily init snapshot storage map
core/state: fix flawed meter on storage reads
core/state: make statedb/stateobjects reuse a hasher
core/blockchain, core/state: implement new trie prefetcher
core: make trie prefetcher deliver tries to statedb
core/state: refactor trie_prefetcher, export storage tries
blockchain: re-enable the next-block-prefetcher
state: remove panics in trie prefetcher
core/state/trie_prefetcher: address some review concerns

sq

# Conflicts:
#	core/blockchain.go
#	core/state/database.go
#	core/state/state_object.go
#	core/state/statedb.go
2021-03-09 11:15:19 +01:00
meowsbits
71a78fafd0 downloader: extract findAncestor search functions (#21744)
This is a simple refactoring, extracting common ancestor
negotiation logic to named function
2021-03-09 11:13:39 +01:00
ucwong
13eecf52cf go.mod: upgrade golang-lru (#22134)
# Conflicts:
#	go.mod
#	go.sum
2021-03-09 11:13:31 +01:00
Marius van der Wijden
14a8a75552 oss-fuzz: fix abi fuzzer (#22199) 2021-03-09 10:51:27 +01:00
gary rong
773ca75cc9 eth, les: add new config field SyncFromCheckpoint (#22123)
This PR introduces a new config field SyncFromCheckpoint for light client.

In some special scenarios, it's required to start synchronization from some
arbitrary checkpoint or even from the scratch. So this PR offers this
flexibility to users so that the synchronization start point can be configured.

There are two relevant configs: SyncFromCheckpoint and Checkpoint.

- If the SyncFromCheckpoint is true, the light client will try to sync from the
  specified checkpoint.

- If the Checkpoint is not configured, then the light client will sync from the
  scratch(from the latest header if the database is not empty)

Additional notes: these two configs are not visible in the CLI flags but only
accessable in the config file.

Example Usage:

[Eth]
SyncFromCheckpoint = true

[Eth.Checkpoint]
SectionIndex = 100
SectionHead = "0xabc"
CHTRoot = "0xabc"
BloomRoot = "0xabc"

PS. Historical checkpoint can be retrieved from the synced full node or light
client via les_getCheckpoint API.
# Conflicts:
#	eth/gen_config.go
#	les/client_handler.go
#	les/commons.go
#	les/sync.go
#	les/sync_test.go
#	les/test_helper.go
2021-03-09 10:51:19 +01:00
Alex Mazalov
ed42a61cc8 cmd/geth: graceful shutdown if disk is full (#22103)
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/cmd.go
2021-03-09 10:50:14 +01:00
gary rong
6236523169 core/state/snapshot: write snapshot generator in batch (#22163)
* core/state/snapshot: write snapshot generator in batch

* core: refactor the tests

* core: update tests

* core: update tests
# Conflicts:
#	core/blockchain_snapshot_test.go
#	core/state/snapshot/generate.go
#	core/state/snapshot/journal.go
2021-03-09 10:49:30 +01:00
Marius van der Wijden
e78f1a9f6e cmd/geth: dump config for metrics (#22083)
* cmd/geth: dump config

* cmd/geth: dump config

* cmd/geth: properly read config again

* cmd/geth: override metrics if flags are set

* cmd/geth: write metrics regardless if enabled

* cmd/geth: renamed to metricsfromcliargs

* metrics: add default configuration
# Conflicts:
#	cmd/geth/config.go
2021-03-09 10:49:12 +01:00
Marius van der Wijden
519d6e3edf tests/fuzzers/abi: better test generation (#22158)
* tests/fuzzers/abi: better test generation

* tests/fuzzers/abi: fixed packing issue

* oss-fuzz: enable abi fuzzer
# Conflicts:
#	tests/fuzzers/abi/abifuzzer.go
2021-03-09 10:48:53 +01:00
gary rong
6d9c22a5a0 les: remove useless protocol defines (#22115)
This PR has two changes in the les protocol:

- the auxRoot is not supported. See ethereum/devp2p#171 for more information
- the empty response will be returned in GetHelperTrieProofsMsg request if the merkle
   proving is failed. note, for backward compatibility, the empty merkle proof as well as
   the request auxiliary data will still be returned in  les2/3 protocol no matter the proving
   is successful or not. the proving failure can happen e.g. request the proving for a
   non-included entry in helper trie (unstable header).
# Conflicts:
#	les/benchmark.go
#	les/handler_test.go
#	les/odr_requests.go
#	les/server_handler.go
2021-03-09 10:38:55 +01:00
Dan DeGreef
a3d76f177b eth/protocols/eth: fix slice resize flaw (#22181) 2021-03-09 10:38:31 +01:00
gary rong
ffc675a3c2 les: don't drop sentTo for normal cases (#22048)
# Conflicts:
#	les/retrieve.go
2021-03-09 10:38:25 +01:00
Péter Szilágyi
6df58fe9eb core/state/snapshot: add generation logs to storage too
# Conflicts:
#	core/state/snapshot/generate.go
2021-03-09 10:38:08 +01:00
Péter Szilágyi
c933238203 cmd/faucet: update the embedded website asset
# Conflicts:
#	cmd/faucet/website.go
2021-03-09 10:37:52 +01:00
Martin Holst Swende
94556a7db7 graphql: fix spurious travis failure (#22166)
The tests sometimes failed with certain go versions because
the behavior of http.Server.Shutdown changed over time. A bug
that was fixed in Go 1.15 could cause active connections on unrelated
servers to close unexpectedly. This is fixed by avoiding use of the
same port number in all tests.

# Conflicts:
#	graphql/graphql_test.go
2021-03-09 10:37:26 +01:00
Marius van der Wijden
33201429e0 consensus/ethash: increase seal timeout for tests (#22162)
It seems that the 2 second timeout is not enough for Travis CI:

   --- FAIL: TestTestMode (2.00s)
       ethash_test.go:53: sealing result timeout
2021-03-09 10:36:52 +01:00
Martin Holst Swende
2fe0c82161 graphql: fix spurious error in test (#22164)
This solves an issue in graphql tests:

    graphql_test.go:38: could not create new node: datadir already used by another process
2021-03-09 10:36:43 +01:00
Martin Holst Swende
57c26675fc tests: update the reference tests (#22009) 2021-03-09 10:36:34 +01:00
Guillaume Ballet
e235d3c43a cmd/utils: don't enumerate USB unless --usb is set (#22130)
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
# Conflicts:
#	cmd/geth/accountcmd_test.go
#	cmd/geth/consolecmd_test.go
#	cmd/geth/dao_test.go
#	cmd/geth/genesis_test.go
#	cmd/geth/les_test.go
2021-03-09 10:36:28 +01:00
Martin Holst Swende
adf219ea76 go.mod: use github.com/holiman/bloomfilter/v2 (#22044)
* deps: use improved bloom filter implementation

* eth/handler, trie: use 4 keys for syncbloom + minor fixes

* eth/protocols, trie: revert change on syncbloom method signature
# Conflicts:
#	core/state/snapshot/difflayer.go
#	go.mod
#	go.sum
#	trie/sync.go
#	turbo/trie/sync_bloom.go
2021-03-09 10:36:02 +01:00
meowsbits
b82525b79b cmd/utils: avoid making console preloads absolute (#22109)
Resolves https://github.com/etclabscore/core-geth/issues/273

jsre.JSRE already handles establishing preload
file paths relative to the 'assets' path (aka docroot),
where it joins the assets dir and the file path if relative,
or uses the file path only if absolute.

The duplication of this logic by MakeConsolePreloads
caused preloaded files to have paths which contained
duplicate references to the assets dir path.

Date: 2020-12-30 08:25:01-06:00
Signed-off-by: meows <b5c6@protonmail.com>
2021-03-09 10:33:36 +01:00
rene
a12aeca94b eth: return error from eth_chainID during sync before EIP-155 activates (#21686)
This changes the chainID RPC method to return an error when EIP-155 is not yet
active at the current block height. It used to simply return zero in this case, but
that's confusing.
2021-03-09 10:33:28 +01:00
rene
8af432b89e graphql: fix issue with unmarshalling int32 into Long type #22153 2021-03-09 10:33:20 +01:00
Marius van der Wijden
993d973de4 eth: improve log message (#22146)
* eth: fixed typos

* eth: fixed log message
2021-03-09 10:32:29 +01:00
Péter Szilágyi
7b7969408d common/prque: pull in tests and benchmarks from upstream 2021-03-09 10:32:21 +01:00
gary rong
df56644121 core: persist bad blocks (#21827)
* core: persist bad blocks

* core, eth, internal: address comments

* core/rawdb: add badblocks to inspector

* core, eth: update

* internal: revert

* core, eth: only save 10 bad blocks

* core/rawdb: address comments

* core/rawdb: fix

* core: address comments
# Conflicts:
#	core/blockchain.go
#	core/rawdb/accessors_chain_test.go
#	core/rawdb/database.go
#	core/rawdb/schema.go
2021-03-09 10:31:26 +01:00
Chris Ziogas
07503e3d11 eth/downloader: fix race condition in tests (#22140)
* downloader: fix race condition in tests

* eth/downloader: fix race condition in tests

* Revert "downloader: fix race condition in tests"

This reverts commit 108033ebc6985de83791d375b6e6647a77d28d5a.
2021-03-09 10:26:12 +01:00
Marius van der Wijden
867df20910 ethclient: better test suite for ethclient package (#22127)
This commit extends the ethclient test suite and increases code coverage of the ethclient
package from ~15% to >55%. These tests act as early smoke tests to signal issues in the
RPC-interface. E.g. if a functionality like eth_chainId or eth_call breaks, the test
will break.
# Conflicts:
#	ethclient/ethclient_test.go
2021-03-09 10:26:10 +01:00
Martin Holst Swende
332d481ebe cmd/faucet: fix nonce-gap problem (#22145)
* cmd/faucet: avoid encoding for each client

* cmd/faucet: fix flaw in clearing of txs, avoid sending more than necessary

* cmd/faucet: fix flaw in tx cropping

* cmd/faucet: revert change to not always send tx info

* cmd/faucet: review fixes

* cmd/faucet: revert #22018, fix order in UI

* cmd/faucet: fix lock error

* cmd/faucet: revert json changes

* squashme
# Conflicts:
#	cmd/faucet/faucet.go
#	cmd/faucet/faucet.html
2021-03-09 10:24:39 +01:00
gary rong
83471766f3 les: remove transaction propagation limits (#22125)
# Conflicts:
#	les/txrelay.go
2021-03-09 10:24:22 +01:00
Péter Szilágyi
736a88c9c1 cmd/faucet: switch Facebook auth over to mobile site
# Conflicts:
#	cmd/faucet/faucet.go
2021-03-09 10:24:05 +01:00
Martin Holst Swende
ec6741bab0 eth/protocols/snap: speed up hash checks (#22023)
* eth/protocols/snap: speed up hash checks

* eth/protocols/snap: nit fix

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-03-09 10:23:30 +01:00
Martin Holst Swende
b128d8aeea eth/protocols/snap: track reverts when peer rejects request (#22016)
* eth/protocols/snap: reschedule missed deliveries

* eth/protocols/snap: clarify log message

* eth/protocols/snap: revert failures async and update runloop

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-03-09 10:23:19 +01:00
Péter Szilágyi
608b052481 cmd/faucet: fix websocket race regression after switching to gorilla
# Conflicts:
#	cmd/faucet/faucet.go
2021-03-09 10:23:07 +01:00
Melvin Junhee Woo
deb2dff0bc snapshot, trie: fixed typos, mostly in snapshot pkg (#22133)
# Conflicts:
#	core/state/snapshot/conversion.go
#	core/state/snapshot/difflayer.go
#	core/state/snapshot/difflayer_test.go
#	core/state/snapshot/disklayer.go
#	core/state/snapshot/disklayer_test.go
#	core/state/snapshot/iterator.go
#	core/state/snapshot/iterator_binary.go
#	core/state/snapshot/iterator_fast.go
#	core/state/snapshot/snapshot.go
#	turbo/trie/database.go
2021-03-09 10:22:48 +01:00
rene
49f677bc1d graphql: return decimal for estimateGas and cumulativeGas queries (#22126)
* estimateGas, cumulativeGas
* linted
* add test for estimateGas
2021-03-09 10:22:24 +01:00
Marius van der Wijden
9f29fe3ea3 crypto: fix ineffectual assignments (#22124)
* crypto/bls12381: fixed ineffectual assignment

* crypto/signify: fix ineffectual assignment
2021-03-09 10:22:15 +01:00
Péter Szilágyi
a50f261e87 cmd/utils, eth/downloader: minor snap nitpicks 2021-03-09 10:22:04 +01:00
jk-jeongkyun
18852aa9b3 eth/downloader: enhanced test cases for downloader queue (#22114)
# Conflicts:
#	eth/downloader/queue_test.go
2021-03-09 10:21:52 +01:00
Marius van der Wijden
8d6f77a735 common/compiler: fix parsing of solc output with solidity v.0.8.0 (#22092)
Solidity 0.8.0 changes the way that output is marshalled. This patch allows to parse both
the legacy format used previously and the new format.

See also https://docs.soliditylang.org/en/breaking/080-breaking-changes.html#interface-changes
2021-03-09 10:18:03 +01:00
Marius van der Wijden
78213e6c0e cmd/geth: added --mainnet flag (#21932)
* cmd/geth: added --mainnet flag

* cmd/utils: set default genesis if --mainnet is specified

* cmd/utils: addressed comments
# Conflicts:
#	cmd/geth/chaincmd.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
2021-03-09 10:17:55 +01:00
Antoine Toulme
886d2a4780 graphql: use a decimal representation for gas limit and gas used (#21883)
This changes the JSON encoding of blocks returned by the API
to have decimal instead of hexadecimal numbers. The spec wants
it this way.

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	graphql/graphql_test.go
2021-03-09 10:17:02 +01:00
Guillaume Ballet
f50f13d70c cmd/geth: usb is off by default (#21984)
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
2021-03-09 10:16:32 +01:00
Martin Holst Swende
20413ad716 miner: avoid sleeping in miner (#22108)
This PR removes a logic in the miner, which was originally intended to help temporary testnets based on ethash from "running off into the future". If the difficulty was low, and a few computers started mining several blocks per second, the ethash rules (which demand 1s delay between blocks) would push the blocktimes further and further away.
The solution was to make the miner sleep while this happened.

Nowadays, this problem is solved instead by PoA chains, and it's recommended to let testnets and devnets be based on clique instead. The existing logic is problematic, since it can cause stalls within the miner making it difficult for remote workers to submit work if the channel is blocked on a sleep.

Credits to Saar Tochner for reporting this via the bug bounty
# Conflicts:
#	miner/worker.go
2021-03-09 10:15:27 +01:00
Péter Szilágyi
7b9565fe10 cmd: support v1.1 Twitter API in faucet, fix puppeth
# Conflicts:
#	cmd/faucet/faucet.go
2021-03-09 10:14:38 +01:00
Suriyaa Sundararuban
9e08f7759f SECURITY.md: link to release page (#22067)
Add links to go-ethereum's GitHub release page.

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-03-09 10:14:23 +01:00
rene
b734ae5e54 node: rename startNetworking to openEndpoints (#22105) 2021-03-09 10:14:13 +01:00
Vie
767a37fbeb cmd/geth: update copyright year (#22099)
# Conflicts:
#	cmd/geth/main.go
2021-03-09 10:14:06 +01:00
Melvin Junhee Woo
0699dae18a core/state/snapshot: gethring -> gathering typo (#22104)
# Conflicts:
#	core/state/snapshot/generate.go
2021-03-09 10:13:46 +01:00