Commit Graph

2642 Commits

Author SHA1 Message Date
Igor Mandrigin
a122920fa8 and more lint fixes 2021-03-18 11:53:34 +01:00
Igor Mandrigin
36297dd057 more linter fixes 2021-03-18 11:44:22 +01:00
Igor Mandrigin
7b000b5644 linter fixups 2021-03-18 11:17:04 +01:00
Alexey Sharp
534f38f756 Fix lints 2021-03-18 08:30:14 +00:00
Alexey Sharp
1ff525e6a3 Fix lints 2021-03-18 08:00:21 +00:00
Alexey Sharp
469e2534da Fix lints 2021-03-18 07:38:06 +00:00
Alexey Sharp
c1de4bd0f7 Fix lints 2021-03-18 07:27:50 +00:00
Alexey Sharp
cdca7f94ad Fix lints 2021-03-17 22:24:46 +00:00
Alexey Sharp
50ea8f4e20 Fix cmd/rpcdaemon tests 2021-03-17 16:49:02 +00:00
Igor Mandrigin
0cc9fdcb60 remove ethclient and checkpoint-admin (unused) 2021-03-17 16:59:06 +01:00
Igor Mandrigin
0a55895b33 Merge branch 'master' into geth-1-10-1 2021-03-17 16:50:20 +01:00
Alexey Sharp
0a91c9fa9a More compilation fixes 2021-03-15 19:37:37 +00:00
Alexey Sharp
0b083d8557 Fixes 2021-03-15 18:21:10 +00:00
Alexey Sharp
7fd3417f52 Fix more compile errors 2021-03-14 19:07:19 +00:00
Alexey Sharp
af495562e4 Fix rcpdaemon compilation 2021-03-14 18:52:15 +00:00
Alexey Sharp
519b19f8aa Compilation fixes 2021-03-14 08:00:37 +00:00
Alex Sharov
3f305673aa
Trie: add invariant - first level of trie must be in DB (to ensure having 100% trie structure in trie_account table) (#1522) 2021-03-13 09:30:54 +07:00
Marius van der Wijden
e052d8e268 accounts/keystore: use github.com/google/uuid (#22217)
This replaces the github.com/pborman/uuid dependency with
github.com/google/uuid because the former is only a wrapper for
the latter (since v1.0.0).

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	accounts/keystore/key.go
#	accounts/keystore/passphrase.go
#	accounts/keystore/presale.go
#	cmd/ethkey/generate.go
#	go.mod
#	go.sum
2021-03-12 15:52:15 +01:00
Péter Szilágyi
d23002e76a all: define Berlin hard fork spec
# Conflicts:
#	cmd/geth/config.go
#	cmd/geth/main.go
#	core/genesis.go
#	core/state_transition.go
#	core/vm/evm.go
#	core/vm/interpreter.go
#	eth/backend.go
#	eth/ethconfig/config.go
#	les/client.go
#	light/txpool.go
#	tests/state_test_util.go
2021-03-12 15:50:09 +01:00
rene
ba494be5a9 cmd/devp2p: add eth66 test suite (#22363)
Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	go.mod
2021-03-12 15:40:44 +01:00
lightclient
50d975302d all: add support for EIP-2718, EIP-2930 transactions (#21502)
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.

There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.

The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID.

This change also adds support for the YoloV3 testnet.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	cmd/evm/internal/t8ntool/execution.go
#	cmd/evm/internal/t8ntool/transition.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	core/bench_test.go
#	core/state/statedb.go
#	core/state_prefetcher.go
#	core/state_processor.go
#	core/state_transition.go
#	core/tx_pool.go
#	core/types/block.go
#	core/types/derive_sha.go
#	core/types/gen_tx_json.go
#	core/types/receipt.go
#	core/types/receipt_test.go
#	core/types/transaction.go
#	core/types/transaction_signing.go
#	core/types/transaction_test.go
#	ethclient/ethclient.go
#	ethclient/signer.go
#	graphql/graphql.go
#	internal/ethapi/api.go
#	internal/guide/guide_test.go
#	les/benchmark.go
#	les/odr_test.go
#	light/odr_test.go
#	light/txpool.go
#	miner/worker.go
#	miner/worker_test.go
#	signer/core/api.go
#	tests/state_test_util.go
#	trie/stacktrie_test.go
#	turbo/stages/blockchain_test.go
2021-03-12 15:40:24 +01:00
Marius van der Wijden
96a424ee84 internal/ethapi: reject non-replay-protected txs over RPC (#22339)
This PR prevents users from submitting transactions without EIP-155 enabled. This behaviour can be overridden by specifying the flag --rpc.allow-unprotected-txs=true.
# Conflicts:
#	cmd/geth/main.go
#	eth/backend.go
#	les/api_backend.go
#	les/client.go
#	node/config.go
2021-03-10 17:00:47 +01:00
rene
254295d9d1 cmd/devp2p/internal/ethtest: use shared message types (#22315)
This updates the eth protocol test suite to use the message type
definitions of the 'production' protocol implementation in eth/protocols/eth.
# Conflicts:
#	cmd/devp2p/internal/ethtest/chain_test.go
#	cmd/devp2p/internal/ethtest/suite.go
#	cmd/devp2p/internal/ethtest/types.go
2021-03-10 11:08:48 +01:00
Guillaume Ballet
fbb3e0e206 cmd/utils: add workaround for FreeBSD statfs quirk (#22310)
Make geth build on FreeBSD, fixes #22309.
2021-03-10 11:02:02 +01:00
rene
33ae8fd923 cmd/devp2p: fix documentation for eth-test (#22298) 2021-03-10 11:00:41 +01:00
Péter Szilágyi
e7bffb64eb cmd/utils, eth/ethconfig: unindex txs older than ~1 year
# Conflicts:
#	eth/ethconfig/config.go
2021-03-10 11:00:31 +01:00
gary rong
a9e6250e72 all: bloom-filter based pruning mechanism (#21724)
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	core/blockchain.go
#	core/blockchain_snapshot_test.go
#	core/genesis.go
#	core/rawdb/database.go
#	core/rawdb/schema.go
#	core/state/snapshot/conversion.go
#	core/state/snapshot/snapshot.go
#	core/state/snapshot/snapshot_test.go
#	eth/backend.go
#	go.mod
#	go.sum
#	tests/block_test_util.go
#	tests/state_test_util.go
#	trie/stacktrie.go
2021-03-10 10:58:22 +01:00
gary rong
7d799473e5 eth: move eth.Config to a common package (#22205)
This moves the eth config definition into a separate package, eth/ethconfig.
Packages eth and les can now import this common package instead of
importing eth from les, reducing dependencies.

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/faucet/faucet.go
#	cmd/geth/config.go
#	cmd/utils/cmd.go
#	cmd/utils/flags.go
#	cmd/utils/flags_legacy.go
#	console/console_test.go
#	eth/backend.go
#	eth/bloombits.go
#	eth/ethconfig/config.go
#	ethclient/ethclient_test.go
#	graphql/graphql_test.go
#	les/api_test.go
#	les/client.go
#	les/commons.go
#	les/costtracker.go
#	les/server.go
#	les/test_helper.go
#	mobile/geth.go
2021-03-10 10:25:55 +01:00
alex.sharov
e79d3b414a table stat metrics 2021-03-10 11:11:03 +07:00
Alex Sharov
060f3b9a60
Grafana: add Btree metrics (#1547) 2021-03-10 10:58:57 +07:00
Alex Prut
1fb76cca04 all: remove unneeded parentheses (#21921)
* remove uneeded convertion type

* remove redundant type in composite literal

* omit explicit type where implicit

* remove unused redundant parenthesis

* remove redundant import alias duktape
# Conflicts:
#	core/state/snapshot/conversion.go
#	eth/tracers/tracer.go
#	metrics/cpu_syscall.go
#	trie/trie_test.go
2021-03-09 13:59:30 +01:00
rene
a0af38304e cmd/geth, node: allow configuring JSON-RPC on custom path prefix (#22184)
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.

Fixes #21826

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	go.sum
#	graphql/graphql_test.go
#	node/node_test.go
#	node/rpcstack_test.go
2021-03-09 13:58:42 +01:00
Martin Holst Swende
227c57cd2e cmd/clef: don't check file permissions on windows (#22251)
Fixes #20123
2021-03-09 13:51:33 +01:00
Sina Mahmoodi
c376039049 cmd,core,eth,params,tests: define yolov3 + enable EIP-2565 (#22213)
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3.
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	cmd/geth/chaincmd.go
#	cmd/geth/consolecmd.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	core/state_processor.go
#	core/vm/evm.go
#	core/vm/interpreter.go
#	tests/block_test.go
2021-03-09 13:47:37 +01:00
Felföldi Zsolt
0d8a69b8c6 les: switch to new discv5 (#21940)
This PR enables running the new discv5 protocol in both LES client
and server mode. In client mode it mixes discv5 and dnsdisc iterators
(if both are enabled) and filters incoming ENRs for "les" tag and fork ID.
The old p2p/discv5 package and all references to it are removed.

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/bootnode/main.go
#	cmd/faucet/faucet.go
#	cmd/utils/flags.go
#	les/client.go
#	les/commons.go
#	les/enr_entry.go
#	les/server.go
#	les/serverpool.go
#	les/serverpool_test.go
#	mobile/discover.go
#	mobile/params.go
#	p2p/discv5/database.go
#	p2p/discv5/metrics.go
#	p2p/discv5/net.go
#	p2p/discv5/net_test.go
#	p2p/discv5/node.go
#	p2p/discv5/node_test.go
#	p2p/discv5/sim_test.go
#	p2p/discv5/table.go
#	p2p/discv5/table_test.go
#	p2p/discv5/ticket.go
#	p2p/discv5/topic.go
#	p2p/discv5/topic_test.go
#	p2p/discv5/udp.go
#	p2p/server.go
2021-03-09 12:54:06 +01:00
Alex Mazalov
ed42a61cc8 cmd/geth: graceful shutdown if disk is full (#22103)
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/cmd.go
2021-03-09 10:50:14 +01:00
Marius van der Wijden
e78f1a9f6e cmd/geth: dump config for metrics (#22083)
* cmd/geth: dump config

* cmd/geth: dump config

* cmd/geth: properly read config again

* cmd/geth: override metrics if flags are set

* cmd/geth: write metrics regardless if enabled

* cmd/geth: renamed to metricsfromcliargs

* metrics: add default configuration
# Conflicts:
#	cmd/geth/config.go
2021-03-09 10:49:12 +01:00
Guillaume Ballet
e235d3c43a cmd/utils: don't enumerate USB unless --usb is set (#22130)
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
# Conflicts:
#	cmd/geth/accountcmd_test.go
#	cmd/geth/consolecmd_test.go
#	cmd/geth/dao_test.go
#	cmd/geth/genesis_test.go
#	cmd/geth/les_test.go
2021-03-09 10:36:28 +01:00
meowsbits
b82525b79b cmd/utils: avoid making console preloads absolute (#22109)
Resolves https://github.com/etclabscore/core-geth/issues/273

jsre.JSRE already handles establishing preload
file paths relative to the 'assets' path (aka docroot),
where it joins the assets dir and the file path if relative,
or uses the file path only if absolute.

The duplication of this logic by MakeConsolePreloads
caused preloaded files to have paths which contained
duplicate references to the assets dir path.

Date: 2020-12-30 08:25:01-06:00
Signed-off-by: meows <b5c6@protonmail.com>
2021-03-09 10:33:36 +01:00
Péter Szilágyi
a50f261e87 cmd/utils, eth/downloader: minor snap nitpicks 2021-03-09 10:22:04 +01:00
Marius van der Wijden
78213e6c0e cmd/geth: added --mainnet flag (#21932)
* cmd/geth: added --mainnet flag

* cmd/utils: set default genesis if --mainnet is specified

* cmd/utils: addressed comments
# Conflicts:
#	cmd/geth/chaincmd.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
2021-03-09 10:17:55 +01:00
Guillaume Ballet
f50f13d70c cmd/geth: usb is off by default (#21984)
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
2021-03-09 10:16:32 +01:00
Péter Szilágyi
7b9565fe10 cmd: support v1.1 Twitter API in faucet, fix puppeth
# Conflicts:
#	cmd/faucet/faucet.go
2021-03-09 10:14:38 +01:00
Alex Sharov
175828aa7f
add periodical db statistic metrics (#1546) 2021-03-09 13:41:09 +07:00
Alex Sharov
4fe019a099
add periodical db statistic metrics (#1545) 2021-03-09 13:34:13 +07:00
Péter Szilágyi
693459073e core, eth: split eth package, implement snap protocol (#21482)
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	cmd/geth/misccmd.go
#	cmd/utils/flags.go
#	core/blockchain.go
#	core/blockchain_snapshot_test.go
#	core/forkid/forkid.go
#	core/rawdb/accessors_snapshot.go
#	core/rawdb/schema.go
#	core/state/snapshot/generate.go
#	core/state/statedb.go
#	eth/api_test.go
#	eth/backend.go
#	eth/downloader/downloader.go
#	eth/downloader/downloader_test.go
#	eth/downloader/modes.go
#	eth/downloader/queue.go
#	eth/downloader/statesync.go
#	eth/gen_config.go
#	eth/handler.go
#	eth/handler_test.go
#	eth/helper_test.go
#	eth/peer.go
#	eth/protocol.go
#	eth/protocol_test.go
#	eth/sync.go
#	eth/sync_test.go
#	ethstats/ethstats.go
#	les/client.go
#	les/enr_entry.go
#	les/handler_test.go
#	les/peer.go
#	les/server_handler.go
#	tests/block_test_util.go
#	trie/proof.go
#	trie/proof_test.go
#	trie/trie.go
#	turbo/trie/notary.go
#	turbo/trie/sync_bloom.go
2021-03-08 17:07:38 +01:00
Shiming
2dfac7aedb cmd/abigen: clarify abigen alias flag usage (#21875)
* doc: clarify abigen alias flag usage

update the `abigen --alias` flag help info, give an example to make it more clear

related issue: https://github.com/ethereum/go-ethereum/issues/21846

* Update cmd/abigen/main.go

Co-authored-by: ligi <ligi@ligi.de>

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: ligi <ligi@ligi.de>
2021-03-08 14:39:15 +01:00
alex.sharov
9d39708253 use non-app metric to parse in grafana variables 2021-03-08 20:36:56 +07:00
Mudit Gupta
0c2992922f cmd/faucet: use Twitter API instead of scraping webpage (#21850)
This PR adds support for using Twitter API to query the tweet and author details. There are two reasons behind this change:

- Twitter will be deprecating the legacy website on 15th December. The current method is expected to stop working then.
- More importantly, the current system uses Twitter handle for spam protection but the Twitter handle can be changed via automated calls. This allows bots to use the same tweet to withdraw funds infinite times as long as they keep changing their handle between every request. The Rinkeby as well as the Goerli faucet are being actively drained via this method. This PR changes the spam protection to be based on Twitter IDs instead of usernames. A user can not change their Twitter ID.

# Conflicts:
#	cmd/faucet/faucet.go
2021-03-08 14:31:14 +01:00
Martin Holst Swende
f9dec8c450 cmd/devp2p: add node filter for snap + fix arg error (#21950) 2021-03-08 12:48:01 +01:00
Felföldi Zsolt
ea7b33c639 les: fix nodiscover option (#21906)
# Conflicts:
#	eth/backend.go
#	eth/discovery.go
#	les/client.go
#	les/enr_entry.go
2021-03-08 12:47:13 +01:00
Marius van der Wijden
5ebb5b13e1 cmd/devp2p/internal/ethtest: add transaction tests (#21857)
# Conflicts:
#	cmd/devp2p/internal/ethtest/suite.go
2021-03-08 12:45:43 +01:00
Alex Prut
f032b6d18e all: remove redundant conversions and import names (#21903)
# Conflicts:
#	accounts/accounts.go
#	accounts/keystore/wallet.go
#	accounts/scwallet/wallet.go
#	accounts/usbwallet/wallet.go
#	cmd/clef/main.go
#	cmd/devp2p/dnscmd.go
#	cmd/evm/compiler.go
#	cmd/evm/disasm.go
#	cmd/evm/runner.go
#	cmd/evm/staterunner.go
#	cmd/geth/config.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	consensus/ethash/ethash.go
#	core/state/statedb.go
#	core/tx_pool_test.go
#	core/vm/evm.go
#	core/vm/gas_table.go
#	core/vm/instructions.go
#	core/vm/operations_acl.go
#	eth/downloader/api.go
#	eth/filters/api.go
#	eth/filters/filter_system.go
#	eth/filters/filter_system_test.go
#	eth/tracers/tracer.go
#	internal/debug/flags.go
#	internal/ethapi/api.go
#	internal/flags/helpers.go
#	miner/unconfirmed_test.go
#	mobile/ethereum.go
#	trie/iterator.go
#	turbo/trie/database.go
2021-03-08 12:43:02 +01:00
Alex Prut
86dd1f91dd all: simplify nested complexity and if blocks ending with a return statement (#21854)
Changes:

    Simplify nested complexity
    If an if blocks ends with a return statement then remove the else nesting.

Most of the changes has also been reported in golint https://goreportcard.com/report/github.com/ethereum/go-ethereum#golint
# Conflicts:
#	cmd/utils/flags.go
#	console/bridge.go
#	crypto/bls12381/g2.go
#	les/benchmark.go
#	les/lespay/server/balance.go
#	les/lespay/server/balance_tracker.go
#	les/lespay/server/prioritypool.go
#	les/odr_requests.go
#	les/serverpool.go
#	les/serverpool_test.go
#	p2p/nodestate/nodestate_test.go
#	trie/committer.go
2021-03-08 12:34:16 +01:00
Marius van der Wijden
59351705b7 cmd/devp2p/internal/ethtest: add 'large announcement' tests (#21792)
* cmd/devp2p/internal/ethtest: added large announcement tests

* cmd/devp2p/internal/ethtest: added large announcement tests

* cmd/devp2p/internal/ethtest: refactored stuff a bit

* cmd/devp2p/internal/ethtest: added TestMaliciousStatus/Handshake

* cmd/devp2p/internal/ethtest: fixed rebasing issue

* happy linter, happy life

* cmd/devp2p/internal/ethtest: used readAndServe

* stuff

* cmd/devp2p/internal/ethtest: fixed test cases
# Conflicts:
#	cmd/devp2p/internal/ethtest/suite.go
#	cmd/devp2p/internal/ethtest/types.go
2021-03-08 12:32:14 +01:00
gary rong
b2351da9d6 all: disable recording preimage of trie keys (#21402)
* cmd, core, eth, light, trie: disable recording preimage by default

* core, eth: fix unit tests

* core: fix import

* all: change to nopreimage

* cmd, core, eth, trie: use cache.preimages flag

* cmd: enable preimages for archive node

* cmd/utils, trie: simplify preimage tracking a bit

* core: fix linter

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	core/blockchain.go
#	core/genesis.go
#	core/state/database.go
#	core/state/state_test.go
#	eth/api_test.go
#	eth/api_tracer.go
#	eth/backend.go
#	light/postprocess.go
#	trie/secure_trie.go
#	turbo/trie/database.go
2021-03-08 12:26:27 +01:00
Marius van der Wijden
bcc51ea117 core, all: split vm.Context into BlockContext and TxContext (#21672)
* all: core: split vm.Config into BlockConfig and TxConfig

* core: core/vm: reset EVM between tx in block instead of creating new

* core/vm: added docs
# Conflicts:
#	accounts/abi/bind/backends/simulated.go
#	cmd/evm/internal/t8ntool/execution.go
#	cmd/geth/retesteth.go
#	core/evm.go
#	core/state_prefetcher.go
#	core/state_processor.go
#	core/state_transition.go
#	core/vm/evm.go
#	core/vm/gas_table_test.go
#	core/vm/instructions.go
#	core/vm/instructions_test.go
#	eth/api_tracer.go
#	eth/tracers/tracer_test.go
#	eth/tracers/tracers_test.go
#	les/api_backend.go
#	les/odr_test.go
#	light/odr_test.go
#	tests/vm_test_util.go
2021-03-08 12:17:35 +01:00
ledgerwatch
b743b88218
Revert "Extract validating interface (#1120)" (#1543)
* Revert "Extract validating interface (#1120)"

This reverts commit e4e36c152e.

* Fix go mod tidy

* Fix lint

* Fix lint

* fix lint

* Test fix

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-05 20:34:23 +00:00
Greg Colvin
8ec8cfcd01 Merge branch 'master' of https://github.com/ledgerwatch/turbo-geth into extractCode 2021-03-04 15:03:18 +00:00
Greg Colvin
c0d0c1885b gofmt 2021-03-04 13:46:57 +00:00
Greg Colvin
7899048eac add action to extract all contract code as hex string 2021-03-04 13:01:46 +00:00
Alex Sharov
eb7d2ad16a
Add validation of senders amount (#1539) 2021-03-04 16:34:49 +07:00
Alex Sharov
827e47e544
grafana: unstack charts (#1540) 2021-03-04 12:27:16 +07:00
alex.sharov
ceb332afdd More documentation for integration tool 2021-03-04 09:42:05 +07:00
Alex Sharov
17f99a5d3f
Grafana: to use iops units (#1537) 2021-03-03 16:18:01 +07:00
Alex Sharov
141b039b48
fix history check (#1534) 2021-03-03 10:16:37 +07:00
ledgerwatch
c00f85ab6c
Simplification of eth_estimageGas (#1531)
* Simplification of eth_estimageGas

* Set GasCap by default

* Add a test

* Fix crash

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-02 22:47:44 +00:00
alex.sharov
b7996c66d9 add more db conversion tools 2021-03-02 09:57:15 +07:00
alex.sharov
66bc517834 add more db conversion tools 2021-03-02 09:56:43 +07:00
alex.sharov
61f8a1ef0c revive 2021-03-02 09:52:05 +07:00
Alex Sharov
904a992d3c
remove old grafana boards (#1527) 2021-03-01 12:34:33 +07:00
Alex Sharov
827461039e
restore cache exec (#1525) 2021-03-01 12:03:49 +07:00
Alex Sharov
8538ec85b3
don't open db in exclusive mode if no migrations (#1524) 2021-03-01 11:02:22 +07:00
Alex Sharov
6d0d84cda6
log git branch and git commit (#1523) 2021-03-01 11:02:06 +07:00
ledgerwatch
da74bd8f38
Fixes to unwind logic for the new staged sync (#1521)
* Make DB visible

* Further fixes to unwind logic

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-28 08:47:31 +00:00
Alex Sharov
f556af6276
Prometheus metrics: more virtual memory visibility, more cross-platform metrics (#1520) 2021-02-28 11:11:28 +07:00
Evgeny Danilenko
e4e36c152e
Extract validating interface (#1120)
* interface

* generalize interface

* linters

* fix deadlock

* fix linters

* close goroutine

* fix

* debug

* id+ttl

* refactor downloader tests

* tests

* lru

* handle genesis, extract fake consensuses

* fix fake consensus

* test uncles, verify

* after a new master

* fmt

* fix close

* debug

* debug

* fix chain length

* remove test field

* use single account

* fix data race on closing channel

* remove postponed blocks queue

* miner test

* VerifyHeaderRequests

* fmt

* fmt

* fix data race

* handle validating errors

* simplify matchParents

* remove copy-paste

* move sort to constructor

* clean up

* debug for 10 parents

* debug

* debug

* batch responses

* batch requests

* works for many ancestors

* remove debug

* always Close an engine

* linters

* ancestors deduplication

* fix test

* reduce interface

* api

* clique

* green clique sync

* stable

* perpermance per second

* full sync

* linters

* gitignore

* deps

* fix panic after master merge

* init consensus

* clique tests

* fix tests

* fix tests

* clean up

* reuse snap

* store vefified snapshots

* optimize snapshots

* safe close

* cleanup loop

* after downloader

* downloader and consensus tests

* update tests

* hack.go

* clique flags

* fix cliuqe config

* review

* gitignore

* remove additional bucket

* blk/sec instead of blk/microsecond

* rename

* deps

* optimize

* debug

* test

* tests without extracted validation process

* same base performance as on master

* benchmark

* simplify ethash verification

* ethash

* ethash

* linters

* ethash

* master stats

* cleanup

* gomod

* linters

* tests

* better locks

* Fix

* Remove logging for verifyHeaders

* Verification speed in the logs

* Fix compile error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-25 19:40:45 +00:00
ledgerwatch
d3bac21f2e
Fix combined mode for the header download (#1513)
* Fix combined mode for new headers downloader

* Fix for streams

* Return error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-24 14:51:50 +00:00
ledgerwatch
be7b93d809
Fixes and simplifications for headers download (#1511)
* Simplify header insert, fix recovery from files

* Fix to the header processed log

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-24 07:57:02 +00:00
Alex Sharov
d4ef1f8516
blocks exec unwind - to support graceful shutdown (#1494)
* quit exec unwind

* quit exec unwind

* lint
2021-02-23 17:14:32 +00:00
Alex Sharov
6894d9b356
Remove cursor features (#1508) 2021-02-23 09:02:54 +07:00
ledgerwatch
c30ad646db
Better ergonomics for new header/bodies downloader, more test friendly, README (#1507)
* Ergonomic changes

* Add little doc and remove coreAddr

* Fix gRPC streaming

* Fix lint

* Remove peer penalty logging

* Fix lint

* Fix lint

* Fix lint

* Fix peerID in error message

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-22 21:59:23 +00:00
ledgerwatch
dc6b4c0e43
Fixes for trace_callMany (#1506)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-21 20:18:59 +00:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record (#1500)
* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE
2021-02-21 18:41:59 +00:00
ledgerwatch
e8501bbf43
eth_getTransactionReceipt to return nil for transactions not in the d… (#1505)
* eth_getTransactionReceipt to return nil for transactions not in the database

* Fix compile error
2021-02-21 09:38:00 +01:00
canepat
3388c1f1af
Make vegetaWrite function usable from all benchs (#1502)
Add vegetaWrite support to bench8 (disabled)
Add trace log with comparative timings in bench8
2021-02-19 13:26:28 +07:00
Alex Sharov
7aedc36baa
Tool for benchmarking execution long ranges of blocks (integration loop_exec) (#1501) 2021-02-18 20:33:59 +07:00
ledgerwatch
b9acd62280
Embed header/bodies download into a mini-staged sync (#1498)
* Embed header/bodies download into a mini-staged sync

* Move code around to prevent import cycles

* Include new files

* Fix lint

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-16 15:47:15 +00:00
ledgerwatch
6e56097056
Bring in refactored p2p sentry interface and adjust downloader for it (#1497)
* Initial commit

* Add sentry gRPC interface

* p2psentry directory

* Update README.md

* Update README.md

* Update README.md

* Add go package

* Correct syntax

* add external downloader interface (#2)

* Add txpool (#3)

* Add private API (#4)

* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Use new p2psentry interface

* Remove

* Fix lint

* Fix lint

* Separate upload messages into its own stream (#6)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Add header upload

* Remove

* Add upload of block headers

* Add block bodies upload

* Fix lint

* Bugfix for blockBodies

* Fix for body upload logging

* Print peerId for GetBlockHeaders

* Invoke block hashes stage to enable block body upload

* Fix test

* Fix stuck bodies download

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-15 18:54:45 +00:00
ledgerwatch
7c81e9198d
Fixes to trace_call, trace_callMany, dual RPC daemon mode (#1491)
* Fixes to trace_call, trace_callMany, dual RPC daemon mode

* Fix compile error

* Fix compile error

* Compile error fix

* Compile error fix

* Fix typo

* Fix discrepancy when trace is empty

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-12 16:47:32 +00:00
Evgeny Danilenko
0fdd296d34
Performance tests for ValidateHeaders on mainnet and rinkeby networks (#1489)
* test

* rename
2021-02-11 16:36:22 +03:00
alex.sharov
a49b882d1c add to_mdbx finish notification 2021-02-11 16:57:04 +07:00
ledgerwatch
bb2cc5a10e
debug_accountAt (#1487)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-10 20:43:13 +00:00
ledgerwatch
2daa71e6cb
Support for trace_callMany (#1486)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-10 17:04:47 +00:00
Artem Vorotnikov
5c34713e71
Remove nested transactions (#1483) 2021-02-10 17:04:22 +00:00
Alex Sharov
b77dfec055
remove debug-web-ui and restapi (#1480) 2021-02-09 19:42:14 +07:00
Alex Sharov
bca54f0e52
delete old code from hack.go (#1481) 2021-02-09 17:25:36 +07:00
ledgerwatch
5834189019
Bodies download and simplify header download (#1471)
* Only insert hard-coded tips if both DB and files recovery failed

* Prevent deadlock

* Report some efficiency numbers

* Count properly unrequested bodies

* Initialise allRequests

* Increase timeout

* Fixes to scheduling

* Small fix

* Simplified scheduling

* Remove separate bodyLoop goroutine

* Update from DB at the beginnig of block bodies forward

* Timeout for repeating the request cycle

* Fix timeout

* Fix

* Increase timeout

* Increase timeout

* Try to make flow if possible

* Fix flow

* Lower timeout

* timeout for each blockNum

* Adjustable timeout

* Better log timing

* Track peers

* copy peerID

* fix scheduling

* Too much logging

* Print delivery speed

* Print committed blocks

* Fix race

* Sentry to only reset back-off timer when response to a request is received

* Print bytes/sec and wasted traffic

* Fix bandwidth accounting

* Less logging

* not to wake up on deliveries

* Spam every second

* Print peer map

* Fix npe, print requests

* Timestamps

* Improved logging

* Penalty for peers

* Log penalties only when disconnecting

* Try with smaller window

* window parameter

* Dealing with partially delivered requests

* Init bodyReq

* Fix array index

* More fix for NPE

* More NPE checks

* Print out body progress and header progress

* Fix ending condition

* Bring back waking up and penalties

* Fix duplicate tip

* Duplicate segment

* Fix lint

* Fix lint

* fix lint

* Fix lint

* Hard coded headers in the source files

* Fix lint

* Replace hix-sized header serialisation with rlp (to support clique)

* Remove anchor records from the files

* Fixeds for DB recovery

* Fix compilation

* Fix compile errors

* Fix formatting

* Fix lint

* Fix comments

* Remove headerLoop

* Properly terminate body download

* Support for StatusData p2p proto

* Fix forkid test

* Fix test

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-08 09:25:10 +00:00
Alex Sharov
bf04a6e73d
make_history_check_optional (#1467) 2021-02-05 16:46:43 +07:00
Artem Vorotnikov
91f7798c4a
Remove hashed changesets (#1464)
* Remove hashed changesets

* try this

* oops

* simplify

* better diagnostics

* don't hash account key and use bitmap 64 to check history index

* update docs of bucket

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2021-02-04 12:59:04 +03:00
Alex Sharov
f0bb3a6dfa
Mdbx new beta version with fixed blockers (#1461) 2021-02-01 20:57:41 +07:00
alex.sharov
5a9c86e552 devel up 2021-01-28 09:37:24 +07:00
Alex Sharov
e943a73ceb
improve integration (#1456) 2021-01-26 12:19:53 +07:00
Alex Sharov
58c4e03861
Revert "Remove hashed changesets (#1444)" (#1446)
This reverts commit aadb6e47cc.
2021-01-20 14:16:20 +07:00
Artem Vorotnikov
aadb6e47cc
Remove hashed changesets (#1444) 2021-01-19 17:42:26 +07:00
ledgerwatch
7c8bae8591
Implementation of debug_traceCall, add stateDiff to trace_call (#1442)
* Implementation of debug_traceCall

* Fix compile error

* Implementing stateDiff in trace_call - WIP

* fixes to stateDiff, better parsing in rpctest

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-18 11:13:19 +00:00
ledgerwatch
820216b77f
Block body download - more recent updates (#1441)
* Combined

* Fix lint

* Fix lint

* Initial bodydownload package

* Updates

* fix lint and process incoming body

* Fix compile errors

* Use deliveries

* Ready for initial testing

* Better handling of missing temp files dir

* Intermediate testing

* First working code

* Fix races, more efficient request scheduling

* Remove excessive logging

* Fix lint

* Interruptibility, fix lint

* Remove more logging

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-15 10:58:04 +00:00
Alex Sharov
7fd1f19435
txdb lazy cursors creation, grpc version up (#1440)
* txdb lazy, grpc version up

* tidy

* fix lint
2021-01-15 09:38:09 +00:00
Alex Sharov
e6f1e43e6f
lmdb: move fsync inside commit (#1433) 2021-01-04 08:17:18 +00:00
ledgerwatch
baafc90bc6
Fixup after merge (#1432)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-02 20:22:35 +00:00
b00ris
8db5790838
State snapshot sync (#1417)
* move experiments to new branch&reorganise kv_snapshot

* walk&modify tests

* added delete from snapshot tests

* fmt

* state snapshot debug

* snapshot validation passed. copy state snapshot

* debug

* snapshot cursor.Prev test

* Prev works correct. Added Current check

* add err check

* added walk forward and backward test

* before refactoring

* refactoring

* execution with snapshot debug

* fix

* remove useless test

* before dupcursor implimentation

* tests with prev and delete works

* execution based on state snapshot passed

* remove useless tests

* blocks to 1140000 passed

* clean verifier

* cleanup state generation

* clean verify && seeder

* remove debug code

* tests passed

* fix lint

* save state

* test passed

* fix lint

* add state hash

* fix lint
2021-01-02 19:28:37 +00:00
Alex Sharov
75cb938980
rpcdaemon performance improve - less reading blocks, less allocs on server (#1426)
* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* don't use global variables

* df

* avoid use non-transactional db instance to reduce amount of cursors
2021-01-02 19:28:22 +00:00
Alex Sharov
fa1d92bd15
Rm zstd (#1427) 2020-12-28 22:14:16 +07:00
Alex Sharov
d7d9ce0a8e
fix go vet (#1425) 2020-12-26 10:18:16 +07:00
Alex Sharov
311fb591c4
fix check index (#1424) 2020-12-26 09:01:00 +07:00
Artem Vorotnikov
8c0ecaeea9
Remove StageData (#1419) 2020-12-21 11:01:37 +00:00
Chase Wright
9db5debc6c
Fix order of print to match stage numbers (#1416)
IntermediateHashes and HashState are flipped
2020-12-17 21:11:54 +07:00
ledgerwatch
3287b44c2e
Bad unwind of recreation of CREATE2 contract - unit test and potentia… (#1412)
* Bad unwind of recreation of CREATE2 contract - unit test and potential fix

* Fix unit test

* Fix lint

* Fix lint

* Fixup hack

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-16 17:16:37 +00:00
Alex Sharov
7262fd503b
Mdbx devel 5 (#1413)
* tx

* fix tests

* fix tests

* fix tests

* sequence
2020-12-16 14:35:14 +00:00
ledgerwatch
71ef201c17
Extract bodies utility (#1409)
* Extract bodies

* Print headers too
2020-12-14 12:09:59 +00:00
Thomas Jay Rush
76e244f24f
Adding specific Postman testing for tracing (#1402)
* Adding specific Postman testing for tracing

* Updating tests
2020-12-14 11:28:03 +00:00
ledgerwatch
0953fd42cb
WIP trace_call implementation (#1399)
* trace_call initial

* Fix tests

* More tracing

* Add more fields to the action

* Completed first example query

* Add initial bench11 to compare trace_call with OpenEthereum

* Exclude precompile calls from call traces

* Add self-destruct, call types, more comparison in rpctest

* Support for execution errors

* Stack underflow error and delegatecall value

* Fix lint

* Fix suicide traceAddress, Bad instruction error

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-14 11:27:52 +00:00
ledgerwatch
2cc36dac87
further cleanup, unit tests for RPC daemon and tracing improvements (#1392)
* Cleanup and rpcdaemon unit tests

* Fix

* Fix

* Fix lint

* Test for debug_traceTransaction

* Add NoRefunds option

* Compile fix, test for no refunds

* Fix compile

* Add poly contract, fix compile errors

* No refunds now work

* Fix NPE in rpcdaemon

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-09 18:24:08 +00:00
ledgerwatch
5ea590c18e
State cache switching writes to reads during commit (#1368)
* State cache init

* More code

* Fix lint

* More tests

* More tests

* More tests

* Fix test

* Transformations

* remove writeQueue, before fixing the tests

* Fix tests

* Add more tests, incarnation to the code items

* Fix lint

* Fix lint

* Remove shards prototype, add incarnation to the state reader code

* Clean up and replace cache in call_traces stage

* fix flaky test

* Save changes

* Readers to use addrHash, writes - addresses

* Fix lint

* Fix lint

* More accurate tracking of size

* Optimise for smaller write batches

* Attempt to integrate state cache into Execution stage

* cacheSize to default flags

* Print correct cache sizes and batch sizes

* cacheSize in the integration

* Fix tests

* Fix lint

* Remove print

* Fix exec stage

* Fix test

* Refresh sequence on write

* No double increment

* heap.Remove

* Try to fix alignment

* Refactoring, adding hashItems

* More changes

* Fix compile errors

* Fix lint

* Wrapping cached reader

* Wrap writer into cached writer

* Turn state cache off by default

* Fix plain state writer

* Fix for code/storage mixup

* Fix tests

* Fix clique test

* Better fix for the tests

* Add test and fix some more

* Fix compile error|

* More functions

* Fixes

* Fix for the tests

* sepatate DeletedFlag and AbsentFlag

* Minor fixes

* Test refactoring

* More changes

* Fix some tests

* More test fixes

* More test fixes

* Fix lint

* Move blockchain_test to be able to use stagedsync

* More fixes

* Fixes and cleanup

* Fix tests in turbo/stages

* Fix lint

* Fix lint

* Intemediate

* Fix tests

* Intemediate

* More fixes

* Compilation fixes

* More fixes

* Fix compile errors

* More test fixes

* More fixes

* More test fixes

* Fix compile error

* Fixes

* Fix

* Fix

* More fixes

* Fixes

* More fixes and cleanup

* Further fix

* Check gas used and bloom with header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-08 09:44:29 +00:00
Alex Sharov
98b36126b2
History bitmap 64 (#1374)
* squash

* squash

* squash

* squash

* squash

* squash

* subscription_doesnt_preserve_shutdown
2020-12-04 21:16:51 +00:00
Alex Sharov
27a24746b7
remove GetLogsByHash (#1390) 2020-12-04 10:27:24 +00:00
Thomas Jay Rush
362e9a1773
Update docs testing (#1385)
* Remove unused files

* Updating testing definition

* Updating auto-generated docs

* Update go code to make agree with silkrpc testing
2020-12-04 10:24:49 +00:00
Igor Mandrigin
63ec2ec9f1 post-rebase fixes 2020-12-03 18:59:17 +01:00
rene
040dcebf57 cmd/devp2p/internal/ethtest: add correct chain files and improve test output (#21782)
This PR replaces the old test genesis.json and chain.rlp files in the testdata
directory for the eth protocol test suite, and also adds documentation for
running the eth test suite locally.

It also improves the test output text and adds more timeouts.

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/devp2p/internal/ethtest/suite.go
2020-12-03 18:59:17 +01:00
Felix Lange
6dee5bbdb5 cmd/devp2p, internal/utesting: implement TAP output (#21760)
TAP is a text format for test results. Parsers for it are available in many languages,
making it easy to consume. I want TAP output from our protocol tests because the
Hive wrapper around them needs to know about the test names and their individual
results and logs. It would also be possible to just write this info as JSON, but I don't
want to invent a new format.

This also improves the normal console output for tests (when running without --tap).
It now prints -- RUN lines before any output from the test, and indents the log output
by one space.
# Conflicts:
#	cmd/devp2p/discv4cmd.go
#	cmd/devp2p/discv5cmd.go
#	cmd/devp2p/rlpxcmd.go
2020-12-03 18:59:17 +01:00
Martin Holst Swende
39c494f4f3 utils, params: add yolov2 bootnode 2020-12-03 18:59:17 +01:00
Felix Lange
eb955580c0 cmd/devp2p/internal/ethtest: update test chain (#21742)
The old one was wrong in two ways: the first block in chain.rlp was the
genesis block, and the genesis difficulty was below minimum difficulty.

This also contains some other fixes to the test.
2020-12-03 18:59:17 +01:00
Igor Mandrigin
5d98f8d40d post-rebase. fix imports 2020-12-03 17:37:38 +01:00
Martin Holst Swende
ca3d54a32c all: implement EIP-2929 (gas cost increases for state access opcodes) + yolo-v2 (#21509)
* core/vm, core/state: implement EIP-2929 + YOLOv2

* core/state, core/vm: fix some review concerns

* core/state, core/vm: address review concerns

* core/vm: address review concerns

* core/vm: better documentation

* core/vm: unify sload cost as fully dynamic

* core/vm: fix typo

* core/vm/runtime: fix compilation flaw

* core/vm/runtime: fix renaming-err leftovers

* core/vm: renaming

* params/config: use correct yolov2 chainid for config

* core, params: use a proper new genesis for yolov2

* core/state/tests: golinter nitpicks
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	cmd/geth/chaincmd.go
#	cmd/geth/consolecmd.go
#	cmd/geth/main.go
#	cmd/geth/usage.go
#	cmd/utils/flags.go
#	core/state/statedb.go
#	core/state/statedb_test.go
#	core/vm/evm.go
#	core/vm/interpreter.go
#	core/vm/logger.go
#	core/vm/runtime/runtime.go
#	eth/api_tracer.go
2020-12-03 17:37:38 +01:00
ledgerwatch
9a233da93d
Add test for getChainConfig (#1375)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-28 15:08:17 +00:00
Andrew Ashikhmin
8a877c227a
Integrate Silkworm execution (#1344)
* Expose C Handle of lmdbTx

* LoadExecutionFunctionPointer

* silkworm ExecuteBlocks

* linter

* CLI flag for SilkwormExecutionFunc

* linter

* Call SilkwormExecutionFunc in SpawnExecuteBlocksStage

* Commit transaction after Silkworm execution

* Fix batch initialization

* Make batch_size uint64_t instead of size_t

* max_block in silkworm_execute_blocks

* More accurate logProgress

* Use CHandle exposed by lmdb-go

* Enable Silkworm on Linux only

* Update lmdb-go

* Error when attempting to use Silkworm not on Linux

* Move SilkwormFlag from cmd/utils/flags.go to turbo/cli/flags.go

* Integrate Silkworm into cmd/integration

* Check against ReaderBuilder/WriterBuilder/ChangeSetHook in Silkworm execution

* Refactor SpawnExecuteBlocksStage

* linters

* Small clean-ups

* Move ChangeSetHook inside executeBlockWithGo

* No need to tamper with batch size in executeBlocksWithSilkworm

* fix

* Fix

* Fixup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-28 15:08:02 +00:00
Alex Sharov
4565bd43c1
Mdbx v0.9.2 (#1373)
* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* switch to master

* mdbx 0.9.2

* mdbx 0.9.2

* %<<$!%%%%%%<o'{}k.###

* %<<$!%%%%%%<o'{}k.###

* merge

* merge

* merge

* merge

* ad

* ad

* ad

* ad
2020-11-28 14:26:28 +00:00
Alex Sharov
67de2a3a46
To mdbx dupfixed (#1371) 2020-11-23 11:18:21 +07:00
Alex Sharov
ccd1ad5832
Sequence in to mdbx (#1370) 2020-11-23 11:15:43 +07:00
Alex Sharov
068463dff4
Store transactions individually (#1358)
* Store transactions individually

* Store transactions individually

* save progress

* checkIndex

* merge
2020-11-22 21:25:26 +00:00
Chase Wright
a8b7a59977
Update rpcdaemon README with eth_(un)subscribe (#1365)
* `eth_subscribe` is supported over WebSocket
* Only `newHeads` is supported at this time
* `eth_unsubscribe is supported
2020-11-19 15:39:13 +01:00
lightclient
377b979c98
Accept discover,netrestrict,staticpeer arguments for sentry (#1364)
* accept discover,netrestrict,staticpeer arguments for sentry

* appease go linter
2020-11-19 08:14:41 +00:00
ledgerwatch
1a0e78a2e2
Remove changeset-based decorators from WalkAsOf (#1356)
* Remove decorators

* Current changes

* Fix and skip some tests for now

* Fix lint

* Specialise WalkAsOf

* Fix compile errors

* Fix lint

* Scan index files for key sizes

* More cleanup

* Fix to remote server

* Fix for walkasofstorage

* Fix/skip tests

* Fix tests

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-18 14:58:52 +00:00
Igor Mandrigin
393c9965ae
rpcdaemon: subscriptions, newHeads (#1359)
* fix `make grpc` on new checkouts

* update proto files

* add some stub

* prototype with fake events

* notifying about events

* pass events

* events are being sent

* transfer headers to filters

* create the “filters” struct

* implement new heads

* PoC of New Heads subscription

* fix keep alive

* fixups for the client

* add “type” to the event

* support header event type on client

* better stage refactor

* fixup for the eth backend

* fixups

* fix tests

* fix tests

* fix linters

* address comments

* remove unused log
2020-11-17 19:13:41 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342)
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-11-16 12:08:28 +00:00
b00ris
6464da7670
Remote snapshot downloader (#1343)
* save state

* save state

* save state

* refactoring

* fix

* save state

* save state

* fmt

* fix lint

* restore torrents for external downloader

* fix lint

* download

* skip debug test

* debug

* remote debug

* small cli fixes

* skip debug test

* external snapshot predownloader

* get rid of remote downloader

* fix lint

* clean makefile

* fix lint

* fix lint

* cleanup

* fix ci

* fmt

* remove proto from interfaces

* Squashed 'interfaces/' content from commit acd02bb94

git-subtree-dir: interfaces
git-subtree-split: acd02bb94c5a421aa8f8d1fd76cd8aad668e9fcb
2020-11-13 16:16:47 +00:00
ledgerwatch
10f338b8f7
Add lmdb guide visualisations (#1348)
* Add lmdb guide visualisations

* updated names, remove double-tap visualisations
2020-11-11 17:18:34 +00:00
ledgerwatch
e71c3cae70
Lmdb visualisation + freelist stability fixes (#1345)
* Visualisations

* Use graphviz records

* Fix lint

* Fix lint

* Reduced leaf pages

* Fix

* Overflow pages

* Fix lint

* Fix lint

* Fix lint

* Add dupsort generate

* DupSort experiments

* Add dupsort nodes

* More interesting dupsort

* freelist

* Fragmented freelist

* Fix lint

* drop T2 too

* Handle overflow pages in FREE_DBI

* Skipping in the MAIN_DBI leaves

* Switch to double-tap LMDB

* Fix to double-tap

* With/without reader visualisations

* Flip flopping without reader

* Tidy mods

* Fix lint, switch to new lmdb-go release

* mod tidy
2020-11-11 09:35:58 +00:00
Igor Mandrigin
ed9672620b
Granular rpc control (Allow list for RPC daemon) (#1341) 2020-11-10 10:08:42 +01:00
Thomas Jay Rush
546b91f47e
Updating RPC tests in Postman (#1340) 2020-11-09 09:52:18 +01:00