Commit Graph

12375 Commits

Author SHA1 Message Date
Martin Holst Swende
5d0b4ca3ff rpc, internal/ethapi: default rpc gascap at 25M + better error message (#21229)
* rpc, internal/ethapi: default rpc gascap at 50M + better error message

* eth,internal: make globalgascap uint64

* core/tests: fix compilation failure

* eth/config: gascap at 25M + minor review concerns
# Conflicts:
#	core/tx_pool_test.go
#	internal/ethapi/api.go
#	les/api_backend.go
2020-08-07 12:27:17 +02:00
Marius van der Wijden
84f3ed1b88 core, txpool: less allocations when handling transactions (#21232)
* core: use uint64 for total tx costs instead of big.Int

* core: added local tx pool test case

* core, crypto: various allocation savings regarding tx handling

* Update core/tx_list.go

* core: added tx.GasPriceIntCmp for comparison without allocation

adds a method to remove unneeded allocation in comparison to tx.gasPrice

* core: handle pools full of locals better

* core/tests: benchmark for tx_list

* core/txlist, txpool: save a reheap operation, avoid some bigint allocs

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	common/math/integer.go
#	core/tx_list.go
#	core/tx_pool.go
#	core/types/transaction.go
2020-08-07 12:25:38 +02:00
Marius van der Wijden
6eaa9d340b rlp: avoid list header allocation in encoder (#21274)
List headers made up 11% of all allocations during sync. This change
removes most of those allocations by keeping the list header values
cached in the encoder buffer instead. Since encoder buffers are pooled,
list headers are no longer allocated in the common case where an
encoder buffer is available for reuse.

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	rlp/encode.go
2020-08-07 11:53:22 +02:00
Adam Schmideg
d6f7b4b92c cmd/geth: LES priority client test (#20719)
This adds a regression test for the LES priority client API.
2020-08-07 11:51:50 +02:00
Marius van der Wijden
12de9c3f65 eth/downloader: fixes data race between synchronize and other methods (#21201)
* eth/downloaded: fixed datarace between synchronize and Progress

There was a race condition between `downloader.synchronize()` and `Progress` `syncWithPeer` `fetchHeight` `findAncestors` and `processHeaders`
This PR changes the behavior of the downloader a bit.
Previously the functions `Progress` `syncWithPeer` `fetchHeight` `findAncestors` and `processHeaders` read the syncMode anew within their loops. Now they read the syncMode at the start of their function and don't change it during their runtime.

* eth/downloaded: comment

* eth/downloader: added comment
# Conflicts:
#	eth/downloader/downloader.go
2020-08-07 11:51:38 +02:00
Marius van der Wijden
9de5600229 core: types: less allocations when hashing and tx handling (#21265)
* core, crypto: various allocation savings regarding tx handling

* core: reduce allocs for gas price comparison

This change reduces the allocations needed for comparing different transactions to each other.
A call to `tx.GasPrice()` copies the gas price as it has to be safe against modifications and
also needs to be threadsafe. For comparing and ordering different transactions we don't need
these guarantees

* core: added tx.GasPriceIntCmp for comparison without allocation

adds a method to remove unneeded allocation in comparison to tx.gasPrice

* core/types: pool legacykeccak256 objects in rlpHash

rlpHash is by far the most used function in core that allocates a legacyKeccak256 object on each call.
Since it is so widely used it makes sense to add pooling here so we relieve the GC.
On my machine these changes result in > 100 MILLION less allocations and > 30 GB less allocated memory.

* reverted some changes

* reverted some changes

* trie: use crypto.KeccakState instead of replicating code

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	core/tx_list.go
#	core/tx_pool.go
#	core/types/block.go
#	core/types/transaction.go
#	trie/committer.go
#	trie/hasher.go
2020-08-07 11:46:33 +02:00
Martin Holst Swende
14e3fda786 cmd/evm: add state transition tool for testing (#20958)
This PR implements the EVM state transition tool, which is intended
to be the replacement for our retesteth client implementation.
Documentation is present in the cmd/evm/README.md file.

Co-authored-by: Felix Lange <fjl@twurst.com>
# Conflicts:
#	cmd/evm/main.go
#	core/state/dump.go
#	core/state/state_test.go
#	tests/state_test_util.go
2020-08-07 11:38:07 +02:00
Binacs
5f3516ccf7 cmd: abstract getPassPhrase functions into one (#21219)
* [cmd] Abstract `getPassPhrase` functions into one.

* cmd/ethkey: fix compilation failure

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
# Conflicts:
#	cmd/clef/main.go
#	cmd/ethkey/utils.go
#	cmd/geth/accountcmd.go
2020-08-07 11:25:25 +02:00
meowsbits
f106dd529e utils: fix ineffectual miner config flags (#21271)
Without use of global, these flags didn't actually modify
miner configuration, since we weren't grabbing from the
proper context scope, which should be global (vs. subcommand).

Signed-off-by: meows <b5c6@protonmail.com>
2020-08-07 11:23:24 +02:00
Marius van der Wijden
0cd5872378 eth: returned revert reason in traceTx (#21195)
* eth: returned revert reason in traceTx

* eth: return result data
2020-08-07 11:23:14 +02:00
meowsbits
8930bd2892 cmd/geth: make import cmd exit with 1 if import errors occurred (#21244)
The import command should not return a 0 status
code if the import finishes prematurely becaues
of an import error.

Returning the error causes the program to exit with 1
if the err is non nil.

Signed-off-by: meows <b5c6@protonmail.com>
# Conflicts:
#	cmd/geth/chaincmd.go
2020-08-07 11:23:06 +02:00
Marius van der Wijden
d3d58d9d07 core/vm: fix incorrect computation of BLS discount (#21253)
* core/vm: fix incorrect computation of discount

During testing on Yolov1 we found that the way geth calculates the discount
is not in line with the specification. Basically what we did is calculate
128 * Bls12381GXMulGas * discount / 1000 whenever we received more than 128 pairs
of values. Correct would be to calculate k * Bls12381... for k > 128.

* core/vm: better logic for discount calculation

* core/vm: better calculation logic, added worstcase benchmarks

* core/vm: better benchmarking logic
2020-08-07 11:22:28 +02:00
Marius van der Wijden
4d3004c152 trie: reduce allocs in insertPreimage (#21261) 2020-08-07 11:22:18 +02:00
gary rong
20de87c1bf eth: don't block if transaction broadcast loop fails (#21255)
* eth: don't block if transaction broadcast loop is returned

* eth: kick out peer if we failed to send message

* eth: address comment
2020-08-07 11:22:09 +02:00
Felix Lange
bc23fc13ab crypto/secp256k1: enable 128-bit int code and endomorphism optimization (#21203)
* crypto/secp256k1: enable use of __int128

This speeds up scalar & field calculations a lot.

* crypto/secp256k1: enable endomorphism optimization
2020-08-07 11:21:59 +02:00
Felix Lange
273230e3c5 cmd/devp2p: add commands for node key management (#21202)
These commands mirror the key/URL generation functions of cmd/bootnode.

    $ devp2p key generate mynode.key
    $ devp2p key to-enode mynode.key -ip 203.0.113.21 -tcp 30304
    enode://78a7746089baf4b8615f54a5f0b67b22b1...
2020-08-07 11:21:48 +02:00
Péter Szilágyi
d73075bd3d p2p: measure packet throughput too, not just bandwidth 2020-08-07 11:21:39 +02:00
Guillaume Ballet
d04e4b25c8 common/fdlimit: build on DragonflyBSD (#21241)
* common/fdlimit: build on DragonflyBSD

* review feedback
# Conflicts:
#	go.mod
#	go.sum
2020-08-07 11:21:19 +02:00
AusIV
f14fc201dc core/rawdb: fix high memory usage in freezer (#21243)
The ancients variable in the freezer is a list of hashes, which
identifies all of the hashes to be frozen. The slice is being allocated
with a capacity of `limit`, which is the number of the last block
this batch will attempt to add to the freezer. That means we are
allocating memory for all of the blocks in the freezer, not just
the ones to be added.

If instead we allocate `limit - f.frozen`, we will only allocate
enough space for the blocks we're about to add to the freezer. On
mainnet this reduces usage by about 320 MB.
# Conflicts:
#	core/rawdb/freezer.go
2020-08-07 11:20:23 +02:00
ucwong
c9e176b59d go.mod: update snappy dependency (#21237)
# Conflicts:
#	go.mod
#	go.sum
2020-08-07 11:19:47 +02:00
gary rong
2740fc90d9 cmd, eth, internal, les: add txfee cap (#21212)
* cmd, eth, internal, les: add gasprice cap

* cmd/utils, eth: add default value for gasprice cap

* all: use txfee cap

* cmd, eth: add fix

* cmd, internal: address comments
# Conflicts:
#	les/api_backend.go
2020-08-07 11:18:39 +02:00
zhangsoledad
c16e980e6c core/rawdb: swap tailId and itemOffset for deleted items in freezer (#21220)
* fix(freezer): tailId filenum offset were misplaced

* core/rawdb: assume first item in freezer always start from zero
# Conflicts:
#	core/rawdb/freezer_table.go
#	core/rawdb/freezer_table_test.go
2020-08-07 11:18:24 +02:00
sixdays
287abd928b eth/downloader: don't use defer for unlock before return (#21227)
Co-authored-by: linjing <linjingjing@baidu.com>
2020-08-07 11:17:56 +02:00
HackyMiner
36a0d2f9ef internal/web3ext: add missing params to debug.accountRange (#21208) 2020-08-07 11:17:46 +02:00
Yang Hau
e6056c0acc build, internal/ethapi, crypto/bls12381: fix typos (#21210)
speicifc -> specific
assigened -> assigned
frobenious -> frobenius
2020-08-07 11:17:36 +02:00
gary rong
f5b6ae0da1 core, eth, internal: include read storage entries in structlog output (#21204)
* core, eth, internal: extend structLog tracer

* core/vm, internal: add storage view

* core, internal: add slots to storage directly

* core: remove useless

* core: address martin's comment

* core/vm: fix tests
# Conflicts:
#	core/vm/logger.go
#	core/vm/logger_test.go
2020-08-07 11:17:27 +02:00
Martin Holst Swende
bb5f890c0c eth/downloader: fix spuriously failing tests (#21149)
* eth/downloader tests: fix spurious failing test due to race between receipts/headers

* miner tests: fix travis failure on arm64

* eth/downloader: tests - store td in ancients too
# Conflicts:
#	eth/downloader/downloader_test.go
#	miner/worker_test.go
2020-08-07 11:15:45 +02:00
Natsu Kagami
7034679fd7 accounts/abi: make GetType public again (#21157) 2020-08-07 11:13:20 +02:00
Pau
4be8cfe42c accounts/keystore: fix typo in error message (#21200) 2020-08-07 11:13:12 +02:00
Péter Szilágyi
d25bb87a6f core/state: avoid escape analysis fault when accessing cached state
# Conflicts:
#	core/state/state_object.go
#	core/state/statedb.go
2020-08-07 11:04:56 +02:00
Paweł Bylica
924c03abf9 core/vm: use uint256 in EVM implementation (#20787)
* core/vm: use fixed uint256 library instead of big

* core/vm: remove intpools

* core/vm: upgrade uint256, fixes uint256.NewFromBig

* core/vm: use uint256.Int by value in Stack

* core/vm: upgrade uint256 to v1.0.0

* core/vm: don't preallocate space for 1024 stack items (only 16)

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	core/vm/common.go
#	core/vm/contract.go
#	core/vm/eips.go
#	core/vm/evm.go
#	core/vm/gas_table.go
#	core/vm/instructions.go
#	core/vm/instructions_test.go
#	core/vm/interpreter.go
#	core/vm/logger.go
#	core/vm/logger_json.go
#	core/vm/logger_test.go
#	core/vm/stack/stack.go
#	go.mod
#	go.sum
2020-08-07 11:04:19 +02:00
Igor Mandrigin
bbb918ea6c etl: update docs with the correct sort method 2020-08-06 18:22:58 +02:00
ledgerwatch
2f3cc6f62c
Header download spec - first draft (#879)
* header download docs

* Continue

* Fix

* More picture and text

* Push

* More doc

* More docs

* First version of the doc
2020-08-06 15:37:41 +01:00
Igor Mandrigin
715874f060 update version 2020-08-06 14:09:36 +02:00
ucwong
f30682ce7c ethstats: use timer instead of time.sleep (#20924) 2020-08-06 14:08:02 +02:00
libby kent
70112ee9de cmd/ethkey: support --passwordfile in generate command (#21183) 2020-08-06 14:07:43 +02:00
Felix Lange
9df275d7e0 params: begin v1.9.16 release cycle 2020-08-06 14:07:26 +02:00
Hao Duan
1ca4a9263f core: filter out txs with invalid signatures as soon as possible
Once we detect an invalid transaction during recovering signatures, we should
directly exclude this transaction to avoid validating the signatures hereafter.

This should optimize the validations times of transactions with invalid signatures
to only one time.
2020-08-06 14:06:31 +02:00
Igor Mandrigin
d1bf85ddeb add merging thing 2020-08-06 14:05:34 +02:00
Igor Mandrigin
ba9f9ee7dd
Add docs for common/etl (#878)
* ETL readme

* add a link to the explainer

* fixups

* a note on marshaling
2020-08-06 14:02:41 +02:00
Igor Mandrigin
1495979983
etl: only print file sizes for actual files, not RAM (#877) 2020-08-06 09:33:09 +02:00
pk
25ec16f223
Update Readme.md (#876) 2020-08-05 18:29:54 +01:00
Igor Mandrigin
fa89cdefd5
log ETL temp files datasize (#873) 2020-08-05 16:33:58 +01:00
Alex Sharov
f41959f030
drop badger support (#869)
* drop badger support

* rebase master
2020-08-05 16:33:45 +01:00
Alexey Akhunov
b72fae75c9 Fix some snags in stage 6 2020-08-05 11:31:41 +01:00
Alexey Akhunov
11637c4c89 Fix some snags in stage 6 2020-08-05 11:25:57 +01:00
Alex Sharov
d458c4cc1e
Migrations: use stage name as db key (#868) 2020-08-05 17:13:35 +07:00
Igor Mandrigin
8bc293b3f7
Remove unnecessary memory profiling (#872)
* remove separate memory profiling from staged sync

* don’t run pprof globally always
2020-08-05 10:54:19 +01:00
Igor Mandrigin
422282fe7b
Merge pull request #870 from ledgerwatch/lmdb-kv-override
Add lmdb.mapSize param
2020-08-05 11:03:27 +02:00
Igor Mandrigin
d48b9141be
Merge pull request #871 from ledgerwatch/staged-sync-piechart
docs: add a piechart with stages
2020-08-05 11:03:12 +02:00