Commit Graph

1855 Commits

Author SHA1 Message Date
Igor Mandrigin
bd61245ece test fixups 2020-08-29 14:35:39 +02:00
Igor Mandrigin
948cd29be7 fixes 2020-08-29 13:43:44 +02:00
Igor Mandrigin
78e466d06a fix imports 2020-08-29 13:31:46 +02:00
Martin Holst Swende
991a1c063d core: do less lookups when writing fast-sync block bodies (#21468) 2020-08-29 13:25:33 +02:00
gary rong
550bd11f9d core, eth, les, trie: add a prefix to contract code (#21080)
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	consensus/clique/clique.go
#	consensus/ethash/consensus.go
#	core/block_validator.go
#	core/blockchain.go
#	core/blockchain_test.go
#	core/genesis.go
#	core/rawdb/accessors_indexes_test.go
#	core/rawdb/accessors_metadata.go
#	core/rawdb/chain_iterator_test.go
#	core/rawdb/database.go
#	core/rawdb/schema.go
#	core/state/database.go
#	core/state/iterator_test.go
#	core/state/statedb.go
#	core/state/sync.go
#	core/state/sync_test.go
#	core/tx_pool_test.go
#	core/types/block_test.go
#	core/types/derive_sha.go
#	eth/downloader/downloader.go
#	eth/downloader/queue.go
#	eth/downloader/statesync.go
#	eth/fetcher/block_fetcher.go
#	eth/fetcher/block_fetcher_test.go
#	eth/handler.go
#	les/odr_requests.go
#	les/server_handler.go
#	light/odr.go
#	light/odr_test.go
#	light/trie.go
#	miner/worker.go
#	trie/database.go
#	trie/secure_trie.go
#	trie/sync.go
#	trie/sync_bloom.go
#	trie/sync_test.go
2020-08-29 13:25:20 +02:00
Péter Szilágyi
bfac425c87 core: define and test chain rewind corner cases (#21409)
* core: define and test chain reparation cornercases

* core: write up a variety of set-head tests

* core, eth: unify chain rollbacks, handle all the cases

* core: make linter smile

* core: remove commented out legacy code

* core, eth/downloader: fix review comments

* core: revert a removed recovery mechanism
# Conflicts:
#	core/blockchain.go
#	core/blockchain_test.go
#	core/headerchain.go
#	core/rawdb/database.go
#	core/rawdb/freezer.go
#	core/rawdb/schema.go
#	eth/downloader/downloader.go
#	eth/sync.go
#	trie/sync.go
2020-08-29 12:29:00 +02:00
Marius van der Wijden
08d3993cef core: more detailed metering for reorgs (#21420) 2020-08-27 21:23:51 +02:00
Giuseppe Bertone
5388193c2c core/state: fixed some comments (#21450)
# Conflicts:
#	core/state/state_object.go
#	core/state/statedb.go
#	core/state/statedb_test.go
2020-08-27 17:38:12 +02:00
Igor Mandrigin
12aae871fd skip_analysis: update block number 2020-08-27 10:35:32 +02:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975)
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
Alex Sharov
522287ac18
Transactional cycle (#966)
* v0

* v1

* v3

* v4

* clean

* temporary fix of txpool

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* add condition

* tx pool to not hold own db

* try to enable TxPool in integration

* exclude tx pool from tx

* exclude tx pool from integration

* reduce limit

* fix integration

* clean

* clean

* clean

* clean

* clean

* exclude tx pool unwind

* exclude tx pool unwind in integration

* fix integration tx pool

* fix commit

* fix current stage after unwind

* fix current stage after unwind

* fix linter

* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.

* move body and headers unwind out of tx

* fix unwind order after reboot

* add support external tx to exec stage

* clean

* clean

* clean

* clean

* clean

* add logs

* better id check

* better id check
2020-08-26 07:02:10 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Alex Sharov
2facf92533
Better logs2 (#964) 2020-08-23 06:55:17 +07:00
Alex Sharov
a6be18b915
ticker-based logs (#954)
* timer-based logs

* timer-based logs

* delegate progress calculation to user

* delegate progress calculation to user

* delegate progress calculation to user

* clear

* add logs to senders recovery

* use default dir in integration

* more logs

* more logs
2020-08-22 12:12:33 +02:00
ledgerwatch
7c16160087
Add IntraBlockState copy function to RPC API (#960) 2020-08-21 21:53:23 +01:00
ledgerwatch
fd7e91a51c
Update skip_analysis (#942) 2020-08-19 20:34:45 +01:00
b00ris
eb46dd68df
Account range testing (#938)
* save state

* save state

* save new timeouts

* compare passed

* save state

* cleanup
2020-08-19 07:33:49 +01:00
Evgeny Danilenko
3b40819444
grpc regenerate (#936)
* grpc regenerate

* use string type for id in net_version
2020-08-18 20:22:49 +03:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928)
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size (#918)
* add logging to loader

* use pure tx in etl loading, logs in mutation commit

* clean

* bletter logging and more cleanup

* bletter logging and more cleanup

* increase batch size to 500M

* better batch commit logging

* async fsync

* sync fsync

* sync fsync

* unify logging

* fix corner-case when etl can use empty bucket name

* fix tests

* better logging

* better logging

* rebase master

* remove lmdb.NoMetaSync flag for now

* consistent walk and multi-walk

* clean

* sub tx

* add consistent multi-put

* implement dupsort support in one new cursor method

* clear
2020-08-17 07:45:52 +01:00
Alex Sharov
6379516a0f
add support of blocks compression to txpool stage (#925) 2020-08-17 10:17:40 +07:00
Igor Mandrigin
2f881f3865
Use batches when unwinding indexes (#931) 2020-08-16 18:54:33 +02:00
Igor Mandrigin
d848233275 Merge branch 'master' into geth-1.9.19
# Conflicts:
#	cmd/geth/consolecmd.go
#	cmd/rpcdaemon/service/service.go
#	cmd/utils/flags.go
#	eth/backend.go
2020-08-15 21:55:09 +02:00
Igor Mandrigin
71758e1fcc fixups 2020-08-15 21:34:22 +02:00
Igor Mandrigin
8346124959 fix core/types 2020-08-15 21:15:44 +02:00
Igor Mandrigin
201524c33e fix node tests 2020-08-15 20:21:30 +02:00
Igor Mandrigin
d6fe697ee1 fixups. make geth successful 2020-08-15 19:32:05 +02:00
Igor Mandrigin
9da7a7c608 fixez 2020-08-15 17:30:34 +02:00
Alex Sharov
90fa69a9e5
Swap IH and HashState stages (#922)
* move ih stage v0

* save progress

* save working version

* baby step

* baby step

* baby step

* non-baby step

* cleanup
2020-08-15 14:49:36 +01:00
Igor Mandrigin
3ba734f657 fixups 2020-08-14 17:43:48 +02:00
Martin Holst Swende
9393ba7d8f core/vm: avoid map lookups for accessing jumpdest analysis
# Conflicts:
#	core/vm/contract.go
2020-08-14 16:35:53 +02:00
Hao Duan
44d9fba09a core: avoid modification of accountSet cache in tx_pool (#21159)
* core: avoid modification of accountSet cache in tx_pool

when runReorg, we may copy the dirtyAccounts' accountSet cache to promoteAddrs
in which accounts will be promoted, however, if we have reset request at the
same time, we may reuse promoteAddrs and modify the cache content which is
against the original intention of accountSet cache. So, we need to make a new
slice here to avoid modify accountSet cache.

* core: fix flatten condition + comment

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-08-14 16:34:29 +02:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object (#914) 2020-08-14 13:41:18 +07:00
Evgeny Danilenko
5d39fe8873
RPCDeamon coinbase, net_version (#911)
* use eth.backend

* net_version

* add coinbase to the console test
2020-08-12 14:47:59 +01:00
ucwong
9cf277a175 core/bloombits: use single channel for shutdown (#20878)
This replaces the two-stage shutdown scheme with the one we
use almost everywhere else: a single quit channel signalling
termination.

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-08-12 12:43:50 +02:00
Péter Szilágyi
0933e1621c core: API-less transaction time sorting
# Conflicts:
#	core/types/transaction.go
#	core/types/transaction_test.go
2020-08-12 12:05:10 +02:00
Hendrik Hofstadt
f8993938b4 core: sort txs at the same gas price by received time
# Conflicts:
#	core/types/transaction.go
#	core/types/transaction_test.go
2020-08-12 12:03:23 +02:00
gary rong
1bc2bef061 cmd, core, eth, light, trie: dump clean cache periodically (#20391)
* cmd, core, eth, light, trie: dump clean cache periodically

* eth: update config

* trie: minor fix

* core, trie: address comments

* eth: remove useless

* trie: print clean cache dump start too

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	core/blockchain.go
#	core/genesis.go
#	core/state/database.go
#	eth/api_tracer.go
#	eth/backend.go
#	eth/config.go
#	light/postprocess.go
#	trie/database.go
2020-08-12 12:01:13 +02:00
Evgeny Danilenko
3980fa7d45
Grps eth_sendTransaction (#882)
* implementation

* tidy gomod

* linters

* fix cmd test

* fix

* fix lint
2020-08-11 22:09:30 +01:00
ledgerwatch
381f721f01
Update skip analysis block number for mainnet (#906) 2020-08-11 11:23:07 +01:00
Alex Sharov
d9d9e14f45
change bucket type to string (#894) 2020-08-11 06:55:32 +07:00
Igor Mandrigin
8600616c3d geth 1.9.18 post-rebase fixups 2020-08-08 17:33:35 +02:00
Martin Holst Swende
e3e00089f4 eth/downloader: refactor downloader + queue (#21263)
* eth/downloader: refactor downloader + queue

downloader, fetcher: throttle-metrics, fetcher filter improvements, standalone resultcache

downloader: more accurate deliverytime calculation, less mem overhead in state requests

downloader/queue: increase underlying buffer of results, new throttle mechanism

eth/downloader: updates to tests

eth/downloader: fix up some review concerns

eth/downloader/queue: minor fixes

eth/downloader: minor fixes after review call

eth/downloader: testcases for queue.go

eth/downloader: minor change, don't set progress unless progress...

eth/downloader: fix flaw which prevented useless peers from being dropped

eth/downloader: try to fix tests

eth/downloader: verify non-deliveries against advertised remote head

eth/downloader: fix flaw with checking closed-status causing hang

eth/downloader: hashing avoidance

eth/downloader: review concerns + simplify resultcache and queue

eth/downloader: add back some locks, address review concerns

downloader/queue: fix remaining lock flaw

* eth/downloader: nitpick fixes

* eth/downloader: remove the *2*3/4 throttling threshold dance

* eth/downloader: print correct throttle threshold in stats

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	eth/downloader/downloader.go
#	eth/downloader/downloader_test.go
#	eth/downloader/queue.go
#	eth/downloader/statesync.go
#	eth/fetcher/block_fetcher.go
2020-08-07 14:23:49 +02:00
rjl493456442
b702eee5f4 core: fix heartbeat in txpool
core: address comment

# Conflicts:
#	core/tx_pool.go
2020-08-07 13:53:54 +02:00
villanuevawill
18ef3d01d8 core: fix queued transaction eviction
Solves issue#20582. Non-executable transactions should not be evicted on each tick if there are no promote transactions or if a pending/reset empties the pending list. Tests and logging expanded to handle these cases in the future.

core/tx_pool: use a ts for each tx in the queue, but only update the heartbeat on promotion or pending replaced

queuedTs proper naming

# Conflicts:
#	core/tx_pool.go
2020-08-07 13:53:16 +02:00
Sammy Libre
349f8e121f ethclient: serialize negative block number as "pending" (#21177)
Fixes #21175

Co-authored-by: sammy007 <sammy007@users.noreply.github.com>
Co-authored-by: Adam Schmideg <adamschmideg@users.noreply.github.com>
2020-08-07 13:49:28 +02:00
Péter Szilágyi
4701dcb508 core/vm: use pointers to operations vs. copy by value
# Conflicts:
#	core/vm/interpreter.go
2020-08-07 13:47:25 +02:00
Nikola Madjarevic
c0a26827a3 core: raise gas limit in --dev mode, seed blake precompile (#21323)
* Set gasLimit in --dev mode to be 9m.

* core: Set gasLimit to 11.5 milion and add 1 wei allocation for BLAKE2b
2020-08-07 13:46:40 +02:00
Martin Holst Swende
a4b70f9ee9 core/vm: less allocations for various call variants (#21222)
* core/vm/runtime/tests: add more benchmarks

* core/vm: initial work on improving alloc count for calls to precompiles

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6     117ms ±75%      43ms ± 1%  -63.09%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   79.6ms ± 4%    70.5ms ± 1%  -11.42%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    24.4MB ± 0%     4.9MB ± 0%  -79.94%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 0%    13.2kB ± 0%     ~     (p=0.357 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6      382k ± 0%      153k ± 0%  -59.99%  (p=0.000 n=5+4)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* core/vm: don't allocate big.int for touch

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6    43.3ms ± 1%    42.4ms ± 7%     ~     (p=0.151 n=5+5)
SimpleLoop/loop-10M-6                   70.5ms ± 1%    76.7ms ± 1%   +8.67%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    4.90MB ± 0%    2.46MB ± 0%  -49.83%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 0%    13.2kB ± 1%     ~     (p=0.571 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6      153k ± 0%       76k ± 0%  -49.98%  (p=0.029 n=4+4)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* core/vm: reduce allocs in staticcall

name                                  old time/op    new time/op    delta
SimpleLoop/identity-precompile-10M-6    42.4ms ± 7%    37.5ms ± 6%  -11.68%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   76.7ms ± 1%    69.1ms ± 1%   -9.82%  (p=0.008 n=5+5)

name                                  old alloc/op   new alloc/op   delta
SimpleLoop/identity-precompile-10M-6    2.46MB ± 0%    0.02MB ± 0%  -99.35%  (p=0.008 n=5+5)
SimpleLoop/loop-10M-6                   13.2kB ± 1%    13.2kB ± 0%     ~     (p=0.143 n=5+5)

name                                  old allocs/op  new allocs/op  delta
SimpleLoop/identity-precompile-10M-6     76.4k ± 0%      0.1k ± 0%     ~     (p=0.079 n=4+5)
SimpleLoop/loop-10M-6                     40.0 ± 0%      40.0 ± 0%     ~     (all equal)

* trie: better use of hasher keccakState

* core/state/statedb: reduce allocations in getDeletedStateObject

* core/vm: reduce allocations in all call derivates

* core/vm: reduce allocations in call variants

- Make returnstack `uint32`
- Use a `sync.Pool` of `stack`s

* core/vm: fix tests

* core/vm: goimports

* core/vm: tracer fix + staticcall gas fix

* core/vm: add back snapshot to staticcall

* core/vm: review concerns + make returnstack pooled + enable returndata in traces

* core/vm: fix some test tracer method signatures

* core/vm: run gencodec, minor comment polish

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	core/state/statedb.go
#	core/vm/contracts_test.go
#	core/vm/evm.go
#	core/vm/instructions.go
#	core/vm/interpreter.go
#	core/vm/logger.go
#	core/vm/logger_json.go
#	core/vm/logger_test.go
#	core/vm/runtime/runtime_test.go
#	core/vm/stack/stack.go
#	eth/tracers/tracer.go
#	eth/tracers/tracer_test.go
#	trie/secure_trie.go
2020-08-07 13:46:26 +02:00
Martin Holst Swende
64565f4b2b core: transaction pool optimizations (#21328)
* core: added local tx pool test case

* core, crypto: various allocation savings regarding tx handling

* core/txlist, txpool: save a reheap operation, avoid some bigint allocs

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2020-08-07 13:23:40 +02:00