Commit Graph

12936 Commits

Author SHA1 Message Date
Felix Lange
0c54d2e649 p2p/discover: implement v5.1 wire protocol (#21647)
This change implements the Discovery v5.1 wire protocol and
also adds an interactive test suite for this protocol.
# Conflicts:
#	cmd/devp2p/discv5cmd.go
#	p2p/discover/v5_encoding.go
#	p2p/discover/v5_encoding_test.go
#	p2p/discover/v5_udp.go
#	p2p/discover/v5_udp_test.go
#	p2p/discover/v5wire/session.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
26069ae7a4 miner: set etherbase even if mining isn't possible at the moment (#21707) 2020-10-26 17:16:00 +01:00
gary rong
ee6e0cbd5c params: update CHTs (#21706) 2020-10-26 17:16:00 +01:00
Marius van der Wijden
ac604670a3 accounts/keystore: fix flaky test (#21703)
* accounts/keystore: add timeout to test to prevent failure on travis

The TestWalletNotifications test sporadically fails on travis.
This is because we shutdown the event collection before all events are received.
Adding a small timeout (10 milliseconds) allows the collector to be scheduled
and to consume all pending events before we shut it down.

* accounts/keystore: added newlines back in

* accounts/keystore: properly fix the walletNotifications test
2020-10-26 17:16:00 +01:00
Martin Holst Swende
735c5b631c miner: don't interrupt mining after successful sync (#21701)
* miner: exit loop when downloader Done or Failed

Following the logic of the comment at the method,
this fixes a regression introduced at 7cf56d6f064869cb62b1673f9ee437020c595391
, which would allow external parties to DoS with
blocks, preventing mining progress.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: remove ineff assign (lint)

Signed-off-by: meows <b5c6@protonmail.com>

* miner: update test re downloader events

Signed-off-by: meows <b5c6@protonmail.com>

* Revert "miner: remove ineff assign (lint)"

This reverts commit eaefcd34ab4862ebc936fb8a07578aa2744bc058.

* Revert "miner: exit loop when downloader Done or Failed"

This reverts commit 23abd34265aa246c38fc390bb72572ad6ae9fe3b.

* miner: add test showing imprecise TestMiner

Signed-off-by: meows <b5c6@protonmail.com>

* miner: fix waitForMiningState precision

This helper function would return an affirmation
on the first positive match on a desired bool.

This was imprecise; it return false positives
by not waiting initially for an 'updated' value.

This fix causes TestMiner_2 to fail, which is
expected.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: remove TestMiner_2 demonstrating broken test

This test demonstrated the imprecision of the test
helper function waitForMiningState. This function
has been fixed with 6d365c2851, and this test test
may now be removed.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: fix test regarding downloader event/mining expectations

See comment for logic.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: add test describing expectations for downloader/mining events

We expect that once the downloader emits a DoneEvent,
signaling a successful sync, that subsequent StartEvents
are not longer permitted to stop the miner.

This prevents a security vulnerability where forced syncs via
fake high blocks would stall mining operation.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: use 'canStop' state to fix downloader event handling

- Break downloader event handling into event
separating Done and Failed events. We need to
treat these cases differently since a DoneEvent
should prevent the miner from being stopped on
subsequent downloader Start events.

- Use canStop state to handle the one-off
case when a downloader first succeeds.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: improve comment wording

Signed-off-by: meows <b5c6@protonmail.com>

* miner: start mining on downloader events iff not already mining

Signed-off-by: meows <b5c6@protonmail.com>

* miner: refactor miner update logic w/r/t downloader events

This makes mining pause/start logic regarding downloader
events more explicit. Instead of eternally handling downloader
events after the first done event, the subscription is closed
when downloader events are no longer actionable.

Signed-off-by: meows <b5c6@protonmail.com>

* miner: fix handling downloader events on subcription closed

Signed-off-by: meows <b5c6@protonmail.com>

* miner: (lint:gosimple) use range over chan instead of for/select

Signed-off-by: meows <b5c6@protonmail.com>

* miner: refactor update loop to remove race condition

The go routine handling the downloader events handling
vars in parallel with the parent routine, causing a
race condition.

This change, though ugly, remove the condition while
still allowing the downloader event subscription to be
closed when the miner has no further use for it (ie DoneEvent).

* miner: alternate fix for miner-flaw

Co-authored-by: meows <b5c6@protonmail.com>
# Conflicts:
#	miner/miner_test.go
2020-10-26 17:16:00 +01:00
Giuseppe Bertone
c690231219 cmd/geth, cmd/utils: fixed flags name (#21700)
# Conflicts:
#	cmd/geth/main.go
#	cmd/geth/usage.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
926cc78870 eth, p2p: use truncated names (#21698)
* peer: return localAddr instead of name to prevent spam

We currently use the name (which can be freely set by the peer) in several log messages.
This enables malicious actors to write spam into your geth log.
This commit returns the localAddr instead of the freely settable name.

* p2p: reduce usage of peer.Name in warn messages

* eth, p2p: use truncated names

* Update peer.go

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2020-10-26 17:16:00 +01:00
mr_franklin
57817f3159 internal/web3ext: improve some web3 apis (#21639)
* imporve some web3-ext apis

* Update web3ext.go

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-10-26 17:16:00 +01:00
Martin Holst Swende
5d9c4ff6ee trie: fix flaw in stacktrie pool reuse (#21699)
# Conflicts:
#	trie/stacktrie.go
2020-10-26 17:16:00 +01:00
mr_franklin
a1f42ea04a consensus/clique: unexport calcDifficulty and improve comment (#21619)
# Conflicts:
#	consensus/clique/clique.go
2020-10-26 17:16:00 +01:00
mr_franklin
4d59c11d20 all: replace RWMutex with Mutex in places where RLock is not used (#21622)
# Conflicts:
#	les/txrelay.go
#	miner/unconfirmed.go
2020-10-26 17:16:00 +01:00
Hanjiang Yu
1bbd51d31b console: fix admin.sleepBlocks (#21629) 2020-10-26 17:16:00 +01:00
mr_franklin
865e01a1a5 accouts, consensus, core: fix some comments (#21617)
# Conflicts:
#	core/state/state_object.go
2020-10-26 17:16:00 +01:00
gary rong
c53124d931 trie: polish commit function (#21692)
* trie: polish commit function

* trie: fix typo
# Conflicts:
#	trie/stacktrie.go
#	trie/trie_test.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
f488346129 core: fix txpool off-by-one error (#21683) 2020-10-26 17:16:00 +01:00
Marius van der Wijden
e40950ebd2 eth/downloader: cache parent hash instead of recomputing (#21678) 2020-10-26 17:16:00 +01:00
gary rong
99120e0c07 eth/downloader: fix data race around the ancientlimit (#21681)
* eth/downloader: fix data race around the ancientlimit

* eth/downloader: initialize the ancientlimit as 0
# Conflicts:
#	eth/downloader/downloader.go
2020-10-26 17:16:00 +01:00
Guillaume Ballet
6ae9473675 core/vm: dedup config check in markdown logger (#21655)
* core/vm: dedup config check

* review feedback: reuse buffer
# Conflicts:
#	core/vm/logger.go
2020-10-26 17:16:00 +01:00
Felix Lange
121fe99e7e p2p/discover: remove use of shared hash instance for key derivation (#21673)
For some reason, using the shared hash causes a cryptographic incompatibility
when using Go 1.15. I noticed this during the development of Discovery v5.1
when I added test vector verification.

The go library commit that broke this is golang/go@97240d5, but the
way we used HKDF is slightly dodgy anyway and it's not a regression.
2020-10-26 17:16:00 +01:00
Martin Holst Swende
350db1cfa6 signer/core: don't mismatch reject and no accounts (#21677)
* signer/core: don't mismatch reject and zero accounts, fixes #21674

* signer/core: docs
2020-10-26 17:16:00 +01:00
Felix Lange
e18b9ef673 node: relax websocket connection header check (#21646)
This makes it accept the "upgrade,keep-alive" header value, which
apparently is a thing.
2020-10-26 17:16:00 +01:00
rene
d25a8f7282 cmd/devp2p/internal/ethtest: improve eth test suite (#21615)
This fixes issues with the protocol handshake and status exchange
and adds support for responding to GetBlockHeaders requests.
# Conflicts:
#	cmd/devp2p/internal/ethtest/suite.go
#	cmd/devp2p/internal/ethtest/types.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
96d0e0ff9c core/types: optimize bloom filters (#21624)
* core/types: tests for bloom

* core/types: refactored bloom filter for receipts, added tests

core/types: replaced old bloom implementation

core/types: change interface of bloom add+test

* core/types: refactor bloom

* core/types: minor tweak on LogsBloom

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
# Conflicts:
#	core/types/bloom9.go
#	core/types/bloom9_test.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
0338641873 core/bloombits: faster generator (#21625)
* core/bloombits: add benchmark

* core/bloombits: optimize inserts
2020-10-26 17:16:00 +01:00
Raw Pong Ghmoa
1e869888ec params: update goerli testnet bootnodes (#21659)
* params: update pegasys besu bootnode

* params: update goerli initiative bootnodes
2020-10-26 17:16:00 +01:00
Felix Lange
adb60a3a71 cmd/faucet: enable DNS discovery for known networks (#21636)
# Conflicts:
#	cmd/faucet/faucet.go
#	cmd/utils/flags.go
2020-10-26 17:16:00 +01:00
Péter Szilágyi
f399aa1456 core/state/snapshot: stop generator if it hits missing trie nodes (#21649)
* core/state/snapshot: exit Geth if generator hits missing trie nodes

* core/state/snapshot: error instead of hard die on generator fault

* core/state/snapshot: don't enable logging on the tests
# Conflicts:
#	core/state/snapshot/generate.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
a7f3f55434 trie: add Commit-sequence tests for stacktrie commit (#21643)
# Conflicts:
#	trie/trie_test.go
2020-10-26 17:16:00 +01:00
gary rong
d6a69fd294 trie: polishes to trie committer (#21351)
* trie: update tests to check commit integrity

* trie: polish committer

* trie: fix typo

* trie: remove hasvalue notion

According to the benchmarks, type assertion between the pointer and
interface is extremely fast.

BenchmarkIntmethod-12           1000000000               1.91 ns/op
BenchmarkInterface-12           1000000000               2.13 ns/op
BenchmarkTypeSwitch-12          1000000000               1.81 ns/op
BenchmarkTypeAssertion-12       2000000000               1.78 ns/op

So the overhead for asserting whether the shortnode has "valuenode"
child is super tiny. No necessary to have another field.

* trie: linter nitpicks

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	trie/committer.go
#	trie/hasher.go
#	trie/trie.go
#	trie/trie_test.go
2020-10-26 17:16:00 +01:00
Martin Holst Swende
328becb968 accounts, signer: implement gnosis safe support (#21593)
* accounts, signer: implement gnosis safe support

* common/math: add type for marshalling big to dec

* accounts, signer: properly sign gnosis requests

* signer, clef: implement account_signGnosisTx

* signer: fix auditlog print, change rpc-name (signGnosisTx to signGnosisSafeTx)

* signer: pass validation-messages/warnings to the UI for gnonsis-safe txs

* signer/core: minor change to validationmessages of typed data
2020-10-26 17:16:00 +01:00
Guillaume Ballet
ee0da1e9e1 trie: use stacktrie for Derivesha operation (#21407)
core/types: use stacktrie for derivesha

trie: add stacktrie file

trie: fix linter

core/types: use stacktrie for derivesha

rebased: adapt stacktrie to the newer version of DeriveSha

Co-authored-by: Martin Holst Swende <martin@swende.se>

More linter fixes

review feedback: no key offset for nodes converted to hashes

trie: use EncodeRLP for full nodes

core/types: insert txs in order in derivesha

trie: tests for derivesha with stacktrie

trie: make stacktrie use pooled hashers

trie: make stacktrie reuse tmp slice space

trie: minor polishes on stacktrie

trie/stacktrie: less rlp dancing

core/types: explain the contorsions in DeriveSha

ci: fix goimport errors

trie: clear mem on subtrie hashing

squashme: linter fix

stracktrie: use pooling, less allocs (#3)

trie: in-place hex prefix, reduce allocs and add rawNode.EncodeRLP

Reintroduce the `[]node` method, add the missing `EncodeRLP` implementation for `rawNode` and calculate the hex prefix in place.

Co-authored-by: Martin Holst Swende <martin@swende.se>

Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
#	core/block_validator.go
#	core/types/derive_sha.go
#	eth/downloader/queue.go
#	eth/handler.go
#	turbo/trie/database.go
#	turbo/trie/encoding.go
#	turbo/trie/encoding_test.go
#	turbo/trie/stacktrie.go
#	turbo/trie/stacktrie_test.go
2020-10-26 17:16:00 +01:00
mr_franklin
bcf31729f7 cmd/bootnode,internal/debug: fix some comments (#21623) 2020-10-26 17:16:00 +01:00
Marius van der Wijden
63c40d94b8 build: keep geth-sources.jar build result for JavaDoc (#21596)
* ci: tooltips for javadoc for mobile app

* f space
# Conflicts:
#	Makefile
#	build/ci.go
2020-10-26 17:16:00 +01:00
mr_franklin
d48b34b7c0 p2p/enode: remove unused code (#21612) 2020-10-26 17:16:00 +01:00
shigeyuki azuchi
6e46109a6c light: fix wrong description in a comment (#21573)
# Conflicts:
#	light/odr.go
2020-10-26 17:16:00 +01:00
mr_franklin
141b59de6d internal/web3ext: improve eth_getBlockByNumber and eth_getBlockByHash console api (#21608) 2020-10-26 17:16:00 +01:00
aaronbuchwald
527155fc34 core: free pointer from slice after popping element from price heap (#21572)
* Fix potential memory leak in price heap

* core: nil free pointer slice (alternative version)

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-10-26 17:16:00 +01:00
Martin Holst Swende
618f9bedfa core/vm, params: make 2200 in line with spec (#21605)
# Conflicts:
#	core/vm/gas_table.go
2020-10-26 17:16:00 +01:00
Marius van der Wijden
117f3186c1 mobile: added constructor for big int (#21597)
* mobile: added constructor for big int

* mobile: tiny nitpick
# Conflicts:
#	mobile/big.go
2020-10-26 17:16:00 +01:00
Marius van der Wijden
3949c520a4 accounts/abi: ABI explicit difference between Unpack and UnpackIntoInterface (#21091)
* accounts/abi: refactored abi.Unpack

* accounts/abi/bind: fixed error

* accounts/abi/bind: modified template

* accounts/abi/bind: added ToStruct for conversion

* accounts/abi: reenabled tests

* accounts/abi: fixed tests

* accounts/abi: fixed tests for packing/unpacking

* accounts/abi: fixed tests

* accounts/abi: added more logic to ToStruct

* accounts/abi/bind: fixed template

* accounts/abi/bind: fixed ToStruct conversion

* accounts/abi/: removed unused code

* accounts/abi: updated template

* accounts/abi: refactored unused code

* contracts/checkpointoracle: updated contracts to sol ^0.6.0

* accounts/abi: refactored reflection logic

* accounts/abi: less code duplication in Unpack*

* accounts/abi: fixed rebasing bug

* fix a few typos in comments

* rebase on master

Co-authored-by: Guillaume Ballet <gballet@gmail.com>
# Conflicts:
#	accounts/abi/bind/base_test.go
#	accounts/abi/bind/bind_test.go
#	accounts/abi/pack_test.go
#	go.mod
#	mobile/bind.go
2020-10-26 17:16:00 +01:00
Igor Mandrigin
6dee4768b8 add to-merge.txt 2020-10-26 17:16:00 +01:00
alex.sharov
78e5bd37e4 try to use exclusive lock from prometheus 2020-10-26 19:08:36 +07:00
ledgerwatch
7554428884
POC of header downloader - splitting into 2 processes: sentry + downloader (via gRPC) (#1291)
* Splitting sentry and downloader - the beginning

* A bit more

* More on sentry

* More gRPC

* Sentry and downloader separated

* Update binding for stable version of grpc

* Better bufferSize flag

* Fix lint

* Send pelanties

* Fix lint

* Remove hard-coded tips on connect

* Tidy the logs a bit

* Deal with hardTips on Recovery

* Print hard tips

* Hide empty anchors

* Request headers after receiving a message

* Better waking up

* Print hard-coded block numbers

* Print outgoing requests

* Debug logging

* In the middle protection

* Sentry not to lose peers when core disconnects
2020-10-25 22:05:37 +00:00
Evgeny Danilenko
b6687fc9aa
initial (#1296) 2020-10-25 21:34:00 +00:00
ledgerwatch
dd2c047cdf
[WIP] Defrag utility for LMDB database (#1268)
* Initial

* Read freelist pages

* Fix

* Fix lint

* Fix lint

* Fix lint
2020-10-25 10:10:55 +00:00
Alex Sharov
6f27e3043b
lmdb_readonly_dont_create_folder (#1293) 2020-10-25 08:39:09 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271)
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-25 08:38:55 +00:00
Ricardo Guilherme Schmidt
e9572ae981
Fix infinite rpc.gascap on eth_estimateGas (#1292)
* Fix infinite rpc.gascap on eth_estimateGas

* use `gasCap.Sign() > 0` instead of `gasCap.Uint64() != 0`
2020-10-25 08:19:59 +00:00
Alex Sharov
c2f41d7723
stop on error: headers stage (#1289) 2020-10-24 18:05:12 +01:00
Thomas Jay Rush
1af3e09d75
1209 automate documentation (#1290)
* Automate documentation

* Updating

* Updating

* Fixing lint

* Updating readthedocs documentation

* Updating testing

* Updating testing
2020-10-24 18:03:52 +01:00