Commit Graph

640 Commits

Author SHA1 Message Date
battlmonstr
c6649f584b
p2p: refactor MaxPendingPeers handling (#3981)
* use semaphore instead of a chan struct{}
* move MaxPendingPeers default value to DefaultConfig.P2P
* log Error if Accept fails
* replace quit channel with context
2022-04-28 09:21:52 +07:00
battlmonstr
7558730a54
p2p: speed-up TestUDPv4_LookupIterator (#4000)
The test was slow, because it was trying to find
predefined nodeIDs (lookupTestnet) by generating random keys
and trying to find their neighbours
until it hits all nodes of the lookupTestnet.
In addition each FindNode response was waited for 0.5 sec (respTimeout).
This could take up to 30 sec and fail the test suite.

A fake random key generator is now used during the test.
It issues the expected keys, and the lookup converges quickly.
The reply timeout is reduced for the test.
Now it normally takes less than.1 sec.
2022-04-28 09:18:42 +07:00
battlmonstr
13d66198bc
p2p/discover/v4wire: use optional RLP field for EIP-868 seq (#3963)
This changes the definitions of Ping and Pong, adding an optional field
for the sequence number. This field was previously encoded/decoded using
the "tail" struct tag, but using "optional" is much nicer.

see https://github.com/ethereum/go-ethereum/pull/22842

Co-authored-by: Felix Lange <fjl@twurst.com>
2022-04-26 15:37:00 +01:00
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os (#3946) 2022-04-23 15:43:00 +01:00
battlmonstr
92db317e06
Observer - P2P network crawler (#3928)
Observer crawls the Ethereum network and collects information about the nodes.
2022-04-22 12:02:12 +01:00
ledgerwatch
f56d4c5881
Switch peerId from 256 to 512 bit (as in stable) (#3862)
* Switch peerId from 256 to 512 bit (as in stable)

* go mod tidy

* Fix some tests

* Fixed

* Fixes

* Fix tests

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-10 08:01:25 +01:00
Alex Sharov
4fc88fee59
docker on alpine 3.15 (#3844)
* save

* save
2022-04-08 15:53:40 +07:00
battlmonstr
930d662f21
Refactor crypto.FromECDSAPub to MarshalPubkey/Std (#3797)
Most places that used this method were cutting off the 1st byte.
Refactor this idea to a common place.

* better naming: MarshalPubkey matches existing UnmarshalPubkey
* "Std" suffix for the ANSI standard encoding without cut off
* docs
2022-03-31 11:06:20 +07:00
battlmonstr
564fc68996
Fix NodeKeyConfig.save() (#3720)
The logic was not saving the key file.
2022-03-16 16:58:56 +00:00
battlmonstr
5715a3f69b
Discovery: speed up lookup tests (#3677) 2022-03-11 12:04:42 +07:00
battlmonstr
04f07a0f08
Discovery: split node records to a sepatate DB table (#3581) (#3667)
Problem:
QuerySeeds will poke 150 random entries in the whole node DB and ignore hitting "field" entries.
In a bootstrap scenario it might hit hundreds of :lastping :lastpong entries,
and very few true "node record" entries.
After running for 15 minutes I've got totalEntryCount=1508 nodeRecordCount=114 entries.
There's a 1/16 chance of hitting a "node record" entry.
It means finding just about 10 nodes of 114 total on average from 150 attempts.

Solution:
Split "node record" entries to a separate table such that QuerySeeds doesn't do idle cycle hits.
2022-03-10 14:57:25 +01:00
battlmonstr
ca97325448
Discovery: throttle node DB commits (#3581) (#3656)
UpdateFindFails/UpdateLastPingReceived/UpdateLastPongReceived events
are causing bursty DB commits (100 per minute).

This optimization throttles the disk writes to happen at most once in a few seconds,
because this info doesn't need to be persisted immediately.

This helps on HDD drives.
2022-03-10 14:02:40 +01:00
battlmonstr
42d128e24b
Revert node DB cache (#3581) (#3674)
Revert "Prevent frequent commits to the node DB in sentries (#2505)".
This reverts commit 65a9a265c2.
2022-03-10 13:39:54 +01:00
battlmonstr
0242b5a4e2
Discovery: refactor public key to node ID conversions. (#3634)
Encode and hash logic was duplicated in multiple places.
* Move encoding to p2p/discover/v4wire
* Move hashing to p2p/enode/idscheme

* Change newRandomLookup to create a proper random key on a curve.
2022-03-03 21:58:31 +07:00
Alex Sharov
8ca388a7d3
less qcheck iterations (#3622) 2022-02-25 13:47:48 +07:00
battlmonstr
f5a0c9d834
p2p: Disable port mapping task with --nat extip (#3612)
If --nat extip:1.2.3.4 option is specified, the port mapping logic
(AddMapping/DeleteMapping) does nothing.
This optimization avoids running a goroutine for doing nothing.
2022-02-24 15:09:56 +00:00
battlmonstr
6fceb72a74
Refactor node private key setup. (#3593)
Extract private key setup from the global config setup to make it reusable.
Return errors instead of panicing.

* NodeKey() method is removed - it is unused.
2022-02-23 15:05:27 +00:00
battlmonstr
082194d538
Discovery: add Context to Listen. (#3577)
Add explicit Context to ListenV4 and ListenV5.
This makes it possible to stop listening by an external signal.
2022-02-22 18:18:43 +00:00
battlmonstr
4337871f7f
Rename log/logger to follow conventions. (#3579)
* use "log" for struct fields
* use "logger" for function parameters and local vars

This is a compromise between:
1) using logger := log.New() to avoid aliasing (log := log.New())
2) and keeping it short when logging e.g.: srv.log.Info(...)
2022-02-22 18:17:15 +00:00
Alex Sharov
01380316a5
close nodes db on server start error (#3424) 2022-02-04 13:04:37 +07:00
Dmitry Savonin
a49d409457
Full BSC support with validator mode (#3233)
* migrated consensus and chain config files for bsc support

* migrated more files from bsc

* fixed consensus crashing

* updated erigon lib for parlia snapshot prefix

* added staticpeers for bsc

* [+] added system contracts
[*] fixed bug with loading snapshot
[+] enabled gas bailout
[+] added fix to prevent syncing more than 1000 headers (for testing only)
[*] fixed bug with crashing sender recover sometimes

* migrated system contract calls

* [*] fixed bug with returning mutable balance object
[+] migrated lightclient contracts from bsc
[*] fixed parlia consensus config param

* [*] fixed tendermint deps

* [+] added some logs

* [+] enabled bsc forks
[*] fixed syscalls from coinbase
[*] more logging

* Fix call sys contract gas calculation

* [*] fixed executing system transactions

* [*] enabled receipt hash, gas and bloom filter checks

* [-] removed some logging scripts
[*] set header checkpoint to 10 million blocks (for testing forks)

* [*] fixed bug with commiting dirty inter block state state after system transaction execution
[-] removed some extra logs and comments

* [+] added chapel and rialto testnet support

* [*] fixed chapel allocs

* [-] removed 6 mil block limit for headers sync

* Fix hardforks on chapel and other testnets

* [*] fixed header sync issue after merge

* [*] tiny code cleanup

* [-] removed some comments

* [*] increased mdbx map size to 4 TB

* [*] increased max chaindata size to 6 tb

* [*] bring more compatibility with origin erigon and some code cleanup

* [+] added support of validator mode for BSC chain

* [*] enable private key load for bsc, rialto and chapel chains

* [*] fixed running BSC validator node

* Fix the branch list

* [*] tiny fixes for linter

* [*] formatted imports for core and parlia packages

* [*] fixed import rules in other files

* Revert "[*] formatted imports for core and parlia packages"

This reverts commit c764b58b34fedc2b14d69458583ba0dad114f227.

* [*] changed import rules in more packages

* [*] fixed type mismatch in hack command

* [*] fixed crash on new epoch, enabled bootstrap flags

* [*] fixed linter errors

* [*] fixed missing err check for syscalls

* [*] now BSC implementation is fully compatible with erigon original sources

* Revert "Add chain config and CLI changes for Binance Smart Chain support (#3131)"

This reverts commit 3d048b7f1a.

* Revert "Add Parlia consensus engine for Binance Smart Chain support (#3086)"

This reverts commit ee99f17fbe.

* [*] fixed several issues after merge

* [*] fixed integration compilation

* Revert "Fix the branch list"

This reverts commit 8150ca57e5f2707a84a9f6a1c5b809b7cc84547b.

* [-] removed receipt repair migration

* [*] fixed parlia fork numbers output

* [*] bring more devel compatibility, fixed bsc address list for access list calculation

* [*] fixed bug with commiting state transition for bad blocks in BSC

* [*] fixed bsc changes apply for integration command and updated config print for parlia

* [*] fixed bug with applying bsc forks for chapel and rialto testnet chains
[*] let's use finalize and assemble for mining to  let consensus know for what it's finalizing block

* Fix compilation errors in hack.go

* Fix lint

* reset changes in erigon-snapshots to devel

* Remove unrelated changes

* Fix embed

* Remove more unrelated changes

* Remove more unrelated changes

* Restore clique and aura miner config

* Refactor interfaces not to use slice pointers

* Refactor parlia functions to return tx and receipt instead of dealing with slices

* Fix for header panic

* Fix lint, restore system contract addresses

* Remove more unrelated changes, unify GatherForks

Co-authored-by: Dmitry Ivanov <convexman18@gmail.com>
Co-authored-by: j75689 <j75689@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 19:06:35 +00:00
Alex Sharov
178fd1931d
Genesis sync from existing snapshots (#3087) 2021-12-05 09:03:08 +07:00
TBC Dev
36fabdb69c
Change eth.Peer.ID() type from string to enode.ID (#3063)
* Change eth.Peer.ID() type from string to enode.ID

* Misc enode.ID.String() usage
2021-11-30 22:55:51 +00:00
ledgerwatch
14f81db038
Not try to create temporary file with enode (#3035)
* Update server.go

* Update server.go
2021-11-25 18:36:15 +00:00
TBC Dev
e1c44cd19b
Change sentry peer_id from H512 pubkey to H256 keccak256(pubkey) (#3013)
* Rename protoHandshake.ID to protoHandshake.Pubkey

* Fix enode.ID comment descriptions

* Change sentry peer_id from H512 pubkey to H256 keccak256(pubkey)

* Simplify PeerInfo helpers
2021-11-22 05:39:31 +00:00
Dmitry Savelev
77a31a8b36
Fix Sentry server initialization (#3008) 2021-11-21 19:36:26 +00:00
Alex Sharov
5b634a790e
Canonical tx ids (#2986)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2021-11-18 14:07:55 +00:00
Alex Sharov
3d3a1d30cc
Remove Docker from deps (#2930) 2021-11-08 21:11:57 +07:00
Alex Sharov
70461e4e68
Recsplit: add first byte of hash to txn payload (#2902) 2021-11-02 14:35:09 +07:00
alex.sharov
d296617b2e up enode db limit 2021-10-27 08:19:25 +07:00
Alex Sharov
da00e949c4
RemoteRPC: coherentCache for kv.Code; LocalRPC: enable small blocksLRU (#2815) 2021-10-12 12:04:04 +07:00
alex.sharov
161f59bb7d less warnings 2021-10-07 14:50:50 +07:00
Alexandr Borodulin
6336f9ab1d
Ignore MaxPeers param for staticpeers (#2789)
Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
2021-10-07 08:33:13 +07:00
Andrea Lanfranchi
3bc3be4b5f
Clean up DEBUG category logs (#2776)
- move many DEBUG logs into TRACE category
2021-10-05 08:14:04 +07:00
Andrea Lanfranchi
c913f35c2e
Inner errors (#2774) 2021-10-04 22:16:52 +07:00
Alex Sharov
400c71b7ce
Pool: txID-based cache (#2697) 2021-09-17 10:31:20 +07:00
Alex Sharov
8b22477fd5
Enode quit race (#2673) 2021-09-13 13:07:34 +07:00
Alex Sharov
a4c14c5f4c
increase nodedb limit (#2579) 2021-08-25 09:06:58 +07:00
Alex Sharov
e7574a6d14
RPC: batch - preserve order, streaming to in-mem buf (#2541)
* preserve order in batch

* fix batch order

* base fee in header json

* less logs in tests

* less logs in tests

* save

* save
2021-08-19 09:26:06 +07:00
Alex Sharov
d5b9abab7d
reduce log level in tests (#2546)
* hack

* hack

* hack

* less debug logs in tests

* less debug logs in tests
2021-08-18 14:25:03 +07:00
ledgerwatch
39e066a121
Fix deadlock in nodedb (#2524)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-13 11:16:09 +01:00
ledgerwatch
65a9a265c2
Prevent frequent commits to the node DB in sentries (#2505)
* Prevent frequent commits to the node DB in sentries

* Commit when btree goes over limit

* iterator for SeedQuery

* Fixing test

* Fix tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-09 09:52:42 +01:00
Alex Sharov
e17e7b6656
enable http body close linter (#2479) 2021-08-02 14:52:54 +07:00
Alex Sharov
286572e99f
Remove some unused code related to metrics (#2469) 2021-07-29 22:37:48 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
69dc2776dc
MIT licensed metrics lib (#2462) 2021-07-29 17:27:46 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
054581f507
staged syncs automatically add error prefix about stage number and name (#2445) 2021-07-26 13:19:01 +07:00