Commit Graph

252 Commits

Author SHA1 Message Date
Andrew Ashikhmin
0fdd60a0d1
Execution Spec Tests & HashCheck() (#6444)
Run tests from a new repo
https://github.com/ethereum/execution-spec-tests.

Also introduce `HashCheck()` function that checks correctness of block's
uncle, transaction, and withdrawals hashes.
2022-12-28 17:01:40 +01:00
Alex Sharov
dfa6505f93
e3: kv/temporal prototype (#6367) 2022-12-19 15:38:54 +07:00
Andrew Ashikhmin
0fc2022a0e
Extend fork ID to timestamp-based forks (#6324)
Starting from
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md),
forks are based on timestamps rather than block heights (see PR #6238).
This PR extends [EIP-2124](https://eips.ethereum.org/EIPS/eip-2124) Fork
ID to include timestamp-based blocks. See also
https://github.com/ethereum/go-ethereum/pull/25878.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-12-16 12:08:54 +01:00
ledgerwatch
0104698033
Not kick static and trusted peers due to empty responses (#6325)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-15 12:25:35 +00:00
ledgerwatch
71fb4ccc8e
Workaround for gnosis chain POS downloader (#6315)
Workaround for https://github.com/ledgerwatch/erigon/issues/6308

Very inefficient, because it get 1 single header from the request of 192
headers, so moves quite slowly

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-15 11:13:28 +00:00
ledgerwatch
47692bb334
Prevent empty responses to skeleton requests (#6305)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-14 12:02:23 +00:00
ledgerwatch
f03d29293c
Kick useless peers instead of putting them on stand-by (#6296)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-13 22:21:10 +00:00
ledgerwatch
5616ec2529
Disable validator set check in historical due to BEP-131, restore Ske… (#6290)
…letons

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-12 20:46:20 +00:00
awskii
a124bcf6ac
added --p2p.allowed-port flag (#6263)
Regarding https://github.com/ledgerwatch/erigon/issues/6260

added flag `--p2p.allowed-ports=<porta>,<portb>` to restrict which ports
to use for sentries for different protocol versions.

Default for this flag is `30303, 30304` (first port is inherited from
`--port` flag defaults.
If `--port` is changed and it's new value is not presented in allowed
port list, provided port will be allowed as well as list provided via
`--p2p.allowed-ports`

Port picking is straightforward, we create sentry gRPC server for
protocol over first allowed port that is not already taken.
If there are no allowed ports left, erigon exits with hint.
2022-12-12 13:25:47 +00:00
ledgerwatch
1c3c486d00
Avoid using peers returning empty responses, not sending skeleton requests (#6281)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-11 23:03:10 +00:00
ledgerwatch
4ab76849d8
Use old method of peer selection for non-POS networks (#6279)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-11 15:14:49 +00:00
ledgerwatch
cf237138d7
Kludge for BSC eth/67 errors (unknown message code 11) (#6248)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-08 08:01:55 +00:00
Håvard Anda Estensen
31ec791d5a
Avoiding leaking the popped item (#6193)
Set the deleted item to its zero val so it can be garbage collected.
2022-12-04 11:17:02 +07:00
Alex Sharov
9bcd7fe6c3
e3: reduce state write lock (#6096) 2022-11-21 10:38:22 +07:00
Alex Sharov
92e6e56120
Fix test e3 (#6087) 2022-11-20 10:58:20 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
Nathan (Blaise) Bruer
ca89620e56
SendMessageToRandomPeers() now is truely pseudo-random (#6050)
Fixes an issue where if the peer count didn't change much, we'd send the
message to the same peers nearly every time.

We now pick new random peers on every call to
SendMessageToRandomPeers().
2022-11-18 02:44:43 +03:00
Alex Sharov
66e1dc2dfd
better format p2p error (#6066) 2022-11-16 19:46:25 +03:00
Andrew Ashikhmin
14c0643476
Enable both eth/66 and eth/67 by default (#6048) 2022-11-15 10:41:56 +01:00
Andrew Ashikhmin
5b3f2b8c5f
Update interfaces (#6031)
Pick up
813d816908
& https://github.com/ledgerwatch/interfaces/pull/131.
2022-11-11 17:16:39 +01:00
ledgerwatch
7f12b809ec
Improvements to the downloader (#6012)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-10 10:33:10 +00:00
ledgerwatch
5b44f7d63b
Rework the peer finding (#5999)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-08 14:16:17 +00:00
ledgerwatch
61559477a6
fixes for POS header downloader (#5991)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-07 16:13:59 +00:00
Max Revitt
345e668832
feat(pos download): send header request to multiple peers (#5948) 2022-11-04 09:07:46 +00:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
606ce5c99a
add logging cli flags to various cmd (#5816) 2022-10-21 12:36:17 +07:00
Max Revitt
07ffa36d44
File system logging (#5812)
- lives in internal/logging
- all log flags moved to internal/logging/flags
- allows continued use of root logger via log.Info etc.
- update logger to take change allowing string to lvl for 'trace'

Verbosity flag is overridden by log.console.verbosity. Logs will be
colocated if all run as one process, only split where progs are run as
separate processes, in a future update this will be addressed so for
example rpcdeamon will always log to it's own file
2022-10-20 19:25:06 +01:00
ledgerwatch
a170d730c0
Fixing issues related to sync delays (#5689)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-11 08:04:40 +01:00
ledgerwatch
7018c5e746
Ergonomic fixes for POS (#5574)
* Various POS fixes

* More quiet

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-29 22:17:16 +01:00
Alex Sharov
cb60382e6d
erigon3: rename "history.v2" to "history.v3" to avoid naming miss-match with "erigon3" (#5519) 2022-09-26 10:54:42 +07:00
Alex Sharov
caa23b9532
erigon22: simplify interhashes stage (#5315) 2022-09-08 14:02:50 +07:00
ledgerwatch
da90c9752e
Fix for goerli resync (#5269)
* Fix for goerli resync

* Fix

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-04 14:51:36 +01:00
Michelangelo Riccobene
590276d3e1
Print peer-id using hex encoding to avoid unprintable characters (#5248) 2022-09-01 15:04:49 +01:00
Max Revitt
4c4baf4bbf
tweak(sentry): downgrade inbound message handling err to debug (#5215) 2022-08-29 19:03:53 +07:00
Alex Sharov
d33096eaa7
erigon22: tests support (#5196) 2022-08-26 16:54:54 +07:00
Andrew Ashikhmin
62bb2c47f7
Disable PoW sync for networks that have already passed through the Merge (#5172)
* Disable PoW sync for networks that have already passed through the Merge

* Fix FirstPoSHeight
2022-08-26 09:16:39 +02:00
Håvard Anda Estensen
ea6b92b132
Enable gosec linter (#5083)
* Ignore pseudo random generators

* Enable gosec linter
2022-08-17 12:18:19 +07:00
Håvard Anda Estensen
8460fd8e25
Remove deprecated functions (#5086) 2022-08-17 12:17:32 +07:00
Alex Sharov
778792e20e
Sentry doc (#5040)
* save

* save

* save
2022-08-13 21:37:51 +07:00
Giulio rebuffo
8ade569cc5
Save PoW blocks for processing received through P2P (#5015)
* save terminal blocks

* ops

* save less PoW blocks

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-08-12 21:54:20 +02:00
Max Revitt
7a87da9497
fix(sentry): add locks in sentry grpc (#4981) (#5025) 2022-08-12 21:18:43 +07:00
Giulio rebuffo
348be13508
fixed gossip post TTD (#5012)
* gossip all gossip

* gossip all gossip

* fixed tests

* fixed gossip post TTD

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-08-11 19:55:59 +02:00
Giulio rebuffo
b09de9e0d5
[P2P] gossip all gossip (#4995)
* gossip all gossip

* gossip all gossip

* fixed tests

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-08-10 22:08:56 +02:00
Andrew Ashikhmin
ea6305339b
Revert "Log data if cannot decode BlockBodiesPacket66 (#4850)" (#4929)
This reverts commit 7ae99884af.
2022-08-04 12:01:36 +02:00
Alex Sharov
454cea4811
docs cli antipattern (#4859)
* save

* save
2022-07-29 09:48:24 +07:00
Andrew Ashikhmin
7ae99884af
Log data if cannot decode BlockBodiesPacket66 (#4850) 2022-07-28 17:26:34 +02:00
hexoscott
75151e0700
fix memory leak in HandleInboundMessage (#4848) 2022-07-28 12:19:34 +01:00
Artem Tsebrovskiy
a1777accd8
fixed passing of raw byte slices to tx processing (#4782) 2022-07-22 15:47:33 +07:00
Andrew Ashikhmin
eec5fa4d41
Add support for eth/67 (#4564)
* Add eth/67

* Listen to eth/66 on a separate port

* Fix compilation error

* Fix cfg66.ListenAddr

* Update erigon ports in README

* Expose port 30304 in docker

* P2pProtocolVersionFlag instead of second sentry

* Remove "66 by default" from usage

* Small comment
2022-07-08 11:14:16 +02:00
Andrew Ashikhmin
dc5d3ffaac
Revert "Switch from eth/66 to eth/67 (#4549)" (#4562)
This reverts commit 16e57aa8a2.
2022-06-28 13:42:35 +02:00
Andrew Ashikhmin
16e57aa8a2
Switch from eth/66 to eth/67 (#4549)
* Switch from eth/66 to eth/67

* Fix a compilation error

* Update erigon-lib

* Refresh erigon-lib
2022-06-28 10:12:48 +02:00
Alex Sharov
359ae889b8
don't crush on p2p message handling panic (#4557)
* save

* save
2022-06-28 10:19:57 +06:00
ledgerwatch
b1572a15d5
No reorgs when fork choice head points to a canonical header (#4466)
* No reorgs when fork choice head points to a canonical header

* Return immediately

* Fix sentry

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-16 16:07:09 +01:00
ledgerwatch
df3eea6414
Not send sentry task to closed tasks channel (#4467)
* Not send sentry task to closed tasks channel

* Add comments

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-16 11:17:53 +01:00
ledgerwatch
1cc5b92dd8
Another fix for stuck header download (#4433)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-11 08:13:40 +01:00
ledgerwatch
43f5f0ae27
Close peerInfo tasks channel (#4429)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-10 11:59:31 +01:00
Alex Sharov
e146b66e35
more usage of dirs object #4390 2022-06-07 11:54:04 +07:00
Alex Sharov
7c79d6d453
Cfg dirs (#4388) 2022-06-07 11:00:37 +07:00
Alex Sharov
a53642b4bf
datadir.Dirs configuration object to group dir config (#4387) 2022-06-07 10:24:50 +07:00
ledgerwatch
b2f9b25300
For for overwritten anchor (#4379)
* For for overwritten anchor

* Switch to time types, log retry time in diagnostics

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-06 18:23:41 +01:00
ledgerwatch
4a5f97527b
Better diagnostic for stuck headers (#4372)
* More stuck headers diagnostics

* Temp print

* Remove temp print

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 15:00:13 +01:00
battlmonstr
0c5d1d64a3
observer: sentry candidates intake (#4321)
* sentry client: log connected peer info

* observer: unseen sentry peers report

* observer: refactoring node.go to node_utils

* observer: sentry candidates intake
2022-06-01 22:48:24 +01:00
Alex Sharov
955c669d21
Group few sync configs (#4303)
* save

* save
2022-05-30 17:08:49 +07:00
ledgerwatch
974e01bcd3
Allow skeleton requests to more peers (#4290)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-05-28 13:38:22 +01:00
battlmonstr
da45be9319
sentry: log peer events in erigon (#4275)
* sentry: log peer events in erigon

* refactor pumpStreamLoop to support streams with arbitrary message types
* subscribe to PeerEvents

* fixup! use generics
2022-05-27 08:33:00 +07:00
Andrew Ashikhmin
86de64a2a3
TangerineWhistle instead of EIP150 & SpuriousDragon instead of EIP155/158 (#4273)
* EIP150Block -> TangerineWhistleBlock

* Merge EIP158Block into EIP155Block

* EIP155Block -> SpuriousDragonBlock
2022-05-26 12:08:59 +02:00
battlmonstr
8a47a7ce33
sentry: PeerById method (#4260) 2022-05-26 10:53:23 +07:00
battlmonstr
6b1f91407d
sentry: multi sentry client refactoring (#4265) 2022-05-26 10:45:35 +07:00
Alex Sharov
fc1a37e6dc
create turbo/services pkg (#4269) 2022-05-26 10:31:06 +07:00
Alex Sharov
52d6f1d551
kv_mdbx: atomic closed flag #464 (#4247) 2022-05-24 12:24:20 +07:00
ledgerwatch
6974a98aca
Clean anchors, forward sort of headers (#4225)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-21 18:00:26 +01:00
battlmonstr
d845eb2a69
sentry: refactor eth handshake checks (#4163) 2022-05-16 20:28:03 +07:00
ledgerwatch
7125593cf5
Fix nil deference in ProcessHeadersPOS (#4155)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-05-15 17:09:27 +01:00
ledgerwatch
e02c77bf5a
Further simplify header download, req/resp metrics (#4121)
* Further simplify header download, req/resp metrics

* Not break the loop

* Move anchors correctly

* Log anchor state

* Print minNum

* Print hash

* Always add links

* Remove obsolete test

* cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alex Sharp <alexsharp@alexs-macbook-pro.home>
2022-05-13 11:18:30 +01:00
Alex Sharov
4e0f3c8c4c
Sentry: batch events (#4140) 2022-05-13 11:00:44 +07:00
Alex Sharov
691ceb1c3d
save (#4126) 2022-05-12 10:31:45 +07:00
ledgerwatch
8001cf2ad6
Simplify header downloader (#4104)
* Simplify header downloader

* Remove VerifyQueue

* Fix

* More fixes

* Fix

* Break out of the loop

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-10 08:06:16 +01:00
Alex Sharov
2c26583f6f
Rename sentry.ControlServerImpl to sentry.MultyClient and sentry.SentryServerImpl to sentry.GrpcServer #444 2022-05-10 12:17:44 +07:00
ledgerwatch
ba987cf262
Remove preverified hashes (#4088)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-06 15:15:01 +01:00
Giulio rebuffo
6927ffc54b
penalize naughty peers on PoS (#4060)
* penalize naughty peers

* lint
2022-05-04 10:39:34 +02:00
Andrew Ashikhmin
6c9d515968
No NewBlock gossip after Merge (#3995) 2022-04-27 14:03:06 +02:00
battlmonstr
07367c2312
sentry: refactor flags, add maxpeers. (#3961) 2022-04-25 21:47:25 +07:00
battlmonstr
fc3eecae5e
RPC: admin.peers() (#3960)
* RPC: admin.peers()

This RPC method returns information about the connected remote nodes.
https://geth.ethereum.org/docs/rpc/ns-admin#admin_peers

The peers are collected from all configured sentries.
See: https://github.com/ledgerwatch/interfaces/pull/102

Test with:
curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc": "2.0", "method": "admin_peers", "params": [], "id":1}' localhost:8545

* save

* liner fix

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-25 14:40:04 +01:00
battlmonstr
3ad25c929c
Sentry GRPC: rename Peers to PeerEvents (#3944)
* Sentry GRPC: rename Peers to PeerEvents

see https://github.com/ledgerwatch/interfaces/pull/101

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-23 17:52:22 +01:00
Alex Sharov
89c3049244
limit amount of sentries in broadcasts (#3920)
* save

* save
2022-04-20 22:00:58 +07:00
ledgerwatch
f56d4c5881
Switch peerId from 256 to 512 bit (as in stable) (#3862)
* Switch peerId from 256 to 512 bit (as in stable)

* go mod tidy

* Fix some tests

* Fixed

* Fixes

* Fix tests

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-10 08:01:25 +01:00
Andrew Ashikhmin
c2f1a6082f
Fix runPeer lint (#3847) 2022-04-07 13:47:28 +02:00
Alex Sharov
6f7a9dbace
Snapshots: nat support (#3834) 2022-04-06 15:25:32 +07:00
Alex Sharov
b7afdf4a83
Single root ctx (#3796) 2022-03-30 16:51:43 +07:00
Andrew Ashikhmin
816ad30eff
Small Merge-related clarifications & improvements (#3785)
* Hd.Fetching -> FetchingNew

* ReadHeadHeaderHash makes more sense in verifyAndSaveNewPoSHeader

* Potentially more correct LatestValidHash in case of payload with invalid transactions
2022-03-29 11:47:07 +02:00
Giulio rebuffo
b6f707d811
fixed bootstrap nodes bug (#3688)
* fix transition warning

* fixed commit genesis

* fixed bootstrap nodes bug

* Update genesis.go
2022-03-14 09:47:56 +01:00
battlmonstr
47df98a499
Use "err" key for logging errors. (#3632)
log.Warn/Error uses "err" key to log errors in most places.
This renames "error" to "err" in some places to adhere to this convention.
2022-03-01 15:40:51 +00:00
Alex Sharov
0cac29d1d2
Update README.md 2022-02-26 19:55:17 +07:00
battlmonstr
47edcd1b6b
Refactor discovery bootnodes parsing. (#3596)
Simplify node URL list logic.
Handle errors.
2022-02-23 20:24:13 +00:00
battlmonstr
082194d538
Discovery: add Context to Listen. (#3577)
Add explicit Context to ListenV4 and ListenV5.
This makes it possible to stop listening by an external signal.
2022-02-22 18:18:43 +00:00
Alex Sharov
20452c3dd6
close channel by producer (#3569) 2022-02-22 16:25:02 +07:00
Enrique Jose Avila Asapche
e348b3ea93
deleted kovan (#3546) 2022-02-19 08:15:44 +00:00
Alex Sharov
b60faba5d9
more stable grpc err chec (#3541) 2022-02-18 09:54:38 +07:00
Alex Sharov
226982d293
Use class dir.Rw - to separate Ro and Rw access to snapshotDir (#3534)
* save

* save

* save

* save

* save

* save

* save
2022-02-18 09:24:17 +07:00
Alex Sharov
e652b5fa87
snapshots: header stage fixes (#3518) 2022-02-16 08:38:54 +07:00