Commit Graph

16864 Commits

Author SHA1 Message Date
ledgerwatch
5deaa9cec1
Fix panic in txpool (due to modificatin with read lock) and snapshot … (#5931)
…access (due to inconsistent indices)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-04 12:43:59 +00:00
Alex Sharov
b1fc489024
e3: fix tests (#5961) 2022-11-04 16:14:58 +07:00
Max Revitt
345e668832
feat(pos download): send header request to multiple peers (#5948) 2022-11-04 09:07:46 +00:00
Alex Sharov
d4fc7c7ef0
e3: reduce write-lock of rws (#5959) 2022-11-04 15:09:29 +07:00
Alex Sharov
f3d8533879
e3: cheaper recon estimate (#5958) 2022-11-04 13:21:44 +07:00
Alex Sharov
bc2a873c9d
e3: disable txlookup stage (#5957) 2022-11-04 10:51:09 +07:00
Alex Sharov
655e32a199
e3: commit between blocks (#5955) 2022-11-04 09:50:35 +07:00
Alex Sharov
7516c33527
e3: idx size metric (#5954) 2022-11-04 09:31:30 +07:00
Alex Sharov
d4c406d3f0
e3: aggressive prune on seq exec (#5953) 2022-11-04 09:26:34 +07:00
Giulio rebuffo
63a8b21bb8
switch BLS library (#5950) 2022-11-03 23:55:59 +01:00
Mike Neuder
5518e69005
Finish BeaconBlocksByRange decoder (#5943)
issue: https://github.com/ledgerwatch/erigon/issues/5884

this PR finishes the decoding of BeaconBlocksByRange responses.
`stream_test.go` now checks two sample ranges, of length 2 and 3
respectively (avoiding doing super long ranges to keep the files
relatively smaller).

https://beaconcha.in/slot/5000007 is the source i used to verify the
decoded data.
2022-11-03 19:04:49 +01:00
Alex Sharov
d418069553
e3: fix retainList on unwind (#5947) 2022-11-03 15:12:53 +07:00
Willian Mitsuda
84ac1b919c
Use pooled bitmaps (#5946) 2022-11-03 12:50:16 +07:00
Willian Mitsuda
11f4978ed4
Upstream all Otterscan patches to devel (#5945)
Following our previous discussion on erigon's discord, this PR requests
to upstream all Otterscan modifications to erigon's repo.

That decision comes after getting feedback from lots of users at events
this year, and although it may introduce some friction for development,
it will make integrators life easier by having all our modifications
available out of box, e.g., dappnode users will get our RPCs since their
official packages are built from erigon repo.

I'm submitting the source-code as-is, please let me know if you think
there is a better code organization.

The current set of modifications comprises only new RPCs. There are some
proposals for extra-stages that would add new tables, but they are still
WIP and will be submitted separately in future after more testing.
2022-11-03 11:32:15 +07:00
Alex Sharov
32eaa82fba
e3: prune debug log (#5944) 2022-11-03 11:15:03 +07:00
Leonard Chinonso
805230ba63
Setup web socket endpoints to listen for new transactions after sending (#5940)
- Added listening methods for WebSocket subscriptions 
- Listened for new blocks using the newHeads method to determine when to
look for a transaction
- Added new util methods and tests for them
- Simplified communication to the user upon initiating the devnet tool
2022-11-03 09:45:36 +07:00
a
ad7be30679
jsonrpc HTTP Get handler & logs topic filter with map (#5922)
this pr has two things in it

1. changed filter logs to use a map for the topics. this will speed up
queries with many topics in them. I still don't have a use case for this
though. i put is as a method of Logs, since that made sense to me, happy
to move it back out though.

2. allows json-rpc over http get request. since firefox is a great json
viewer (can search through, collapse large results) i often use it to
debug. it is also useful for sharing data with those who are less
familiar with command line / programming.
example get request:

http://rpcdaemon/?method=eth_getLogs&params=[{"fromBlock":"0xf2316b","toBlock":"0xf2316b"}]

it is based on the old jsonrpc http specification
https://www.jsonrpc.org/historical/json-rpc-over-http.html#encoded-parameters
except we also accept not base64 encoded params. since every eth rpc
request has a [], it will immediately fail validation for base64 and
attempt to use the parameters as a they are. otherwise it will attempt
to parse the rest of the payload as base64 and use that.

Co-authored-by: a <a@a.a>
Co-authored-by: gfx <86091021+gfxlabs@users.noreply.github.com>
2022-11-03 09:09:04 +07:00
Giulio rebuffo
4840234ce9
Remove printf (#5941) 2022-11-02 23:48:34 +01:00
Mike Neuder
7590411ffc
BeaconBlocksByRange req/resp domain encoding work. (#5917)
I am trying to decode these messages from the req/resp domain. see
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#beaconblocksbyrange
for the spec.

this is part of https://github.com/ledgerwatch/erigon/issues/5884
2022-11-02 15:48:22 +01:00
Andrew Ashikhmin
7134d72738
Unify EVM Call flavours (#5939) 2022-11-02 15:04:37 +01:00
Giulio rebuffo
5c3245d4e0
Added Checkpoint sync for CL full client (#5934) 2022-11-02 13:12:44 +01:00
Alex Sharov
7208b0dc60
remove fmt (#5937) 2022-11-02 10:08:25 +07:00
Alex Sharov
96deee29bb
e3: build files sequentially, flush indices in-advance before commit (#5935) 2022-11-02 10:04:12 +07:00
hexoscott
c611c7489f
reduce estimated RAM from 80% to 50% (#5930) 2022-11-01 21:54:39 +07:00
Andrew Ashikhmin
d4c2acc152
AuRa system calls to Certifier should be const (#5929)
This fixes gas used in
https://gnosisscan.io/tx/0x8f3b839a3d179102084a58dc4c3ec91fe13bb00cc1450cd69d22add7f25444e3
2022-11-01 14:37:45 +01:00
Alex Sharov
58c8baf2ef
e3: parallel exec - spend time for prune instead of flush when much data in db (#5925) 2022-11-01 14:17:25 +07:00
Thomas Taylor
4310aa4032
etl-temp location changed so cleaning up docs (#5924)
It seems this got changed around so that `temp` is used but the docs
were not changed.
2022-11-01 13:19:28 +07:00
Alex Sharov
f19b4367ed
e3: seq exec (#5923) 2022-11-01 10:39:57 +07:00
Alex Sharov
e490d2d0c7
e3: background merge (#5921) 2022-11-01 09:55:38 +07:00
net.wyman
7548755b62
fix potential deadlock in eth_filters logs event (#5912)
Like the newPendingTransaction event, the logs event also has the
potential for deadlock when unsubscribed
2022-11-01 08:59:01 +07:00
Alex Sharov
8783521ba4
lint (#5920) 2022-11-01 08:57:05 +07:00
net.wyman
98420edd30
Update README.md (#5914)
the rpc_daemon README.md is out of date and miss the eth_subscribe logs
event
2022-11-01 08:56:22 +07:00
Andrew Ashikhmin
9ffc457cbb
EIP-3860: Limit and meter initcode (#5892)
[EIP-3860](https://eips.ethereum.org/EIPS/eip-3860) is
[included](https://github.com/ethereum/execution-specs/pull/633) into
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md).

TODO: similar changes to `txpool`.

Co-authored-by: Andrei Maiboroda <andrei@ethereum.org>
2022-10-31 13:40:41 +01:00
Leonard Chinonso
3163f40e58
Added Mechanism For Sending Raw Transaction (#5825)
- Added new requests for sending raw transactions
- Made provisions for other types of transactions generated, including
contracts
- Modified the models and services to accommodate general values
2022-10-31 17:46:49 +07:00
Alex Sharov
4ead27fad9
e3: add "make test3" to CI (#5910) 2022-10-31 14:25:52 +07:00
Alex Sharov
381bea5b92
e3: eth_traceTransaction implementation (#5909) 2022-10-31 12:31:38 +07:00
Alex Sharov
fb5ce32de9
e3: more mainnet snapshots (#5867) 2022-10-31 09:20:58 +07:00
Leonard Chinonso
bb780a06d3
Tested functionalities for request_generator (#5905)
- Added tests for request_generator in devnet
- Added tests for request_generator in rpctests
2022-10-30 19:54:09 +07:00
Giulio rebuffo
adf5fc7b82
refactored files regarding lightclient (#5904) 2022-10-29 21:51:32 +02:00
Alex Sharov
8c36c3fe99
e3: recon progress metric (#5902) 2022-10-29 21:51:01 +07:00
Alex Sharov
0e23ecda87
e3: more merge logs (#5900) 2022-10-29 21:13:12 +07:00
Alex Sharov
fde630f76b
e3: wal preallocate historyKey #713 (#5899) 2022-10-29 19:18:35 +07:00
Mike Neuder
dab1ba44d0
Test req/resp domain response decoding (#5886)
https://github.com/ledgerwatch/erigon/issues/5884 has more context on
this issue. we are trying to understand the encoding of the responses.

from the Ethereum R&D discord, AgeManning said that the prefix is
supposed to be a varint representing the length of the payload:
https://developers.google.com/protocol-buffers/docs/encoding#varints.

that works for the `&cltypes.MetadataV1{}` type, but not for
`&cltypes.MetadataV2{}` or `&cltypes.LightClientFinalityUpdate{}`.
2022-10-29 13:02:26 +02:00
Alex Sharov
ba02673e4d
e3: try move old data from db as early as possible #712 (#5897) 2022-10-29 14:58:34 +07:00
Alex Sharov
f766a490b5
e3: merge logs (#5896) 2022-10-29 14:44:11 +07:00
Alex Sharov
8fe89f4e5b
e3: less small merges, parallel merge (#5895) 2022-10-29 13:21:06 +07:00
Alex Sharov
d8d4a0a240
e3: wait group for apply loop (#5894) 2022-10-29 11:48:53 +07:00
Andrew Ashikhmin
dd43d486f3
Enable London in AllEthashProtocolChanges (#5891)
`AllEthashProtocolChanges` should contain all the EIPs accepted by the
core developers into the Ethash consensus, including the
[London](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md)
protocol upgrade.
2022-10-28 17:19:18 +02:00
Alex Sharov
0074ad0e48
e3: parallel exec, rotate and partial-flush of indices wal - while main thread is idle (#5889)
While main thread is idle (apply is on RoTx now):
- periodically still hisotry/index etl collectors and load them to db 
- prune old history/index in db

Also:
- fixed RAM estimation calculation of State22
2022-10-28 21:13:53 +07:00
Andrew Ashikhmin
f46300e37e
core/vm: Rename SHA3 instruction to KECCAK256 (#5890)
Cherry-pick https://github.com/ethereum/go-ethereum/pull/23976

Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>
2022-10-28 13:38:34 +02:00