Commit Graph

16774 Commits

Author SHA1 Message Date
Alex Sharov
606ce5c99a
add logging cli flags to various cmd (#5816) 2022-10-21 12:36:17 +07:00
Levi Aul
e85d4b76aa
Return bor receipts correctly in getTransactionReceipt and getBlockReceipts (#5787)
Fixes a nil-pointer dereference in Decode when attempting to call
`getTransactionReceipt` / `getBlockReceipts` on blocks containing
stateSync events.
2022-10-20 19:25:46 +01:00
Max Revitt
07ffa36d44
File system logging (#5812)
- lives in internal/logging
- all log flags moved to internal/logging/flags
- allows continued use of root logger via log.Info etc.
- update logger to take change allowing string to lvl for 'trace'

Verbosity flag is overridden by log.console.verbosity. Logs will be
colocated if all run as one process, only split where progs are run as
separate processes, in a future update this will be addressed so for
example rpcdeamon will always log to it's own file
2022-10-20 19:25:06 +01:00
Giulio rebuffo
889164e847
Strenghtened safety system for LC (#5811) 2022-10-20 16:07:17 +02:00
Leonard Chinonso
4c376b06da
Tests for lightclient/sentinel (#5772) 2022-10-20 15:41:28 +02:00
Alex Sharov
5d5d14532d
mdbx_to_mdbx preserve destination (#5810) 2022-10-20 19:20:22 +07:00
Alex Sharov
9d2d51e0ae
e3: mainnet (#5809) 2022-10-20 19:07:25 +07:00
a
d697b6fe7c
large performance optimization for filterLogs (getLogs) when doing queries with many addresses and logs. (#5805)
this pr changes filterLogs to use a pre computed hashset of addresses,
instead of iterating across the list of addresses once per log.

this greatly increases the speed of filter queries that use many
addresses and also return a large number of logs. In our case, we are
performing a query for all the trades performed in a uniswap v3 pool in
a 250 block range.
 
my benchmarks were performed with the data & code below:
 
address list gist is here
[addrs](2c30b0df43/gistfile1.txt)

 ```
 c := NewRpcClient()
 addrs := []common.Address{AddressListGist}
logs, err := c.FilterLogs(context.TODO(), ethereum.FilterQuery{
	FromBlock:big.NewInt(15640000),
	ToBlock:   big.NewInt(15640250),
	Addresses: addrs,
	Topics: [][]common.Hash{
		{

common.HexToHash("c42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"),
		},
	},
```
the query contains 8442 addresses, while the response contains 1277 logs

On average, current devel averages a 15.57 second response time on my machine after 10 runs, while the new filterLogs averages 1.05 seconds.

for CURRENT DEVEL, the profile is here: https://pprof.aaaaa.news/cd8dkv0tidul37sctmi0/flamegraph 

for the filterLogs branch, the profile is here: https://pprof.aaaaa.news/cd8dlmgtidul37sctmig/flamegraph

while the tests pass with this branch, I am not really sure why filterLogs was originally programmed the way it was. Is there some sort of edge case / compatibility thing that I am missing with this change?

Co-authored-by: a <a@a.a>
2022-10-20 17:34:20 +07:00
Andrew Ashikhmin
af46c3c069
txpool: prevent index out of range (#5808)
Pick up https://github.com/ledgerwatch/erigon-lib/pull/690
2022-10-20 11:34:21 +02:00
Alex Sharov
c5d905fae2
e3: more snapshots (#5806) 2022-10-20 13:02:59 +07:00
Alex Sharov
6405425465
add fileName to decompressor panic (#5804) 2022-10-20 12:51:02 +07:00
Alex Sharov
0db6b9b44b
e3: move skip analysis out of the loop (#5803) 2022-10-20 10:39:40 +07:00
Andrew Ashikhmin
832e8e01fe
Remove verbose and partially redundant log (#5796)
This log message is mostly redundant after PR #5792.
2022-10-19 14:19:30 +02:00
Alex Sharov
c1313a1669
e3: Close recon db earlier to free ram (#5794) 2022-10-19 16:45:55 +07:00
Alex Sharov
bab0aaa24d
e3: close recon db earlier (#5793) 2022-10-19 16:44:06 +07:00
Andrew Ashikhmin
8ff674acc5
Fix potential nil pointer dereference in NewBlockBuilder (#5792)
This should fix the following error reported by stickx:

![PXL_20221017_004749004](https://user-images.githubusercontent.com/34320705/196652634-2abf82d0-71da-451f-a63a-e6e622f8d97a.jpeg)
2022-10-19 11:38:03 +02:00
Alex Sharov
965619ff83
e3: fix parallel exec deadlock (#5790) 2022-10-19 12:25:57 +07:00
Alex Sharov
2b5755c51a
grafana: more mdbx internals (#5789) 2022-10-19 12:07:31 +07:00
Alex Sharov
60270df639
up grpc and log deps (#5788) 2022-10-19 09:31:15 +07:00
Giulio rebuffo
39dd749db0
Added verbosity flag to lightclient (#5785)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-18 12:46:48 +02:00
Andrew Ashikhmin
0fb74172a8
FinalizeTx after engine.Initialize (#5783)
AuRa storage changes made by `onEpochBegin` should be finalized.
Otherwise, `SSTORE` cost might be incorrect in block transactions when
[EIP-2200](https://eips.ethereum.org/EIPS/eip-2200) is active.
https://gnosisscan.io/tx/0xa6d45d903c977ad237165737ebc4e34fb7756675719d112a199af858bb1275e0
is an example of a transaction on Gnosis Chain whose gas used was
previously calculated incorrectly.
2022-10-18 10:44:56 +02:00
Alex Sharov
52c418993e
estimate: don't rely on 100% of ram (#5784) 2022-10-18 15:38:14 +07:00
Alex Sharov
31b9d37cdc
estimate more ram for indexing (#5782) 2022-10-18 15:29:59 +07:00
Alex Sharov
14493d5a8d
e3: eth_getLogs fix (#5780) 2022-10-18 14:42:00 +07:00
Alex Sharov
dcf6216591
e3: tests (#5779) 2022-10-18 13:06:25 +07:00
Alex Sharov
3f0712027d
e3: nil writer fix (#5778) 2022-10-18 11:54:31 +07:00
Alex Sharov
c6faa9fca3
e3: tests (#5776) 2022-10-18 11:53:54 +07:00
Alex Sharov
b76aa8d681
e3: nil writer fix (#5777) 2022-10-18 11:39:27 +07:00
Alex Sharov
4f68f1ec15
e3: nil writer fix (#5775) 2022-10-18 11:09:34 +07:00
Alex Sharov
b909bfa581
e3 tests (#5774) 2022-10-18 11:08:14 +07:00
gfx
d1e9149c93
add more sync stages to metrics (#5740)
hi! 

this pr adds more sync stages to the prometheus metrics. node now shows 

```
sync{stage="account_history_index"} 15742138
sync{stage="block_hashes"} 15742138
sync{stage="bodies"} 15742138
sync{stage="call_traces"} 15742138
sync{stage="cumulative_index"} 15742138
sync{stage="execution"} 15742138
sync{stage="finish"} 15742138
sync{stage="hash_state"} 15742138
sync{stage="headers"} 15742138
sync{stage="intermediate_hashes"} 15742138
sync{stage="log_index"} 15742138
sync{stage="senders"} 15742138
sync{stage="snapshots"} 15742138
sync{stage="storage_history_index"} 15742138
sync{stage="translation"} 0
sync{stage="tx_lookup"} 15742138
sync{stage="verkle_trie"} 0
sync{stage="watch_the_burn"} 0
```

Co-authored-by: a <a@a.a>
2022-10-18 10:22:57 +07:00
Alex Sharov
3aa97c57c1
fix ram estimation for indexing (#5773) 2022-10-18 10:19:58 +07:00
juroberttyb
64fe6708fa
Correct failed link in README.md (#5762)
[RPC calls] link path (../../cmd/rpcdaemon/Readme.md) corrected to
(../../cmd/rpcdaemon/README.md)
2022-10-18 09:23:44 +07:00
Max Revitt
bd7b75173f
docs(examples): initial erigon single process example (#5767) 2022-10-18 09:23:13 +07:00
hexoscott
adadef62c8
remove estimated duration from stage execution (#5768) 2022-10-18 09:22:44 +07:00
Giulio rebuffo
147c1d0460
fixed messaage id (more efficient gossip, reduction in bandwidth). (#5770)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-17 19:13:23 +02:00
J1ang
2177c8faa5
fix: issue #5723 type conversion bug (#5763)
fix:
fix for issue #5723 type conversion bug by adding check. `expValue`
should be `uint64[]`.
2022-10-17 11:25:17 +01:00
Enrique Jose Avila Asapche
19f0776a20
Adding flags (#5759) 2022-10-16 19:54:12 +02:00
Giulio rebuffo
b5697214ff
Segmented dump files (#5765) 2022-10-16 18:48:58 +02:00
Giulio rebuffo
527c13fb90
fixed portion of CL verification for NextSyncCommittee (#5764) 2022-10-16 18:31:41 +02:00
Giulio rebuffo
ab3a0b0fbd
Switched to validating lightclient (#5761)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-16 00:07:27 +02:00
Giulio rebuffo
d075618ecc
Importing of ETH2 blocks from Lightclient (#5760) 2022-10-15 23:01:52 +02:00
Giulio rebuffo
6f1c2be4e5
Implemented clause 3 of phase 4 (#5758) 2022-10-15 19:03:31 +02:00
Giulio rebuffo
45f1c99b50
Commented out BLS validation because it breaks CI (#5755)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-15 12:38:37 +02:00
ledgerwatch
90d26a3f46
Fix lint (#5756)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 10:34:12 +01:00
Alex Sharov
b8dbb53d3b
e3: write history and indices to etl (#5742) 2022-10-15 08:20:58 +07:00
Alex Sharov
3be0fd7d19
e3: history mainnet,bsc snapshots (#5726) 2022-10-15 08:19:12 +07:00
ledgerwatch
108ab66d19
Bump version (#5753) 2022-10-15 00:12:48 +01:00
ledgerwatch
525a3a7cbb
Fix for empty blocks being proposed (#5749)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-15 00:12:38 +01:00
Giulio rebuffo
8e1b559ad3
implemented clauses 4 (i) and (ii) (#5752) 2022-10-15 00:14:04 +02:00