Commit Graph

17924 Commits

Author SHA1 Message Date
Alex Sharov
f027324f01
torrent: suppress some warning (#7124) 2023-03-17 03:53:35 +00:00
Alex Sharov
3da0df736f
grpc_middleware_up (#7123) 2023-03-17 03:33:59 +00:00
Max Revitt
82217f5411
fix(release): download page https redirect (#7120)
Updated initial traefik config so the http -> https redirect on
download.erigon.sh will work.
2023-03-17 03:29:09 +00:00
awskii
7cee93aa10
E4 metrics upd (#7122) 2023-03-17 00:38:38 +00:00
hexoscott
3b36d5d57a
get localnode address up front on creation to save potential data race (#7111)
the log line here was the culprit for the race. made sense to just
capture this on localnode creation instead and hold onto it for when the
server is started.

ran test with `-race` and `-count=5000` to double check and all looks
good
2023-03-16 03:44:00 +00:00
Max Revitt
87c958b5c1
fix(release): run download page update separately (#7112)
Checksum not working as the file isn't available in the assets when run
in the same job.
2023-03-16 03:40:42 +00:00
Alex Sharov
59bc028762
txpool: senders batch commit optimize (#7118) 2023-03-16 03:39:58 +00:00
Alex Sharov
60fb9c12e6
e3: don't loose nil-value in reconstitution (#7117) 2023-03-16 03:27:09 +00:00
Alex Sharov
157a380be7
e3: history no auto-increment (#7097) 2023-03-15 08:03:57 +00:00
alex.sharov
ec76e0c5c1 e3: less merge logs 2023-03-15 13:35:04 +07:00
alex.sharov
f6759518d6 don't show "block number" in txpool logs it confusing users 2023-03-15 11:27:03 +07:00
hexoscott
165a6a950d
check for nil stream when running the null check in handler (#7105) 2023-03-15 04:03:16 +00:00
alex.sharov
94729b1e82 up goprotobuf 2023-03-15 10:26:44 +07:00
hexoscott
84ec0a0b4b
check for nil before returning invalid json in rpc streaming calls (#7104)
should handle nil having already been written in any rpc call before
writing it again causing invalid json to be returned.
2023-03-14 11:07:53 +00:00
Alex Sharov
2ba3b084d7
torrent: don't cancel storage, because lib can't handle such error and can graceful-shutdown anyway (#7102) 2023-03-14 08:42:39 +00:00
hexoscott
4f91dff773
fix for reading yaml/toml config in cmd/integration (#7101) 2023-03-14 08:33:59 +00:00
Alex Sharov
0ef5c76d89
torrent lib version up to remove some warning log (#7100) 2023-03-14 07:50:00 +00:00
Alex Sharov
bbe56620a3
move more parts to lru2 (#7098) 2023-03-14 07:37:23 +00:00
Victor Shyba
158fb2b606
Optimize memory buffer, simplify set32, use sha256-simd (#7060)
Hi,

I'm syncing Gnosis on a Celeron N5100 to get familiar with the codebase.
In the process I managed to optimize some things from profiling.
Since I'm not yet on the project Discord, I decided to open this PR as a
suggestion. This pass all tests here and gave me a nice boost for that
platform, although I didn't have time to benchmark it yet.

* reuse VM Memory objects with sync.Pool. It starts with 4k as `evmone`
[code
suggested](0897edb001/lib/evmone/execution_state.hpp (L49))
as a good value.
* set32 simplification: mostly cosmetic
* sha256-simd: Celeron has SHA instructions. We should probably do the
same for torrent later, but this already helped as it is very CPU bound
on such a low end processor. Maybe that helps ARM as well.
2023-03-14 07:17:04 +00:00
hexoscott
efd541028c
read metrics config from yaml file (#7089) 2023-03-14 00:07:05 +00:00
Jason Yellick
b21569ccc5
Backfill eth getproof tests (#7092)
This PR adds missing tests for eth_getProof and does some mild
refactoring with switching from strings to more strict types. It's
likely best/most easily reviewed commit by commit.

Note, the tests include quite a number of helper types and functions for
doing the proof validation. This is largely because unlike Geth,
Erigon's approach to trie computations only requires serializing the
trie nodes, not deserializing them. Consequently, it wasn't obvious how
to leverage the existing trie code for doing deserialization and proof
checks. I checked on Discord, but, there were no suggestions. Of course,
any feedback is welcome and I'd be happy to remove this code if it can
be avoided.

Additionally, I've opted to change the interface type for `GetProof` to
use a `common.Hash` for the storage keys instead of a `string`. I
_think_ this should be fairly safe, as until very recently it was
unimplemented. That being said, since it's an interface, it has the
potential to break other consumers, anyone who was generating mocks
against it etc. There's additionally a `GetStorageAt` that follows the
same parameter. I'd be happy to submit a PR modifying this one as well
if desired.

Also, as a small note, there is test code for checking storage proofs,
but, storage proofs aren't currently supported by the implementation. My
hope is to add storage proofs and historic proofs in a followup PR.

---------

Co-authored-by: Jason Yellick <jason@enya.ai>
2023-03-14 00:04:52 +00:00
Jason Yellick
4f6d76928a
Fix broken link in doc (#7094)
The doc in
[dupsort.md](https://github.com/ledgerwatch/erigon/blob/devel/docs/programmers_guide/dupsort.md#erigon)
points to a non-existent `AbstractKV.md`

As best as I can tell, the `AbstractKV.md` was reworked and renamed in
commit

0bc61c06ed

then, this was later moved into erigon-lib. This commit simply repoints
the doc at this new location.

Co-authored-by: Jason Yellick <jason@enya.ai>
2023-03-14 00:03:09 +00:00
Jason Yellick
aec1dcdeef
Delete retain_list_builder.go (#7096)
I was walking the code to try to understand in a bit more detail how the
state root hash is constructed and stumbled across
`retain_list_builder.go` as a consumer of the retain list APIs. But, as
far as I can tell, this file doesn't seem to actually be used anywhere
(including tests), and, it's seen no development (other than import
fixes) since 2020 or so. All linting and tests still pass for me locally
without it, so, I believe it's safe to simply remove.

Co-authored-by: Jason Yellick <jason@enya.ai>
2023-03-14 00:02:07 +00:00
Giulio rebuffo
bc546c72da
Added hard fork transition support to Erigon-CL. (#7088) 2023-03-13 12:10:36 +00:00
Alex Sharov
f2d99f9d85
e3: split "changed keys" iterator to simplify (#7086) 2023-03-13 08:06:50 +00:00
hexoscott
9f6b842d03
check for free messages when calling trace_transaction (#7073)
Gateway reported an issue with a trace returning an odd result, similar
to the recent problem we'd seen with gnosis. I found that
debug_traceTransaction worked fine so found where the differences were.
trace_transaction wasn't checking for service transactions so the trace
failed around fees when it shouldn't.

A number of code paths use callManyTransactions so they should all now
check for service messages where needed.
2023-03-13 07:36:40 +00:00
alex.sharov
470547f07a erigon-lib up 2023-03-13 11:07:21 +07:00
Alex Sharov
f432cde16b
e3: reconst: run workers in errgroup (#7071) 2023-03-13 04:04:49 +00:00
Alex Sharov
68f264675a
e3: simplify wal (#7085) 2023-03-13 02:35:18 +00:00
alex.sharov
2155147417 go mod tidy 2023-03-13 09:33:10 +07:00
Alex Sharov
a44e0afbd6
Downloader main loop wait on close (#7082) 2023-03-13 02:12:30 +00:00
Kevin Ingersoll
ec071b4195
fix typo in eth_call error (#7084)
Spotted this typo when poking at Cloudflare's public gateway and mapped
it back here 🙈
2023-03-13 02:12:08 +00:00
Giulio rebuffo
dbf1225cb2
Added partial SSZ library (#7083)
Implements SSZ encode/decoding and hashtreeroot for simple list-lacking
data structures. does not account offset case.
2023-03-12 14:41:53 +00:00
Giulio rebuffo
9167429a75
Added phase0 support to Erigon-CL. (#7066)
Added phase 0 support.
2023-03-11 19:27:21 +00:00
alex.sharov
e3f0f99cef save 2023-03-11 15:37:46 +07:00
Alex Sharov
4ab63db6e2
nil ptr in delete ancient (#7081) 2023-03-11 05:46:32 +00:00
awskii
a96293ae8a
e4: added some metrics to code (#7078) 2023-03-10 19:21:10 +00:00
Andrew Ashikhmin
3270720cb7
Remove ETC-specific DAOForkSupport=false functionality (#7075) 2023-03-10 12:10:11 +00:00
Andrew Ashikhmin
2212e21192
Remove archaic eip150Hash functionality (#7074) 2023-03-10 10:55:59 +00:00
alex.sharov
7190d89af3 save 2023-03-10 16:01:55 +07:00
Alex Sharov
81c7936df4
sentry: handle "retry later" grpc stream (#6852) 2023-03-10 05:28:25 +00:00
Rafael Matias
43960fec34
params: remove EF azure bootnodes (#7061)
Same has been done in geth:
https://github.com/ethereum/go-ethereum/pull/26828
2023-03-10 05:27:20 +00:00
alex.sharov
8189fdd587 cleanup 2023-03-10 12:20:50 +07:00
alex.sharov
99f5be5e21 fix build 2023-03-10 12:18:19 +07:00
Alex Sharov
5b3ba433eb
e3: aggressive pruning (#7070) 2023-03-10 04:31:44 +00:00
Alex Sharov
570c3d65ab
up moq version, up some lru version (#7069) 2023-03-10 04:03:32 +00:00
awskii
36e3c94897
run E4 via integration binary (#7063)
added subcommand `state_domains` of `bin/integration` to start
processing of existing chaindata with use of state domains and
commitment.
It creates two directory in `datadir`: `state` and `statedb` for files
and mdbx respectively.

This version runs blocks one after another and produces merged files. 
Want to add some metrics to export later.
2023-03-10 03:27:12 +00:00
Giulio rebuffo
7f6d1c9575
Added SSZ support for Phase0 state. (#7065) 2023-03-09 17:34:49 +00:00
Michael Montour
d5fbc5011e
Partial EIP1186 eth_getProof implementation (#6560)
This is a partial implementation of eth_getProof (see issue #1349),
supporting only a request for the latest block and an empty list of
storage keys (i.e. Account proof only). I don't know if there's a better
way of implementing this, but this was what I could come up with.
Posting it here in case it's useful.

Example output:
```
> eth.getProof("0x67b1d87101671b127f5f8714789C7192f7ad340e", [], 'latest')
{
  accountProof: ["0xf90131a0252c9d4ed347b4cf3fdccaea3ccef0a507e6bd4dbe4dcd98609b7195347c4062a0ab8cdb808c8303bb61fb48e276217be9770fa83ecf3f90f2234d558885f5abf18080a01a697e814758281972fcd13bc9707dbcd2f195986b05463d7b78426508445a04a0b5d7a91be5ee273cce27e2ad9a160d2faadd5a6ba518d384019b68728a4f62f4a0c2c799b60a0cd6acd42c1015512872e86c186bcf196e85061e76842f3b7cf86080a0e73919d9f472eec11f6da95518503f5527a98b9428f7a02c4f55bf51854214e480a06301b39b2ea8a44df8b0356120db64b788e71f52e1d7a6309d0d2e5b86fee7cb8080a01b7779e149cadf24d4ffb77ca7e11314b8db7097e4d70b2a173493153ca2e5a0a066a7662811491b3d352e969506b420d269e8b51a224f574b3b38b3463f43f0098080", "0xf8518080808080a0a00135c9ec2655cb6a47ab7ad27d6fc150d9cba8b3d4a702e879179116a68a60808080808080a02fb46956347985b9870156b5747712899d213b1636ad4fe553c63e33521d567a80808080", "0xf873a02056274a27dd7524955417c11ecd917251cc7c4c8310f4c7e4bd3c304d3d9a79b850f84e808a021e19e0c9bab2400000a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a0c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470"],
  address: "0x67b1d87101671b127f5f8714789c7192f7ad340e",
  balance: "0x21e19e0c9bab2400000",
  codeHash: "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470",
  nonce: "0x0",
  storageHash: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  storageProof: []
}
> eth.getBlock('latest').stateRoot
"0x6a0673c691edfa4c4528323986bb43c579316f436ff6f8b4ac70854bbd95340b"
```
2023-03-09 03:04:09 +00:00
Alex Sharov
d19fcd870b
e3: use iterators composition in invIndex.Range (#7056) 2023-03-09 02:26:40 +00:00