this first major move separates the transient beacon state cache from
the underlying tree.
leaf updates are enforced in the setters, which should make programming
easier.
all exported methods of the raw.BeaconState should be safe to call
(without disrupting internal state)
changes many functions to consume *raw.BeaconState in perparation for
interface
beyond refactor it also:
adds a pool for the leaves of the validator ssz hash
adds a pool for the snappy writers
removed the parallel hash experiment (high memory use)
To expose context flags through diagnostic endpoint
`/debug/metrics/flags`.
The current `/debug/metrics/cmdline` only provides the command run by
user. In the case when user runs Erigon using config file, `cmdline`’s
info does not truly reflect the ‘launch setting' and flags set that
Erigon is running on.
## Example
### Command
```
erigon --datadir /tmp/data --config test.yaml
```
### Pseudo config file (in yaml)
```
datadir : '/tmp/data'
chain : "sepolia"
http : true
metrics: true
private.api.addr : "localhost:9090"
http.api : ["eth","debug","net"]
```
### Output
```
SUCCESS
chain=sepolia
config=test.yaml
datadir=/tmp/data
http=true
http.api=eth,debug,net
metrics=true
private.api.addr=localhost:9090
```
This addresses the last known deficiency of the eth_getProof
implementation. The previous code would return an error in the event
that the element was not found in the trie. EIP-1186 allows for
'negative' proofs where a proof demonstrates that an element cannot be
in the trie, so this commit updates the logic to support that case.
Co-authored-by: Jason Yellick <jason@enya.ai>
This PR extends the function merged in #7337 to cover the proof
generation paths as well.
The first commit simply migrates the internal proof checking code from
the rpc `eth_getProof` test to be available externally exported under
`turbo/trie`. In the process, it translates the in place testing
assertions to return more normally as errors and adapts to use the
pre-existing trie node types which are defined slightly differently
(although this code is still intended only for testing purposes).
The second commit refactors the existing trie fuzzing tests from the
previous PR and adds new fuzzing tests for the proof generation. In
order to validate the proofs, these fuzzing tests fundamentally do two
things. Firstly, after bootstrapping the test (by seeding, and modifying
the db), for each key we compute the 'naive' proof utilizing the
existing code in `proof.go` and the in memory `trie.Trie` structure. The
`trie.Trie` code actually had a couple small bugs which are fixed in
this PR (not handling value nodes, and not honoring the `NewTestRLPTrie`
contract of pre-RLP encoded nodes in proof generation). Secondly, we
re-compute the same proof for the flatDB production variant, and verify
that it is exactly the same proof as computed by the naive
implementation.
This fuzzing has been run for ~72 hours locally with no errors. Although
this code hasn't identified any new bugs in the proof generation path,
it improves coverage and should help to prevent regressions. Additional
extensions will be provided in a subsequent PR.
---------
Co-authored-by: Jason Yellick <jason@enya.ai>
There are currently a number of bugs in the flat DB trie hash
computation. These bugs are improbable in 'real' scenarios (hence the
ability to sync the assorted large chains), and, the repercussions of
encountering them are low (generally re-computing the hash at a later
block will succeed). Still, they can cause the process to crash, or
deadlock, and a clever adversary could feasibly construct a block or
storage update designed to trigger these bugs. (Or, an unlucky block may
trigger them inadvertently). Based on the tracing in the code, it seems
that some of these bugs may have been witnessed in the wild, but not
reliably reproduced (for instance when `maxlen >= len(curr)`).
1. There is an infinite loop that can occur in
_nextSiblingOfParentInMem. This occurs in the account path when the
c.k[1] entry is 'nil' and the code will continuously retry resolving
this entry indefinitely.
2. When the next trie node is deeper in the trie than the previous, but
is not a descendent of the previous trie node, then the old trie node is
inadvertently left in memory instead of nil-ed. This results in an
incorrect hash being computed.
3. When the last trie node being processed is comprised entirely of 'f'
nibbles, the 'FirstNotCoveredPrefix' returns an empty byte array,
because there is no next nibble sub-tree. This causes keys to
inappropriately be reprocessed triggering either an index out of bounds
panic or incorrect hash results.
4. When the _nextSiblingInDB path is triggered, if the next nibble
subtree contains no trie table entries, then any keys with a prefix in
that subtree will be skipped resulting in incorrect hash results.
The fuzzing seeds included cover all four of these cases (for both
accounts and storage where appropriate). As fuzzing is baked into the
native go 1.18 toolchain, running 'go test' (as the Makefile currently
does) is adequate to cover these failures. To run additional fuzzing the
`-fuzz` flag may be provided as documented in the go test doc.
Co-authored-by: Jason Yellick <jason@enya.ai>
types.NewMessage now expects maxFeePerDataGas param, which will be used
in transaction verification (preCheck). GetPayloadV3 method added to
EngineAPI. Some cosmetic changes applied.
According to EIP-1186 the `proof` parts of the response to eth_getProof
should be returned "starting with the stateRoot-Node, following the path
of the SHA3 (address) as key." Currently, the proof is returned in
traversal order, rather than from the root.
Although all of the proof elements are there and correct, this is
contrary to the EIP and will cause problems for some clients. The
existing rpc test uses a map to lookup proof elements by hash, rather
than by index, so this bug was not initially caught.
This commit fixes the behavior, updates the existing test, and adds
additional checks to the rpc test.
Co-authored-by: Jason Yellick <jason@enya.ai>
In this PR Header's ExcessDataGas is set to the actual value, but it's
still unused. It will be used to compute data fee for eip-4844 data
blobs, logic of which will be added in later PRs. Also eip-4844 header
verification logic added.
`excessDataGas` has been partially made eip-4844 ready, so instead of
passing nils to functions, now it actually assigned to some value (it is
expected to be nil until cancun update).
Pre-Shanghai blocks should have `nil` withdrawals, while post-Shanghai
blocks should have non-nil withdrawals (empty or non-empty slice, but
not `nil`). Judging from Issue #6976, that's not always a case. PR #7279
attempted to fix the issue, but unfortunately it only masks the root
cause.
This reverts commit c60a6a2962.
This PR completes the implementation of `eth_getProof` by adding support
for storage proofs.
Because storage proofs are potentially overlapping, the existing
strategy of simply aggregating all proofs together into a single result
was challenging. Instead, this commit rewires things to introduce a
ProofRetainer, which aggregates proofs and their corresponding nibble
encoded paths in the trie. Once all of the proofs have been aggregated,
the caller requests the proof result, which then iterates over the
aggregated proofs, placing each into the relevant proof array into the
result.
Although there are tests for `eth_getProof` as an RPC and for the new
`ProofRetainer` code, the code coverage for the proof generation over
complex tries is lacking. But, since this is not a new problem I'll plan
to follow up this PR with an additional one adding more coverage into
`turbo/trie`.
---------
Co-authored-by: Jason Yellick <jason@enya.ai>
@shyba hi, seems this lib doesn't work with
ghcr.io/goreleaser/goreleaser-cross (which producing release binaries)
removing it for now, feel free to add it in future - if can make it work
with goreleaser-cross
see: https://github.com/ledgerwatch/erigon/issues/7210
Small change in core.NewEVMBlockContext and now it expects
excessDataGas. This will be used in state transition to compute data fee
for eip-4844 data blobs. The logic that computes it will be added in the
next PRs.
This PR contains very small EIP-4844 additions. GasPool is modified and
now it is a struct with 2 fields "gas" and "dataGas" (blobs are priced
in dataGas). ExcessDataGas block header field added. ExcessDataGas
needed to compute the data gas price. EIP-4844 helper functions are
added as well.
