- always RLock all snapshots - to guarantee consistency
- introduce class View (analog of RoTx and MakeContext)
- move read methods to View object
- View object will be managed by temporal_tx
---------
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
## What's this PR is about?
Minor fix in args usage message of support flag. The current message
says that the flag should be 'metrics.url' but it reality it should be
'metrics.urls'
Blob transactions are SSZ encoded, so it had to be added to decoding.
There are 2 encoding forms: `network` and `minimal` (usual). Network
encoded blob transactions include "wrapper data" which are `kzgs`,
`blobs` and `proofs`, and decoded by `DecodeWrappedTransaction`. For
previous types of transactions the network encoding is no different.
Execution-payloads / blocks use the minimal encoding of transactions. In
the transaction-pool and local transaction-journal the network encoding
is used.
Concerns:
1. Possible performance reduction caused by these changes, not sure if
streams are better then slices. Go slices in this modifications are
read-only, so they should be referred to the same underlying array and
passed by a reference.
2. If `DecodeWrappedTransaction` and `DecodeTransaction` will create
confusion and should be merged into one function.
This is the beginning of the series of changes to make it possible to
run multiple instances of erigon inside a single process (as devnet tool
does), with the logging from these processes going to respective log
files correctly.
This is the first part where the initial infrastructure is being
established
---------
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
this first major move separates the transient beacon state cache from
the underlying tree.
leaf updates are enforced in the setters, which should make programming
easier.
all exported methods of the raw.BeaconState should be safe to call
(without disrupting internal state)
changes many functions to consume *raw.BeaconState in perparation for
interface
beyond refactor it also:
adds a pool for the leaves of the validator ssz hash
adds a pool for the snappy writers
removed the parallel hash experiment (high memory use)
To expose context flags through diagnostic endpoint
`/debug/metrics/flags`.
The current `/debug/metrics/cmdline` only provides the command run by
user. In the case when user runs Erigon using config file, `cmdline`’s
info does not truly reflect the ‘launch setting' and flags set that
Erigon is running on.
## Example
### Command
```
erigon --datadir /tmp/data --config test.yaml
```
### Pseudo config file (in yaml)
```
datadir : '/tmp/data'
chain : "sepolia"
http : true
metrics: true
private.api.addr : "localhost:9090"
http.api : ["eth","debug","net"]
```
### Output
```
SUCCESS
chain=sepolia
config=test.yaml
datadir=/tmp/data
http=true
http.api=eth,debug,net
metrics=true
private.api.addr=localhost:9090
```
This addresses the last known deficiency of the eth_getProof
implementation. The previous code would return an error in the event
that the element was not found in the trie. EIP-1186 allows for
'negative' proofs where a proof demonstrates that an element cannot be
in the trie, so this commit updates the logic to support that case.
Co-authored-by: Jason Yellick <jason@enya.ai>
This PR extends the function merged in #7337 to cover the proof
generation paths as well.
The first commit simply migrates the internal proof checking code from
the rpc `eth_getProof` test to be available externally exported under
`turbo/trie`. In the process, it translates the in place testing
assertions to return more normally as errors and adapts to use the
pre-existing trie node types which are defined slightly differently
(although this code is still intended only for testing purposes).
The second commit refactors the existing trie fuzzing tests from the
previous PR and adds new fuzzing tests for the proof generation. In
order to validate the proofs, these fuzzing tests fundamentally do two
things. Firstly, after bootstrapping the test (by seeding, and modifying
the db), for each key we compute the 'naive' proof utilizing the
existing code in `proof.go` and the in memory `trie.Trie` structure. The
`trie.Trie` code actually had a couple small bugs which are fixed in
this PR (not handling value nodes, and not honoring the `NewTestRLPTrie`
contract of pre-RLP encoded nodes in proof generation). Secondly, we
re-compute the same proof for the flatDB production variant, and verify
that it is exactly the same proof as computed by the naive
implementation.
This fuzzing has been run for ~72 hours locally with no errors. Although
this code hasn't identified any new bugs in the proof generation path,
it improves coverage and should help to prevent regressions. Additional
extensions will be provided in a subsequent PR.
---------
Co-authored-by: Jason Yellick <jason@enya.ai>
There are currently a number of bugs in the flat DB trie hash
computation. These bugs are improbable in 'real' scenarios (hence the
ability to sync the assorted large chains), and, the repercussions of
encountering them are low (generally re-computing the hash at a later
block will succeed). Still, they can cause the process to crash, or
deadlock, and a clever adversary could feasibly construct a block or
storage update designed to trigger these bugs. (Or, an unlucky block may
trigger them inadvertently). Based on the tracing in the code, it seems
that some of these bugs may have been witnessed in the wild, but not
reliably reproduced (for instance when `maxlen >= len(curr)`).
1. There is an infinite loop that can occur in
_nextSiblingOfParentInMem. This occurs in the account path when the
c.k[1] entry is 'nil' and the code will continuously retry resolving
this entry indefinitely.
2. When the next trie node is deeper in the trie than the previous, but
is not a descendent of the previous trie node, then the old trie node is
inadvertently left in memory instead of nil-ed. This results in an
incorrect hash being computed.
3. When the last trie node being processed is comprised entirely of 'f'
nibbles, the 'FirstNotCoveredPrefix' returns an empty byte array,
because there is no next nibble sub-tree. This causes keys to
inappropriately be reprocessed triggering either an index out of bounds
panic or incorrect hash results.
4. When the _nextSiblingInDB path is triggered, if the next nibble
subtree contains no trie table entries, then any keys with a prefix in
that subtree will be skipped resulting in incorrect hash results.
The fuzzing seeds included cover all four of these cases (for both
accounts and storage where appropriate). As fuzzing is baked into the
native go 1.18 toolchain, running 'go test' (as the Makefile currently
does) is adequate to cover these failures. To run additional fuzzing the
`-fuzz` flag may be provided as documented in the go test doc.
Co-authored-by: Jason Yellick <jason@enya.ai>
types.NewMessage now expects maxFeePerDataGas param, which will be used
in transaction verification (preCheck). GetPayloadV3 method added to
EngineAPI. Some cosmetic changes applied.
