This PR contains 3 fixes for interaction between the Bor mining loop and
the TX pool which where causing the regular creation of blocks with zero
transactions.
* Mining/Tx pool block synchronization
The synchronization of the tx pool between the sync loop and the mining
loop has been changed so that both are triggered by the same event and
synchronized via a sync.Cond rather than a polling loop with a hard
coded loop limit. This means that mining now waits for the pool to be
updated from the previous block before it starts the mining process.
* Txpool Startup consolidated into its MainLoop
Previously the tx pool start process was dynamically triggered at
various points in the code. This has all now been moved to the start of
the main loop. This is necessary to avoid a timing hole which can leave
the mining loop hanging waiting for a previously block broadcast which
it missed due to its delay start.
* Mining listens for block broadcast to avoid duplicate mining
operations
The mining loop for bor has a recommit timer in case blocks re not
produced on time. However in the case of sprint transitions where the
seal publication is delayed this can lead to duplicate block production.
This is suppressed by introducing a `waiting` state which is exited upon
the block being broadcast from the sealing operation.
* fix "genesis hash does not match" when dev nodes connect
The "dev" nodes need to have the same --miner.etherbase in order to
generate the same genesis ExtraData by DeveloperGenesisBlock(). Override
DevnetEtherbase global var that's used if --miner.etherbase is not
passed. (for NonBlockProducer case)
* fix missing private key for the hardcoded DevnetEtherbase
Fixes panic if SigKey is not found. Bor non-producers will use a default
`DevnetEtherbase` while Dev nodes modify it. Save hardcoded
DevnetEtherbase/DevnetSignPrivateKey into accounts so that SigKey can
recover it.
* refactor devnet.node to contain Node config
This avoids interface{} type casts and fixes an error with
Heimdall.validatorSet == nil
* add connection retries to rpcCall and Subscribe of requestGenerator
Fixes "connection refused" errors due to node not ready to handle early
RPC requests.
* fix deadlock in Heimdall.NodeStarted
* fix GetBlockByNumber
Fixes "cannot unmarshal string into Go struct field body.transactions of
type jsonrpc.RPCTransaction"
* demote "no of blocks on childchain is less than confirmations
required" to Info (#8626)
* demote "mismatched pending subpool size" to Debug (#8615)
* revert wiggle testing code
This request is extending the devnet functionality to more fully handle
contract processing by adding support for the following calls:
* trace_call,
* trace_transaction
* debug_accountAt,
* eth_getCode
* eth_estimateGas
* eth_gasPrice
It also contains an initial rationalization of the devnet subscription
code to use the erigon client code directly rather than using its own
intermediate subscription management.
This is used to implement a general purpose block waiter - which can be
used in any scenario step - rather than being specific to transaction
processing.
This pull also contains an end to end tested sync processor for bor and
associated support services:
* Heimdall (supports sync event transfer)
* Faucet - allows the creation and funding of arbitary test specific
accounts (cross chain)
Notes and Caveats:
* Code generation for contracts requires `--evm-version paris`. For
chains which don't support push0 for solc over 0.8.19
* The bor log processing post the application of sync events causes a
panic - this will be the subject of a seperate smaller push as it is not
devnet specific
* The bor code seems to make repeated calls for the same sync events and
also reverts requests - this needs further investigation. This is the
behaviour of the current implementation and may be required - although
it does seem to generate repeat processing - which could be avoided.
This request implements an end to end Polygon state sync in the devnet.
It does this by deploying smart contracts ont the L2 & L2 chain which
follow the polygon fx portal model with security checks removed to
simplify the code. The sync events generated are routed through a local
mock heimdal - to avoid the consensus process for testing purposes.
The commit also includes support code to help the delivery of additional
contract based scenratios.
An update to the devnet to introduce a local heimdall to facilitate
multiple validators without the need for an external process, and hence
validator registration/staking etc.
In this initial release only span generation is supported.
It has the following changes:
* Introduction of a local grpc heimdall interface
* Allocation of accounts via a devnet account generator ()
* Introduction on 'Services' for the network config
"--chain bor-devnet --bor.localheimdall" will run a 2 validator network
with a local service
"--chain bor-devnet --bor.withoutheimdall" will sun a single validator
with no heimdall service as before
---------
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
The check in catches errors in the node start-up code and makes sure
that the network is stopped if any node fails to start cleanly, and
that5 it returns an error - so that any calling code can take
appropriate action.
This is an update to the devnet code which introduces the concept of
configurable scenarios. This replaces the previous hard coded execution
function.
The intention is that now both the network and the operations to run on
the network can be described in a data structure which is configurable
and composable.
The operating model is to create a network and then ask it to run
scenarios:
```go
network.Run(
runCtx,
scenarios.Scenario{
Name: "all",
Steps: []*scenarios.Step{
&scenarios.Step{Text: "InitSubscriptions", Args: []any{[]requests.SubMethod{requests.Methods.ETHNewHeads}}},
&scenarios.Step{Text: "PingErigonRpc"},
&scenarios.Step{Text: "CheckTxPoolContent", Args: []any{0, 0, 0}},
&scenarios.Step{Text: "SendTxWithDynamicFee", Args: []any{recipientAddress, services.DevAddress, sendValue}},
&scenarios.Step{Text: "AwaitBlocks", Args: []any{2 * time.Second}},
},
})
```
The steps here refer to step handlers which can be defined as follows:
```go
func init() {
scenarios.MustRegisterStepHandlers(
scenarios.StepHandler(GetBalance),
)
}
func GetBalance(ctx context.Context, addr string, blockNum requests.BlockNumber, checkBal uint64) {
...
```
This commit is an initial implementation of the scenario running - which
is working, but will need to be enhanced to make it more usable &
developable.
The current version of the code is working and has been tested with the
dev network, and bor withoutheimdall. There is a multi miner bor
heimdall configuration but this is yet to be tested.
Note that by default the scenario runner picks nodes at random on the
network to send transactions to. this causes the dev network to run very
slowly as it seems to take a long time to include transactions where the
nonce is incremented across nodes. It seems to take a long time for the
nonce to catch up in the transaction pool processing. This is yet to be
investigated.
This branch is intended to allow the devnet to be used for testing
multiple consents types beyond the default clique. It is initially being
used to test Bor consensus for polygon.
It also has the following refactoring:
### 1. Network configuration
The two node arg building functions miningNodeArgs and nonMiningNodeArgs
have been replaced with a configuration struct which is used to
configure:
```go
network := &node.Network{
DataDir: dataDir,
Chain: networkname.DevChainName,
//Chain: networkname.BorDevnetChainName,
Logger: logger,
BasePrivateApiAddr: "localhost:9090",
BaseRPCAddr: "localhost:8545",
Nodes: []node.NetworkNode{
&node.Miner{},
&node.NonMiner{},
},
}
```
and start multiple nodes
```go
network.Start()
```
Network start will create a network of nodes ensuring that all nodes are
configured with non clashing network ports set via command line
arguments on start-up.
### 2. Request Routing
The `RequestRouter` has been updated to take a 'target' rather than
using a static dispatcher which routes to a single node on the network.
Each node in the network has its own request generator so command and
services have more flexibility in request routing and
`ExecuteAllMethods` currently takes the `node.Network` as an argument
and can pick which node (node 0 for the moment) to send requests to.
@shyba hi, seems this lib doesn't work with
ghcr.io/goreleaser/goreleaser-cross (which producing release binaries)
removing it for now, feel free to add it in future - if can make it work
with goreleaser-cross
see: https://github.com/ledgerwatch/erigon/issues/7210
- Fix tx numbers expected
- Check for baseFee subpool size too
- Fix tx counting
- Wait for blocks in baseFee subpool to be promoted and mined
- Add BlockNumber request
**Current Problem**
When X transactions are created, with M transactions out of them having
`gasFeeCap` greater than the current `baseFeePerGas` for the block, and
N transactions having `gasFeeCap` lower, all X transactions get added to
the yellow pool (baseFee) in the txpool and they never get mined.
However when X transactions all having `gasFeeCap` higher than the
`baseFeePerGas` are created, they all get added to the green pool
(pending) in the txpool and thus get mined.
This phenomenon can be inspected in the
`signEIP1559TxsLowerAndHigherThanBaseFee2` function where the number of
transactions that should have `gasFeeCap` higher than and lower than the
current `baseFeePerGas` is specified in its parameter - `func
signEIP1559TxsLowerAndHigherThanBaseFee2(amountLower, amountHigher
int...`.
When `amountLower` is set as `0`, all transactions created from
`amountHigher` will be mined because they have `gasFeeCap` >
`baseFeePerGas`. When `amountLower` has a value > `0`, all the
transactions created (including transactions created with
`amountHigher`) will go to the yellow pool (baseFee) and thus do not get
mined.
- Added a new method and type for contract transactions.
- Added functions to emit fallback events from contract transactions.
- Added GetLogs request generator
- Added tests for GetLogs request generator
- Added listening methods for WebSocket subscriptions
- Listened for new blocks using the newHeads method to determine when to
look for a transaction
- Added new util methods and tests for them
- Simplified communication to the user upon initiating the devnet tool
- Added new requests for sending raw transactions
- Made provisions for other types of transactions generated, including
contracts
- Modified the models and services to accommodate general values
* Rebuilding the devnet tool for better quality
* Added README
* deleted former devnettest folder
* Correction changes
* Implementation node connection through enode for two nodes
* synced two nodes, moving to logging
* Fixed lint
