Commit Graph

37 Commits

Author SHA1 Message Date
Mark Holt
d8b91c4d02
Fix startup sync for txpool processing for bor block production (#9219)
When the sync loop first runs it suppresses block sync events both in
the initial loop and when the blocks being processed are greater than
1000.

This fix removed the first check, because otherwise the first block
received by the process ends up not getting sent to the tx pool. Which
means it won't produce new block for polygon.

As well as this fix - I have also moved the gas initialization to the
txpool start method rather than prompting it with a 'synthetic block
event'

As the txpool start has access to the core & tx DB's it can find the
current block and chain config internally so that it doesn't need to be
externally activated it can just do this itself on start up. This has
the advantage of making the txpool more self contained.
2024-01-13 10:33:34 +00:00
Håvard Anda Estensen
4873502818
turbo: run tests in parallel (#8738)
Tests that don't affect each other should run in parallel
2023-11-16 16:29:31 +07:00
a
436493350e
Sentinel refactor (#8296)
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Somnath Banerjee
f51d9b61a0
Txpool 4844 upgrades Part 2 (#8213)
Some peer-review changes from the last related PR. 
Addition of a flag for BlobSlots - for max allowed blobs per account in
txpool.
Use BlobFee from the block to validate txs in the pool.

See also https://github.com/ledgerwatch/erigon-lib/pull/1125
2023-09-20 17:29:30 +05:30
Somnath Banerjee
a699f64761
Txpool upgrades for EIP-4844 Blob Transactions (#8004)
See https://github.com/ledgerwatch/erigon-lib/pull/1075
2023-09-11 09:38:58 +07:00
Mark Holt
8ea0096d56
moved metrics sub packages types to metrics (#8119)
This is a non functional change which consolidates the various packages
under metrics into the top level package now that the dead code is
removed.

It is a precursor to the removal of Victoria metrics after which all
erigon metrics code will be contained in this single package.
2023-09-03 08:09:27 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization (#8094)
This is an update of:

https://github.com/ledgerwatch/erigon/pull/7846

which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.

It also includes code to address the duplicate metrics issue identified
here:

https://github.com/ledgerwatch/erigon/issues/8053

It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.

e.g. 

```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than

```
# TYPE p2p_peers gauge
p2p_peers 0
```

---------

Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Andrew Ashikhmin
9387abf66b
[interfaces] lower_snake_case field names (#7293)
Pick up https://github.com/ledgerwatch/interfaces/pull/163
2023-04-12 10:35:54 +00:00
Alex Sharov
201572c6f5
enable more linters #954 (#7179) 2023-03-25 05:13:27 +00:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
Alex Sharov
4ffd50c716
hashers to use sync.pool (#6200)
for https://github.com/ledgerwatch/erigon/issues/6195
2022-12-04 17:23:44 +07:00
Andrew Ashikhmin
5b3f2b8c5f
Update interfaces (#6031)
Pick up
813d816908
& https://github.com/ledgerwatch/interfaces/pull/131.
2022-11-11 17:16:39 +01:00
hexoscott
636586c1b5
cache state check (#5844)
draft for now to get some early feedback on approach
2022-11-07 13:04:31 +00:00
ledgerwatch
94f4ea805d
Fixing hive SideChain reorg test (#5620)
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)

so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-10-05 05:42:38 +01:00
Andrew Ashikhmin
ab28089583
Still fixing index out of range in (*Accumulator) ChangeStorage (#4751) 2022-07-19 11:11:08 +02:00
Andrew Ashikhmin
5862899979
Fix index out of range in (*Accumulator) ChangeStorage (#4738) 2022-07-18 19:03:38 +02:00
Alex Sharov
2415fecb26
--downloader.verfiy flag to verify once on startup (#4597)
* save

* save

* save

* save

* save

* save
2022-07-01 16:52:43 +06:00
Alex Sharov
3e4fb5dd49
up linter version (#4108)
* save

* save

* save
2022-05-10 10:14:09 +07:00
ledgerwatch
0bb6478d3d
Add blockGasLimit to the state stream (#3321)
* Add blockGasLimit to the state stream

* Send block gas limit

* Switch to new erigon-lib

* Update

* Update txpool - safe operations

* Update erigon-lib

* Txpool working

* Update

* Update

* Update

* Fix lint

* fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-25 10:49:28 +00:00
Alex Sharov
a148e71d9d
Pool: switch to pendingBaseFee (#2732) 2021-09-26 23:09:36 +07:00
Alex Sharov
400c71b7ce
Pool: txID-based cache (#2697) 2021-09-17 10:31:20 +07:00
Alex Sharov
76dd448c70
Pool: switch cache to tx.ID(), add blockNum to tx info to drop from full pool old txs first (#2688) 2021-09-15 19:34:05 +07:00
Alex Sharov
77db993cfc
Pool: add coherent cache (#2678) 2021-09-13 14:58:25 +07:00
ledgerwatch
15b4095718
Move ETL to erigon-lib (#2667)
* Move ETL to erigon-lib

* Update link in the readme

* go mod tidy

* Use common/chan.go from erigon-lib

* Clean up

* Fix lint

* Fix test

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-12 08:50:17 +01:00
Alex Sharov
b7ada2a595
create new Change object if incarnation increased (#2539)
* create new Change object if incarnation increased

* create new Change object if incarnation increased
2021-08-18 09:41:07 +07:00
Alex Sharov
6c7edb0d34
state diff send to txpool (#2532) 2021-08-17 15:52:55 +07:00
Alex Sharov
69dc2776dc
MIT licensed metrics lib (#2462) 2021-07-29 17:27:46 +07:00
Alex Sharov
43af5d42b8
simplify staged sync world (#2336)
* simplify world

* simplify world

* simplify world
2021-07-10 16:43:58 +07:00
ledgerwatch
a05d1be531
Pull grpc bindings into erigon-lib (#2265)
* Pull grpc bindings into erigon-lib

* Remove mdbx dist

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-01 22:31:14 +01:00
ledgerwatch
835b954f5c
State stream to RPC daemon (#1986)
* Remove interfaces

* Squashed 'interfaces/' content from commit e06631eb4

git-subtree-dir: interfaces
git-subtree-split: e06631eb4d0926c2d6a4f552497b920b4ed8d1bd

* Update KV interface

* Squashed 'interfaces/' changes from e06631eb4..014677ffe

014677ffe Merge remote-tracking branch 'origin/master' into stream
08c32a09e add version method to txPool and ethbackend
5b6bf70b9 Update README.md
7712cb267 Update README.md
f895ece4c save (#37)

git-subtree-dir: interfaces
git-subtree-split: 014677ffe5bff0dee1a333f06c92e6110d791468

* Fix old interfaces

* Squashed 'interfaces/' changes from 014677ffe..df31e1146

df31e1146 remove action from storage change

git-subtree-dir: interfaces
git-subtree-split: df31e1146c368eda2e2b15ab252b78fba7a0a6f3

* add Accumulator

* add location

* Squashed 'interfaces/' changes from df31e1146..472584639

472584639 Merge remote-tracking branch 'origin/master' into stream
dd6a42724 Refactor of consensus interface (#28)

git-subtree-dir: interfaces
git-subtree-split: 472584639f637189dfb906ef1ed03665f98d55d2

* Fix compilation in cons

* Pass accumulator to Execution stage

* Fix test

* Pass accumulator to unwind and plain writer

* Add accumulator use to plain writer

* Squashed 'interfaces/' changes from 472584639f..5c36f038b8

5c36f038b8 State change stream to KV (#38)
REVERT: 472584639f Merge remote-tracking branch 'origin/master' into stream
REVERT: df31e1146c remove action from storage change
REVERT: 014677ffe5 Merge remote-tracking branch 'origin/master' into stream
REVERT: e06631eb4d Fix
REVERT: 9c10d79d2d Fix
REVERT: 61ae9307de Fix
REVERT: 4fcf34ecc5 State change stream to KV

git-subtree-dir: interfaces
git-subtree-split: 5c36f038b87096ffb6b07e90c6762c21b864cd3b

* Add state.stream flag

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-23 15:53:18 +01:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
Alex Sharov
cf464302a4
check_right_cache_size (#1711) 2021-04-11 11:23:39 +07:00
Alex Sharov
3f305673aa
Trie: add invariant - first level of trie must be in DB (to ensure having 100% trie structure in trie_account table) (#1522) 2021-03-13 09:30:54 +07:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record (#1500)
* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE
2021-02-21 18:41:59 +00:00
ledgerwatch
2daa71e6cb
Support for trace_callMany (#1486)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-10 17:04:47 +00:00
ledgerwatch
5ea590c18e
State cache switching writes to reads during commit (#1368)
* State cache init

* More code

* Fix lint

* More tests

* More tests

* More tests

* Fix test

* Transformations

* remove writeQueue, before fixing the tests

* Fix tests

* Add more tests, incarnation to the code items

* Fix lint

* Fix lint

* Remove shards prototype, add incarnation to the state reader code

* Clean up and replace cache in call_traces stage

* fix flaky test

* Save changes

* Readers to use addrHash, writes - addresses

* Fix lint

* Fix lint

* More accurate tracking of size

* Optimise for smaller write batches

* Attempt to integrate state cache into Execution stage

* cacheSize to default flags

* Print correct cache sizes and batch sizes

* cacheSize in the integration

* Fix tests

* Fix lint

* Remove print

* Fix exec stage

* Fix test

* Refresh sequence on write

* No double increment

* heap.Remove

* Try to fix alignment

* Refactoring, adding hashItems

* More changes

* Fix compile errors

* Fix lint

* Wrapping cached reader

* Wrap writer into cached writer

* Turn state cache off by default

* Fix plain state writer

* Fix for code/storage mixup

* Fix tests

* Fix clique test

* Better fix for the tests

* Add test and fix some more

* Fix compile error|

* More functions

* Fixes

* Fix for the tests

* sepatate DeletedFlag and AbsentFlag

* Minor fixes

* Test refactoring

* More changes

* Fix some tests

* More test fixes

* More test fixes

* Fix lint

* Move blockchain_test to be able to use stagedsync

* More fixes

* Fixes and cleanup

* Fix tests in turbo/stages

* Fix lint

* Fix lint

* Intemediate

* Fix tests

* Intemediate

* More fixes

* Compilation fixes

* More fixes

* Fix compile errors

* More test fixes

* More fixes

* More test fixes

* Fix compile error

* Fixes

* Fix

* Fix

* More fixes

* Fixes

* More fixes and cleanup

* Further fix

* Check gas used and bloom with header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-08 09:44:29 +00:00
ledgerwatch
cceb8c2c3a
Sharded state experiment based on call_traces stage run via integration (#1329)
* Initial work on state shards

* Fix compile errors

* Intermediate

* Implement dispatcher

* Implemented sharded reader

* No cache

* Print comms

* No prepopulation

* Print dispatcher activity

* Print more

* Print errors

* Fix dispatcher

* Remove logging

* Dispatcher to enforce initial barrier

* Don't print that much

* Turn off cache pre-seeting for state sharding

* Artificial latency, fixed number of blocks to execute

* Fix
2020-10-31 11:51:56 +00:00