Commit Graph

876 Commits

Author SHA1 Message Date
Andrew Ashikhmin
d82c778ab3
Withdrawals part 1 (#6009)
This PR partially implements
[EIP-4895](https://eips.ethereum.org/EIPS/eip-4895): Beacon chain push
withdrawals as operations. The new Engine API methods
(https://github.com/ethereum/execution-apis/pull/195) are implemented.

_Body downloader and saving withdrawals into DB are not implemented
yet!_
2022-12-01 09:15:01 +01:00
Andrew Ashikhmin
424f38a9b6
AuRa: iterate over durations in order (#6013)
Iteration order over go maps is random.
2022-11-10 13:47:57 +01:00
Alex Sharov
8d1ed547b0
E3: parallel exec, apply on roTx (#5879) 2022-10-28 08:47:45 +07:00
Andrew Ashikhmin
64a3156112
Support BaseFee in AuRa headers (#5820)
BaseFee is required in AuRa headers when
[EIP-1559](https://eips.ethereum.org/EIPS/eip-1559) is activated.

Also:
- Basic AuRa header verification
- Extract some common RLP methods
- Tiny log clean-up
2022-10-21 12:43:44 +02:00
Andrew Ashikhmin
832e8e01fe
Remove verbose and partially redundant log (#5796)
This log message is mostly redundant after PR #5792.
2022-10-19 14:19:30 +02:00
hexoscott
e50110562a
revert execution batch size logic to size on disk (#5732)
Revert to older batch size logic to keep memory usage down during
execution phase and closer to the --batchSize flag size.

Spotted a "leak" in key generation as well. The unsafe pointer keeps the
byte slice around for as long as the batch is not committed, takes up a
fair chunk of memory surprisingly doing that so removed the unsafe
pointer usage giving the GC some chance to clean up along the way.

Moved the batch rollback into a defer func call rather than allowing
them to stack in the for loop. If this isn't going to work just let me
know and can change it back.
2022-10-13 09:10:26 +01:00
ledgerwatch
d95a905b2b
Fix Re-Org Back into Canonical Chain hive test (#5647)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-06 18:25:24 +01:00
ledgerwatch
94f4ea805d
Fixing hive SideChain reorg test (#5620)
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)

so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-10-05 05:42:38 +01:00
Andrew Ashikhmin
9ecc564d03
Demote "stage loop is busy" to debug (#5597) 2022-10-02 16:10:11 +01:00
Alex Sharov
37f76c9a9e
bitmapdb (#5593) 2022-10-02 10:54:40 +07:00
a
7c8b3c2ba1
[rpcdaemon] greatly increase speed of {eth,erigon}_getLogs (#5572) 2022-10-02 10:51:19 +07:00
ledgerwatch
7018c5e746
Ergonomic fixes for POS (#5574)
* Various POS fixes

* More quiet

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-29 22:17:16 +01:00
hexoscott
705673c336
increased logging around POS mining (#5442) 2022-09-20 13:15:56 +01:00
hexoscott
cd8cad6a89
allow multiple log subscriptions at the same time (#5358) 2022-09-16 14:36:25 +01:00
Andrew Ashikhmin
e5afa7292e
Potential improvement to CL timeouts (#5378)
* Strengthen early return when PoS syncing

* Be less aggressive about new payloads

* Less scary log message

* Clean up LVH in EngineStatus_SYNCING

* Slightly more correct message

* Try to fix Hive tests
2022-09-16 10:59:46 +02:00
Andrew Ashikhmin
18554a2f90
Revert "fix: relax deposit contract prune to first deposit (#5295)" (#5317)
This reverts commit d8c9b1151d.
2022-09-08 10:19:45 +02:00
banteg
d8c9b1151d
fix: relax deposit contract prune to first deposit (#5295) 2022-09-07 22:33:31 +03:00
ledgerwatch
693017c554
Cleanup Tevm experimental code (#5259)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-01 19:49:29 +01:00
Andrew Ashikhmin
4a7bcc73e1
Restore check against Beacon Chain requests before TTD (#5234)
* getPayloadStatusFromHashIfPossible -> getQuickPayloadStatusIfPossible

* Restore TTD check

* Amend log warning

* Restore full stop in comment

* 3 Hive tests are fixed now
2022-08-31 11:01:02 +02:00
fynn.z
9c713d8168
Fixed mining system contract txn lost (#5212)
* fix mining exec error

* fix invalid memory

* remove logs

Co-authored-by: Jeff Rossiter <jeffrey.rossiter@me.com>
2022-08-30 09:47:47 +07:00
Andrew Ashikhmin
465c586ef4
Fix description of --prune=r flag (#5193)
* Fix description of --prune=r flag

* Small correction
2022-08-26 18:36:31 +07:00
Andrew Ashikhmin
e2667d33cf
Amend log message and level (#5165) 2022-08-24 12:13:21 +02:00
Andrew Ashikhmin
b3d99f8e8c
Small cleanups (#5162) 2022-08-24 11:35:39 +02:00
Andrew Ashikhmin
4ca7394726
Small fix: log block RLP as hex (#5149) 2022-08-23 10:49:20 +02:00
Enrique Jose Avila Asapche
1cfb2fc7b4
fixed prune flag not working with htrc (#5139)
* fixed prune flag not working with htrc

* deleted repetition

* unnecesary check
2022-08-23 09:16:05 +07:00
Giulio rebuffo
1303023cb8
Optimized backend pre-checks and fixed FCU deadlock (#5124)
* Optimized backend pre-checks and fixed FCU deadlock

* fixed tests

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-08-20 16:27:51 +02:00
Andrew Ashikhmin
21699d929a
Log RLP of built PoS blocks (#5107) 2022-08-18 18:11:37 +02:00
Andrew Ashikhmin
25bdbe3de1
Access to EthBackendServer.builders should be protected by lock (#5106) 2022-08-18 18:11:12 +02:00
Enrique Jose Avila Asapche
c1eef706b3
checking if flag on (#5075)
Pruning is turning on by default when it shouldn't
2022-08-16 16:06:50 +03:00
Enrique Jose Avila Asapche
f0c7235f97
Pruning Receipts Default (#5065) 2022-08-15 17:21:18 +03:00
dmitriyselivanov
4499e04a05
rpcdaemon: wire the most recent of EthBackend.builders.block as a pending block into eth_getBlockByNumber (#5061) 2022-08-15 15:19:45 +07:00
Alex Sharov
52fd0d0e8b
Aggregator22.Unwind() (#5039)
* save

* save
2022-08-13 18:51:25 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Giulio rebuffo
7a64fe44eb
UX improvement for pre-merge sync with Teku (#4955)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-08-08 11:26:34 +02:00
Leonard Chinonso
592fe32217
Refactoring To Increase Code Quality (#4842)
* General cleanup and fixes

* Cleaning part 2

* cleanup part 3

* Refactored staterunner to have a handler function

* Fixed linting

* Cleanup part 4

* Changed the initialization of the require package and updated .golangci.yml

* Fixed lint errors from golangci-lint

* deleted state_recon_1.go

* fixed build error

* Added ReadHeaderTimeouts for httpServer

* made changes to golangci

* Reset golangci.yml
2022-08-04 20:23:00 +07:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-07-28 16:57:38 +07:00
Andrew Ashikhmin
c591266914
Small PoS clean-ups (#4834)
* Amend comment

* More watchful sleep

* improve comment

* Cosmetics for lock/unlock

* statusRef -> statusDeref

* Comment on obsure logic
2022-07-27 12:14:35 +02:00
Andrew Ashikhmin
c1f848746d
Update consensus tests to v11 (#4724)
* Consensus tests update 11

* Add GrayGlacier fork

* Wire currentRandom from tests into PREVRANDAO

* Serenity engine

* Support insertion of PoS blocks in MockSentry

* Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()

* Revert "Introduce marshallTypedTransactionsAsRlpStrings arg into (*Block) RawBody()"

This reverts commit 903fca572be03c7de33318ce177a03a4be34927b.

* Post-merge fix

* Don't wait for Beacon Chain in tests

* Skip powToPosBlockRejection transition test

* ForkChoice in insertPoSBlocks

* Add withPosDownloader arg to MockWithEverything in order to fix TestPoSDownloader
2022-07-26 09:35:38 +02:00
Alex Sharov
b6ea28ea80
kv.Del() remove second parameter (#4832)
* save

* save

* save

* save

* save

* save

* save
2022-07-26 12:47:05 +07:00
Alex Sharov
3aac12a5f1
use PrioritizedSend method (#4830) 2022-07-26 10:37:51 +07:00
Giulio rebuffo
a372746397
Move some checks out enginePayload and fcu (#4805)
* made in refactoring

* test

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-07-24 16:20:08 +02:00
Giulio rebuffo
1cb6be02a5
Avoid constantly triggering stageloop when using Engine API (#4797)
* avoid constantly triggering stageloop when using Engine API

* fix lint + test

* fixed comments

* ops

* little fixes here and there

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-07-23 18:57:23 +02:00
Andrew Ashikhmin
6060b87840
Fix binary vs raw confusion for PoS transaction (#4781)
* Replace PayloadMessage with Block

* RawTransactions -> BinaryTransactions for clarity

* add a log warning
2022-07-21 19:40:00 +02:00
Giulio rebuffo
d83b7a4ae0
Proper PoS Error reporting (#4631)
* better reporting

* removed debug log

* proper error reporting
2022-07-05 01:31:16 +02:00
Giulio rebuffo
3fc51f5ef7
Revert "Proper Pos block checker when INVALID/ACCEPTED status is sent (#4604)" (#4616)
This reverts commit e90e03ae31.
2022-07-03 13:11:16 +02:00
Giulio rebuffo
e90e03ae31
Proper Pos block checker when INVALID/ACCEPTED status is sent (#4604)
* added proper PoS block checker

* proper invalid lvh

* p

* fixed smol thingy

* fix more

* fixed engine API

* fixed engine API

* better nil hash

* added 0x0 checks

* full support
2022-07-03 11:58:57 +02:00
Giulio rebuffo
8558778ee2
fixed engine unit tests (#4581)
* fixed engine unit tests

* done
2022-06-30 02:27:34 +02:00
Giulio rebuffo
687295f01b
fixed stall (#4576) 2022-06-29 17:49:33 +02:00
Andrew Ashikhmin
dc5d3ffaac
Revert "Switch from eth/66 to eth/67 (#4549)" (#4562)
This reverts commit 16e57aa8a2.
2022-06-28 13:42:35 +02:00
Andrew Ashikhmin
16e57aa8a2
Switch from eth/66 to eth/67 (#4549)
* Switch from eth/66 to eth/67

* Fix a compilation error

* Update erigon-lib

* Refresh erigon-lib
2022-06-28 10:12:48 +02:00
Enrique Jose Avila Asapche
29760d1425
Getting rid of unnecessary cache (#4494)
* whitelistedTable used

* added descriptions

* no cache

* lint
2022-06-19 19:39:43 +01:00
ledgerwatch
64067a2b77
Debugging Engine API (#4488)
* defer tx rollback

* Add http.trace flag

* Fixed http.trace flag

* Fix lint

* Fix lint

* Delete unused tests

* Fix lint

* Fix lint

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-19 13:40:28 +01:00
Andrew Ashikhmin
d1aab4e59b
Bring a corner case in line with the Engine API spec (#4477) 2022-06-17 19:24:36 +01:00
Giulio rebuffo
2e600316e3
Used memory batches from Erigon (#4469)
* modded summed

* deleted

* ops
2022-06-16 16:39:56 +02:00
Giulio rebuffo
631d485476
Renamed mutations types to be exported (#4463)
* renamed

* ops
2022-06-15 19:59:20 +02:00
Giulio rebuffo
663ccfd4dc
better flush (#4459) 2022-06-15 18:45:05 +02:00
Giulio rebuffo
ff5cbcb75d
Made in memory mutation compatible with all buckets (#4454)
* progress #1

* progress #2

* proper file naming

* more mature memory mutation
2022-06-15 01:10:41 +02:00
Giulio rebuffo
cad32ee1cf
added deletion support (#4451) 2022-06-14 21:43:44 +02:00
Giulio rebuffo
1f36d76e09
Added Flush function to memory mutation (#4439) 2022-06-13 18:20:18 +02:00
Andrew Ashikhmin
eb497372ae
Interruptible PoS block building (#4438)
* Fix a typo

* BlockBuilder dummy

* BlockProposerParametersPOS -> BlockBuilderParameters

* Pass tx to MiningStep

* BlockBuilderFunc

* Interrupt in MiningExec Stage

* Draft implementation of BlockBuilder

* Fail back to empty header

* Add a comment

* cosmetic change

* Cosmetic change again

* It's not safe to pass transactions between goroutines
2022-06-13 15:43:09 +02:00
Giulio rebuffo
4b4218d2f6
added licensing (#4422) 2022-06-09 16:57:00 +01:00
Giulio rebuffo
b8e5cbf820
fixed timeout (#4419) 2022-06-09 13:16:11 +02:00
Alex Sharov
0e142e33a0
Reclaimable space in logs (#4417)
* save

* save

* save

* save

* save

* save

* save
2022-06-09 11:47:38 +01:00
Giulio rebuffo
07df1eb598
Fix mining PoS for Deleting of state entries (#4399)
* fixed delete hits

* added tests
2022-06-07 17:39:19 +01:00
Giulio rebuffo
a4a466bb75
Fixed Last in the mining mutation (#4397)
* removed left over from previous mining algo

* added ethereum database

* test improvement
2022-06-07 15:13:02 +01:00
Giulio rebuffo
135fe6f0a6
removed left over from previous mining algo (#4377) 2022-06-06 08:35:53 +07:00
Andrew Ashikhmin
45cf6ee978
Fix LatestValidHash nil pointer (#4338) 2022-06-02 16:35:42 +02:00
Giulio rebuffo
40230d5ab5
Refactor mining batch (#4322)
* refactooor

* updated ethdb

* more fixing

* more fixing

* more

* refactoor

* fixed mdbx crash
2022-06-01 22:48:42 +01:00
Giulio rebuffo
576d736570
Added first prototype of mining mutation (#4266)
* added first prototype of mining mutation

* tests and lint

* fixed some stuff

* more progress

* bug fixing

* more refactoring

* account only modifications are allowed

* Added account trie memory calculation

* SeekBothRange adjustments

* more on both range

* finished

* lint

* fix account tries

* fixes

* more fixes to storage tries

* more fixes

* just saving progress

* better version

* more progress

* done please

* lint
2022-06-01 01:26:08 +02:00
Andrew Ashikhmin
4d2b80639b
Update Engine API errors (#4318) 2022-05-31 15:06:00 +02:00
Andrew Ashikhmin
4b117e600d
LatestValidHash should be 0x00... instead of nil if TTD not reached yet (#4314) 2022-05-31 12:33:15 +02:00
Alex Sharov
fc1a37e6dc
create turbo/services pkg (#4269) 2022-05-26 10:31:06 +07:00
ledgerwatch
8ea2bb55d0
Prevent deletion of extra block worth of tx lookup records when pruning (#4232)
* Print

* Prevent underflow, return 0 block if tx lookup failed

* Return nil receipt for 0 block

* Update to main erigon-lib

* Fix other places

* Fix other places

* Comment

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-22 20:35:02 +01:00
Giulio rebuffo
52c4846b26
deprecated old status (#4196) 2022-05-18 21:33:35 +02:00
Giulio rebuffo
1e3ad39202
added FCU hash on proposing (#4193) 2022-05-18 09:40:41 +02:00
Giulio rebuffo
e383aae147
fixed proposing on invalid fcu (#4189) 2022-05-17 22:20:57 +02:00
Giulio rebuffo
1a811f0aa1
added latest valid hash (#4188) 2022-05-17 21:25:39 +02:00
Giulio rebuffo
12829209c0
fixed bug in proposing (#4184) 2022-05-17 18:57:32 +02:00
Giulio rebuffo
23b648d7ce
ops (#4183) 2022-05-17 17:44:25 +02:00
Giulio rebuffo
01a98a4582
fixed finalized portion (#4181) 2022-05-17 15:26:47 +02:00
Håvard Anda Estensen
0a75065e7a
Use go:build lines and remove obsolete +build lines (#4175) 2022-05-17 14:46:53 +07:00
leonardchinonso
3b87f8c726
Fix disabled value for --prune.* flags (#4165)
* Changes:
* Proceeded to check if other h, r, t or c flags for 'before' and 'older' are set before returning from the default state of --prune
Concern:
* Does the 'disabled' value for '--prune=' need to be updated if the '--prune.*' flags are set but the '--prune=' flag is not?

* Modify mode.Initialised only when it is changed

* Updated layout for --prune=*

* save

* save

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-05-17 09:31:14 +07:00
Giulio rebuffo
e8a1e64df6
Revert "fixed finalized portion (#4170)" (#4171)
This reverts commit 46f9411e79.
2022-05-16 23:22:16 +02:00
Giulio rebuffo
46f9411e79
fixed finalized portion (#4170) 2022-05-16 23:21:07 +02:00
Alex Sharov
e1dec529d4
Generic sort of slices (no allocs, inlinable) (#4161)
* save

* save

* save

* save

* save
2022-05-16 08:24:12 +01:00
Alex Sharov
f590bf682f
RPCDaemon - open snapshots only when they are ready (and indices ready). 2022-05-04 12:55:58 +07:00
Alex Sharov
76c36c2c09
typed sender (#4016)
* save

* save
2022-04-28 15:35:14 +07:00
Alex Sharov
352869fc3a
rename field "type" (#4015)
* save

* save

* save
2022-04-28 15:02:06 +07:00
battlmonstr
fc3eecae5e
RPC: admin.peers() (#3960)
* RPC: admin.peers()

This RPC method returns information about the connected remote nodes.
https://geth.ethereum.org/docs/rpc/ns-admin#admin_peers

The peers are collected from all configured sentries.
See: https://github.com/ledgerwatch/interfaces/pull/102

Test with:
curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc": "2.0", "method": "admin_peers", "params": [], "id":1}' localhost:8545

* save

* liner fix

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2022-04-25 14:40:04 +01:00
Giulio rebuffo
5a805abdd4
revert (#3908) 2022-04-18 08:55:30 +07:00
Giulio rebuffo
94e21d0c41
done (#3889) 2022-04-16 17:57:39 +02:00
Giulio rebuffo
ef14176007
Caching Least Recently Used Contract Code in Stage Execution (#3887)
* caching gets

* whitelist

* fix

* less ram

* comment
2022-04-14 23:34:03 +02:00
Giulio rebuffo
e9dba096b1
fixed SIGINT during stage execution in hashmaps (#3849) 2022-04-08 10:24:27 +07:00
alex.sharov
d259bb2d77 return err on interruption 2022-04-06 20:25:53 +07:00
Andrew Ashikhmin
634e9e4f3c
Prohibit EIP-2718 txn wrapped as RLP strings (#3827) 2022-04-06 10:40:44 +02:00
Andrew Ashikhmin
44ece33dff
Check TerminalTotalDifficulty for NewPayload & ForkChoice (#3826)
* Check TerminalTotalDifficulty for NewPayload & ForkChoice

* Fix copy-paste leftover

* forkChoiceMessage -> forkChoice
2022-04-05 17:23:17 +02:00
Giulio rebuffo
1b0d7c80bf
Better batches for Execution (Hashmaps based instead of BTree based) (#3814) 2022-04-04 14:11:59 +07:00
ledgerwatch
9ea6398524
Fixes to subscribe logs (#3769)
* Fixes to subscribe logs

* Add criteria to logs subscription

* Skeleton of RPC daemon event log distribution

* Simplify

* Send aggregated filter to Erigon

* Change API

* Print

* Fixes

* Fix topics filtering

* Fill txHash and blockHash

* Timing logs, fill tx index

* Print

* More print

* Print

* Asynchronous sending of log events to RPC daemon

* Remove prints

* Only extract logs if there are subscribers

* Check empty when RPC daemon is removed

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-26 18:21:31 +00:00
Alex Sharov
a357421719
Snapshots: notify rpcdaemon about new snapshot (#3755) 2022-03-24 09:25:38 +07:00
Giulio rebuffo
b7fb85df6a
demoted logs (#3753) 2022-03-23 10:28:32 +01:00