Commit Graph

2941 Commits

Author SHA1 Message Date
Alex Sharov
623a3feeee
RPC: return all stages progress inside eth_syncing (#2353) 2021-07-13 09:17:05 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00
Alex Sharov
b38d3fc34b
don't print peer logs on shutdown (#2341) 2021-07-11 16:16:17 +07:00
Alex Sharov
188dfb14b8
txPool: propagate on peer connect (#2335) 2021-07-11 14:01:16 +07:00
Alex Sharov
e1c17e035a
rawdb.TxLookup - to return err, remove docs of rpcdaemon dual mode (#2340) 2021-07-11 12:25:21 +07:00
Alex Sharov
5eaf4c8cc3
rpc: 1559 in call and tracing (#2327) 2021-07-11 04:05:56 +00:00
Alex Sharov
2ec64aee1d
Simplify mining (#2339)
* save

* save
2021-07-11 04:05:33 +00:00
Alex Sharov
43af5d42b8
simplify staged sync world (#2336)
* simplify world

* simplify world

* simplify world
2021-07-10 16:43:58 +07:00
Alex Sharov
e98340d806
Simplify stagedsync.Prepare (#2317) 2021-07-08 20:52:22 +07:00
Alex Sharov
7c60cc0fee
step towards kv interface (#2313) 2021-07-08 09:56:09 +07:00
Alex Sharov
a78cb54132
step towards kv interface (#2311) 2021-07-07 23:15:49 +07:00
Alex Sharov
360735318b
rpc: add --rpc.batch.concurrency flag (#2308) 2021-07-07 10:48:21 +07:00
b00ris
57473175ff
Body snapshot (#2100)
* fix test

* get rid of ObjectDatabase

* sn_builder_prototype2

* save state

* save state

* integration step1

* fix lint

* fix

* fix test

* integrate migrator.finish

* fix lint

* fix build

* fix typo

* save state

* body snapshot test

* unique tx

* body snapshot generation using walk

* move methods out of test

* block data verification added

* fix lint

* test with remove works correctly

* fix lint

* remove experiment test

* fix test

* add comment

* add second layer of remove test

* rename test

* fix typos

* fix lint

* revert testdata

* body snapshot migration save state

* fix body snapshot migration

* fix after merge

* remove debug test

* debug windows build

* fix build

* fix

* fix lint

* debug

* fix

* fix windows build

* simplify snapshot management&&get rid of lazy tx

* fix lint

* fix windows path

* debug

* debug

* debug

* debug

* remove geometry experiments

* skip windows tests

* clean

* fix

* fix ;int
2021-07-06 23:33:26 +01:00
Alex Sharov
a68b5ba361
Replace ChainConfig.WithEIPsFlags by go-ethereum's ChainConfig.Rules (#2304)
* use chainRules

* use chainRules

* use chainRules

* use chainRules

* use chainRules
2021-07-05 19:52:50 +01:00
ledgerwatch
9023f2a29c
Investigate slow receipt repair (#2306)
* Collect data about broken receipts

* Actually collect data

* Fix CBOR problem more efficiently

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-05 19:14:58 +01:00
Alex Sharov
9683b26483
remove ctx from state writer interface (#2302) 2021-07-05 15:42:44 +07:00
ledgerwatch
63ccabc72f
Receipt repair migration (#2291)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

* Better mechanism for detecting broken receipts

* Fixes

* Introduce receipt_repair migration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-04 14:48:13 +01:00
Alex Sharov
afd8674c4b
Peer log (#2294)
* Revert "Revert "better handshake""

This reverts commit 4141c54c95.

* better peers logs

* better peers logs
2021-07-04 08:51:08 +01:00
Alex Sharov
ba902f7ef3
separate_config_from_node_and_eth (#2289) 2021-07-04 08:50:32 +01:00
Alex Sharov
3db7e85994
Simplify GetAsOf (#2285)
* no reason to copy if we work with ethdb.Tx (user of ethdb.Tx expecting that all data valid until end of tx)

* less objects
2021-07-04 08:49:31 +01:00
Alex Sharov
4baad9f4b9
Sentry: better handshake (#2293) 2021-07-04 11:30:31 +07:00
ledgerwatch
f5e42b1e7a
Fixes in rpctest and rpcdaemon for debug_traceTransaction, and opcode tracer, tool for repairing broken receipts (#2284)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-03 14:34:23 +01:00
Alex Sharov
aa70f3c5a8
speedup tracing by not recalculating senders (#2286) 2021-07-03 13:55:14 +01:00
Alex Sharov
6a5f86423c
Remove "numgc" from logs (#2275) 2021-07-03 14:44:23 +07:00
Alex Sharov
b9be0cadad
don't stop iterate over peers on error (#2274) 2021-07-03 08:03:27 +01:00
ledgerwatch
ba55611f29
rpctest for eth_getTransactionReceipt (#2271)
* rpctest bench for eth_getTransationReceipt

* Add blockHash to re-generated receipts

* Modify scanReceipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-02 14:34:20 +01:00
Alex Sharov
c12ef4478b
don't call http handler after ws handler (#2267) 2021-07-02 11:10:54 +01:00
ledgerwatch
a05d1be531
Pull grpc bindings into erigon-lib (#2265)
* Pull grpc bindings into erigon-lib

* Remove mdbx dist

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-01 22:31:14 +01:00
Alex Sharov
ea81761991
enable private rpc by default in Erigon, better logs in the beginning (#2262) 2021-07-01 22:30:55 +01:00
Zhengyan Gao
f4ef314e50
add GasPrice for the DynamicFeeTransaction. (#2261) 2021-07-01 11:29:32 +07:00
Alex Sharov
bfe4d97fae
fix_receipt_status (#2250) 2021-06-30 12:56:00 +01:00
Alex Sharov
6df7230108
Sokol v0 - better header rlp, steps to generate blocks (#2256) 2021-06-30 16:30:21 +07:00
Alex Sharov
3d6d45a82f
remove subtrie loader and dcmp (#2258) 2021-06-30 10:31:36 +07:00
Zhengyan Gao
2fb9b2ebc8
add effectiveGasPrice field for GetTransactionReceipt, GetBlockReceipts (#2252) 2021-06-30 09:50:39 +07:00
Alex Sharov
8f6aa2f7ae
propagate new tx to p2p (#2254) 2021-06-29 17:00:22 +07:00
Alex Sharov
a6f4b08716
Revert "Propagate new txs to P2P (#2249)" (#2253)
This reverts commit e9ae6d688f.
2021-06-29 15:46:00 +07:00
Alex Sharov
e9ae6d688f
Propagate new txs to P2P (#2249) 2021-06-29 14:40:23 +07:00
Alex Sharov
96ecce60e4
ChainID alias for ChainId (#2248) 2021-06-28 18:36:07 +07:00
Andrea Lanfranchi
bc16c0127f
Update README.md 2021-06-28 12:55:14 +02:00
Alex Sharov
f30c3b8951
Rpcdaemon help add namespaces (#2244)
* add namespaces list to rpcdaemon help

* add namespaces list to rpcdaemon help

* add namespaces list to rpcdaemon help
2021-06-27 22:14:14 +01:00
Zhengyan Gao
31cd8afe98
update tx block base fee, tips,fee cap json fields. (eip1559) (#2237)
* update tx block base fee, tips,fee cap json fields. (eip1559)

* gofmt core/types/block.go
2021-06-27 22:14:01 +01:00
Alex Sharov
cba7861164
Send raw tx test (#2234) 2021-06-27 13:41:21 +07:00
ledgerwatch
7baf3ec484
Move secp256k1 into a separate go module, hook up erigon-lib (#2239)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-26 15:03:05 +01:00
Alex Sharov
0edac73451
RPC: trace_replayTransaction and trace_replayBlockTransactions methods (stateDiff only) (#2238)
* trace_replay

* trace_replay

* trace_replay

* trace_replay

* trace_replay
2021-06-26 13:27:29 +01:00
Alex Sharov
605d7a8d59
Sokol v0 (#2228)
* handle chain flag in integration

* handle chain flag in integration

* handle chain flag in integration

* save

* save

* merge devel

* save

* noopWriter - one is enough

* chain spec parser

* chain spec parser

* embed

* embed

* embed

* embed

* embed

* embed

* embed

* clean

* clean

* correct alloc after reset state

* correct alloc after reset state

* correct alloc after reset state

* integration reset state now does re-apply genesis and chainConfig

* eips summary

* eips summary

* eips summary
2021-06-25 19:13:40 +01:00
Alex Sharov
d8608ac22e
simplify eth_callBundle (#2233)
* simplify eth_callBundle

* simplify eth_callBundle
2021-06-25 17:25:12 +01:00
Roman Koblov
18fa156d92
Make default gas for eth_call to be maximum gas capacity instead of 0 (#2232)
* Make default gas for eth_Call to be maximum gas capacity instead of 0

* Fix nil pointer dereference
2021-06-24 15:48:54 +01:00
ledgerwatch
adf4766191
Scan transaction to select test cases (#2225)
* Scan txs

* Find examples with access lists

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 23:34:17 +01:00
ledgerwatch
876dde506f
Fix range for eth_getLogs (#2227)
* Fix range for eth_getLogs

* Use Or and measure time

* more measurement

* more measurement

* Not compute tx hashes

* Not compute tx hashes

* Not compute tx hashes

* filter logs before derive fields

* Fix filtering by block number

* fix

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 23:33:45 +01:00
ledgerwatch
6292056503
Remove root from eth_getTransactionReceipt (#2211)
* Remove root from eth_getTransactionReceipt

* Simplifaction of status

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-23 15:52:31 +01:00
ledgerwatch
5ad449ef22
Migration to fix receipts (broken due to change of CBOR codec) (#2221)
* Scan receipts

* Not check tx type before Berlin

* Reverse

* scan receipt code in the migration

* Turn migration on

* Apply migration

* Fix

* Print last

* Remove last

* Not remove last

* Not updata in migration

* no cursors in migration

* Handle empty blocks

* Handle empty blocks

* Use the same code in hack

* Print

* Print

* Save

* Fix

* Fix lint

* Fix lint

* Fix for empty DB

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 15:05:16 +01:00
Alex Sharov
59d05dc5fe
hide file exists err (#2218) 2021-06-22 11:09:45 +01:00
Alex Sharov
2b59b45216
Do log tables size, db metrics - avoid concurrency, check stale readers hourly (#2216)
* add table size logs and metrics

* log stale readers

* - don't collect db metrics in background
- do check stale readers once an hour

* execution to update metrics when print logs

* hide file exists err

* hide file exists err
2021-06-22 11:08:47 +01:00
ledgerwatch
ab685e90d9
Improvement to the trimTxs (#2215)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-22 08:35:34 +01:00
ledgerwatch
a79ed2fe0b
Utilities to fix DB corruption (#2214)
* Detect broken ancestry

* Print last canonical headers

* Add backExec

* Cleanup

* Add unwind util

* Add trimTxs utility

* Actually delete tx records

* Avoid infinite loop

* Change strategy

* Close cursor after commmit

* Fix RwBegin issue

* 10m records per transaction

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 21:52:51 +01:00
Alex Sharov
7d1e17712f
don't create block context twice (#2213) 2021-06-21 19:54:41 +01:00
ledgerwatch
04943eaa52
Fix for state corruption without performance regression (#2208)
* extractHeaders

* Print diff

* Fix for state corruption without performance regression

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 06:56:45 +01:00
Artem Vorotnikov
d8a009837b
Nuke preimage remnants (#2203) 2021-06-20 13:00:22 +07:00
Alex Sharov
a535e8f30c
switch to mdbx-go module, and db-tools to libmdbx submodule (#2199)
* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule
2021-06-19 21:30:12 +01:00
Alex Sharov
4736bac190
save (#2197) 2021-06-19 21:29:02 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Alex Sharov
de24b427bf
save (#2196) 2021-06-19 13:32:35 +07:00
mempirate
ca926ce94d
net_peerCount for remote sentries (#2192)
* net_peerCount for remote sentries

* cleanup

* Simplify NetPeerCount using SentryClient

* Refactor id -> count
2021-06-18 14:34:15 +01:00
ledgerwatch
535ce8fb9c
Try to solve sentry hanging problem + p2p impovement from geth (#2191)
* Try to solve sentry hanging problem + p2p impovement from geth

* Fix import

* Finer stream lock

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-18 12:47:46 +01:00
Alex Sharov
e9f510c2b7
erigon as lib (#2186) 2021-06-18 10:35:11 +07:00
mempirate
e51b09da0d
Implements net_peerCount RPC call for internal sentries (#2172)
* Implement API and protobuf

* Squashed 'interfaces/' content from commit cd3eca28e

git-subtree-dir: interfaces
git-subtree-split: cd3eca28e50367af9de52fca863b1d7528bff774

* Implement net_peerCount RPC call

* Squashed 'interfaces/' content from commit cd3eca28e

git-subtree-dir: interfaces
git-subtree-split: cd3eca28e50367af9de52fca863b1d7528bff774

* Fixing merge conflicts

* Remove p2p.Server from Ethereum backend

* Fix peerCount calculation for internal sentries

* Bump eth backend API version

* Update rpcdaemon README

* Document version change
2021-06-17 22:55:20 +01:00
ledgerwatch
9d03cd4bf4
trace compatibility with OE (sometimes bug for bug) (#2174)
* Initial investigations

* Add compatibility option, cleanup

* More cleanup

* Better error handling for streaming RPC requests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-16 18:24:56 +01:00
ledgerwatch
8a166204b4
Fix for peer not found messages (#2181)
* Fix for peer not found messages

* Print proto on disconnects too

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-16 17:18:19 +01:00
Alex Sharov
f63d12405a
Sender stage hangs (#2177)
* - add context for goroutines shutdown
- use 1 select for all channels to avoid deadlock

* - add context for goroutines shutdown
- use 1 select for all channels to avoid deadlock
- integration: senders to use right chainconfig
2021-06-16 14:04:44 +01:00
Artem Vorotnikov
af836a6200
Nuke LMDB (#2167) 2021-06-16 13:57:58 +03:00
Alex Sharov
372ab9060a
add chain support to (#2176) 2021-06-16 11:31:25 +07:00
Alex Sharov
dcf850dd75
fix fieldalignment linter warnings (#2171) 2021-06-16 11:00:35 +07:00
Artem Vorotnikov
9a8c0f9669
Always enable eth65 (#2166)
* Always enable eth65

* Add ListenAddr65 to defaults
2021-06-15 17:57:26 +01:00
Alex Sharov
b160e30652
Send GetBlockHeaders immediately if TD of connected peer >= our TD (#2169)
* empty engine

* start sync from peer right after handshake if it's TD is higher

* fixCanonicalChain to print logs

* clean

* >
2021-06-15 13:43:55 +01:00
Artem Vorotnikov
0a4e0d1fa1
Various code simplificaitons (#2165)
* simplify randomaccount

* simplify execution unwind

* Remove standalone account accessors
2021-06-14 19:52:30 +01:00
ledgerwatch
2c1fe2a3a7
Restore TD records (#2164)
* Insert missing TD values

* Print missing TD records

* Fix typo

* Apply fix

* Fixes in execution stage

* Fix tx/sec output

* Advance exec

* Reset unwind

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-14 19:51:57 +01:00
Alex Sharov
b79823d1b9
create dir for node keyfile (#2161) 2021-06-14 07:35:22 +01:00
ledgerwatch
0f64df626a
Util to dump state (#2159)
* Util to dump state

* Extract bodies info

* Fix for extractBodies

* Fix typo

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-14 07:34:36 +01:00
mempirate
89edfe39db
Add --maxpeers cli option (#2150) (#2158)
* add --maxpeers cli options (#2150)

* cleanup
2021-06-13 21:14:52 +01:00
BitBaseBit
7ed337cdcc
Implemented panic handling, graceful shutdown and reporting for all goroutines that don't explicitly handle them. (#2153)
* implemented crash reporting for all goroutine panics that aren't handled explicitly

* implemented crash reporting for all goroutine panics that aren't handled explicitly

* changed node defaults back to originals after testing

* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports

* handling panics on all goroutines gracefully

* updated missing call

* error assignment

* implemented suggestions

* path.Join added

* implemented Evgeny's suggestions

* changed path.Join to filepath.Join for cross-platform

* added err check

* updated RecoverStackTrace to LogPanic

* updated closures

* removed call of common.Go to some goroutines

* updated scope capture

* removed testing files

* reverted back to original method, I feel like its less intrusive

* update filename for clarity
2021-06-13 17:41:39 +01:00
ledgerwatch
74862ce607
Upgrade tests to 9.0.2 (#2147)
* Upgrade tests to 9.0.1

* Fix gas limit rule

* Clean up output

* Update testdata

* Fix tip assigment

* Fix genesis discrepancy

* Skip typeTwo berlin test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-13 14:01:11 +01:00
Alex Sharov
925d17a238
remove lmdb from hack, integration, state packages. remove lmdb dbtools (#2146) 2021-06-11 20:31:37 +07:00
ledgerwatch
d1e4edb180
Better compatibility with OE for trace_call and trace_callMany (#2137)
* Better compatibility for trace_call

* Also for trace_callMany

* non canonical hash error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-11 13:19:10 +01:00
Marin Ivanov
5226d98379
Add ws.compression flag to rpcdaemon (#2143)
* Add ws.compression flag to rpcdaemon

* Fix WebSocketHandler in tests
2021-06-11 10:21:39 +01:00
Alex Sharov
e1ad20460c
Less obj db in snapshot (#2142)
* less obj db in snapshot

* less obj db in snapshot

* save
2021-06-11 09:35:26 +01:00
Alex Sharov
eedabfca85
txpool_content (#2127)
* txpool_content

* txpool_content

* txpool_content

* txpool_content

* Squashed 'interfaces/' content from commit 1e916d013

git-subtree-dir: interfaces
git-subtree-split: 1e916d0137aff7bd9e9356cc203cea5f0ea9c7f1

* add test

* add test

* add test

* merge devel

* save
2021-06-11 09:34:47 +01:00
Alex Sharov
ba56e7a139
extract p2p config out of sentry (#2126)
* extract p2p config out of sentry

* extract p2p config out of sentry

* extract p2p config out of sentry

* rebase devel

* rebase devel

* add bootnodes to sentry

* add bootnodes to sentry
2021-06-11 09:34:37 +01:00
Evgeny Danilenko
de26634d8d
TEVM status pruning (#2034)
* use trace for TEVM status

* restore testdata

* fix getting contract from plain state

* unwind

* linters

* reproducible build debug

* debug

* debug

* remove debug
2021-06-11 09:34:25 +01:00
Evgeny Danilenko
20ff2c5d65
EVM test deadlock (#2136)
* fix EVM test deadlock

* set stateRoot even in case of an error
2021-06-10 16:45:45 +01:00
Alex Sharov
77aba80d8b
fix p2p nodeName (#2129) 2021-06-10 08:53:06 +07:00
Alex Sharov
de7d5a94f6
Update Readme.md 2021-06-10 07:50:20 +07:00
Alex Sharov
8abf0317c9
Update Readme.md 2021-06-10 07:49:02 +07:00
ledgerwatch
aae8fddaee
Update README.md (#2125) 2021-06-09 11:20:46 +01:00
Alex Sharov
338fc0df26
move snapshot config to constructor (#2121) 2021-06-08 09:43:41 +07:00
Alex Sharov
40eaf77eb8
Sokol genesis and chainconfig (#2116)
* sokol init

* test

* precompiles

* precompiles

* add code

* sokol test

* save

* save

* save

* save

* save

* save
2021-06-07 12:01:54 +01:00
ledgerwatch
c2c63a3715
Clean up DefaultStages (#2114)
* Clean up DefaultStages

* Fix compile

* Fix compile

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-07 12:00:16 +01:00
ledgerwatch
73bb98f686
Move tests 6 (#2107)
* Move tests 6

* Compile fixes

* Fix lint

* Compile fixes

* Fix compile

* compile fixes

* Compile fix

* Fix consesus/clique

* Cleanup

* Add gas limit

* Print

* Prints

* More print

* Fix

* Reinstate TestGetBlockReceipts66

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-06 22:44:14 +01:00
alex.sharov
3aeb186bd1 db faq about RAM 2021-06-06 20:02:15 +07:00
Alex Sharov
dea4378927
move (#2113) 2021-06-06 18:17:03 +07:00
Alex Sharov
35fd972a9e
move sync.Run() to kv (#2112)
* less objdb in tests

* move sync.Run to tx

* move

* move

* move
2021-06-06 10:32:55 +01:00