Commit Graph

2696 Commits

Author SHA1 Message Date
ledgerwatch
128f446e7c
Download v2 fixes (#1903)
* [download.v2] Fix endianness of Difficulty in Status message

* Add comments

* Finish stage to use RwTx

* Print header age in stage 1

* Print new block messages

* TopSeenHeight to be updated from any header

* block in the future handling and skip the test

* Fix downloader test

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-09 08:22:08 +01:00
ledgerwatch
5c512fbddb
Trace filter 2 (#1898)
* Fix error message about write protection

* Clean up of trace_filter

* Add CallTraces to Execution Stage

* Fix compile

* Say something

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 16:17:17 +01:00
ledgerwatch
4a214d38e4
Fix integration after snapshot PR (#1901)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 11:42:34 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync (#1899) 2021-05-08 00:07:49 +03:00
ledgerwatch
fe0a882a45
Trace filter (#1895)
* Trace filter

* Fix self-destruct traces

* Update ci.yml

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-07 10:05:47 +01:00
Alex Sharov
80ee37540e
penalize for abandoned anchor (#1896) 2021-05-07 06:17:40 +01:00
Alex Sharov
06cec6d535
Move exec to tx (#1887) 2021-05-07 09:06:44 +07:00
Alex Sharov
b2f424e2cd
Canonical fix in --download.v2 (#1886) 2021-05-07 09:06:00 +07:00
ledgerwatch
59e5502744
Initial work on RPC streaming (#1880)
* Initial work on RPC streaming

* Create the stream and pass it on

* trace_filter expressed as streamable (not finished)

* Reenable call trace index

* Add new line

* Remove storage mode override

* Tool for overriding storage mode

* Diagnostics

* Make trace_filter more streamy

* Streaming debug_traceTransaction and debug_traceCall

* Fix test

* Log error on stream flush

* Enable streaming for http

* Flush the stream too

* Make trace_filter flush too

* Stop streaming if client is not interested

* Try to quiet annoying test|

* Revert "Try to quiet annoying test|"

This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.

* Debug log for test

* Proceed with handshake regardless of whether peer notification worked

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 18:37:38 +01:00
Alex Sharov
4323d56376
Mdbx tx limit metric (#1891) 2021-05-06 17:02:50 +07:00
Alex Sharov
49225f7319
Mdbx page ops metrics (#1885) 2021-05-06 10:51:49 +07:00
Evgeny Danilenko
871bb912b1
[WIP] FuzzyVM for TG (#1884)
* restore StateRoot

* empty error should be ommited

* calc state root

* logger json remove code duplication

* fix
2021-05-05 21:55:15 +01:00
Alex Sharov
a627bc2d14
use same keepalive settings in sentry and rpcdaemon (#1883)
* use same keepalive settings in sentry and rpcdaemon

* use same keepalive settings in sentry and rpcdaemon
2021-05-05 11:32:04 +01:00
Alex Sharov
e87d4932f7
Changeset writer move to tx (#1871) 2021-05-05 14:31:40 +07:00
ledgerwatch
7c60ec3ec0
Fix for eth_getTransaction... returning tx from the pool (#1881)
* Fix for eth_getTransaction... returning tx from the pool

* Fix

* Fix more issues

* Fix for EOF

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 22:58:13 +01:00
ledgerwatch
d604bfbb4f
Fix transaction stream test (#1879)
* Fix transaction stream test

* Fix

* Update events.go

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-04 17:12:59 +01:00
Alex Sharov
c7eedf0091
Add kv to stage settings (because it doesn't change in runtime, tx will be passed as Stage func argument) (#1872)
* add kv to config

* move forward tx use

* move forward tx use

* gc stat

* gc stat

* changeset writer - move to tx

* clean

* clean

* clean

* clean

* clean
2021-05-04 13:36:03 +01:00
ledgerwatch
ea2014f5c6
Making trace_filter work in first approximation with the new call trace index (#1877)
* Fix RPC daemon

* Initial fixes for trace_filter

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 13:34:17 +01:00
Artem Vorotnikov
2148885637
Simplify ChangeSet implementation and tests (#1878)
* Move storage changeset code into one file

* Simplify storage changeset tests

* simplify more

* more

* more

* more

* move account changeset into one file

* simplify account changeset test

* NewStorageChangesetPlain -> NewStorageChangeSet

* EncodeStoragePlain -> EncodeStorage

* StorageChangeSetPlain -> StorageChangeSet

* AccountChangeSetPlain -> AccountChangeSet

* more

* more

* more

* Simplify EncodeStorage

* more

* Simplify FromDBFormat

* more rename

* rename changeset buckets (constant names only)
2021-05-04 13:34:08 +01:00
ledgerwatch
99578ec58d
Fix RPC daemon (#1873)
* Fix RPC daemon

* Skip the test to fix RPC daemon

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 12:31:07 +01:00
ledgerwatch
793c13e20e
trace_filter and CallTrace Index (derivative of #1626) (#1867)
* Trace Index

* trace_filter

* hrtc -> hrt

* Fix CallTraces

* wip

* wip

* Fix for incorrect gas

* fix

* Fix Tracer

* Add block and uncle coinbases to trace index

* rewind coinbase

* Commit every 100k blocks after block 3m

* Simplify

* Safe unwinds

* Safe unwind 2

* Cleanup

* Simplification

* Remove intermediate JSON transform

* Reinstate optional CallIndex stage

* Create rpctest bench for trace_filter

* Fix trace_filter generator

* Restore unwind order

* Ignore the storage mode

* Ignore the storage mode

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 07:23:54 +01:00
Alex Sharov
fc196d4b9b
pending txs methods (#1869) 2021-05-04 06:51:28 +01:00
Alex Sharov
f3d2af1e33
Rpcdaemon: move tx pool to own grpc service, subscribe to pending txs (#1863) 2021-05-04 08:37:17 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods (#1861)
* less use deprecated methods

* less use deprecated methods

* auto Dp limit

* auto Dp limit

* add test
2021-05-03 21:01:01 +01:00
ledgerwatch
5168287784
Make trace_block results closer to what OpenEthereum returns (#1862)
* Add block rewards to trace_block

* Add rewards to trace_block

* Remove printouts

* Fix trace_block and trace_transaction

* Fix getBlockByNumber

* Fix for parent/non parent block

* Reverse fix for trace_call

* Fix eth_getTransactionBy

* Fixes for TIMESTAMP etc opcodes

* More fixes

* Fixes to tracers

* Don't call CaptureEnd twice

* Corrent gasUsed for CaptureEnd in create

* Do CaptureFault consistently

* Correct gasUsed for create

* Remove insufficient balance trace

* Catch contract collision error

* Compatibility

* Compatibility

* More error names

* Out of gas

* Clean up

* more error messages

* Restore CaptureFault

* Errors

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-03 20:49:55 +01:00
ledgerwatch
4432618cb9
Support for Baikal devnet (#1853)
* EIP-3529

* Test fix and EIP-3541

* Fix tests

* Fix

* Fix stagedsync test

* Fix

* Remove dup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 10:34:19 +01:00
Alex Sharov
7eb088e862
Download request more headers on ext (#1858)
* use lru

* add comment

* add comment
2021-05-02 08:12:17 +01:00
ledgerwatch
4303465e96
Address dump utility (#1856)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-01 21:04:02 +01:00
Alex Sharov
08f6299257
broadcast new headers (#1847)
* broadcast new headers

* don't use eth type

* don't use eth type

* add seenAnnounces scet

* use lru
2021-05-01 21:03:51 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
ledgerwatch
34c1fdac21
Aleut fix for GASPRICE opcode (#1852)
* Aleut fix for GASPRICE opcode

* Fix for tip+basefee > feecap

* Unbork

* Fix tests

* Remove setHead test

* Test fix

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-30 19:20:18 +01:00
Alex Sharov
249b3098cb
add --sentry.api.addr flag (#1850)
* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr
2021-04-30 16:09:03 +01:00
ledgerwatch
75ca6b8c76
Initial work on integration tests (#1797)
* Initial work on integration tests

* Delete subtree

* Squashed 'interfaces/' content from commit 41a082ba4

git-subtree-dir: interfaces
git-subtree-split: 41a082ba4bde38647325eb0416cb1da1b4ca2b12

* Add consensus interfaces

* More stuff

* comments

* Fix compile

* Squashed 'interfaces/' changes from 41a082ba4..1b13a42a7

1b13a42a7 Add chainspec to consensus interface

git-subtree-dir: interfaces
git-subtree-split: 1b13a42a7803f5464722867a71065c27a7ebe8c3

* Squashed 'interfaces/' changes from 1b13a42a7..93a072c4c

93a072c4c Add missing import

git-subtree-dir: interfaces
git-subtree-split: 93a072c4c099d169322a3a53b95e40203276820b

* New consensus interfaces

* More on clique

* Fix tests

* Squashed 'interfaces/' changes from 93a072c4c..62f4ec4b2

62f4ec4b2 Add test service for consensus engine

git-subtree-dir: interfaces
git-subtree-split: 62f4ec4b263107635ffa3aabd5d634af22e813c6

* Squashed 'interfaces/' changes from 62f4ec4b2..061a63543

061a63543 Fix

git-subtree-dir: interfaces
git-subtree-split: 061a63543babdeb51ab7e3a96dec56b2485d4389

* Configuring clique engine with toml specs - start

* More toml parsing

* Constructed rinkeby genesis

* Simplify VerifyHeaders functions

* Fix lint

* Remove concurrent verification tests

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-29 16:14:10 +01:00
Thomas Jay Rush
711f157cf5
Re-enable trace_transaction and trace_get - issue 1833 (#1835)
* Fixes trace_transaction to use new OETracer from trace_block

* Fixes trace_get to use OETracer from trace_block

* Trace only up to txIndex as per suggestion in trace_transaction

* Cleaning up a few old comments
2021-04-29 05:54:48 +01:00
ledgerwatch
6cbfff354d
[download-v2] Use Clique instead of EtHash for clique networks (#1832)
* Use Clique instead of EtHash for clique networks

* Start tx fetcher, enable GetPooledTx in sentry

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-28 14:44:29 +01:00
ledgerwatch
03beaf4df7
Connect txPool to download.v2 to resolve 1 reason for "frozen download" (#1830)
* Pass txPool into new staged sync

* Remove poolStart function

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-27 16:37:03 +01:00
Alex Sharov
9cc4a0a97c
rebort db metrics - not guilty (#1828) 2021-04-27 15:32:41 +07:00
ledgerwatch
e8161541e2
Use RLP encoding instead of json for subscriptions (#1825)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-27 09:23:04 +01:00
ledgerwatch
6b84a92b7d
Fix nil pointer dereference when subscribing to pending txs (#1823)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-26 18:38:20 +01:00
ledgerwatch
b341b5aed5
Fix for RPC daemon crashing TG when pending txs are transmitted (#1819)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 17:56:02 +01:00
ledgerwatch
04bbd07749
Move sentry message to log.Debug for download.v2 mode (#1817)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 15:40:46 +01:00
Alex Sharov
01ae962459
Clean tmp on start (#1808)
* fininish to remove chain context

* clean

* move migrations folder from
<datadir>/etl-tmp/migrations
to
<datadir>/migrations
2021-04-26 13:51:01 +01:00
ledgerwatch
117838e213
Move compatibility check (#1814)
* Move compatibility check

* Implement Alex's suggestion

* Use root context

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 13:39:34 +01:00
Alex Sharov
544c597151
Pending txs part 2 (#1816) 2021-04-26 18:15:05 +07:00
Alex Sharov
4e80d6eeed
Pending txs subscription (#1810) 2021-04-26 16:53:38 +07:00
Alex Sharov
c40c3b7a7a
exec unwind etl (#1806) 2021-04-26 10:41:31 +01:00
Alex Sharov
472e161e33
fininish to remove chain context (#1807) 2021-04-26 14:23:21 +07:00
Alex Sharov
eee9438625
deprecate PlainDbState in favor of PlainKvState (#1805) 2021-04-26 12:37:48 +07:00
Alex Sharov
3462059ae8
mdbx rpcdaemon support, mdbx - switch to master branch (#1804) 2021-04-26 12:00:20 +07:00