Commit Graph

1761 Commits

Author SHA1 Message Date
ledgerwatch
793c13e20e
trace_filter and CallTrace Index (derivative of #1626) (#1867)
* Trace Index

* trace_filter

* hrtc -> hrt

* Fix CallTraces

* wip

* wip

* Fix for incorrect gas

* fix

* Fix Tracer

* Add block and uncle coinbases to trace index

* rewind coinbase

* Commit every 100k blocks after block 3m

* Simplify

* Safe unwinds

* Safe unwind 2

* Cleanup

* Simplification

* Remove intermediate JSON transform

* Reinstate optional CallIndex stage

* Create rpctest bench for trace_filter

* Fix trace_filter generator

* Restore unwind order

* Ignore the storage mode

* Ignore the storage mode

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 07:23:54 +01:00
Alex Sharov
f3d2af1e33
Rpcdaemon: move tx pool to own grpc service, subscribe to pending txs (#1863) 2021-05-04 08:37:17 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods (#1861)
* less use deprecated methods

* less use deprecated methods

* auto Dp limit

* auto Dp limit

* add test
2021-05-03 21:01:01 +01:00
ledgerwatch
39d3314b99
[download.v2] Prevent deadlock by starting txPool early (#1860)
* Prevent deadlock by starting txPool early

* Reduce error levels in the logs

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 12:15:53 +01:00
Alex Sharov
08f6299257
broadcast new headers (#1847)
* broadcast new headers

* don't use eth type

* don't use eth type

* add seenAnnounces scet

* use lru
2021-05-01 21:03:51 +01:00
Alex Sharov
df8c5e3095
All stages - finish fast if current progress > target (#1773) 2021-05-01 17:08:59 +07:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
ledgerwatch
34c1fdac21
Aleut fix for GASPRICE opcode (#1852)
* Aleut fix for GASPRICE opcode

* Fix for tip+basefee > feecap

* Unbork

* Fix tests

* Remove setHead test

* Test fix

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-30 19:20:18 +01:00
Alex Sharov
249b3098cb
add --sentry.api.addr flag (#1850)
* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr

* add --sentry.api.addr
2021-04-30 16:09:03 +01:00
Alex Sharov
c313238e4a
less blockchain objects in tests (#1844) 2021-04-29 16:14:19 +01:00
ledgerwatch
75ca6b8c76
Initial work on integration tests (#1797)
* Initial work on integration tests

* Delete subtree

* Squashed 'interfaces/' content from commit 41a082ba4

git-subtree-dir: interfaces
git-subtree-split: 41a082ba4bde38647325eb0416cb1da1b4ca2b12

* Add consensus interfaces

* More stuff

* comments

* Fix compile

* Squashed 'interfaces/' changes from 41a082ba4..1b13a42a7

1b13a42a7 Add chainspec to consensus interface

git-subtree-dir: interfaces
git-subtree-split: 1b13a42a7803f5464722867a71065c27a7ebe8c3

* Squashed 'interfaces/' changes from 1b13a42a7..93a072c4c

93a072c4c Add missing import

git-subtree-dir: interfaces
git-subtree-split: 93a072c4c099d169322a3a53b95e40203276820b

* New consensus interfaces

* More on clique

* Fix tests

* Squashed 'interfaces/' changes from 93a072c4c..62f4ec4b2

62f4ec4b2 Add test service for consensus engine

git-subtree-dir: interfaces
git-subtree-split: 62f4ec4b263107635ffa3aabd5d634af22e813c6

* Squashed 'interfaces/' changes from 62f4ec4b2..061a63543

061a63543 Fix

git-subtree-dir: interfaces
git-subtree-split: 061a63543babdeb51ab7e3a96dec56b2485d4389

* Configuring clique engine with toml specs - start

* More toml parsing

* Constructed rinkeby genesis

* Simplify VerifyHeaders functions

* Fix lint

* Remove concurrent verification tests

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-29 16:14:10 +01:00
ledgerwatch
6cbfff354d
[download-v2] Use Clique instead of EtHash for clique networks (#1832)
* Use Clique instead of EtHash for clique networks

* Start tx fetcher, enable GetPooledTx in sentry

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-28 14:44:29 +01:00
ledgerwatch
03beaf4df7
Connect txPool to download.v2 to resolve 1 reason for "frozen download" (#1830)
* Pass txPool into new staged sync

* Remove poolStart function

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-27 16:37:03 +01:00
Alex Sharov
9cc4a0a97c
rebort db metrics - not guilty (#1828) 2021-04-27 15:32:41 +07:00
Alex Sharov
9d82456f17
remove chain indexer and core.setHead func (#1826) 2021-04-27 14:48:59 +07:00
ledgerwatch
04bbd07749
Move sentry message to log.Debug for download.v2 mode (#1817)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 15:40:46 +01:00
Alex Sharov
01ae962459
Clean tmp on start (#1808)
* fininish to remove chain context

* clean

* move migrations folder from
<datadir>/etl-tmp/migrations
to
<datadir>/migrations
2021-04-26 13:51:01 +01:00
emhane
4823039a3d
Update README.md (#1815)
Correct typo.
2021-04-26 11:21:51 +01:00
Alex Sharov
4e80d6eeed
Pending txs subscription (#1810) 2021-04-26 16:53:38 +07:00
Alex Sharov
c40c3b7a7a
exec unwind etl (#1806) 2021-04-26 10:41:31 +01:00
Alex Sharov
472e161e33
fininish to remove chain context (#1807) 2021-04-26 14:23:21 +07:00
Alex Sharov
eee9438625
deprecate PlainDbState in favor of PlainKvState (#1805) 2021-04-26 12:37:48 +07:00
Alex Sharov
3af0b9b591
add hashstate, trie, history, logIndex stages (#1799)
* add hashstate and trie stages

* add history stages

* add history stages

* add history stages

* add logindex stage

* add logindex stage

* add other stages
2021-04-25 09:29:39 +01:00
ledgerwatch
88e8828bbd
Not rewind chain on hard forks (#1798)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-25 07:14:10 +01:00
Alex Sharov
d959986466
Connect TxFetcher to Sentry, Add --download.v2 option to TG (#1796) 2021-04-25 11:20:50 +07:00
ledgerwatch
e3f3dd3c9b
Integration tests 1 (#1793)
* Initial commit

* Add sentry gRPC interface

* p2psentry directory

* Update README.md

* Update README.md

* Update README.md

* Add go package

* Correct syntax

* add external downloader interface (#2)

* Add txpool (#3)

* Add private API (#4)

* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Separate upload messages into its own stream (#6)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Only send changed accounts to listeners (#7)

* Txpool interface doc (#9)

* Add architecture diagram source and picture (#10)

* Typed hashes (#11)

* Typed hashes

* Fix PeerId

* 64-bit tx nonce

* Add proper golang packages, max_block into p2p sentry Status (#12)

* Add proper golang packages, max_block into p2p sentry Status

* Change EtherReply to address

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Add Rust infrastructure (#13)

* DB stats methods removed by https://github.com/ledgerwatch/turbo-geth/pull/1665

* more p2p methods (#15)

* add mining methods (#16)

* First draft of Consensus gRPC interface (#14)

* Update Rust build

* Fix interfaces in architecture diagram (#17)

* Fix KV interface provider

* Fix Consensus interface provider

* drop java attributes (#18)

* tx pool remove unused import (#19)

* ethbackend: add protocol version and client version (#20)

* Add missing ethbackend I/F (#21)

* Add interface versioning mechanism (#23)

Add versioning in KV interface

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>

* spec of tx pool method (#24)

* spec of tx pool method (#25)

* Update version.proto

* Refactor interface versioning

* Refactor interface versioning

* Testing interface

* Remove tree

* Fix

* Build testing protos

* Fix

* Fix

* Update to the newer interfaces

* Add ProtocolVersion and ClientVersion stubs

* Hook up ProtocolVersion and ClientVersion

* Remove service

* Add compatibility checks for RPC daemon

* Fix typos

* Properly update DB schema version

* Fix test

* Add test for KV compatibility|

* Info messages about compability for RPC daemon

* DB schema version to be one key

* Update release intructions

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: canepat <16927169+canepat@users.noreply.github.com>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Co-authored-by: canepat <tullio.canepa@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-24 16:46:29 +01:00
Alex Sharov
ac7ece4a4d
Revert txpool grpc (#1794) 2021-04-24 10:48:06 +07:00
Evgeny Danilenko
984daa4d7a
Clique snapshot timer fixed (#1789)
* fix clique snapshot storage by timer

* after merge

* remove debug

* remove debug

* debug logging

* correct t.Fatal
2021-04-23 20:47:06 +03:00
Alex Sharov
d7d790725b
Add stage exec to downloader (#1788) 2021-04-23 17:27:07 +07:00
Alex Sharov
10c41491fa
Use Config struct for headers/body/senders stages. Add StageSenders to new downloader. (#1787)
* config for headers/body/senders

* clean
2021-04-23 16:16:45 +07:00
ledgerwatch
12cde41772
Aleut support (Eip1559) (#1704)
* Where I am at

* Refactoring of transaction types

* More refactoring

* Use Homested signer in rpc daemon

* Unified signer

* Continue unified signer

* A bit more

* Fixes and down the rabbit hole...

* More tx pool fixes

* More refactoring fixes

* More fixes'

* more fixes

* More fixes

* More compile fixes

* More RLP hand-writing

* Finish RLP encoding/decoding of transactions

* Fixes to header encoding, start on protocol packets

* Transaction decoding

* Use DecodeTransaction function

* Decoding BlockBodyPacket

* Encode and decode for pool txs

* Start fixing tests

* Introduce SigningHash

* Fixes to SignHash

* RLP encoding fixes

* Fixes for encoding/decoding

* More test fixes

* Fix more tests

* More test fixes

* More test fixes

* Fix core tests

* More fixes for signer

* Fix for tx

* Fixes to string encoding/size

* Fix eip2930 test

* Fix rest of ./tests

* More fixes

* Fix compilation

* More test fixes

* More test fixes

* Test fixes

* More fixes

* Reuse EncodingSize in EncodeRLP for accessList

* Rearrange things in dynamic fee tx

* Add MarshalBinary

* More fixes

* Make V,R,S non-pointers

* More NPE fixes

* More fixes

* Receipt fixes

* Fix core/types

* Fix ./eth

* More compile fixes for tests

* More test fixes

* More test fixes

* Try to see lint errors better

* Try to see lint errors better

* Fix lint

* Debugging eip1559 test

* Fix TestEIP1559Transition test

* Fix NewBlockPacket encoding/decoding

* Fix calculation of TxHash

* Fix perf problem with senders

* Update aleut config values

* Try adding static peers

* Add staticpeers to defaul flags

* Change aleut networkID

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
ledgerwatch
80a3edae2f
Fixes to the sentry/downloader (#1767)
* Complete eth/66 migration, temporary disable broadcast

* Update header stage progress correctly

* Fix restarting

* Not pass blockPropagator to headers stage

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-21 23:24:01 +01:00
Alex Sharov
4b803bfa6a
Notify RPCDaemon after sync cycle commit (#1772) 2021-04-21 17:58:23 +07:00
Artem Vorotnikov
9a3e47a498
Remove StateCache where unused (#1770) 2021-04-21 16:14:23 +07:00
Alex Sharov
bbe9af3e25
Stages as const (#1774) 2021-04-21 15:02:21 +07:00
Alex Sharov
41fa29d421
Mining fixes (#1771) 2021-04-21 12:01:25 +07:00
Artem Vorotnikov
c967655c1d
More cleanups (#1765) 2021-04-21 08:48:37 +07:00
alex.sharov
e25b57511d fix nil pointer in peer 2021-04-20 23:34:54 +07:00
alex.sharov
b08532b278 fix nil pointer in peer 2021-04-20 23:24:09 +07:00
Alex Sharov
bf1c219531
Move txPool.Get method behind direct-grpc (#1760) 2021-04-20 21:41:46 +07:00
Alex Sharov
22980c11a1
create make_protocols_method similar to eth.MakeProtocols (#1754) 2021-04-20 12:36:25 +07:00
Alex Sharov
bf6f042a5d
Start mining on new tx (#1740) 2021-04-20 09:12:08 +07:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB (#1703)
* debug

* debug

* it works

* rename clique bucket

* remove genesis special case

* copy snapshot

* remove debug

* migration

* debug

* regenerate snapshots

* simplify

* regeneration

* after merge

* tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
e44b5eb84e
less blockchain object use step 5 (#1751) 2021-04-19 15:36:50 +07:00
Alex Sharov
5019610cfb
remove database field from Ethereum object (#1750) 2021-04-19 14:56:44 +07:00
Alex Sharov
368f37a02f
Simplify blockchain obj, step 5 (#1739) 2021-04-18 12:27:07 +07:00
Alex Sharov
b6a58b8263
Eth 66 sentry (#1721)
* eth66

* eth66

* test v0

* save

* sentry handshake test

* sentry handshake test

* sentry handshake test

* clean

* clean

* clean

* clean

* clean

* clean
2021-04-16 14:05:35 +01:00
Alex Sharov
9273090761
Move TxPool stage to rwtx (#1734) 2021-04-16 13:30:58 +07:00
Alex Sharov
a6541b5402
[merge after release] remove trie cache cli flags and logic (#1725) 2021-04-15 17:06:38 +07:00
Alex Sharov
5703059f29
[merge after release] Move history stage to rwtx (#1727) 2021-04-15 17:06:30 +07:00