ledgerwatch
835b954f5c
State stream to RPC daemon ( #1986 )
...
* Remove interfaces
* Squashed 'interfaces/' content from commit e06631eb4
git-subtree-dir: interfaces
git-subtree-split: e06631eb4d0926c2d6a4f552497b920b4ed8d1bd
* Update KV interface
* Squashed 'interfaces/' changes from e06631eb4..014677ffe
014677ffe Merge remote-tracking branch 'origin/master' into stream
08c32a09e add version method to txPool and ethbackend
5b6bf70b9 Update README.md
7712cb267 Update README.md
f895ece4c save (#37 )
git-subtree-dir: interfaces
git-subtree-split: 014677ffe5bff0dee1a333f06c92e6110d791468
* Fix old interfaces
* Squashed 'interfaces/' changes from 014677ffe..df31e1146
df31e1146 remove action from storage change
git-subtree-dir: interfaces
git-subtree-split: df31e1146c368eda2e2b15ab252b78fba7a0a6f3
* add Accumulator
* add location
* Squashed 'interfaces/' changes from df31e1146..472584639
472584639 Merge remote-tracking branch 'origin/master' into stream
dd6a42724 Refactor of consensus interface (#28 )
git-subtree-dir: interfaces
git-subtree-split: 472584639f637189dfb906ef1ed03665f98d55d2
* Fix compilation in cons
* Pass accumulator to Execution stage
* Fix test
* Pass accumulator to unwind and plain writer
* Add accumulator use to plain writer
* Squashed 'interfaces/' changes from 472584639f..5c36f038b8
5c36f038b8 State change stream to KV (#38 )
REVERT: 472584639f Merge remote-tracking branch 'origin/master' into stream
REVERT: df31e1146c remove action from storage change
REVERT: 014677ffe5 Merge remote-tracking branch 'origin/master' into stream
REVERT: e06631eb4d Fix
REVERT: 9c10d79d2d Fix
REVERT: 61ae9307de Fix
REVERT: 4fcf34ecc5 State change stream to KV
git-subtree-dir: interfaces
git-subtree-split: 5c36f038b87096ffb6b07e90c6762c21b864cd3b
* Add state.stream flag
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-23 15:53:18 +01:00
Alex Sharov
450c79e3ab
move integration to kv ( #1996 )
2021-05-23 12:41:42 +07:00
Alex Sharov
a8e8cab4c5
Less blockchain obj in test ( #1995 )
2021-05-23 10:48:31 +07:00
alex.sharov
defd59d490
move setry mock and StageLoopStep to Kv
2021-05-23 10:39:25 +07:00
ledgerwatch
6a19321d82
Drop header references from persistent links ( #1991 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-22 12:03:53 +01:00
Artem Vorotnikov
5655409616
Do not write old block changesets in pruning storage mode ( #1964 )
...
* Remove incremental staged sync
* Do not write old block changesets in pruning storage mode
* Remove pruning flag, put behaviour behind !StorageMode.History
2021-05-22 11:57:10 +01:00
Alex Sharov
545fe32f68
Mdbx - make it default db. Lazy buckets renaming. ( #1982 )
2021-05-22 10:03:02 +07:00
Alex Sharov
0be3044b7e
rename ( #1978 )
...
* rename
* rename "make grpc"
* rename "abi bindings templates"
* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
Alex Sharov
215623da80
Windows support ( #1944 )
2021-05-20 23:46:12 +07:00
Alex Sharov
50934fa5d3
Move chain maiker to kv ( #1972 )
2021-05-20 18:49:33 +07:00
Alex Sharov
9d66aa0fe3
better support for mdbx in snapshots ( #1971 )
2021-05-20 18:12:07 +07:00
ledgerwatch
4b897d679f
Reduce persistent link kept ( #1969 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-20 07:57:17 +01:00
Alex Sharov
57880b8817
fixes for dv2 cancelation ( #1968 )
...
* fixes
* fixes
* fixes
* fixes
* fixes
2021-05-20 06:33:55 +01:00
ledgerwatch
79084ede47
[download.v2] Make default ( #1963 )
...
* [download.v2] Make default
* Update server.go
* Fix for tmp files appearing in the current dir
* Fix test
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
2021-05-19 16:19:29 +01:00
Alex Sharov
781a67a4ee
Auto-Close db in tests by using t.Cleanup ( #1956 )
2021-05-19 10:47:28 +07:00
ledgerwatch
2b2153f387
[download.v2] No double p2p server ( #1960 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-19 00:49:38 +01:00
ledgerwatch
19b602a135
Stuck unwind fix, test, and recovery ( #1959 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-19 00:00:57 +01:00
Alex Sharov
cceaf75b16
use t.TempDir() ( #1955 )
2021-05-18 19:13:16 +07:00
ledgerwatch
b2d00c97d8
Mock sentry and initial tests ( #1952 )
...
* More on mock
* Working test with mock sentry
* Simulate unwinding
* Fix MDBX overlapping txs
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-17 21:26:18 +01:00
Alex Sharov
0c91bfbf3e
RPCDaemon: support Pending block ( #1942 )
...
* clean
* save
* pubsub
* pubsub
* pubsub
* pubsub
* pubsub
* pubsub
* save
* tx pub-sub
* tx pub-sub
* clean
* clean
* save
* save
* save
* save
* save
* Squashed 'interfaces/' content from commit c469f3ae0
git-subtree-dir: interfaces
git-subtree-split: c469f3ae073b60c8821b61fed2910191080ef835
* save
* save
* save
* save
* Squashed 'interfaces/' changes from c469f3ae0..958dfc669
958dfc669 save
git-subtree-dir: interfaces
git-subtree-split: 958dfc669f8daeefe686a13aa852fb95f1537886
* save
* save
* up some deps
* up some deps
* clean
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
2021-05-17 13:15:19 +01:00
Alex Sharov
617baac56b
move db tests to kv interface ( #1946 )
2021-05-17 16:54:29 +07:00
ledgerwatch
86a94e8478
Avoid parsing transactions at Stage3 ( #1940 )
...
* RawBody
* Do not parse transaction during Stage3
* Reinstate delivery channel capacity, add comment
* Correct calculation of tx encoding for hashing
* fix logging of senders stage
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-16 20:01:14 +01:00
ledgerwatch
f6ac04e53e
Skip flaky tests ( #1943 )
...
* Update send_transaction_test.go
* Update snapshot_builder_test.go
2021-05-16 19:09:12 +01:00
ledgerwatch
b47367eb8d
Defer tx pool start until the initial cycle end ( #1937 )
...
* Defer tx pool start until the initial cycle end
* Not print dropped messages
* Fix mock sentry test
* Fix test import cycle|
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-15 12:35:28 +01:00
ledgerwatch
7eefcbe6a6
fix RPC notification, add boilerplate for tests ( #1936 )
...
* fix RPC notification, add boilerplate for tests
* Fix test
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-14 22:16:16 +01:00
ledgerwatch
c4f746f54f
[download.v2] Recover from unwind panic ( #1935 )
...
* [download.v2] Recover from unwind panic
* Fix notifications and unwind
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-14 15:24:50 +01:00
Alex Sharov
2ff6ce7d92
move staged sync to tx ( #1897 )
2021-05-14 18:00:20 +07:00
b00ris
48733c31cd
test fix ( #1930 )
2021-05-13 21:04:47 +01:00
b00ris
7948eafcc2
Property tests for kv snapshot ( #1928 )
...
* model tests for snapshot kv
* cursor test
* first failing test
* fix
* fix lint
* remove fmt
2021-05-13 09:40:32 +01:00
ledgerwatch
44cb94b906
Use chainname for preverified hashes ( #1927 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-13 09:32:09 +01:00
ledgerwatch
ebf20b1adc
Peer logging in sentry + fresh hashes for mainnet and ropsten ( #1924 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-13 01:04:07 +01:00
ledgerwatch
61df34b080
Fix headers subscriptions for download.v2 ( #1923 )
...
* Fix headers subscriptions for download.v2
* Use datadir to place nodes database for sentry
* Enable request chaining only after initial cycle
* Support for baikal and other in sentry
* Properly solve problem of stage1
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-12 22:37:43 +01:00
Alex Sharov
47d22773b9
[wip] dv2: Validate body ( #1913 )
...
* validate body
* validate body
* save
* save
* body deliveries messages handling into staged sync
* body deliveries messages handling into staged sync
* body deliveries messages handling into staged sync
* body deliveries messages handling into staged sync
* body deliveries messages handling into staged sync
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
* penalize right after verifyUncles and stop, don't batch
2021-05-12 14:57:35 +01:00
ledgerwatch
319c167279
Improvements to stage1 in download.v2 ( #1920 )
...
* Improvements to stage1 in download.v2
* Stop timer earlier
* Stop timer earlier
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-12 12:40:23 +01:00
ledgerwatch
447f20d395
Baikal testnet configuration ( #1910 )
...
* Baikal testnet configuration
* Fixes
* Change AsMessage signature
* Fixes in t8ntool
* More updates
* Fix misspelling
* Skip eip1559 transition test
* More bootnodes
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-11 10:54:20 +01:00
Alex Sharov
921c6a53c9
set default mdbx log level to warning ( #1912 )
2021-05-11 15:22:34 +07:00
ledgerwatch
a99d22c313
Trace filter 4 + fixes for download.v2 ( #1907 )
...
* Introduce TraceCallSet bucket and fill it up
* Fix compile
* Reset TraceCallSet
* Add database.verbosity
* Warn about dropped messages
* Unwind for call traces
* Fixes for headers download
* Fix for the slow stage 3
* Remove wakeup channels
* Permit system for peer load balancing
* Reduce aggressiveness
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-10 16:07:22 +01:00
Artem Vorotnikov
80bbbf62c2
Misc fixes ( #1908 )
...
* Fix debug levels
* try this
* remove adapter
* fix
* fix bodies stage
2021-05-10 06:59:29 +01:00
ledgerwatch
128f446e7c
Download v2 fixes ( #1903 )
...
* [download.v2] Fix endianness of Difficulty in Status message
* Add comments
* Finish stage to use RwTx
* Print header age in stage 1
* Print new block messages
* TopSeenHeight to be updated from any header
* block in the future handling and skip the test
* Fix downloader test
* Fix test
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-09 08:22:08 +01:00
ledgerwatch
8907e95d4a
Trace filter 3 ( #1904 )
...
* Prevent the leak
* Default storage mode, fix leak again
* TopSeenHeight to be updated from any header
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 23:07:16 +01:00
b00ris
565a4250d6
Snapshot sync headers stage ( #1836 )
...
* save state
* snapshot update works
* save state
* snapshot migrator
* tx test
* save state
* migrations stages refactor
* refactor snapshot migrator
* compilation fixed
* integrate snapshot migrator
* goerli sync headers
* debug async snapshotter on goerly
* move verify headers, remove experiments, fix remove old snapshot
* save state
* refactor snapshotsync injection
* fix deadlock
* replace snapshot generation stage logic to migrate method
* change done for body snapshot
* clean
* clean&&change deleted value
* clean
* fix hash len
* fix hash len
* remove one of wrap methods, add remove snapshots on start
* add err check
* fix shadowing
* stages unwind order debug
* matryoshka experiments
* steam test
* fix build
* fix test
* fix lint
* fix test
* fix test datarace
* add get test
* return timeout
* fix mdbx overlap
* fix after merge
* change epoch size
* clean todo
* fix
* return testdata
* added return from sndownloader gorutine
* fix review comments
* Fix
* More info
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync ( #1899 )
2021-05-08 00:07:49 +03:00
Alex Sharov
80ee37540e
penalize for abandoned anchor ( #1896 )
2021-05-07 06:17:40 +01:00
Alex Sharov
b2f424e2cd
Canonical fix in --download.v2 ( #1886 )
2021-05-07 09:06:00 +07:00
ledgerwatch
59e5502744
Initial work on RPC streaming ( #1880 )
...
* Initial work on RPC streaming
* Create the stream and pass it on
* trace_filter expressed as streamable (not finished)
* Reenable call trace index
* Add new line
* Remove storage mode override
* Tool for overriding storage mode
* Diagnostics
* Make trace_filter more streamy
* Streaming debug_traceTransaction and debug_traceCall
* Fix test
* Log error on stream flush
* Enable streaming for http
* Flush the stream too
* Make trace_filter flush too
* Stop streaming if client is not interested
* Try to quiet annoying test|
* Revert "Try to quiet annoying test|"
This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.
* Debug log for test
* Proceed with handshake regardless of whether peer notification worked
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 18:37:38 +01:00
Alex Sharov
e8255abdd0
deprecate tds and trie ( #1890 )
2021-05-06 15:48:30 +07:00
Alex Sharov
c7eedf0091
Add kv to stage settings (because it doesn't change in runtime, tx will be passed as Stage func argument) ( #1872 )
...
* add kv to config
* move forward tx use
* move forward tx use
* gc stat
* gc stat
* changeset writer - move to tx
* clean
* clean
* clean
* clean
* clean
2021-05-04 13:36:03 +01:00
Alex Sharov
f3d2af1e33
Rpcdaemon: move tx pool to own grpc service, subscribe to pending txs ( #1863 )
2021-05-04 08:37:17 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods ( #1861 )
...
* less use deprecated methods
* less use deprecated methods
* auto Dp limit
* auto Dp limit
* add test
2021-05-03 21:01:01 +01:00
ledgerwatch
39d3314b99
[download.v2] Prevent deadlock by starting txPool early ( #1860 )
...
* Prevent deadlock by starting txPool early
* Reduce error levels in the logs
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 12:15:53 +01:00
ledgerwatch
4432618cb9
Support for Baikal devnet ( #1853 )
...
* EIP-3529
* Test fix and EIP-3541
* Fix tests
* Fix
* Fix stagedsync test
* Fix
* Remove dup
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 10:34:19 +01:00
Alex Sharov
7eb088e862
Download request more headers on ext ( #1858 )
...
* use lru
* add comment
* add comment
2021-05-02 08:12:17 +01:00
Alex Sharov
08f6299257
broadcast new headers ( #1847 )
...
* broadcast new headers
* don't use eth type
* don't use eth type
* add seenAnnounces scet
* use lru
2021-05-01 21:03:51 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface ( #1854 )
2021-05-01 14:42:23 +07:00
Alex Sharov
ad598652cf
less blockchain object in tests ( #1848 )
2021-05-01 10:51:10 +07:00
ledgerwatch
34c1fdac21
Aleut fix for GASPRICE opcode ( #1852 )
...
* Aleut fix for GASPRICE opcode
* Fix for tip+basefee > feecap
* Unbork
* Fix tests
* Remove setHead test
* Test fix
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-30 19:20:18 +01:00
Alex Sharov
249b3098cb
add --sentry.api.addr flag ( #1850 )
...
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
* add --sentry.api.addr
2021-04-30 16:09:03 +01:00
ledgerwatch
75ca6b8c76
Initial work on integration tests ( #1797 )
...
* Initial work on integration tests
* Delete subtree
* Squashed 'interfaces/' content from commit 41a082ba4
git-subtree-dir: interfaces
git-subtree-split: 41a082ba4bde38647325eb0416cb1da1b4ca2b12
* Add consensus interfaces
* More stuff
* comments
* Fix compile
* Squashed 'interfaces/' changes from 41a082ba4..1b13a42a7
1b13a42a7 Add chainspec to consensus interface
git-subtree-dir: interfaces
git-subtree-split: 1b13a42a7803f5464722867a71065c27a7ebe8c3
* Squashed 'interfaces/' changes from 1b13a42a7..93a072c4c
93a072c4c Add missing import
git-subtree-dir: interfaces
git-subtree-split: 93a072c4c099d169322a3a53b95e40203276820b
* New consensus interfaces
* More on clique
* Fix tests
* Squashed 'interfaces/' changes from 93a072c4c..62f4ec4b2
62f4ec4b2 Add test service for consensus engine
git-subtree-dir: interfaces
git-subtree-split: 62f4ec4b263107635ffa3aabd5d634af22e813c6
* Squashed 'interfaces/' changes from 62f4ec4b2..061a63543
061a63543 Fix
git-subtree-dir: interfaces
git-subtree-split: 061a63543babdeb51ab7e3a96dec56b2485d4389
* Configuring clique engine with toml specs - start
* More toml parsing
* Constructed rinkeby genesis
* Simplify VerifyHeaders functions
* Fix lint
* Remove concurrent verification tests
* Fix lint
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-29 16:14:10 +01:00
ledgerwatch
03beaf4df7
Connect txPool to download.v2 to resolve 1 reason for "frozen download" ( #1830 )
...
* Pass txPool into new staged sync
* Remove poolStart function
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-27 16:37:03 +01:00
Alex Sharov
768bdcd850
--database.verbosity flag ( #1829 )
...
* --database.verbosity flag
* --database.verbosity flag
2021-04-27 13:31:00 +01:00
Artem Vorotnikov
57c88868a3
GetAsOf: do not return error on nil ( #1820 )
2021-04-26 18:05:29 +01:00
Alex Sharov
eee9438625
deprecate PlainDbState in favor of PlainKvState ( #1805 )
2021-04-26 12:37:48 +07:00
Alex Sharov
3af0b9b591
add hashstate, trie, history, logIndex stages ( #1799 )
...
* add hashstate and trie stages
* add history stages
* add history stages
* add history stages
* add logindex stage
* add logindex stage
* add other stages
2021-04-25 09:29:39 +01:00
Alex Sharov
d959986466
Connect TxFetcher to Sentry, Add --download.v2 option to TG ( #1796 )
2021-04-25 11:20:50 +07:00
ledgerwatch
e3f3dd3c9b
Integration tests 1 ( #1793 )
...
* Initial commit
* Add sentry gRPC interface
* p2psentry directory
* Update README.md
* Update README.md
* Update README.md
* Add go package
* Correct syntax
* add external downloader interface (#2 )
* Add txpool (#3 )
* Add private API (#4 )
* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5 )
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Separate upload messages into its own stream (#6 )
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Only send changed accounts to listeners (#7 )
* Txpool interface doc (#9 )
* Add architecture diagram source and picture (#10 )
* Typed hashes (#11 )
* Typed hashes
* Fix PeerId
* 64-bit tx nonce
* Add proper golang packages, max_block into p2p sentry Status (#12 )
* Add proper golang packages, max_block into p2p sentry Status
* Change EtherReply to address
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Add Rust infrastructure (#13 )
* DB stats methods removed by https://github.com/ledgerwatch/turbo-geth/pull/1665
* more p2p methods (#15 )
* add mining methods (#16 )
* First draft of Consensus gRPC interface (#14 )
* Update Rust build
* Fix interfaces in architecture diagram (#17 )
* Fix KV interface provider
* Fix Consensus interface provider
* drop java attributes (#18 )
* tx pool remove unused import (#19 )
* ethbackend: add protocol version and client version (#20 )
* Add missing ethbackend I/F (#21 )
* Add interface versioning mechanism (#23 )
Add versioning in KV interface
Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
* spec of tx pool method (#24 )
* spec of tx pool method (#25 )
* Update version.proto
* Refactor interface versioning
* Refactor interface versioning
* Testing interface
* Remove tree
* Fix
* Build testing protos
* Fix
* Fix
* Update to the newer interfaces
* Add ProtocolVersion and ClientVersion stubs
* Hook up ProtocolVersion and ClientVersion
* Remove service
* Add compatibility checks for RPC daemon
* Fix typos
* Properly update DB schema version
* Fix test
* Add test for KV compatibility|
* Info messages about compability for RPC daemon
* DB schema version to be one key
* Update release intructions
Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: canepat <16927169+canepat@users.noreply.github.com>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Co-authored-by: canepat <tullio.canepa@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-24 16:46:29 +01:00
Alex Sharov
ac7ece4a4d
Revert txpool grpc ( #1794 )
2021-04-24 10:48:06 +07:00
Evgeny Danilenko
984daa4d7a
Clique snapshot timer fixed ( #1789 )
...
* fix clique snapshot storage by timer
* after merge
* remove debug
* remove debug
* debug logging
* correct t.Fatal
2021-04-23 20:47:06 +03:00
Alex Sharov
d7d790725b
Add stage exec to downloader ( #1788 )
2021-04-23 17:27:07 +07:00
Alex Sharov
10c41491fa
Use Config struct for headers/body/senders stages. Add StageSenders to new downloader. ( #1787 )
...
* config for headers/body/senders
* clean
2021-04-23 16:16:45 +07:00
ledgerwatch
12cde41772
Aleut support (Eip1559) ( #1704 )
...
* Where I am at
* Refactoring of transaction types
* More refactoring
* Use Homested signer in rpc daemon
* Unified signer
* Continue unified signer
* A bit more
* Fixes and down the rabbit hole...
* More tx pool fixes
* More refactoring fixes
* More fixes'
* more fixes
* More fixes
* More compile fixes
* More RLP hand-writing
* Finish RLP encoding/decoding of transactions
* Fixes to header encoding, start on protocol packets
* Transaction decoding
* Use DecodeTransaction function
* Decoding BlockBodyPacket
* Encode and decode for pool txs
* Start fixing tests
* Introduce SigningHash
* Fixes to SignHash
* RLP encoding fixes
* Fixes for encoding/decoding
* More test fixes
* Fix more tests
* More test fixes
* More test fixes
* Fix core tests
* More fixes for signer
* Fix for tx
* Fixes to string encoding/size
* Fix eip2930 test
* Fix rest of ./tests
* More fixes
* Fix compilation
* More test fixes
* More test fixes
* Test fixes
* More fixes
* Reuse EncodingSize in EncodeRLP for accessList
* Rearrange things in dynamic fee tx
* Add MarshalBinary
* More fixes
* Make V,R,S non-pointers
* More NPE fixes
* More fixes
* Receipt fixes
* Fix core/types
* Fix ./eth
* More compile fixes for tests
* More test fixes
* More test fixes
* Try to see lint errors better
* Try to see lint errors better
* Fix lint
* Debugging eip1559 test
* Fix TestEIP1559Transition test
* Fix NewBlockPacket encoding/decoding
* Fix calculation of TxHash
* Fix perf problem with senders
* Update aleut config values
* Try adding static peers
* Add staticpeers to defaul flags
* Change aleut networkID
* Fix test
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
Alex Sharov
1f839f5b54
Txpool minor fixes ( #1778 )
2021-04-22 10:33:34 +07:00
ledgerwatch
80a3edae2f
Fixes to the sentry/downloader ( #1767 )
...
* Complete eth/66 migration, temporary disable broadcast
* Update header stage progress correctly
* Fix restarting
* Not pass blockPropagator to headers stage
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-21 23:24:01 +01:00
Artem Vorotnikov
9a3e47a498
Remove StateCache where unused ( #1770 )
2021-04-21 16:14:23 +07:00
Artem Vorotnikov
ffcb4b62a6
Adapters refactor ( #1768 )
2021-04-21 09:18:05 +07:00
Artem Vorotnikov
c967655c1d
More cleanups ( #1765 )
2021-04-21 08:48:37 +07:00
alex.sharov
a6fe31e1f2
fix nil pointer in peer
2021-04-20 23:31:42 +07:00
Alex Sharov
bf1c219531
Move txPool.Get method behind direct-grpc ( #1760 )
2021-04-20 21:41:46 +07:00
ledgerwatch
3d0fbb488f
Try adding static peers ( #1762 )
...
* Try adding static peers
* Add staticpeers to defaul flags
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-20 12:32:06 +01:00
Artem Vorotnikov
9e3aae1b9b
Cleanup StateReader ( #1759 )
2021-04-20 13:23:16 +07:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB ( #1703 )
...
* debug
* debug
* it works
* rename clique bucket
* remove genesis special case
* copy snapshot
* remove debug
* migration
* debug
* regenerate snapshots
* simplify
* regeneration
* after merge
* tests
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
5019610cfb
remove database field from Ethereum object ( #1750 )
2021-04-19 14:56:44 +07:00
Alex Sharov
00f700675b
add --datadir parameter to integration, snapshot generator, header downloader ( #1749 )
2021-04-19 14:25:26 +07:00
Alex Sharov
22b9522112
RPCDaemon: add --datadir flag, use --chaindata only if wanna redefine default chaindata folder ( #1747 )
2021-04-19 12:44:14 +07:00
Alex Sharov
368f37a02f
Simplify blockchain obj, step 5 ( #1739 )
2021-04-18 12:27:07 +07:00
Alex Sharov
b6a58b8263
Eth 66 sentry ( #1721 )
...
* eth66
* eth66
* test v0
* save
* sentry handshake test
* sentry handshake test
* sentry handshake test
* clean
* clean
* clean
* clean
* clean
* clean
2021-04-16 14:05:35 +01:00
Alex Sharov
a6541b5402
[merge after release] remove trie cache cli flags and logic ( #1725 )
2021-04-15 17:06:38 +07:00
Alex Sharov
4ef5e4b2f6
Continue move to rwtx ( #1722 )
2021-04-15 12:23:10 +07:00
Alex Sharov
cf464302a4
check_right_cache_size ( #1711 )
2021-04-11 11:23:39 +07:00
ledgerwatch
6febaf8dd1
Refine turbo mine parameters ( #1702 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-09 17:44:25 +01:00
Alex Sharov
a6c73b3ab5
Sentry node identity ( #1701 )
...
* clean
* clean
2021-04-09 13:01:00 +01:00
ledgerwatch
a3cf656b68
Fixes to sentry ( #1698 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-08 20:07:06 +01:00
Alex Sharov
35872a4708
[merge after release] remove etl onLoadCommit hook, because etl doesn't manage transactions anymore (it's low-level code) ( #1692 )
2021-04-08 18:03:33 +07:00
ledgerwatch
acea6548a2
Replace testnet flags with one chain option ( #1686 )
...
* Replace testnet flags with one chain option
* Update README.md
* Update README.md
* Update README.md
* Update config.go
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-08 08:39:40 +01:00
Alex Sharov
7222cdd640
delete tds.unwindTo method ( #1688 )
2021-04-08 10:47:10 +07:00
Alex Sharov
9527671e5e
Remove blockchain object step3 ( #1684 )
2021-04-07 12:38:43 +07:00
Alex Sharov
a913ae8847
Remove chainContext object ( #1683 )
2021-04-06 16:52:53 +07:00
Alex Sharov
7452cc7f7f
Broadcast new blocks - basic functions ( #1675 )
2021-04-06 12:54:50 +07:00
Alex Sharov
cc539ce266
Remove blockchain.InsertChain method ( #1681 )
2021-04-06 12:06:07 +07:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it ( #1676 )
2021-04-05 20:04:58 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs ( #1669 )
2021-04-03 09:26:00 +03:00
Artem Vorotnikov
e0a2d47139
Remove ancients and some old skipped tests ( #1668 )
2021-04-03 13:08:35 +07:00
Alex Sharov
fbfc43da7e
etl to use rwtx ( #1660 )
2021-04-02 18:22:25 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) ( #1656 )
2021-04-02 13:36:49 +07:00
Alex Sharov
b4a2d823c4
old loader remove v1 ( #1641 )
2021-03-31 14:43:12 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface ( #1627 )
2021-03-30 12:53:54 +03:00
Alex Sharov
86ccfd0338
RPC: Pending transactions/blocks/logs ( #1625 )
2021-03-30 14:09:00 +07:00
Artem Vorotnikov
9b8cdc0f22
Fix lints and remove more unused code ( #1621 )
2021-03-29 10:58:45 +07:00
Alex Sharov
8ccc6b2664
Mining: remove local pow mining (from ethash), --miner.notify is required now, cycle is non-blocking now ( #1617 )
2021-03-28 21:40:27 +07:00
Alex Sharov
5f6513de31
Remove blockchain object from some tests ( #1612 )
2021-03-28 20:41:47 +07:00
Alex Sharov
416a69ecb3
Genesis.toBlock remove db from arguments ( #1611 )
2021-03-28 16:27:06 +07:00
Artem Vorotnikov
afe551ba1c
Remove puppeth & account management ( #1610 )
2021-03-26 21:05:42 +03:00
Alex Sharov
2bb70fca9f
Genesis: calculate root by common components ( #1608 )
2021-03-26 17:05:35 +07:00
Artem Vorotnikov
e67be2b1be
Remove unused code ( #1604 )
...
* Remove graphql, legacy options
* Remove CODEOWNERS
* Remove EWASM and custom EVM interpreter
2021-03-26 09:08:01 +07:00
ledgerwatch
ea15c4a4fd
Fixes to new downloader ( #1599 )
...
* Fix for prefetched bodies
* Do not reset headers stage to 0
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-24 22:37:29 +00:00
ledgerwatch
69702c8235
Correct forkID computation in sentry, use blocks prefetched by NewBlockPacket, Fix NaN delivery speed output ( #1596 )
...
* Use proper forkID
* Remove NaN from Write block bodies messages
* Move Prefetched blocks
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-24 09:46:31 +00:00
Artem Vorotnikov
a6742a4cf8
Kill cmd/state/stateless ( #1592 )
...
* Move stuff around
* Remove
* tidy
* Fix for writeReceipt printing
* fix lint typo
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-23 16:33:58 +00:00
Alex Sharov
aff859edc0
Mining stage ( #1554 )
2021-03-23 16:00:07 +07:00
Alex Sharov
47c6bdce9d
RPCDaemon: increase throughput by allowing more parallel grpc streams and not lockthread for read tx ( #1580 )
2021-03-23 14:28:04 +07:00
ledgerwatch
0986ce43fb
Fix some conformance tests broken after rebase ( #1587 )
...
* Uncomment
* Try to fix
* Try again
* Compile fix
* Compile fix
* Compile fix
* Compile fix
* Compile fix
* fix lint
* Enable more tests
* Fix core tests
* Remove comment outs
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-23 07:25:10 +00:00
Alex Sharov
91e4126ca7
db commit - remove unused return param ( #1584 )
2021-03-22 19:41:52 +07:00
Alex Sharov
7024a7a35f
mining uncles ( #1581 )
2021-03-22 18:29:33 +07:00
Alex Sharov
40dd00fee7
remove ideal batch size ( #1582 )
2021-03-22 13:47:01 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions ( #1577 )
2021-03-21 20:15:25 +07:00
alex.sharov
67e1f9f6d4
Revert "remove old flags"
...
This reverts commit 009c098b87
2021-03-21 17:50:07 +07:00
alex.sharov
e7034c0eba
revert
2021-03-21 17:50:05 +07:00
alex.sharov
1fcf349ce2
fix tests
2021-03-21 17:41:09 +07:00
alex.sharov
a649eef6b9
fix tests
2021-03-21 17:26:54 +07:00
alex.sharov
009c098b87
remove old flags
2021-03-21 17:07:28 +07:00
alex.sharov
970c7e93a8
remove old flags
2021-03-21 17:05:42 +07:00
Alex Sharov
97eec4288a
Tests: use transaction instead of db.MemCopy in chain_makers.go ( #1570 )
...
* mdbx tests speedup
* mdbx tests speedup
* devel up
* devel up
* devel up
* lint
* less db copy
* don't loose error in test
* clean
2021-03-20 19:33:40 +00:00
ledgerwatch
81ea5bab78
More updates to downloader, new p2psentry protocol ( #1559 )
...
* Initial commit
* Add sentry gRPC interface
* p2psentry directory
* Update README.md
* Update README.md
* Update README.md
* Add go package
* Correct syntax
* add external downloader interface (#2 )
* Add txpool (#3 )
* Add private API (#4 )
* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5 )
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Separate upload messages into its own stream (#6 )
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Only send changed accounts to listeners (#7 )
* Txpool interface doc (#9 )
* More additions
* More additions
* Fix locking
* Intermediate
* Fix separation of phases
* Intermediate
* Fix test
* More transformations
* New simplified way of downloading headers
* Fix hard-coded header sync
* Fixed syncing near the tip of the chain
* Add architecture diagram source and picture (#10 )
* More fixes
* rename tip to link
* Use preverified hashes instead of preverified headers
* Fix preverified hashes generation
* more parametrisation
* Continue parametrisation
* Fix grpc data limit, interruption of headers stage
* Add ropsten preverified hashes
* Typed hashes (#11 )
* Typed hashes
* Fix PeerId
* 64-bit tx nonce
* Disable penalties
* Add goerli settings, bootstrap nodes
* Try to fix goerly sync
* Remove interfaces
* Add proper golang packages, max_block into p2p sentry Status
* Prepare for proto overhaul
* Squashed 'interfaces/' content from commit ce36053c2
git-subtree-dir: interfaces
git-subtree-split: ce36053c24db2f56e48ac752808de60afa1dfb4b
* Change EtherReply to address
* Adaptations to new types
* Switch to new types
* Fixes
* Fix formatting
* Fix lint
* Lint fixes, reverse order in types
* Fix lint
* Fix lint
* Fix lint
* Fix test
* Not supporting eth/66 yet
* Fix shutdown
* Fix lint
* Fix lint
* Fix lint
* return stopped check
Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: canepat <16927169+canepat@users.noreply.github.com>
2021-03-19 21:24:49 +00:00
b00ris
c03da8fb8e
Split header prefix bucket to headers, td and canonical ( #1556 )
...
* split headers prefix
* migration
* fix downloader bug
* test for migration
* fix lint
* uncomment t.Prallel
* fix postprocessing test
2021-03-19 12:54:47 +00:00
Alex Sharov
909fd42bee
go 1.16 support ( #1509 )
2021-03-19 16:32:52 +07:00
Artem Vorotnikov
0b2e4df0ff
Align Cursor API with LMDB/MDBX ( #1558 )
2021-03-19 14:45:01 +07:00
Alexey Sharp
0d24a38813
Fix TestEIP2718Transition
2021-03-18 13:19:21 +01:00
Igor Mandrigin
0a55895b33
Merge branch 'master' into geth-1-10-1
2021-03-17 16:50:20 +01:00
Igor Mandrigin
6fbc0cfff8
fix compilation in blockchain_test.go
2021-03-16 11:55:42 +01:00
Alexey Sharp
af495562e4
Fix rcpdaemon compilation
2021-03-14 18:52:15 +00:00
Alexey Sharp
519b19f8aa
Compilation fixes
2021-03-14 08:00:37 +00:00
Alex Sharov
3f305673aa
Trie: add invariant - first level of trie must be in DB (to ensure having 100% trie structure in trie_account table) ( #1522 )
2021-03-13 09:30:54 +07:00
lightclient
50d975302d
all: add support for EIP-2718, EIP-2930 transactions ( #21502 )
...
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.
There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.
The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID.
This change also adds support for the YoloV3 testnet.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
# Conflicts:
# accounts/abi/bind/backends/simulated.go
# cmd/evm/internal/t8ntool/execution.go
# cmd/evm/internal/t8ntool/transition.go
# cmd/geth/main.go
# cmd/geth/usage.go
# core/bench_test.go
# core/state/statedb.go
# core/state_prefetcher.go
# core/state_processor.go
# core/state_transition.go
# core/tx_pool.go
# core/types/block.go
# core/types/derive_sha.go
# core/types/gen_tx_json.go
# core/types/receipt.go
# core/types/receipt_test.go
# core/types/transaction.go
# core/types/transaction_signing.go
# core/types/transaction_test.go
# ethclient/ethclient.go
# ethclient/signer.go
# graphql/graphql.go
# internal/ethapi/api.go
# internal/guide/guide_test.go
# les/benchmark.go
# les/odr_test.go
# light/odr_test.go
# light/txpool.go
# miner/worker.go
# miner/worker_test.go
# signer/core/api.go
# tests/state_test_util.go
# trie/stacktrie_test.go
# turbo/stages/blockchain_test.go
2021-03-12 15:40:24 +01:00
Alex Sharov
060f3b9a60
Grafana: add Btree metrics ( #1547 )
2021-03-10 10:58:57 +07:00
Martin Holst Swende
adf219ea76
go.mod: use github.com/holiman/bloomfilter/v2 ( #22044 )
...
* deps: use improved bloom filter implementation
* eth/handler, trie: use 4 keys for syncbloom + minor fixes
* eth/protocols, trie: revert change on syncbloom method signature
# Conflicts:
# core/state/snapshot/difflayer.go
# go.mod
# go.sum
# trie/sync.go
# turbo/trie/sync_bloom.go
2021-03-09 10:36:02 +01:00
Alex Sharov
4fe019a099
add periodical db statistic metrics ( #1545 )
2021-03-09 13:34:13 +07:00
Péter Szilágyi
693459073e
core, eth: split eth package, implement snap protocol ( #21482 )
...
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md ) , but does not enable it by default.
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
# cmd/geth/misccmd.go
# cmd/utils/flags.go
# core/blockchain.go
# core/blockchain_snapshot_test.go
# core/forkid/forkid.go
# core/rawdb/accessors_snapshot.go
# core/rawdb/schema.go
# core/state/snapshot/generate.go
# core/state/statedb.go
# eth/api_test.go
# eth/backend.go
# eth/downloader/downloader.go
# eth/downloader/downloader_test.go
# eth/downloader/modes.go
# eth/downloader/queue.go
# eth/downloader/statesync.go
# eth/gen_config.go
# eth/handler.go
# eth/handler_test.go
# eth/helper_test.go
# eth/peer.go
# eth/protocol.go
# eth/protocol_test.go
# eth/sync.go
# eth/sync_test.go
# ethstats/ethstats.go
# les/client.go
# les/enr_entry.go
# les/handler_test.go
# les/peer.go
# les/server_handler.go
# tests/block_test_util.go
# trie/proof.go
# trie/proof_test.go
# trie/trie.go
# turbo/trie/notary.go
# turbo/trie/sync_bloom.go
2021-03-08 17:07:38 +01:00
gary rong
b2351da9d6
all: disable recording preimage of trie keys ( #21402 )
...
* cmd, core, eth, light, trie: disable recording preimage by default
* core, eth: fix unit tests
* core: fix import
* all: change to nopreimage
* cmd, core, eth, trie: use cache.preimages flag
* cmd: enable preimages for archive node
* cmd/utils, trie: simplify preimage tracking a bit
* core: fix linter
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
# cmd/geth/main.go
# cmd/geth/usage.go
# cmd/utils/flags.go
# core/blockchain.go
# core/genesis.go
# core/state/database.go
# core/state/state_test.go
# eth/api_test.go
# eth/api_tracer.go
# eth/backend.go
# light/postprocess.go
# trie/secure_trie.go
# turbo/trie/database.go
2021-03-08 12:26:27 +01:00
ledgerwatch
b743b88218
Revert "Extract validating interface ( #1120 )" ( #1543 )
...
* Revert "Extract validating interface (#1120 )"
This reverts commit e4e36c152e
2021-03-05 20:34:23 +00:00
Alex Sharov
722f1a387a
trie: do not use extension hash ( #1535 )
2021-03-03 15:54:42 +07:00
Alex Sharov
6d0d84cda6
log git branch and git commit ( #1523 )
2021-03-01 11:02:06 +07:00
ledgerwatch
da74bd8f38
Fixes to unwind logic for the new staged sync ( #1521 )
...
* Make DB visible
* Further fixes to unwind logic
* Fix lint
* Fix lint
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-28 08:47:31 +00:00
