Commit Graph

13960 Commits

Author SHA1 Message Date
ledgerwatch
c4f746f54f
[download.v2] Recover from unwind panic (#1935)
* [download.v2] Recover from unwind panic

* Fix notifications and unwind

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-14 15:24:50 +01:00
Alex Sharov
2ff6ce7d92
move staged sync to tx (#1897) 2021-05-14 18:00:20 +07:00
ledgerwatch
61cba00081
Minor fixes (#1934)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-14 08:57:18 +01:00
Alex Sharov
594097a523
Up some deps (geth has similar) (#1933) 2021-05-14 09:00:08 +07:00
b00ris
48733c31cd
test fix (#1930) 2021-05-13 21:04:47 +01:00
Igor Mandrigin
da4f25637f begin 2021.05.03 release cycle 2021-05-13 16:08:48 +02:00
Igor Mandrigin
4e4d2032a3 update skip_analysis 2021-05-13 16:07:04 +02:00
Alex Sharov
f3a9eb340c
mdbx_setup_debug - handle response correct way (#1931) 2021-05-13 21:01:23 +07:00
b00ris
7948eafcc2
Property tests for kv snapshot (#1928)
* model tests for snapshot kv

* cursor test

* first failing test

* fix

* fix lint

* remove fmt
2021-05-13 09:40:32 +01:00
ledgerwatch
44cb94b906
Use chainname for preverified hashes (#1927)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-13 09:32:09 +01:00
Alex Sharov
3e1bdbb465
Mdbx: fix for MDBX_PROBLEM (#1922) 2021-05-13 14:40:44 +07:00
ledgerwatch
ebf20b1adc
Peer logging in sentry + fresh hashes for mainnet and ropsten (#1924)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-13 01:04:07 +01:00
ledgerwatch
61df34b080
Fix headers subscriptions for download.v2 (#1923)
* Fix headers subscriptions for download.v2

* Use datadir to place nodes database for sentry

* Enable request chaining only after initial cycle

* Support for baikal and other in sentry

* Properly solve problem of stage1

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-12 22:37:43 +01:00
ledgerwatch
ddcf271ca5
Trace filter 6 (#1921)
* Improvements to stage1 in download.v2

* Stop timer earlier

* Stop timer earlier

* Print apends

* Fix traceCursor

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-12 17:35:46 +01:00
Alex Sharov
47d22773b9
[wip] dv2: Validate body (#1913)
* validate body

* validate body

* save

* save

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch
2021-05-12 14:57:35 +01:00
ledgerwatch
319c167279
Improvements to stage1 in download.v2 (#1920)
* Improvements to stage1 in download.v2

* Stop timer earlier

* Stop timer earlier

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-12 12:40:23 +01:00
ledgerwatch
0aabf85c1d
Sentry fixes (for very frequent peer not found error) (#1918)
* Extra prints

* Remove printouts

* Printing back

* Fix the stuck stage 1

* Not allow peers to connect more than once

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-12 09:36:43 +01:00
ledgerwatch
7f51f13804
Fix for removed peers (#1917)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-11 20:35:59 +01:00
ledgerwatch
bf0ea48823
Deadline-based permits in sentry (#1915)
* Not increase permit on announcements

* Init permits

* Deadlines

* Fix NPEs

* Fix NPE

* Fix findPeer

* Reduce max permits to 4

* 8 max permits, but 30 seconds timeout

* Back to 4

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-11 18:20:00 +01:00
ledgerwatch
0b7d185d1f
Trace filter 5 (and sentry improvement) (#1914)
* Append/AppendDup

* Print appends

* Remove printing

* Increase permit on announcements

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-11 13:58:43 +01:00
ledgerwatch
1351383f68
Add files via upload 2021-05-11 13:40:10 +01:00
ledgerwatch
5478355e91
Add files via upload 2021-05-11 13:29:41 +01:00
ledgerwatch
0d9ffa7875
Add files via upload 2021-05-11 13:28:30 +01:00
ledgerwatch
447f20d395
Baikal testnet configuration (#1910)
* Baikal testnet configuration

* Fixes

* Change AsMessage signature

* Fixes in t8ntool

* More updates

* Fix misspelling

* Skip eip1559 transition test

* More bootnodes

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-11 10:54:20 +01:00
Alex Sharov
0163329a5a
KV and RemoteKV: to reuse stateless cursors during transaction (#1909) 2021-05-11 15:23:17 +07:00
Alex Sharov
921c6a53c9
set default mdbx log level to warning (#1912) 2021-05-11 15:22:34 +07:00
ledgerwatch
a99d22c313
Trace filter 4 + fixes for download.v2 (#1907)
* Introduce TraceCallSet bucket and fill it up

* Fix compile

* Reset TraceCallSet

* Add database.verbosity

* Warn about dropped messages

* Unwind for call traces

* Fixes for headers download

* Fix for the slow stage 3

* Remove wakeup channels

* Permit system for peer load balancing

* Reduce aggressiveness

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-10 16:07:22 +01:00
ledgerwatch
c73783c985
Add files via upload 2021-05-10 08:57:24 +01:00
ledgerwatch
589ed0e34c
Delete Consensus Engine-Page-3.png 2021-05-10 08:55:01 +01:00
ledgerwatch
2919c3cb88
Delete Consensus Engine-Page-2.png 2021-05-10 08:54:54 +01:00
ledgerwatch
23f6c1dab6
Delete Consensus Engine-Page-1.png 2021-05-10 08:54:47 +01:00
ledgerwatch
51b94db1b4
Add files via upload 2021-05-10 08:51:43 +01:00
Artem Vorotnikov
80bbbf62c2
Misc fixes (#1908)
* Fix debug levels

* try this

* remove adapter

* fix

* fix bodies stage
2021-05-10 06:59:29 +01:00
Alex Sharov
3f36e362c2
mdbx switch to release (#1906) 2021-05-09 15:35:47 +07:00
ledgerwatch
128f446e7c
Download v2 fixes (#1903)
* [download.v2] Fix endianness of Difficulty in Status message

* Add comments

* Finish stage to use RwTx

* Print header age in stage 1

* Print new block messages

* TopSeenHeight to be updated from any header

* block in the future handling and skip the test

* Fix downloader test

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-09 08:22:08 +01:00
ledgerwatch
8907e95d4a
Trace filter 3 (#1904)
* Prevent the leak

* Default storage mode, fix leak again

* TopSeenHeight to be updated from any header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 23:07:16 +01:00
ledgerwatch
5c512fbddb
Trace filter 2 (#1898)
* Fix error message about write protection

* Clean up of trace_filter

* Add CallTraces to Execution Stage

* Fix compile

* Say something

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 16:17:17 +01:00
ledgerwatch
56dc23691c
Fix DB performance regression (#1902)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 12:38:32 +01:00
ledgerwatch
4a214d38e4
Fix integration after snapshot PR (#1901)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 11:42:34 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
e0dd6bafc6
Enable CI for pushes to master (#1900) 2021-05-08 00:43:04 +03:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync (#1899) 2021-05-08 00:07:49 +03:00
ledgerwatch
fe0a882a45
Trace filter (#1895)
* Trace filter

* Fix self-destruct traces

* Update ci.yml

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-07 10:05:47 +01:00
Alex Sharov
80ee37540e
penalize for abandoned anchor (#1896) 2021-05-07 06:17:40 +01:00
Alex Sharov
06cec6d535
Move exec to tx (#1887) 2021-05-07 09:06:44 +07:00
Alex Sharov
b2f424e2cd
Canonical fix in --download.v2 (#1886) 2021-05-07 09:06:00 +07:00
ledgerwatch
9fa21db40e
Qucker reaction to new block headers (#1894)
* Qucker reaction to new block headers

* Always print header lag

* Print verification time

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 19:40:12 +01:00
ledgerwatch
59e5502744
Initial work on RPC streaming (#1880)
* Initial work on RPC streaming

* Create the stream and pass it on

* trace_filter expressed as streamable (not finished)

* Reenable call trace index

* Add new line

* Remove storage mode override

* Tool for overriding storage mode

* Diagnostics

* Make trace_filter more streamy

* Streaming debug_traceTransaction and debug_traceCall

* Fix test

* Log error on stream flush

* Enable streaming for http

* Flush the stream too

* Make trace_filter flush too

* Stop streaming if client is not interested

* Try to quiet annoying test|

* Revert "Try to quiet annoying test|"

This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.

* Debug log for test

* Proceed with handshake regardless of whether peer notification worked

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 18:37:38 +01:00
Chase Wright
d8888439b7
Add tzdata to alpine container for log timestamps (#1892)
Alpine containers, by default, do not include `tzdata` package, so they do not honor the `TZ` environment variable. When running in a docker container the logs timestamps will always be in UTC without this package. I propose we add `tzdata` so that the timezone can be changed.
2021-05-06 18:22:11 +01:00
Artem Vorotnikov
bb305e183e
Update ci.yml (#1893) 2021-05-06 18:22:01 +01:00