Commit Graph

8 Commits

Author SHA1 Message Date
Alex Sharov
a72543bc16
Stop dial scheduler asap (#1334)
* stop dial scheduler asap

* stop dial scheduler asap
2020-11-02 21:07:26 +00:00
ledgerwatch
cceb8c2c3a
Sharded state experiment based on call_traces stage run via integration (#1329)
* Initial work on state shards

* Fix compile errors

* Intermediate

* Implement dispatcher

* Implemented sharded reader

* No cache

* Print comms

* No prepopulation

* Print dispatcher activity

* Print more

* Print errors

* Fix dispatcher

* Remove logging

* Dispatcher to enforce initial barrier

* Don't print that much

* Turn off cache pre-seeting for state sharding

* Artificial latency, fixed number of blocks to execute

* Fix
2020-10-31 11:51:56 +00:00
ledgerwatch
97a03a1ad7
[WIP] add interfaces subtree and switch headers POC to using shared proto files (#1315)
* Squashed 'interfaces/' content from commit c4e14d6dc

git-subtree-dir: interfaces
git-subtree-split: c4e14d6dc80a157c60acd42f0152d631d250601e

* Squashed 'interfaces/' changes from c4e14d6dc..84c5faa5f

84c5faa5f Update README.md
07db484a1 Update README.md
ea14fa4e3 Update README.md

git-subtree-dir: interfaces
git-subtree-split: 84c5faa5f956ea52119911f39ac5864b282dcec2

* Change proto paths

* Squashed 'interfaces/' changes from 84c5faa5f..eb9859d41

eb9859d41 Correct syntax
ae78fe955 Add go package

git-subtree-dir: interfaces
git-subtree-split: eb9859d4157351fac3b8bd2f477307a09978e6df

* Switch to shared proto files

* Fix generation

* Remove duplicate files

* Init hard-coded tips when not recovering

* Only check initiation when extending up
2020-10-29 16:38:46 +00:00
ledgerwatch
7554428884
POC of header downloader - splitting into 2 processes: sentry + downloader (via gRPC) (#1291)
* Splitting sentry and downloader - the beginning

* A bit more

* More on sentry

* More gRPC

* Sentry and downloader separated

* Update binding for stable version of grpc

* Better bufferSize flag

* Fix lint

* Send pelanties

* Fix lint

* Remove hard-coded tips on connect

* Tidy the logs a bit

* Deal with hardTips on Recovery

* Print hard tips

* Hide empty anchors

* Request headers after receiving a message

* Better waking up

* Print hard-coded block numbers

* Print outgoing requests

* Debug logging

* In the middle protection

* Sentry not to lose peers when core disconnects
2020-10-25 22:05:37 +00:00
ledgerwatch
14288b8cd0
Headers poc 5 - Intermediate (#1145)
* Add headers persistence

* Print flushBuffer

* Fix indexing problem

* Not skip hard-coded headers if the files are empty

* Fix lint

* print anchor state after init

* Properly construct file names

* Add check sub-command

* Fix lint

* Fix lint

* Fix lint

* Print more info when recovering

* Fix recovering

* Fix recovery

* Add anchors also as tips

* 2-level priority queue for tips

* Initialise tipQueue in anchor

* update maxTipHeight

* fix type

* Add anchors to the anchorQueue and rebuild anchorQueue on deletion

* Fix NPE

* fix recovery

* User buffersize, add hard coded headers to buffer

* Reinit anchorQueue

* Schedule requests after recovery

* No fix

* Remove duplicates

* Report duplicate headers

* Log buffer additions

* Fix for duplicate headers

* Try to fix duplicate headers again

* remove TODO comment

* Use LLRB instead of heap for anchors

* Print reserveTip info

* Correctly replace anchors in the tree

* Remove excessive logging

* Print tips attached to the anchor

* Print tips better

* limitTips instead of reserveTip

* Print forked headers

* Use pointers in llrb

* Print tipStretch

* Print limitTips

* Mininise AnchorItem

* Put anchors into the tree

* Fix totalDiff calculation, but is it needed?

* Remove totalDifficulty from anchors

* CheckInitiation

* Fix tests

* Fix lint, print more at check initiation

* Better output for hard-coded anchors

* Print hard-coded anchors better

* Prioritise anchors with short stretches

* Prioritise by chainSize

* Use linked list instead of heap for requestQueue

* Fix problem of no requests

* Push front

* Fix lint

* Not verify PoW for too far in the past and future

* Fix Calculation of totalDifficulty when Connect

* Fix hard tips

* Another fix for tips
2020-09-27 21:32:05 +01:00
ledgerwatch
5aab794c9b
Headers poc 4 - intermediate changes (#1125)
* Process BlockHeadersMsg

* Print invalidations

* Print reason for parent-child mismatch

* Fix off by one error

* Fix ExtendDown

* Replace tipLimiter with an LRU cache

* Retrieve 256 headers at a time

* Hard coding of tips, process new block hashes

* extract headers

* Write hard-coded headers into a file

* Add total difficulty

* Insert hard coded headers

* Log i

* Initiate download from hard-coded anchors

* Print the state of anchors

* Add hard-coded headers as tips

* Spare the peers

* Refine LRU caching of tips

* Print length of anchor trees

* Limit requests to 192 headers

* range error fix

* Print better, retain at least one tip in the LRU cache

* Skip duplicate line ends

* Throttle requests, display peer name

* Back off, intermediate tip queue

* Print block header timestamp

* Better logging

* Log less

* More logging of evictions

* Fix queue push

* Fix queue pop

* Exclude duplicate tips

* Fix tips logging

* Fix moving

* Add port flag

* Check headers

* Print extra

* Add mixDigest to serialisation

* Fix difficulty in hard-coded headers

* Avoid duplicate header requests

* Update peer characteristics on NewBlockMsg

* Remove backoff when peer responds

* Fix tests

* Fix lint

* Fix lint
2020-09-20 10:47:24 +01:00
ledgerwatch
ab4c456fec
Headers poc 3 - intermediate changes (#1093)
* Use log instead of Printf

* Better output

* Display NewBlockHeadersMsg

* Print tx hashes

* Separate sentry and downloader

* Fix lint

* Requesting headers

* Fix lint

* Prevent deadlock, fix NPE

* Not quit when did not restore

* Fix types

* Fix types again

* Warn on penalty

* Prints

* Avoid deadlock

* Better request logic

* Print height

* Actually send header requests

* Actually send header request

* Fix compile errors, sending

* Print BlockHeadersMsg

* Fix lint
2020-09-11 07:35:51 +01:00
ledgerwatch
256e4262e1
2nd batch of Headers PoC changes (#1067)
* PoC utility

* Headers cmd

* headers download command

* Fix lint

* Send handshake

* Receive Status message

* Better error messages

* Messages

* Fix

* Fix lint

* Print block number for NewBlockMsg

* Fix lint
2020-09-07 07:03:12 +01:00