erigon-pulse/cmd/headers/check/check.go

34 lines
869 B
Go
Raw Normal View History

Headers poc 5 - Intermediate (#1145) * Add headers persistence * Print flushBuffer * Fix indexing problem * Not skip hard-coded headers if the files are empty * Fix lint * print anchor state after init * Properly construct file names * Add check sub-command * Fix lint * Fix lint * Fix lint * Print more info when recovering * Fix recovering * Fix recovery * Add anchors also as tips * 2-level priority queue for tips * Initialise tipQueue in anchor * update maxTipHeight * fix type * Add anchors to the anchorQueue and rebuild anchorQueue on deletion * Fix NPE * fix recovery * User buffersize, add hard coded headers to buffer * Reinit anchorQueue * Schedule requests after recovery * No fix * Remove duplicates * Report duplicate headers * Log buffer additions * Fix for duplicate headers * Try to fix duplicate headers again * remove TODO comment * Use LLRB instead of heap for anchors * Print reserveTip info * Correctly replace anchors in the tree * Remove excessive logging * Print tips attached to the anchor * Print tips better * limitTips instead of reserveTip * Print forked headers * Use pointers in llrb * Print tipStretch * Print limitTips * Mininise AnchorItem * Put anchors into the tree * Fix totalDiff calculation, but is it needed? * Remove totalDifficulty from anchors * CheckInitiation * Fix tests * Fix lint, print more at check initiation * Better output for hard-coded anchors * Print hard-coded anchors better * Prioritise anchors with short stretches * Prioritise by chainSize * Use linked list instead of heap for requestQueue * Fix problem of no requests * Push front * Fix lint * Not verify PoW for too far in the past and future * Fix Calculation of totalDifficulty when Connect * Fix hard tips * Another fix for tips
2020-09-27 20:32:05 +00:00
package check
import (
"time"
"github.com/ledgerwatch/turbo-geth/common"
Headers poc 5 - Intermediate (#1145) * Add headers persistence * Print flushBuffer * Fix indexing problem * Not skip hard-coded headers if the files are empty * Fix lint * print anchor state after init * Properly construct file names * Add check sub-command * Fix lint * Fix lint * Fix lint * Print more info when recovering * Fix recovering * Fix recovery * Add anchors also as tips * 2-level priority queue for tips * Initialise tipQueue in anchor * update maxTipHeight * fix type * Add anchors to the anchorQueue and rebuild anchorQueue on deletion * Fix NPE * fix recovery * User buffersize, add hard coded headers to buffer * Reinit anchorQueue * Schedule requests after recovery * No fix * Remove duplicates * Report duplicate headers * Log buffer additions * Fix for duplicate headers * Try to fix duplicate headers again * remove TODO comment * Use LLRB instead of heap for anchors * Print reserveTip info * Correctly replace anchors in the tree * Remove excessive logging * Print tips attached to the anchor * Print tips better * limitTips instead of reserveTip * Print forked headers * Use pointers in llrb * Print tipStretch * Print limitTips * Mininise AnchorItem * Put anchors into the tree * Fix totalDiff calculation, but is it needed? * Remove totalDifficulty from anchors * CheckInitiation * Fix tests * Fix lint, print more at check initiation * Better output for hard-coded anchors * Print hard-coded anchors better * Prioritise anchors with short stretches * Prioritise by chainSize * Use linked list instead of heap for requestQueue * Fix problem of no requests * Push front * Fix lint * Not verify PoW for too far in the past and future * Fix Calculation of totalDifficulty when Connect * Fix hard tips * Another fix for tips
2020-09-27 20:32:05 +00:00
"github.com/ledgerwatch/turbo-geth/log"
"github.com/ledgerwatch/turbo-geth/turbo/stages/headerdownload"
)
func Check(filesDir string) error {
log.Info("Checking", "directory", filesDir)
hd := headerdownload.NewHeaderDownload(
common.Hash{}, /* initialHash */
Headers poc 5 - Intermediate (#1145) * Add headers persistence * Print flushBuffer * Fix indexing problem * Not skip hard-coded headers if the files are empty * Fix lint * print anchor state after init * Properly construct file names * Add check sub-command * Fix lint * Fix lint * Fix lint * Print more info when recovering * Fix recovering * Fix recovery * Add anchors also as tips * 2-level priority queue for tips * Initialise tipQueue in anchor * update maxTipHeight * fix type * Add anchors to the anchorQueue and rebuild anchorQueue on deletion * Fix NPE * fix recovery * User buffersize, add hard coded headers to buffer * Reinit anchorQueue * Schedule requests after recovery * No fix * Remove duplicates * Report duplicate headers * Log buffer additions * Fix for duplicate headers * Try to fix duplicate headers again * remove TODO comment * Use LLRB instead of heap for anchors * Print reserveTip info * Correctly replace anchors in the tree * Remove excessive logging * Print tips attached to the anchor * Print tips better * limitTips instead of reserveTip * Print forked headers * Use pointers in llrb * Print tipStretch * Print limitTips * Mininise AnchorItem * Put anchors into the tree * Fix totalDiff calculation, but is it needed? * Remove totalDifficulty from anchors * CheckInitiation * Fix tests * Fix lint, print more at check initiation * Better output for hard-coded anchors * Print hard-coded anchors better * Prioritise anchors with short stretches * Prioritise by chainSize * Use linked list instead of heap for requestQueue * Fix problem of no requests * Push front * Fix lint * Not verify PoW for too far in the past and future * Fix Calculation of totalDifficulty when Connect * Fix hard tips * Another fix for tips
2020-09-27 20:32:05 +00:00
filesDir,
32*1024, /* bufferLimit */
16*1024, /* tipLimit */
1024, /* initPowDepth */
nil,
nil,
3600, /* newAnchor future limit */
3600, /* newAnchor past limit */
)
Bodies download and simplify header download (#1471) * Only insert hard-coded tips if both DB and files recovery failed * Prevent deadlock * Report some efficiency numbers * Count properly unrequested bodies * Initialise allRequests * Increase timeout * Fixes to scheduling * Small fix * Simplified scheduling * Remove separate bodyLoop goroutine * Update from DB at the beginnig of block bodies forward * Timeout for repeating the request cycle * Fix timeout * Fix * Increase timeout * Increase timeout * Try to make flow if possible * Fix flow * Lower timeout * timeout for each blockNum * Adjustable timeout * Better log timing * Track peers * copy peerID * fix scheduling * Too much logging * Print delivery speed * Print committed blocks * Fix race * Sentry to only reset back-off timer when response to a request is received * Print bytes/sec and wasted traffic * Fix bandwidth accounting * Less logging * not to wake up on deliveries * Spam every second * Print peer map * Fix npe, print requests * Timestamps * Improved logging * Penalty for peers * Log penalties only when disconnecting * Try with smaller window * window parameter * Dealing with partially delivered requests * Init bodyReq * Fix array index * More fix for NPE * More NPE checks * Print out body progress and header progress * Fix ending condition * Bring back waking up and penalties * Fix duplicate tip * Duplicate segment * Fix lint * Fix lint * fix lint * Fix lint * Hard coded headers in the source files * Fix lint * Replace hix-sized header serialisation with rlp (to support clique) * Remove anchor records from the files * Fixeds for DB recovery * Fix compilation * Fix compile errors * Fix formatting * Fix lint * Fix comments * Remove headerLoop * Properly terminate body download * Support for StatusData p2p proto * Fix forkid test * Fix test * Fix lint Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-08 09:25:10 +00:00
if recovered, err := hd.RecoverFromFiles(uint64(time.Now().Unix()), make(map[common.Hash]headerdownload.HeaderRecord)); err != nil || !recovered {
Headers poc 5 - Intermediate (#1145) * Add headers persistence * Print flushBuffer * Fix indexing problem * Not skip hard-coded headers if the files are empty * Fix lint * print anchor state after init * Properly construct file names * Add check sub-command * Fix lint * Fix lint * Fix lint * Print more info when recovering * Fix recovering * Fix recovery * Add anchors also as tips * 2-level priority queue for tips * Initialise tipQueue in anchor * update maxTipHeight * fix type * Add anchors to the anchorQueue and rebuild anchorQueue on deletion * Fix NPE * fix recovery * User buffersize, add hard coded headers to buffer * Reinit anchorQueue * Schedule requests after recovery * No fix * Remove duplicates * Report duplicate headers * Log buffer additions * Fix for duplicate headers * Try to fix duplicate headers again * remove TODO comment * Use LLRB instead of heap for anchors * Print reserveTip info * Correctly replace anchors in the tree * Remove excessive logging * Print tips attached to the anchor * Print tips better * limitTips instead of reserveTip * Print forked headers * Use pointers in llrb * Print tipStretch * Print limitTips * Mininise AnchorItem * Put anchors into the tree * Fix totalDiff calculation, but is it needed? * Remove totalDifficulty from anchors * CheckInitiation * Fix tests * Fix lint, print more at check initiation * Better output for hard-coded anchors * Print hard-coded anchors better * Prioritise anchors with short stretches * Prioritise by chainSize * Use linked list instead of heap for requestQueue * Fix problem of no requests * Push front * Fix lint * Not verify PoW for too far in the past and future * Fix Calculation of totalDifficulty when Connect * Fix hard tips * Another fix for tips
2020-09-27 20:32:05 +00:00
if err != nil {
log.Error("Recovery from file failed, will start from scratch", "error", err)
} else {
log.Info("Nothing recovered")
}
}
log.Info(hd.AnchorState())
return nil
}