mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-25 13:07:17 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
15,342 Commits 5 Branches 14 Tags 294 MiB
f03d08c5ce
Commit Graph

379 Commits

Author SHA1 Message Date
Alex Sharov
f03d08c5ce
Snapshots: retire testing tool (#3684) 2022-03-12 17:26:11 +07:00
battlmonstr
04f07a0f08
Discovery: split node records to a sepatate DB table (#3581) (#3667) 
Problem:
QuerySeeds will poke 150 random entries in the whole node DB and ignore hitting "field" entries.
In a bootstrap scenario it might hit hundreds of :lastping :lastpong entries,
and very few true "node record" entries.
After running for 15 minutes I've got totalEntryCount=1508 nodeRecordCount=114 entries.
There's a 1/16 chance of hitting a "node record" entry.
It means finding just about 10 nodes of 114 total on average from 150 attempts.

Solution:
Split "node record" entries to a separate table such that QuerySeeds doesn't do idle cycle hits.
 2022-03-10 14:57:25 +01:00
battlmonstr
ca97325448
Discovery: throttle node DB commits (#3581) (#3656) 
UpdateFindFails/UpdateLastPingReceived/UpdateLastPongReceived events
are causing bursty DB commits (100 per minute).

This optimization throttles the disk writes to happen at most once in a few seconds,
because this info doesn't need to be persisted immediately.

This helps on HDD drives.
 2022-03-10 14:02:40 +01:00
Alex Sharov
b003d07839
Add system-txs to begin end of block (#3654) 
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
 2022-03-10 07:48:58 +00:00
ledgerwatch
3cab0c786f
[erigon2] Allow uncompressed words (#3655) 
* Optimise compressor

* Update erigon-lib

* Update

* Update

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-03-09 20:45:14 +00:00
Alex Sharov
0b2ce6f845
rpcdaemon: limit amount of read transactions (#3637) 
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* a

* a

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* fefefe

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
 2022-03-08 10:43:14 +07:00
Alex Sharov
6d9baab566
up torrent version (#3647) 2022-03-05 10:05:31 +07:00
Alex Sharov
902824a8cf
kv: forAmount noop on 0 amount (#3644) 2022-03-04 09:14:54 +07:00
Alex Sharov
230a177d4c
mdbx v0.11.5(#3624) 2022-03-04 09:07:51 +07:00
Giulio rebuffo
cb8aacae87
Added Engine Authentication [JWT] (#3531) 
* jwt

* fuuuuture

* added auth

* merge

* merge

* Update jwt to the latest version

* ops

* comments

* cleanup

* bad

* gut

* maybe

* mod sum

Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
 2022-02-28 12:07:09 +01:00
Giulio rebuffo
99f688c3ec
PrevRandao Renaming (#3616) 
* PrevRandao Renaming

* lint
 2022-02-24 18:16:05 +01:00
Alex Sharov
f685d6bfe5
logger version up (#3608) 
* snapshots: fix indexing of recently merged range

* logger version up
 2022-02-24 15:14:41 +07:00
Alex Sharov
3067cf9999
snapshots: ensure tmpdir exists (#3602) 2022-02-24 08:37:16 +07:00
ledgerwatch
e784f2b809
[erigon2] ETL optimisations (#3550) 2022-02-21 09:31:06 +07:00
ledgerwatch
80d7734ea8
[erigon2] fix history creation (txNum confusion) (#3547) 
* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-19 08:15:35 +00:00
Alex Sharov
952085a83d
last version of kanzi-go (#3543) 2022-02-18 10:14:00 +07:00
Alex Sharov
2755af2345
more_stable_grpc_err_check (#3542) 2022-02-18 09:55:39 +07:00
Alex Sharov
3de83e2d96
RPCDaemon: to start with disabled txpool (#3539) 2022-02-18 09:45:20 +07:00
Alex Sharov
226982d293
Use class dir.Rw - to separate Ro and Rw access to snapshotDir (#3534) 
* save

* save

* save

* save

* save

* save

* save
 2022-02-18 09:24:17 +07:00
Alex Sharov
e74b9ef9ff
Embed rpcdaemon fixes (#3535) 
* save

* save
 2022-02-17 11:24:21 +07:00
ledgerwatch
c5b4d3e352
[erigon2] Thin commitment (2nd attempt) (#3515) 
* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-02-17 10:56:20 +07:00
Alex Sharov
6d9ea162ab
rpc daemon embed (step 3) (#3519) 2022-02-16 11:24:51 +07:00
ledgerwatch
dffcc57ecc
[erigon2] Fix history bug, optimise bitmap search (#3510) 
* Fix history bug, optimise bitmap search

* Optimise SeekInBitmap64

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-13 20:02:18 +00:00
Alex Sharov
7566e1dbbe
return EOF on direct stream end (#3504) 2022-02-13 09:34:02 +07:00
ledgerwatch
2074b98568
Tools for gathering data on erigon2 history and bitmaps (#3488) 
* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-12 15:48:38 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
Alex Sharov
479c7f3d9d
Embed rpcdaemon: prepared direct clients (#3492) 2022-02-12 19:47:19 +07:00
Alex Sharov
e5ba5c8c59
grpc disabled server txpool (#3485) 2022-02-12 12:24:19 +07:00
ledgerwatch
eb6a168603
[erigon2] Chain history and bitmap indices (#3437) 
* First update

* Update to latest erigon-lib

* Fix lint

* Update

* Update

* Update

* Add history2 command

* Updates

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-02-12 00:06:57 +00:00
Alex Sharov
a1318f869a
make services net-independent (#3481) 2022-02-11 17:11:59 +07:00
Alex Sharov
3d3315132c
downloader switch back to bolt backend (#3478) 2022-02-11 12:44:18 +07:00
Alex Sharov
2e17b8f515
snapshot: parallel transaction index, recsplit: reset offset collector, etl: faster flush and load (#3465) 2022-02-10 14:40:29 +07:00
Alex Sharov
e695a6e58b
state and newHead: 1 goroutine per subscriber (#3452) 2022-02-10 08:25:58 +07:00
Andrew Ashikhmin
8466cb1b7b
New Engine API semantics (Kiln v1) (#3340) 
* Disable PoS sync temporarily

* Resore PoS sync

* Handle Ctrl^C in HeadersPOS

* Consistent naming

* Extract verifyAndSavePoSHeader & downloadMissingPoSHeaders

* Preparation for EngineForkChoiceUpdated re-orgs

* Extract ForkingPoint

* Comments

* New proto for Engine API

* EngineExecutePayload -> EngineNewPayload

* Return INVALID_BLOCK_HASH if block hash is invalid

* Return EngineStatus_ACCEPTED for side chain blocks

* Update erigon-lib (PR 268)

* Fix payload2Hash

* reverseDownloadCh -> beaconPayloadCh

* Update erigon-lib

* Engine API updated

* ExecutionStatus -> PayloadStatus

* Introduce forkChoiceCh

* Mock ForkChoiceMessage/PayloadStatus

* Add ValidationError to PayloadStatus

* Small clean-ups

* Add INVALID_TERMINAL_BLOCK to EngineStatus

* Add a comment

* Extract handleNewPayload & handleForkChoice

* Partially implement handleForkChoice

* Update erigon-lib

* short vs long re-org

* Move header insertion out of downloadMissingPoSHeaders

* Update erigon-lib

* Refactor ProcessSegmentPOS

* Fix imports

* Wire downloadMissingPoSHeaders into handleForkChoice

* evictOldPendingPayloads

* nolint:unused for assertSegment

* Try nolint instead of nolint:unused

* Small comment improvements

* HeadHeaderHash/StageProgress in handleForkChoice

* TODO: bodyDownloader.AddToPrefetch(block)

* Review call suggestions

* Don't use ReadHeaderNumber in ProcessSegmentPOS

* Don't leave ethbackend waiting when server is stopping

* Update erigon-lib

* More explicit signature of downloadMissingPoSHeaders
 2022-02-09 08:33:22 +01:00
Alex Sharov
1d6f8a04ac
Embed downloader (#3450) 
* save

* save

* save

* save

* save

* save
 2022-02-09 13:22:43 +07:00
Enrique Jose Avila Asapche
35fcd3a829
Merging Turbo bor into devel (#3372) 
* implemented bor consensus

* add bor flags to default

* change bucket into snapshot to clique

* enable stateSync

* bypass reciept checks

* fix receipt calculation and bor logs

* fix: contract call wrt bor

* Update mumbai config

* Add: bor-mainnet flag and config

* Add bor consensus to integration

* use header coinbase in block context

* london fork mumbai changes

* fix genesis error

* Jaipur fork for mumbai

* add sysCall to verifyHeader

* added bor related rpc method implementation

* added bor specific rpc extensions

* fixes in snapshot implementation, major refactor for bor rpc

* modify consensus specific db path for bor

* fix: remove parallel compute for get root hash rpc method

* Added bor-receipt flow

* Use turbo-bor-lib and bor tables

* Use bor table in RPC snapshot

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* update rpc readme

* link rpc docs in readme

* Update Readme

* Update Readme

* move erigon namespace rpc methods to eth

* rm: erigon namespace

* rm: erigon namespace, update list of available rpc methods, add example

* fix: binary name in rpc readme

* fix: max db size

* Add london to bor-mainnet

* updated node.go

* add system req to readme

* golang version fix readme

* added networknames in correct place

* nil

* ran gofmt

* erigon

* fixed fake.go

* dont need turbor-lib

* old readme

* fixing readme

* half

* other half

* changed return

* fixing return

* fixed return

* fixed flags

* gofmt

* merge with devel

* latest erigon-lib

* fixed context.coinbase

* took out syscall

* fixed params in hash

* bor type now is consensus.Engine

* parlia is consensus.Engine

* missing arg and repeated importation

* repeated importation

* fixed eth_receipts.go

* deleted duplicate issuance

* part of consensus.Engine type

* added eth_api issuance

* networkname

* added erigon_system file

* fork struct taken out

* added erigon block

* getLogByHash for erigonImpl

* gofmt

* fixed lint

* ops

* gofmt

* gofmt

* added APIImple functions

* fixed clique test

* took out print

* fixed state added balance

* fixed README

* fixed rpcDaemon README

* fixed integration README

* updated blockchain.go

* lint

* added bor back into blockchain.go

* took out comment

* lint

* updated daemon

* updated wtb

* removed duplicate

* removed VerifyHeaders

* prevent use of wrong Transfer

* fixed state_processor.go

* fixed state_transition.go

* fixed headers

* returning err

* error handling in bor read tx look up

* put for txLookUp

* dealing with error

* lint

* traces

* more traces

* fixed receipt in execution

* getTrasanction receipt for bor or others

* nil

* lint

* ops

* deleted syscall

* took out else

* Merge branch 'devel

* tests syscalls

* changed borReceipt to receipt

* reset header algos

* arguments fix

* took out prefixes

* lint

* erigon-named

* borReceiptKey = blocknumber

* reverts e3b60c2e159d03efcb855f7ab3da5a098dd60c33.

* correct hashing tx

* dont need it here

* lint

* added txlookup for bor

* change to uint256

* outputs for isBor

* wrapper

* added isBor and isParlia

* isBor

* fixed BorTransfer

* not readBody

* correct prefix

* added blockNum

* added readStorageBody

* readStorageBody

* lint

* got rid of unnecessary bor_receipt func

* onlny if bor

* use clone

* append

* writeToSlice

* added isBor flag

* fixed writeToSlice

* normal sorting

* lint

* Reset erigon-snapshots

* Move bor prefix into if

Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Uttam Singh <uttamkhanduja@yahoo.in>
Co-authored-by: Giulio Rebuffo <giulio.rebuffo@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-02-07 21:30:46 +00:00
Alex Sharov
4da9a663f8
torrent: downgrade lib version #3433 2022-02-05 15:53:51 +07:00
ledgerwatch
bb8a17e422
[erigon2] Proper reset of decompressor getter (#3418) 
* Trying to reset

* Update

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-04 20:43:34 +00:00
Alex Sharov
29a4adfbaa
ParallelCompressor: Remove intermediate ETL collectors (#3427) 
* save

* save

* save
 2022-02-04 16:48:16 +07:00
Alex Sharov
f89669674d
torrent version 1.40.1 (#3422) 2022-02-04 09:27:21 +07:00
ledgerwatch
71909c44db
Fix txpool leak (#3417) 
* Fix txpool leak

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-03 18:51:51 +00:00
ledgerwatch
093bfbad39
[erigon2] Optimisations (#3412) 
* Update

* [erigon2] optimisations

* Update to latest erigon-lib

* Update to latest erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-02-03 09:36:27 +00:00
Alex Sharov
fc6e5d7e63
pool: fix race in async broadcast (#3402) 
* pool: better async broadcast

* save

* save
 2022-02-02 14:57:30 +07:00
Alex Sharov
12e91b39fa
pool: hotfix for slow "new txs notifications" (#3398) 
* hotfix for slow "new txs notifications"

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-01 16:08:38 +00:00
ledgerwatch
f0c635b48f
[erigon2] Fix deadlock (#3397) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-01 12:04:39 +00:00
Alex Sharov
5567eebb23
Decompressor.WithReadAhead (#3395) 
* readahead

* readahead
 2022-02-01 11:19:16 +07:00
ledgerwatch
4ef14215b4
[erigon2] three threads, better stats in the logs (#3378) 
* [erigon2] Use compact keys in commitments

* Better stats in the logs

* Update

* Update

* Update

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-31 23:40:51 +00:00
ledgerwatch
02ab452cfa
[erigon2] Use compact keys in commitments (#3368) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-28 22:16:23 +00:00
Alex Sharov
c87a18a9a3
parallel compressor: less read of dat file (#3361) 
* save

* save

* save
 2022-01-27 17:13:32 +07:00
Alex Sharov
6385a35ee8
remove one more logger (#3360) 2022-01-27 13:31:53 +07:00