Commit Graph

124 Commits

Author SHA1 Message Date
ledgerwatch
15b4095718
Move ETL to erigon-lib (#2667)
* Move ETL to erigon-lib

* Update link in the readme

* go mod tidy

* Use common/chan.go from erigon-lib

* Clean up

* Fix lint

* Fix test

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-12 08:50:17 +01:00
Alex Sharov
99317663f8
No open db in main, right clique sign, periodically release write tx on devnet (#2642) 2021-09-08 11:13:34 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
f735997fa1
Sokol v0: remove unused parameters and methods (#2435) 2021-07-24 11:04:07 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition (#2411) 2021-07-21 18:13:26 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00
Alex Sharov
76bc954283
Sokol v0: can process 1-st block (#2310) 2021-07-08 19:40:43 +07:00
Alex Sharov
8e00da2035
Sokol v0: preparations for contracts run, proofs support (#2270) 2021-07-02 15:00:41 +07:00
Alex Sharov
6df7230108
Sokol v0 - better header rlp, steps to generate blocks (#2256) 2021-06-30 16:30:21 +07:00
Alex Sharov
605d7a8d59
Sokol v0 (#2228)
* handle chain flag in integration

* handle chain flag in integration

* handle chain flag in integration

* save

* save

* merge devel

* save

* noopWriter - one is enough

* chain spec parser

* chain spec parser

* embed

* embed

* embed

* embed

* embed

* embed

* embed

* clean

* clean

* correct alloc after reset state

* correct alloc after reset state

* correct alloc after reset state

* integration reset state now does re-apply genesis and chainConfig

* eips summary

* eips summary

* eips summary
2021-06-25 19:13:40 +01:00
Alex Sharov
59d05dc5fe
hide file exists err (#2218) 2021-06-22 11:09:45 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Alex Sharov
b160e30652
Send GetBlockHeaders immediately if TD of connected peer >= our TD (#2169)
* empty engine

* start sync from peer right after handshake if it's TD is higher

* fixCanonicalChain to print logs

* clean

* >
2021-06-15 13:43:55 +01:00
BitBaseBit
7ed337cdcc
Implemented panic handling, graceful shutdown and reporting for all goroutines that don't explicitly handle them. (#2153)
* implemented crash reporting for all goroutine panics that aren't handled explicitly

* implemented crash reporting for all goroutine panics that aren't handled explicitly

* changed node defaults back to originals after testing

* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports

* handling panics on all goroutines gracefully

* updated missing call

* error assignment

* implemented suggestions

* path.Join added

* implemented Evgeny's suggestions

* changed path.Join to filepath.Join for cross-platform

* added err check

* updated RecoverStackTrace to LogPanic

* updated closures

* removed call of common.Go to some goroutines

* updated scope capture

* removed testing files

* reverted back to original method, I feel like its less intrusive

* update filename for clarity
2021-06-13 17:41:39 +01:00
Alex Sharov
ccc028daf6
Fix clique snapshot err check (#2115)
* save

* save

* save

* save
2021-06-07 12:00:38 +01:00
ledgerwatch
73bb98f686
Move tests 6 (#2107)
* Move tests 6

* Compile fixes

* Fix lint

* Compile fixes

* Fix compile

* compile fixes

* Compile fix

* Fix consesus/clique

* Cleanup

* Add gas limit

* Print

* Prints

* More print

* Fix

* Reinstate TestGetBlockReceipts66

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-06 22:44:14 +01:00
Alex Sharov
5ba3ea162a
Simulated backend and genesis to kv (#2104)
* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* merge

* merge

* merge

* merge

* merge

* merge

* merge

* merge
2021-06-05 16:17:04 +01:00
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018)
* turbo-geth to erigon

* tg, turbo to erigon
2021-05-26 11:35:39 +01:00
ledgerwatch
7d98b6504a
Improvements in the mock sentry tests (#1997)
* Improvements in the mock sentry tests

* GenerateChain to return chainpack

* Fix consensus

* Fix core

* Fix core/state

* Fix eth

* Fix eth/download

* Fix eth/fetcher

* Fix eth/filters

* Fix eth/gasprice

* Fix eth/protocols/eth

* Fix tests

* Fix turbo/stages

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-23 17:10:34 +01:00
Alex Sharov
a8e8cab4c5
Less blockchain obj in test (#1995) 2021-05-23 10:48:31 +07:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
Alex Sharov
50934fa5d3
Move chain maiker to kv (#1972) 2021-05-20 18:49:33 +07:00
Alex Sharov
781a67a4ee
Auto-Close db in tests by using t.Cleanup (#1956) 2021-05-19 10:47:28 +07:00
Alex Sharov
47d22773b9
[wip] dv2: Validate body (#1913)
* validate body

* validate body

* save

* save

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* body deliveries messages handling into staged sync

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch

* penalize right after verifyUncles and stop, don't batch
2021-05-12 14:57:35 +01:00
ledgerwatch
447f20d395
Baikal testnet configuration (#1910)
* Baikal testnet configuration

* Fixes

* Change AsMessage signature

* Fixes in t8ntool

* More updates

* Fix misspelling

* Skip eip1559 transition test

* More bootnodes

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-11 10:54:20 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync (#1899) 2021-05-08 00:07:49 +03:00
ledgerwatch
8c9a76d3bc
Add Baikal checks properly (#1868)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 06:50:06 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
ledgerwatch
75ca6b8c76
Initial work on integration tests (#1797)
* Initial work on integration tests

* Delete subtree

* Squashed 'interfaces/' content from commit 41a082ba4

git-subtree-dir: interfaces
git-subtree-split: 41a082ba4bde38647325eb0416cb1da1b4ca2b12

* Add consensus interfaces

* More stuff

* comments

* Fix compile

* Squashed 'interfaces/' changes from 41a082ba4..1b13a42a7

1b13a42a7 Add chainspec to consensus interface

git-subtree-dir: interfaces
git-subtree-split: 1b13a42a7803f5464722867a71065c27a7ebe8c3

* Squashed 'interfaces/' changes from 1b13a42a7..93a072c4c

93a072c4c Add missing import

git-subtree-dir: interfaces
git-subtree-split: 93a072c4c099d169322a3a53b95e40203276820b

* New consensus interfaces

* More on clique

* Fix tests

* Squashed 'interfaces/' changes from 93a072c4c..62f4ec4b2

62f4ec4b2 Add test service for consensus engine

git-subtree-dir: interfaces
git-subtree-split: 62f4ec4b263107635ffa3aabd5d634af22e813c6

* Squashed 'interfaces/' changes from 62f4ec4b2..061a63543

061a63543 Fix

git-subtree-dir: interfaces
git-subtree-split: 061a63543babdeb51ab7e3a96dec56b2485d4389

* Configuring clique engine with toml specs - start

* More toml parsing

* Constructed rinkeby genesis

* Simplify VerifyHeaders functions

* Fix lint

* Remove concurrent verification tests

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-29 16:14:10 +01:00
Alex Sharov
9d82456f17
remove chain indexer and core.setHead func (#1826) 2021-04-27 14:48:59 +07:00
ledgerwatch
302e85ba1e
Fix for nil parent (#1802)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-25 14:36:38 +01:00
ledgerwatch
9c0b3f59fa
Simplify clique (#1801)
* Initial

* More fixes

* Fix

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-25 09:28:53 +01:00
Evgeny Danilenko
984daa4d7a
Clique snapshot timer fixed (#1789)
* fix clique snapshot storage by timer

* after merge

* remove debug

* remove debug

* debug logging

* correct t.Fatal
2021-04-23 20:47:06 +03:00
ledgerwatch
b308afce1e
Clique fixes (#1785)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-23 09:28:28 +01:00
ledgerwatch
d9402bfcba
Compile error fix (#1784)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-04-22 18:52:30 +01:00
ledgerwatch
5501af8367
Clean up runPeer (#1780)
* Clean up runPeer

* Improvements

* Fixes

* Clique fixes

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:12:13 +01:00
ledgerwatch
12cde41772
Aleut support (Eip1559) (#1704)
* Where I am at

* Refactoring of transaction types

* More refactoring

* Use Homested signer in rpc daemon

* Unified signer

* Continue unified signer

* A bit more

* Fixes and down the rabbit hole...

* More tx pool fixes

* More refactoring fixes

* More fixes'

* more fixes

* More fixes

* More compile fixes

* More RLP hand-writing

* Finish RLP encoding/decoding of transactions

* Fixes to header encoding, start on protocol packets

* Transaction decoding

* Use DecodeTransaction function

* Decoding BlockBodyPacket

* Encode and decode for pool txs

* Start fixing tests

* Introduce SigningHash

* Fixes to SignHash

* RLP encoding fixes

* Fixes for encoding/decoding

* More test fixes

* Fix more tests

* More test fixes

* More test fixes

* Fix core tests

* More fixes for signer

* Fix for tx

* Fixes to string encoding/size

* Fix eip2930 test

* Fix rest of ./tests

* More fixes

* Fix compilation

* More test fixes

* More test fixes

* Test fixes

* More fixes

* Reuse EncodingSize in EncodeRLP for accessList

* Rearrange things in dynamic fee tx

* Add MarshalBinary

* More fixes

* Make V,R,S non-pointers

* More NPE fixes

* More fixes

* Receipt fixes

* Fix core/types

* Fix ./eth

* More compile fixes for tests

* More test fixes

* More test fixes

* Try to see lint errors better

* Try to see lint errors better

* Fix lint

* Debugging eip1559 test

* Fix TestEIP1559Transition test

* Fix NewBlockPacket encoding/decoding

* Fix calculation of TxHash

* Fix perf problem with senders

* Update aleut config values

* Try adding static peers

* Add staticpeers to defaul flags

* Change aleut networkID

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB (#1703)
* debug

* debug

* it works

* rename clique bucket

* remove genesis special case

* copy snapshot

* remove debug

* migration

* debug

* regenerate snapshots

* simplify

* regeneration

* after merge

* tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
a913ae8847
Remove chainContext object (#1683) 2021-04-06 16:52:53 +07:00
Alex Sharov
8ccc6b2664
Mining: remove local pow mining (from ethash), --miner.notify is required now, cycle is non-blocking now (#1617) 2021-03-28 21:40:27 +07:00
Alex Sharov
416a69ecb3
Genesis.toBlock remove db from arguments (#1611) 2021-03-28 16:27:06 +07:00
Artem Vorotnikov
afe551ba1c
Remove puppeth & account management (#1610) 2021-03-26 21:05:42 +03:00
ledgerwatch
0986ce43fb
Fix some conformance tests broken after rebase (#1587)
* Uncomment

* Try to fix

* Try again

* Compile fix

* Compile fix

* Compile fix

* Compile fix

* Compile fix

* fix lint

* Enable more tests

* Fix core tests

* Remove comment outs

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-23 07:25:10 +00:00
Guillaume Ballet
1ccf0ee51c consensus: remove seal verification from the consensus engine interface (#22274)
# Conflicts:
#	consensus/ethash/algorithm_test.go
2021-03-10 10:27:50 +01:00
ledgerwatch
b743b88218
Revert "Extract validating interface (#1120)" (#1543)
* Revert "Extract validating interface (#1120)"

This reverts commit e4e36c152e.

* Fix go mod tidy

* Fix lint

* Fix lint

* fix lint

* Test fix

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-05 20:34:23 +00:00
Evgeny Danilenko
da9c92f559
fix (#1529) 2021-03-01 19:28:32 +00:00
Evgeny Danilenko
e4e36c152e
Extract validating interface (#1120)
* interface

* generalize interface

* linters

* fix deadlock

* fix linters

* close goroutine

* fix

* debug

* id+ttl

* refactor downloader tests

* tests

* lru

* handle genesis, extract fake consensuses

* fix fake consensus

* test uncles, verify

* after a new master

* fmt

* fix close

* debug

* debug

* fix chain length

* remove test field

* use single account

* fix data race on closing channel

* remove postponed blocks queue

* miner test

* VerifyHeaderRequests

* fmt

* fmt

* fix data race

* handle validating errors

* simplify matchParents

* remove copy-paste

* move sort to constructor

* clean up

* debug for 10 parents

* debug

* debug

* batch responses

* batch requests

* works for many ancestors

* remove debug

* always Close an engine

* linters

* ancestors deduplication

* fix test

* reduce interface

* api

* clique

* green clique sync

* stable

* perpermance per second

* full sync

* linters

* gitignore

* deps

* fix panic after master merge

* init consensus

* clique tests

* fix tests

* fix tests

* clean up

* reuse snap

* store vefified snapshots

* optimize snapshots

* safe close

* cleanup loop

* after downloader

* downloader and consensus tests

* update tests

* hack.go

* clique flags

* fix cliuqe config

* review

* gitignore

* remove additional bucket

* blk/sec instead of blk/microsecond

* rename

* deps

* optimize

* debug

* test

* tests without extracted validation process

* same base performance as on master

* benchmark

* simplify ethash verification

* ethash

* ethash

* linters

* ethash

* master stats

* cleanup

* gomod

* linters

* tests

* better locks

* Fix

* Remove logging for verifyHeaders

* Verification speed in the logs

* Fix compile error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-25 19:40:45 +00:00