Commit Graph

78 Commits

Author SHA1 Message Date
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
6801208497
move some tests to kv interface (#2438) 2021-07-24 14:14:11 +07:00
Alex Sharov
7c60cc0fee
step towards kv interface (#2313) 2021-07-08 09:56:09 +07:00
Alex Sharov
a68b5ba361
Replace ChainConfig.WithEIPsFlags by go-ethereum's ChainConfig.Rules (#2304)
* use chainRules

* use chainRules

* use chainRules

* use chainRules

* use chainRules
2021-07-05 19:52:50 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Alex Sharov
1f13f73045
up version of uint256 lib (#2082)
* eip 1559_tx_constraints

* eip 1559_tx_constraints

* return error on uint256 overflow

* up uint256 version

* merge master

* fix master

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2021-06-04 17:25:28 +01:00
Alex Sharp
7fd335308d Switch more tests to new staged sync 2021-05-31 19:21:53 +01:00
ledgerwatch
7ffd67e973
Move tests 4 (#2057)
* Convert tests to new staged sync 4

* More

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-30 22:58:00 +01:00
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018)
* turbo-geth to erigon

* tg, turbo to erigon
2021-05-26 11:35:39 +01:00
ledgerwatch
7d98b6504a
Improvements in the mock sentry tests (#1997)
* Improvements in the mock sentry tests

* GenerateChain to return chainpack

* Fix consensus

* Fix core

* Fix core/state

* Fix eth

* Fix eth/download

* Fix eth/fetcher

* Fix eth/filters

* Fix eth/gasprice

* Fix eth/protocols/eth

* Fix tests

* Fix turbo/stages

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-23 17:10:34 +01:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
Alex Sharov
50934fa5d3
Move chain maiker to kv (#1972) 2021-05-20 18:49:33 +07:00
Alex Sharov
781a67a4ee
Auto-Close db in tests by using t.Cleanup (#1956) 2021-05-19 10:47:28 +07:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
ledgerwatch
12cde41772
Aleut support (Eip1559) (#1704)
* Where I am at

* Refactoring of transaction types

* More refactoring

* Use Homested signer in rpc daemon

* Unified signer

* Continue unified signer

* A bit more

* Fixes and down the rabbit hole...

* More tx pool fixes

* More refactoring fixes

* More fixes'

* more fixes

* More fixes

* More compile fixes

* More RLP hand-writing

* Finish RLP encoding/decoding of transactions

* Fixes to header encoding, start on protocol packets

* Transaction decoding

* Use DecodeTransaction function

* Decoding BlockBodyPacket

* Encode and decode for pool txs

* Start fixing tests

* Introduce SigningHash

* Fixes to SignHash

* RLP encoding fixes

* Fixes for encoding/decoding

* More test fixes

* Fix more tests

* More test fixes

* More test fixes

* Fix core tests

* More fixes for signer

* Fix for tx

* Fixes to string encoding/size

* Fix eip2930 test

* Fix rest of ./tests

* More fixes

* Fix compilation

* More test fixes

* More test fixes

* Test fixes

* More fixes

* Reuse EncodingSize in EncodeRLP for accessList

* Rearrange things in dynamic fee tx

* Add MarshalBinary

* More fixes

* Make V,R,S non-pointers

* More NPE fixes

* More fixes

* Receipt fixes

* Fix core/types

* Fix ./eth

* More compile fixes for tests

* More test fixes

* More test fixes

* Try to see lint errors better

* Try to see lint errors better

* Fix lint

* Debugging eip1559 test

* Fix TestEIP1559Transition test

* Fix NewBlockPacket encoding/decoding

* Fix calculation of TxHash

* Fix perf problem with senders

* Update aleut config values

* Try adding static peers

* Add staticpeers to defaul flags

* Change aleut networkID

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB (#1703)
* debug

* debug

* it works

* rename clique bucket

* remove genesis special case

* copy snapshot

* remove debug

* migration

* debug

* regenerate snapshots

* simplify

* regeneration

* after merge

* tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
b4a2d823c4
old loader remove v1 (#1641) 2021-03-31 14:43:12 +07:00
Artem Vorotnikov
9b8cdc0f22
Fix lints and remove more unused code (#1621) 2021-03-29 10:58:45 +07:00
alex.sharov
e7034c0eba revert 2021-03-21 17:50:05 +07:00
alex.sharov
970c7e93a8 remove old flags 2021-03-21 17:05:42 +07:00
Alex Sharov
97eec4288a
Tests: use transaction instead of db.MemCopy in chain_makers.go (#1570)
* mdbx tests speedup

* mdbx tests speedup

* devel up

* devel up

* devel up

* lint

* less db copy

* don't loose error in test

* clean
2021-03-20 19:33:40 +00:00
ledgerwatch
b743b88218
Revert "Extract validating interface (#1120)" (#1543)
* Revert "Extract validating interface (#1120)"

This reverts commit e4e36c152e.

* Fix go mod tidy

* Fix lint

* Fix lint

* fix lint

* Test fix

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-05 20:34:23 +00:00
Evgeny Danilenko
e4e36c152e
Extract validating interface (#1120)
* interface

* generalize interface

* linters

* fix deadlock

* fix linters

* close goroutine

* fix

* debug

* id+ttl

* refactor downloader tests

* tests

* lru

* handle genesis, extract fake consensuses

* fix fake consensus

* test uncles, verify

* after a new master

* fmt

* fix close

* debug

* debug

* fix chain length

* remove test field

* use single account

* fix data race on closing channel

* remove postponed blocks queue

* miner test

* VerifyHeaderRequests

* fmt

* fmt

* fix data race

* handle validating errors

* simplify matchParents

* remove copy-paste

* move sort to constructor

* clean up

* debug for 10 parents

* debug

* debug

* batch responses

* batch requests

* works for many ancestors

* remove debug

* always Close an engine

* linters

* ancestors deduplication

* fix test

* reduce interface

* api

* clique

* green clique sync

* stable

* perpermance per second

* full sync

* linters

* gitignore

* deps

* fix panic after master merge

* init consensus

* clique tests

* fix tests

* fix tests

* clean up

* reuse snap

* store vefified snapshots

* optimize snapshots

* safe close

* cleanup loop

* after downloader

* downloader and consensus tests

* update tests

* hack.go

* clique flags

* fix cliuqe config

* review

* gitignore

* remove additional bucket

* blk/sec instead of blk/microsecond

* rename

* deps

* optimize

* debug

* test

* tests without extracted validation process

* same base performance as on master

* benchmark

* simplify ethash verification

* ethash

* ethash

* linters

* ethash

* master stats

* cleanup

* gomod

* linters

* tests

* better locks

* Fix

* Remove logging for verifyHeaders

* Verification speed in the logs

* Fix compile error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-25 19:40:45 +00:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record (#1500)
* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE
2021-02-21 18:41:59 +00:00
Artem Vorotnikov
91f7798c4a
Remove hashed changesets (#1464)
* Remove hashed changesets

* try this

* oops

* simplify

* better diagnostics

* don't hash account key and use bitmap 64 to check history index

* update docs of bucket

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2021-02-04 12:59:04 +03:00
Alex Sharov
58c4e03861
Revert "Remove hashed changesets (#1444)" (#1446)
This reverts commit aadb6e47cc.
2021-01-20 14:16:20 +07:00
Artem Vorotnikov
aadb6e47cc
Remove hashed changesets (#1444) 2021-01-19 17:42:26 +07:00
ledgerwatch
3287b44c2e
Bad unwind of recreation of CREATE2 contract - unit test and potentia… (#1412)
* Bad unwind of recreation of CREATE2 contract - unit test and potential fix

* Fix unit test

* Fix lint

* Fix lint

* Fixup hack

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-16 17:16:37 +00:00
ledgerwatch
5ea590c18e
State cache switching writes to reads during commit (#1368)
* State cache init

* More code

* Fix lint

* More tests

* More tests

* More tests

* Fix test

* Transformations

* remove writeQueue, before fixing the tests

* Fix tests

* Add more tests, incarnation to the code items

* Fix lint

* Fix lint

* Remove shards prototype, add incarnation to the state reader code

* Clean up and replace cache in call_traces stage

* fix flaky test

* Save changes

* Readers to use addrHash, writes - addresses

* Fix lint

* Fix lint

* More accurate tracking of size

* Optimise for smaller write batches

* Attempt to integrate state cache into Execution stage

* cacheSize to default flags

* Print correct cache sizes and batch sizes

* cacheSize in the integration

* Fix tests

* Fix lint

* Remove print

* Fix exec stage

* Fix test

* Refresh sequence on write

* No double increment

* heap.Remove

* Try to fix alignment

* Refactoring, adding hashItems

* More changes

* Fix compile errors

* Fix lint

* Wrapping cached reader

* Wrap writer into cached writer

* Turn state cache off by default

* Fix plain state writer

* Fix for code/storage mixup

* Fix tests

* Fix clique test

* Better fix for the tests

* Add test and fix some more

* Fix compile error|

* More functions

* Fixes

* Fix for the tests

* sepatate DeletedFlag and AbsentFlag

* Minor fixes

* Test refactoring

* More changes

* Fix some tests

* More test fixes

* More test fixes

* Fix lint

* Move blockchain_test to be able to use stagedsync

* More fixes

* Fixes and cleanup

* Fix tests in turbo/stages

* Fix lint

* Fix lint

* Intemediate

* Fix tests

* Intemediate

* More fixes

* Compilation fixes

* More fixes

* Fix compile errors

* More test fixes

* More fixes

* More test fixes

* Fix compile error

* Fixes

* Fix

* Fix

* More fixes

* Fixes

* More fixes and cleanup

* Further fix

* Check gas used and bloom with header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-08 09:44:29 +00:00
Alex Sharov
9712e858e2
Method delete to accept second param: allow delete dupsort values (#1297)
* aloow_delete_dupsort_values

* check canonical receipt

* exclusive lock for mdbx

* readme_load_dump_tables

* readme_load_dump_tables

* readme_load_dump_tables

* Remove IdealBatchSize change

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-29 13:19:31 +00:00
Igor Mandrigin
0f1b2f3595 geth-1.9.23: post-rebase fixups 2020-10-26 17:16:00 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Alex Sharov
ecc94a63f0
DupSort for Plain and Hashed state buckets (behind feature flags) (#854)
* return error from rawdb

* squash

* v14

* improve performance of put

* clean

* clean

* hide feature behind ENV variable

* cleanup

* cleanup

* disable ipc and make Readme less confusing (people thought points are depend on each-other)

* fix test

* cleanup

* cleanup
2020-08-02 12:53:08 +01:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation (#851)
* Jumpdest skipping optimisation

* Fix formatting

* Move skipAnalysis into vmConfig, introduce tracing ability

* Improve detection logging

* Added release instructions

* Fix lint
2020-08-01 17:56:57 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly (#748)
* close goruotines

* fix using defer

* close pm managers a bit later

* fmt

* finalizer for in-mem db

* after cr

* revert finalizer

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
665b1d88b9
Speed up GenerateChain by using intermediate hashes (#736)
* GenerateChain to support intermediate hashes

* Fix formatting

* Changeset stats

* Fix formatting
2020-07-10 22:37:34 +01:00
ledgerwatch
887fc4f99c
GenerateChain without using TrieDbState (#726)
* Different chain generation

* More fixes

* Fix TestSideLogRebirth

* Fix the rest of core tests

* Fix accounts/abi/bind

* Fix compile errors for cmd/pics

* Fix consensus/clique tests

* More fixes

* Fix core/state tests

* Fix eth tests

* Fix eth/downloader tests

* Fix more tests

* Fix compile errors

* Some more fixes

* Save

* Exists

* Fix last tests

* Clean up

* Fix formatting

* Fix linter

* Fix formatting

* Fix linter

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint
2020-07-09 07:15:28 +01:00
Alex Sharov
8d04689041
close db in tests (#669) 2020-06-16 12:05:40 +07:00
ledgerwatch
dec30b698d
Fix execution for block 10094566 (#665)
* First sketch of the fix

* Re-enable stage4

* Fix formatting

* Park the changes

* Save updates

* Save update

* Cleanup

* Cleanup

* Cleanup

* Fix lint

* Cleanup

* Update

* Tx tracing

* Update

* Cleanup

* Switch tests back to bolt

* Test cleanup

* Update

* Polymorthic CREATE2 test

* Add assertions

* Fix formatting

* Fix test

* Cleanup

* Cleanup

* Cleanup
2020-06-15 22:24:08 +01:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap (#626)
* optimize codeBitmap

* add lru

* remove custom lru

* jumpDests as explocit parameter

* lint

* lint

* linters

* default jumpDestsCache

* remove common.rand
2020-06-06 21:49:06 +01:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger (#624)
* lmdb tests

* trigger ci

* fix tests

* disable parallelism

* disable parallelism

* cleanup resources

* cleanup resources

* reduce concurency

* try run tests on bolt

* try run tests on bolt

* fix downloader test

* run bolt tests

* rely on interface instead of exact instance

* Rename AbstractKV to KV

* don't use separator for badger

* don't initialize badger cursor - because it not used here

* fix linter

* try reduce badger compactors

* compat with master

* try lmdb

* try lmdb

* try lmdb

* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option

* allow to close closed db

* release

* release

* ideal batch size for badger

* ideal batch size for badger
2020-06-05 10:25:33 +01:00
Andrew Ashikhmin
33c28f7fac
[GC] uint256 rather than big.Int in Transaction (#614)
* uint256 in rlp

* uint256 rather than big.Int in Transation

* linters

* more linters

* still linters

* Reduce garbage in writeUint256

* Experiment with GC in writeByteArray
2020-06-04 08:43:08 +01:00
ledgerwatch
4ce69916dc
Fix rebuilding the Intermediate Hash bucket (#602)
* Changes

* First attempt to fix

* Sync works

* Fix compile

* Fix linter

* Fix test

* Fix test

* Debug

* Propagate tracing

* Trace default receiver

* More tracing

* Correct unfurl

* Remove tracing

* Try to retain trie for debugging
2020-06-02 08:46:21 +01:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb (#584)
* Not hash, keep the files

* Calculate savings

* Fix

* Fix

* Fix

* Fix

* RestAPI to support local boltdb

* Not error on read-only db

* Changes so far

* Continue

* More

* Roll back a bit

* Restore newline

* something compiles

* Fix restapi

* Fix block number

* Fix reads

* Use plain writer

* Maps for storage reads and writes

* Clean up coersions

* Fix accounts/abi/bind

* Fix tests

* More fixes

* more fixes

* More fixes

* Fixes

* Fixed core/state

* Fixed eth tests

* Move code, fix linter

* Fix test

* Fix linter

* Fix linter

* Fix linter, badger_db to support AbstractKV

* Increase IdealBatchSize for badger

* Fix linter

* Fix linter
2020-05-27 17:24:34 +01:00
Andrew Ashikhmin
5bbdb6a4b9
Use uint256 for account balance (#580)
* Use uint256 for account balance

* linter
2020-05-26 17:53:50 +01:00
Andrew Ashikhmin
b16e560529
Use uint256.Int rather than common.Hash for storage values to reduce memory allocation in opSload & opSstore (#575)
* Produce less garbage in GetState

* Still playing with mem allocation in GetCommittedState

* Pass key by pointer in GetState as well

* linter

* Avoid a memory allocation in opSload

* Use uint256.Int rather than common.Hash for storage values to reduce memory allocation in opSload & opSstore

* linter

* linters

* small clean up
2020-05-25 12:12:25 +01:00
Andrew Ashikhmin
df28575420
Use pointers to hashes in Get(Commited)State to reduce memory allocation (#573)
* Produce less garbage in GetState

* Still playing with mem allocation in GetCommittedState

* Pass key by pointer in GetState as well

* linter

* Avoid a memory allocation in opSload
2020-05-24 18:43:54 +02:00
Igor Mandrigin
0dae3ade0d post-rebase fixups 2020-05-20 15:26:22 +03:00