Alex Sharov
50934fa5d3
Move chain maiker to kv ( #1972 )
2021-05-20 18:49:33 +07:00
Alex Sharov
781a67a4ee
Auto-Close db in tests by using t.Cleanup ( #1956 )
2021-05-19 10:47:28 +07:00
Artem Vorotnikov
66a8c23c37
Remove TrieDbState + other things ( #1925 )
...
* Remove old table from script
* Simplify ChangeSetWriter
* Move state tests to KV
* Simplify TrieCfg
* Port state tests away from TrieDbState
* Remove old MakePreState
* Upgrade t8ntool's MakePreState
* MakePreState2 -> MakePreState
* Remove TrieDbState
2021-05-16 06:21:29 +01:00
b00ris
565a4250d6
Snapshot sync headers stage ( #1836 )
...
* save state
* snapshot update works
* save state
* snapshot migrator
* tx test
* save state
* migrations stages refactor
* refactor snapshot migrator
* compilation fixed
* integrate snapshot migrator
* goerli sync headers
* debug async snapshotter on goerly
* move verify headers, remove experiments, fix remove old snapshot
* save state
* refactor snapshotsync injection
* fix deadlock
* replace snapshot generation stage logic to migrate method
* change done for body snapshot
* clean
* clean&&change deleted value
* clean
* fix hash len
* fix hash len
* remove one of wrap methods, add remove snapshots on start
* add err check
* fix shadowing
* stages unwind order debug
* matryoshka experiments
* steam test
* fix build
* fix test
* fix lint
* fix test
* fix test datarace
* add get test
* return timeout
* fix mdbx overlap
* fix after merge
* change epoch size
* clean todo
* fix
* return testdata
* added return from sndownloader gorutine
* fix review comments
* Fix
* More info
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync ( #1899 )
2021-05-08 00:07:49 +03:00
Alex Sharov
e8255abdd0
deprecate tds and trie ( #1890 )
2021-05-06 15:48:30 +07:00
Alex Sharov
e87d4932f7
Changeset writer move to tx ( #1871 )
2021-05-05 14:31:40 +07:00
Artem Vorotnikov
2148885637
Simplify ChangeSet implementation and tests ( #1878 )
...
* Move storage changeset code into one file
* Simplify storage changeset tests
* simplify more
* more
* more
* more
* move account changeset into one file
* simplify account changeset test
* NewStorageChangesetPlain -> NewStorageChangeSet
* EncodeStoragePlain -> EncodeStorage
* StorageChangeSetPlain -> StorageChangeSet
* AccountChangeSetPlain -> AccountChangeSet
* more
* more
* more
* Simplify EncodeStorage
* more
* Simplify FromDBFormat
* more rename
* rename changeset buckets (constant names only)
2021-05-04 13:34:08 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface ( #1854 )
2021-05-01 14:42:23 +07:00
Artem Vorotnikov
57c88868a3
GetAsOf: do not return error on nil ( #1820 )
2021-04-26 18:05:29 +01:00
Alex Sharov
c40c3b7a7a
exec unwind etl ( #1806 )
2021-04-26 10:41:31 +01:00
Artem Vorotnikov
63835b6a30
Remove locking from IntraBlockState ( #1795 )
2021-04-26 10:40:25 +01:00
Alex Sharov
eee9438625
deprecate PlainDbState in favor of PlainKvState ( #1805 )
2021-04-26 12:37:48 +07:00
ledgerwatch
12cde41772
Aleut support (Eip1559) ( #1704 )
...
* Where I am at
* Refactoring of transaction types
* More refactoring
* Use Homested signer in rpc daemon
* Unified signer
* Continue unified signer
* A bit more
* Fixes and down the rabbit hole...
* More tx pool fixes
* More refactoring fixes
* More fixes'
* more fixes
* More fixes
* More compile fixes
* More RLP hand-writing
* Finish RLP encoding/decoding of transactions
* Fixes to header encoding, start on protocol packets
* Transaction decoding
* Use DecodeTransaction function
* Decoding BlockBodyPacket
* Encode and decode for pool txs
* Start fixing tests
* Introduce SigningHash
* Fixes to SignHash
* RLP encoding fixes
* Fixes for encoding/decoding
* More test fixes
* Fix more tests
* More test fixes
* More test fixes
* Fix core tests
* More fixes for signer
* Fix for tx
* Fixes to string encoding/size
* Fix eip2930 test
* Fix rest of ./tests
* More fixes
* Fix compilation
* More test fixes
* More test fixes
* Test fixes
* More fixes
* Reuse EncodingSize in EncodeRLP for accessList
* Rearrange things in dynamic fee tx
* Add MarshalBinary
* More fixes
* Make V,R,S non-pointers
* More NPE fixes
* More fixes
* Receipt fixes
* Fix core/types
* Fix ./eth
* More compile fixes for tests
* More test fixes
* More test fixes
* Try to see lint errors better
* Try to see lint errors better
* Fix lint
* Debugging eip1559 test
* Fix TestEIP1559Transition test
* Fix NewBlockPacket encoding/decoding
* Fix calculation of TxHash
* Fix perf problem with senders
* Update aleut config values
* Try adding static peers
* Add staticpeers to defaul flags
* Change aleut networkID
* Fix test
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
Alex Sharov
0af3085461
Revert "GetAsOf: do not return error on nil ( #1769 )" ( #1782 )
...
This reverts commit 60196abf50
2021-04-22 13:45:04 +01:00
Alex Sharov
e7a78a2d9f
Mdbx: speedup tests ( #1776 )
2021-04-21 20:03:08 +07:00
Artem Vorotnikov
60196abf50
GetAsOf: do not return error on nil ( #1769 )
2021-04-21 10:47:46 +07:00
Artem Vorotnikov
c967655c1d
More cleanups ( #1765 )
2021-04-21 08:48:37 +07:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB ( #1703 )
...
* debug
* debug
* it works
* rename clique bucket
* remove genesis special case
* copy snapshot
* remove debug
* migration
* debug
* regenerate snapshots
* simplify
* regeneration
* after merge
* tests
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
e473e0b4d9
lint fixes ( #1717 )
2021-04-13 21:48:20 +07:00
Alex Sharov
7222cdd640
delete tds.unwindTo method ( #1688 )
2021-04-08 10:47:10 +07:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it ( #1676 )
2021-04-05 20:04:58 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs ( #1669 )
2021-04-03 09:26:00 +03:00
Alex Sharov
9a9c5b298a
bitmapdb to use rwtx ( #1661 )
2021-04-03 08:52:45 +07:00
Alex Sharov
fbfc43da7e
etl to use rwtx ( #1660 )
2021-04-02 18:22:25 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) ( #1656 )
2021-04-02 13:36:49 +07:00
Alex Sharov
fbd9690251
Remove tds in few tests ( #1657 )
2021-04-02 12:14:43 +07:00
Alex Sharov
abe5bd879c
Remove old loader, step 3 (remove some tds and blockchain objects in tests) ( #1647 )
2021-04-01 12:36:22 +07:00
Alex Sharov
b4a2d823c4
old loader remove v1 ( #1641 )
2021-03-31 14:43:12 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface ( #1627 )
2021-03-30 12:53:54 +03:00
Artem Vorotnikov
9b8cdc0f22
Fix lints and remove more unused code ( #1621 )
2021-03-29 10:58:45 +07:00
Artem Vorotnikov
e67be2b1be
Remove unused code ( #1604 )
...
* Remove graphql, legacy options
* Remove CODEOWNERS
* Remove EWASM and custom EVM interpreter
2021-03-26 09:08:01 +07:00
Alex Sharov
b31813dcb5
Mining - don't write changesets to batch ( #1595 )
2021-03-24 12:09:29 +07:00
Artem Vorotnikov
a6742a4cf8
Kill cmd/state/stateless ( #1592 )
...
* Move stuff around
* Remove
* tidy
* Fix for writeReceipt printing
* fix lint typo
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-23 16:33:58 +00:00
Alex Sharov
91e4126ca7
db commit - remove unused return param ( #1584 )
2021-03-22 19:41:52 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions ( #1577 )
2021-03-21 20:15:25 +07:00
alex.sharov
e7034c0eba
revert
2021-03-21 17:50:05 +07:00
alex.sharov
970c7e93a8
remove old flags
2021-03-21 17:05:42 +07:00
Alex Sharov
97eec4288a
Tests: use transaction instead of db.MemCopy in chain_makers.go ( #1570 )
...
* mdbx tests speedup
* mdbx tests speedup
* devel up
* devel up
* devel up
* lint
* less db copy
* don't loose error in test
* clean
2021-03-20 19:33:40 +00:00
Artem Vorotnikov
0b2e4df0ff
Align Cursor API with LMDB/MDBX ( #1558 )
2021-03-19 14:45:01 +07:00
Alexey Sharp
519b19f8aa
Compilation fixes
2021-03-14 08:00:37 +00:00
Martin Holst Swende
d85afb7d20
core/state: fix eta calculation on pruning ( #22386 )
2021-03-12 15:52:25 +01:00
Martin Holst Swende
1f9850da82
core/state/pruner: fix compaction range error
2021-03-10 10:58:50 +01:00
Péter Szilágyi
97fc3ba0d2
core/state/pruner: fix compaction after pruning
2021-03-10 10:58:39 +01:00
gary rong
a9e6250e72
all: bloom-filter based pruning mechanism ( #21724 )
...
* cmd, core, tests: initial state pruner
core: fix db inspector
cmd/geth: add verify-state
cmd/geth: add verification tool
core/rawdb: implement flatdb
cmd, core: fix rebase
core/state: use new contract code layout
core/state/pruner: avoid deleting genesis state
cmd/geth: add helper function
core, cmd: fix extract genesis
core: minor fixes
contracts: remove useless
core/state/snapshot: plugin stacktrie
core: polish
core/state/snapshot: iterate storage concurrently
core/state/snapshot: fix iteration
core: add comments
core/state/snapshot: polish code
core/state: polish
core/state/snapshot: rebase
core/rawdb: add comments
core/rawdb: fix tests
core/rawdb: improve tests
core/state/snapshot: fix concurrent iteration
core/state: run pruning during the recovery
core, trie: implement martin's idea
core, eth: delete flatdb and polish pruner
trie: fix import
core/state/pruner: add log
core/state/pruner: fix issues
core/state/pruner: don't read back
core/state/pruner: fix contract code write
core/state/pruner: check root node presence
cmd, core: polish log
core/state: use HEAD-127 as the target
core/state/snapshot: improve tests
cmd/geth: fix verification tool
cmd/geth: use HEAD as the verification default target
all: replace the bloomfilter with martin's fork
cmd, core: polish code
core, cmd: forcibly delete state root
core/state/pruner: add hash64
core/state/pruner: fix blacklist
core/state: remove blacklist
cmd, core: delete trie clean cache before pruning
cmd, core: fix lint
cmd, core: fix rebase
core/state: fix the special case for clique networks
core/state/snapshot: remove useless code
core/state/pruner: capping the snapshot after pruning
cmd, core, eth: fixes
core/rawdb: update db inspector
cmd/geth: polish code
core/state/pruner: fsync bloom filter
cmd, core: print warning log
core/state/pruner: adjust the parameters for bloom filter
cmd, core: create the bloom filter by size
core: polish
core/state/pruner: sanitize invalid bloomfilter size
cmd: address comments
cmd/geth: address comments
cmd/geth: address comment
core/state/pruner: address comments
core/state/pruner: rename homedir to datadir
cmd, core: address comments
core/state/pruner: address comment
core/state: address comments
core, cmd, tests: address comments
core: address comments
core/state/pruner: release the iterator after each commit
core/state/pruner: improve pruner
cmd, core: adjust bloom paramters
core/state/pruner: fix lint
core/state/pruner: fix tests
core: fix rebase
core/state/pruner: remove atomic rename
core/state/pruner: address comments
all: run go mod tidy
core/state/pruner: avoid false-positive for the middle state roots
core/state/pruner: add checks for middle roots
cmd/geth: replace crit with error
* core/state/pruner: fix lint
* core: drop legacy bloom filter
* core/state/snapshot: improve pruner
* core/state/snapshot: polish concurrent logs to report ETA vs. hashes
* core/state/pruner: add progress report for pruning and compaction too
* core: fix snapshot test API
* core/state: fix some pruning logs
* core/state/pruner: support recovering from bloom flush fail
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
# cmd/geth/main.go
# cmd/geth/usage.go
# core/blockchain.go
# core/blockchain_snapshot_test.go
# core/genesis.go
# core/rawdb/database.go
# core/rawdb/schema.go
# core/state/snapshot/conversion.go
# core/state/snapshot/snapshot.go
# core/state/snapshot/snapshot_test.go
# eth/backend.go
# go.mod
# go.sum
# tests/block_test_util.go
# tests/state_test_util.go
# trie/stacktrie.go
2021-03-10 10:58:22 +01:00
Péter Szilágyi
40a366086e
core/state: convert prefetcher to concurrent per-trie loader
...
# Conflicts:
# accounts/abi/bind/backends/simulated.go
# core/blockchain.go
# core/state/state_object.go
# core/state/state_test.go
# core/state/statedb.go
# core/state/statedb_test.go
# eth/api_tracer.go
# miner/worker.go
2021-03-09 12:30:09 +01:00
Martin Holst Swende
6c3d78d0d9
core: implement background trie prefetcher
...
Squashed from the following commits:
core/state: lazily init snapshot storage map
core/state: fix flawed meter on storage reads
core/state: make statedb/stateobjects reuse a hasher
core/blockchain, core/state: implement new trie prefetcher
core: make trie prefetcher deliver tries to statedb
core/state: refactor trie_prefetcher, export storage tries
blockchain: re-enable the next-block-prefetcher
state: remove panics in trie prefetcher
core/state/trie_prefetcher: address some review concerns
sq
# Conflicts:
# core/blockchain.go
# core/state/database.go
# core/state/state_object.go
# core/state/statedb.go
2021-03-09 11:15:19 +01:00
ledgerwatch
b743b88218
Revert "Extract validating interface ( #1120 )" ( #1543 )
...
* Revert "Extract validating interface (#1120 )"
This reverts commit e4e36c152e
2021-03-05 20:34:23 +00:00
Evgeny Danilenko
e4e36c152e
Extract validating interface ( #1120 )
...
* interface
* generalize interface
* linters
* fix deadlock
* fix linters
* close goroutine
* fix
* debug
* id+ttl
* refactor downloader tests
* tests
* lru
* handle genesis, extract fake consensuses
* fix fake consensus
* test uncles, verify
* after a new master
* fmt
* fix close
* debug
* debug
* fix chain length
* remove test field
* use single account
* fix data race on closing channel
* remove postponed blocks queue
* miner test
* VerifyHeaderRequests
* fmt
* fmt
* fix data race
* handle validating errors
* simplify matchParents
* remove copy-paste
* move sort to constructor
* clean up
* debug for 10 parents
* debug
* debug
* batch responses
* batch requests
* works for many ancestors
* remove debug
* always Close an engine
* linters
* ancestors deduplication
* fix test
* reduce interface
* api
* clique
* green clique sync
* stable
* perpermance per second
* full sync
* linters
* gitignore
* deps
* fix panic after master merge
* init consensus
* clique tests
* fix tests
* fix tests
* clean up
* reuse snap
* store vefified snapshots
* optimize snapshots
* safe close
* cleanup loop
* after downloader
* downloader and consensus tests
* update tests
* hack.go
* clique flags
* fix cliuqe config
* review
* gitignore
* remove additional bucket
* blk/sec instead of blk/microsecond
* rename
* deps
* optimize
* debug
* test
* tests without extracted validation process
* same base performance as on master
* benchmark
* simplify ethash verification
* ethash
* ethash
* linters
* ethash
* master stats
* cleanup
* gomod
* linters
* tests
* better locks
* Fix
* Remove logging for verifyHeaders
* Verification speed in the logs
* Fix compile error
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-25 19:40:45 +00:00
Alex Sharov
d4ef1f8516
blocks exec unwind - to support graceful shutdown ( #1494 )
...
* quit exec unwind
* quit exec unwind
* lint
2021-02-23 17:14:32 +00:00
