Marius van der Wijden
4b8f938436
eth: updated comments ( #21490 )
...
# Conflicts:
# eth/api.go
2020-09-11 16:04:25 +02:00
ledgerwatch
ab4c456fec
Headers poc 3 - intermediate changes ( #1093 )
...
* Use log instead of Printf
* Better output
* Display NewBlockHeadersMsg
* Print tx hashes
* Separate sentry and downloader
* Fix lint
* Requesting headers
* Fix lint
* Prevent deadlock, fix NPE
* Not quit when did not restore
* Fix types
* Fix types again
* Warn on penalty
* Prints
* Avoid deadlock
* Better request logic
* Print height
* Actually send header requests
* Actually send header request
* Fix compile errors, sending
* Print BlockHeadersMsg
* Fix lint
2020-09-11 07:35:51 +01:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support ( #1080 )
...
* etl.Loader - allow use of custom comparator
* log timing
* try now
* try now
* more performance
* etl.Loader - allow use of custom comparator
* working version
* simplify IH cursor
* clean
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* clean
* add only unwind support
* squash
* squash
* clean
* fix test
* clean
* clean
* clean
2020-09-10 13:35:58 +01:00
Alex Sharov
da31494cc3
transactional migrations ( #1079 )
2020-09-08 20:39:43 +01:00
Alex Sharov
c45a710ce6
Warmup logs, less overhead, warmup code bucket ( #1054 )
...
* warmup logs and less overhead.
* warmup logs and less overhead.
* move WarmUp to common func
2020-09-08 08:28:37 +01:00
Alex Sharov
9d29a4b480
log_all_stages_timings ( #1078 )
...
* log_all_stages_timings
* clean
2020-09-08 07:19:25 +01:00
Alex Sharov
50d13fd9a8
increase tx limit ( #1047 )
2020-09-08 07:09:33 +01:00
ledgerwatch
256e4262e1
2nd batch of Headers PoC changes ( #1067 )
...
* PoC utility
* Headers cmd
* headers download command
* Fix lint
* Send handshake
* Receive Status message
* Better error messages
* Messages
* Fix
* Fix lint
* Print block number for NewBlockMsg
* Fix lint
2020-09-07 07:03:12 +01:00
ledgerwatch
fe6d4b5213
Better log message about commit ( #1060 )
2020-09-06 14:10:33 +01:00
ledgerwatch
787bba4f48
Print pending tx stats, gather data for gas price chart ( #1058 )
...
* Print tx pool stat
* Print tx pool stats
* Chart gas prices
* Print gaps
* Don't check canonical
* Hack
* Remove hack
* Fix stages
* fixState
* Canonical gaps
* Gas burnt vs avg gas price
* Revert changes in tx pool
* Fix NPE
* Remove fixStages
* Fix NPE
2020-09-05 18:42:37 +01:00
Igor Mandrigin
2a6478d799
turbo-api: create a second executable file with a custom stage example ( #1055 )
2020-09-05 18:07:27 +02:00
Alex Sharov
b9bdd003a7
KV: Raw (no custom logic) cursors for DupSort and DupFixed buckets ( #1020 )
2020-09-04 10:54:15 +07:00
ledgerwatch
f4b34102eb
Properly pass datadir into stage2 ( #1044 )
2020-09-03 23:15:24 +01:00
Evgeny Danilenko
e4f495fa44
Get logs ( #1028 )
...
* it compiles
* after recent master
* fix linters warnings
* grpcV7
* go mod tidy
* unmarshall adresses or adress
* fix linters
* after cr
* after cr
* after cr
* after cr
* fix tests
* remove dev version
* it compiles
* mod tidy
* fix bin deps
* use stable version of grpc
* switch back to master constructor
* switch back to master constructor
* add a bit docs
* add a bit docs
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-09-03 08:51:19 +01:00
Igor Mandrigin
5b92d47e9d
Initialize Staged Sync with stages. ( #1033 )
...
* externalize default stages
* initialize with the right stages
* fix some bugs
* fix
* linters
2020-09-02 19:11:41 +01:00
ledgerwatch
c6975414e8
[WIP] Warmup the state bucket for HDD sync ( #1032 )
...
* Warmup
* Log the warmup
* Make warmup interruptible
* Make warmup a command line option for HDD
* Activate hdd flag
2020-09-02 12:24:27 +01:00
ledgerwatch
d8c0c16e40
Bad rewind issues fix ( #1024 )
...
* Unwind over unwound, no rollbacks
* Exclude bits only for StagedSync
* Nitpick
2020-09-01 12:50:36 +01:00
ledgerwatch
c06dbdadf9
Use Append for changesets and receipts ( #1018 )
...
* Use Append for changesets and receipts
* Fix lint
* Delete all receipts on unwind
* Fix lint
* Fix unwind keys for changeset
* Fix test
* Use writeReceipts flag
* Revert to Append
* Delete receipts too when resetting
2020-09-01 07:48:25 +01:00
ledgerwatch
1a0e9275e1
Headers PoC - intermediate changes ( #1016 )
...
* First algorithm
* Update doc
* Second algorithm implemented
* Prepend (unfinished)
* More on prepend
* More fixes
* Change CalcDifficulty interface
* fix make test
* Fix formatting
* Another test
* More on Preprend
* Another test for Prepend
* More tests for Prepend
* More tests for Prepend
* Move files to turbo
* Add another test for Prepend
* Started adding Append algorithm
* More work on Append
* Start changing the terminology
* Split trees into segments
* More fixes
* Fix compilation
* Connect
* FindAnchors, FindTip
* Tip has anchor and not anchorParent field
* Remove Prepend and Append
* Add Connect
* NewAnchor algorithm
* More to recovery from files
* Add more to AddAnchor
* Request more headers
* Header fix-sizes serialisation
* Upgrade CI to 1.15
* Fix lint
* Fix lint
* replace RLP with fixed-sized serialisation
* Fix lint
* More algos
* Fix lint
2020-08-31 06:27:51 +01:00
Igor Mandrigin
7bc892e886
Prefetch block bodies ( #1013 )
...
* prefetched blocks 1
* fixups
* fix logs
* fixups
* linters
* remove logs
* fix an NPE in tests
* setup the correct origin
* run blocks on prefetched bodies
* fix log
* smaller cache size
2020-08-30 18:34:40 +01:00
Igor Mandrigin
a1a7f6607a
rpcdaemon, stagedsync: make sure we update the current block number after a sync cycle ( #1015 )
...
* rpcdaemon, stagedsync: make sure we update the current block number after a sync cycle
fixes #1014
* linters
2020-08-30 17:51:06 +01:00
Igor Mandrigin
5365b8e402
stagedsync: create a StagedSync object, initialize in ProtocolManager ( #1012 )
...
* stagedsync: create a StagedSync object, initialize in ProtocolManager
* fix compile error
2020-08-30 13:24:58 +02:00
ledgerwatch
22dc832d92
RPC method fixes ( #1007 )
...
* Fix some RPC issues
* Fudge the getBlockByNumber
* Add two more PRC methods
* Fix rpc tests
2020-08-29 22:28:09 +01:00
Igor Mandrigin
15e86b4b34
fix compilation issue
2020-08-29 15:30:06 +02:00
Igor Mandrigin
637493e8f1
Merge branch 'master' into geth-1.9.20
2020-08-29 15:28:03 +02:00
Igor Mandrigin
bd61245ece
test fixups
2020-08-29 14:35:39 +02:00
Igor Mandrigin
4b4326020c
geth compiles
2020-08-29 14:03:41 +02:00
Igor Mandrigin
948cd29be7
fixes
2020-08-29 13:43:44 +02:00
Igor Mandrigin
78e466d06a
fix imports
2020-08-29 13:31:46 +02:00
Martin Holst Swende
fa3ee4bf6b
eth: utilize sync bloom for getNodeData ( #21445 )
...
* eth/downloader, eth/handler: utilize sync bloom for getNodeData
* trie: handle if bloom is nil
* trie, downloader: check bloom nilness externally
# Conflicts:
# eth/handler.go
2020-08-29 13:26:15 +02:00
gary rong
550bd11f9d
core, eth, les, trie: add a prefix to contract code ( #21080 )
...
# Conflicts:
# cmd/evm/internal/t8ntool/execution.go
# consensus/clique/clique.go
# consensus/ethash/consensus.go
# core/block_validator.go
# core/blockchain.go
# core/blockchain_test.go
# core/genesis.go
# core/rawdb/accessors_indexes_test.go
# core/rawdb/accessors_metadata.go
# core/rawdb/chain_iterator_test.go
# core/rawdb/database.go
# core/rawdb/schema.go
# core/state/database.go
# core/state/iterator_test.go
# core/state/statedb.go
# core/state/sync.go
# core/state/sync_test.go
# core/tx_pool_test.go
# core/types/block_test.go
# core/types/derive_sha.go
# eth/downloader/downloader.go
# eth/downloader/queue.go
# eth/downloader/statesync.go
# eth/fetcher/block_fetcher.go
# eth/fetcher/block_fetcher_test.go
# eth/handler.go
# les/odr_requests.go
# les/server_handler.go
# light/odr.go
# light/odr_test.go
# light/trie.go
# miner/worker.go
# trie/database.go
# trie/secure_trie.go
# trie/sync.go
# trie/sync_bloom.go
# trie/sync_test.go
2020-08-29 13:25:20 +02:00
Martin Holst Swende
b670cc44a5
eth/downloader: fix rollback issue on short chains
2020-08-29 12:30:17 +02:00
Péter Szilágyi
bfac425c87
core: define and test chain rewind corner cases ( #21409 )
...
* core: define and test chain reparation cornercases
* core: write up a variety of set-head tests
* core, eth: unify chain rollbacks, handle all the cases
* core: make linter smile
* core: remove commented out legacy code
* core, eth/downloader: fix review comments
* core: revert a removed recovery mechanism
# Conflicts:
# core/blockchain.go
# core/blockchain_test.go
# core/headerchain.go
# core/rawdb/database.go
# core/rawdb/freezer.go
# core/rawdb/schema.go
# eth/downloader/downloader.go
# eth/sync.go
# trie/sync.go
2020-08-29 12:29:00 +02:00
Alex Sharov
46a5d9c275
clean ( #1001 )
2020-08-29 17:07:12 +07:00
Alex Sharov
ca0bc897dc
clear logs ( #1000 )
2020-08-29 17:06:53 +07:00
Alex Sharov
8c9a55bb21
[To test on HDD] Write changesets to tx instead of mutation ( #978 )
...
* use mutation over tx
* clear
* add .CommitAndBegin() method
* multip put 2
* right sorting
* write non-random things to tx
* write non-random things to tx
* disable multi put 2
* clean
* clean
* remove multiput2
* clean
* clean
* clean
* clean
* clean
* add receipts
2020-08-29 08:32:51 +01:00
Alex Sharov
07ca5c643a
eth_syncing ( #991 )
...
* eth_syncing
* linters
* fix readme
* cleanup cli params
* go mod tidy
* remove memsize ui dep
* remove bloomfilter
* implement net_peerCount
* remove legacy flags
2020-08-29 08:24:50 +01:00
Alex Sharov
de16812a94
move tx limit from 16 to 32 blocks ( #988 )
2020-08-28 07:28:08 +01:00
Alex Sharov
df05a581dd
Add tx pool stage to transaction ( #981 )
...
* add tx_pool stage to tx
* clean
* don't call commit in read tx
* don't call commit in read tx
* clean
* more logs
* fix tx pool race
* more logs
* clean
2020-08-28 07:27:48 +01:00
Alex Sharov
af81bb0de6
[merge after release] Simplified loader ( #979 )
...
* v0
* more docs, less customization
* more docs, less customization
* more docs, less customization
* don't store db reference
* rely on fact that lmdb key/value valid until end of transaction
* clean
* clean
2020-08-27 19:21:50 +01:00
Alex Sharov
584a5581c8
show "Done!" message only if stage took more than 30 sec ( #977 )
2020-08-27 12:43:37 +07:00
Alex Sharov
714e5041e2
Better logs3 ( #976 )
2020-08-26 19:06:51 +07:00
Alex Sharov
522287ac18
Transactional cycle ( #966 )
...
* v0
* v1
* v3
* v4
* clean
* temporary fix of txpool
* Add debug logs about tx start/commit
* Add debug logs about tx start/commit
* Add debug logs about tx start/commit
* add condition
* tx pool to not hold own db
* try to enable TxPool in integration
* exclude tx pool from tx
* exclude tx pool from integration
* reduce limit
* fix integration
* clean
* clean
* clean
* clean
* clean
* exclude tx pool unwind
* exclude tx pool unwind in integration
* fix integration tx pool
* fix commit
* fix current stage after unwind
* fix current stage after unwind
* fix linter
* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.
* move body and headers unwind out of tx
* fix unwind order after reboot
* add support external tx to exec stage
* clean
* clean
* clean
* clean
* clean
* add logs
* better id check
* better id check
2020-08-26 07:02:10 +01:00
Alex Sharov
c61f8a5bdb
After reboot unwind stack has wrong order ( #972 )
...
* fix unwind order after reboot
* Fix test
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-08-24 18:15:22 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync ( #970 )
...
* Log in GetAsOf
* Default plain state
* trace
* More test fixes
* Fix lint
* Fix
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix for events
* Repair events
* Fix lint
* Fix lint
* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Alex Sharov
43a4e34a6e
[to discuss] exec blocks without ObjectDb: Mutation over TxDb ( #947 )
...
* use mutation over tx
* clean
* clear
* add .CommitAndBegin() method
* clean
* increase timings for logging
* return ideal batch size
2020-08-24 12:07:59 +01:00
Alex Sharov
a6be18b915
ticker-based logs ( #954 )
...
* timer-based logs
* timer-based logs
* delegate progress calculation to user
* delegate progress calculation to user
* delegate progress calculation to user
* clear
* add logs to senders recovery
* use default dir in integration
* more logs
* more logs
2020-08-22 12:12:33 +02:00
Igor Mandrigin
18df6cd182
Fix parallel recovery senders ( #962 )
...
* fix parallel context creation
* fix a typo
2020-08-22 12:08:47 +02:00
Igor Mandrigin
fbcf48f29d
Store ethash datafiles in a separate directory. ( #949 )
2020-08-20 15:59:06 +01:00
ledgerwatch
01f211f3b8
Revert "IH: no retain list ( #946 )" ( #948 )
...
This reverts commit f86ed64288
2020-08-20 13:37:34 +01:00
