Commit Graph

1870 Commits

Author SHA1 Message Date
Igor Mandrigin
2a6478d799
turbo-api: create a second executable file with a custom stage example (#1055) 2020-09-05 18:07:27 +02:00
Alex Sharov
b9bdd003a7
KV: Raw (no custom logic) cursors for DupSort and DupFixed buckets (#1020) 2020-09-04 10:54:15 +07:00
ledgerwatch
136ecb4401
Update skip analysis (#1041) 2020-09-03 09:19:42 +01:00
Evgeny Danilenko
e4f495fa44
Get logs (#1028)
* it compiles

* after recent master

* fix linters warnings

* grpcV7

* go mod tidy

* unmarshall adresses or adress

* fix linters

* after cr

* after cr

* after cr

* after cr

* fix tests

* remove dev version

* it compiles

* mod tidy

* fix bin deps

* use stable version of grpc

* switch back to master constructor

* switch back to master constructor

* add a bit docs

* add a bit docs

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-09-03 08:51:19 +01:00
ledgerwatch
c06dbdadf9
Use Append for changesets and receipts (#1018)
* Use Append for changesets and receipts

* Fix lint

* Delete all receipts on unwind

* Fix lint

* Fix unwind keys for changeset

* Fix test

* Use writeReceipts flag

* Revert to Append

* Delete receipts too when resetting
2020-09-01 07:48:25 +01:00
ledgerwatch
1a0e9275e1
Headers PoC - intermediate changes (#1016)
* First algorithm

* Update doc

* Second algorithm implemented

* Prepend (unfinished)

* More on prepend

* More fixes

* Change CalcDifficulty interface

* fix make test

* Fix formatting

* Another test

* More on Preprend

* Another test for Prepend

* More tests for Prepend

* More tests for Prepend

* Move files to turbo

* Add another test for Prepend

* Started adding Append algorithm

* More work on Append

* Start changing the terminology

* Split trees into segments

* More fixes

* Fix compilation

* Connect

* FindAnchors, FindTip

* Tip has anchor and not anchorParent field

* Remove Prepend and Append

* Add Connect

* NewAnchor algorithm

* More to recovery from files

* Add more to AddAnchor

* Request more headers

* Header fix-sizes serialisation

* Upgrade CI to 1.15

* Fix lint

* Fix lint

* replace RLP with fixed-sized serialisation

* Fix lint

* More algos

* Fix lint
2020-08-31 06:27:51 +01:00
Igor Mandrigin
9e3d4b8b2a linters 2020-08-29 15:43:33 +02:00
Igor Mandrigin
7396ccc37e linters 2020-08-29 15:37:27 +02:00
Igor Mandrigin
637493e8f1 Merge branch 'master' into geth-1.9.20 2020-08-29 15:28:03 +02:00
Igor Mandrigin
1e69e76f6d fix tests 2020-08-29 15:15:37 +02:00
Igor Mandrigin
bd61245ece test fixups 2020-08-29 14:35:39 +02:00
Igor Mandrigin
948cd29be7 fixes 2020-08-29 13:43:44 +02:00
Igor Mandrigin
78e466d06a fix imports 2020-08-29 13:31:46 +02:00
Martin Holst Swende
991a1c063d core: do less lookups when writing fast-sync block bodies (#21468) 2020-08-29 13:25:33 +02:00
gary rong
550bd11f9d core, eth, les, trie: add a prefix to contract code (#21080)
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	consensus/clique/clique.go
#	consensus/ethash/consensus.go
#	core/block_validator.go
#	core/blockchain.go
#	core/blockchain_test.go
#	core/genesis.go
#	core/rawdb/accessors_indexes_test.go
#	core/rawdb/accessors_metadata.go
#	core/rawdb/chain_iterator_test.go
#	core/rawdb/database.go
#	core/rawdb/schema.go
#	core/state/database.go
#	core/state/iterator_test.go
#	core/state/statedb.go
#	core/state/sync.go
#	core/state/sync_test.go
#	core/tx_pool_test.go
#	core/types/block_test.go
#	core/types/derive_sha.go
#	eth/downloader/downloader.go
#	eth/downloader/queue.go
#	eth/downloader/statesync.go
#	eth/fetcher/block_fetcher.go
#	eth/fetcher/block_fetcher_test.go
#	eth/handler.go
#	les/odr_requests.go
#	les/server_handler.go
#	light/odr.go
#	light/odr_test.go
#	light/trie.go
#	miner/worker.go
#	trie/database.go
#	trie/secure_trie.go
#	trie/sync.go
#	trie/sync_bloom.go
#	trie/sync_test.go
2020-08-29 13:25:20 +02:00
Péter Szilágyi
bfac425c87 core: define and test chain rewind corner cases (#21409)
* core: define and test chain reparation cornercases

* core: write up a variety of set-head tests

* core, eth: unify chain rollbacks, handle all the cases

* core: make linter smile

* core: remove commented out legacy code

* core, eth/downloader: fix review comments

* core: revert a removed recovery mechanism
# Conflicts:
#	core/blockchain.go
#	core/blockchain_test.go
#	core/headerchain.go
#	core/rawdb/database.go
#	core/rawdb/freezer.go
#	core/rawdb/schema.go
#	eth/downloader/downloader.go
#	eth/sync.go
#	trie/sync.go
2020-08-29 12:29:00 +02:00
Alex Sharov
8c9a55bb21
[To test on HDD] Write changesets to tx instead of mutation (#978)
* use mutation over tx

* clear

* add .CommitAndBegin() method

* multip put 2

* right sorting

* write non-random things to tx

* write non-random things to tx

* disable multi put 2

* clean

* clean

* remove multiput2

* clean

* clean

* clean

* clean

* clean

* add receipts
2020-08-29 08:32:51 +01:00
Alex Sharov
d30df505c8
Don't read cwd in blockchain validator, remove debug code (#992)
* print cwd

* don't read cwd, remove debug code

* oops, overdelete

* clean
2020-08-28 10:30:22 +01:00
Alex Sharov
df05a581dd
Add tx pool stage to transaction (#981)
* add tx_pool stage to tx

* clean

* don't call commit in read tx

* don't call commit in read tx

* clean

* more logs

* fix tx pool race

* more logs

* clean
2020-08-28 07:27:48 +01:00
Suhabe Bugrara
8b31944611
Control Flow Analysis (#990)
* First

* More on SA interpreter

* Fixup

* Add cfg action to hack binary that invokes the SaInterpreter. Added an operation handler for PUSH1

* refactor cfg tests into separate file

* Move cfg tests into separate file

* More refactoring into new file

* dataflow interpreter

* work on cfg0

* finish cfg0

* df works on base examples

* refactor into dataflow spec

* add bounded stack

* add harder example

* fix switch pass thru

* fix switch pass thru

* bug fix, and better printing

* manual merge

* restore call to test gencfg

* abstract interpretation based cfg analysis

* fix post signature

* use uint256 instead uint64, add post function

* preprocess stmts

* initial implementation of resolve

* fix resolve

* fix resolve

* print stmts for edges

* print stmts for edges

* print states

* print states

* bug fixes, debugging

* fix jumpi dest - first working impl

* reachability analysis to filter out dead edges

* add all transfer functions

* larger contract bytecodes from solc compiler

* simple solidity contract goes thru

* add deposit contract bytecode

* rename deposit contract test

* fix new contract arg

* Address non-determinism leading to imprecise results

* improve debugging output

* improve debugging output

* improve debugging output

* fix for bug causing incorrect analysis results

* fix for bug causing incorrect analysis results

* fix for bug causing incorrect analysis results

* add more test cases

* fix coverage bug

* debugging for non-termination

* fix bad fixpoint check

* fix data inference

* fix transfer function for halting stmts

* switch to deposit contract test, disable debugging

* add anly counter to viz, fix stmt.valid check

* show all preds, adjust anlycounter behavior

* dfs instead of bfs to fail earlier

* viz improvements

* add worklist size to viz

* add test case for private functions

* valueset analysis

* add more checks to fail earlier in the analysis to help debugging, improve debugging output, catch additional bad jumps

* delete old code

* delete old code

* delete old code

* fix up minor changes to jump table

* copy over comments from cgf-1 branch

* remove minor diffs

* add recompiled deposit contract

* graph viz

* cleanup/refactoring

* initial impl of viz

* script to run cfg anly and generate dot file

* div example

* accept bytecode from cmd line

* add minimal deposit contract example

* replace valueset analysis with stackset analysis

* get in sync with master

* sync with master

* fix linting

* fix linting

* fix linting

* reformatting

* fix linting

* fix linting

* fix linting

* fix linting

* fix linting

* fix linting

* fix linting

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-08-28 07:26:49 +01:00
Marius van der Wijden
08d3993cef core: more detailed metering for reorgs (#21420) 2020-08-27 21:23:51 +02:00
Alex Sharov
af81bb0de6
[merge after release] Simplified loader (#979)
* v0

* more docs, less customization

* more docs, less customization

* more docs, less customization

* don't store db reference

* rely on fact that lmdb key/value valid until end of transaction

* clean

* clean
2020-08-27 19:21:50 +01:00
Giuseppe Bertone
5388193c2c core/state: fixed some comments (#21450)
# Conflicts:
#	core/state/state_object.go
#	core/state/statedb.go
#	core/state/statedb_test.go
2020-08-27 17:38:12 +02:00
Igor Mandrigin
12aae871fd skip_analysis: update block number 2020-08-27 10:35:32 +02:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975)
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
Alex Sharov
522287ac18
Transactional cycle (#966)
* v0

* v1

* v3

* v4

* clean

* temporary fix of txpool

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* add condition

* tx pool to not hold own db

* try to enable TxPool in integration

* exclude tx pool from tx

* exclude tx pool from integration

* reduce limit

* fix integration

* clean

* clean

* clean

* clean

* clean

* exclude tx pool unwind

* exclude tx pool unwind in integration

* fix integration tx pool

* fix commit

* fix current stage after unwind

* fix current stage after unwind

* fix linter

* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.

* move body and headers unwind out of tx

* fix unwind order after reboot

* add support external tx to exec stage

* clean

* clean

* clean

* clean

* clean

* add logs

* better id check

* better id check
2020-08-26 07:02:10 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Alex Sharov
2facf92533
Better logs2 (#964) 2020-08-23 06:55:17 +07:00
Alex Sharov
a6be18b915
ticker-based logs (#954)
* timer-based logs

* timer-based logs

* delegate progress calculation to user

* delegate progress calculation to user

* delegate progress calculation to user

* clear

* add logs to senders recovery

* use default dir in integration

* more logs

* more logs
2020-08-22 12:12:33 +02:00
ledgerwatch
7c16160087
Add IntraBlockState copy function to RPC API (#960) 2020-08-21 21:53:23 +01:00
ledgerwatch
fd7e91a51c
Update skip_analysis (#942) 2020-08-19 20:34:45 +01:00
b00ris
eb46dd68df
Account range testing (#938)
* save state

* save state

* save new timeouts

* compare passed

* save state

* cleanup
2020-08-19 07:33:49 +01:00
Evgeny Danilenko
3b40819444
grpc regenerate (#936)
* grpc regenerate

* use string type for id in net_version
2020-08-18 20:22:49 +03:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928)
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size (#918)
* add logging to loader

* use pure tx in etl loading, logs in mutation commit

* clean

* bletter logging and more cleanup

* bletter logging and more cleanup

* increase batch size to 500M

* better batch commit logging

* async fsync

* sync fsync

* sync fsync

* unify logging

* fix corner-case when etl can use empty bucket name

* fix tests

* better logging

* better logging

* rebase master

* remove lmdb.NoMetaSync flag for now

* consistent walk and multi-walk

* clean

* sub tx

* add consistent multi-put

* implement dupsort support in one new cursor method

* clear
2020-08-17 07:45:52 +01:00
Alex Sharov
6379516a0f
add support of blocks compression to txpool stage (#925) 2020-08-17 10:17:40 +07:00
Igor Mandrigin
2f881f3865
Use batches when unwinding indexes (#931) 2020-08-16 18:54:33 +02:00
Igor Mandrigin
d848233275 Merge branch 'master' into geth-1.9.19
# Conflicts:
#	cmd/geth/consolecmd.go
#	cmd/rpcdaemon/service/service.go
#	cmd/utils/flags.go
#	eth/backend.go
2020-08-15 21:55:09 +02:00
Igor Mandrigin
71758e1fcc fixups 2020-08-15 21:34:22 +02:00
Igor Mandrigin
8346124959 fix core/types 2020-08-15 21:15:44 +02:00
Igor Mandrigin
201524c33e fix node tests 2020-08-15 20:21:30 +02:00
Igor Mandrigin
d6fe697ee1 fixups. make geth successful 2020-08-15 19:32:05 +02:00
Igor Mandrigin
9da7a7c608 fixez 2020-08-15 17:30:34 +02:00
Alex Sharov
90fa69a9e5
Swap IH and HashState stages (#922)
* move ih stage v0

* save progress

* save working version

* baby step

* baby step

* baby step

* non-baby step

* cleanup
2020-08-15 14:49:36 +01:00
Igor Mandrigin
3ba734f657 fixups 2020-08-14 17:43:48 +02:00
Martin Holst Swende
9393ba7d8f core/vm: avoid map lookups for accessing jumpdest analysis
# Conflicts:
#	core/vm/contract.go
2020-08-14 16:35:53 +02:00
Hao Duan
44d9fba09a core: avoid modification of accountSet cache in tx_pool (#21159)
* core: avoid modification of accountSet cache in tx_pool

when runReorg, we may copy the dirtyAccounts' accountSet cache to promoteAddrs
in which accounts will be promoted, however, if we have reset request at the
same time, we may reuse promoteAddrs and modify the cache content which is
against the original intention of accountSet cache. So, we need to make a new
slice here to avoid modify accountSet cache.

* core: fix flatten condition + comment

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-08-14 16:34:29 +02:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object (#914) 2020-08-14 13:41:18 +07:00
Evgeny Danilenko
5d39fe8873
RPCDeamon coinbase, net_version (#911)
* use eth.backend

* net_version

* add coinbase to the console test
2020-08-12 14:47:59 +01:00
ucwong
9cf277a175 core/bloombits: use single channel for shutdown (#20878)
This replaces the two-stage shutdown scheme with the one we
use almost everywhere else: a single quit channel signalling
termination.

Co-authored-by: Felix Lange <fjl@twurst.com>
2020-08-12 12:43:50 +02:00