Commit Graph

1790 Commits

Author SHA1 Message Date
Igor Mandrigin
3aedcbcd7d Use datadir for temp files when generating indexes 2020-08-02 12:32:41 +02:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation ()
* Jumpdest skipping optimisation

* Fix formatting

* Move skipAnalysis into vmConfig, introduce tracing ability

* Improve detection logging

* Added release instructions

* Fix lint
2020-08-01 17:56:57 +01:00
Evgeny Danilenko
e44dc2ad6a
RpcDeamon eth_call and eth_estimateGas ()
* eth_call and eth_estimateGas

* use NewPlainDBState
2020-07-29 17:21:34 +01:00
ledgerwatch
ba6c6aaef2
Move BadHashes to stage1 () 2020-07-29 15:33:01 +01:00
ledgerwatch
38ecda3500
Remove last fast block message, and possibilty for db reset () 2020-07-29 14:59:51 +01:00
Alex Sharov
f500148b86
remove BoltDatabase, BadgerDatabase, RemoteDatabase classes (replaced by objectDb) () 2020-07-27 22:59:54 +01:00
Alex Sharov
f037aed6e5
wait for verify_headers method () 2020-07-27 19:26:24 +07:00
Alex Sharov
c4a1ffa4f0
Grpc-based remoteDb () 2020-07-27 19:15:48 +07:00
b00ris
9fc51a3efd
Account range ()
* save state

* cleanup

* add walkasof test for storage with chunks

* fix lint

* fix lint

* remove useless comments

* fix dump test

* move accountRange test to plain state

* get last block fixes
2020-07-27 07:09:50 +01:00
Alex Sharov
95365c8f37
remove ReadLastBlockNumber method = read execution stage status instead ()
* remove ReadLastBlockNumber method = read execution stage status instead

* make linter happy
2020-07-24 14:46:49 +01:00
ledgerwatch
cf5b0c8836
Integrate txPool into staged sync, fix tx_pool_test ()
* cmd/tester to initiate reorg

* Second commit

* Turn on profile

* More

* Unwind function

* More fixes

* Before fixing tests

* Fix compile error

* Fix lint

* Fix formatting

* Fix lint

* Fix nil in tests

* Fix lint

* Decouple txpool from rawdb

* Fix some tx tests

* One more test

* Fix the rest of tests in tx_pool_test.go

* Fix formatting
2020-07-23 21:52:41 +01:00
Alex Sharov
e65b743878
Prometheus dashboard revisit () 2020-07-21 15:33:03 +07:00
Alex Sharov
a7e5c4b453
Remove remote_chain package, fix rpc methods: eth_blockNumber and eth_getBlockByNumber () 2020-07-21 15:19:04 +07:00
ledgerwatch
b765ff0e33
cmd/tester initiates staged sync, and unwind (but not reorg) ()
* First push

* Remove batch from Blockchain

* Fix compile error in cmd/integration

* Fix compile errors in cmd/state

* Fix compile errors

* Save changes

* Fix core tests (skip tx tests)

* More test fixes

* Fix compile error

* Fix formatting

* Fix formatting

* Fix lint

* Fixes

* Skip some miner tests

* Fix tests

* Cleanup
2020-07-19 09:11:53 +01:00
b00ris
12aa55bcbb
Remove incarnation inversion ()
* remove incarnation inversion

* fix lint

* remove incarnation inversion

* remove comment

* remove inversion from new code

* revert for unwind
2020-07-17 15:18:31 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly ()
* close goruotines

* fix using defer

* close pm managers a bit later

* fmt

* finalizer for in-mem db

* after cr

* revert finalizer

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
18e67cd07d
cmd/tester to initiate correct staged sync (but not reorg yet) ()
* Disable caches

* Fix cmd/integration?

* state root hash matches

* Cleanup

* Skip TestForkIDSplit

* Try to fix the test
2020-07-15 13:26:59 +01:00
ledgerwatch
3359ba7a04
Make cmd/tester initiate staged sync, remove some pools and caches ()
* no pools

* Remove excessive logging

* Revert

* Initial commit

* Cleanup

* Fix some compile errors

* Fix core tests

* Fix more tests

* Fix eth/downloader

* go mod tidy

* Fix core/vm

* Fix formatting

* Fix formatting

* Fix lint

* Fix lint

* Switch tests to StagedSync

* Cleanup

* Reuse cache in stage4

* Fix

* Fix formatting

* Try to fix test
2020-07-15 07:15:48 +01:00
Evgeny Danilenko
2cccf6d02c
Tx pool: prevent double start or stop ()
* use isStarted

* set on stop
2020-07-14 14:14:43 +01:00
Alex Sharov
6043c09ddf
Replace global buff pool by local, because of buffer leaks to larger pools () 2020-07-14 08:56:47 +07:00
Evgeny Danilenko
9d1ba168de
safe closing txpool reorgShutdownCh ()
* fix

* linters

* buffered chan to prevent blocking

* create channel on start

* fmt

* fix
2020-07-12 08:27:08 +01:00
ledgerwatch
665b1d88b9
Speed up GenerateChain by using intermediate hashes ()
* GenerateChain to support intermediate hashes

* Fix formatting

* Changeset stats

* Fix formatting
2020-07-10 22:37:34 +01:00
Alex Sharov
8335700331
Integration: compare to reference_db after all cycles, better cancelation, add to makefile ()
* cmp to reference db if --reference_chaindata provided

* graceful shutdown

* add ./cmd/integration to makefile

* add interruption support for different stages

* don't compare errors directly
2020-07-10 07:03:18 +01:00
ledgerwatch
72dcf478d3
Fix debug_storageRangeAt in rpc daemon ()
* Fix storageRangeAt

* Fix formatting

* Fix lint

* Fix lint

* Prevent racing by closing db after the blockchain
2020-07-10 06:44:01 +01:00
Alex Sharov
d98382fdf8
Integration add checkChangeSet into state_stages command ()
Integration add checkChangeSet into state_stages command ()
2020-07-10 10:25:40 +07:00
Evgeny Danilenko
5f77570556
Make transaction pool startable and stoppable. Staged sync. ()
* after concurrency testing

* fixes for linters

* add an extra parameter

* change init order for protocolManager

* green tests
2020-07-09 13:53:35 +01:00
ledgerwatch
887fc4f99c
GenerateChain without using TrieDbState ()
* Different chain generation

* More fixes

* Fix TestSideLogRebirth

* Fix the rest of core tests

* Fix accounts/abi/bind

* Fix compile errors for cmd/pics

* Fix consensus/clique tests

* More fixes

* Fix core/state tests

* Fix eth tests

* Fix eth/downloader tests

* Fix more tests

* Fix compile errors

* Some more fixes

* Save

* Exists

* Fix last tests

* Clean up

* Fix formatting

* Fix linter

* Fix formatting

* Fix linter

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint
2020-07-09 07:15:28 +01:00
b00ris
eca0f233ec
[WIP] WalkAsOf ()
* save state

* walkAsOf by cs test passed

* cs search for plain state

* save state

* fix accounts tests

* refactor walkAsOf account tests

* fix storage test

* refactor walkAsOf storage tests

* fix lint

* fix test

* save state

* save state

* add test with fixed bits

* fmt

* add stages check

* fix lint

* fix lint

* remove obsoleted methods
2020-07-09 07:13:45 +01:00
Igor Mandrigin
9881a2ccb3
Fix broken witness generation. () 2020-07-07 12:24:55 +01:00
Alex Sharov
305f8aff02
Integration tests2 ()
```
go run ./cmd/integration reset --chaindata=...
go run ./cmd/integration state_stages -h
go run ./cmd/integration state_stages  --chaindata=... --verbosity=3 --block=2_000_000 --unwind=10 --unwind_every=1_000 
```
Also, it inherits flags from geth:
```
--pprof.cpuprofile=./cpu.out   // to file
--pprof --pprof.port=6060     // launch pprof server
--metrics                  //  sends to prometheus 
```
2020-07-07 11:00:25 +07:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes ()
* Fixing history generation

* Try fixes

* Reset from 0

* Cleanup
2020-07-06 07:34:24 +01:00
Alex Sharov
2aadd94acb
remove mgr feature and stateSize calculation feature () 2020-07-05 07:17:27 +01:00
ledgerwatch
49181a298b
Special bucket for transaction senders ()
* Senders in the separate bucket

* Pre-create txSenders bucket

* Fix compile error

* Fix compile error

* Fix compile error, remove block transformations

* Fix formatting

* Fix test
2020-07-03 08:23:49 +01:00
Giulio rebuffo
011a15350b
fix stage 2 ()
* removed unused actions in stage 2

* fixed stage 2
2020-07-02 19:44:08 +01:00
Giulio rebuffo
3890900dd0
removed unused actions in stage 2 () 2020-07-02 08:38:25 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets ()
* Debugging changesets

* Kinda works

* Fix compile error

* Fix formatting

* Fix lint

* Duplicate entries kludge

* Fix compile error

* Cleanup
2020-07-01 15:56:56 +01:00
Giulio rebuffo
3ee1435ac2
integration of insertBodies instead of insertChain ()
* integration of insertBodies instead of insertChain

* remove seal verification from stage2

* fixed seals
2020-07-01 08:23:15 +01:00
Giulio rebuffo
85ab6ff8aa
added insertBodies () 2020-06-29 17:26:33 +01:00
ledgerwatch
ed866e6934
non-concurrent ETL, debug_traceTransaction in rcpdaemon ()
* Fixing history index

* Remove chunk generation, fix formatting

* Fix compile error, clean up hack.go

* Fix output tests

* Fix index generator test

* Fixed checkChangeSets

* Fix linter
2020-06-28 07:10:27 +01:00
ledgerwatch
4c456a649f
Separating HashState and IntermediateHashes stages again, fixing errors ()
* TestWatchNoDir not to be parallel

* Print ks and inc

* Print addrHash

* Change the buffer

* Print loading

* Skip

* More logging

* Error out earlier

* Handle empty codes

* Handle empty codes

* Remove logging

* Compare states

* Not do stage5

* compareBucket

* Preimage

* Clearer errors

* No need to clean up contract code

* Restore stage6

* Printing

* Skip the skipping

* Print all

* Change buffer type

* Add limit to stage5

* Always fail

* Remove exessive logging

* Restore buffer type

* Revert

* Print when exception

* Reenable skipping

* Skip storage items for deleted contracts

* not shortcut

* Remove removeAccount

* Re-enable state hashing

* Default to plain state

* Disable hashing state

* Reenable reset5

* Print unfurl list

* Enable removingAccount

* No printing

* Reenable stage5 commit

* Swap order of stages

* Prevent backwards promotion, reset tx lookup

* reset finish

* Introduce storage item replacement

* See if unwind works

* Restore removingAccount

* Don't do removeAccount for unwinding

* Possible fix

* Proper(er) fix

* Don't exclude unwinding

* Remove unwinding flag

* Fix formatting

* Fix lint

* Not to ignore blocks if they cause reorg

* Fix test, separate stages again

* Fix TestUnwind

* Fix stage

* Swap unwinding

* Revert to unwinding flag

* Print unfurl list

* Print

* Print inside receive

* Print after

* No printing

* Cleanup

* Not use blockCache when doing GetBlock
2020-06-26 07:46:05 +01:00
b00ris
e8b617bbf2
Add sort.Search to account changeset ()
* fast search

* fix

* remove copy bytes
2020-06-16 15:03:35 +01:00
Alex Sharov
57777e7c60
Prepare codebase for future default DB change ()
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* choose db based on file suffix

* implement db.id method

* implement db.id method

* use ethdb.NewDatabase method

* use ethb.MustOpen method

* cleanup

* support TEST_DB env flag

* create db path automatically needed

* bolt - don't change prefix on happy path
2020-06-16 14:36:16 +01:00
Alex Sharov
8d04689041
close db in tests () 2020-06-16 12:05:40 +07:00
ledgerwatch
dec30b698d
Fix execution for block 10094566 ()
* First sketch of the fix

* Re-enable stage4

* Fix formatting

* Park the changes

* Save updates

* Save update

* Cleanup

* Cleanup

* Cleanup

* Fix lint

* Cleanup

* Update

* Tx tracing

* Update

* Cleanup

* Switch tests back to bolt

* Test cleanup

* Update

* Polymorthic CREATE2 test

* Add assertions

* Fix formatting

* Fix test

* Cleanup

* Cleanup

* Cleanup
2020-06-15 22:24:08 +01:00
Igor Mandrigin
93f6c4c01a post-rebase fixups 2020-06-15 19:38:54 +03:00
Ev
f67b3d7d45 core: fix typo in comments () 2020-06-15 19:38:13 +03:00
Marius van der Wijden
cc69b76852 core/vm, crypt/bls12381: fixed comments in bls ()
* core/vm: crypto/bls12381: minor code comments

* crypto/bls12381: fix comment
2020-06-15 19:38:13 +03:00
Martin Holst Swende
7347d036b0 cmd, core, params: inital support for yolo-v1 testnet ()
* core,params,puppeth: inital support for yolo-v1 testnet

* cmd/geth, core: add yolov1 console flag

* cmd, core, params: YoloV1 bakein fixups

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	core/vm/interpreter.go
#	params/config.go
2020-06-15 19:38:13 +03:00
kilic
0921861edb core/vm, crypto/bls12381, params: add bls12-381 elliptic curve precompiles ()
* crypto: add bls12-381 elliptic curve wrapper

* params: add bls12-381 precompile gas parameters

* core/vm: add bls12-381 precompiles

* core/vm: add bls12-381 precompile tests

* go.mod, go.sum: use latest bls12381 lib

* core/vm: move point encode/decode functions to base library

* crypto/bls12381: introduce bls12-381 library init function

* crypto/bls12381: import bls12381 elliptic curve implementation

* go.mod, go.sum: remove bls12-381 library

* remove unsued frobenious coeffs

supress warning for inp that used in asm

* add mappings tests for zero inputs

fix swu g2 minus z inverse constant

* crypto/bls12381: fix typo

* crypto/bls12381: better comments for bls12381 constants

* crypto/bls12381: swu, use single conditional for e2

* crypto/bls12381: utils, delete empty line

* crypto/bls12381: utils, use FromHex for string to big

* crypto/bls12381: g1, g2, strict length check for FromBytes

* crypto/bls12381: field_element, comparision changes

* crypto/bls12381: change swu, isogeny constants with hex values

* core/vm: fix point multiplication comments

* core/vm: fix multiexp gas calculation and lookup for g1 and g2

* core/vm: simpler imput length check for multiexp and pairing precompiles

* core/vm: rm empty multiexp result declarations

* crypto/bls12381: remove modulus type definition

* crypto/bls12381: use proper init function

* crypto/bls12381: get rid of new lines at fatal desciprtions

* crypto/bls12-381: fix no-adx assembly multiplication

* crypto/bls12-381: remove old config function

* crypto/bls12381: update multiplication backend

this commit changes mul backend to 6limb eip1962 backend

mul assign operations are dropped

* core/vm/contracts_tests: externalize test vectors for precompiles

* core/vm/contracts_test: externalize failure-cases for precompiles

* core/vm: linting

* go.mod: tiny up sum file

* core/vm: fix goimports linter issues

* crypto/bls12381: build tags for plain ASM or ADX implementation

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
# Conflicts:
#	core/vm/contracts.go
#	core/vm/contracts_test.go
#	go.sum
2020-06-15 19:38:13 +03:00
chenglin
efb387fff9 core: collect NewTxsEvent items without holding reorg lock ()
# Conflicts:
#	core/tx_pool.go
2020-06-15 19:38:13 +03:00