Commit Graph

227 Commits

Author SHA1 Message Date
Alex Sharov
4565bd43c1
Mdbx v0.9.2 (#1373)
* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* switch to master

* mdbx 0.9.2

* mdbx 0.9.2

* %<<$!%%%%%%<o'{}k.###

* %<<$!%%%%%%<o'{}k.###

* merge

* merge

* merge

* merge

* ad

* ad

* ad

* ad
2020-11-28 14:26:28 +00:00
Alex Sharov
068463dff4
Store transactions individually (#1358)
* Store transactions individually

* Store transactions individually

* save progress

* checkIndex

* merge
2020-11-22 21:25:26 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342)
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-11-16 12:08:28 +00:00
b00ris
6464da7670
Remote snapshot downloader (#1343)
* save state

* save state

* save state

* refactoring

* fix

* save state

* save state

* fmt

* fix lint

* restore torrents for external downloader

* fix lint

* download

* skip debug test

* debug

* remote debug

* small cli fixes

* skip debug test

* external snapshot predownloader

* get rid of remote downloader

* fix lint

* clean makefile

* fix lint

* fix lint

* cleanup

* fix ci

* fmt

* remove proto from interfaces

* Squashed 'interfaces/' content from commit acd02bb94

git-subtree-dir: interfaces
git-subtree-split: acd02bb94c5a421aa8f8d1fd76cd8aad668e9fcb
2020-11-13 16:16:47 +00:00
Horacio Mijail Antón Quiles
ed1819ec58
Opcode tracer (#1320)
* OpcodeTracer

* Cleanup, rename to opcode_tracer

* Bring back check_change_sets, remove repeated vars

* Cleanup, gofmt

* Linter, cleanup

* Rename "segments" to "basic blocks", "bblocks"

* gofmt

* Lintci madness
2020-10-30 19:44:00 +00:00
Horacio Mijail Antón Quiles
f8477e29ab
Move batch management into the writeReceipts case (#1321) 2020-10-30 08:41:35 +00:00
Alex Sharov
531d36efcb
mdbx support (#1235) 2020-10-28 10:18:10 +07:00
Alex Sharov
6f27e3043b
lmdb_readonly_dont_create_folder (#1293) 2020-10-25 08:39:09 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271)
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-25 08:38:55 +00:00
Alex Sharov
e85b388061
Protect from wrong class use: make ethdb.Tx incompatible with rawdb.DatabaseReader (#1288)
* protect_from_wrong_class_use

* more cases
2020-10-24 09:09:20 +01:00
ledgerwatch
88acfc55ca
Vary batch size for execution stage (#1274)
* Vary batch size for execution stage

* Fix in logging

* Improve log of timings

* Log improvements

* Log improvements

* Fix tests

* More log improvments, fix tests

* Fix tests

* Better logging for Bodies and Interhash

* Fix tests

* Minor fixes

* Fix test
2020-10-21 18:01:40 +01:00
b00ris
e5f6412da2
Update snapshot hash (#1243)
* remove useless bucket

* update snapshot hashes

* remove log
2020-10-17 08:48:20 +01:00
ledgerwatch
cd706d5081
Download/seed block bodies and block headers via BitTorrent (#1218)
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

* tidy mods

* Fix compile

* Fix lint

* Fix rpcdaemon running in the docker

Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-10-13 13:56:16 +01:00
ledgerwatch
b747ab5324
[WIP] CallTraces index (#1157)
* Initial commit for CallTraces index

* Fix compilation

* fix lint, add comment

* Fix integration

* Add Close function to ethdb.Cursor, fix some compile errors

* Try to stop cursor leak in Get

* Fix compile errors in RPC daemon

* Fix compile errors

* fixing another way

* Some fixes

* More fixes

* More fixes

* More fixes

* Fixes to core/state

* Fix lint

* Fix lint

* Fixes

* Stage caching for call trace stage

* Add mem stats

* Try to stop the leak

* Turn off debug

* Chunks for 10k blocks

* Print

* Revert "Print"

This reverts commit 5ffada4828d61e00e5dad1ca12c98258dfbbad00.

* Revert "Chunks for 10k blocks"

This reverts commit cfb9d498e782e5583d41c30abf0e2137da27383e.

* Trying to fix the leak

* Don't compute receipts in re-tracing

* Not compose block

* Print speed, fix receipts, bigger caches

* Fix lint

* Utilise changeset info

* Counters

* Use NoReceipts and ReadOnly

* ReadOnly is incompatible with caching

* Skip test leaking transactions

* Fix block test

* Change disable message for call-traces stage

* Use block option for call traces integration

* Fix retracing due to incarnation
2020-10-12 09:39:04 +01:00
Alex Sharov
88a099702b
rawdb.ReadCanonicalHash to return err (#1204)
* read_canonical_hash_return_err

* hack hugeFreelist

* fix tests
2020-10-10 07:05:56 +01:00
ledgerwatch
1af6cbe37b
Revert "Download/seed headers and block bodies via BitTorrent (#1195)" (#1202)
This reverts commit 94360212cb.
2020-10-07 21:49:44 +01:00
ledgerwatch
94360212cb
Download/seed headers and block bodies via BitTorrent (#1195)
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-10-07 13:15:49 +01:00
ledgerwatch
bdb0155856
Revert "Add sync by bittorrent snapshots (#1160)" (#1194)
This reverts commit 4ebade3583.
2020-10-06 22:31:57 +01:00
b00ris
4ebade3583
Add sync by bittorrent snapshots (#1160)
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-06 20:24:48 +01:00
Alex Sharov
61fbe34024
Better cbor support (#1163) 2020-10-02 10:56:13 +07:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124)
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-28 18:18:36 +01:00
Horacio Mijail Antón Quiles
252863fedf
Use the passed VMConfig (#1141)
* Use the passed VMConfig

* Explicitly pass the vmConfig into runBlock
2020-09-27 21:31:49 +01:00
Giulio rebuffo
d33302c2be
Mutual TLS authentication (for now without verifying the Common Name) (#1113)
* added tls auth

* added client side

* put --tls

* fixed flag

* Add key/cert generation instructions, turn off Common Name verification

* Add CLI arguments and Warning

* Lint

* Update the doc about Internal IP

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-19 15:16:04 +01:00
Igor Mandrigin
adf52465e3
move ./trie to ./turbo/trie (#1114)
Useful for minimizing merge conflicts when rebasing new geth
2020-09-14 11:33:39 +01:00
Giulio rebuffo
23d89c0941
Added TLS handshake to RPCDaemon (#1089)
* added eth_getStorageAt

* used uint32

* now its 256

* incarnation

* added TLS handshake

* code minified

* lint

* minified client

* Update flags.go

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-11 21:17:37 +01:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support (#1080)
* etl.Loader - allow use of custom comparator

* log timing

* try now

* try now

* more performance

* etl.Loader - allow use of custom comparator

* working version

* simplify IH cursor

* clean

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* clean

* add only unwind support

* squash

* squash

* clean

* fix test

* clean

* clean

* clean
2020-09-10 13:35:58 +01:00
Alex Sharov
fd9c7c3b2d
kill bolt (#1085)
* kill bolt

* kill bolt

* fix test

* clean
2020-09-09 11:09:55 +01:00
Alex Sharov
5ce73d438a
fix integration (#999) 2020-08-29 15:27:20 +07:00
ledgerwatch
335aedc7e3
Cut down the stateGrowth generation code (#973)
* Cut down the stateGrowth generation

* Fix NPE

* Fix for lastTimestamp

* Adjust to the current state

* Potential fix

* Fix for shorter keys in the state

* Extract code

* Iterate over code
2020-08-28 10:31:37 +01:00
Alex Sharov
d73b036cff
python_bublic_print_cli (#994) 2020-08-28 14:19:34 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975)
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Giulio rebuffo
0ffe32ae16
State growth restore (#951)
* new state growth

* lint
2020-08-21 07:32:11 +01:00
Alex Sharov
f827f86c11
Rpcdaemon as lib 2 (#943) 2020-08-20 10:52:27 +07:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928)
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
b2e35eb63f
rename "remote2" to "remote" (#932) 2020-08-17 07:47:06 +01:00
Alex Sharov
e04c4ebe58
KV: move bucket.Get and bucket.Cursor methods to Tx object (#914) 2020-08-14 13:41:18 +07:00
Alex Sharov
3aa5526d5a
fix miss-type (#916) 2020-08-13 12:31:17 +07:00
Alex Sharov
9047f4fea0
python: don't hardcode buckets list (#915) 2020-08-13 11:40:30 +07:00
Alex Sharov
760fe0f0b6
DupSort of hash state (#896) 2020-08-12 09:57:55 +07:00
ledgerwatch
6f88e4058f
Fix (#908) 2020-08-11 20:48:37 +01:00
Alex Sharov
d9d9e14f45
change bucket type to string (#894) 2020-08-11 06:55:32 +07:00
Alex Sharov
81c487803e
add python docs (#900) 2020-08-10 16:36:46 +07:00
Igor Mandrigin
8600616c3d geth 1.9.18 post-rebase fixups 2020-08-08 17:33:35 +02:00
Alex Sharov
d458c4cc1e
Migrations: use stage name as db key (#868) 2020-08-05 17:13:35 +07:00
ledgerwatch
0514501937
Revert "Use stage name as db key (#858)" (#867)
This reverts commit 7290bf519e.
2020-08-04 11:05:27 +01:00
Alex Sharov
7290bf519e
Use stage name as db key (#858) 2020-08-04 16:25:28 +07:00
Alex Sharov
ecc94a63f0
DupSort for Plain and Hashed state buckets (behind feature flags) (#854)
* return error from rawdb

* squash

* v14

* improve performance of put

* clean

* clean

* hide feature behind ENV variable

* cleanup

* cleanup

* disable ipc and make Readme less confusing (people thought points are depend on each-other)

* fix test

* cleanup

* cleanup
2020-08-02 12:53:08 +01:00
Igor Mandrigin
3aedcbcd7d Use datadir for temp files when generating indexes 2020-08-02 12:32:41 +02:00
ledgerwatch
cf799157cc
Jumpdest skip optimisation (#851)
* Jumpdest skipping optimisation

* Fix formatting

* Move skipAnalysis into vmConfig, introduce tracing ability

* Improve detection logging

* Added release instructions

* Fix lint
2020-08-01 17:56:57 +01:00
Alex Sharov
c4a1ffa4f0
Grpc-based remoteDb (#788) 2020-07-27 19:15:48 +07:00
Alex Sharov
95365c8f37
remove ReadLastBlockNumber method = read execution stage status instead (#780)
* remove ReadLastBlockNumber method = read execution stage status instead

* make linter happy
2020-07-24 14:46:49 +01:00
Alex Sharov
a7e5c4b453
Remove remote_chain package, fix rpc methods: eth_blockNumber and eth_getBlockByNumber (#769) 2020-07-21 15:19:04 +07:00
Alex Sharov
1faa9dbcd4
add senders bucket to py (#763) 2020-07-20 16:11:47 +07:00
ledgerwatch
b765ff0e33
cmd/tester initiates staged sync, and unwind (but not reorg) (#758)
* First push

* Remove batch from Blockchain

* Fix compile error in cmd/integration

* Fix compile errors in cmd/state

* Fix compile errors

* Save changes

* Fix core tests (skip tx tests)

* More test fixes

* Fix compile error

* Fix formatting

* Fix formatting

* Fix lint

* Fixes

* Skip some miner tests

* Fix tests

* Cleanup
2020-07-19 09:11:53 +01:00
b00ris
12aa55bcbb
Remove incarnation inversion (#755)
* remove incarnation inversion

* fix lint

* remove incarnation inversion

* remove comment

* remove inversion from new code

* revert for unwind
2020-07-17 15:18:31 +01:00
Evgeny Danilenko
5ead20e288
close tx_cache and peer.handle goruotines properly (#748)
* close goruotines

* fix using defer

* close pm managers a bit later

* fmt

* finalizer for in-mem db

* after cr

* revert finalizer

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-07-16 14:27:24 +01:00
ledgerwatch
3359ba7a04
Make cmd/tester initiate staged sync, remove some pools and caches (#746)
* no pools

* Remove excessive logging

* Revert

* Initial commit

* Cleanup

* Fix some compile errors

* Fix core tests

* Fix more tests

* Fix eth/downloader

* go mod tidy

* Fix core/vm

* Fix formatting

* Fix formatting

* Fix lint

* Fix lint

* Switch tests to StagedSync

* Cleanup

* Reuse cache in stage4

* Fix

* Fix formatting

* Try to fix test
2020-07-15 07:15:48 +01:00
b00ris
eca0f233ec
[WIP] WalkAsOf (#724)
* save state

* walkAsOf by cs test passed

* cs search for plain state

* save state

* fix accounts tests

* refactor walkAsOf account tests

* fix storage test

* refactor walkAsOf storage tests

* fix lint

* fix test

* save state

* save state

* add test with fixed bits

* fmt

* add stages check

* fix lint

* fix lint

* remove obsoleted methods
2020-07-09 07:13:45 +01:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes (#716)
* Fixing history generation

* Try fixes

* Reset from 0

* Cleanup
2020-07-06 07:34:24 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets (#698)
* Debugging changesets

* Kinda works

* Fix compile error

* Fix formatting

* Fix lint

* Duplicate entries kludge

* Fix compile error

* Cleanup
2020-07-01 15:56:56 +01:00
ledgerwatch
ed866e6934
non-concurrent ETL, debug_traceTransaction in rcpdaemon (#692)
* Fixing history index

* Remove chunk generation, fix formatting

* Fix compile error, clean up hack.go

* Fix output tests

* Fix index generator test

* Fixed checkChangeSets

* Fix linter
2020-06-28 07:10:27 +01:00
Igor Mandrigin
69a20de5cc
Fix broken export file usage. (#680) 2020-06-20 21:07:39 +01:00
Alex Sharov
57777e7c60
Prepare codebase for future default DB change (#670)
* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* Add kv.tx.bucket.Clear() and db.ClearBuckets() methods

* choose db based on file suffix

* implement db.id method

* implement db.id method

* use ethdb.NewDatabase method

* use ethb.MustOpen method

* cleanup

* support TEST_DB env flag

* create db path automatically needed

* bolt - don't change prefix on happy path
2020-06-16 14:36:16 +01:00
Igor Mandrigin
93f6c4c01a post-rebase fixups 2020-06-15 19:38:54 +03:00
Alex Sharov
15096f273d
Remove ctx from Open. Stop goroutines on Close. (#650)
* remove ctx from MustOpen

* remove ctx from Open. Stop goroutines on Close.

* remove ctx from Open. Stop goroutines on Close.

* remove ctx from remote open (we have DialTimeout field to manage connection timeouts)

* enable RawReads and add native implementation of Get/Has methods
2020-06-12 10:31:21 +01:00
Alex Sharov
9961fef39e
experiment with rewriting gas_limits to python+lmdb (just to see how it can be) (#654) 2020-06-12 09:25:40 +01:00
ledgerwatch
853db762e2
Make GetAsOf work with partially missing history index (#648)
* Debug infos

* bucketStats lmdb by default

* Use bools instead of strings

* Fix linters

* Fix linter

* Finally fix

* Fix compile error

* Fix test

* Fix formatting

* Fix shadowing

* Cleanup

* cleanup
2020-06-11 07:19:15 +01:00
b00ris
b4ba764fb1
[WIP] TxLookup stage (#646)
* save state

* txlookup full results

* save state

* save state

* remove experiments

* some fix&lint

* add end key to txLookup and index generation

* change log message

* change log

* fix lint

* lint

* fix test
2020-06-10 23:07:14 +03:00
Andrew Ashikhmin
cf7b9a75a7
Issue 635: Stage4 to optionally generate persisted receipts (#644)
* Euphemerally -> Ephemerally

* Move StorageMode to ethdb and pass it to PrepareStagedSync

* linter

* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go

* Optionally write receipts in the execute stage

* memory profiler

* linter

* proper linter fix

* linter

* typo

* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
2020-06-09 14:11:09 +01:00
Alex Sharov
9f11dabede
Enable golint back but with conditions (#641)
* enable_golint_with_conditions

* enable_golint_with_conditions
2020-06-09 12:00:58 +01:00
ledgerwatch
462edc2345
CheckChangeSets to work with plain state and changeSets, optionally generate receipts (#639)
* Start from 9m7

* Regenerate IH + receipts

* Only stats for iH bucket

* Persist receipts

* Go all in

* Start from block 10m

* Convert DbState to use plain state

* Fix findHistory

* Hard-code export

* More fixes

* Fix test

* Fix formatting

* Introduce PlainDbState

* Actually return PlainDbState

* Fix formatting

* Fix name style lint

* Fix linters

* Fix history_test

* Fix blockchain_test

* Fix compile error

* Bucket stats from all buckets
2020-06-09 06:52:38 +01:00
Evgeny Danilenko
41ee460a27
optimize codeBitmap (#626)
* optimize codeBitmap

* add lru

* remove custom lru

* jumpDests as explocit parameter

* lint

* lint

* linters

* default jumpDestsCache

* remove common.rand
2020-06-06 21:49:06 +01:00
Evgeny Danilenko
8d7e603f07
stack pool (#627) 2020-06-05 17:45:56 +03:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger (#624)
* lmdb tests

* trigger ci

* fix tests

* disable parallelism

* disable parallelism

* cleanup resources

* cleanup resources

* reduce concurency

* try run tests on bolt

* try run tests on bolt

* fix downloader test

* run bolt tests

* rely on interface instead of exact instance

* Rename AbstractKV to KV

* don't use separator for badger

* don't initialize badger cursor - because it not used here

* fix linter

* try reduce badger compactors

* compat with master

* try lmdb

* try lmdb

* try lmdb

* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option

* allow to close closed db

* release

* release

* ideal batch size for badger

* ideal batch size for badger
2020-06-05 10:25:33 +01:00
Andrew Ashikhmin
33c28f7fac
[GC] uint256 rather than big.Int in Transaction (#614)
* uint256 in rlp

* uint256 rather than big.Int in Transation

* linters

* more linters

* still linters

* Reduce garbage in writeUint256

* Experiment with GC in writeByteArray
2020-06-04 08:43:08 +01:00
ledgerwatch
7ab10c85af
Separate Collector pattern out of ETL, hash collector for rebuilding Intermediate Hashes (#597)
* Introduce hashCollector

* Add HashCollector to SubTrieLoader

* Fix linter

* Reset hashed state

* Not to regenerate the hashed state

* Not to delete state

* Fix linter

* Print expected hash in the beginning

* Simplify

* Remove initialTrie

* Use etl to buffer intermediate hashes

* Copy values, not insert empty key

* Compress instead of decompress

* Enhance file buffer logging, fix linter

* Fix compile errors

* Fix log

* Fix logging

* Exclude zero key again

* Add rewind

* Restrict timestamps

* Fix

* Fix formatting

* Incorporate separation

* Extract identityLoadFunction

* Fix formatting
2020-05-31 13:23:34 +01:00
b00ris
266f9a5208
Plain state index (#595)
* plain state indexes

* generalize index generation&tests

* add plain state tests

* fix regenerate index

* сoncurrent changeset chunks processing

* remove concurrency

* fix lint

* remove comments

* add test to truncate

* fix conflicts

* fmt

* remove shadowing
2020-05-31 07:57:47 +01:00
Alex Sharov
916a1f8b83
[wip] Lmdb: AbstractKV and DB interfaces (#589)
* resetIH from scratch if needed

* lmdb

* add AbstractKV to loader, added new Object accessor around AbstractKV

* add lmdb cli flag

* add requirement of k!=nil on error in docs

* add Size method for compatibility

* read after put tests

* fix multiput nils

* simplify loops

* increase mmap size

* better error messages

* better error messages

* fix tests

* better error messages

* cleanup

* avoid bolt usage in test

* move hardcoded bucket name to dbutils

* register more buckets

* register more buckets

* fix test
2020-05-30 09:12:21 +01:00
Igor Mandrigin
ebe7aec14e
Generalize "insert into DB though sorted files" pattern. (#592) 2020-05-30 10:00:35 +03:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb (#584)
* Not hash, keep the files

* Calculate savings

* Fix

* Fix

* Fix

* Fix

* RestAPI to support local boltdb

* Not error on read-only db

* Changes so far

* Continue

* More

* Roll back a bit

* Restore newline

* something compiles

* Fix restapi

* Fix block number

* Fix reads

* Use plain writer

* Maps for storage reads and writes

* Clean up coersions

* Fix accounts/abi/bind

* Fix tests

* More fixes

* more fixes

* More fixes

* Fixes

* Fixed core/state

* Fixed eth tests

* Move code, fix linter

* Fix test

* Fix linter

* Fix linter

* Fix linter, badger_db to support AbstractKV

* Increase IdealBatchSize for badger

* Fix linter

* Fix linter
2020-05-27 17:24:34 +01:00
Andrew Ashikhmin
5bbdb6a4b9
Use uint256 for account balance (#580)
* Use uint256 for account balance

* linter
2020-05-26 17:53:50 +01:00
ledgerwatch
f06db2f37b
Stages 6 and 7 for generating history indices (#569)
* save state

* add current index feature

* fix test

* remove logs

* Only execute 1000 blocks

* Reset history index

* Correct action

* Increase batch size

* Increase chunk size, print memory stats

* Fix linter

* Remove unused from

* Split into 2 staged

* Use storage history gen

* remove log

* Not to run tx_cacher in staged mode

* Not to recover during stage 2

* Not to recover during stage 2

* Remove counter

Co-authored-by: b00ris <b00ris@mail.ru>
2020-05-23 10:19:56 +01:00
Igor Mandrigin
0dae3ade0d post-rebase fixups 2020-05-20 15:26:22 +03:00
Andrew Ashikhmin
5bc45f2426
Use uint256 in EVM implementation (kudos to Pawel Bylica) (#551)
* core/vm: improve jumpdest lookup

* Use uint256 in EVM implementation (kudos to Pawel Bylica)

* Safety precaution in opMulmod

Co-authored-by: Martin Holst Swende <martin@swende.se>
2020-05-18 08:10:59 +01:00
ledgerwatch
9f3f768ab4
Rename Resolver => SubTrieLoader, ResolveSet => RetainList (#544)
* Rename Resolver => SubTrieLoader, ResolveSet => RetainList

* Fix linter

* Fix linter

* Fix linter

* Fix linter
2020-05-12 15:24:43 +01:00
ledgerwatch
82a67f4d48
eth_getProof step2 - Decouple Resolver from Trie, remove ResolveRequest (#543)
* Removing PrepareResolveParams and NeedResolution

* Remove ResolveRequest

* Decouple resolver from trie

* Remove hooks from resolver

* Fix util compilation

* More test fixes

* Compiled tests in trie

* Fix trie tests

* Fix linter

* Fix linter

* Fix linter

* Fix linter
2020-05-12 08:22:45 +01:00
ledgerwatch
41b5fce078
resolver: merge HashBuilder-s and ResolveSet-s, decouple from ResolveRequest-s (#540)
* merge hbStorage and hb in resolver

* Fix prepare

* Remove tracing by default

* cleanup

* Fix linter

* Not to include incarnation into the ResolveRequest

* Fix linter

* Fix linter

* Fix test

* Fix linter

* Revert

* Fix for infinite loop

* Fix linter

* Collapse rss into rs

* remove reliance of requests in resolver

* Remove currentReq

* Remove fields from ResolveRequest

* Fix linter
2020-05-11 05:46:07 +01:00
Alex Sharov
2686bf2f3b
Remove topLevels param of resolver (#515)
* remove topLevels param of resolver

* revert to master version of ih
2020-05-05 21:08:00 +01:00
Alex Sharov
af4a99b977
Merge account and storage resolvers (#504)
* add_incarnation_to_acc_root_in_ih

* merge cached resolver into stateful resolver

* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash

* remove blockNR and bucket params from walker

* fix out of range panic

* calc acc.Root on the fly

* remove fieldSet field from resolver, make logic of root - lazy

* remove 2 parameters

* working version of forward-only walk over Acc and Storage

* improve test

* rebase master

* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go

* skip old incarnations

* don't rebuild when 0 requests

* fix tests

* start from account key when need resolve storage

* Error: stateless prototype faced hashNode when extracting witness

* Statless works: copy touches

* Remove getAccRoot function

* Remove "isAccount" parameter from resolver signature

* Fix: use correct storageResolveSet in finaliseStorageRoot

* Fix: when startKey changed - reset storage buffers also

* Fix: if account incarnation=0 - set EmptyRoot

* Fix: remove account roots by default from IntermediateHash bucket

* Fix: skip abandoned storage - which appeared just after startKey

* Fix: did reset acc key incorrectly

* Fix: clean previous key if receive IH

* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)

* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests

* Remove accNode from IH cycle

* Fix flags

* Fix: reset succStorage also

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: use rssStorage to HashOnly check

* Fix: remove termination symbol from resolveRequest

* cleanup

* Fix: skip abandoned storage after IH

* Debug

This reverts commit 9c5eb69465f25607d546b03359b2cbcb1bd46689.

* Fix linters

* add_incarnation_to_acc_root_in_ih

* merge cached resolver into stateful resolver

* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash

* remove blockNR and bucket params from walker

* fix out of range panic

* calc acc.Root on the fly

* remove fieldSet field from resolver, make logic of root - lazy

* remove 2 parameters

* working version of forward-only walk over Acc and Storage

* improve test

* rebase master

* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go

* skip old incarnations

* don't rebuild when 0 requests

* fix tests

* start from account key when need resolve storage

* Error: stateless prototype faced hashNode when extracting witness

* Statless works: copy touches

* Remove getAccRoot function

* Remove "isAccount" parameter from resolver signature

* Fix: use correct storageResolveSet in finaliseStorageRoot

* Fix: when startKey changed - reset storage buffers also

* Fix: if account incarnation=0 - set EmptyRoot

* Fix: remove account roots by default from IntermediateHash bucket

* Fix: skip abandoned storage - which appeared just after startKey

* Fix: did reset acc key incorrectly

* Fix: clean previous key if receive IH

* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)

* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests

* Remove accNode from IH cycle

* Fix flags

* Fix: reset succStorage also

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: skip IH if it has wrong incarnation

* Fix: use rssStorage to HashOnly check

* Fix: remove termination symbol from resolveRequest

* cleanup

* Fix: skip abandoned storage after IH

* remove inc

* remove inc from rss

* tr.succStorage.Reset()

* remove inc from rss

* Remove hard-coding

* succ.Reset

* Enable CalcTrieRoots

* Proper dumping of the trie

* Debug

* Fix for CalcTrieRoot

* Fix another inteference bug

* Temp

* Fix test

* Cleanup

* remove STORE_ACCOUNT_ROOT=true flag

* Fix linter

* Fix linter

* Disable getnodedata by default

* Fix test

* Fix test

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-05-02 19:00:57 +01:00
Igor Mandrigin
7d9bcc1e01
Fix initial state of the ExportFileBlockProvider (#500)
* Fix initial state of the `ExportFileBlockProvider`

* fixup
2020-04-29 14:17:06 +03:00
b00ris
cc137af1ce
[WIP] New storage changeset encoding (#493)
* new storage changeset encoding

* fix lint
2020-04-28 08:41:55 +01:00
Igor Mandrigin
a1facf39be
Fix broken --block in the stateless prototype. 2020-04-27 17:40:34 +03:00
ledgerwatch
0c6a44b5e5
Index bits to mark creation of accounts or storage items (#483)
* Index bits

* Fix linter

* Fix linter

* Not to call getModifiedAccountsByNumber for go-ethereum

* Remove the 0x000 kludge
2020-04-25 15:50:32 +01:00
ledgerwatch
623545a47c
Fix index chunks (#478)
* First stab at fixing index chunks

* Fix compilation

* Fix tests

* Fix walkAsOf for storage

* Fix

* Fix for walkAsOf accounts

* Fix linter
2020-04-21 09:15:40 +01:00
b00ris
a9f71298aa
Index chunking (#473)
* index generation

* save state

* save state

* save state

* save state

* save state

* save state

* save state

* save state

* fix

* save state

* fix
2020-04-20 11:35:33 +01:00
Giulio rebuffo
6e2fc0d936
[WIP] 459 merge storage account (#466)
* bt commit

* merged buckets

* adjusted cmd/

* fixed sync
2020-04-19 20:51:32 +01:00
Igor Mandrigin
8c3d19fd4c
geth 1.9.13 (#469)
* core: initial version of state snapshots

* core/state: lazy sorting, snapshot invalidation

* core/state/snapshot: extract and split cap method, cover corners

* snapshot: iteration and buffering optimizations

* core/state/snapshot: unlink snapshots from blocks, quad->linear cleanup

* 123

* core/rawdb, core/state/snapshot: runtime snapshot generation

* core/state/snapshot: fix difflayer origin-initalization after flatten

* add "to merge"

* core/state/snapshot: implement snapshot layer iteration

* core/state/snapshot: node behavioural difference on bloom content

* core: journal the snapshot inside leveldb, not a flat file

* core/state/snapshot: bloom, metrics and prefetcher fixes

* core/state/snapshot: move iterator out into its own files

* core/state/snapshot: implement iterator priority for fast direct data lookup

* core/state/snapshot: full featured account iteration

* core/state/snapshot: faster account iteration, CLI integration

* core: fix broken tests due to API changes + linter

* core/state: fix an account resurrection issue

* core/tests: test for destroy+recreate contract with storage

* squashme

* core/state/snapshot, tests: sync snap gen + snaps in consensus tests

* core/state: extend snapshotter to handle account resurrections

* core/state: fix account root hash update point

* core/state: fix resurrection state clearing and access

* core/state/snapshot: handle deleted accounts in fast iterator

* core: more blockchain tests

* core/state/snapshot: fix various iteration issues due to destruct set

* core: fix two snapshot iterator flaws, decollide snap storage prefix

* core/state/snapshot/iterator: fix two disk iterator flaws

* core/rawdb: change SnapshotStoragePrefix to avoid prefix collision with preimagePrefix

* params: begin v1.9.13 release cycle

* cmd/checkpoint-admin: add some documentation (#20697)

* go.mod: update duktape to fix sprintf warnings (#20777)

This revision of go-duktype fixes the following warning

```
duk_logging.c: In function ‘duk__logger_prototype_log_shared’:
duk_logging.c:184:64: warning: ‘Z’ directive writing 1 byte into a region of size between 0 and 9 [-Wformat-overflow=]
  184 |  sprintf((char *) date_buf, "%04d-%02d-%02dT%02d:%02d:%02d.%03dZ",
      |                                                                ^
In file included from /usr/include/stdio.h:867,
                 from duk_logging.c:5:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 25 and 85 bytes into a destination of size 32
   36 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   37 |       __bos (__s), __fmt, __va_arg_pack ());
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* core/rawdb: fix freezer table test error check

Fixes: Condition is always 'false' because 'err' is always 'nil'

* core/rawdb: improve table database (#20703)

This PR fixes issues in TableDatabase.

TableDatabase is a wrapper of underlying ethdb.Database with an additional prefix.
The prefix is applied to all entries it maintains. However when we try to retrieve entries
from it we don't handle the key properly. In theory the prefix should be truncated and
only user key is returned. But we don't do it in some cases, e.g. the iterator and batch
replayer created from it. So this PR is the fix to these issues.

* eth: when triggering a sync, check the head header TD, not block

* internal/web3ext: fix clique console apis to work on missing arguments

* rpc: dont log an error if user configures --rpcapi=rpc... (#20776)

This just prevents a false negative ERROR warning when, for some unknown
reason, a user attempts to turn on the module rpc even though it's already going
to be on.

* node, cmd/clef: report actual port used for http rpc (#20789)

* internal/ethapi: don't set sender-balance to maxuint, fixes #16999 (#20783)

Prior to this change, eth_call changed the balance of the sender account in the
EVM environment to 2^256 wei to cover the gas cost of the call execution.
We've had this behavior for a long time even though it's super confusing.

This commit sets the default call gasprice to zero instead of updating the balance,
which is better because it makes eth_call semantics less surprising. Removing
the built-in balance assignment also makes balance overrides work as expected.

* metrics: disable CPU stats (gosigar) on iOS

* cmd/devp2p: tweak DNS TTLs (#20801)

* cmd/devp2p: tweak DNS TTLs

* cmd/devp2p: bump treeNodeTTL to four weeks

* cmd/devp2p: lower route53 change limit again (#20819)

* cmd/devp2p: be very correct about route53 change splitting (#20820)

Turns out the way RDATA limits work is documented after all,
I just didn't search right. The trick to make it work is to
count UPSERTs twice.

This also adds an additional check to ensure TTL changes are
applied on existing records.

* graphql, node, rpc: fix typos in comments (#20824)

* eth: improve shutdown synchronization (#20695)

* eth: improve shutdown synchronization

Most goroutines started by eth.Ethereum didn't have any shutdown sync at
all, which lead to weird error messages when quitting the client.

This change improves the clean shutdown path by stopping all internal
components in dependency order and waiting for them to actually be
stopped before shutdown is considered done. In particular, we now stop
everything related to peers before stopping 'resident' parts such as
core.BlockChain.

* eth: rewrite sync controller

* eth: remove sync start debug message

* eth: notify chainSyncer about new peers after handshake

* eth: move downloader.Cancel call into chainSyncer

* eth: make post-sync block broadcast synchronous

* eth: add comments

* core: change blockchain stop message

* eth: change closeBloomHandler channel type

* eth/filters: fix typo on unindexedLogs function's comment (#20827)

* core: bump txpool tx max size to 128KB

* snapshotter/tests: verify snapdb post-state against trie (#20812)

* core/state/snapshot: basic trie-to-hash implementation

* tests: validate snapshot after test

* core/state/snapshot: fix review concerns

* cmd, consensus: add option to disable mmap for DAG caches/datasets (#20484)

* cmd, consensus: add option to disable mmap for DAG caches/datasets

* consensus: add benchmarks for mmap with/with lock

* cmd/clef: add newaccount command (#20782)

* cmd/clef: add newaccount command

* cmd/clef: document clef_New, update API versioning

* Update cmd/clef/intapi_changelog.md

Co-Authored-By: ligi <ligi@ligi.de>

* Update signer/core/uiapi.go

Co-Authored-By: ligi <ligi@ligi.de>

Co-authored-by: ligi <ligi@ligi.de>

* eth: add debug_accountRange API (#19645)

This new API allows reading accounts and their content by address range.

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>

* travis: allow cocoapods deploy to fail (#20833)

* metrics: improve TestTimerFunc (#20818)

The test failed due to what appears to be fluctuations in time.Sleep, which is
not the actual method under test. This change modifies it so we compare the
metered Max to the actual time instead of the desired time.

* README: update private network genesis spec with istanbul (#20841)

* add istanbul and muirGlacier to genesis states in README

* remove muirGlacier, relocate istanbul

* cmd/evm: Rework execution stats (#20792)

- Dump stats also for --bench flag.
- From memory stats only show number and size of allocations. This is what `test -bench` shows. I doubt others like number of GC runs are any useful, but can be added if requested.
- Now the mem stats are for single execution in case of --bench.

* cmd/devp2p, cmd/wnode, whisper: add missing calls to Timer.Stop (#20843)

* p2p/server: add UDP port mapping goroutine to wait group (#20846)

* accounts/abi faster unpacking of int256 (#20850)

* p2p/discv5: add missing Timer.Stop calls (#20853)

* miner/worker: add missing timer.Stop call (#20857)

* cmd/geth: fix bad genesis test (#20860)

* eth/filters: add missing Ticker.Stop call (#20862)

* eth/fetcher: add missing timer.Stop calls (#20861)

* event: add missing timer.Stop call in TestFeed (#20868)

* metrics: add missing calls to Ticker.Stop in tests (#20866)

* ethstats: add missing Ticker.Stop call (#20867)

* p2p/discv5, p2p/testing: add missing Timer.Stop calls in tests (#20869)

* core: add missing Timer.Stop call in TestLogReorgs (#20870)

* rpc: add missing timer.Stop calls in websocket tests (#20863)

* crypto/ecies: improve concatKDF (#20836)

This removes a bunch of weird code around the counter overflow check in
concatKDF and makes it actually work for different hash output sizes.

The overflow check worked as follows: concatKDF applies the hash function N
times, where N is roundup(kdLen, hashsize) / hashsize. N should not
overflow 32 bits because that would lead to a repetition in the KDF output.

A couple issues with the overflow check:

- It used the hash.BlockSize, which is wrong because the
  block size is about the input of the hash function. Luckily, all standard
  hash functions have a block size that's greater than the output size, so
  concatKDF didn't crash, it just generated too much key material.
- The check used big.Int to compare against 2^32-1.
- The calculation could still overflow before reaching the check.

The new code in concatKDF doesn't check for overflow. Instead, there is a
new check on ECIESParams which ensures that params.KeyLen is < 512. This
removes any possibility of overflow.

There are a couple of miscellaneous improvements bundled in with this
change:

- The key buffer is pre-allocated instead of appending the hash output
  to an initially empty slice.
- The code that uses concatKDF to derive keys is now shared between Encrypt
  and Decrypt.
- There was a redundant invocation of IsOnCurve in Decrypt. This is now removed
  because elliptic.Unmarshal already checks whether the input is a valid curve
  point since Go 1.5.

Co-authored-by: Felix Lange <fjl@twurst.com>

* rpc: metrics for JSON-RPC method calls (#20847)

This adds a couple of metrics for tracking the timing
and frequency of method calls:

- rpc/requests gauge counts all requests
- rpc/success gauge counts requests which return err == nil
- rpc/failure gauge counts requests which return err != nil
- rpc/duration/all timer tracks timing of all requests
- rpc/duration/<method>/<success/failure> tracks per-method timing

* mobile: use bind.NewKeyedTransactor instead of duplicating (#20888)

It's better to reuse the existing code to create a keyed transactor
than to rewrite the logic again.

* internal/ethapi: add CallArgs.ToMessage method (#20854)

ToMessage is used to convert between ethapi.CallArgs and types.Message.
It reduces the length of the DoCall method by about half by abstracting out
the conversion between the CallArgs and the Message. This should improve the
code's maintainability and reusability.

* eth, les: fix flaky tests (#20897)

* les: fix flaky test

* eth: fix flaky test

* cmd/geth: enable metrics for geth import command (#20738)

* cmd/geth: enable metrics for geth import command

* cmd/geth: enable metrics-flags for import command

* core/vm: use a callcontext struct (#20761)

* core/vm: use a callcontext struct

* core/vm: fix tests

* core/vm/runtime: benchmark

* core/vm: make intpool push inlineable, unexpose callcontext

* docs/audits: add discv5 protocol audits from LA and C53 (#20898)

* .github: change gitter reference to discord link in issue template (#20896)

* couple of fixes to docs in clef (#20900)

* p2p/discover: add initial discovery v5 implementation (#20750)This adds an implementation of the current discovery v5 spec.There is full integration with cmd/devp2p and enode.Iterator in thisversion. In theory we could enable the new protocol as a replacement ofdiscovery v4 at any time. In practice, there will likely be a few morechanges to the spec and implementation before this can happen.

* build: upgrade to golangci-lint 1.24.0 (#20901)

* accounts/scwallet: remove unnecessary uses of fmt.Sprintf

* cmd/puppeth: remove unnecessary uses of fmt.Sprintf

* p2p/discv5: remove unnecessary use of fmt.Sprintf

* whisper/mailserver: remove unnecessary uses of fmt.Sprintf

* core: goimports -w tx_pool_test.go

* eth/downloader: goimports -w downloader_test.go

* build: upgrade to golangci-lint 1.24.0

* accounts/abi/bind: Refactored topics  (#20851)

* accounts/abi/bind: refactored topics

* accounts/abi/bind: use store function to remove code duplication

* accounts/abi/bind: removed unused type defs

* accounts/abi/bind: error on tuples in topics

* Cosmetic changes to restart travis build

Co-authored-by: Guillaume Ballet <gballet@gmail.com>

* node: allow websocket and HTTP on the same port (#20810)

This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546.

    geth --rpc --rpcport 8545 --ws --wsport 8545

This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.

* crypto: improve error messages in LoadECDSA (#20718)

This improves error messages when the file is too short or too long.
Also rewrite the test for SaveECDSA because LoadECDSA has its own
test now.

Co-authored-by: Felix Lange <fjl@twurst.com>

* changed date of rpcstack.go since new file (#20904)

* accounts/abi/bind: fixed erroneous filtering of negative ints (#20865)

* accounts/abi/bind: fixed erroneous packing of negative ints

* accounts/abi/bind: added test cases for negative ints in topics

* accounts/abi/bind: fixed genIntType for go 1.12

* accounts/abi: minor  nitpick

* cmd: deprecate --testnet, use named networks instead (#20852)

* cmd/utils: make goerli the default testnet

* cmd/geth: explicitly rename testnet to ropsten

* core: explicitly rename testnet to ropsten

* params: explicitly rename testnet to ropsten

* cmd: explicitly rename testnet to ropsten

* miner: explicitly rename testnet to ropsten

* mobile: allow for returning the goerli spec

* tests: explicitly rename testnet to ropsten

* docs: update readme to reflect changes to the default testnet

* mobile: allow for configuring goerli and rinkeby nodes

* cmd/geth: revert --testnet back to ropsten and mark as legacy

* cmd/util: mark --testnet flag as deprecated

* docs: update readme to properly reflect the 3 testnets

* cmd/utils: add an explicit deprecation warning on startup

* cmd/utils: swap goerli and ropsten in usage

* cmd/geth: swap goerli and ropsten in usage

* cmd/geth: if running a known preset, log it for convenience

* docs: improve readme on usage of ropsten's testnet datadir

* cmd/utils: check if legacy `testnet` datadir exists for ropsten

* cmd/geth: check for legacy testnet path in console command

* cmd/geth: use switch statement for complex conditions in main

* cmd/geth: move known preset log statement to the very top

* cmd/utils: create new ropsten configurations in the ropsten datadir

* cmd/utils: makedatadir should check for existing testnet dir

* cmd/geth: add legacy testnet flag to the copy db command

* cmd/geth: add legacy testnet flag to the inspect command

* les, les/lespay/client: add service value statistics and API (#20837)

This PR adds service value measurement statistics to the light client. It
also adds a private API that makes these statistics accessible. A follow-up
PR will add the new server pool which uses these statistics to select
servers with good performance.

This document describes the function of the new components:
https://gist.github.com/zsfelfoldi/3c7ace895234b7b345ab4f71dab102d4

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* README: update min go version to 1.13 (#20911)

* travis, appveyor, build, Dockerfile: bump Go to 1.14.2 (#20913)

* travis, appveyor, build, Dockerfile: bump Go to 1.14.2

* travis, appveyor: force GO111MODULE=on for every build

* core/rawdb: fix data race between Retrieve and Close (#20919)

* core/rawdb: fixed data race between retrieve and close

closes https://github.com/ethereum/go-ethereum/issues/20420

* core/rawdb: use non-atomic load while holding mutex

* all: simplify and fix database iteration with prefix/start (#20808)

* core/state/snapshot: start fixing disk iterator seek

* ethdb, rawdb, leveldb, memorydb: implement iterators with prefix and start

* les, core/state/snapshot: iterator fixes

* all: remove two iterator methods

* all: rename Iteratee.NewIteratorWith -> NewIterator

* ethdb: fix review concerns

* params: update CHTs for the 1.9.13 release

* params: release Geth v1.9.13

* added some missing files

* post-rebase fixups

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: gary rong <garyrong0905@gmail.com>
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Co-authored-by: meowsbits <45600330+meowsbits@users.noreply.github.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rene <41963722+renaynay@users.noreply.github.com>
Co-authored-by: Ha ĐANG <dvietha@gmail.com>
Co-authored-by: Hanjiang Yu <42531996+de1acr0ix@users.noreply.github.com>
Co-authored-by: ligi <ligi@ligi.de>
Co-authored-by: Wenbiao Zheng <delweng@gmail.com>
Co-authored-by: Adam Schmideg <adamschmideg@users.noreply.github.com>
Co-authored-by: Jeff Wentworth <jeff@curvegrid.com>
Co-authored-by: Paweł Bylica <chfast@gmail.com>
Co-authored-by: ucwong <ucwong@126.com>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Luke Champine <luke.champine@gmail.com>
Co-authored-by: Boqin Qin <Bobbqqin@gmail.com>
Co-authored-by: William Morriss <wjmelements@gmail.com>
Co-authored-by: Guillaume Ballet <gballet@gmail.com>
Co-authored-by: Raw Pong Ghmoa <58883403+q9f@users.noreply.github.com>
Co-authored-by: Felföldi Zsolt <zsfelfoldi@gmail.com>
2020-04-19 18:31:47 +01:00
Alex Sharov
59f997a663
Move account root to ih (#465)
* move_account_root_to_ih

* temptorary comment test
2020-04-18 21:09:44 +01:00
Igor Mandrigin
33d18f376e
Fix hash mismatch at block 76328 (#467) 2020-04-18 16:12:51 +03:00