Commit Graph

1915 Commits

Author SHA1 Message Date
Felix Lange
85941edbea cmd/devp2p: fix Route53 TXT record splitting (#20626)
For longer records and subtree entries, the deployer created two
separate TXT records. This doesn't work as intended because the client
will receive the two records in arbitrary order. The fix is to encode
longer values as "string1""string2" instead of "string1", "string2".
This encoding creates a single record on AWS Route53.
2020-02-27 17:21:19 +03:00
meowsbits
9ebb2a4d2f cmd/geth: add 'dumpgenesis' command (#20191)
Adds the 'geth dumpgenesis' command, which writes the configured
genesis in JSON format to stdout. This provides a way to generate the
data (structure and content) that can then be used with the 'geth init'
command.
2020-02-27 17:20:36 +03:00
Martin Holst Swende
f3630b8949 retesteth: clean txpool on rewind, default dao support (#20596) 2020-02-27 17:20:36 +03:00
Felix Lange
cf03897b25 rpc: check module availability at startup (#20597)
Fixes #20467

Co-authored-by: meowsbits <45600330+meowsbits@users.noreply.github.com>
2020-02-27 17:20:36 +03:00
Guillaume Ballet
80fcbd3040 console, internal/jsre: use github.com/dop251/goja (#20470)
This replaces the JavaScript interpreter used by the console with goja,
which is actively maintained and a lot faster than otto. Clef still uses otto
and eth/tracers still uses duktape, so we are currently dependent on three
different JS interpreters. We're looking to replace the remaining uses of otto
soon though.
2020-02-27 17:20:36 +03:00
Guillaume Ballet
91a19add0b cmd/evm: accept --input for disasm command (#20548) 2020-02-27 17:20:36 +03:00
Martin Holst Swende
2f469b2c81 cmd/geth/retesteth: use canon head instead of keeping alternate count (#20572) 2020-02-27 17:20:36 +03:00
Martin Holst Swende
906f989762 cmd: implement abidump (#19958)
* abidump: implement abi dump command

* cmd/abidump: add license
2020-02-27 17:20:36 +03:00
Felix Lange
5f7f638ac5 log, internal/debug: delete RotatingFileHandler (#20586)
* log: delete RotatingFileHandler

We added this for the dashboard, which is gone now. The
handler never really worked well and had data race and file
handling issues.

* internal/debug: remove unused RotatingFileHandler setup code
2020-02-27 17:20:36 +03:00
Alex Sharov
fdbba5202b
Trie: store self-destructed accounts (#355)
* squash commits

* enable storage cache

* make linter happy

* fix subtree prefix len check

* save cahnges to test master

* remove restriction on prefix len

* fix comparison of last bits

* fix wrong alignment

* remove debug prints

* commit current state

* commit current state

* avoid changing state of resolver from multiwalk

* remove debug code

* remove debug code

* remove debug code

* remove unnecessary copy

* make code more readable

* reduce rebuildHashes initial resolution

* fix test after rebase to master

* make code more readable

* improve pruner

* pruner add IntermediateCache bucket

* fix panic in Walk on short keys

* reduce allocations for storage keys decompression by increasing default buffer size

* re-run CI

* fix iterator behaviour

* rename cache to hash for unification

* re-run ci

* avoid using underlying DB

* hash all subtree nodes before unload

* fix getNode method

* need to check node type, not parent - before put to hashBucket

* return back parent type check, doesn't work without it.

* don't recalculate hash again

* move unloadFunc from trie to pruner

* rename bucket to shorter name

* rename bucket to shorter name

* clean

* rebase to master
2020-02-12 13:52:59 +00:00
ledgerwatch
767a374c2e
Tx pool use triedbstate, fixes to GetNodeData and move it to experimental (#364)
* Use TrieDbState for tx pool

* Not initialise tx pool until state is loaded

* Add preimage

* Fix account

* Print codehash

* Print correct code hash

* Print incarnatin

* Print incarnatin

* Use proper incarnation

* Print dbValue

* Actually fix

* Actually fix

* Fix verifySnapshot

* readAccount to get code hash

* Next incarnation

* Print addrHashes with 0 incarnations

* Print storage history

* Print storage history

* Print storage history

* Print storage history

* Print all storage history

* print change set keys

* print change set keys

* print change set keys

* print change set keys

* Not print codebucket info

* Fixes

* Fix for incarnation

* Fix for storage history bucket

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Try to fix the leak

* Fix embedded nodes

* Hasher

* Fix

* Test fixes

* Add experimental debug flag

* Fix tx_pool_test

* Disable GetNodeData test unless in experiment

* Fix more tests

* Fix lint and revert some changes

* Fix lint

* Fix lint
2020-02-10 17:05:32 +00:00
Igor Mandrigin
aafacd04d7
Debug Web UI prototype (#362) 2020-02-09 13:31:52 +03:00
Igor Mandrigin
f4d0747949
stateless: flush the partial witnesses DB for every saved witness (#354)
* stateless: flush the partial witnesses DB for every saved witness.

it seems to fix to the out of memory error I had on blocks over 2.5M;

* fix linters

* simplify
2020-02-06 13:58:58 +03:00
Andrew Ashikhmin
8585516071
Serve GetNodeData for nodes in memory (Issue #300) 2020-02-06 11:53:09 +01:00
Evgeny Danilenko
17a4a56634
Refactor mining. Remove a few mining goroutines (#338)
* add context

* extract chain events

* run commit in goroutines

* mine only on canonical

* typo

* linters

* fmt

* mark unused methods

* restore stress test

* test single miner

* remove unsafe Trie storage

* remove locks from miner

* restore interrupt

* remove result goroutine

* remove unconfirmedBlocks

* cherry-pick 04a1d475ff1a36ad8f92fec80385df18c52bdc1f

* extract uncles

* one miner succeeded

* restore context cancel

* cleanup

* skip an unstable test

* remove pending state

* use context instead of interrupt func

* calculate sealHash only once

* comment out unstable test

* after merge

* fix after merge

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2020-02-03 15:02:26 +03:00
Alex Sharov
0dd4386a37
Intermediate trie hashes phase 1 (#334)
* add intermediate trie hashes bucket
* update docs
* use version of CompressNibbles with buf pool (less GC) but without io.ByteWriter (slow)
2020-01-31 11:11:20 +07:00
ledgerwatch
dca36e8b29
Restore the functionality CalcTrieRoots (compute trie root without modifying the trie) (#327)
* Trace first block

* Fixes for CalcTrieRoots

* Timings of the CalcTrieRoot

* Fix lint

* Add memory profiling

* Reduce allocations in StreamHash

* Fix

* Fix

* Fix

* Optimise allocations

* Reuse streams

* Fixes

* Fix

* Unit test fix

* Fix lint

* Reuse hashbuilder

* No loop

* Reuse resolver

* Fixed tests

* Fix test

* Fix test

* Fix test

* Fix witness threshold

* Optimise allocations in RLP transform

* Optimise allocations in RLP transform

* Optimise branchHash

* 100 times again

* Replace ToStream with Iterator

* StreamMergeIterator

* No streams

* Amplification

* Minimise the use of hashOnly

* 100 times

* Reduce stack operations

* Reduce appends

* More optimisations

* More optimisations

* More optimisations

* local hex

* Small fix

* Remove test

* Fix lint

* Fix lint

* Fix lint

* Add test for empty

* Fix lint

* More tests

* Fix lint

* Add measurement of stateless exec
2020-01-30 13:16:12 +00:00
Igor Mandrigin
785d0eafd8 post-rebase fixups v1.9.10 2020-01-30 13:36:30 +02:00
Felix Lange
3558ce7af4 cmd/devp2p: submit Route53 changes in batches (#20524)
This change works around the 32k RDATA character limit per change
request and fixes several issues in the deployer which prevented it from
working for our production trees.
2020-01-30 13:36:30 +02:00
Yole
2b25a7951b cmd/geth: update copyright year (#20512)
Update copyright from 2013-2019 to 2013-2020
2020-01-30 13:36:30 +02:00
Kumar Anirudha
d17526e43d cmd/puppeth: change dashboard title to not use "testnet" (#20513) 2020-01-30 13:36:30 +02:00
Guillaume Ballet
802b0ac7ff cmd/geth: fix forked exe leak in console tests (#20480) 2020-01-30 13:36:30 +02:00
Paweł Bylica
0ec69b46f0 cmd/evm: Add --bench flag for benchmarking (#20330)
The --bench flag uses the testing.B to execute the EVM bytecode many times and get the average exeuction time out of it.
2020-01-30 13:36:30 +02:00
Guillaume Ballet
8806691dda cmd/abigen: Sanitize vyper's combined json names (#20419)
* cmd/abigen: Sanitize vyper's combined json names

* Review feedback: handle full paths
2020-01-30 13:36:30 +02:00
Felix Lange
3e10e5be5f cmd/devp2p: implement AWS Route53 enrtree deployer (#20446) 2020-01-30 13:36:30 +02:00
Felix Lange
c8d82bfc5c p2p/dnsdisc: add enode.Iterator API (#20437)
* p2p/dnsdisc: add support for enode.Iterator

This changes the dnsdisc.Client API to support the enode.Iterator
interface.

* p2p/dnsdisc: rate-limit DNS requests

* p2p/dnsdisc: preserve linked trees across root updates

This improves the way links are handled when the link root changes.
Previously, sync would simply remove all links from the current tree and
garbage-collect all unreachable trees before syncing the new list of
links.

This behavior isn't great in certain cases: Consider a structure where
trees A, B, and C reference each other and D links to A. If D's link
root changed, the sync code would first remove trees A, B and C, only to
re-sync them later when the link to A was found again.

The fix for this problem is to track the current set of links in each
clientTree and removing old links only AFTER all links are synced.

* p2p/dnsdisc: deflake iterator test

* cmd/devp2p: adapt dnsClient to new p2p/dnsdisc API

* p2p/dnsdisc: tiny comment fix
2020-01-30 13:36:30 +02:00
Igor Mandrigin
1115b23832
Stateless Resolver: store and use witnesses for resolved subtries (#326) 2020-01-24 11:58:01 +01:00
ledgerwatch
4f5c858f00
Collect data about Keccak256 invocations for evaluation of STARK proof sizes and performance (#315)
* Gathering start data on witnesses

* Fix number parsing

* Proper filename, actually outputting

* Correct block

* Visualise states

* Code and value fix

* Visualise code hash and storage hash

* Current block number

* Print bucket

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix typo

* Fixes after merging master

* Fix lint

Co-authored-by: b00ris <b00ris@mail.ru>
2020-01-15 17:33:36 +00:00
b00ris
df8798b94b
Thin history for storage (#324)
* save state

* save state

* fmt

* add check to test

* move incarnation length to common

* remove line

* fix lint
2020-01-15 17:55:43 +03:00
ledgerwatch
571742cf94
Ongoing changes in the db_walkthrough (#301)
* Fixes in the db walkthrough

* Fix in picture generation

* Fix image

* more edits

* More edits
2020-01-15 13:59:08 +00:00
Igor Mandrigin
b7710ff7fa
New Witness Format (and remove the rest of the tapes) (#317) 2020-01-15 14:56:50 +01:00
Alex Sharov
4a3abcea11
Make ComputeTxEnv public and reuse it in Retesteth and RpcDaemon (#309)
* Make ComputeTxEnv public and reuse it in Retesteth and RpcDaemon
2020-01-15 20:13:47 +07:00
Alex Sharov
5ab8345114
Retesteth: storageRangeAt compatibility (#310)
* fix callDataCopyOffset test
2020-01-15 19:56:25 +07:00
Alex Sharov
fc7291ec34
Use local db in analytics (#308)
* Use local db in analytics (#308)
* pool of encoders
* incr and decr funcs
2020-01-15 19:47:13 +07:00
Alex Sharov
438476d1ba
Serialization bench (#288)
* bench shows that need to pass pointer to []byte when use encoder.Encode()
2020-01-08 13:09:07 +07:00
Alex Sharov
879ca6d364
rpcdaemon - simple lrucache (#290) 2020-01-07 17:52:20 +07:00
Giulio rebuffo
bcf04bbbcf Change "for storage" encoding of Account from RLP to bitmask + fields encoding (#287)
* migration tool + CBOR encoding for storage of account bucket

* documentation updated

* fixed

* removed noncontract in EncodeForStorage

* code adjustments

* updated tests

* switched to custom encoding

* documentation updated
2020-01-07 09:02:44 +00:00
Alex Sharov
33308dd5c2
State analytic less memory and checkpoints (#299)
* added prefix tree to analyses to reduce memory usage
* make new partition every day
* merge concepts of reporter and snapshot
* tests for .FirstKey() and .NextKey()
2020-01-07 09:27:19 +07:00
Alex Sharov
85009dabe6
return error from .First() and .Next() methods of RemoteDb (#292)
* return error from .First() and .Next() methods of RemoteDb
* re-run CI
2020-01-06 18:38:21 +07:00
ledgerwatch
dbc37b5660
bolt vs badger log parser and chart (#284) 2019-12-24 13:26:29 +00:00
Alex Sharov
805f9aa501
benchmark rpc daemon
* tracing, write bench1() result into files

* add context to computeTxEnv, remote_db cursor lazy cache allocation,

* more results

* added

* added

* added

* more info

* make linter happy

* exit from tracing gorutines

* remove unused param

* re-run CI

* re-run CI

* remove concept txHandle

* batch cursor in reports

* remove debug output

* fix shadow variables and unhandled errors
2019-12-22 21:10:46 +06:00
Evgeny Danilenko
9d5949a0b9 Retesteth restore mining (#278)
* initial commit

* add callers

* restore mining for retesteth

* unlock even if panics

* try to restore accountRange

* fmt

* rename MineBlock

* fmt

* linters
2019-12-19 13:44:43 +03:00
alex.sharov
31889869d9 better functions naming 2019-12-16 20:55:10 +06:00
alex.sharov
e6b33b376f make ci happy 2019-12-16 20:55:10 +06:00
alex.sharov
f57015c4ea re-run ci 2019-12-16 20:55:10 +06:00
alex.sharov
88b05bd162 make linter happy 2019-12-16 20:54:30 +06:00
alex.sharov
d7a3c031f9 remove unused field 2019-12-16 20:34:51 +06:00
alex.sharov
4fa90b9856 make linter happy 2019-12-16 20:27:49 +06:00
alex.sharov
54d24190b9 compute state to exact transaction precision (but only for mainnet) 2019-12-13 09:56:30 +06:00
alex.sharov
73b50e080d debug_StorageRangeAt v0, got wrong response on request: {"jsonrpc":"2.0","method":"debug_storageRangeAt","params":["0x2bf07c790737be3bc4c57cbf3dedb231806f6bfef434657d59dcc9ddbe4665ab", 1,"0x8b3b3b624c3c0397d3da8fd861512393d51dcbac","0xfade75560a6cfb895f5dc7c4ab3fa10089ac2372c98aa78280d029ab36285ad6",1024],"id":1377} 2019-12-13 09:56:30 +06:00