Commit Graph

220 Commits

Author SHA1 Message Date
Alex Sharov
eb7d2ad16a
Add validation of senders amount (#1539) 2021-03-04 16:34:49 +07:00
ledgerwatch
c00f85ab6c
Simplification of eth_estimageGas (#1531)
* Simplification of eth_estimageGas

* Set GasCap by default

* Add a test

* Fix crash

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-02 22:47:44 +00:00
Evgeny Danilenko
e4e36c152e
Extract validating interface (#1120)
* interface

* generalize interface

* linters

* fix deadlock

* fix linters

* close goroutine

* fix

* debug

* id+ttl

* refactor downloader tests

* tests

* lru

* handle genesis, extract fake consensuses

* fix fake consensus

* test uncles, verify

* after a new master

* fmt

* fix close

* debug

* debug

* fix chain length

* remove test field

* use single account

* fix data race on closing channel

* remove postponed blocks queue

* miner test

* VerifyHeaderRequests

* fmt

* fmt

* fix data race

* handle validating errors

* simplify matchParents

* remove copy-paste

* move sort to constructor

* clean up

* debug for 10 parents

* debug

* debug

* batch responses

* batch requests

* works for many ancestors

* remove debug

* always Close an engine

* linters

* ancestors deduplication

* fix test

* reduce interface

* api

* clique

* green clique sync

* stable

* perpermance per second

* full sync

* linters

* gitignore

* deps

* fix panic after master merge

* init consensus

* clique tests

* fix tests

* fix tests

* clean up

* reuse snap

* store vefified snapshots

* optimize snapshots

* safe close

* cleanup loop

* after downloader

* downloader and consensus tests

* update tests

* hack.go

* clique flags

* fix cliuqe config

* review

* gitignore

* remove additional bucket

* blk/sec instead of blk/microsecond

* rename

* deps

* optimize

* debug

* test

* tests without extracted validation process

* same base performance as on master

* benchmark

* simplify ethash verification

* ethash

* ethash

* linters

* ethash

* master stats

* cleanup

* gomod

* linters

* tests

* better locks

* Fix

* Remove logging for verifyHeaders

* Verification speed in the logs

* Fix compile error

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-25 19:40:45 +00:00
ledgerwatch
dc6b4c0e43
Fixes for trace_callMany (#1506)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-21 20:18:59 +00:00
ledgerwatch
e8501bbf43
eth_getTransactionReceipt to return nil for transactions not in the d… (#1505)
* eth_getTransactionReceipt to return nil for transactions not in the database

* Fix compile error
2021-02-21 09:38:00 +01:00
ledgerwatch
7c81e9198d
Fixes to trace_call, trace_callMany, dual RPC daemon mode (#1491)
* Fixes to trace_call, trace_callMany, dual RPC daemon mode

* Fix compile error

* Fix compile error

* Compile error fix

* Compile error fix

* Fix typo

* Fix discrepancy when trace is empty

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-12 16:47:32 +00:00
ledgerwatch
bb2cc5a10e
debug_accountAt (#1487)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-10 20:43:13 +00:00
ledgerwatch
2daa71e6cb
Support for trace_callMany (#1486)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-10 17:04:47 +00:00
Artem Vorotnikov
5c34713e71
Remove nested transactions (#1483) 2021-02-10 17:04:22 +00:00
ledgerwatch
7c8bae8591
Implementation of debug_traceCall, add stateDiff to trace_call (#1442)
* Implementation of debug_traceCall

* Fix compile error

* Implementing stateDiff in trace_call - WIP

* fixes to stateDiff, better parsing in rpctest

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-18 11:13:19 +00:00
Alex Sharov
7fd1f19435
txdb lazy cursors creation, grpc version up (#1440)
* txdb lazy, grpc version up

* tidy

* fix lint
2021-01-15 09:38:09 +00:00
b00ris
8db5790838
State snapshot sync (#1417)
* move experiments to new branch&reorganise kv_snapshot

* walk&modify tests

* added delete from snapshot tests

* fmt

* state snapshot debug

* snapshot validation passed. copy state snapshot

* debug

* snapshot cursor.Prev test

* Prev works correct. Added Current check

* add err check

* added walk forward and backward test

* before refactoring

* refactoring

* execution with snapshot debug

* fix

* remove useless test

* before dupcursor implimentation

* tests with prev and delete works

* execution based on state snapshot passed

* remove useless tests

* blocks to 1140000 passed

* clean verifier

* cleanup state generation

* clean verify && seeder

* remove debug code

* tests passed

* fix lint

* save state

* test passed

* fix lint

* add state hash

* fix lint
2021-01-02 19:28:37 +00:00
Alex Sharov
75cb938980
rpcdaemon performance improve - less reading blocks, less allocs on server (#1426)
* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* don't use global variables

* df

* avoid use non-transactional db instance to reduce amount of cursors
2021-01-02 19:28:22 +00:00
Artem Vorotnikov
8c0ecaeea9
Remove StageData (#1419) 2020-12-21 11:01:37 +00:00
Thomas Jay Rush
76e244f24f
Adding specific Postman testing for tracing (#1402)
* Adding specific Postman testing for tracing

* Updating tests
2020-12-14 11:28:03 +00:00
ledgerwatch
0953fd42cb
WIP trace_call implementation (#1399)
* trace_call initial

* Fix tests

* More tracing

* Add more fields to the action

* Completed first example query

* Add initial bench11 to compare trace_call with OpenEthereum

* Exclude precompile calls from call traces

* Add self-destruct, call types, more comparison in rpctest

* Support for execution errors

* Stack underflow error and delegatecall value

* Fix lint

* Fix suicide traceAddress, Bad instruction error

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-14 11:27:52 +00:00
ledgerwatch
2cc36dac87
further cleanup, unit tests for RPC daemon and tracing improvements (#1392)
* Cleanup and rpcdaemon unit tests

* Fix

* Fix

* Fix lint

* Test for debug_traceTransaction

* Add NoRefunds option

* Compile fix, test for no refunds

* Fix compile

* Add poly contract, fix compile errors

* No refunds now work

* Fix NPE in rpcdaemon

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-09 18:24:08 +00:00
ledgerwatch
5ea590c18e
State cache switching writes to reads during commit (#1368)
* State cache init

* More code

* Fix lint

* More tests

* More tests

* More tests

* Fix test

* Transformations

* remove writeQueue, before fixing the tests

* Fix tests

* Add more tests, incarnation to the code items

* Fix lint

* Fix lint

* Remove shards prototype, add incarnation to the state reader code

* Clean up and replace cache in call_traces stage

* fix flaky test

* Save changes

* Readers to use addrHash, writes - addresses

* Fix lint

* Fix lint

* More accurate tracking of size

* Optimise for smaller write batches

* Attempt to integrate state cache into Execution stage

* cacheSize to default flags

* Print correct cache sizes and batch sizes

* cacheSize in the integration

* Fix tests

* Fix lint

* Remove print

* Fix exec stage

* Fix test

* Refresh sequence on write

* No double increment

* heap.Remove

* Try to fix alignment

* Refactoring, adding hashItems

* More changes

* Fix compile errors

* Fix lint

* Wrapping cached reader

* Wrap writer into cached writer

* Turn state cache off by default

* Fix plain state writer

* Fix for code/storage mixup

* Fix tests

* Fix clique test

* Better fix for the tests

* Add test and fix some more

* Fix compile error|

* More functions

* Fixes

* Fix for the tests

* sepatate DeletedFlag and AbsentFlag

* Minor fixes

* Test refactoring

* More changes

* Fix some tests

* More test fixes

* More test fixes

* Fix lint

* Move blockchain_test to be able to use stagedsync

* More fixes

* Fixes and cleanup

* Fix tests in turbo/stages

* Fix lint

* Fix lint

* Intemediate

* Fix tests

* Intemediate

* More fixes

* Compilation fixes

* More fixes

* Fix compile errors

* More test fixes

* More fixes

* More test fixes

* Fix compile error

* Fixes

* Fix

* Fix

* More fixes

* Fixes

* More fixes and cleanup

* Further fix

* Check gas used and bloom with header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-08 09:44:29 +00:00
Alex Sharov
98b36126b2
History bitmap 64 (#1374)
* squash

* squash

* squash

* squash

* squash

* squash

* subscription_doesnt_preserve_shutdown
2020-12-04 21:16:51 +00:00
Alex Sharov
27a24746b7
remove GetLogsByHash (#1390) 2020-12-04 10:27:24 +00:00
Thomas Jay Rush
362e9a1773
Update docs testing (#1385)
* Remove unused files

* Updating testing definition

* Updating auto-generated docs

* Update go code to make agree with silkrpc testing
2020-12-04 10:24:49 +00:00
ledgerwatch
9a233da93d
Add test for getChainConfig (#1375)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-28 15:08:17 +00:00
Alex Sharov
068463dff4
Store transactions individually (#1358)
* Store transactions individually

* Store transactions individually

* save progress

* checkIndex

* merge
2020-11-22 21:25:26 +00:00
Chase Wright
a8b7a59977
Update rpcdaemon README with eth_(un)subscribe (#1365)
* `eth_subscribe` is supported over WebSocket
* Only `newHeads` is supported at this time
* `eth_unsubscribe is supported
2020-11-19 15:39:13 +01:00
ledgerwatch
1a0e78a2e2
Remove changeset-based decorators from WalkAsOf (#1356)
* Remove decorators

* Current changes

* Fix and skip some tests for now

* Fix lint

* Specialise WalkAsOf

* Fix compile errors

* Fix lint

* Scan index files for key sizes

* More cleanup

* Fix to remote server

* Fix for walkasofstorage

* Fix/skip tests

* Fix tests

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-18 14:58:52 +00:00
Igor Mandrigin
393c9965ae
rpcdaemon: subscriptions, newHeads (#1359)
* fix `make grpc` on new checkouts

* update proto files

* add some stub

* prototype with fake events

* notifying about events

* pass events

* events are being sent

* transfer headers to filters

* create the “filters” struct

* implement new heads

* PoC of New Heads subscription

* fix keep alive

* fixups for the client

* add “type” to the event

* support header event type on client

* better stage refactor

* fixup for the eth backend

* fixups

* fix tests

* fix tests

* fix linters

* address comments

* remove unused log
2020-11-17 19:13:41 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342)
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-11-16 12:08:28 +00:00
b00ris
6464da7670
Remote snapshot downloader (#1343)
* save state

* save state

* save state

* refactoring

* fix

* save state

* save state

* fmt

* fix lint

* restore torrents for external downloader

* fix lint

* download

* skip debug test

* debug

* remote debug

* small cli fixes

* skip debug test

* external snapshot predownloader

* get rid of remote downloader

* fix lint

* clean makefile

* fix lint

* fix lint

* cleanup

* fix ci

* fmt

* remove proto from interfaces

* Squashed 'interfaces/' content from commit acd02bb94

git-subtree-dir: interfaces
git-subtree-split: acd02bb94c5a421aa8f8d1fd76cd8aad668e9fcb
2020-11-13 16:16:47 +00:00
Igor Mandrigin
ed9672620b
Granular rpc control (Allow list for RPC daemon) (#1341) 2020-11-10 10:08:42 +01:00
Thomas Jay Rush
546b91f47e
Updating RPC tests in Postman (#1340) 2020-11-09 09:52:18 +01:00
Igor Mandrigin
c0f9b57b42
rpcdaemon: read chainConfig from the DB (remove hardcode) (#1339)
* read chainConfig from the db

* get dbtx properly
2020-11-08 06:46:53 +01:00
Igor Mandrigin
ab878ed59a
Ethstats (#1338) 2020-11-06 08:59:50 +01:00
b00ris
d2d5da8b09
add snapshot support for rpcdaemon (#1304) 2020-10-27 22:31:47 +00:00
Evgeny Danilenko
b6687fc9aa
initial (#1296) 2020-10-25 21:34:00 +00:00
Alex Sharov
6f27e3043b
lmdb_readonly_dont_create_folder (#1293) 2020-10-25 08:39:09 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271)
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-10-25 08:38:55 +00:00
Ricardo Guilherme Schmidt
e9572ae981
Fix infinite rpc.gascap on eth_estimateGas (#1292)
* Fix infinite rpc.gascap on eth_estimateGas

* use `gasCap.Sign() > 0` instead of `gasCap.Uint64() != 0`
2020-10-25 08:19:59 +00:00
Thomas Jay Rush
1af3e09d75
1209 automate documentation (#1290)
* Automate documentation

* Updating

* Updating

* Fixing lint

* Updating readthedocs documentation

* Updating testing

* Updating testing
2020-10-24 18:03:52 +01:00
Alex Sharov
e85b388061
Protect from wrong class use: make ethdb.Tx incompatible with rawdb.DatabaseReader (#1288)
* protect_from_wrong_class_use

* more cases
2020-10-24 09:09:20 +01:00
Alex Sharov
77a4b3305d
rawdb to return errors: part 1 (#1286)
* ReadBlockByNumber and ReadChainConfig return error

* Let's make rawdb package to return error. But by small steps.
High-level methods need to move first:

ReadBlockByNumber
ReadChainConfig
ReadBlockByHash
ReadHeaderByHash
WriteChainConfig
WriteDatabaseVersion
WriteBlock
DeleteTd
WriteTd
ReadTd
2020-10-24 07:57:09 +01:00
Alex Sharov
6584b43b12
Allow start read transactions from ethdb.Database interface (#1287)
* allow_read_transactions

* fix lint

* fix lint

* more cases
2020-10-24 07:55:43 +01:00
Alex Sharov
5e53c79f50
stable version of grpc codegen, drop kv_remote_v1 support (#1285) 2020-10-24 07:54:03 +01:00
Thomas Jay Rush
2aa6fb50ca
Cleans up some custom RPC commands (#1270)
* Moving a few files to be more consistent

* Collecting together custom code under tg_ namespace

* Small cleanups

* Updating test cases

* Uncomment eth_gasPrice but still unimplemented
2020-10-20 22:16:28 +01:00
Alex Sharov
5dbe1724be
Bitmap etl (#1227)
* save progress

* save progress

* save progress

* save progress

* etl bimaps

* etl bimaps

* etl bimaps

* etl bimaps

* a

* a

* a

* a

* a

* a

* a

* save progress

* save progress

* save progress

* a

* fix_race_condition_on_zstd_build

* clean

* clean

* clean

* clean

* clean

* go mod tidy

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* fix corner case

* commit every

* call traces etl

* call traces etl

* call traces etl

* return missed error

* go mod tidy

* go mod tidy

* go mod tidy

* rebase to master

* remove trash

* print full key if it has 8 leading zeroes
2020-10-19 14:11:01 +01:00
Evgeny Danilenko
3bec75cdc4
Extract nonstandard APIs and introduce tg namespace (#1248)
* initial

* update rpc deamon readme

* use read-only transactions
2020-10-18 20:44:28 +01:00
Evgeny Danilenko
946d4fac5b
ForkID api (#1244)
* forkID api

* merge fork calls

* optional param
2020-10-16 15:15:10 +03:00
Thomas Jay Rush
c6c82b7569
Finish moving functions to consistent locations to ease testing/documentation of RPC (#1241)
* Cleaning up comments to start

* Renaming get_receipts.go to eth_receipts.go

* Stubbing out RPC mining routines

* Move block-related commands to eth_block.go

* Move transaction-related commands to eth_txs.go

* Move account-related commands to eth_accounts.go

* Move system-related commands to eth_system.go

* Final cleanup

* Updating testing
2020-10-14 16:59:42 +01:00
Thomas Jay Rush
dcdcc1a30f
Fixes issue #1232 (#1233)
* Fixes issue #1232

* Adding/updating test cases

* Updating as per comments

* Use tx instead of db in GetModifiedAccounts and clean up a bit

* option to specify LMDB map size for cmd/integration commands (#1238)

* Add lmdb.mapSize to integration

* Make lmdb.mapSize effective

* Use mapSize for first opening

* Fix for cobra

* Extend lmdb.mapSize for all commands

* Fix lint

* Fixes issue #1232

* Adding/updating test cases

* Updating as per comments

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2020-10-14 12:01:44 +01:00
Thomas Jay Rush
d6587ba250
Simple file renaming to make automated testing easier (#1230)
* Simple file renaming to make automated testing easier

* Fixing comment
2020-10-12 15:17:34 +01:00
Thomas Jay Rush
1bd46089e9
Start of re-working of RPC code to align with testing better (#1225)
* Starting process of rearranging RPC code

* First version of automated testing

* Putting RPC docs in their own folder

* Fixing build

* Making varible names more consistent

* Adding deprecated RPC commands for db_

* Adding deprecated RPC commands for shh_

* Adding deprecated RPC commands for eth_
2020-10-12 09:39:33 +01:00
ledgerwatch
b747ab5324
[WIP] CallTraces index (#1157)
* Initial commit for CallTraces index

* Fix compilation

* fix lint, add comment

* Fix integration

* Add Close function to ethdb.Cursor, fix some compile errors

* Try to stop cursor leak in Get

* Fix compile errors in RPC daemon

* Fix compile errors

* fixing another way

* Some fixes

* More fixes

* More fixes

* More fixes

* Fixes to core/state

* Fix lint

* Fix lint

* Fixes

* Stage caching for call trace stage

* Add mem stats

* Try to stop the leak

* Turn off debug

* Chunks for 10k blocks

* Print

* Revert "Print"

This reverts commit 5ffada4828d61e00e5dad1ca12c98258dfbbad00.

* Revert "Chunks for 10k blocks"

This reverts commit cfb9d498e782e5583d41c30abf0e2137da27383e.

* Trying to fix the leak

* Don't compute receipts in re-tracing

* Not compose block

* Print speed, fix receipts, bigger caches

* Fix lint

* Utilise changeset info

* Counters

* Use NoReceipts and ReadOnly

* ReadOnly is incompatible with caching

* Skip test leaking transactions

* Fix block test

* Change disable message for call-traces stage

* Use block option for call traces integration

* Fix retracing due to incarnation
2020-10-12 09:39:04 +01:00
Alex Sharov
fc3cd4d5c3
Rpcdaemon: private api V2 (#1214)
* save progress

* GracefulShutdown grpc server, send to server close signal before canceling context

* GracefulShutdown json server

* GracefulShutdown json server

* fix lint

* clean

* hack hugeFreelist

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* save progress

* fix_race_condition_on_zstd_build

* fix_race_condition_on_zstd_build

* better close cursor

* save progress

* open read tx in all api methods

* clean

* clean
2020-10-10 13:24:56 +01:00
Alex Sharov
9cf10a8f85
Rpcdaemon: Gracefull shutdown, better grpc stream close (#1196)
* save progress

* GracefulShutdown grpc server, send to server close signal before canceling context

* GracefulShutdown grpc server, send to server close signal before canceling context

* GracefulShutdown json server

* GracefulShutdown json server

* fix lint

* fix lint

* can

* clean

* test

* test

* test

* increase keepalive timeout, increase vegeta timeout, mark server-side errors

* increase keepalive timeout, increase vegeta timeout, mark server-side errors

* hack hugeFreelist

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* up streams limit

* fix_race_condition_on_zstd_build
2020-10-10 07:06:54 +01:00
Alex Sharov
88a099702b
rawdb.ReadCanonicalHash to return err (#1204)
* read_canonical_hash_return_err

* hack hugeFreelist

* fix tests
2020-10-10 07:05:56 +01:00
ledgerwatch
d912c9929e
No prefetch on remote cursors in RPC daemon (#1201) 2020-10-07 21:50:03 +01:00
Alex Sharov
85219b651e
Revert "rpcdaemon to cache chainconfig (#1173)" (#1177)
This reverts commit cc4dd3695d.
2020-10-03 15:07:49 +07:00
Nikola
ef7c837f2c
Update rpcdaemon README (#1175) 2020-10-02 15:03:53 +02:00
Alex Sharov
cc4dd3695d
rpcdaemon to cache chainconfig (#1173)
* rpcdaemon to cache chainconfig

* topic normalization

* topic normalization
2020-10-02 13:52:02 +01:00
Alex Sharov
6ccf57dace
switch to go implementation of roaring bitmaps for alpine support (#1165) 2020-10-02 15:16:21 +07:00
Alex Sharov
48cf5fb43e
History-style sharding for bitmap indices (#1159) 2020-10-02 10:54:11 +07:00
Thomas Jay Rush
5ebf997160
Fixes to a few tracing issues (#1161)
* Starting to look at performance

* Fixes issue #1142 - incorrect error strings

* Uses a recent fix in Geth callTracer code to return balance to self-destruct traces
2020-10-01 07:02:11 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124)
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-28 18:18:36 +01:00
Thomas Jay Rush
784bc9dc85 Fixes issue #1139 - crash when running chaindata mode (#1140)
* Fixes issue #1139 - crash when running chaindata mode

* Cleaning up error messages as per feedback
2020-09-27 12:11:05 -04:00
Thomas Jay Rush
5b4445ecea Revert "Fixes issue #1139 - crash when running chaindata mode (#1140)"
This reverts commit 00415eee76.
2020-09-27 12:06:37 -04:00
Thomas Jay Rush
00415eee76
Fixes issue #1139 - crash when running chaindata mode (#1140)
* Fixes issue #1139 - crash when running chaindata mode

* Cleaning up error messages as per feedback
2020-09-26 22:01:11 +01:00
ledgerwatch
4be6174a15
Add comment about ContractAddress in receipts, followed by some simplification (#1137)
* Add comment about receipts

* Panic on error

* Ignore errors in the simulated backend

* Fix lint
2020-09-26 13:38:10 +01:00
Thomas Jay Rush
c0a03ad728
Responses to comments on previous tracing PR (#1138)
* Responses to comments on previous tracing PR

* Cleanup

* Cleanup
2020-09-26 07:41:34 +01:00
Thomas Jay Rush
b437fab006
[WIP] Tracing enhancements (#1133)
* Adding some testdata for RPC tracing

* Adding command line option to allow switching trace export format

* Moved types into a separate file for clarity

* Added endpoints and stub functions for other trace routines

* Added experimental end points for issuance, blockReward, and uncleReward

* Moved supporting functions for filtering to _filtering file

* Moved Filter function to _filtering file

* Reordering functions for clarity

* Cleaning up

* Implemented trace_get - 7 of 8 tests pass

* Implemented trace_transaction - 2 of 8 tests pass

* Updating test cases

* Turning off lint temporarily. Will be turned back on

* Extended trace_filter (parity traces, reward traces) - 10 of 13 tests pass

* Implemented trace_block - 4 of 6 tests passed

* Cleanup test cases

* Added refundAddress and selfDestructedAddress to self destruct traces

* Total hack fix to missing gasUsed in some tests. Fixes tests, needs to be replaced
2020-09-25 13:12:36 +01:00
ledgerwatch
28e2b45f5e
Return zero balance for non-existent accounts instead of crashing (#1135) 2020-09-23 20:18:23 +02:00
Giulio rebuffo
d33302c2be
Mutual TLS authentication (for now without verifying the Common Name) (#1113)
* added tls auth

* added client side

* put --tls

* fixed flag

* Add key/cert generation instructions, turn off Common Name verification

* Add CLI arguments and Warning

* Lint

* Update the doc about Internal IP

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-19 15:16:04 +01:00
ledgerwatch
64e1a8fbd4
Fix chaindata option in RPCdaemon (#1123) 2020-09-18 11:23:35 +01:00
Alex Sharov
a112183ceb
fix logidx (#1122)
* fix logidx

* derive fields already dose set index

* start from zero
2020-09-17 13:19:11 +02:00
Thomas Jay Rush
f596a631af
1075 RPC additions and cleanup of README (#1117)
* Rewrite of README for clarity and expanding implementation table

* Small cleanups for web3_api.go

* Changed GetReceipts function to non-export (getRecipts) and cleaned up comments

* Added a comment for GetLogsByHash

* Add support for eth_protocolVersion and eth_chainId

* Adding comments and re-ordering fields to agree with Parity's ordering

* Added support for eth_listening. Moved net_version to same file as others

* Setup for adding eth_gasPrice
2020-09-15 17:44:28 +01:00
Thomas Jay Rush
e5f8073d75
1084 various rpc fixes (#1111)
* Fixes issue #1110 - eth_getStorageAt returning inconsistent values

* Adds support for various eth_getUncle calls

* Adding a couple of comments

* Adding support for eth_getTransactionCount

* Cleaning up README's

* Cleaning up README for rpcdaemon

Co-authored-by: tjayrush <jrush@greathill.com>
2020-09-14 07:59:07 +01:00
Thomas Jay Rush
065e658f42
Corrects incorrect return from eth_getCode for non-contract accounts (#1109) 2020-09-13 07:06:39 +01:00
Giulio rebuffo
23d89c0941
Added TLS handshake to RPCDaemon (#1089)
* added eth_getStorageAt

* used uint32

* now its 256

* incarnation

* added TLS handshake

* code minified

* lint

* minified client

* Update flags.go

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-11 21:17:37 +01:00
Thomas Jay Rush
fdb82d87f1
1070 eth get code (#1100)
* Fixing issue #1070 Add eth_getCode

* Adding eth_getCode to README
2020-09-11 20:20:00 +01:00
Thomas Jay Rush
3f18b9eaa3
1074 web3 client version (#1094)
* Removed old TODOs

* Adding support for web3_clientVersion and web3_sha3

* Fixing formatting

* Adding new endpoints to README

* Responding to comments by removing date from compile-time values
2020-09-11 14:12:38 +01:00
Alex Sharov
84b52a5547
rpc daemon: log request params on debug verbosity (#1087)
* rpc_daemon_access_log

* rpc_daemon_access_log

* clean

* remove response

* remove accesslog concept

* clean
2020-09-09 21:21:19 +01:00
Alex Sharov
fd9c7c3b2d
kill bolt (#1085)
* kill bolt

* kill bolt

* fix test

* clean
2020-09-09 11:09:55 +01:00
Thomas Jay Rush
7f1e7d5212
1073 eth get transaction by block number and index (#1083)
* Added missing README entry for getTransactionByHash

* Adding support for eth_getTransactioBy* RPC routines

* Added missing README entry for getTransactionByHash

* Adding support for eth_getTransactioBy* RPC routines

* Copying code from ./internal/ethapi/api.go with modifications
2020-09-08 20:39:31 +01:00
Giulio rebuffo
c590fa476e
Eth get storage at (32 => 256) (#1065)
* added eth_getStorageAt

* used uint32

* now its 256

* incarnation
2020-09-06 20:57:58 +01:00
ledgerwatch
3a92b2b39d
Fix for RPC daemon leak (#1059)
* Start memory prof

* Fix rpctest

* Fix rpctest

* Attempt to fix the leak

* Remove http pprof
2020-09-05 21:58:51 +01:00
Giulio rebuffo
e48b28bc5c
added eth_getStorageAt (#1048) 2020-09-04 19:00:22 +01:00
Giulio rebuffo
f235bf1d98
add getTxByHash (#1042) 2020-09-03 21:41:06 +01:00
Evgeny Danilenko
e4f495fa44
Get logs (#1028)
* it compiles

* after recent master

* fix linters warnings

* grpcV7

* go mod tidy

* unmarshall adresses or adress

* fix linters

* after cr

* after cr

* after cr

* after cr

* fix tests

* remove dev version

* it compiles

* mod tidy

* fix bin deps

* use stable version of grpc

* switch back to master constructor

* switch back to master constructor

* add a bit docs

* add a bit docs

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-09-03 08:51:19 +01:00
Giulio rebuffo
d8126d8812
fixed offset (#1036)
* fixed offset

* rm unused
2020-09-03 07:04:23 +01:00
Giulio rebuffo
cad769ee67
Websocket work over json rpc (#1030)
* added ws support

* fixed log

* ws over json rpc

* lint

* fixed bad req
2020-09-02 06:56:48 +01:00
Giulio rebuffo
050ef8773f
fixed chainconfig (#1027) 2020-09-01 20:44:05 +01:00
Giulio rebuffo
b47b8cb989
Ws daemon (#1026)
* added ws support

* fixed log
2020-09-01 17:00:47 +01:00
ledgerwatch
8a3dad0914
Use Finish stage for eth_syncing (#1017) 2020-08-31 06:27:35 +01:00
Igor Mandrigin
a1a7f6607a
rpcdaemon, stagedsync: make sure we update the current block number after a sync cycle (#1015)
* rpcdaemon, stagedsync: make sure we update the current block number after a sync cycle

fixes #1014

* linters
2020-08-30 17:51:06 +01:00
Chase Wright
f2902bb47c
Update totalDifficulty from Big to hexutil.Big (#1009)
* Update totalDifficulty from Big to hexutil.Big
2020-08-30 10:55:31 +07:00
ledgerwatch
22dc832d92
RPC method fixes (#1007)
* Fix some RPC issues

* Fudge the getBlockByNumber

* Add two more PRC methods

* Fix rpc tests
2020-08-29 22:28:09 +01:00
Giulio rebuffo
33316fec6a
Implemented trace_filter (#998)
* added message for --rpc

* more messages for other unused flags

* implemented trace_filter

* lint

* cleanup

* cleanup
2020-08-29 16:50:24 +01:00
Alex Sharov
07ca5c643a
eth_syncing (#991)
* eth_syncing

* linters

* fix readme

* cleanup cli params

* go mod tidy

* remove memsize ui dep

* remove bloomfilter

* implement net_peerCount

* remove legacy flags
2020-08-29 08:24:50 +01:00
Chase Wright
9d3c641664
Update Supported JSON-RPC calls in README (#983)
* Update Supported JSON-RPC calls in README

* Fix Hyperlinks
* Update list

* Include net / net_version

* Rename net.go to net_api.go

* Fix hyperlink for renamed net_api.go file
2020-08-27 19:19:58 +01:00
Alex Sharov
325785aba4
add net namespace (#974) 2020-08-25 20:04:01 +07:00
Giulio rebuffo
c2b3d137f7
implemented eth_getblockbyhash in rpcdaemon (#963) 2020-08-22 21:13:38 +01:00
Alex Sharov
f827f86c11
Rpcdaemon as lib 2 (#943) 2020-08-20 10:52:27 +07:00