Commit Graph

12704 Commits

Author SHA1 Message Date
Alex Sharov
6ccf57dace
switch to go implementation of roaring bitmaps for alpine support (#1165) 2020-10-02 15:16:21 +07:00
Alex Sharov
61fbe34024
Better cbor support (#1163) 2020-10-02 10:56:13 +07:00
Alex Sharov
48cf5fb43e
History-style sharding for bitmap indices (#1159) 2020-10-02 10:54:11 +07:00
lightclient
8009a30149
make some minor edits to db walkthrough doc to improve readability (#1169) 2020-10-01 21:57:46 +01:00
Giulio rebuffo
ffd17375eb
added basic installation guide (#1167) 2020-10-01 21:31:07 +01:00
Igor Mandrigin
da9f333a2b begin 2020.10.02 release cycle 2020-10-01 11:21:06 +02:00
Igor Mandrigin
1e023fe278 workaround issue with MUSL and roaring bitmaps 2020-10-01 11:09:49 +02:00
ledgerwatch
96d9639570
Update skip analysis (#1164) 2020-10-01 09:25:44 +01:00
Thomas Jay Rush
5ebf997160
Fixes to a few tracing issues (#1161)
* Starting to look at performance

* Fixes issue #1142 - incorrect error strings

* Uses a recent fix in Geth callTracer code to return balance to self-destruct traces
2020-10-01 07:02:11 +01:00
Alex Sharov
0e62fa4d59
add return (#1158) 2020-09-30 10:28:36 +01:00
Alex Sharov
0559d56dae
Pin protoc version and store it in ./build/bin (#1154)
* use ./build/bin folder to store protobuf binaries

* use ./build/bin folder to store protobuf binaries
2020-09-29 15:09:50 +01:00
ledgerwatch
7bb9cb4274
Remove BloomStatus from the API (#1155)
* Remove BloomStatus from the API

* Trying to find missing space

* Fix lint
2020-09-29 15:09:29 +01:00
Alex Sharov
5a74fe4884
use ./build/bin folder to store protobuf binaries (#1153) 2020-09-29 10:55:41 +01:00
Giulio rebuffo
c53393c9cd
added basic types to rtd (#1149)
* added basic types to rtd

* fixed typo

* fixed doc
2020-09-29 10:34:56 +01:00
Alex Sharov
dae72a8589
(related to lmdb assert) Avoid deleteCurrent on cursor which used for iterations (#1151)
* save progress

* save progress

* avoid delete current

* undupsort_ih
2020-09-29 10:28:49 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124)
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-28 18:18:36 +01:00
ledgerwatch
14288b8cd0
Headers poc 5 - Intermediate (#1145)
* Add headers persistence

* Print flushBuffer

* Fix indexing problem

* Not skip hard-coded headers if the files are empty

* Fix lint

* print anchor state after init

* Properly construct file names

* Add check sub-command

* Fix lint

* Fix lint

* Fix lint

* Print more info when recovering

* Fix recovering

* Fix recovery

* Add anchors also as tips

* 2-level priority queue for tips

* Initialise tipQueue in anchor

* update maxTipHeight

* fix type

* Add anchors to the anchorQueue and rebuild anchorQueue on deletion

* Fix NPE

* fix recovery

* User buffersize, add hard coded headers to buffer

* Reinit anchorQueue

* Schedule requests after recovery

* No fix

* Remove duplicates

* Report duplicate headers

* Log buffer additions

* Fix for duplicate headers

* Try to fix duplicate headers again

* remove TODO comment

* Use LLRB instead of heap for anchors

* Print reserveTip info

* Correctly replace anchors in the tree

* Remove excessive logging

* Print tips attached to the anchor

* Print tips better

* limitTips instead of reserveTip

* Print forked headers

* Use pointers in llrb

* Print tipStretch

* Print limitTips

* Mininise AnchorItem

* Put anchors into the tree

* Fix totalDiff calculation, but is it needed?

* Remove totalDifficulty from anchors

* CheckInitiation

* Fix tests

* Fix lint, print more at check initiation

* Better output for hard-coded anchors

* Print hard-coded anchors better

* Prioritise anchors with short stretches

* Prioritise by chainSize

* Use linked list instead of heap for requestQueue

* Fix problem of no requests

* Push front

* Fix lint

* Not verify PoW for too far in the past and future

* Fix Calculation of totalDifficulty when Connect

* Fix hard tips

* Another fix for tips
2020-09-27 21:32:05 +01:00
Horacio Mijail Antón Quiles
252863fedf
Use the passed VMConfig (#1141)
* Use the passed VMConfig

* Explicitly pass the vmConfig into runBlock
2020-09-27 21:31:49 +01:00
Giulio rebuffo
f33ddec7ab
first sphinx doc portion (#1144)
* first sphinx doc portion

* added .readthedocs.yml

* fixed typos
2020-09-27 20:40:48 +01:00
Thomas Jay Rush
784bc9dc85 Fixes issue #1139 - crash when running chaindata mode (#1140)
* Fixes issue #1139 - crash when running chaindata mode

* Cleaning up error messages as per feedback
2020-09-27 12:11:05 -04:00
Thomas Jay Rush
5b4445ecea Revert "Fixes issue #1139 - crash when running chaindata mode (#1140)"
This reverts commit 00415eee76.
2020-09-27 12:06:37 -04:00
Thomas Jay Rush
00415eee76
Fixes issue #1139 - crash when running chaindata mode (#1140)
* Fixes issue #1139 - crash when running chaindata mode

* Cleaning up error messages as per feedback
2020-09-26 22:01:11 +01:00
ledgerwatch
4be6174a15
Add comment about ContractAddress in receipts, followed by some simplification (#1137)
* Add comment about receipts

* Panic on error

* Ignore errors in the simulated backend

* Fix lint
2020-09-26 13:38:10 +01:00
Thomas Jay Rush
c0a03ad728
Responses to comments on previous tracing PR (#1138)
* Responses to comments on previous tracing PR

* Cleanup

* Cleanup
2020-09-26 07:41:34 +01:00
Thomas Jay Rush
b437fab006
[WIP] Tracing enhancements (#1133)
* Adding some testdata for RPC tracing

* Adding command line option to allow switching trace export format

* Moved types into a separate file for clarity

* Added endpoints and stub functions for other trace routines

* Added experimental end points for issuance, blockReward, and uncleReward

* Moved supporting functions for filtering to _filtering file

* Moved Filter function to _filtering file

* Reordering functions for clarity

* Cleaning up

* Implemented trace_get - 7 of 8 tests pass

* Implemented trace_transaction - 2 of 8 tests pass

* Updating test cases

* Turning off lint temporarily. Will be turned back on

* Extended trace_filter (parity traces, reward traces) - 10 of 13 tests pass

* Implemented trace_block - 4 of 6 tests passed

* Cleanup test cases

* Added refundAddress and selfDestructedAddress to self destruct traces

* Total hack fix to missing gasUsed in some tests. Fixes tests, needs to be replaced
2020-09-25 13:12:36 +01:00
Igor Mandrigin
5aedee8e6f begin 2020.10.01 release cycle 2020-09-24 10:56:17 +02:00
Igor Mandrigin
ab18d19743 update skip_analysis 2020-09-24 10:30:37 +02:00
ledgerwatch
28e2b45f5e
Return zero balance for non-existent accounts instead of crashing (#1135) 2020-09-23 20:18:23 +02:00
Thomas Jay Rush
47f59e8fcd
Small change to accumulateRewards to make it available to the RPC (#1130)
* Small change to accumulateRewards to make it availble in RPC

* Cleanup comments

* Removed unused 'state' parameter from AccumulateRewards
2020-09-23 12:10:54 +01:00
Alex Sharov
e2acbfecda
try to remove sleep (#1126) 2020-09-23 12:10:13 +01:00
Igor Mandrigin
65d8a12bb1
turbo-api: Add docs to some public structs, methods and fields (#1127) 2020-09-21 16:10:25 +02:00
ledgerwatch
5aab794c9b
Headers poc 4 - intermediate changes (#1125)
* Process BlockHeadersMsg

* Print invalidations

* Print reason for parent-child mismatch

* Fix off by one error

* Fix ExtendDown

* Replace tipLimiter with an LRU cache

* Retrieve 256 headers at a time

* Hard coding of tips, process new block hashes

* extract headers

* Write hard-coded headers into a file

* Add total difficulty

* Insert hard coded headers

* Log i

* Initiate download from hard-coded anchors

* Print the state of anchors

* Add hard-coded headers as tips

* Spare the peers

* Refine LRU caching of tips

* Print length of anchor trees

* Limit requests to 192 headers

* range error fix

* Print better, retain at least one tip in the LRU cache

* Skip duplicate line ends

* Throttle requests, display peer name

* Back off, intermediate tip queue

* Print block header timestamp

* Better logging

* Log less

* More logging of evictions

* Fix queue push

* Fix queue pop

* Exclude duplicate tips

* Fix tips logging

* Fix moving

* Add port flag

* Check headers

* Print extra

* Add mixDigest to serialisation

* Fix difficulty in hard-coded headers

* Avoid duplicate header requests

* Update peer characteristics on NewBlockMsg

* Remove backoff when peer responds

* Fix tests

* Fix lint

* Fix lint
2020-09-20 10:47:24 +01:00
Giulio rebuffo
d33302c2be
Mutual TLS authentication (for now without verifying the Common Name) (#1113)
* added tls auth

* added client side

* put --tls

* fixed flag

* Add key/cert generation instructions, turn off Common Name verification

* Add CLI arguments and Warning

* Lint

* Update the doc about Internal IP

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-19 15:16:04 +01:00
ledgerwatch
64e1a8fbd4
Fix chaindata option in RPCdaemon (#1123) 2020-09-18 11:23:35 +01:00
Igor Mandrigin
be2100766c begin 2020.09.04 release cycle 2020-09-17 14:27:55 +02:00
Igor Mandrigin
f956550671 update skip_analysis 2020-09-17 14:24:04 +02:00
Alex Sharov
a112183ceb
fix logidx (#1122)
* fix logidx

* derive fields already dose set index

* start from zero
2020-09-17 13:19:11 +02:00
Thomas Jay Rush
f596a631af
1075 RPC additions and cleanup of README (#1117)
* Rewrite of README for clarity and expanding implementation table

* Small cleanups for web3_api.go

* Changed GetReceipts function to non-export (getRecipts) and cleaned up comments

* Added a comment for GetLogsByHash

* Add support for eth_protocolVersion and eth_chainId

* Adding comments and re-ordering fields to agree with Parity's ordering

* Added support for eth_listening. Moved net_version to same file as others

* Setup for adding eth_gasPrice
2020-09-15 17:44:28 +01:00
Igor Mandrigin
4f77472717
Expose TX instead of DB (#1118) 2020-09-15 16:36:35 +02:00
Igor Mandrigin
53557c3137
Update flags.go (#1116) 2020-09-14 15:48:08 +02:00
Igor Mandrigin
adf52465e3
move ./trie to ./turbo/trie (#1114)
Useful for minimizing merge conflicts when rebasing new geth
2020-09-14 11:33:39 +01:00
Thomas Jay Rush
e5f8073d75
1084 various rpc fixes (#1111)
* Fixes issue #1110 - eth_getStorageAt returning inconsistent values

* Adds support for various eth_getUncle calls

* Adding a couple of comments

* Adding support for eth_getTransactionCount

* Cleaning up README's

* Cleaning up README for rpcdaemon

Co-authored-by: tjayrush <jrush@greathill.com>
2020-09-14 07:59:07 +01:00
Alex Sharov
3a4eb3db3e
Fix metrics conflict with pporf (#1112) 2020-09-14 12:05:35 +07:00
Thomas Jay Rush
065e658f42
Corrects incorrect return from eth_getCode for non-contract accounts (#1109) 2020-09-13 07:06:39 +01:00
ledgerwatch
a98469c6cd
Remove changeset processing in FindHistory (#1105) 2020-09-12 08:24:13 +01:00
ledgerwatch
5d89a7e61b
Add files via upload 2020-09-12 07:30:10 +01:00
ledgerwatch
ad1d9dfdd9
Add files via upload 2020-09-12 07:26:36 +01:00
Thomas Jay Rush
752793264b
Fixes issue #1103 (#1106) 2020-09-12 06:33:07 +01:00
ledgerwatch
0f24a01677
Close the ticker (#1104) 2020-09-11 21:44:24 +01:00
Giulio rebuffo
23d89c0941
Added TLS handshake to RPCDaemon (#1089)
* added eth_getStorageAt

* used uint32

* now its 256

* incarnation

* added TLS handshake

* code minified

* lint

* minified client

* Update flags.go

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-09-11 21:17:37 +01:00