Commit Graph

13873 Commits

Author SHA1 Message Date
ledgerwatch
56dc23691c
Fix DB performance regression (#1902)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 12:38:32 +01:00
ledgerwatch
4a214d38e4
Fix integration after snapshot PR (#1901)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 11:42:34 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
e0dd6bafc6
Enable CI for pushes to master (#1900) 2021-05-08 00:43:04 +03:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync (#1899) 2021-05-08 00:07:49 +03:00
ledgerwatch
fe0a882a45
Trace filter (#1895)
* Trace filter

* Fix self-destruct traces

* Update ci.yml

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-07 10:05:47 +01:00
Alex Sharov
80ee37540e
penalize for abandoned anchor (#1896) 2021-05-07 06:17:40 +01:00
Alex Sharov
06cec6d535
Move exec to tx (#1887) 2021-05-07 09:06:44 +07:00
Alex Sharov
b2f424e2cd
Canonical fix in --download.v2 (#1886) 2021-05-07 09:06:00 +07:00
ledgerwatch
9fa21db40e
Qucker reaction to new block headers (#1894)
* Qucker reaction to new block headers

* Always print header lag

* Print verification time

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 19:40:12 +01:00
ledgerwatch
59e5502744
Initial work on RPC streaming (#1880)
* Initial work on RPC streaming

* Create the stream and pass it on

* trace_filter expressed as streamable (not finished)

* Reenable call trace index

* Add new line

* Remove storage mode override

* Tool for overriding storage mode

* Diagnostics

* Make trace_filter more streamy

* Streaming debug_traceTransaction and debug_traceCall

* Fix test

* Log error on stream flush

* Enable streaming for http

* Flush the stream too

* Make trace_filter flush too

* Stop streaming if client is not interested

* Try to quiet annoying test|

* Revert "Try to quiet annoying test|"

This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.

* Debug log for test

* Proceed with handshake regardless of whether peer notification worked

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 18:37:38 +01:00
Chase Wright
d8888439b7
Add tzdata to alpine container for log timestamps (#1892)
Alpine containers, by default, do not include `tzdata` package, so they do not honor the `TZ` environment variable. When running in a docker container the logs timestamps will always be in UTC without this package. I propose we add `tzdata` so that the timezone can be changed.
2021-05-06 18:22:11 +01:00
Artem Vorotnikov
bb305e183e
Update ci.yml (#1893) 2021-05-06 18:22:01 +01:00
Igor Mandrigin
c54c471594 begin 2021.05.02 release cycle 2021-05-06 17:06:02 +02:00
Igor Mandrigin
cc6abbcacc update skip_analysis 2021-05-06 17:00:22 +02:00
Alex Sharov
4323d56376
Mdbx tx limit metric (#1891) 2021-05-06 17:02:50 +07:00
Alex Sharov
e8255abdd0
deprecate tds and trie (#1890) 2021-05-06 15:48:30 +07:00
ledgerwatch
618acbffc1
Remove storageMode override (#1889)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 08:39:10 +01:00
Alex Sharov
b1dc618b95
update stage progress before commit - to keep db always consistent (#1888) 2021-05-06 08:18:43 +01:00
Alex Sharov
49225f7319
Mdbx page ops metrics (#1885) 2021-05-06 10:51:49 +07:00
Evgeny Danilenko
871bb912b1
[WIP] FuzzyVM for TG (#1884)
* restore StateRoot

* empty error should be ommited

* calc state root

* logger json remove code duplication

* fix
2021-05-05 21:55:15 +01:00
Alex Sharov
a627bc2d14
use same keepalive settings in sentry and rpcdaemon (#1883)
* use same keepalive settings in sentry and rpcdaemon

* use same keepalive settings in sentry and rpcdaemon
2021-05-05 11:32:04 +01:00
Alex Sharov
e87d4932f7
Changeset writer move to tx (#1871) 2021-05-05 14:31:40 +07:00
Alex Sharov
61c20798db
fix concurrent map write (#1882) 2021-05-05 10:16:57 +07:00
ledgerwatch
7c60ec3ec0
Fix for eth_getTransaction... returning tx from the pool (#1881)
* Fix for eth_getTransaction... returning tx from the pool

* Fix

* Fix more issues

* Fix for EOF

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 22:58:13 +01:00
ledgerwatch
d604bfbb4f
Fix transaction stream test (#1879)
* Fix transaction stream test

* Fix

* Update events.go

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-04 17:12:59 +01:00
Alex Sharov
c7eedf0091
Add kv to stage settings (because it doesn't change in runtime, tx will be passed as Stage func argument) (#1872)
* add kv to config

* move forward tx use

* move forward tx use

* gc stat

* gc stat

* changeset writer - move to tx

* clean

* clean

* clean

* clean

* clean
2021-05-04 13:36:03 +01:00
ledgerwatch
ea2014f5c6
Making trace_filter work in first approximation with the new call trace index (#1877)
* Fix RPC daemon

* Initial fixes for trace_filter

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 13:34:17 +01:00
Artem Vorotnikov
2148885637
Simplify ChangeSet implementation and tests (#1878)
* Move storage changeset code into one file

* Simplify storage changeset tests

* simplify more

* more

* more

* more

* move account changeset into one file

* simplify account changeset test

* NewStorageChangesetPlain -> NewStorageChangeSet

* EncodeStoragePlain -> EncodeStorage

* StorageChangeSetPlain -> StorageChangeSet

* AccountChangeSetPlain -> AccountChangeSet

* more

* more

* more

* Simplify EncodeStorage

* more

* Simplify FromDBFormat

* more rename

* rename changeset buckets (constant names only)
2021-05-04 13:34:08 +01:00
ledgerwatch
99578ec58d
Fix RPC daemon (#1873)
* Fix RPC daemon

* Skip the test to fix RPC daemon

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 12:31:07 +01:00
Alex Sharov
66b683a6b3
RemoteKv version up (#1874)
* clean

* clean

* clean
2021-05-04 09:56:54 +01:00
ledgerwatch
793c13e20e
trace_filter and CallTrace Index (derivative of #1626) (#1867)
* Trace Index

* trace_filter

* hrtc -> hrt

* Fix CallTraces

* wip

* wip

* Fix for incorrect gas

* fix

* Fix Tracer

* Add block and uncle coinbases to trace index

* rewind coinbase

* Commit every 100k blocks after block 3m

* Simplify

* Safe unwinds

* Safe unwind 2

* Cleanup

* Simplification

* Remove intermediate JSON transform

* Reinstate optional CallIndex stage

* Create rpctest bench for trace_filter

* Fix trace_filter generator

* Restore unwind order

* Ignore the storage mode

* Ignore the storage mode

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 07:23:54 +01:00
Alex Sharov
4b7873708a
clean (#1870) 2021-05-04 13:21:51 +07:00
Alex Sharov
fc196d4b9b
pending txs methods (#1869) 2021-05-04 06:51:28 +01:00
ledgerwatch
8c9a76d3bc
Add Baikal checks properly (#1868)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 06:50:06 +01:00
Alex Sharov
f3d2af1e33
Rpcdaemon: move tx pool to own grpc service, subscribe to pending txs (#1863) 2021-05-04 08:37:17 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods (#1861)
* less use deprecated methods

* less use deprecated methods

* auto Dp limit

* auto Dp limit

* add test
2021-05-03 21:01:01 +01:00
ledgerwatch
5168287784
Make trace_block results closer to what OpenEthereum returns (#1862)
* Add block rewards to trace_block

* Add rewards to trace_block

* Remove printouts

* Fix trace_block and trace_transaction

* Fix getBlockByNumber

* Fix for parent/non parent block

* Reverse fix for trace_call

* Fix eth_getTransactionBy

* Fixes for TIMESTAMP etc opcodes

* More fixes

* Fixes to tracers

* Don't call CaptureEnd twice

* Corrent gasUsed for CaptureEnd in create

* Do CaptureFault consistently

* Correct gasUsed for create

* Remove insufficient balance trace

* Catch contract collision error

* Compatibility

* Compatibility

* More error names

* Out of gas

* Clean up

* more error messages

* Restore CaptureFault

* Errors

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-03 20:49:55 +01:00
Alex Sharov
964f8f76a2
Pin OS version on CI, Add older os to CI (#1864) 2021-05-03 16:59:35 +07:00
Alex Sharov
64d68bcc04
switch to roaring release version #1865 2021-05-03 16:58:14 +07:00
ledgerwatch
39d3314b99
[download.v2] Prevent deadlock by starting txPool early (#1860)
* Prevent deadlock by starting txPool early

* Reduce error levels in the logs

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 12:15:53 +01:00
ledgerwatch
4432618cb9
Support for Baikal devnet (#1853)
* EIP-3529

* Test fix and EIP-3541

* Fix tests

* Fix

* Fix stagedsync test

* Fix

* Remove dup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-02 10:34:19 +01:00
Alex Sharov
7eb088e862
Download request more headers on ext (#1858)
* use lru

* add comment

* add comment
2021-05-02 08:12:17 +01:00
alex.sharov
8a74b65ba2 remove kv_remove prefix and prefetch 2021-05-02 10:53:19 +07:00
Artem Vorotnikov
2e2e3835aa
Remove MdbxCursor.prefix (#1857) 2021-05-02 10:49:00 +07:00
ledgerwatch
4303465e96
Address dump utility (#1856)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-01 21:04:02 +01:00
Alex Sharov
08f6299257
broadcast new headers (#1847)
* broadcast new headers

* don't use eth type

* don't use eth type

* add seenAnnounces scet

* use lru
2021-05-01 21:03:51 +01:00
Alex Sharov
df8c5e3095
All stages - finish fast if current progress > target (#1773) 2021-05-01 17:08:59 +07:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
Alex Sharov
ad598652cf
less blockchain object in tests (#1848) 2021-05-01 10:51:10 +07:00