Commit Graph

562 Commits

Author SHA1 Message Date
Alex Sharov
9d66aa0fe3
better support for mdbx in snapshots (#1971) 2021-05-20 18:12:07 +07:00
ledgerwatch
79084ede47
[download.v2] Make default (#1963)
* [download.v2] Make default

* Update server.go

* Fix for tmp files appearing in the current dir

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
2021-05-19 16:19:29 +01:00
Alex Sharov
369b41dfc2
clean (#1962) 2021-05-19 15:55:10 +07:00
Alex Sharov
ba185eff83
tx.Cursor() ignored error (maybe it's fix for MDBX_PROBLEM) (#1961) 2021-05-19 11:21:17 +07:00
Alex Sharov
781a67a4ee
Auto-Close db in tests by using t.Cleanup (#1956) 2021-05-19 10:47:28 +07:00
b00ris
777882ff52
Fix corner case for KV_Snapshot.Next (#1957)
* fix next

* comment test
2021-05-18 19:32:04 +01:00
Alex Sharov
cceaf75b16
use t.TempDir() (#1955) 2021-05-18 19:13:16 +07:00
Alex Sharov
0c91bfbf3e
RPCDaemon: support Pending block (#1942)
* clean

* save

* pubsub

* pubsub

* pubsub

* pubsub

* pubsub

* pubsub

* save

* tx pub-sub

* tx pub-sub

* clean

* clean

* save

* save

* save

* save

* save

* Squashed 'interfaces/' content from commit c469f3ae0

git-subtree-dir: interfaces
git-subtree-split: c469f3ae073b60c8821b61fed2910191080ef835

* save

* save

* save

* save

* Squashed 'interfaces/' changes from c469f3ae0..958dfc669

958dfc669 save

git-subtree-dir: interfaces
git-subtree-split: 958dfc669f8daeefe686a13aa852fb95f1537886

* save

* save

* up some deps

* up some deps

* clean

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test
2021-05-17 13:15:19 +01:00
Alex Sharov
617baac56b
move db tests to kv interface (#1946) 2021-05-17 16:54:29 +07:00
Alex Sharov
9bef3fb656
move some tests to mdbx (#1947) 2021-05-17 16:14:08 +07:00
Alex Sharov
2ff6ce7d92
move staged sync to tx (#1897) 2021-05-14 18:00:20 +07:00
b00ris
48733c31cd
test fix (#1930) 2021-05-13 21:04:47 +01:00
Alex Sharov
f3a9eb340c
mdbx_setup_debug - handle response correct way (#1931) 2021-05-13 21:01:23 +07:00
b00ris
7948eafcc2
Property tests for kv snapshot (#1928)
* model tests for snapshot kv

* cursor test

* first failing test

* fix

* fix lint

* remove fmt
2021-05-13 09:40:32 +01:00
Alex Sharov
3e1bdbb465
Mdbx: fix for MDBX_PROBLEM (#1922) 2021-05-13 14:40:44 +07:00
Alex Sharov
0163329a5a
KV and RemoteKV: to reuse stateless cursors during transaction (#1909) 2021-05-11 15:23:17 +07:00
Artem Vorotnikov
80bbbf62c2
Misc fixes (#1908)
* Fix debug levels

* try this

* remove adapter

* fix

* fix bodies stage
2021-05-10 06:59:29 +01:00
Alex Sharov
3f36e362c2
mdbx switch to release (#1906) 2021-05-09 15:35:47 +07:00
ledgerwatch
8907e95d4a
Trace filter 3 (#1904)
* Prevent the leak

* Default storage mode, fix leak again

* TopSeenHeight to be updated from any header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-08 23:07:16 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Artem Vorotnikov
09788dbc02
Pruning flag, incremental staged sync (#1899) 2021-05-08 00:07:49 +03:00
ledgerwatch
59e5502744
Initial work on RPC streaming (#1880)
* Initial work on RPC streaming

* Create the stream and pass it on

* trace_filter expressed as streamable (not finished)

* Reenable call trace index

* Add new line

* Remove storage mode override

* Tool for overriding storage mode

* Diagnostics

* Make trace_filter more streamy

* Streaming debug_traceTransaction and debug_traceCall

* Fix test

* Log error on stream flush

* Enable streaming for http

* Flush the stream too

* Make trace_filter flush too

* Stop streaming if client is not interested

* Try to quiet annoying test|

* Revert "Try to quiet annoying test|"

This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.

* Debug log for test

* Proceed with handshake regardless of whether peer notification worked

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-06 18:37:38 +01:00
Alex Sharov
4323d56376
Mdbx tx limit metric (#1891) 2021-05-06 17:02:50 +07:00
Alex Sharov
49225f7319
Mdbx page ops metrics (#1885) 2021-05-06 10:51:49 +07:00
Alex Sharov
a627bc2d14
use same keepalive settings in sentry and rpcdaemon (#1883)
* use same keepalive settings in sentry and rpcdaemon

* use same keepalive settings in sentry and rpcdaemon
2021-05-05 11:32:04 +01:00
Alex Sharov
e87d4932f7
Changeset writer move to tx (#1871) 2021-05-05 14:31:40 +07:00
Alex Sharov
61c20798db
fix concurrent map write (#1882) 2021-05-05 10:16:57 +07:00
ledgerwatch
7c60ec3ec0
Fix for eth_getTransaction... returning tx from the pool (#1881)
* Fix for eth_getTransaction... returning tx from the pool

* Fix

* Fix more issues

* Fix for EOF

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 22:58:13 +01:00
ledgerwatch
d604bfbb4f
Fix transaction stream test (#1879)
* Fix transaction stream test

* Fix

* Update events.go

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-04 17:12:59 +01:00
Alex Sharov
c7eedf0091
Add kv to stage settings (because it doesn't change in runtime, tx will be passed as Stage func argument) (#1872)
* add kv to config

* move forward tx use

* move forward tx use

* gc stat

* gc stat

* changeset writer - move to tx

* clean

* clean

* clean

* clean

* clean
2021-05-04 13:36:03 +01:00
ledgerwatch
99578ec58d
Fix RPC daemon (#1873)
* Fix RPC daemon

* Skip the test to fix RPC daemon

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 12:31:07 +01:00
Alex Sharov
66b683a6b3
RemoteKv version up (#1874)
* clean

* clean

* clean
2021-05-04 09:56:54 +01:00
ledgerwatch
793c13e20e
trace_filter and CallTrace Index (derivative of #1626) (#1867)
* Trace Index

* trace_filter

* hrtc -> hrt

* Fix CallTraces

* wip

* wip

* Fix for incorrect gas

* fix

* Fix Tracer

* Add block and uncle coinbases to trace index

* rewind coinbase

* Commit every 100k blocks after block 3m

* Simplify

* Safe unwinds

* Safe unwind 2

* Cleanup

* Simplification

* Remove intermediate JSON transform

* Reinstate optional CallIndex stage

* Create rpctest bench for trace_filter

* Fix trace_filter generator

* Restore unwind order

* Ignore the storage mode

* Ignore the storage mode

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-04 07:23:54 +01:00
Alex Sharov
4b7873708a
clean (#1870) 2021-05-04 13:21:51 +07:00
Alex Sharov
fc196d4b9b
pending txs methods (#1869) 2021-05-04 06:51:28 +01:00
Alex Sharov
f3d2af1e33
Rpcdaemon: move tx pool to own grpc service, subscribe to pending txs (#1863) 2021-05-04 08:37:17 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods (#1861)
* less use deprecated methods

* less use deprecated methods

* auto Dp limit

* auto Dp limit

* add test
2021-05-03 21:01:01 +01:00
alex.sharov
8a74b65ba2 remove kv_remove prefix and prefetch 2021-05-02 10:53:19 +07:00
Artem Vorotnikov
2e2e3835aa
Remove MdbxCursor.prefix (#1857) 2021-05-02 10:49:00 +07:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
Alex Sharov
e7849498d5
Degbug build support (for delve debugger and for getting C code profiling, traces) (#1843) 2021-04-29 21:29:58 +07:00
Alex Sharov
07a063cb8a
txn full fix - cursors leak (#1838) 2021-04-28 18:00:32 +01:00
ledgerwatch
92c20e1f67
[eip-1559] Apply gas limit elasticity (#1834)
* Apply gas limit elasticity

* Fix unmarshalling tx

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-28 15:32:48 +01:00
Alex Sharov
768bdcd850
--database.verbosity flag (#1829)
* --database.verbosity flag

* --database.verbosity flag
2021-04-27 13:31:00 +01:00
Alex Sharov
9cc4a0a97c
rebort db metrics - not guilty (#1828) 2021-04-27 15:32:41 +07:00
ledgerwatch
e8161541e2
Use RLP encoding instead of json for subscriptions (#1825)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-27 09:23:04 +01:00
ledgerwatch
b341b5aed5
Fix for RPC daemon crashing TG when pending txs are transmitted (#1819)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 17:56:02 +01:00
ledgerwatch
117838e213
Move compatibility check (#1814)
* Move compatibility check

* Implement Alex's suggestion

* Use root context

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-26 13:39:34 +01:00
Alex Sharov
544c597151
Pending txs part 2 (#1816) 2021-04-26 18:15:05 +07:00
Alex Sharov
4e80d6eeed
Pending txs subscription (#1810) 2021-04-26 16:53:38 +07:00