Commit Graph

62 Commits

Author SHA1 Message Date
Alex Sharov
3462059ae8
mdbx rpcdaemon support, mdbx - switch to master branch (#1804) 2021-04-26 12:00:20 +07:00
Alex Sharov
e7a78a2d9f
Mdbx: speedup tests (#1776) 2021-04-21 20:03:08 +07:00
Alex Sharov
a95bc0e34d
Mdbx devel up 3 (#1763) 2021-04-20 16:01:28 +07:00
Alex Sharov
00f700675b
add --datadir parameter to integration, snapshot generator, header downloader (#1749) 2021-04-19 14:25:26 +07:00
Alex Sharov
22b9522112
RPCDaemon: add --datadir flag, use --chaindata only if wanna redefine default chaindata folder (#1747) 2021-04-19 12:44:14 +07:00
Alex Sharov
acf1118300
MDBX: enable MADV_RANDOM (#1712) 2021-04-12 21:30:19 +07:00
Artem Vorotnikov
19924040bc
KV: Unified Has (#1671) 2021-04-03 19:04:58 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs (#1669) 2021-04-03 09:26:00 +03:00
Alex Sharov
ad342b27ab
Mdb no db size log (#1665) 2021-04-03 10:30:28 +07:00
Alex Sharov
6179933872
more (#1658) 2021-04-02 16:15:41 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) (#1656) 2021-04-02 13:36:49 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface (#1627) 2021-03-30 12:53:54 +03:00
Artem Vorotnikov
9b8cdc0f22
Fix lints and remove more unused code (#1621) 2021-03-29 10:58:45 +07:00
Alex Sharov
416a69ecb3
Genesis.toBlock remove db from arguments (#1611) 2021-03-28 16:27:06 +07:00
Alex Sharov
aff859edc0
Mining stage (#1554) 2021-03-23 16:00:07 +07:00
Alex Sharov
47c6bdce9d
RPCDaemon: increase throughput by allowing more parallel grpc streams and not lockthread for read tx (#1580) 2021-03-23 14:28:04 +07:00
Alex Sharov
4c82b15c36
no_panic_be_happy (#1586) 2021-03-22 22:45:24 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions (#1577) 2021-03-21 20:15:25 +07:00
Alex Sharov
97eec4288a
Tests: use transaction instead of db.MemCopy in chain_makers.go (#1570)
* mdbx tests speedup

* mdbx tests speedup

* devel up

* devel up

* devel up

* lint

* less db copy

* don't loose error in test

* clean
2021-03-20 19:33:40 +00:00
Artem Vorotnikov
b5f9cdd7b2
KV: split Sequence into ReadSequence and MakeSequence (#1569) 2021-03-20 21:12:54 +07:00
Andrew Ashikhmin
e303aeb5ee
Simple test for layout of TrieOfAccountsBucket (#1563) 2021-03-20 16:38:45 +07:00
Alex Sharov
0953c11c04
grpc version up (#1568) 2021-03-20 16:35:13 +07:00
Artem Vorotnikov
01f6542729
KV: Remove reserve, putcurrent, cmp, dcmp (#1552) 2021-03-20 16:35:02 +07:00
Artem Vorotnikov
0b2e4df0ff
Align Cursor API with LMDB/MDBX (#1558) 2021-03-19 14:45:01 +07:00
Alex Sharov
3ffa029497
Add mdbx_drop cli tool (#1553) 2021-03-15 09:41:02 +07:00
Alex Sharov
3f305673aa
Trie: add invariant - first level of trie must be in DB (to ensure having 100% trie structure in trie_account table) (#1522) 2021-03-13 09:30:54 +07:00
Alex Sharov
060f3b9a60
Grafana: add Btree metrics (#1547) 2021-03-10 10:58:57 +07:00
Alex Sharov
4fe019a099
add periodical db statistic metrics (#1545) 2021-03-09 13:34:13 +07:00
alex.sharov
61f8a1ef0c revive 2021-03-02 09:52:05 +07:00
Alex Sharov
d284f4ede0
clear bucket must not change DBI handle (#1526) 2021-03-01 11:15:59 +07:00
Alex Sharov
25393f46ea
mdbx: fix master (#1517) 2021-02-25 13:20:57 +07:00
Alex Sharov
303d697619
mdbx: same dirty list size as lmdb (#1515) 2021-02-25 10:00:51 +07:00
Alex Sharov
de1806c173
same options as lmdb (#1514) 2021-02-25 09:41:57 +07:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record (#1500)
* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE
2021-02-21 18:41:59 +00:00
Alex Sharov
101c7d78c0
compatibility fix (#1495) 2021-02-14 11:38:28 +07:00
alex.sharov
9b1ec50b97 increase mdbx OptRpAugmentLimit 2021-02-11 12:54:21 +07:00
Alex Sharov
d69f9bd4b2
revert mdbx bindings (#1488) 2021-02-11 11:53:14 +07:00
Artem Vorotnikov
5c34713e71
Remove nested transactions (#1483) 2021-02-10 17:04:22 +00:00
Artem Vorotnikov
739c0d30a7
Remove DupFixed (#1482) 2021-02-10 16:56:44 +07:00
alex.sharov
f88607c7f4 revert: enable lifo 2021-02-09 16:05:04 +07:00
alex.sharov
b9aae2bfcb enable lifo 2021-02-09 16:04:36 +07:00
Alex Sharov
7464b7924d
Print tables size during execution stage (#1479) 2021-02-09 12:02:00 +07:00
Alex Sharov
679d70fca8
Print tables size at end of cycle if table>10Gb, and print freelist size (#1478) 2021-02-09 11:42:10 +07:00
Alex Sharov
5ef9a5a9f0
mdbx: switch CI to mdbx, fix SeekExact, print slow/big transactions info (#1475)
DEBUG_BIG_RO_TX_KB - print logs with info about large read-only transactions
DEBUG_BIG_RW_TX_KB - print logs with info about large read-write transactions
DEBUG_SLOW_COMMIT_MS - print logs with commit timing details if commit is slower than this threshold
2021-02-09 09:31:37 +07:00
Alex Sharov
f351eea439
Mdbx increase reclaim limit (#1465)
* mdbx_increase_reclaim_limit

* readable
2021-02-05 14:11:42 +07:00
Alex Sharov
f0bb3a6dfa
Mdbx new beta version with fixed blockers (#1461) 2021-02-01 20:57:41 +07:00
Alex Sharov
7262fd503b
Mdbx devel 5 (#1413)
* tx

* fix tests

* fix tests

* fix tests

* sequence
2020-12-16 14:35:14 +00:00
Alex Sharov
98b36126b2
History bitmap 64 (#1374)
* squash

* squash

* squash

* squash

* squash

* squash

* subscription_doesnt_preserve_shutdown
2020-12-04 21:16:51 +00:00
Alex Sharov
ed2b36ff04
log db size if stage took longer than 1Minute (#1378)
* log db size

* log db size
2020-11-30 08:58:06 +00:00
Andrew Ashikhmin
8a877c227a
Integrate Silkworm execution (#1344)
* Expose C Handle of lmdbTx

* LoadExecutionFunctionPointer

* silkworm ExecuteBlocks

* linter

* CLI flag for SilkwormExecutionFunc

* linter

* Call SilkwormExecutionFunc in SpawnExecuteBlocksStage

* Commit transaction after Silkworm execution

* Fix batch initialization

* Make batch_size uint64_t instead of size_t

* max_block in silkworm_execute_blocks

* More accurate logProgress

* Use CHandle exposed by lmdb-go

* Enable Silkworm on Linux only

* Update lmdb-go

* Error when attempting to use Silkworm not on Linux

* Move SilkwormFlag from cmd/utils/flags.go to turbo/cli/flags.go

* Integrate Silkworm into cmd/integration

* Check against ReaderBuilder/WriterBuilder/ChangeSetHook in Silkworm execution

* Refactor SpawnExecuteBlocksStage

* linters

* Small clean-ups

* Move ChangeSetHook inside executeBlockWithGo

* No need to tamper with batch size in executeBlocksWithSilkworm

* fix

* Fix

* Fixup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-11-28 15:08:02 +00:00