mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-28 14:47:16 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,110 Commits 5 Branches 14 Tags 294 MiB
de24b427bf
Commit Graph

55 Commits

Author SHA1 Message Date
Alex Sharov
dcf850dd75
fix fieldalignment linter warnings (#2171) 2021-06-16 11:00:35 +07:00
Alex Sharov
5ba3ea162a
Simulated backend and genesis to kv (#2104) 
* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* merge

* merge

* merge

* merge

* merge

* merge

* merge

* merge
 2021-06-05 16:17:04 +01:00
Alex Sharov
eee871aedf
Add db label (later will add txpool db) (#2081) 
* add db label - later will add txpool db

* move commit marker to kv_mdbx behind Chain label

* Update migrations.go

* clean

Co-authored-by: ledgerwatch <akhounov@gmail.com>
 2021-06-04 15:56:49 +01:00
Alex Sharov
c4e20896fa
mdbx ensure lib version match (for .dll and .o compatibility) (#1988) 2021-05-22 11:18:04 +07:00
Alex Sharov
0be3044b7e
rename (#1978) 
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
 2021-05-20 19:25:53 +01:00
Alex Sharov
49225f7319
Mdbx page ops metrics (#1885) 2021-05-06 10:51:49 +07:00
Alex Sharov
f230ba1da4
Less use rawdb deprecated methods (#1861) 
* less use deprecated methods

* less use deprecated methods

* auto Dp limit

* auto Dp limit

* add test
 2021-05-03 21:01:01 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
Alex Sharov
41fa29d421
Mining fixes (#1771) 2021-04-21 12:01:25 +07:00
Alex Sharov
bf1c219531
Move txPool.Get method behind direct-grpc (#1760) 2021-04-20 21:41:46 +07:00
Alex Sharov
a95bc0e34d
Mdbx devel up 3 (#1763) 2021-04-20 16:01:28 +07:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it (#1676) 2021-04-05 20:04:58 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs (#1669) 2021-04-03 09:26:00 +03:00
Artem Vorotnikov
e0a2d47139
Remove ancients and some old skipped tests (#1668) 2021-04-03 13:08:35 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) (#1656) 2021-04-02 13:36:49 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface (#1627) 2021-03-30 12:53:54 +03:00
Alex Sharov
91e4126ca7
db commit - remove unused return param (#1584) 2021-03-22 19:41:52 +07:00
Alex Sharov
40dd00fee7
remove ideal batch size (#1582) 2021-03-22 13:47:01 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions (#1577) 2021-03-21 20:15:25 +07:00
Artem Vorotnikov
72ff17e0b7
Hide TxDb (#1575) 2021-03-21 12:20:44 +07:00
Artem Vorotnikov
b5f9cdd7b2
KV: split Sequence into ReadSequence and MakeSequence (#1569) 2021-03-20 21:12:54 +07:00
Artem Vorotnikov
01f6542729
KV: Remove reserve, putcurrent, cmp, dcmp (#1552) 2021-03-20 16:35:02 +07:00
Alex Sharov
879112f347
TxDb: Walk method - to reuse cursors (#1548) 2021-03-12 16:27:04 +07:00
alex.sharov
5ae602e9a4 less metrics performance impact 2021-03-08 18:58:44 +07:00
Alex Sharov
cdc090ebfd
Delete MultiWalk and GetIndexChunk funcs from Database interface (#1544) 2021-03-08 12:08:37 +07:00
Alex Sharov
f556af6276
Prometheus metrics: more virtual memory visibility, more cross-platform metrics (#1520) 2021-02-28 11:11:28 +07:00
Artem Vorotnikov
5c34713e71
Remove nested transactions (#1483) 2021-02-10 17:04:22 +00:00
Alex Sharov
7fd1f19435
txdb lazy cursors creation, grpc version up (#1440) 
* txdb lazy, grpc version up

* tidy

* fix lint
 2021-01-15 09:38:09 +00:00
Alex Sharov
342c5fee89
[to discuss] More compact append implementation (#1372) 
* more compact append implementation

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* fix tests

* fix tests
 2020-11-28 14:24:47 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342) 
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-11-16 12:08:28 +00:00
Alex Sharov
1dd283efe2
Sequence for each table (#1354) 
* history_early_stop

* auto_increment

* auto_increment

* tx_table
 2020-11-14 13:48:29 +00:00
Alex Sharov
9712e858e2
Method delete to accept second param: allow delete dupsort values (#1297) 
* aloow_delete_dupsort_values

* check canonical receipt

* exclusive lock for mdbx

* readme_load_dump_tables

* readme_load_dump_tables

* readme_load_dump_tables

* Remove IdealBatchSize change

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-29 13:19:31 +00:00
Alex Sharov
531d36efcb
mdbx support (#1235) 2020-10-28 10:18:10 +07:00
ledgerwatch
32278539ac
Use BTree instead of maps for Mutation (batch) (#1308) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 22:30:18 +00:00
ledgerwatch
1e6c29a9ef
Revert "[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294)" (#1305) 
This reverts commit a9ae477d8f.
 2020-10-27 14:35:25 +00:00
ledgerwatch
a9ae477d8f
[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 13:37:37 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271) 
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-25 08:38:55 +00:00
Alex Sharov
6584b43b12
Allow start read transactions from ethdb.Database interface (#1287) 
* allow_read_transactions

* fix lint

* fix lint

* more cases
 2020-10-24 07:55:43 +01:00
Alex Sharov
5dbe1724be
Bitmap etl (#1227) 
* save progress

* save progress

* save progress

* save progress

* etl bimaps

* etl bimaps

* etl bimaps

* etl bimaps

* a

* a

* a

* a

* a

* a

* a

* save progress

* save progress

* save progress

* a

* fix_race_condition_on_zstd_build

* clean

* clean

* clean

* clean

* clean

* go mod tidy

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* fix corner case

* commit every

* call traces etl

* call traces etl

* call traces etl

* return missed error

* go mod tidy

* go mod tidy

* go mod tidy

* rebase to master

* remove trash

* print full key if it has 8 leading zeroes
 2020-10-19 14:11:01 +01:00
ledgerwatch
d2e5cc5cb5
Better migration - receipts_cbor_encode not to increase size of the database (#1250) 
* Add print_migration command to integration

* Not run migrations for print commands

* Reverse migration

* Fix lint

* Add datadir to run_migrations

* Add lmdb.mapSize to integration

* Make lmdb.mapSize effective

* Fix

* receipt sizes

* fix print

* Switch to another version of lmdb

* Collector from existing files

* Not load

* Commit after clearing

* Add methods for clear/drop buckets by multiple transactions

* Fix to lmdb

* Not remove temp files

* Not load

* Re-enable loading

* Prints

* Try to fix refill

* Migration in 3 steps

* tidy mod

* Fix lint

* Fix tests

* Migrations for CBOR receipts in 3 txs

* Fix lint

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-10-17 14:00:12 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124) 
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-28 18:18:36 +01:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support (#1080) 
* etl.Loader - allow use of custom comparator

* log timing

* try now

* try now

* more performance

* etl.Loader - allow use of custom comparator

* working version

* simplify IH cursor

* clean

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* clean

* add only unwind support

* squash

* squash

* clean

* fix test

* clean

* clean

* clean
 2020-09-10 13:35:58 +01:00
Alex Sharov
fd9c7c3b2d
kill bolt (#1085) 
* kill bolt

* kill bolt

* fix test

* clean
 2020-09-09 11:09:55 +01:00
Alex Sharov
da31494cc3
transactional migrations (#1079) 2020-09-08 20:39:43 +01:00
Alex Sharov
7e71c4512e
restrict metric by 1 bucket (#1052) 2020-09-05 13:09:47 +07:00
Alex Sharov
95e985ef8f
add commit event to dashboard (#1050) 2020-09-05 10:10:17 +07:00
Alex Sharov
0c75304261
Os page cache monitoring (#1049) 2020-09-05 09:36:42 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975) 
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
 2020-08-26 07:03:50 +01:00
Alex Sharov
522287ac18
Transactional cycle (#966) 
* v0

* v1

* v3

* v4

* clean

* temporary fix of txpool

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* add condition

* tx pool to not hold own db

* try to enable TxPool in integration

* exclude tx pool from tx

* exclude tx pool from integration

* reduce limit

* fix integration

* clean

* clean

* clean

* clean

* clean

* exclude tx pool unwind

* exclude tx pool unwind in integration

* fix integration tx pool

* fix commit

* fix current stage after unwind

* fix current stage after unwind

* fix linter

* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.

* move body and headers unwind out of tx

* fix unwind order after reboot

* add support external tx to exec stage

* clean

* clean

* clean

* clean

* clean

* add logs

* better id check

* better id check
 2020-08-26 07:02:10 +01:00
Alex Sharov
43a4e34a6e
[to discuss] exec blocks without ObjectDb: Mutation over TxDb (#947) 
* use mutation over tx

* clean

* clear

* add .CommitAndBegin() method

* clean

* increase timings for logging

* return ideal batch size
 2020-08-24 12:07:59 +01:00