mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-28 14:47:16 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,110 Commits 5 Branches 14 Tags 294 MiB
de24b427bf
Commit Graph

91 Commits

Author SHA1 Message Date
Alex Sharov
dcf850dd75
fix fieldalignment linter warnings (#2171) 2021-06-16 11:00:35 +07:00
Alex Sharov
5ba3ea162a
Simulated backend and genesis to kv (#2104) 
* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* merge

* merge

* merge

* merge

* merge

* merge

* merge

* merge
 2021-06-05 16:17:04 +01:00
Alex Sharov
0be3044b7e
rename (#1978) 
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
 2021-05-20 19:25:53 +01:00
Alex Sharov
4323d56376
Mdbx tx limit metric (#1891) 2021-05-06 17:02:50 +07:00
Alex Sharov
d959986466
Connect TxFetcher to Sentry, Add --download.v2 option to TG (#1796) 2021-04-25 11:20:50 +07:00
Alex Sharov
bf1c219531
Move txPool.Get method behind direct-grpc (#1760) 2021-04-20 21:41:46 +07:00
Alex Sharov
cc539ce266
Remove blockchain.InsertChain method (#1681) 2021-04-06 12:06:07 +07:00
Artem Vorotnikov
70181c4af8
Mutation: implement StatelessRwTx (#1670) 2021-04-05 13:00:35 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs (#1669) 2021-04-03 09:26:00 +03:00
Artem Vorotnikov
e0a2d47139
Remove ancients and some old skipped tests (#1668) 2021-04-03 13:08:35 +07:00
Alex Sharov
ad342b27ab
Mdb no db size log (#1665) 2021-04-03 10:30:28 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) (#1656) 2021-04-02 13:36:49 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface (#1627) 2021-03-30 12:53:54 +03:00
Artem Vorotnikov
9b8cdc0f22
Fix lints and remove more unused code (#1621) 2021-03-29 10:58:45 +07:00
Alex Sharov
91e4126ca7
db commit - remove unused return param (#1584) 2021-03-22 19:41:52 +07:00
Alex Sharov
40dd00fee7
remove ideal batch size (#1582) 2021-03-22 13:47:01 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions (#1577) 2021-03-21 20:15:25 +07:00
Artem Vorotnikov
b5f9cdd7b2
KV: split Sequence into ReadSequence and MakeSequence (#1569) 2021-03-20 21:12:54 +07:00
Artem Vorotnikov
01f6542729
KV: Remove reserve, putcurrent, cmp, dcmp (#1552) 2021-03-20 16:35:02 +07:00
Alex Sharov
cdc090ebfd
Delete MultiWalk and GetIndexChunk funcs from Database interface (#1544) 2021-03-08 12:08:37 +07:00
ledgerwatch
8d2db43e2b
Fix for slow stage3 (#1377) 
* Fix for slow stage3

* Cache sequence

* Small optimisation

* mutation sequence lock and sequence(0) use read transaction

* tx

* Fixes

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-11-30 11:49:52 +00:00
Alex Sharov
342c5fee89
[to discuss] More compact append implementation (#1372) 
* more compact append implementation

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* fix tests

* fix tests
 2020-11-28 14:24:47 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342) 
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-11-16 12:08:28 +00:00
Alex Sharov
1dd283efe2
Sequence for each table (#1354) 
* history_early_stop

* auto_increment

* auto_increment

* tx_table
 2020-11-14 13:48:29 +00:00
Alex Sharov
73358fab1b
up master (#1332) 2020-11-01 08:28:11 +00:00
Alex Sharov
9712e858e2
Method delete to accept second param: allow delete dupsort values (#1297) 
* aloow_delete_dupsort_values

* check canonical receipt

* exclusive lock for mdbx

* readme_load_dump_tables

* readme_load_dump_tables

* readme_load_dump_tables

* Remove IdealBatchSize change

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-29 13:19:31 +00:00
Alex Sharov
531d36efcb
mdbx support (#1235) 2020-10-28 10:18:10 +07:00
ledgerwatch
32278539ac
Use BTree instead of maps for Mutation (batch) (#1308) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 22:30:18 +00:00
ledgerwatch
1e6c29a9ef
Revert "[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294)" (#1305) 
This reverts commit a9ae477d8f.
 2020-10-27 14:35:25 +00:00
ledgerwatch
a9ae477d8f
[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 13:37:37 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271) 
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-25 08:38:55 +00:00
Alex Sharov
6584b43b12
Allow start read transactions from ethdb.Database interface (#1287) 
* allow_read_transactions

* fix lint

* fix lint

* more cases
 2020-10-24 07:55:43 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124) 
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-28 18:18:36 +01:00
Alex Sharov
95e985ef8f
add commit event to dashboard (#1050) 2020-09-05 10:10:17 +07:00
ledgerwatch
c06dbdadf9
Use Append for changesets and receipts (#1018) 
* Use Append for changesets and receipts

* Fix lint

* Delete all receipts on unwind

* Fix lint

* Fix unwind keys for changeset

* Fix test

* Use writeReceipts flag

* Revert to Append

* Delete receipts too when resetting
 2020-09-01 07:48:25 +01:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975) 
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
 2020-08-26 07:03:50 +01:00
Alex Sharov
43a4e34a6e
[to discuss] exec blocks without ObjectDb: Mutation over TxDb (#947) 
* use mutation over tx

* clean

* clear

* add .CommitAndBegin() method

* clean

* increase timings for logging

* return ideal batch size
 2020-08-24 12:07:59 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size (#918) 
* add logging to loader

* use pure tx in etl loading, logs in mutation commit

* clean

* bletter logging and more cleanup

* bletter logging and more cleanup

* increase batch size to 500M

* better batch commit logging

* async fsync

* sync fsync

* sync fsync

* unify logging

* fix corner-case when etl can use empty bucket name

* fix tests

* better logging

* better logging

* rebase master

* remove lmdb.NoMetaSync flag for now

* consistent walk and multi-walk

* clean

* sub tx

* add consistent multi-put

* implement dupsort support in one new cursor method

* clear
 2020-08-17 07:45:52 +01:00
Alex Sharov
16f09be0a8
add method .Last() (#909) 2020-08-12 10:49:52 +07:00
Alex Sharov
d9d9e14f45
change bucket type to string (#894) 2020-08-11 06:55:32 +07:00
Alex Sharov
0a367001d1
implement db stats page (#803) 2020-07-29 05:31:46 +01:00
Alex Sharov
e65b743878
Prometheus dashboard revisit (#767) 2020-07-21 15:33:03 +07:00
ledgerwatch
887fc4f99c
GenerateChain without using TrieDbState (#726) 
* Different chain generation

* More fixes

* Fix TestSideLogRebirth

* Fix the rest of core tests

* Fix accounts/abi/bind

* Fix compile errors for cmd/pics

* Fix consensus/clique tests

* More fixes

* Fix core/state tests

* Fix eth tests

* Fix eth/downloader tests

* Fix more tests

* Fix compile errors

* Some more fixes

* Save

* Exists

* Fix last tests

* Clean up

* Fix formatting

* Fix linter

* Fix formatting

* Fix linter

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix lint
 2020-07-09 07:15:28 +01:00
Alex Sharov
57358730a4
Minor lmdb related improvements (#667) 
* don't call initCursor on happy path

* don't call initCursor on happy path

* don't run stale reads goroutine for inMem mode

* don't call initCursor on happy path

* remove buffers from cursor object - they are useful only in Badger implementation

* commit kv benchmark

* remove buffers from cursor object - they are useful only in Badger implementation

* remove buffers from cursor object - they are useful only in Badger implementation

* cancel server before return pipe to pool

* try  to fix test

* set field db in managed tx
 2020-06-15 13:30:54 +01:00
Alex Sharov
b05fafc628
Optimize multi put (#657) 
* remove unused slice from MultiPut

* mutation: reuse tuples slice and preallocate bucketPuts

* use bucketPool in kv_lmdb

* remove duplicated check of context status

* more benchmarks

* remove reusage of puts
 2020-06-12 18:59:06 +01:00
Alex Sharov
f0bc2b2146
Run tests on lmdb and badger (#624) 
* lmdb tests

* trigger ci

* fix tests

* disable parallelism

* disable parallelism

* cleanup resources

* cleanup resources

* reduce concurency

* try run tests on bolt

* try run tests on bolt

* fix downloader test

* run bolt tests

* rely on interface instead of exact instance

* Rename AbstractKV to KV

* don't use separator for badger

* don't initialize badger cursor - because it not used here

* fix linter

* try reduce badger compactors

* compat with master

* try lmdb

* try lmdb

* try lmdb

* reduce badger's MaxTableSize, reduce badger's minGoMaxProc for inMem option

* allow to close closed db

* release

* release

* ideal batch size for badger

* ideal batch size for badger
 2020-06-05 10:25:33 +01:00
Alex Sharov
7aa050ca79
[Breaking change in db!] UI for Buckets stat (#607) 
* UI for buckets size
 2020-06-04 16:35:42 +07:00
Alex Sharov
916a1f8b83
[wip] Lmdb: AbstractKV and DB interfaces (#589) 
* resetIH from scratch if needed

* lmdb

* add AbstractKV to loader, added new Object accessor around AbstractKV

* add lmdb cli flag

* add requirement of k!=nil on error in docs

* add Size method for compatibility

* read after put tests

* fix multiput nils

* simplify loops

* increase mmap size

* better error messages

* better error messages

* fix tests

* better error messages

* cleanup

* avoid bolt usage in test

* move hardcoded bucket name to dbutils

* register more buckets

* register more buckets

* fix test
 2020-05-30 09:12:21 +01:00
ledgerwatch
dba3363b52
Fix RestAPI for tx_retrace and make it work for local boltdb (#584) 
* Not hash, keep the files

* Calculate savings

* Fix

* Fix

* Fix

* Fix

* RestAPI to support local boltdb

* Not error on read-only db

* Changes so far

* Continue

* More

* Roll back a bit

* Restore newline

* something compiles

* Fix restapi

* Fix block number

* Fix reads

* Use plain writer

* Maps for storage reads and writes

* Clean up coersions

* Fix accounts/abi/bind

* Fix tests

* More fixes

* more fixes

* More fixes

* Fixes

* Fixed core/state

* Fixed eth tests

* Move code, fix linter

* Fix test

* Fix linter

* Fix linter

* Fix linter, badger_db to support AbstractKV

* Increase IdealBatchSize for badger

* Fix linter

* Fix linter
 2020-05-27 17:24:34 +01:00
Igor Mandrigin
db746bba7a
"Unwind" for the execution phase when plain state is selected (#571) 
* "Unwind" for the execution phase when plain state is selected

* test stub (fails)

* tests (one with incarnations fails)

* test fixups

* fix tests: cleanup contract code bucket
 2020-05-25 12:11:56 +01:00