mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-26 05:27:19 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
13,960 Commits 5 Branches 14 Tags 294 MiB
87d24be960
Commit Graph

84 Commits

Author SHA1 Message Date
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018) 
* turbo-geth to erigon

* tg, turbo to erigon
 2021-05-26 11:35:39 +01:00
Alex Sharov
c4e20896fa
mdbx ensure lib version match (for .dll and .o compatibility) (#1988) 2021-05-22 11:18:04 +07:00
Alex Sharov
0be3044b7e
rename (#1978) 
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
 2021-05-20 19:25:53 +01:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it (#1676) 2021-04-05 20:04:58 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs (#1669) 2021-04-03 09:26:00 +03:00
Artem Vorotnikov
e0a2d47139
Remove ancients and some old skipped tests (#1668) 2021-04-03 13:08:35 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) (#1656) 2021-04-02 13:36:49 +07:00
Artem Vorotnikov
d4c10c9a47
Port rpcdaemon to KV interface (#1627) 2021-03-30 12:53:54 +03:00
Artem Vorotnikov
7968637d42
ObjectDatabase cleanup (#1598) 
* ObjectDatabase cleanup

* Remove ObjectDatabase::WarmUp
 2021-03-24 22:48:08 +00:00
Alex Sharov
4c82b15c36
no_panic_be_happy (#1586) 2021-03-22 22:45:24 +07:00
Alex Sharov
40dd00fee7
remove ideal batch size (#1582) 2021-03-22 13:47:01 +07:00
Artem Vorotnikov
6a6f5469a1
KV: Split RO and RW transactions (#1577) 2021-03-21 20:15:25 +07:00
Artem Vorotnikov
b5f9cdd7b2
KV: split Sequence into ReadSequence and MakeSequence (#1569) 2021-03-20 21:12:54 +07:00
Artem Vorotnikov
01f6542729
KV: Remove reserve, putcurrent, cmp, dcmp (#1552) 2021-03-20 16:35:02 +07:00
alex.sharov
5ae602e9a4 less metrics performance impact 2021-03-08 18:58:44 +07:00
Alex Sharov
cdc090ebfd
Delete MultiWalk and GetIndexChunk funcs from Database interface (#1544) 2021-03-08 12:08:37 +07:00
Alex Sharov
f556af6276
Prometheus metrics: more virtual memory visibility, more cross-platform metrics (#1520) 2021-02-28 11:11:28 +07:00
Alex Sharov
f1ee8d507e
New Trie db layout - store trie structure info and multiple hashes per record (#1500) 
* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE

* SE
 2021-02-21 18:41:59 +00:00
Artem Vorotnikov
5c34713e71
Remove nested transactions (#1483) 2021-02-10 17:04:22 +00:00
Artem Vorotnikov
123710adcd
ChangeSet: remove Walker::Walk (#1476) 2021-02-09 17:26:00 +07:00
alex.sharov
fe193723f7 make lmdb default 2021-02-09 09:36:59 +07:00
Alex Sharov
5ef9a5a9f0
mdbx: switch CI to mdbx, fix SeekExact, print slow/big transactions info (#1475) 
DEBUG_BIG_RO_TX_KB - print logs with info about large read-only transactions
DEBUG_BIG_RW_TX_KB - print logs with info about large read-write transactions
DEBUG_SLOW_COMMIT_MS - print logs with commit timing details if commit is slower than this threshold
 2021-02-09 09:31:37 +07:00
Alex Sharov
7262fd503b
Mdbx devel 5 (#1413) 
* tx

* fix tests

* fix tests

* fix tests

* sequence
 2020-12-16 14:35:14 +00:00
ledgerwatch
8d2db43e2b
Fix for slow stage3 (#1377) 
* Fix for slow stage3

* Cache sequence

* Small optimisation

* mutation sequence lock and sequence(0) use read transaction

* tx

* Fixes

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-11-30 11:49:52 +00:00
Alex Sharov
4565bd43c1
Mdbx v0.9.2 (#1373) 
* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* up devel

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* switch to master

* mdbx 0.9.2

* mdbx 0.9.2

* %<<$!%%%%%%<o'{}k.###

* %<<$!%%%%%%<o'{}k.###

* merge

* merge

* merge

* merge

* ad

* ad

* ad

* ad
 2020-11-28 14:26:28 +00:00
Alex Sharov
342c5fee89
[to discuss] More compact append implementation (#1372) 
* more compact append implementation

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* do appned for dupsort buckets

* fix tests

* fix tests
 2020-11-28 14:24:47 +00:00
Alex Sharov
b3f1915d09
ChangeSets dupsort (#1342) 
* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* change_set_dup

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* working version

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* aa

* squash

* squash

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* history_early_stop

* history_early_stop

* vmConfig with ReadOnly false

* auto_increment

* auto_increment

* rebase master

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-11-16 12:08:28 +00:00
Alex Sharov
1dd283efe2
Sequence for each table (#1354) 
* history_early_stop

* auto_increment

* auto_increment

* tx_table
 2020-11-14 13:48:29 +00:00
Alex Sharov
9712e858e2
Method delete to accept second param: allow delete dupsort values (#1297) 
* aloow_delete_dupsort_values

* check canonical receipt

* exclusive lock for mdbx

* readme_load_dump_tables

* readme_load_dump_tables

* readme_load_dump_tables

* Remove IdealBatchSize change

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-29 13:19:31 +00:00
Igor Mandrigin
abf4fa5861
Don't require MDBX if you don't specify it (#1313) 
* fix compilation

* add mdbx flag to the tg binary

* fix makefile

* fixups

* fix makefile

* fix readme

* fix linters

* fix tests
 2020-10-28 12:17:18 +00:00
Alex Sharov
531d36efcb
mdbx support (#1235) 2020-10-28 10:18:10 +07:00
ledgerwatch
32278539ac
Use BTree instead of maps for Mutation (batch) (#1308) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 22:30:18 +00:00
ledgerwatch
1e6c29a9ef
Revert "[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294)" (#1305) 
This reverts commit a9ae477d8f.
 2020-10-27 14:35:25 +00:00
ledgerwatch
a9ae477d8f
[WIP - Discussion - Perf Testing] Use BTree instead of maps for Mutation (batch) (#1294) 
* Try mutation based on B-tree

* Use pointers

* Fix NPE

* Cleanup
 2020-10-27 13:37:37 +00:00
Alex Sharov
6f27e3043b
lmdb_readonly_dont_create_folder (#1293) 2020-10-25 08:39:09 +00:00
Alex Sharov
331dcd45eb
Store receipts separately - one record per tx (#1271) 
* squash

* add --database flag to integration

* clean

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* split to 2 buckets

* save progress

* save progress

* improve test

* improve test

* save progress

* change app logic

* change app logic

* return err from rawdb package

* don't clean automatically

* don't clean automatically

* clean

* clean

* clean

* don't rely on `make clean`

* improve cbor code

* clean

* clean

* clean

* fix tests

* rebase master

* stop on error: headers stage

* make TxDb walk and multiwalk safe

* Fix panics

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-25 08:38:55 +00:00
Alex Sharov
e85b388061
Protect from wrong class use: make ethdb.Tx incompatible with rawdb.DatabaseReader (#1288) 
* protect_from_wrong_class_use

* more cases
 2020-10-24 09:09:20 +01:00
Alex Sharov
6584b43b12
Allow start read transactions from ethdb.Database interface (#1287) 
* allow_read_transactions

* fix lint

* fix lint

* more cases
 2020-10-24 07:55:43 +01:00
ledgerwatch
d2e5cc5cb5
Better migration - receipts_cbor_encode not to increase size of the database (#1250) 
* Add print_migration command to integration

* Not run migrations for print commands

* Reverse migration

* Fix lint

* Add datadir to run_migrations

* Add lmdb.mapSize to integration

* Make lmdb.mapSize effective

* Fix

* receipt sizes

* fix print

* Switch to another version of lmdb

* Collector from existing files

* Not load

* Commit after clearing

* Add methods for clear/drop buckets by multiple transactions

* Fix to lmdb

* Not remove temp files

* Not load

* Re-enable loading

* Prints

* Try to fix refill

* Migration in 3 steps

* tidy mod

* Fix lint

* Fix tests

* Migrations for CBOR receipts in 3 txs

* Fix lint

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-10-17 14:00:12 +01:00
ledgerwatch
cd706d5081
Download/seed block bodies and block headers via BitTorrent (#1218) 
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

* tidy mods

* Fix compile

* Fix lint

* Fix rpcdaemon running in the docker

Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-10-13 13:56:16 +01:00
ledgerwatch
b747ab5324
[WIP] CallTraces index (#1157) 
* Initial commit for CallTraces index

* Fix compilation

* fix lint, add comment

* Fix integration

* Add Close function to ethdb.Cursor, fix some compile errors

* Try to stop cursor leak in Get

* Fix compile errors in RPC daemon

* Fix compile errors

* fixing another way

* Some fixes

* More fixes

* More fixes

* More fixes

* Fixes to core/state

* Fix lint

* Fix lint

* Fixes

* Stage caching for call trace stage

* Add mem stats

* Try to stop the leak

* Turn off debug

* Chunks for 10k blocks

* Print

* Revert "Print"

This reverts commit 5ffada4828d61e00e5dad1ca12c98258dfbbad00.

* Revert "Chunks for 10k blocks"

This reverts commit cfb9d498e782e5583d41c30abf0e2137da27383e.

* Trying to fix the leak

* Don't compute receipts in re-tracing

* Not compose block

* Print speed, fix receipts, bigger caches

* Fix lint

* Utilise changeset info

* Counters

* Use NoReceipts and ReadOnly

* ReadOnly is incompatible with caching

* Skip test leaking transactions

* Fix block test

* Change disable message for call-traces stage

* Use block option for call traces integration

* Fix retracing due to incarnation
 2020-10-12 09:39:04 +01:00
ledgerwatch
1af6cbe37b
Revert "Download/seed headers and block bodies via BitTorrent (#1195)" (#1202) 
This reverts commit 94360212cb.
 2020-10-07 21:49:44 +01:00
ledgerwatch
94360212cb
Download/seed headers and block bodies via BitTorrent (#1195) 
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-10-07 13:15:49 +01:00
ledgerwatch
bdb0155856
Revert "Add sync by bittorrent snapshots (#1160)" (#1194) 
This reverts commit 4ebade3583.
 2020-10-06 22:31:57 +01:00
b00ris
4ebade3583
Add sync by bittorrent snapshots (#1160) 
* save state

* torrent experiments

* torrent experiment passed

* fixes after merge

* snapshot headers processing passed

* save state

* save state

* download headers works after snapshot processing

* save state

* save state

* save state

* save state

* add lazy load tx to snapshots, increase number of trackers

* save state

* speedup getting info

* change logging

* move to turbo package

* save state

* save state

* save state

* cleanup

* save state

* add test test

* save state

* lmdb debugging

* fix readonly mode

* save state

* fix build

* sync works

* save state

* save state

* save state

* allow cmd stages stageSenders use snapshots

* debugging failed hashing

* remove experimental tests

* remove torrent experimental tests

* fix lint

* extract snapshot wrapper

* metainfo checker

* add remote seeder

* add logs

* update gomod

* remove useless code

* fix lint&remove useless code

* extract verify snapshot to separated command

* skip debug test

* fix test

* change type of seedSnapshot flag

* add eth logger to torrent lib

* skip debug test

* add Close method

* review fixes

* fix lint

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-10-06 20:24:48 +01:00
Evgeny Danilenko
ec6be820ac
Don't force sync in case InMemDb (#1174) 
* initial

* fix

* fix chain length
 2020-10-02 14:08:28 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124) 
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-28 18:18:36 +01:00
Alex Sharov
fd9c7c3b2d
kill bolt (#1085) 
* kill bolt

* kill bolt

* fix test

* clean
 2020-09-09 11:09:55 +01:00
Alex Sharov
c45a710ce6
Warmup logs, less overhead, warmup code bucket (#1054) 
* warmup logs and less overhead.

* warmup logs and less overhead.

* move WarmUp to common func
 2020-09-08 08:28:37 +01:00
Alex Sharov
0c75304261
Os page cache monitoring (#1049) 2020-09-05 09:36:42 +07:00