Commit Graph

130 Commits

Author SHA1 Message Date
TKartist
684c3d3697
Tests with iteration have been added (#538) 2022-07-22 16:50:12 +07:00
Alex Sharov
b9efe56597
mdbx: use OS pagesize by default (but > 4Kb, and < 64Kb) #530 2022-07-19 10:39:48 +07:00
TKartist
8b2c02d1fb
Test for kv_mdbx optimized (#528)
* tests for GetOne and Put added

* test configured

* multiple tests added

* conflict resolve

* optimized and tests updated

* conflict fix
2022-07-18 21:36:58 +02:00
TKartist
5821ae7bbc
kv_mdbx_test updated (#522)
* tests for GetOne and Put added

* test configured

* multiple tests added

* conflict resolve
2022-07-18 14:27:03 +07:00
Giulio rebuffo
21aa65c1c3
Silence errors for mdbx_get no data (#521)
* removed empty data cursors operations

* ops

* ops
2022-07-14 18:05:25 +02:00
TKartist
33df7d785c
tests for GetOne and Put added (#519)
* tests for GetOne and Put added

* test configured
2022-07-14 17:57:52 +02:00
Max Revitt
d629e31df7
fix(#4543): BeginRo use semaphore (#520) 2022-07-13 14:37:45 +02:00
Igor Mandrigin
95e361fa1e
Fixup for a possible deadlock if db.env.BeginTxn fails (#516)
* attempt to fix the deadlock

* slightly more correct check

Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-07-07 08:52:30 +02:00
Alex Sharov
a86baf3e81
TxPool: btree with generics (#514) 2022-07-07 11:07:43 +06:00
ledgerwatch
e77f25bce6
[erigon2.2] Temporary tables for state reconstitution (#512)
* Add temporary table for Plain state reconstitution

* Add 2 more temp tables

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-04 08:11:49 +01:00
Alex Sharov
8873903605
Snapshots table (#510)
* save

* save
2022-07-04 10:56:35 +06:00
Alex Sharov
7aa540e6fa
save (#508) 2022-07-01 10:10:23 +06:00
Giulio rebuffo
b2c211d9bf
Refactor sequence handling in mutation 2022-06-25 01:05:27 +02:00
Giulio rebuffo
541dbe8c47
SeekBothRange instead of FirstDup 2022-06-25 01:04:51 +02:00
Giulio rebuffo
8e0df524d8
Added more functionality to the mutation. 2022-06-24 22:10:45 +02:00
ledgerwatch
46bebb3317
[erigon2.2] Add ReadIndices aggregator to collect data (#500)
* [erigon2.2] Add ReadIndices aggregator to collect data

* Try

* Fix for history access

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-20 08:39:29 +01:00
Giulio rebuffo
addc05bae3
Plaxeholder values for dbsize and pagesize 2022-06-19 00:46:31 +02:00
Giulio rebuffo
d57a16c7f6
Implemented ForAmount and MDBX sequencing (#497) 2022-06-18 21:24:30 +02:00
Giulio rebuffo
988a4b5510
Added update txn function 2022-06-17 23:59:04 +02:00
Giulio rebuffo
280c5e9dcc
Added memory mutation to Erigon lib (#493) 2022-06-16 08:01:48 +01:00
ledgerwatch
e2c6ef0058
[erigon2.2] Fixes for inverted indices and domains for the prototype (#489)
* Better control of compress/uncompressed

* Add new function

* more careful pruning

* Printf

* Printf

* Fix DupSort

* Remove copying in prune

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-13 19:32:13 +01:00
Alex Sharov
e4b9cc3884
Reclaimable space in logs (#484)
* save

* save

* save

* save

* save

* save
2022-06-09 13:37:28 +01:00
Alex Sharov
19086759eb
methods to configure db #483 2022-06-09 09:43:12 +07:00
ledgerwatch
157b4299e4
[erigon2] Continuation on domains and inverted indices, putting things together (#476)
* Add scan files tests, create new aggregator type

* Fix lint

* windows test fix

* Add delelte test

* AggCollation

* More functions to Aggregator

* More aggregator functions

* Update

* More functions

* More functions

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-02 21:40:58 +01:00
ledgerwatch
37d9944da9
[erigon2] State domains (move functionality out of aggregator) (#436)
* Domain

* First functions

* change year

* More on domain

* More to test

* More on test

* More on domains

* buildFiles

* More on domains

* Collation test

* Fix collate

* Add test for decompressors

* Restructure history tables

* Split history into 2 tables

* Fix lint

* Check index files in the test

* Close files

* Add file scanning

* Fix lint

* Fix lint

* Add readFromFiles

* Add ef history idx file

* Start cleanup

* More to cleanup, test for ef history

* More test

* Add prune to test

* Test for prune and fix

* Start history access

* History test

* Test for LastDup

* Fix one lint

* Workaround

* History tests

* Debug

* Fix

* Fix in history

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@alexs-macbook-pro.home>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alex Sharp <alexsharp@alexs-mbp.lan>
2022-05-24 18:59:57 +01:00
Alex Sharov
9821dcdff5
kv_mdbx: atomic closed flag (#464) 2022-05-24 12:20:19 +07:00
Alex Sharov
4e257396f4
Kv: GetBool helper (#445)
* save

* save
2022-05-10 14:51:32 +07:00
Alex Sharov
9f5b2b6391
don't log inside library about db close #442 2022-05-10 08:55:32 +07:00
Alex Sharov
3c52757526
Downloader tables (#422)
* save

* save

* save
2022-04-21 10:43:46 +07:00
Alex Sharov
1ac7357005
KV: ReadAhead helper #419 2022-04-18 09:37:01 +07:00
Andrew Ashikhmin
8c89288cbc
Add LastForkchoice table (#405)
* Deprecate TransitionBlock

* Add ForkchoiceHead/Safe/Finalized tables

* Merge Forkchoice* into LastForkchoice
2022-03-31 10:51:23 +02:00
Alex Sharov
54629aaf7c
Mdbx fix after v0.11.6 (#398)
* save

* save
2022-03-27 07:23:35 +01:00
ledgerwatch
4e8d577d1d
[erigon2] Switch back to MDBX (#388)
* Switch back to MDBX

* Fix test

* No restore

* Fix overwrite

* Fix change file opening

* Verify state vs change files

* Add WriteMap

* Print

* Not to go to the end of the file when reading

* Fix rebuild

* prefixLen

* Print

* Print

* Print

* Fix for rebuild

* Remove prints

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-23 14:35:13 +00:00
Alex Sharov
83951a1d62
Enable more linters (#381) 2022-03-19 11:38:37 +07:00
Alex Sharov
2193d570a6
Mdbx: WriteMap fallback on open error (#369)
* save

* save
2022-03-16 11:59:11 +00:00
battlmonstr
3123b6d895
kv: add NodeRecords nodedb table (#359) 2022-03-10 13:15:15 +01:00
battlmonstr
f54e4c7eae
kv: use SetSyncPeriod from mdbx-go v0.22.16 (#358) 2022-03-10 13:04:08 +01:00
Alex Sharov
6512e3c941
add emptyWordsCount field to .seg file header (breaking .seg format) (#355)
* up torrent

* save

* save

* save

* save

* save

* save

* save
2022-03-10 07:48:37 +00:00
battlmonstr
009358ab2f
kv: expose mdbx SafeNoSync and OptSyncPeriod options (#356) 2022-03-10 10:25:11 +07:00
Alex Sharov
a2cbecf373
[wip] KV: limit amount of parallel read txs (#352)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* save

* save

* save

* save

* save
2022-03-07 12:21:59 +00:00
Alex Sharov
56d528b1f0
save (#353) 2022-03-04 09:13:15 +07:00
Alex Sharov
5be29adf1a
Grpc err utils (#334)
* save

* save
2022-02-18 09:40:11 +07:00
alex.sharov
a2ca4cf55f save 2022-02-17 10:10:23 +07:00
Alex Sharov
7aec036bdd
remotedb server: ro db (#313) 2022-02-11 16:02:57 +07:00
Alex Sharov
8e4618c57d
state and newHead: 1 goroutine per subscriber #309 2022-02-10 08:26:08 +07:00
alex.sharov
d814adb13e save 2022-02-03 16:28:43 +07:00
alex.sharov
02c407a14e save 2022-02-03 16:28:32 +07:00
Enrique Jose Avila Asapche
a5c2d83108
Added bor tables (#286)
Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
2022-01-28 21:34:03 +00:00
ledgerwatch
df95313a31
Add blockGasLimit to the state stream (#261)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit eff9f8f

git-subtree-dir: interfaces
git-subtree-split: eff9f8f7273dda7c78575f2474e3103da61825db

* Add blockGasLimit to the state stream

* Pass blockGasLimit into txpool

* Add NotTooMuchGas bit, refactored promote

* Replace effectiveTip with minTip in the queues and slices

* Fix lint

* Refactor

* Improve terminology of ordering function

* Optimise baseFee change

* cleanup

* Safe operations

* Fix UnsafeAdd

* also print pendingBaseFee

* trace txpool_content

* trace txpool_content

* user sender ID inverted index to speed up txpool_content

* copy rlp from Db

* remove debug printing

* change from string to []byte

* safe storage of sender in the map

* Fix compile

* safe storage of sender in the map

* Print errors

* Prints

* Prints

* Prints

* Change worst function

* Change worst function

* Change worst function

* Change worst function

* Fix minTip

* Fix minTip

* Warn on db error, but not stop

* Fix interface

* More prints

* Cleanup

* Cleanup

* Differentiate baseFee pool

* Bump version of remove KV interface

* Not print every block

* Not print every block

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-25 10:20:11 +00:00
Alex Sharov
1c92f7bc00
mdbx: consensus label #265 2022-01-22 12:49:25 +07:00