Commit Graph

923 Commits

Author SHA1 Message Date
Alex Sharov
e824fdff60
remove fuzzbeta build tag, because now go1.18 is minimum requirement (#428) 2022-07-03 14:38:53 +06:00
ledgerwatch
707a89842d
Add function to get history without state (#501)
* Add function to get history without state

* Add recon functions

* Expose endMinimax

* Recon prints

* Add NoState access methods

* MaxTxNum functions

* MaxTxNum functions

* MaxTxNum functions

* MaxTxNum functions

* History iterator

* Iterator

* history iterators to aggregator

* Print

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Print

* Print

* Print

* Fix

* Fix

* Fix

* Fix

* Fix

* Print

* Print

* Print

* Print

* Print

* Add stats

* Remove time measurement

* Contexts for thread safety

* Partial iterators

* Fix

* Fix

* Not use SkipUncompressed

* Print

* Print

* Pass empty vals

* Parallel bitmap collection

* Print

* ReconTx iterator

* ReconTx iterator

* ReconTx iterator

* ReconTx iterator

* Print

* Print

* Remove print

* Print

* Print

* Print

* Print

* Print

* Print

* Dedicated getter for Iterate

* For for storage 0

* Remove print

* do not perform unnecessary changes

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-02 19:38:34 +01:00
Alex Sharov
ed452dbc4b
save (#509) 2022-07-01 10:20:32 +06:00
Alex Sharov
7aa540e6fa
save (#508) 2022-07-01 10:10:23 +06:00
Alex Sharov
2a6be9cb92
Downloader verify (#507)
* Remove interfaces for replacement

* Remove interfaces for replacement

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit f047f61c

git-subtree-dir: interfaces
git-subtree-split: f047f61c2a0388aeeef2750cab1045f7212acfcc

* save
2022-07-01 10:06:58 +06:00
Andrew Ashikhmin
59f7b5b57b
Exempt local transactions from spam protection (#506) 2022-06-29 21:44:34 +06:00
Andrew Ashikhmin
3aa23bded8
Add eth/67 (#505)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 1c7f9c3

git-subtree-dir: interfaces
git-subtree-split: 1c7f9c34740ab1057d9fc3e09da1d52eceffe6b8

* Update auto-generated files

* Add eth/67
2022-06-28 17:24:56 +01:00
Andrew Ashikhmin
2c79430f85
Revert "Switch from eth/66 to eth/67 (#503)" (#504)
This reverts commit eac8c1fa59.
2022-06-28 13:39:45 +02:00
Andrew Ashikhmin
eac8c1fa59
Switch from eth/66 to eth/67 (#503)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit a25a685

git-subtree-dir: interfaces
git-subtree-split: a25a68528428cc6d792b5033dd97d094c14e1dd9

* Regenerate proto

* eth/66 -> eth/67

* Auto-generated leftovers

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 674172f

git-subtree-dir: interfaces
git-subtree-split: 674172f20ef9a77bcff1c97213f5f3865115f213

* Refresh generated proto

* Remove version suffix from MessageId
2022-06-28 09:58:12 +02:00
Alex Sharov
ceafdded8f
Compress: reduce etl buffers to save RAM (#502) 2022-06-25 19:39:36 +06:00
Giulio rebuffo
b2c211d9bf
Refactor sequence handling in mutation 2022-06-25 01:05:27 +02:00
Giulio rebuffo
541dbe8c47
SeekBothRange instead of FirstDup 2022-06-25 01:04:51 +02:00
Giulio rebuffo
8e0df524d8
Added more functionality to the mutation. 2022-06-24 22:10:45 +02:00
ledgerwatch
46bebb3317
[erigon2.2] Add ReadIndices aggregator to collect data (#500)
* [erigon2.2] Add ReadIndices aggregator to collect data

* Try

* Fix for history access

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-20 08:39:29 +01:00
Alex Sharov
5574d68a87
save (#499) 2022-06-19 06:35:29 +01:00
Giulio rebuffo
addc05bae3
Plaxeholder values for dbsize and pagesize 2022-06-19 00:46:31 +02:00
ledgerwatch
234be664fc
Optimise history access for multiple files (#498)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-18 22:54:36 +01:00
Giulio rebuffo
d57a16c7f6
Implemented ForAmount and MDBX sequencing (#497) 2022-06-18 21:24:30 +02:00
Giulio rebuffo
988a4b5510
Added update txn function 2022-06-17 23:59:04 +02:00
ledgerwatch
945b0e9e0f
Fix merge of code files (#495)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 19:24:56 +01:00
ledgerwatch
d7693ce094
[txpool] Prevent clogging up the pending sub pool (#494)
* Debug txpool Best

* Print Best result

* Print

* Trace removeMined

* Remove mined tx from pending subpool

* Remove debug

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 15:46:01 +01:00
ledgerwatch
df49481ddc
[erigon 2.2] Make keys always uncompressed, values compressed only for code (#492)
* Reduce allocations in domain and aggregator

* Make keys always uncompressed, values compressed only for code

* Functions to remake index

* Fix index recreation

* Test for reindex, fix

* Use uncompress vals in history

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 12:39:49 +01:00
Giulio rebuffo
280c5e9dcc
Added memory mutation to Erigon lib (#493) 2022-06-16 08:01:48 +01:00
ledgerwatch
bbf96d0580
Close compressor (#491)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-14 22:38:18 +01:00
ledgerwatch
a450f5c341
MDBX patch for txid to try to reduce GC work on commit (#490)
* MDBX patch for txid to try to reduce GC work on commit

* Update to mdbx-go master

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-14 11:42:14 +01:00
ledgerwatch
e2c6ef0058
[erigon2.2] Fixes for inverted indices and domains for the prototype (#489)
* Better control of compress/uncompressed

* Add new function

* more careful pruning

* Printf

* Printf

* Fix DupSort

* Remove copying in prune

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-13 19:32:13 +01:00
Giulio rebuffo
dba877998b
Fix pool giving too many entries (#488) 2022-06-13 09:00:31 +01:00
ledgerwatch
6cad65e62b
[erigon2.2] Parallel build files and merge, change file names (#487)
* Parallel build files and merge, change file names

* Update ci.yml

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-12 10:14:18 +01:00
ledgerwatch
45d4c21490
Expose inverted index ranges in aggregator (#486)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-11 12:09:24 +01:00
ledgerwatch
7ce8bd589f
[erigon 2.2] Add functions for traces and event logs (#485)
* Add functions for traces and event logs

* Add functions for traces and event logs

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-10 06:51:00 +01:00
ledgerwatch
74ea75f9b8
[erigon2.2] Merge fixes, add historical access (#482)
* Merge fixes, add historical access

* Change API from AfterTxNum to BeforeTxNum

* Change API functions

* Change API functions

* Print

* Fix for non-existent items

* Remove prints

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-09 14:48:16 +01:00
Artem Tsebrovskiy
f8bdadf3e0
HPH with direct reading from state by plainKey (#472)
* dirty trie with direct reading of account/storage data from state

run with fixes

implemented trie with direct reading from state

* cleaner version without updates
2022-06-09 13:46:11 +01:00
Alex Sharov
e4b9cc3884
Reclaimable space in logs (#484)
* save

* save

* save

* save

* save

* save
2022-06-09 13:37:28 +01:00
Alex Sharov
19086759eb
methods to configure db #483 2022-06-09 09:43:12 +07:00
leonardchinonso
13529121f3
Chenged lowercase b meaning bits to uppercase B meaning Bytes (#481) 2022-06-08 19:19:17 +07:00
Andrew Ashikhmin
5278815cd0
Start txpool in AddLocalTxs (#480)
* Start txpool in AddLocalTxs

* Correct tx in fromDB called by AddLocalTxs
2022-06-08 10:39:22 +02:00
ledgerwatch
a77e6425eb
Fixes for the Erigon 2 upgrade 2 prototype (#479)
* Print

* Remove print

* Remove print

* Fix one panic

* Fix duplicate collation

* Print

* Fix print

* fix maxSpan

* Reduce maxSpan

* Remove duplicate join

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 22:32:34 +01:00
ledgerwatch
f16b285631
Adjustments for erigon 2 upgrade prototype (#477)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 16:34:38 +01:00
Alex Sharov
fdf7c6598b
compress.Count() method (#478) 2022-06-03 12:14:58 +07:00
ledgerwatch
157b4299e4
[erigon2] Continuation on domains and inverted indices, putting things together (#476)
* Add scan files tests, create new aggregator type

* Fix lint

* windows test fix

* Add delelte test

* AggCollation

* More functions to Aggregator

* More aggregator functions

* Update

* More functions

* More functions

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-02 21:40:58 +01:00
ledgerwatch
c5a10975ab
[erigon2] Introduce inverted index type (#473)
* [erigon2] Introduce inverted index type

* More inverted index code

* More tests for inverted index

* Think about public and non-public APIs

* Minimise DB access when accessing history

* Work on iterator

* Implementation of inverted iterator

* Test for inverted index

* Assert end of iterators

* Merge of inverted index files and test

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-05-31 18:42:04 +01:00
Alex Sharov
7e00e77ae5
match roaring version with erigon #475 2022-05-30 09:17:47 +07:00
Alex Sharov
b995bd7540
recsplit: configurable etl limit #474 2022-05-30 09:06:11 +07:00
ledgerwatch
990e586823
[erigon2] Continuation of work on domains - merge static files (#466)
* Iteration over files - initial

* Fix interator for multistep

* Add function

* More functions for merge

* Merge files

* More work on the merge

* Fix buildIndex

* Fix history test for test of not completely pruned db

* Prepare for merge test

* Merge file test

* Close files

* Move functions into separate file

* Print

* fix for closing index

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alex Sharp <alexsharp@alexs-mbp.lan>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-29 19:57:09 +01:00
Artem Tsebrovskiy
56a6afc8c9
commitment: fixed hash evaluation for hex patricia trie (#471) 2022-05-28 07:52:24 +01:00
Artem Tsebrovskiy
49e3522a05
added print of decompressed file at panic (#468)
* added print of decompressed file at panic

* more info for recovered decompressing
2022-05-27 08:20:53 +07:00
Andrew Ashikhmin
28d6b37a22
Remove INVALID_TERMINAL_BLOCK (#470)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 981f4ba

git-subtree-dir: interfaces
git-subtree-split: 981f4bae6619ab7100c7222852fd4eae3f837ad4

* regenerate go proto
2022-05-26 17:01:43 +02:00
battlmonstr
8c81bd094c
sentry: grpc.ClientStream.RecvMsg for direct sentry streams (#469) 2022-05-26 20:53:16 +07:00
battlmonstr
da10d41489
sentry: PeerById method (#467) 2022-05-25 19:50:19 +07:00
Alex Sharov
dd8136cde9
save (#465) 2022-05-24 19:00:09 +01:00