Commit Graph

805 Commits

Author SHA1 Message Date
ledgerwatch
71751426bc
Fix windows CI for go1.18 (#383)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-19 12:37:03 +00:00
ledgerwatch
2b14d6da29
[erigon2] debug delays in the main thread (#379)
* Split handover time

* Clean up

* Correctly compute handover time

* not use compression when aggregating

* Fix for etl collector

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-19 12:02:27 +00:00
EXEC
f739b43d8a
Fix OpenFile permission (#382)
`0o755` -> `0755`
2022-03-19 10:50:08 +00:00
Alex Sharov
83951a1d62
Enable more linters (#381) 2022-03-19 11:38:37 +07:00
Alex Sharov
06713f8c96
linter to support go 1.18 (#380)
* save

* save
2022-03-19 10:02:28 +07:00
ledgerwatch
ff3e074549
No handover (#378)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-18 16:09:57 +00:00
ledgerwatch
f93ea948d0
[erigon2] Optimise Huffman decoder (#374)
* Update

* Intermediate

* Huffman decoding

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-18 09:10:18 +00:00
Alex Sharov
1b0bcd89fa
snapshots: grpc event (#377) 2022-03-18 14:47:39 +07:00
Alex Sharov
758de3929f
go 1.18 on ci (#375)
* save

* save
2022-03-18 11:27:44 +07:00
Alex Sharov
e8e8e2f2e6
Pool cost (#373)
* save

* save
2022-03-17 20:16:57 +00:00
Alex Sharov
2193d570a6
Mdbx: WriteMap fallback on open error (#369)
* save

* save
2022-03-16 11:59:11 +00:00
Alex Sharov
05b59ea524
use same version of golang.org/x/crypto with erigon #368 2022-03-16 11:30:33 +07:00
Giulio rebuffo
9b99057af4
fixed bug for h2048 (#367) 2022-03-16 09:59:25 +07:00
Artem Tsebrovskiy
6687e6b4d6
during computeCommitment exits with RootHash when there are no updates to apply (#366) 2022-03-15 17:01:51 +00:00
ledgerwatch
77eb94b53e
Elias fano search and merge (#357)
* Elias fano search and merge

* Add first cut of search

* Iterator and test

* Changes in aggregator

* Elias fano bitmap

* Fix uncompress decompress

* Print

* Print

* No print

* Print

* Print

* Print

* Change to AppendBytes

* Print

* Fix NextUncompressed

* Remove print

* Fix history search

* Fix in history search

* More tracing

* More tracing

* Fix

* Print

* Print key

* More print

* Print

* No deletion for history records

* Remove print

* Fix

* Fix

* Fix test

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-13 22:46:17 +00:00
Alex Sharov
c1f1365f92
cancel compress (#362) 2022-03-12 16:34:58 +07:00
Alex Sharov
c0fcdabf91
compress: less allocs (#361) 2022-03-12 15:33:01 +07:00
ledgerwatch
d3e05fb352
Subscriptions for logs (#279)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 4b4b765

git-subtree-dir: interfaces
git-subtree-split: 4b4b76598f956abcdeae67c50b7c37f055d11b17

* Regenerate bindings

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit a0b1e1c

git-subtree-dir: interfaces
git-subtree-split: a0b1e1cdbce09fd62c369f25e8ef35cf3972357e

* Rebuild gointerfaces

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit eea9297

git-subtree-dir: interfaces
git-subtree-split: eea92977065e217c385db8eefe261671fb72c529

* Update to new interfaces

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 548970d

git-subtree-dir: interfaces
git-subtree-split: 548970d40b288ed7ef80412e5fdb1144c8ffe129

* Update to gointerfaces

* Update pb

* Add subslogs

* Upd pb

* Add direct for logs

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Andrey Rybnov <ryban92@gmail.com>
2022-03-11 16:33:47 +00:00
Alex Sharov
fb9214d8bb
lint up (#360) 2022-03-11 08:49:54 +07:00
Enrique Jose Avila Asapche
c82e350e63
fixed marshalling error with nil (#354)
* fixed marshalling error with nil

* returning empty byte slice

* returning empty byte slice pt2
2022-03-10 20:24:04 +00:00
battlmonstr
3123b6d895
kv: add NodeRecords nodedb table (#359) 2022-03-10 13:15:15 +01:00
battlmonstr
f54e4c7eae
kv: use SetSyncPeriod from mdbx-go v0.22.16 (#358) 2022-03-10 13:04:08 +01:00
Alex Sharov
6512e3c941
add emptyWordsCount field to .seg file header (breaking .seg format) (#355)
* up torrent

* save

* save

* save

* save

* save

* save

* save
2022-03-10 07:48:37 +00:00
battlmonstr
009358ab2f
kv: expose mdbx SafeNoSync and OptSyncPeriod options (#356) 2022-03-10 10:25:11 +07:00
ledgerwatch
75b52ac25e
[compress] Allow uncompressed words (#350)
* Intermediate work

* Allow uncompressed words

* Fix

* Fix tests

* Add NextUncompressed, remove g.word buffer

* Code simplifications, no goroutines when workers == 1

* Fix lint|

* Add test for MatchPrefix

* Work on patricia

* Beginning of new matcher

* Fuzz test for new longest match

* No skip

* Fixes

* Fixes

* More tracing

* Fixes

* Fixes

* Change back to old FindLongestMatches

* Switch to old match finder

* Print mismatches

* Fix

* After fix

* After fix

* After fix

* Print pointers

* Fixes and tests

* Print

* Print

* Print

* More tests

* Intermediate

* Fix

* Fix

* Prints

* Fix

* Fix

* Initialise matchStack

* Compute only once

* Compute only once

* Switch back

* Switch to old Find

* Introduce sais

* Switch patricia to sais

* Use sais in compressor

* Use sais in compressor

* Remove unused code

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-09 17:25:22 +00:00
Alex Sharov
a2cbecf373
[wip] KV: limit amount of parallel read txs (#352)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* save

* save

* save

* save

* save
2022-03-07 12:21:59 +00:00
Alex Sharov
56d528b1f0
save (#353) 2022-03-04 09:13:15 +07:00
Alex Sharov
ebfbf92936
mdbx_v0.11.5 (#351) 2022-02-25 07:42:48 +00:00
Giulio rebuffo
2fc62cd2f5
prevrandao (#349) 2022-02-24 16:18:01 +01:00
racytech
7763945374
Reverted 3 last commits (#348)
* Revert "unnecessary includes removed"

This reverts commit 76406bb78b.

* Revert "local dev setup"

This reverts commit ac06fd9400.

* Revert "compress/cgo-addition"

This reverts commit fae7683d46, reversing
changes made to e3e108c6c4.
2022-02-24 14:39:42 +00:00
Kairat Abylkasymov
fae7683d46 compress/cgo-addition 2022-02-24 06:28:01 -05:00
Kairat Abylkasymov
76406bb78b unnecessary includes removed 2022-02-24 06:21:25 -05:00
Kairat Abylkasymov
ac06fd9400 local dev setup 2022-02-24 06:15:14 -05:00
Alex Sharov
e3e108c6c4
logger version up (#347) 2022-02-24 15:14:37 +07:00
Alex Sharov
3205770ee0
snapshots: fix test (#346) 2022-02-24 08:35:13 +07:00
ledgerwatch
27a61d4611
[txpool] Poking only caused by local transactions (#345)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-24 00:18:46 +00:00
ledgerwatch
efdeb69af0
[txpool] Poke transactions that cannot be replaced (#343)
* Poke transactions that cannot be replaced

* Remove printing

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 23:48:05 +00:00
ledgerwatch
50082e3553
[txpool] Remove unsafe parameter for onSenderStateChange (#341)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-22 08:53:29 +00:00
ledgerwatch
c71ac02a0f
[erigon2] Optimisations in etl collector and compressor (#339)
* Optimisations in etl collector and compressor

* Not copy k and v in the collector

* Fix lint

* Optimisations

* Change Load1 back to Load

* Reduce allocations for tests

* preallocate inv

* counting hits and misses

* Try to fix

* Try to fix

* Relaxation 1

* Relaxation 2

* Add arch tables

* Fix

* Update arch tables and use them

* Not to override larger value

* Increase arch table size

* Increase arch table size

* Fixes to arch

* Print

* Off by one

* Print

* Fix

* Remove print

* Perform update of arch in the background

* Build up huffman tree

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-20 22:14:06 +00:00
EXEC
143cd510bd
Fix typo in README.md (#340)
`currenk` -> `current`
2022-02-20 14:12:06 +00:00
ledgerwatch
91c51b20ad
[erigon2] History reader fixes (#338)
* More tracing for history

* More tracing for history

* More tracing for history

* only under tracing

* Trace

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-18 22:19:38 +00:00
Alex Sharov
c1c45b94bf
last version of kanzi-go dep (#337) 2022-02-18 10:11:29 +07:00
Alex Sharov
7747be5228
more stable grpc err check (#336) 2022-02-18 09:54:27 +07:00
Alex Sharov
5be29adf1a
Grpc err utils (#334)
* save

* save
2022-02-18 09:40:11 +07:00
Alex Sharov
d94c4ada78
dir package - to enforce dir RW lock by types (#332)
* save

* save
2022-02-18 09:24:30 +07:00
Alex Sharov
d309d52fe2
rpcdaemon embed fixes (#333) 2022-02-17 11:24:13 +07:00
alex.sharov
a2ca4cf55f save 2022-02-17 10:10:23 +07:00
ledgerwatch
43b38539b3
[erigon2] Remove empty commitments and add comments (#331)
* Remove empty commitments and add comments

* Remove logging

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 22:38:37 +00:00
ledgerwatch
1c6e82c2b6
[erigon2] Thin commitment (2nd attempt) (#329)
* Another fix for history files

* Half way through

* Another fix

* Correct closing sequence

* Remove first byte insert marker

* More on think commitments

* Fixes

* Fixes

* Print

* Skip touchMap

* Merge branchData from trees and from files

* Fill branch commitment

* Fill branch commitment

* Print

* Fix?

* Merge branchData when updating in the tree

* Better panic

* Prints

* Prints

* Prints

* Create complete branch data if it did not exist before

* Cleanup printing

* Fix merge

* Fix merge use

* Fix transform

* Better startBlock panic

* Preserve touchMap

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Include fieldbits during transform

* Fix history reads

* Print

* Print

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Print largestMerge, lock files in branchFn

* Add storage lock

* Prints

* prefixLen fix

* prefixLen fix

* Fixes

* Remove print

* Remove print

* Set changesets and commitments flags upfront

* Logging instead of printing

* Fix history merge, recsplit panic

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-16 16:44:00 +00:00
Alex Sharov
0c69ccd29d
rpcdaemon embed (#330)
* save

* save

* save
2022-02-16 10:56:50 +07:00