Commit Graph

923 Commits

Author SHA1 Message Date
Kairat Abylkasymov
ac06fd9400 local dev setup 2022-02-24 06:15:14 -05:00
Alex Sharov
e3e108c6c4
logger version up (#347) 2022-02-24 15:14:37 +07:00
Alex Sharov
3205770ee0
snapshots: fix test (#346) 2022-02-24 08:35:13 +07:00
ledgerwatch
27a61d4611
[txpool] Poking only caused by local transactions (#345)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-24 00:18:46 +00:00
ledgerwatch
efdeb69af0
[txpool] Poke transactions that cannot be replaced (#343)
* Poke transactions that cannot be replaced

* Remove printing

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-23 23:48:05 +00:00
ledgerwatch
50082e3553
[txpool] Remove unsafe parameter for onSenderStateChange (#341)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-22 08:53:29 +00:00
ledgerwatch
c71ac02a0f
[erigon2] Optimisations in etl collector and compressor (#339)
* Optimisations in etl collector and compressor

* Not copy k and v in the collector

* Fix lint

* Optimisations

* Change Load1 back to Load

* Reduce allocations for tests

* preallocate inv

* counting hits and misses

* Try to fix

* Try to fix

* Relaxation 1

* Relaxation 2

* Add arch tables

* Fix

* Update arch tables and use them

* Not to override larger value

* Increase arch table size

* Increase arch table size

* Fixes to arch

* Print

* Off by one

* Print

* Fix

* Remove print

* Perform update of arch in the background

* Build up huffman tree

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-20 22:14:06 +00:00
EXEC
143cd510bd
Fix typo in README.md (#340)
`currenk` -> `current`
2022-02-20 14:12:06 +00:00
ledgerwatch
91c51b20ad
[erigon2] History reader fixes (#338)
* More tracing for history

* More tracing for history

* More tracing for history

* only under tracing

* Trace

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-18 22:19:38 +00:00
Alex Sharov
c1c45b94bf
last version of kanzi-go dep (#337) 2022-02-18 10:11:29 +07:00
Alex Sharov
7747be5228
more stable grpc err check (#336) 2022-02-18 09:54:27 +07:00
Alex Sharov
5be29adf1a
Grpc err utils (#334)
* save

* save
2022-02-18 09:40:11 +07:00
Alex Sharov
d94c4ada78
dir package - to enforce dir RW lock by types (#332)
* save

* save
2022-02-18 09:24:30 +07:00
Alex Sharov
d309d52fe2
rpcdaemon embed fixes (#333) 2022-02-17 11:24:13 +07:00
alex.sharov
a2ca4cf55f save 2022-02-17 10:10:23 +07:00
ledgerwatch
43b38539b3
[erigon2] Remove empty commitments and add comments (#331)
* Remove empty commitments and add comments

* Remove logging

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-16 22:38:37 +00:00
ledgerwatch
1c6e82c2b6
[erigon2] Thin commitment (2nd attempt) (#329)
* Another fix for history files

* Half way through

* Another fix

* Correct closing sequence

* Remove first byte insert marker

* More on think commitments

* Fixes

* Fixes

* Print

* Skip touchMap

* Merge branchData from trees and from files

* Fill branch commitment

* Fill branch commitment

* Print

* Fix?

* Merge branchData when updating in the tree

* Better panic

* Prints

* Prints

* Prints

* Create complete branch data if it did not exist before

* Cleanup printing

* Fix merge

* Fix merge use

* Fix transform

* Better startBlock panic

* Preserve touchMap

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments during aggregation

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Merge commitments

* Include fieldbits during transform

* Fix history reads

* Print

* Print

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Prints

* Print largestMerge, lock files in branchFn

* Add storage lock

* Prints

* prefixLen fix

* prefixLen fix

* Fixes

* Remove print

* Remove print

* Set changesets and commitments flags upfront

* Logging instead of printing

* Fix history merge, recsplit panic

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-16 16:44:00 +00:00
Alex Sharov
0c69ccd29d
rpcdaemon embed (#330)
* save

* save

* save
2022-02-16 10:56:50 +07:00
Alex Sharov
9789f53041
aggregator: close idx after build (#327) 2022-02-14 18:38:39 +00:00
Alex Sharov
1f5a1ab9cd
fuzz cases (#328) 2022-02-14 11:53:20 +07:00
ledgerwatch
1363859958
[erigon2] Fix history bug, optimise history search (#326)
* Another fix for DeleteAccount

* Optimise bitmap search

* Optimise bitmap search

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-13 18:56:36 +00:00
Alex Sharov
79aa17d297
Recsplit: use crypto rand seed if user not set (#325) 2022-02-13 16:14:04 +07:00
Alex Sharov
39566b5e50
return EOF on direct stream end (#324)
* save

* save
2022-02-13 09:33:54 +07:00
ledgerwatch
d9367b582a
[erigon2] fix change files for commitments (#318)
* fix change files for commitments

* Not create commitment files unless enabled

* Print loaded files

* More fixes to commitment changes

* More fixes to commitment changes

* Fix for DeleteAccount storage values

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 14:51:23 +00:00
Alex Sharov
6f85066c7e
path -> filepath (path package is for urls) (#321) 2022-02-12 20:11:30 +07:00
alex.sharov
b041b959ca direct: txpool, mining 2022-02-12 19:33:47 +07:00
Alex Sharov
31982dacc8
direct: more compact naming (#320) 2022-02-12 18:48:42 +07:00
Alex Sharov
51dda9de27
direct: txpool, mining (#319) 2022-02-12 18:41:17 +07:00
Alex Sharov
e16bce5233
eth backend: direct (#317) 2022-02-12 16:55:33 +07:00
Alex Sharov
ba6af37677
eth backend: direct (#316)
* eth backend: direct

* eth backend: direct
2022-02-12 16:50:06 +07:00
Alex Sharov
57d026dcca
grpc: disabled server txpool (#314)
* grpc disabled server txpool

* grpc disabled server txpool
2022-02-12 12:24:17 +07:00
ledgerwatch
5f7283d0f1
[erigon2] History reader (#312)
* Initial code for history reader

* Print

* Print

* Fix npe

* Open files of all types

* don't close file in the constructor

* Print foundTxNum

* Print foundTxNum

* Print adding to bitmap

* Print when reducing

* Print when reducing

* Print when reducing

* Enum false for index

* Remove prints

* Remove prints

* Print loaded files

* Tracing

* Tracing

* Tracing

* Tracing

* Tracing

* Optinal commitments

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-11 23:44:30 +00:00
Alex Sharov
7aec036bdd
remotedb server: ro db (#313) 2022-02-11 16:02:57 +07:00
ledgerwatch
441a4c3cde
[erigon2] Chain history and bitmap indices - part 2 (#308)
* corretly shutdown history goroutine

* Different final merge for history files

* Skip value

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Bitmap production bug

* Handle collision

* Handle collision

* Debug

* Debug

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-10 15:30:55 +00:00
primal_concrete_sledge
c4805e0262
WIP: issue/issue-281-create_binding_to_pedersen_hash (#301)
* issue/issue-281-create_binding_to_pedersen_hash

* Add //nolint

* Add more nolints

* move nolint

* Remove nolit

* Add gcc install

* Upd .ci

* Remove staticcheck

* Add envs

* try to exclude pedersen_hash from test

* try to fix mac os build

* Add include for mac os

* Add include for mac os

* Fix runner_os

* remove test for macos

* Change restrictions

* restrict tests to ubuntu

* Try test windows

* Add build constraint
2022-02-10 14:47:28 +00:00
Alex Sharov
3c4f8a759c
recsplit: reset offset collector, etl: faster flush and load (#310) 2022-02-10 14:40:24 +07:00
Alex Sharov
8e4618c57d
state and newHead: 1 goroutine per subscriber #309 2022-02-10 08:26:08 +07:00
Andrew Ashikhmin
2e162c70ba
Update Merge Engine API (Kiln v1) (#268)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 2daa726

git-subtree-dir: interfaces
git-subtree-split: 2daa72683ceaacfe0476005c11a6fc7b6d945780

* Update Merge Engine API

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 0e25bcc

git-subtree-dir: interfaces
git-subtree-split: 0e25bcc642c90fcc63e12930032a7e0f2260e865

* Engine API updated

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit ee8057d

git-subtree-dir: interfaces
git-subtree-split: ee8057d3e8ab57059fac55434f3f55d189b00242

* Add INVALID_TERMINAL_BLOCK to EngineStatus

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 50ece30

git-subtree-dir: interfaces
git-subtree-split: 50ece307faac31b7d7fe9cd06e11ed82c09064b9
2022-02-09 07:49:31 +01:00
Alex Sharov
e649f7ea91
Less alloc etl recsplit (#307)
* less allocs recsplit

* save

* save
2022-02-09 13:22:45 +07:00
ledgerwatch
f5deff3ac9
[erigon2] accumulating chain history and bitmap indices (#306)
* Aggregator refactoring

* Restore the test

* Simplify commitment

* Lint

* onBefore

* Wip smaller commitments

* Fixes

* Fix lint

* Check for zero len

* Print

* Remove restriction

* Correct restrictions

* Correct restrictions

* More merging

* delBitmap

* Switch off merging

* Reintroduce background merge

* Try to fix

* Slight optimisation

* Produce initial changeset files

* Print changeset keys

* Print changeset keys

* Print changeset keys

* Print changeset keys

* Print changeset keys

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

* Merge for history and bitmaps

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-08 21:19:29 +00:00
ledgerwatch
92fc7215c8
[erigon2] fix commitment error (#304)
* Try to fix

* Fix

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-04 18:48:50 +00:00
Alex Sharov
567d9ddfed
ParallelCompressor: Remove intermediate ETL collectors (#302) 2022-02-04 16:48:02 +07:00
alex.sharov
d4b9053aed save 2022-02-04 11:20:02 +07:00
ledgerwatch
55080d5c01
Proper reset of decompressor getter (#299)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 17:58:56 +00:00
ledgerwatch
11615db276
Fix memory leak in txpool (#298)
* Fix memory leak in txpool

* Cannot delete during iteration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 17:52:20 +00:00
alex.sharov
d814adb13e save 2022-02-03 16:28:43 +07:00
alex.sharov
02c407a14e save 2022-02-03 16:28:32 +07:00
ledgerwatch
cbfd733672
[erigon2] optimisations (#297)
* Reuse getter and indexReader

* Properly reset Getter

* Use separate getter and indexReader for merge thread

* Remove DB

* Fixes

* Print

* Fixes

* Cleanup signatures

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 08:49:47 +00:00
Alex Sharov
ec354d1615
Fuzz fixes (#295)
* fuzz tests fixes

* fuzz tests fixes
2022-02-02 09:18:04 +00:00
Alex Sharov
a53f232322
pool: fix race in async broadcast (#294)
* pool: better async broadcast

* save

* save
2022-02-02 14:49:26 +07:00