mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-25 13:07:17 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
12,762 Commits 5 Branches 14 Tags 294 MiB
eb5e01ff5d
Commit Graph

356 Commits

Author SHA1 Message Date
Alexey Akhunov
eb5e01ff5d Do not cancel context when closing remote cursor 2020-10-06 17:11:30 +01:00
ledgerwatch
b9aec74e4b
Kludge fix for node being stuck (#1188) 2020-10-06 07:36:34 +01:00
Evgeny Danilenko
ec6be820ac
Don't force sync in case InMemDb (#1174) 
* initial

* fix

* fix chain length
 2020-10-02 14:08:28 +01:00
Alex Sharov
6ccf57dace
switch to go implementation of roaring bitmaps for alpine support (#1165) 2020-10-02 15:16:21 +07:00
Alex Sharov
61fbe34024
Better cbor support (#1163) 2020-10-02 10:56:13 +07:00
Alex Sharov
48cf5fb43e
History-style sharding for bitmap indices (#1159) 2020-10-02 10:54:11 +07:00
Alex Sharov
0559d56dae
Pin protoc version and store it in ./build/bin (#1154) 
* use ./build/bin folder to store protobuf binaries

* use ./build/bin folder to store protobuf binaries
 2020-09-29 15:09:50 +01:00
ledgerwatch
7bb9cb4274
Remove BloomStatus from the API (#1155) 
* Remove BloomStatus from the API

* Trying to find missing space

* Fix lint
 2020-09-29 15:09:29 +01:00
Alex Sharov
e02d6acc7d
bitmap indices for logs (#1124) 
* save progress

* try now

* don't create bloom inside rlpDecode

* don't create bloom inside ApplyTransaction

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* rename method

* print timings

* print timings

* print timings

* sort before flush

* fix err lint

* clean

* move tests to transactions

* compressed version

* up bound

* up bound

* more tests

* more tests

* more tests

* more tests

* better removal

* clean

* better performance of get/put methods

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* optimize rpcdaemon

* fix test

* fix rpcdaemon

* fix test

* simplify

* simplify

* fix nil pointer

* clean

* revert some changes

* add some logs

* clean

* try without optimize

* clean

* clean

* clean

* clean

* try

* move log_index to own stage

* move log_index to own stage

* integration add log_index stage

* integration add log_index stage

* clean

* clean

* print timing

* remove duplicates at unwind

* extract truncateBitmaps func

* try detect

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* add blackList of topics

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* clean

* sharding 1

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 2

* sharded 3

* sharded 3

* sharded 3

* speedup things by putCurrent and putReserve

* clean

* optimize trim

* clean

* remove blacklist

* add more info to err

* ?

* clean

* clean

* clean

* clean

* clean

* working version

* switch to cgo version of roaring bitmaps

* clean

* clean

* clean

* clean

* more docs

* clean

* clean

* fix logs bloom field

* Fix debug_getModifiedAccountsByNumber

* Try to fix crash

* fix problem with "absent block"

* fix problem with "absent block"

* remove optimize method call

* remove roaring iterator

* fix problem with rebuild indicess

* remove debug prints

* tests for eth_getLogs involving topics

* add tests for new stage, speparate topics into 2 buckets

* version up

* remove debug logs

* remove debug logs

* remove bloom filter implementation

* Optimisation

* Optimisatin not required, make rpctest lenient to geth errors

* Lenient to geth failures

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-28 18:18:36 +01:00
Alex Sharov
e2acbfecda
try to remove sleep (#1126) 2020-09-23 12:10:13 +01:00
Giulio rebuffo
d33302c2be
Mutual TLS authentication (for now without verifying the Common Name) (#1113) 
* added tls auth

* added client side

* put --tls

* fixed flag

* Add key/cert generation instructions, turn off Common Name verification

* Add CLI arguments and Warning

* Lint

* Update the doc about Internal IP

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-19 15:16:04 +01:00
ledgerwatch
0f24a01677
Close the ticker (#1104) 2020-09-11 21:44:24 +01:00
Giulio rebuffo
23d89c0941
Added TLS handshake to RPCDaemon (#1089) 
* added eth_getStorageAt

* used uint32

* now its 256

* incarnation

* added TLS handshake

* code minified

* lint

* minified client

* Update flags.go

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
 2020-09-11 21:17:37 +01:00
Alex Sharov
b76661cd25
Revert "integration_write_receipts (#1096)" (#1097) 
This reverts commit 7a1285bf3c.
 2020-09-11 15:32:45 +07:00
Alex Sharov
7a1285bf3c
integration_write_receipts (#1096) 2020-09-11 15:28:28 +07:00
Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support (#1080) 
* etl.Loader - allow use of custom comparator

* log timing

* try now

* try now

* more performance

* etl.Loader - allow use of custom comparator

* working version

* simplify IH cursor

* clean

* squash

* squash

* squash

* squash

* squash

* squash

* squash

* clean

* add only unwind support

* squash

* squash

* clean

* fix test

* clean

* clean

* clean
 2020-09-10 13:35:58 +01:00
b00ris
bdb7832a59
[lmdb] Stop force sync for ReadOnly mode (#1088) 
* fix warning

* fix log
 2020-09-09 21:07:19 +01:00
Alex Sharov
fd9c7c3b2d
kill bolt (#1085) 
* kill bolt

* kill bolt

* fix test

* clean
 2020-09-09 11:09:55 +01:00
Alex Sharov
da31494cc3
transactional migrations (#1079) 2020-09-08 20:39:43 +01:00
Alex Sharov
c45a710ce6
Warmup logs, less overhead, warmup code bucket (#1054) 
* warmup logs and less overhead.

* warmup logs and less overhead.

* move WarmUp to common func
 2020-09-08 08:28:37 +01:00
b00ris
23e995838f
Fix readonly mode (#1077) 
* fix readonly mode

* fix vet
 2020-09-08 07:19:06 +01:00
Alex Sharov
afe808c88e
time based reopen tx (#1069) 2020-09-07 12:47:08 +07:00
ledgerwatch
3a92b2b39d
Fix for RPC daemon leak (#1059) 
* Start memory prof

* Fix rpctest

* Fix rpctest

* Attempt to fix the leak

* Remove http pprof
 2020-09-05 21:58:51 +01:00
Alex Sharov
e690103c2a
Docs for Cursors and KV (#1046) 2020-09-05 14:26:24 +07:00
Alex Sharov
7e71c4512e
restrict metric by 1 bucket (#1052) 2020-09-05 13:09:47 +07:00
Alex Sharov
95e985ef8f
add commit event to dashboard (#1050) 2020-09-05 10:10:17 +07:00
Alex Sharov
0c75304261
Os page cache monitoring (#1049) 2020-09-05 09:36:42 +07:00
Alex Sharov
b9bdd003a7
KV: Raw (no custom logic) cursors for DupSort and DupFixed buckets (#1020) 2020-09-04 10:54:15 +07:00
Evgeny Danilenko
e4f495fa44
Get logs (#1028) 
* it compiles

* after recent master

* fix linters warnings

* grpcV7

* go mod tidy

* unmarshall adresses or adress

* fix linters

* after cr

* after cr

* after cr

* after cr

* fix tests

* remove dev version

* it compiles

* mod tidy

* fix bin deps

* use stable version of grpc

* switch back to master constructor

* switch back to master constructor

* add a bit docs

* add a bit docs

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
 2020-09-03 08:51:19 +01:00
Alex Sharov
e02f2f6dfc
Revert "simplify cursor.Next and cursor.First implementation (#1019)" (#1039) 
This reverts commit f19bb0a345.
 2020-09-03 13:21:11 +07:00
Alex Sharov
f19bb0a345
simplify cursor.Next and cursor.First implementation (#1019) 2020-09-01 12:50:23 +01:00
ledgerwatch
c06dbdadf9
Use Append for changesets and receipts (#1018) 
* Use Append for changesets and receipts

* Fix lint

* Delete all receipts on unwind

* Fix lint

* Fix unwind keys for changeset

* Fix test

* Use writeReceipts flag

* Revert to Append

* Delete receipts too when resetting
 2020-09-01 07:48:25 +01:00
Alex Sharov
49351b0091
KV: to support multiple databases and multiple set of buckets (#1002) 
* don't use global variables

* up deps

* clean

* use joined flags

* use single non-global buckets config

* clean

* copy map

* fix tests

* fix tests

* clean

* clean
 2020-08-30 18:34:18 +01:00
Alex Sharov
5ce73d438a
fix integration (#999) 2020-08-29 15:27:20 +07:00
Alex Sharov
07ca5c643a
eth_syncing (#991) 
* eth_syncing

* linters

* fix readme

* cleanup cli params

* go mod tidy

* remove memsize ui dep

* remove bloomfilter

* implement net_peerCount

* remove legacy flags
 2020-08-29 08:24:50 +01:00
Alex Sharov
df05a581dd
Add tx pool stage to transaction (#981) 
* add tx_pool stage to tx

* clean

* don't call commit in read tx

* don't call commit in read tx

* clean

* more logs

* fix tx pool race

* more logs

* clean
 2020-08-28 07:27:48 +01:00
Alex Sharov
714e5041e2
Better logs3 (#976) 2020-08-26 19:06:51 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975) 
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
 2020-08-26 07:03:50 +01:00
Alex Sharov
522287ac18
Transactional cycle (#966) 
* v0

* v1

* v3

* v4

* clean

* temporary fix of txpool

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* add condition

* tx pool to not hold own db

* try to enable TxPool in integration

* exclude tx pool from tx

* exclude tx pool from integration

* reduce limit

* fix integration

* clean

* clean

* clean

* clean

* clean

* exclude tx pool unwind

* exclude tx pool unwind in integration

* fix integration tx pool

* fix commit

* fix current stage after unwind

* fix current stage after unwind

* fix linter

* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.

* move body and headers unwind out of tx

* fix unwind order after reboot

* add support external tx to exec stage

* clean

* clean

* clean

* clean

* clean

* add logs

* better id check

* better id check
 2020-08-26 07:02:10 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970) 
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
 2020-08-24 18:00:40 +01:00
Alex Sharov
43a4e34a6e
[to discuss] exec blocks without ObjectDb: Mutation over TxDb (#947) 
* use mutation over tx

* clean

* clear

* add .CommitAndBegin() method

* clean

* increase timings for logging

* return ideal batch size
 2020-08-24 12:07:59 +01:00
Alex Sharov
a6be18b915
ticker-based logs (#954) 
* timer-based logs

* timer-based logs

* delegate progress calculation to user

* delegate progress calculation to user

* delegate progress calculation to user

* clear

* add logs to senders recovery

* use default dir in integration

* more logs

* more logs
 2020-08-22 12:12:33 +02:00
Alex Sharov
ebbaa55672
sub-transaction to not call fsync, sub-transactions to not call runtime.LockThread() (#945) 2020-08-21 07:31:43 +01:00
ledgerwatch
01f211f3b8
Revert "IH: no retain list (#946)" (#948) 
This reverts commit f86ed64288.
 2020-08-20 13:37:34 +01:00
Alex Sharov
f86ed64288
IH: no retain list (#946) 
* experiment of not using retain list

* better handle sub-tx

* save progress

* don't touch collector

* don't run collector on sub-tx because of `mdb_cursor_put: MDB_TXN_FULL: Transaction has too many dirty pages - transaction too big`

* clear

* clear
 2020-08-20 11:42:43 +01:00
b00ris
eb46dd68df
Account range testing (#938) 
* save state

* save state

* save new timeouts

* compare passed

* save state

* cleanup
 2020-08-19 07:33:49 +01:00
Evgeny Danilenko
3b40819444
grpc regenerate (#936) 
* grpc regenerate

* use string type for id in net_version
 2020-08-18 20:22:49 +03:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928) 
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
 2020-08-17 08:24:59 +01:00
Alex Sharov
b2e35eb63f
rename "remote2" to "remote" (#932) 2020-08-17 07:47:06 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size (#918) 
* add logging to loader

* use pure tx in etl loading, logs in mutation commit

* clean

* bletter logging and more cleanup

* bletter logging and more cleanup

* increase batch size to 500M

* better batch commit logging

* async fsync

* sync fsync

* sync fsync

* unify logging

* fix corner-case when etl can use empty bucket name

* fix tests

* better logging

* better logging

* rebase master

* remove lmdb.NoMetaSync flag for now

* consistent walk and multi-walk

* clean

* sub tx

* add consistent multi-put

* implement dupsort support in one new cursor method

* clear
 2020-08-17 07:45:52 +01:00