mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2024-12-22 03:30:37 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
20,764 Commits 5 Branches 14 Tags 294 MiB
master
Commit Graph

82 Commits

Author SHA1 Message Date
Alex Sharov
1ae3f41189
db migration: if stage_snapshots > 0, then create prohibit_new_downloads.lock file (#9036) 
it will protect existing nodes from downloading `v2` files - even if
they do upgrade
 2023-12-20 18:04:43 +07:00
a
436493350e
Sentinel refactor (#8296) 
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
 2023-10-22 01:17:18 +02:00
Alex Sharov
e14664d53b
Store Canonical/NonCanonical/Bad blocks/senders/txs in same tables. Make re-org cheaper (#7648) 
- allow store non-canonical blocks/senders
- optimize re-org: don't update/delete most of data
- allow mark chain as `Bad` - will be not visible by eth_getBlockByHash,
but can read if have hash+num
 2023-06-05 12:36:24 +07:00
Alex Sharov
c8e717c957
rawdb methods to rw db schema version (#7653) 2023-06-04 09:20:22 +07:00
Alex Sharov
31687be599
blockReaders in tests, step4 (#7570) 2023-05-24 15:52:51 +07:00
Alex Sharov
2865b85888
move e2 snapshots management closer to e3: step 1 (#7543) 
- always RLock all snapshots - to guarantee consistency
- introduce class View (analog of RoTx and MakeContext)
- move read methods to View object
- View object will be managed by temporal_tx

---------

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
 2023-05-22 10:09:46 +07:00
ledgerwatch
b382f96f6c
Introduce logger into etl (#7537) 
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
 2023-05-18 21:20:07 +01:00
ledgerwatch
6ef3fc3a3c
[devnet tool] Side-quest logging, step 5 (#7484) 
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
 2023-05-10 19:36:27 +01:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
ledgerwatch
fd52a788b7
Fix creation of block snapshots (#5579) 
* Print snapshot prune

* More print

* Print

* Print

* Print

* Move snapshots stage forward

* Cleanup

* Fix tests

* Print

* Too much logging

* Remove print

* Log, check

* Revert

* No panic, print

* Fix tx numbering

* Harder condition to start retiring blocks

* Disable Pow verification after TTD is reached

* Fix POW verifying

* Print

* Prints

* Fix?

* cleanup

* Add migrations and hack

* More diagnostics

* More print

* Reset sequence only once

* Fix migration

* Remove print reset

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-10-04 11:14:18 +01:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Alex Sharov
b6ea28ea80
kv.Del() remove second parameter (#4832) 
* save

* save

* save

* save

* save

* save

* save
 2022-07-26 12:47:05 +07:00
Alex Sharov
d7d698f565
db migration to reset blocks (#4389) 
* save

* save

* save

* save

* save

* save

* Update reset_blocks.go

* Not to remove too many tx lookup files

* Fix truncate blocks and add reset txlookup

* Fix bodies

* Fix nil pointer

Co-authored-by: ledgerwatch <akhounov@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-06-07 19:59:14 +01:00
Alex Sharov
b003d07839
Add system-txs to begin end of block (#3654) 
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* up torrent

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
 2022-03-10 07:48:58 +00:00
Alex Sharov
a5f4c721bb
fix lost migrations (#3661) 2022-03-09 16:46:48 +07:00
Alex Sharov
191a3f9232
Disallow db version downgrade (#3657) 
* save

* save
 2022-03-08 10:02:35 +07:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
Alex Sharov
bf9e109115
Revert "Change major db version check (#3241)" (#3421) 
This reverts commit db7fa3d0ed.
 2022-02-04 09:21:10 +07:00
Alex Sharov
db7fa3d0ed
Change major db version check (#3241) 
* remove major jump check

* remove major jump check
 2022-01-12 13:30:23 +00:00
ledgerwatch
9f861e1b3f
Check upgrade/downgrade of database schema version to prevent accidental upgrade, remove migrations (#3133) 
* Check upgrade/downgrade

* Remove old migrations

* Fix compilation

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-12-15 21:03:04 +00:00
Alex Sharov
e55256296b
snapshots: read block from snapshots, add sender to txs file (#2996) 2021-11-21 10:32:14 +07:00
Alexandr Borodulin
d7159cd8bd
More convenient pruning for non-PoW consensus: add --prune.*.before flags (#2714) 2021-09-23 09:13:19 +07:00
alex.sharov
12982b21c1 remove old migration 2021-08-02 19:10:14 +07:00
alex.sharov
f5e30f293a remove old example from migrations package 2021-08-02 19:04:51 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
a36a613e61
move migrations to kv interfaces (#2430) 2021-07-24 11:28:05 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399) 
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
 2021-07-20 21:03:19 +01:00
Alex Sharov
4c53f51767
Remove "persistent unwind" concept (#2388) 2021-07-17 21:14:51 +07:00
Alex Sharov
7c60cc0fee
step towards kv interface (#2313) 2021-07-08 09:56:09 +07:00
ledgerwatch
9023f2a29c
Investigate slow receipt repair (#2306) 
* Collect data about broken receipts

* Actually collect data

* Fix CBOR problem more efficiently

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2021-07-05 19:14:58 +01:00
ledgerwatch
954304e43a
Turn off receipt repair migration for investigation (#2303) 
* receipt repair migration - check for r storage mode

* Turn off migration

* Fix lint

* Better way to disable migration

* Better way to disable migration

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-07-05 11:25:18 +01:00
ledgerwatch
63ccabc72f
Receipt repair migration (#2291) 
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

* Better mechanism for detecting broken receipts

* Fixes

* Introduce receipt_repair migration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2021-07-04 14:48:13 +01:00
Alex Sharov
ba902f7ef3
separate_config_from_node_and_eth (#2289) 2021-07-04 08:50:32 +01:00
ledgerwatch
5ad449ef22
Migration to fix receipts (broken due to change of CBOR codec) (#2221) 
* Scan receipts

* Not check tx type before Berlin

* Reverse

* scan receipt code in the migration

* Turn migration on

* Apply migration

* Fix

* Print last

* Remove last

* Not remove last

* Not updata in migration

* no cursors in migration

* Handle empty blocks

* Handle empty blocks

* Use the same code in hack

* Print

* Print

* Save

* Fix

* Fix lint

* Fix lint

* Fix for empty DB

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-06-23 15:05:16 +01:00
Artem Vorotnikov
72e9a34365
Remove unused tables, soft rename, take 3 (#2207) 
* Remove unused tables, soft rename, take 3

* Bump db schema version

* nil check in migration
 2021-06-20 20:46:57 +01:00
Artem Vorotnikov
af836a6200
Nuke LMDB (#2167) 2021-06-16 13:57:58 +03:00
Alex Sharov
eee871aedf
Add db label (later will add txpool db) (#2081) 
* add db label - later will add txpool db

* move commit marker to kv_mdbx behind Chain label

* Update migrations.go

* clean

Co-authored-by: ledgerwatch <akhounov@gmail.com>
 2021-06-04 15:56:49 +01:00
ledgerwatch
43915a73a8
Migration to fix trace_filter (#2095) 
* Migration to fix trace_filter

* Fix to db/tx

* Fix to db/tx

* Error fixes

* Bump KV version

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-06-04 12:45:06 +01:00
ledgerwatch
729bfab988
Split DBSchema version depending on the database type (#1990) 
* Split DBSchema version depending on the database type

* Fix test

* Fix test

* Fix lint

* Update migrations.go

* Update bucket.go

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-05-22 10:20:43 +01:00
Alex Sharov
11d1e9119e
remove pre-berlin migrations (#1987) 2021-05-22 10:29:00 +07:00
Alex Sharov
545fe32f68
Mdbx - make it default db. Lazy buckets renaming. (#1982) 2021-05-22 10:03:02 +07:00
Alex Sharov
0be3044b7e
rename (#1978) 
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
 2021-05-20 19:25:53 +01:00
Alex Sharov
01ae962459
Clean tmp on start (#1808) 
* fininish to remove chain context

* clean

* move migrations folder from
<datadir>/etl-tmp/migrations
to
<datadir>/migrations
 2021-04-26 13:51:01 +01:00
ledgerwatch
e3f3dd3c9b
Integration tests 1 (#1793) 
* Initial commit

* Add sentry gRPC interface

* p2psentry directory

* Update README.md

* Update README.md

* Update README.md

* Add go package

* Correct syntax

* add external downloader interface (#2)

* Add txpool (#3)

* Add private API (#4)

* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Separate upload messages into its own stream (#6)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Only send changed accounts to listeners (#7)

* Txpool interface doc (#9)

* Add architecture diagram source and picture (#10)

* Typed hashes (#11)

* Typed hashes

* Fix PeerId

* 64-bit tx nonce

* Add proper golang packages, max_block into p2p sentry Status (#12)

* Add proper golang packages, max_block into p2p sentry Status

* Change EtherReply to address

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Add Rust infrastructure (#13)

* DB stats methods removed by https://github.com/ledgerwatch/turbo-geth/pull/1665

* more p2p methods (#15)

* add mining methods (#16)

* First draft of Consensus gRPC interface (#14)

* Update Rust build

* Fix interfaces in architecture diagram (#17)

* Fix KV interface provider

* Fix Consensus interface provider

* drop java attributes (#18)

* tx pool remove unused import (#19)

* ethbackend: add protocol version and client version (#20)

* Add missing ethbackend I/F (#21)

* Add interface versioning mechanism (#23)

Add versioning in KV interface

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>

* spec of tx pool method (#24)

* spec of tx pool method (#25)

* Update version.proto

* Refactor interface versioning

* Refactor interface versioning

* Testing interface

* Remove tree

* Fix

* Build testing protos

* Fix

* Fix

* Update to the newer interfaces

* Add ProtocolVersion and ClientVersion stubs

* Hook up ProtocolVersion and ClientVersion

* Remove service

* Add compatibility checks for RPC daemon

* Fix typos

* Properly update DB schema version

* Fix test

* Add test for KV compatibility|

* Info messages about compability for RPC daemon

* DB schema version to be one key

* Update release intructions

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: canepat <16927169+canepat@users.noreply.github.com>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Co-authored-by: canepat <tullio.canepa@gmail.com>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-04-24 16:46:29 +01:00
Alex Sharov
bbe9af3e25
Stages as const (#1774) 2021-04-21 15:02:21 +07:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB (#1703) 
* debug

* debug

* it works

* rename clique bucket

* remove genesis special case

* copy snapshot

* remove debug

* migration

* debug

* regenerate snapshots

* simplify

* regeneration

* after merge

* tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2021-04-19 22:58:05 +01:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it (#1676) 2021-04-05 20:04:58 +07:00
b00ris
c03da8fb8e
Split header prefix bucket to headers, td and canonical (#1556) 
* split headers prefix

* migration

* fix downloader bug

* test for migration

* fix lint

* uncomment t.Prallel

* fix postprocessing test
 2021-03-19 12:54:47 +00:00