Commit Graph

244 Commits

Author SHA1 Message Date
Alex Sharov
60f1adddcd
implement server kv.Snapsthos() method (#4831)
* save

* save
2022-07-26 10:44:47 +07:00
Alex Sharov
68e35417fc
RetireBlocks: encapsulate delete logic 2022-07-23 11:09:16 +07:00
Alex Sharov
cd8b10f57e
snapshot merger: smaller interface (#4786)
* save

* save

* save
2022-07-22 16:54:05 +07:00
Alex Sharov
66758c7960
RetireBlocks: less arguments (#4785)
* save

* save
2022-07-22 13:44:42 +07:00
Alex Sharov
1ecacde3a9
trackerslist: convert from git submodule to go package (#4761) 2022-07-20 09:47:58 +07:00
Alex Sharov
5805d963ea
erigon-snapshot: convert from git sumbodule to golang package (#4760) 2022-07-20 09:34:12 +07:00
Enrique Jose Avila Asapche
e768227d38
Merge range (#4749)
* added merge range into segments

* got rid of missing snapshot errors

* reusing RequestSnapshotDownload

* sleep out of download

* ops

* warning if we are missing snapshots
2022-07-19 19:27:54 +07:00
Alex Sharov
d3b424c9f6
Mdbx: GC BigFoot (#4750) 2022-07-19 16:53:18 +07:00
Enrique Jose Avila Asapche
8dceb6fe82
Auto download snapshots (#4729)
* refactored request download

* keeping track of missing snapshots

* using slice mergeRange

* request snapshots on reopen

* passing arguments

* passed in var

* Revert "passed in var"

This reverts commit 90478978dfa9f2a6dd5b1b051fc1d3f9e5f7a9c5.

* Revert "passing arguments"

This reverts commit 1e39c4152003796f6ff0bcfc188512d4a43bd18d.

* Revert "request snapshots on reopen"

This reverts commit d40212b973bc15db2b25cc1b0abb22051a3debb1.

* added downloadRequest
;

* downloading missing headers at start up

* there shouldnt be an error anymore

* not using nil;
;
2022-07-18 17:42:20 +03:00
Alex Sharov
15ca3d25c1
snapshots: mainnet to 15m #4733 2022-07-18 15:36:53 +07:00
ledgerwatch
329d586464
Try to reproduce and fix eth_getBlockByNumber returning nil (#4608)
* Small optimisation for eth_getBlockByNumber

* Option to not retrieve transactions

* fixes

* Check hash

* Fixes

* Avoid shadowing of err in BlockWithSenders

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-06 14:10:12 +01:00
Alex Sharov
7e2d46cbe4
Support "latests block number" in Oracle Backend (#4635)
* save

* save

* save

* save

* save

* save

* save

* save
2022-07-05 14:49:39 +06:00
ledgerwatch
8599dceec7
[erigon2.2] State reconstitution prototype (#4508)
* reconstitution

* Add history access without state function

* More on state reconstitution

* More on state recon

* More on state recon

* More

* More

* support dao fork

* More on state reconstitution

* Update to erigon-lib

* More

* Added genesis block and filling with history

* update

* Genesis works

* Start on parallel

* Preparation for parallel reconstitution, stats for EfSearch

* continue with parallel work

* Fix history reader

* Remove time measurements

* Fixes

* Fixes and UX improvements

* Fixes

* More tracing

* More fixes

* More fixes

* Fix code size

* Update to latest erigon-lib

* Fix for dao fork

* Remove hacks

* Update to erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 20:48:42 +01:00
Alex Sharov
c03d57356c
prevent downloading new snapshots after initial sync (#4585) 2022-06-30 22:35:44 +06:00
Alex Sharov
6add6ecd0e
Snapshots: new bsc hash #4578 2022-06-29 20:42:05 +06:00
Alex Sharov
538b4fea6c
Snapshots: don't panic after too far reset (#4558)
* save

* save

* save

* save
2022-06-28 10:31:44 +06:00
Alex Sharov
529682aa9d
New goerli snapshot (#4544) 2022-06-26 17:13:07 +06:00
ledgerwatch
93151f0ae6
[erigon2.2] Reduce allocations when replaying historical txs (#4460)
* [erigon2.2] Reduce allocations when replaying historical txs

* import new API, fix code

* Add hack decompress function

* update erigon-lib

* Fix hack

* Update to latest erigon-lib

* Fix reindexing

* Enable skip analysis for tracing calls too

* Enable for eth_getLogs

* Fix skip analysis

* Optimise

* Stop grpc server in the tests

* Print

* No panic on server stop

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 13:40:49 +01:00
ledgerwatch
21e3ebdab5
Update skip analysis, add more BSC snapshot hashes (#4448)
* Update skip analysis, add more BSC snapshot hashes

* Syntax fix

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-14 16:06:42 +01:00
ledgerwatch
fd8adddce8
trace_filter implementation based on erigon 2 update 2 data (#4431)
* trace_filter implementation based on erigon 2 update 2 data

* Fix test compile, extract txNums

* Update to erigon-lib, add generic heap

* Fix getHeaderHash for RPC methods

* Missing files

* Skip tests

* Add reward traces

* Filter trace lint

* Reinstate filtering

* Print

* Print

* Fix

* Print

* Print txNums

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove prints

* Fix nil dereference

* Inclusive toBlock bound

* Print

* Print

* Hack

* remove some deps

* Fix lint

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-12 12:44:01 +01:00
Alex Sharov
c97064173e
Open reset blocks migration: to delete .torrent files also #4404 (#4404) 2022-06-08 09:51:07 +07:00
Alex Sharov
d7d698f565
db migration to reset blocks (#4389)
* save

* save

* save

* save

* save

* save

* Update reset_blocks.go

* Not to remove too many tx lookup files

* Fix truncate blocks and add reset txlookup

* Fix bodies

* Fix nil pointer

Co-authored-by: ledgerwatch <akhounov@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-07 19:59:14 +01:00
Alex Sharov
d655854b53
Snapshots: optimisticaly open at app startup (#4393)
* save

* save
2022-06-07 12:33:33 +07:00
ledgerwatch
1231f8c21c
Add bor mainnet to embed (#4370)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 11:39:27 +01:00
ledgerwatch
7132a90613
Add bor mainnet snapshot hashes (#4368)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 14:36:26 +05:30
Enrique Jose Avila Asapche
9f1cc40c89
changed syncmode flag to snapshots flag (#4353)
* changed syncmode flag to snapshots flag

* using boolT

* Remove syncmode concept

* Add mumbai

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 07:33:55 +01:00
Alex Sharov
f838268e44
[wip] Snapshots: more runtime invariants check (#4350) 2022-06-03 18:50:01 +07:00
Enrique Jose Avila Asapche
49292346b3
Auto correcting sync mode instead of giving an error (#4337) 2022-06-03 09:38:10 +01:00
ledgerwatch
bcbb8bec65
Try to prevent double block retire (#4330)
* Try to prevent double block retire

* Remove print

* More careful setting of segmentsMax

* Print snapshots max

* Clean lists

* Print

* Use Reopen

* Fix

* Remove prints

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-02 16:58:38 +01:00
Enrique Jose Avila Asapche
bc21ab9d97
syncmode changes (#4334)
* log syncmode

* little comment

* ops

* calling log in correct place

* got rid of error instead we warn

* no return
2022-06-02 14:15:00 +01:00
ledgerwatch
b06b4b4c91
Prevent double retiring of the same interval, prevent hiding errors (#4325)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-02 08:49:24 +07:00
Alex Sharov
e13e212288
rpc: add all blockReader methods (#4320)
* save

* save
2022-06-01 09:57:12 +07:00
ledgerwatch
0680671df9
Revert "Warn not error when body is missing (#4309)" (#4311)
This reverts commit 85814cacc5.
2022-05-30 18:31:47 +01:00
ledgerwatch
85814cacc5
Warn not error when body is missing (#4309)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-30 16:23:57 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00
Alex Sharov
815119525a
reduce mem usage on building txs indices (#4299) 2022-05-30 09:33:48 +07:00
Alex Sharov
6eb10ba9a8
Use ETL to fill kv.HeaderNumber from snapshots (#4272)
* merge devel

* save
2022-05-26 21:41:05 +07:00
Alex Sharov
60ba14e2c7
bsc snap hashes to 17.5M #4249 2022-05-24 13:32:54 +07:00
Alex Sharov
52d6f1d551
kv_mdbx: atomic closed flag #464 (#4247) 2022-05-24 12:24:20 +07:00
Alex Sharov
31e94803dc
SnapshotIndex: Add more context to panic #4245 2022-05-24 11:00:55 +07:00
ledgerwatch
8ea2bb55d0
Prevent deletion of extra block worth of tx lookup records when pruning (#4232)
* Print

* Prevent underflow, return 0 block if tx lookup failed

* Return nil receipt for 0 block

* Update to main erigon-lib

* Fix other places

* Fix other places

* Comment

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-22 20:35:02 +01:00
Alex Sharov
7ecdf51d92
methods to read single txn by txnID (#4215) 2022-05-20 14:12:12 +07:00
Alex Sharov
ada46b0344
allow snap sync for mumbai chain (#4208)
* save

* mumbai
2022-05-19 19:31:02 +07:00
Alex Sharov
d9ed307807
allow --syncmode=snap for bor-mainnnet #4206 2022-05-19 18:54:50 +07:00
Alex Sharov
a95d86a99f
allow --syncmode=snap for bor-mainnnet (#4206)
* save

* save
2022-05-19 18:51:05 +07:00
Alex Sharov
726ef081ce
option to disable runtime.ReadMemStats (#4200) 2022-05-19 11:48:07 +07:00
Alex Sharov
4e7807251a
Ensure snapshotDir not created by accident (#4164)
* save

* save

* save
2022-05-17 09:40:45 +07:00
Alex Sharov
e1dec529d4
Generic sort of slices (no allocs, inlinable) (#4161)
* save

* save

* save

* save

* save
2022-05-16 08:24:12 +01:00
Alex Sharov
91e43e52a6
Snapsync for ropsten (#4143)
* save

* save
2022-05-13 14:58:03 +07:00
Alex Sharov
4e0f3c8c4c
Sentry: batch events (#4140) 2022-05-13 11:00:44 +07:00