Commit Graph

591 Commits

Author SHA1 Message Date
ledgerwatch
15b4095718
Move ETL to erigon-lib (#2667)
* Move ETL to erigon-lib

* Update link in the readme

* go mod tidy

* Use common/chan.go from erigon-lib

* Clean up

* Fix lint

* Fix test

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-12 08:50:17 +01:00
Alex Sharov
f3021dd2af
RPC: small optimizations (#2564) 2021-08-24 08:28:58 +01:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
054581f507
staged syncs automatically add error prefix about stage number and name (#2445) 2021-07-26 13:19:01 +07:00
Alex Sharov
12c578b84d
less alloc at hash state key transformation #2441 (#2441) 2021-07-24 18:14:10 +07:00
alex.sharov
c2b0b8be36 remove debug comment 2021-07-22 19:08:19 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition (#2411) 2021-07-21 18:13:26 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399)
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
2021-07-20 21:03:19 +01:00
Alex Sharov
a5a58e97d0
collector already does copy, no reason to copy in buffer (#2396) 2021-07-18 17:25:41 +07:00
Alex Sharov
4c53f51767
Remove "persistent unwind" concept (#2388) 2021-07-17 21:14:51 +07:00
Alex Sharov
8d320aee07
add pruning stage concept, but not enable yet (#2385) 2021-07-17 13:38:44 +07:00
Alex Sharov
a79b3cd6eb
unify stage params (#2384) 2021-07-17 09:09:56 +07:00
alex.sharov
bc65759c84 clien id in rpc 2021-07-15 16:11:39 +07:00
Alex Sharov
75ee5db06f
Block read optimize (#2357) 2021-07-13 16:31:59 +07:00
Alex Sharov
08590211e6
Add defer collector.Close() (#2351)
* add defer collector.Close

* add defer collector.Close
2021-07-13 09:17:21 +07:00
Alex Sharov
92df338ec8
ETL: Enforce "oldest appeared" property at load time (#2352) 2021-07-13 09:17:15 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00
Alex Sharov
134e164fe7
crash reports don't write to files - logger is enough (#2342) 2021-07-12 13:39:44 +07:00
Alex Sharov
5eaf4c8cc3
rpc: 1559 in call and tracing (#2327) 2021-07-11 04:05:56 +00:00
Alex Sharov
7c60cc0fee
step towards kv interface (#2313) 2021-07-08 09:56:09 +07:00
b00ris
57473175ff
Body snapshot (#2100)
* fix test

* get rid of ObjectDatabase

* sn_builder_prototype2

* save state

* save state

* integration step1

* fix lint

* fix

* fix test

* integrate migrator.finish

* fix lint

* fix build

* fix typo

* save state

* body snapshot test

* unique tx

* body snapshot generation using walk

* move methods out of test

* block data verification added

* fix lint

* test with remove works correctly

* fix lint

* remove experiment test

* fix test

* add comment

* add second layer of remove test

* rename test

* fix typos

* fix lint

* revert testdata

* body snapshot migration save state

* fix body snapshot migration

* fix after merge

* remove debug test

* debug windows build

* fix build

* fix

* fix lint

* debug

* fix

* fix windows build

* simplify snapshot management&&get rid of lazy tx

* fix lint

* fix windows path

* debug

* debug

* debug

* debug

* remove geometry experiments

* skip windows tests

* clean

* fix

* fix ;int
2021-07-06 23:33:26 +01:00
Alex Sharov
3db7e85994
Simplify GetAsOf (#2285)
* no reason to copy if we work with ethdb.Tx (user of ethdb.Tx expecting that all data valid until end of tx)

* less objects
2021-07-04 08:49:31 +01:00
Alex Sharov
6a5f86423c
Remove "numgc" from logs (#2275) 2021-07-03 14:44:23 +07:00
Alex Sharov
8e00da2035
Sokol v0: preparations for contracts run, proofs support (#2270) 2021-07-02 15:00:41 +07:00
ledgerwatch
a05d1be531
Pull grpc bindings into erigon-lib (#2265)
* Pull grpc bindings into erigon-lib

* Remove mdbx dist

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-01 22:31:14 +01:00
Alex Sharov
3d6d45a82f
remove subtrie loader and dcmp (#2258) 2021-06-30 10:31:36 +07:00
Alex Sharov
605d7a8d59
Sokol v0 (#2228)
* handle chain flag in integration

* handle chain flag in integration

* handle chain flag in integration

* save

* save

* merge devel

* save

* noopWriter - one is enough

* chain spec parser

* chain spec parser

* embed

* embed

* embed

* embed

* embed

* embed

* embed

* clean

* clean

* correct alloc after reset state

* correct alloc after reset state

* correct alloc after reset state

* integration reset state now does re-apply genesis and chainConfig

* eips summary

* eips summary

* eips summary
2021-06-25 19:13:40 +01:00
Alex Sharov
59d05dc5fe
hide file exists err (#2218) 2021-06-22 11:09:45 +01:00
Artem Vorotnikov
72e9a34365
Remove unused tables, soft rename, take 3 (#2207)
* Remove unused tables, soft rename, take 3

* Bump db schema version

* nil check in migration
2021-06-20 20:46:57 +01:00
Artem Vorotnikov
504c31e97a
Revert "Remove unused tables, soft rename" (#2205) 2021-06-20 15:08:48 +01:00
Artem Vorotnikov
d8a009837b
Nuke preimage remnants (#2203) 2021-06-20 13:00:22 +07:00
Alex Sharov
4736bac190
save (#2197) 2021-06-19 21:29:02 +01:00
Artem Vorotnikov
a9baf37ea1
Remove unused tables, soft rename (#2201) 2021-06-19 21:36:21 +03:00
Alex Sharov
4533b59724
flag to stop before stage (#2194) 2021-06-19 16:01:33 +07:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Alex Sharov
e9f510c2b7
erigon as lib (#2186) 2021-06-18 10:35:11 +07:00
ledgerwatch
a36d5dc2c3
Revert "Remove unused tables, soft rename (#2180)" (#2187)
This reverts commit 1390b3c740.
2021-06-17 09:35:59 +01:00
Artem Vorotnikov
1390b3c740
Remove unused tables, soft rename (#2180) 2021-06-16 16:44:23 +01:00
Artem Vorotnikov
af836a6200
Nuke LMDB (#2167) 2021-06-16 13:57:58 +03:00
BitBaseBit
159777db29
Changed gas price oracle to use effective tip and ignoreUnder parameter for 1559. (#2173)
* added temp file in crashreports because git is stupid and wont add empty folders

* implemented ignoreUnder + EffectiveGas for legacy and dynamic txs

* implemented changes to gas price oracle for 1559

* switched Cmp to Lt/Gt
2021-06-15 21:45:34 +01:00
BitBaseBit
229d6671b9
Added temp file in crashreports because git is stupid (#2157)
* added temp file in crashreports because git is stupid and wont add empty folders

* added proper creating of crash report dir

* changed crashreports dir to one directory up from datadir

* implemented suggestions

* switched back to 5d651381daf9d12cc663a7326b3b16d7bb4c1aa0 for tests/testdata
2021-06-14 07:35:01 +01:00
BitBaseBit
7ed337cdcc
Implemented panic handling, graceful shutdown and reporting for all goroutines that don't explicitly handle them. (#2153)
* implemented crash reporting for all goroutine panics that aren't handled explicitly

* implemented crash reporting for all goroutine panics that aren't handled explicitly

* changed node defaults back to originals after testing

* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports

* handling panics on all goroutines gracefully

* updated missing call

* error assignment

* implemented suggestions

* path.Join added

* implemented Evgeny's suggestions

* changed path.Join to filepath.Join for cross-platform

* added err check

* updated RecoverStackTrace to LogPanic

* updated closures

* removed call of common.Go to some goroutines

* updated scope capture

* removed testing files

* reverted back to original method, I feel like its less intrusive

* update filename for clarity
2021-06-13 17:41:39 +01:00
Evgeny Danilenko
de26634d8d
TEVM status pruning (#2034)
* use trace for TEVM status

* restore testdata

* fix getting contract from plain state

* unwind

* linters

* reproducible build debug

* debug

* debug

* remove debug
2021-06-11 09:34:25 +01:00
Alex Sharov
40eaf77eb8
Sokol genesis and chainconfig (#2116)
* sokol init

* test

* precompiles

* precompiles

* add code

* sokol test

* save

* save

* save

* save

* save

* save
2021-06-07 12:01:54 +01:00
Alex Sharov
5ba3ea162a
Simulated backend and genesis to kv (#2104)
* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* move to kv

* merge

* merge

* merge

* merge

* merge

* merge

* merge

* merge
2021-06-05 16:17:04 +01:00
ledgerwatch
f7fc4d66ec
Move tests 5 (#2069)
* Remove silkworm and reader builders

* Switch tests

* Stop from hanging

* Update tests to 9.0.0

* Move tests back

* Moving code around

* Fix lint

* More fixes

* Intermediate

* Fix compile

* Fix compile errors

* Compile errors

* Fix lint

* Comment out more printing

* More simplifications

* Compile fixes

* Compile fix

* More replacements

* More fixes

* More muddling through

* lint

* Exclude dao split test

* Restore sidechains after unwind

* Errors in senders stage, stopped in bodies stage

* Fix compile

* Fix compile

* Fix tests

* More fixes

* More fixes

* Code cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2021-06-05 11:00:21 +01:00
Alex Sharov
1f13f73045
up version of uint256 lib (#2082)
* eip 1559_tx_constraints

* eip 1559_tx_constraints

* return error on uint256 overflow

* up uint256 version

* merge master

* fix master

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2021-06-04 17:25:28 +01:00