Commit Graph

502 Commits

Author SHA1 Message Date
ledgerwatch
81d106bc9d
Experiment in parallel execution (#4652)
* Restructure tx execution

* fixes

* Fixes and traces

* Tracing

* More tracing

* Drain the result channel

* Intermediate

* more efficient parallel exec

* Sorted buffer

* Fix results size

* fix for the recon

* Fix compilation

* Sort keys in Write and Read sets, fix compilation in rpcdaemon22

* Update to latest erigon-lib

* Update to erigon-lib

* Remove go.mod replace

* Update erigon-lib

* Update to erigon-lib main

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-23 18:39:08 +01:00
Alex Sharov
95f0338ddb
Pool: parse rlp chain id for non-legacy transactions #4802 2022-07-23 10:13:13 +07:00
Alex Sharov
6a759a34f8
pool: metrics (#4800) 2022-07-23 10:06:35 +07:00
Alex Sharov
dd4bae789b
Pool: parse rlp chain id for non-legacy transactions 2022-07-23 09:33:45 +07:00
Giulio rebuffo
1d378b6618
Filter out bad tx with wrong chain id during block building phase. (#4783)
* filter out bad tx with wrong chain id

* report bad txs

Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-07-22 09:18:19 +02:00
Andrew Ashikhmin
9e8f625c53
Fix txn removal in PendingPool (#4770) 2022-07-20 18:28:58 +03:00
Andrew Ashikhmin
00769e3dff
Fix MDBX compilation on macOS (#4767) 2022-07-20 15:16:20 +02:00
Alex Sharov
1ecacde3a9
trackerslist: convert from git submodule to go package (#4761) 2022-07-20 09:47:58 +07:00
Alex Sharov
5805d963ea
erigon-snapshot: convert from git sumbodule to golang package (#4760) 2022-07-20 09:34:12 +07:00
Andrew Ashikhmin
d0399012c2
Fix txn removal in PendingPool (#4754) 2022-07-19 16:31:30 +02:00
Alex Sharov
d3b424c9f6
Mdbx: GC BigFoot (#4750) 2022-07-19 16:53:18 +07:00
Alex Sharov
b70abd7aaf
Grpc up v48 #532
* save

* save
2022-07-19 13:45:53 +07:00
Alex Sharov
5d68f610bc
mdbx: use OS pagesize by default (but > 4Kb, and < 64Kb) #4743 2022-07-19 10:40:02 +07:00
Alex Sharov
b7acf6c108
compressor: generic sort (#4734) 2022-07-18 17:12:13 +07:00
Alex Sharov
ac9b7d8cc2
commitment: generic btree #4731 2022-07-18 12:19:46 +07:00
Giulio rebuffo
759e77c71b
mod (#4717) 2022-07-14 19:08:33 +02:00
Max Revitt
211dbfbb75
fix(#4543): BeginRo use semaphore (erigon-lib bump) (#4712) 2022-07-14 12:01:57 +02:00
Alex Sharov
36586e52a1
Torrent: add fsync after piece download (#4700) 2022-07-13 17:39:20 +07:00
Artem Tsebrovskiy
9637b25a42
[erigon2] print trie variant derived from trie instead switch log (#4672)
* [erigon2] print trie variant derived from trie instead switch log

* updated erigon-lib version

* bumped erigon-lib version
2022-07-11 15:58:08 +01:00
Giulio rebuffo
63f6eab6c7
removed envelope on txpool for PoS (#4683) 2022-07-08 16:48:24 +02:00
sudeep
e13a318e0b
evm t8n to use ExecuteBlockEphemerally api (#4642)
* evm t8n tool to use ExecuteBlockEphemerally api (#4512)

* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc

* fix integration tests

* fix build
2022-07-07 12:47:00 +01:00
Igor Mandrigin
024061018e
Fix a deadlock under RPC load (#4667)
* debugs

* rename logs

* updated erigon lib with the attempt to fix deadlock

* Update erigon-lib that fixes the deadlock

* Revert "rename logs"

This reverts commit 13b6ac555314ce4317de2438d5f1a5e883343865.

* Revert "debugs"

This reverts commit 356de6c8ef3df2a440f8ad6c18ccb63496d8256d.

* revert unnecessary change

* go mod tidy

Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-07-07 08:34:51 +01:00
Alex Sharov
01bca8f1e2
TxPool: generics btree (#4665)
* save

* save

* save

* save
2022-07-07 11:07:48 +06:00
ledgerwatch
73b0659bb7
[erigon2.2] Optimisation of state reconstitution (#4621)
* Init

* Optimise ReconState flushes

* Updates

* Update

* Updates

* Updates

* More on parallel execution

* More on parallel exec

* Fix lint

* Improvements to parallel exec

* comment

* Fix history access, include incarnation

* Close work channel

* TxTask

* more fixes to parallel exec

* Update to latest erigon-lib

* Fix compilation

* Cleanup and timing

* Fixes to recon1

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-06 07:49:00 +01:00
Alex Sharov
2a0fda33dd
snapshots table (#4624) 2022-07-04 11:17:39 +06:00
ledgerwatch
77dc35bb85
Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)" (#4610)
* Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)"

This reverts commit db93d2ea37.

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 23:24:20 +01:00
ledgerwatch
8599dceec7
[erigon2.2] State reconstitution prototype (#4508)
* reconstitution

* Add history access without state function

* More on state reconstitution

* More on state recon

* More on state recon

* More

* More

* support dao fork

* More on state reconstitution

* Update to erigon-lib

* More

* Added genesis block and filling with history

* update

* Genesis works

* Start on parallel

* Preparation for parallel reconstitution, stats for EfSearch

* continue with parallel work

* Fix history reader

* Remove time measurements

* Fixes

* Fixes and UX improvements

* Fixes

* More tracing

* More fixes

* More fixes

* Fix code size

* Update to latest erigon-lib

* Fix for dao fork

* Remove hacks

* Update to erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 20:48:42 +01:00
sudeep
db93d2ea37
evm t8n tool to use ExecuteBlockEphemerally api (#4512)
* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc
2022-07-02 06:52:23 +01:00
Alex Sharov
2415fecb26
--downloader.verfiy flag to verify once on startup (#4597)
* save

* save

* save

* save

* save

* save
2022-07-01 16:52:43 +06:00
Andrew Ashikhmin
7d8c67e853
Exempt local transactions from spam protection (#4580) 2022-06-29 17:50:20 +02:00
Andrew Ashikhmin
dc5d3ffaac
Revert "Switch from eth/66 to eth/67 (#4549)" (#4562)
This reverts commit 16e57aa8a2.
2022-06-28 13:42:35 +02:00
Andrew Ashikhmin
16e57aa8a2
Switch from eth/66 to eth/67 (#4549)
* Switch from eth/66 to eth/67

* Fix a compilation error

* Update erigon-lib

* Refresh erigon-lib
2022-06-28 10:12:48 +02:00
Alex Sharov
0d29c3d47d
Compress: reduce etl buffers to save RAM (#4536) 2022-06-25 19:39:39 +06:00
Alex Sharov
acd69a63ec
"-tags debug" to allow pprof show profiling from C code (#4502) 2022-06-20 15:16:56 +06:00
ledgerwatch
79830adefa
[erigon2.2] collecting read indices (#4499)
* [erigon2.2] collecting read indices

* Fix compile issue

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-20 09:00:45 +01:00
ledgerwatch
2e3a75f2b1
[erigon2.2] Add skip analysis to trace_filter (#4487)
* [erigon2.2] Add skip analysis to trace_filter

* Optimisation

* Update to latest erigon-lib

* Fix test

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-19 07:01:00 +01:00
Alex Sharov
ef749e39c3
Roaring version up (#4492)
* save

* save
2022-06-19 06:27:14 +01:00
ledgerwatch
50873a5b1d
[erigon2.2] Fix for code merge (#4478)
* [erigon2.2 Fix for code merge

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 19:46:27 +01:00
ledgerwatch
ecf528d56a
Prevent clogging up pending subpool with transactions that were already mined but notification missed (#4476)
* Debug tx pool Best

* Print results of Best

* Print

* Update to erigon-lib

* Fix

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 16:10:35 +01:00
ledgerwatch
93151f0ae6
[erigon2.2] Reduce allocations when replaying historical txs (#4460)
* [erigon2.2] Reduce allocations when replaying historical txs

* import new API, fix code

* Add hack decompress function

* update erigon-lib

* Fix hack

* Update to latest erigon-lib

* Fix reindexing

* Enable skip analysis for tracing calls too

* Enable for eth_getLogs

* Fix skip analysis

* Optimise

* Stop grpc server in the tests

* Print

* No panic on server stop

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-17 13:40:49 +01:00
Giulio rebuffo
2e600316e3
Used memory batches from Erigon (#4469)
* modded summed

* deleted

* ops
2022-06-16 16:39:56 +02:00
ledgerwatch
2e4e7bd4de
[erigon2.2] Log stats tool (#4453)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-15 07:10:37 +01:00
ledgerwatch
34d823c15a
txid patch to try to reduce GC work during commit (#4441)
* txid patch to try to reduce GC work during commit

* Switch to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-14 12:31:26 +01:00
ledgerwatch
e47dd15e68
[erigon2.2] Prototype of eth_getLogs (#4437)
* Initial work

* Fix compile errors

* Update

* Debug

* Cleanup

* Temp changes

* Temp

* Remove temp

* Remove print

* Upgate to erigon-lib main

* go mod tidy

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-13 21:25:17 +01:00
ledgerwatch
fd8adddce8
trace_filter implementation based on erigon 2 update 2 data (#4431)
* trace_filter implementation based on erigon 2 update 2 data

* Fix test compile, extract txNums

* Update to erigon-lib, add generic heap

* Fix getHeaderHash for RPC methods

* Missing files

* Skip tests

* Add reward traces

* Filter trace lint

* Reinstate filtering

* Print

* Print

* Fix

* Print

* Print txNums

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove prints

* Fix nil dereference

* Inclusive toBlock bound

* Print

* Print

* Hack

* remove some deps

* Fix lint

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-12 12:44:01 +01:00
ledgerwatch
8e3ac8a21c
Erigon2 upgrade 2 prototype (#4341)
* Erigon2 upgrade 2 prototype

* Latest erigon-lib

* Fixes

* Fix print

* Fix maxSpan

* Reduce maxSpan

* Remove duplicate joins

* TxNum

* Fix resuming

* first draft of history22

* Introduce historical reads

* Update to erigon-lib

* Update erigon-lib

* Update erigon-lib

* Fixes and tracing for checkChangeSets

* More trace

* Print account details

* fix getHeader

* Update to erigon-lib main

* Add tracer indices and event log indices

* Fix calltracer

* Fix calltracer

* Duplicate rpcdaemon into rpcdaemon22

* Fix tests

* Fix tests

* Fix tests

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-10 16:18:43 +01:00
Alex Sharov
059e5d1ea5
roaring: up version #4427 2022-06-10 09:02:00 +07:00
Alex Sharov
b004589610
up mmap-go version to solve Win problem #4426 2022-06-10 08:47:24 +07:00
Alex Sharov
0e142e33a0
Reclaimable space in logs (#4417)
* save

* save

* save

* save

* save

* save

* save
2022-06-09 11:47:38 +01:00
Alex Sharov
935975bebd
methods to configure db (#4412)
* save

* save

* save
2022-06-09 09:45:30 +07:00