Alex Sharov
6043c09ddf
Replace global buff pool by local, because of buffer leaks to larger pools ( #739 )
2020-07-14 08:56:47 +07:00
b00ris
6e8c027d68
fix go run for integration ( #741 )
2020-07-14 08:56:29 +07:00
Evgeny Danilenko
9d1ba168de
safe closing txpool reorgShutdownCh ( #738 )
...
* fix
* linters
* buffered chan to prevent blocking
* create channel on start
* fmt
* fix
2020-07-12 08:27:08 +01:00
Alex Sharov
e01f936b35
improve dupsort experiment ( #737 )
2020-07-11 09:56:29 +07:00
ledgerwatch
665b1d88b9
Speed up GenerateChain by using intermediate hashes ( #736 )
...
* GenerateChain to support intermediate hashes
* Fix formatting
* Changeset stats
* Fix formatting
2020-07-10 22:37:34 +01:00
ledgerwatch
16695eb100
Log Info and duplication fix ( #735 )
2020-07-10 11:40:13 +01:00
Alex Sharov
8335700331
Integration: compare to reference_db after all cycles, better cancelation, add to makefile ( #733 )
...
* cmp to reference db if --reference_chaindata provided
* graceful shutdown
* add ./cmd/integration to makefile
* add interruption support for different stages
* don't compare errors directly
2020-07-10 07:03:18 +01:00
ledgerwatch
72dcf478d3
Fix debug_storageRangeAt in rpc daemon ( #732 )
...
* Fix storageRangeAt
* Fix formatting
* Fix lint
* Fix lint
* Prevent racing by closing db after the blockchain
2020-07-10 06:44:01 +01:00
Alex Sharov
df2241f13e
remove debug ( #734 )
2020-07-10 10:29:30 +07:00
Alex Sharov
de5dd0f55e
use non-empty buffer to encode changest ( #729 )
2020-07-10 10:26:48 +07:00
Alex Sharov
d98382fdf8
Integration add checkChangeSet into state_stages command ( #728 )
...
Integration add checkChangeSet into state_stages command (#728 )
2020-07-10 10:25:40 +07:00
Andrew Ashikhmin
bae2e7094a
Fix limit = 0 (unlimited) ( #731 )
2020-07-09 16:08:26 +01:00
Alex Sharov
686f186545
integration - to use same blockchain config as production ( #727 )
2020-07-09 13:54:10 +01:00
Evgeny Danilenko
5f77570556
Make transaction pool startable and stoppable. Staged sync. ( #730 )
...
* after concurrency testing
* fixes for linters
* add an extra parameter
* change init order for protocolManager
* green tests
2020-07-09 13:53:35 +01:00
ledgerwatch
887fc4f99c
GenerateChain without using TrieDbState ( #726 )
...
* Different chain generation
* More fixes
* Fix TestSideLogRebirth
* Fix the rest of core tests
* Fix accounts/abi/bind
* Fix compile errors for cmd/pics
* Fix consensus/clique tests
* More fixes
* Fix core/state tests
* Fix eth tests
* Fix eth/downloader tests
* Fix more tests
* Fix compile errors
* Some more fixes
* Save
* Exists
* Fix last tests
* Clean up
* Fix formatting
* Fix linter
* Fix formatting
* Fix linter
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Fix lint
2020-07-09 07:15:28 +01:00
b00ris
eca0f233ec
[WIP] WalkAsOf ( #724 )
...
* save state
* walkAsOf by cs test passed
* cs search for plain state
* save state
* fix accounts tests
* refactor walkAsOf account tests
* fix storage test
* refactor walkAsOf storage tests
* fix lint
* fix test
* save state
* save state
* add test with fixed bits
* fmt
* add stages check
* fix lint
* fix lint
* remove obsoleted methods
2020-07-09 07:13:45 +01:00
Alex Sharov
2689b35da9
TxLookup unwind: Remove entries for blocks between unwindPoint+1 and stage.BlockNumber ( #723 )
2020-07-08 05:46:04 +01:00
Igor Mandrigin
9881a2ccb3
Fix broken witness generation. ( #721 )
2020-07-07 12:24:55 +01:00
Alex Sharov
495f95f688
Add commands "compare_states", add command "stage9", fix unwind of stage9 (it walked over whole db) ( #722 )
...
* ./cmd/integration compare_states, ./cmd/integration stage9
* TxLookup fix unwind - it walked over all db
2020-07-07 17:07:14 +07:00
Alex Sharov
fb933bc9e3
move lmdb-go under ledgerwatch org ( #720 )
2020-07-07 16:18:09 +07:00
Alex Sharov
f7cc86452c
Move stages from hack.go to ./cmd/integration ( #719 )
...
* move stages from hack.go to ./cmd/integration
* move stages from hack.go to ./cmd/integration
2020-07-07 08:11:10 +01:00
Alex Sharov
305f8aff02
Integration tests2 ( #714 )
...
```
go run ./cmd/integration reset --chaindata=...
go run ./cmd/integration state_stages -h
go run ./cmd/integration state_stages --chaindata=... --verbosity=3 --block=2_000_000 --unwind=10 --unwind_every=1_000
```
Also, it inherits flags from geth:
```
--pprof.cpuprofile=./cpu.out // to file
--pprof --pprof.port=6060 // launch pprof server
--metrics // sends to prometheus
```
2020-07-07 11:00:25 +07:00
Alex Sharov
8363cb0535
pass deleted code to collector ( #718 )
2020-07-07 09:33:26 +07:00
Alex Sharov
d3ede58ff0
stage3, fix: if header stage is ehead of body stage ( #717 )
...
* fix case: if header stage is ehead of body stage
* fix case: if header stage is ehead of body stage
* fix case: if header stage is ehead of body stage
* allow stage3 to sync to given block
* allow stage3 to sync to given block
2020-07-06 11:11:01 +01:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes ( #716 )
...
* Fixing history generation
* Try fixes
* Reset from 0
* Cleanup
2020-07-06 07:34:24 +01:00
Alex Sharov
98f8ccc561
Command for long and heavy integration tests ( #712 )
...
* cmd for integration tests
* cmd for integration tests
2020-07-05 07:18:21 +01:00
Alex Sharov
2aadd94acb
remove mgr feature and stateSize calculation feature ( #711 )
2020-07-05 07:17:27 +01:00
Alex Sharov
c429e6c3a4
Experiments: how much will take state in DupSort feature and how fast can delete large DupSort key ( #709 )
2020-07-04 21:43:23 +07:00
ledgerwatch
cccc3ab460
Stage3 optimise ( #708 )
...
* Cherry-picked
* Reset3
* Fix formatting
* Print before commit
* Use ETL
* Fix linter
* Use different keys for ETL
* Fix formatting
* Update progress correctly
* Reset stage5
* Fix stage3 hack
2020-07-03 18:55:43 +01:00
ledgerwatch
49181a298b
Special bucket for transaction senders ( #707 )
...
* Senders in the separate bucket
* Pre-create txSenders bucket
* Fix compile error
* Fix compile error
* Fix compile error, remove block transformations
* Fix formatting
* Fix test
2020-07-03 08:23:49 +01:00
ledgerwatch
b498419dcb
Fix loader crash (kludge) ( #706 )
...
* Kludge to remove duplicates
* Fix
* Fix formatting
* Fix in Len function
2020-07-03 08:08:35 +01:00
Giulio rebuffo
011a15350b
fix stage 2 ( #705 )
...
* removed unused actions in stage 2
* fixed stage 2
2020-07-02 19:44:08 +01:00
Alex Sharov
ec70cd2a9c
Revert "Only useful IH cursor ( #694 )" ( #703 )
...
This reverts commit 63c998c91f
2020-07-02 17:55:35 +07:00
ledgerwatch
aaf3ab9161
Fix ( #702 )
2020-07-02 11:23:21 +01:00
Giulio rebuffo
3890900dd0
removed unused actions in stage 2 ( #701 )
2020-07-02 08:38:25 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets ( #698 )
...
* Debugging changesets
* Kinda works
* Fix compile error
* Fix formatting
* Fix lint
* Duplicate entries kludge
* Fix compile error
* Cleanup
2020-07-01 15:56:56 +01:00
Giulio rebuffo
3ee1435ac2
integration of insertBodies instead of insertChain ( #700 )
...
* integration of insertBodies instead of insertChain
* remove seal verification from stage2
* fixed seals
2020-07-01 08:23:15 +01:00
Andrew Ashikhmin
aa80b97e80
Use MOV instead of VMOV in fnvHash16AVX2 to peacify asmdecl ( #699 )
2020-06-30 11:23:28 +01:00
Alex Sharov
e28e455b77
small fix of miss-type in ih stage ( #697 )
2020-06-30 05:44:41 +01:00
Alex Sharov
a7f5b1ca08
KV: defer friendly rollback ( #696 )
...
* AbstractKV: defer friendly rollback
* AbstractKV: defer friendly rollback
2020-06-30 04:48:46 +01:00
Giulio rebuffo
85ab6ff8aa
added insertBodies ( #695 )
2020-06-29 17:26:33 +01:00
Alex Sharov
63c998c91f
Only useful IH cursor ( #694 )
...
* cursor which returns only useful IH
* cursor which returns only useful IH
2020-06-28 13:51:18 +01:00
ledgerwatch
ed866e6934
non-concurrent ETL, debug_traceTransaction in rcpdaemon ( #692 )
...
* Fixing history index
* Remove chunk generation, fix formatting
* Fix compile error, clean up hack.go
* Fix output tests
* Fix index generator test
* Fixed checkChangeSets
* Fix linter
2020-06-28 07:10:27 +01:00
Alex Sharov
488a368e5b
remove "k,v,err" fields in kv_bolt, they are useful only in badger ( #690 )
2020-06-27 15:27:06 +07:00
ledgerwatch
4c456a649f
Separating HashState and IntermediateHashes stages again, fixing errors ( #688 )
...
* TestWatchNoDir not to be parallel
* Print ks and inc
* Print addrHash
* Change the buffer
* Print loading
* Skip
* More logging
* Error out earlier
* Handle empty codes
* Handle empty codes
* Remove logging
* Compare states
* Not do stage5
* compareBucket
* Preimage
* Clearer errors
* No need to clean up contract code
* Restore stage6
* Printing
* Skip the skipping
* Print all
* Change buffer type
* Add limit to stage5
* Always fail
* Remove exessive logging
* Restore buffer type
* Revert
* Print when exception
* Reenable skipping
* Skip storage items for deleted contracts
* not shortcut
* Remove removeAccount
* Re-enable state hashing
* Default to plain state
* Disable hashing state
* Reenable reset5
* Print unfurl list
* Enable removingAccount
* No printing
* Reenable stage5 commit
* Swap order of stages
* Prevent backwards promotion, reset tx lookup
* reset finish
* Introduce storage item replacement
* See if unwind works
* Restore removingAccount
* Don't do removeAccount for unwinding
* Possible fix
* Proper(er) fix
* Don't exclude unwinding
* Remove unwinding flag
* Fix formatting
* Fix lint
* Not to ignore blocks if they cause reorg
* Fix test, separate stages again
* Fix TestUnwind
* Fix stage
* Swap unwinding
* Revert to unwinding flag
* Print unfurl list
* Print
* Print inside receive
* Print after
* No printing
* Cleanup
* Not use blockCache when doing GetBlock
2020-06-26 07:46:05 +01:00
Alex Sharov
392bf6e517
Reduce amount of c.Next calls in loader ( #687 )
...
* reduce amount of c.Next calls
* add timings of gen IH to logs
2020-06-25 08:36:03 +01:00
Alex Sharov
b26b00f72b
increase IO buffers size ( #686 )
...
* increase IO buffers size
* found SSD threshold
2020-06-25 08:34:40 +01:00
Alex Sharov
daba929bd8
seek only when need, calculate next subtrie correctly ( #685 )
2020-06-24 07:54:34 +01:00
Alex Sharov
26ccde4614
use lmdb.Append method ( #684 )
...
* use lmdb.Append method
* linter happiness
2020-06-24 07:54:21 +01:00
b00ris
7ac009a7da
Fix last chunk order ( #683 )
...
* fix commit order
* fix lint
2020-06-21 15:13:17 +01:00
