Commit Graph

12519 Commits

Author SHA1 Message Date
Marius van der Wijden
f6ba57ebc1 metrics: zero temp variable in updateMeter (#21470)
* metrics: zero temp variable in  updateMeter

Previously the temp variable was not updated properly after summing it to count.
This meant we had astronomically high metrics, now we zero out the temp whenever we
sum it onto the snapshot count

* metrics: move temp variable to be aligned, unit tests

Moves the temp variable in MeterSnapshot to be 64-bit aligned because of the atomic bug.
Adds a unit test, that catches the previous bug.
2020-08-29 12:30:07 +02:00
Péter Szilágyi
f734b7a0eb travis, dockerfile, appveyor, build: bump to Go 1.15
# Conflicts:
#	Dockerfile
#	Dockerfile.alltools
#	build/checksums.txt
2020-08-29 12:29:55 +02:00
Péter Szilágyi
bfac425c87 core: define and test chain rewind corner cases (#21409)
* core: define and test chain reparation cornercases

* core: write up a variety of set-head tests

* core, eth: unify chain rollbacks, handle all the cases

* core: make linter smile

* core: remove commented out legacy code

* core, eth/downloader: fix review comments

* core: revert a removed recovery mechanism
# Conflicts:
#	core/blockchain.go
#	core/blockchain_test.go
#	core/headerchain.go
#	core/rawdb/database.go
#	core/rawdb/freezer.go
#	core/rawdb/schema.go
#	eth/downloader/downloader.go
#	eth/sync.go
#	trie/sync.go
2020-08-29 12:29:00 +02:00
Marius van der Wijden
08d3993cef core: more detailed metering for reorgs (#21420) 2020-08-27 21:23:51 +02:00
Martin Holst Swende
cac316dc82 cmd/evm: statet8n output folder + tx hashes on trace filenames (#21406)
* t8ntool: add output basedir

* t8ntool: add txhash to trace filename

* t8ntool: don't default to '.' basedir, allow absolute paths
# Conflicts:
#	cmd/evm/internal/t8ntool/execution.go
#	cmd/evm/internal/t8ntool/transition.go
2020-08-27 17:41:02 +02:00
Péter Szilágyi
8d7d853961 build: drop disco, enable groovy on Ubuntu PPAs
# Conflicts:
#	build/ci.go
2020-08-27 17:38:36 +02:00
Giuseppe Bertone
5388193c2c core/state: fixed some comments (#21450)
# Conflicts:
#	core/state/state_object.go
#	core/state/statedb.go
#	core/state/statedb_test.go
2020-08-27 17:38:12 +02:00
Marius van der Wijden
ece9515974 metrics: make meter updates lock-free (#21446) 2020-08-27 17:35:40 +02:00
Martin Holst Swende
db716f5864 cmd/geth/tests: try to fix spurious travis failure in les tests (#21410)
* cmd/geth/tests: try to fix spurious travis failure in les tests

* cmd/geth: les_test - remove extraneous option during boot
# Conflicts:
#	cmd/geth/les_test.go
2020-08-27 17:35:34 +02:00
Péter Szilágyi
26477944e5 params: begin v1.9.20 release cycle
# Conflicts:
#	params/version.go
2020-08-27 17:35:14 +02:00
Igor Mandrigin
03c1b48180 calver: start 2020.09.01 release 2020-08-27 10:55:52 +02:00
Igor Mandrigin
12aae871fd skip_analysis: update block number 2020-08-27 10:35:32 +02:00
Alex Sharov
f54812184b
make all in Dockerfile (#980) 2020-08-27 13:45:24 +07:00
Alex Sharov
584a5581c8
show "Done!" message only if stage took more than 30 sec (#977) 2020-08-27 12:43:37 +07:00
Alex Sharov
714e5041e2
Better logs3 (#976) 2020-08-26 19:06:51 +07:00
Alex Sharov
9091ce62bb
KV reduce features amount (#975)
* move IdealBatchSize method to Batch object :-)
remove cursor.Walk
remove kv.IdealBatchSize

* remove bolt_db.go file

* move method NoValueCursor to tx object
2020-08-26 07:03:50 +01:00
Alex Sharov
522287ac18
Transactional cycle (#966)
* v0

* v1

* v3

* v4

* clean

* temporary fix of txpool

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* Add debug logs about tx start/commit

* add condition

* tx pool to not hold own db

* try to enable TxPool in integration

* exclude tx pool from tx

* exclude tx pool from integration

* reduce limit

* fix integration

* clean

* clean

* clean

* clean

* clean

* exclude tx pool unwind

* exclude tx pool unwind in integration

* fix integration tx pool

* fix commit

* fix current stage after unwind

* fix current stage after unwind

* fix linter

* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.

* move body and headers unwind out of tx

* fix unwind order after reboot

* add support external tx to exec stage

* clean

* clean

* clean

* clean

* clean

* add logs

* better id check

* better id check
2020-08-26 07:02:10 +01:00
Alex Sharov
325785aba4
add net namespace (#974) 2020-08-25 20:04:01 +07:00
Alex Sharov
c61f8a5bdb
After reboot unwind stack has wrong order (#972)
* fix unwind order after reboot

* Fix test

Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
2020-08-24 18:15:22 +01:00
ledgerwatch
8728e54a72
Another step in migrating tests to staged sync (#970)
* Log in GetAsOf

* Default plain state

* trace

* More test fixes

* Fix lint

* Fix

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Fix for events

* Repair events

* Fix lint

* Fix lint

* Change back to InsertChain
2020-08-24 18:00:40 +01:00
Alex Sharov
43a4e34a6e
[to discuss] exec blocks without ObjectDb: Mutation over TxDb (#947)
* use mutation over tx

* clean

* clear

* add .CommitAndBegin() method

* clean

* increase timings for logging

* return ideal batch size
2020-08-24 12:07:59 +01:00
Giulio rebuffo
2cd825ab63
added message for --rpc (#968)
* added message for --rpc

* more messages for other unused flags
2020-08-24 07:32:14 +01:00
Igor Mandrigin
9b81829616
etl: create a subfolder in datadir for temp files. (#965) 2020-08-23 10:53:01 +01:00
Alex Sharov
2facf92533
Better logs2 (#964) 2020-08-23 06:55:17 +07:00
Giulio rebuffo
c2b3d137f7
implemented eth_getblockbyhash in rpcdaemon (#963) 2020-08-22 21:13:38 +01:00
Alex Sharov
a6be18b915
ticker-based logs (#954)
* timer-based logs

* timer-based logs

* delegate progress calculation to user

* delegate progress calculation to user

* delegate progress calculation to user

* clear

* add logs to senders recovery

* use default dir in integration

* more logs

* more logs
2020-08-22 12:12:33 +02:00
Igor Mandrigin
18df6cd182
Fix parallel recovery senders (#962)
* fix parallel context creation

* fix a typo
2020-08-22 12:08:47 +02:00
ledgerwatch
7c16160087
Add IntraBlockState copy function to RPC API (#960) 2020-08-21 21:53:23 +01:00
Igor Mandrigin
a119a3a35f
Simplify build system (#955) 2020-08-21 14:38:11 +02:00
Giulio rebuffo
0ffe32ae16
State growth restore (#951)
* new state growth

* lint
2020-08-21 07:32:11 +01:00
Alex Sharov
ebbaa55672
sub-transaction to not call fsync, sub-transactions to not call runtime.LockThread() (#945) 2020-08-21 07:31:43 +01:00
Igor Mandrigin
f46a72bdd9
Print approximate progress for ETL stages (#950) 2020-08-21 07:30:30 +01:00
Igor Mandrigin
fbcf48f29d
Store ethash datafiles in a separate directory. (#949) 2020-08-20 15:59:06 +01:00
ledgerwatch
01f211f3b8
Revert "IH: no retain list (#946)" (#948)
This reverts commit f86ed64288.
2020-08-20 13:37:34 +01:00
Alex Sharov
f86ed64288
IH: no retain list (#946)
* experiment of not using retain list

* better handle sub-tx

* save progress

* don't touch collector

* don't run collector on sub-tx because of `mdb_cursor_put: MDB_TXN_FULL: Transaction has too many dirty pages - transaction too big`

* clear

* clear
2020-08-20 11:42:43 +01:00
Igor Mandrigin
3f81472831
Never store Ethash caches to disk (#944)
* always keep ethash caches in ram

* fix tests
2020-08-20 11:12:51 +01:00
Alex Sharov
f827f86c11
Rpcdaemon as lib 2 (#943) 2020-08-20 10:52:27 +07:00
ledgerwatch
fd7e91a51c
Update skip_analysis (#942) 2020-08-19 20:34:45 +01:00
ledgerwatch
53f7efb4b0
Fix cmd/state flags (#941) 2020-08-19 14:52:03 +01:00
Alex Sharov
1dcc2b141a
Rpcdaemon as lib (#940)
* share config object

* create default config and logger

* move db connection to common func

* move server start to cli package

* clear

* clear

* rename cli to rpc

* use unified SetupLogger func

* make all root flag persistent

* use common flags in different packages

* use common flags in different packages

* move TraceTx method to eth package

* use native slice flags

* create package "turbo"

* disable geth api

* disable geth api

* move more data types to turbo/adapter package

* add support for customApiList

* run more

* run more

* run more

* dog-food

* move DoCall

* move DoCall

* fix tests

* fix test
2020-08-19 12:46:20 +01:00
b00ris
eb46dd68df
Account range testing (#938)
* save state

* save state

* save new timeouts

* compare passed

* save state

* cleanup
2020-08-19 07:33:49 +01:00
Evgeny Danilenko
3b40819444
grpc regenerate (#936)
* grpc regenerate

* use string type for id in net_version
2020-08-18 20:22:49 +03:00
ledgerwatch
09440dccd3
Print memory stats (#934) 2020-08-17 16:27:42 +01:00
Evgeny Danilenko
6976f58f6a
net api service (#933) 2020-08-17 16:27:29 +01:00
Alex Sharov
081d02920d
Remove "Bucket" class from KV abstraction (#928)
* drop Bucket object from KV abstraction

* add some non-public methods

* rebase master

* rebase master
2020-08-17 08:24:59 +01:00
Alex Sharov
7138d6740e
add logging to ih stage (#921)
* add logging to loader

* better logging
2020-08-17 08:24:48 +01:00
Alex Sharov
b2e35eb63f
rename "remote2" to "remote" (#932) 2020-08-17 07:47:06 +01:00
Alex Sharov
820d6d0adf
[merge after release] enable IH sequence feature (#920)
* add logging to loader

* enable IH sequence feature

* clean
2020-08-17 07:46:22 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size (#918)
* add logging to loader

* use pure tx in etl loading, logs in mutation commit

* clean

* bletter logging and more cleanup

* bletter logging and more cleanup

* increase batch size to 500M

* better batch commit logging

* async fsync

* sync fsync

* sync fsync

* unify logging

* fix corner-case when etl can use empty bucket name

* fix tests

* better logging

* better logging

* rebase master

* remove lmdb.NoMetaSync flag for now

* consistent walk and multi-walk

* clean

* sub tx

* add consistent multi-put

* implement dupsort support in one new cursor method

* clear
2020-08-17 07:45:52 +01:00
Alex Sharov
6379516a0f
add support of blocks compression to txpool stage (#925) 2020-08-17 10:17:40 +07:00