Commit Graph

13080 Commits

Author SHA1 Message Date
alex.sharov
bb55ea81a4 fix print buckets issue 2021-02-09 14:39:31 +07:00
Alex Sharov
7464b7924d
Print tables size during execution stage (#1479) 2021-02-09 12:02:00 +07:00
Alex Sharov
679d70fca8
Print tables size at end of cycle if table>10Gb, and print freelist size (#1478) 2021-02-09 11:42:10 +07:00
alex.sharov
fe193723f7 make lmdb default 2021-02-09 09:36:59 +07:00
Alex Sharov
5ef9a5a9f0
mdbx: switch CI to mdbx, fix SeekExact, print slow/big transactions info (#1475)
DEBUG_BIG_RO_TX_KB - print logs with info about large read-only transactions
DEBUG_BIG_RW_TX_KB - print logs with info about large read-write transactions
DEBUG_SLOW_COMMIT_MS - print logs with commit timing details if commit is slower than this threshold
2021-02-09 09:31:37 +07:00
ledgerwatch
5834189019
Bodies download and simplify header download (#1471)
* Only insert hard-coded tips if both DB and files recovery failed

* Prevent deadlock

* Report some efficiency numbers

* Count properly unrequested bodies

* Initialise allRequests

* Increase timeout

* Fixes to scheduling

* Small fix

* Simplified scheduling

* Remove separate bodyLoop goroutine

* Update from DB at the beginnig of block bodies forward

* Timeout for repeating the request cycle

* Fix timeout

* Fix

* Increase timeout

* Increase timeout

* Try to make flow if possible

* Fix flow

* Lower timeout

* timeout for each blockNum

* Adjustable timeout

* Better log timing

* Track peers

* copy peerID

* fix scheduling

* Too much logging

* Print delivery speed

* Print committed blocks

* Fix race

* Sentry to only reset back-off timer when response to a request is received

* Print bytes/sec and wasted traffic

* Fix bandwidth accounting

* Less logging

* not to wake up on deliveries

* Spam every second

* Print peer map

* Fix npe, print requests

* Timestamps

* Improved logging

* Penalty for peers

* Log penalties only when disconnecting

* Try with smaller window

* window parameter

* Dealing with partially delivered requests

* Init bodyReq

* Fix array index

* More fix for NPE

* More NPE checks

* Print out body progress and header progress

* Fix ending condition

* Bring back waking up and penalties

* Fix duplicate tip

* Duplicate segment

* Fix lint

* Fix lint

* fix lint

* Fix lint

* Hard coded headers in the source files

* Fix lint

* Replace hix-sized header serialisation with rlp (to support clique)

* Remove anchor records from the files

* Fixeds for DB recovery

* Fix compilation

* Fix compile errors

* Fix formatting

* Fix lint

* Fix comments

* Remove headerLoop

* Properly terminate body download

* Support for StatusData p2p proto

* Fix forkid test

* Fix test

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-02-08 09:25:10 +00:00
Alex Sharov
47b4f52db2
mdbx: C11 atomics performance fix and disable MDBX_ENV_CHECKPID (#1472) 2021-02-08 11:35:02 +07:00
Andrew Ashikhmin
f20c08d18b
Merge pull request #1469 from ledgerwatch/m1
Update snappy to prevent crash on Apple M1
2021-02-05 17:22:44 +01:00
yperbasis
21aba6327f Update snappy to prevent crash on Apple M1 2021-02-05 16:13:28 +01:00
Alex Sharov
bf04a6e73d
make_history_check_optional (#1467) 2021-02-05 16:46:43 +07:00
Alex Sharov
f351eea439
Mdbx increase reclaim limit (#1465)
* mdbx_increase_reclaim_limit

* readable
2021-02-05 14:11:42 +07:00
Artem Vorotnikov
91f7798c4a
Remove hashed changesets (#1464)
* Remove hashed changesets

* try this

* oops

* simplify

* better diagnostics

* don't hash account key and use bitmap 64 to check history index

* update docs of bucket

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2021-02-04 12:59:04 +03:00
Igor Mandrigin
29bf7eb864 begin release cycle 2021.02.02 2021-02-04 09:40:21 +01:00
Igor Mandrigin
aa6db643b9 update skip_analysis 2021-02-04 09:27:38 +01:00
Alex Sharov
0c3b5c70b6
mdbx 0.9.3 (#1462) 2021-02-03 10:08:35 +07:00
alex.sharov
25213a51b1 mdbx 0.9.3 2021-02-03 10:06:42 +07:00
Alex Sharov
f0bb3a6dfa
Mdbx new beta version with fixed blockers (#1461) 2021-02-01 20:57:41 +07:00
Alex Sharov
1828bb32fa
Split ih cursors (#1459) 2021-01-29 10:42:48 +07:00
Igor Mandrigin
906d3eefb6 begin release cycle 2021.02.01 2021-01-28 15:17:28 +01:00
alex.sharov
5a9c86e552 devel up 2021-01-28 09:37:24 +07:00
Alex Sharov
049f19cad2
Update README.md 2021-01-28 09:24:36 +07:00
Alex Sharov
6faec7e834
remove blocks compression (#1453) 2021-01-27 20:11:34 +07:00
Alex Sharov
c35b07df3d
senders logging (#1457) 2021-01-27 20:10:21 +07:00
Alex Sharov
e943a73ceb
improve integration (#1456) 2021-01-26 12:19:53 +07:00
alex.sharov
7f2ceca0b7 prom v2.24.1 2021-01-26 11:48:34 +07:00
Andrei Ivasko
022bdeaf1b
hashstate-shutdown (#1452) 2021-01-24 14:04:37 +07:00
Igor Mandrigin
c972a94c7c begin release cycle 2021.01.03 2021-01-21 14:13:03 +01:00
Igor Mandrigin
f2d4a15644 update skip_analysis 2021-01-21 12:18:52 +01:00
Alex Sharov
b47a2fb2a7
Add Golang requirements to Readme.md 2021-01-21 11:40:06 +07:00
Alex Sharov
58c4e03861
Revert "Remove hashed changesets (#1444)" (#1446)
This reverts commit aadb6e47cc.
2021-01-20 14:16:20 +07:00
Artem Vorotnikov
aadb6e47cc
Remove hashed changesets (#1444) 2021-01-19 17:42:26 +07:00
ledgerwatch
7c8bae8591
Implementation of debug_traceCall, add stateDiff to trace_call (#1442)
* Implementation of debug_traceCall

* Fix compile error

* Implementing stateDiff in trace_call - WIP

* fixes to stateDiff, better parsing in rpctest

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-18 11:13:19 +00:00
Artem Vorotnikov
d0d53c1817
ChangeSet cleanup (#1443) 2021-01-17 08:55:04 +07:00
ledgerwatch
820216b77f
Block body download - more recent updates (#1441)
* Combined

* Fix lint

* Fix lint

* Initial bodydownload package

* Updates

* fix lint and process incoming body

* Fix compile errors

* Use deliveries

* Ready for initial testing

* Better handling of missing temp files dir

* Intermediate testing

* First working code

* Fix races, more efficient request scheduling

* Remove excessive logging

* Fix lint

* Interruptibility, fix lint

* Remove more logging

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-15 10:58:04 +00:00
dependabot[bot]
13cc1d0d9a
build(deps): bump axios from 0.19.2 to 0.21.1 in /debug-web-ui (#1435)
Bumps [axios](https://github.com/axios/axios) from 0.19.2 to 0.21.1.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v0.21.1/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.19.2...v0.21.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-15 09:40:15 +00:00
Alex Sharov
7fd1f19435
txdb lazy cursors creation, grpc version up (#1440)
* txdb lazy, grpc version up

* tidy

* fix lint
2021-01-15 09:38:09 +00:00
Alex Sharov
b2c60a6f56
increase concurrent streams cap (#1438) 2021-01-15 09:37:24 +00:00
Igor Mandrigin
d4abe16d25 begin release cycle 2021.01.02 2021-01-14 16:17:08 +01:00
Igor Mandrigin
cc6647193e update skip_analysis 2021-01-14 16:05:14 +01:00
Peter Lai
8b8999078d
Update broken link (#1434) 2021-01-11 12:01:35 +00:00
Alex Sharov
8fdb11d2a6
less RAM for migrations (#1439) 2021-01-11 09:28:25 +07:00
Artem Vorotnikov
f7c5ac42d2
Remove unused fields from remoteCursor (#1437) 2021-01-09 12:29:04 +07:00
Alex Sharov
e6f1e43e6f
lmdb: move fsync inside commit (#1433) 2021-01-04 08:17:18 +00:00
ledgerwatch
baafc90bc6
Fixup after merge (#1432)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-01-02 20:22:35 +00:00
b00ris
8db5790838
State snapshot sync (#1417)
* move experiments to new branch&reorganise kv_snapshot

* walk&modify tests

* added delete from snapshot tests

* fmt

* state snapshot debug

* snapshot validation passed. copy state snapshot

* debug

* snapshot cursor.Prev test

* Prev works correct. Added Current check

* add err check

* added walk forward and backward test

* before refactoring

* refactoring

* execution with snapshot debug

* fix

* remove useless test

* before dupcursor implimentation

* tests with prev and delete works

* execution based on state snapshot passed

* remove useless tests

* blocks to 1140000 passed

* clean verifier

* cleanup state generation

* clean verify && seeder

* remove debug code

* tests passed

* fix lint

* save state

* test passed

* fix lint

* add state hash

* fix lint
2021-01-02 19:28:37 +00:00
Alex Sharov
75cb938980
rpcdaemon performance improve - less reading blocks, less allocs on server (#1426)
* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* rpcdaemon performance improve - less reading blocks, less allocs on server

* don't use global variables

* df

* avoid use non-transactional db instance to reduce amount of cursors
2021-01-02 19:28:22 +00:00
Alex Sharov
6538df5e08
fix_rpcdaemon_panic (#1428) 2020-12-30 00:34:29 +03:00
Alex Sharov
fa1d92bd15
Rm zstd (#1427) 2020-12-28 22:14:16 +07:00
Alex Sharov
d7d9ce0a8e
fix go vet (#1425) 2020-12-26 10:18:16 +07:00
Alex Sharov
311fb591c4
fix check index (#1424) 2020-12-26 09:01:00 +07:00