Commit Graph

856 Commits

Author SHA1 Message Date
alex.sharov
b48aeef8f1 save 2023-03-07 16:10:25 +07:00
Alex Sharov
34ebf6bbea
up grpc version (#7043) 2023-03-07 09:00:07 +00:00
Alex Sharov
b685407175
attempt to fix TestGolangBindings (#7041) 2023-03-07 06:14:35 +00:00
Alex Sharov
d4e25daf22
etl: distinct empty values from nil (#7039)
Reverts ledgerwatch/erigon#7038
2023-03-07 04:19:00 +00:00
Alex Sharov
7206e482da
Revert "etl: distinct empty values from nil" (#7038)
Reverts ledgerwatch/erigon#6934
2023-03-07 03:22:23 +00:00
Alex Sharov
c5acfd0503
etl: distinct empty values from nil (#6934) 2023-03-07 02:33:21 +00:00
hexoscott
77065ee0f3
remove PeerUseless calls (#7032) 2023-03-06 18:59:14 +00:00
ledgerwatch
1ab083316b
More block snapshots for ETH mainnet, goerli and gnosis (#7026)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-03-06 08:58:58 +00:00
Giulio rebuffo
9101fdadcc
Long keys and finalization markers added. (#7024) 2023-03-05 19:15:17 +00:00
awskii
e48609ee8d
E4 btree-like indexes (#6834)
This branch implements support of btree indexes instead of .kvi files.
Kvi also created, but not used during search/lookup, only for size
comparation.

Btree allows to make seek calls, while recsplit indexes does not support
that query.
During initial tests, btree indexes are ~1-3% smaller than recsplit but
avg lookup time bigger up to 1.5us (which compensates with ability to
fetch next key).

BtAlloc
```
337000 searches, last took 5.417µs avg=4.385µs next_access_last[of 10000 keys] 105ns
338000 searches, last took 3.792µs avg=4.385µs next_access_last[of 10000 keys] 103ns
339000 searches, last took 5.125µs avg=4.386µs next_access_last[of 10000 keys] 103ns
340000 searches, last took 4.75µs avg=4.386µs next_access_last[of 10000 keys] 103ns
341000 searches, last took 3.875µs avg=4.386µs next_access_last[of 10000 keys] 104ns
342000 searches, last took 6.042µs avg=4.387µs next_access_last[of 10000 keys] 107ns
343000 searches, last took 3.209µs avg=4.388µs next_access_last[of 10000 keys] 108ns
344000 searches, last took 2.292µs avg=4.393µs next_access_last[of 10000 keys] 113ns
345000 searches, last took 2.417µs avg=4.394µs next_access_last[of 10000 keys] 106ns
346000 searches, last took 2.625µs avg=4.394µs next_access_last[of 10000 keys] 103ns
347000 searches, last took 4.417µs avg=4.397µs next_access_last[of 10000 keys] 104ns
348000 searches, last took 5.458µs avg=4.398µs next_access_last[of 10000 keys] 103ns
349000 searches, last took 4.875µs avg=4.4µs next_access_last[of 10000 keys] 102ns
350000 searches, last took 4.25µs avg=4.4µs next_access_last[of 10000 keys] 104ns
351000 searches, last took 6.25µs avg=4.403µs next_access_last[of 10000 keys] 106ns
352000 searches, last took 3.125µs avg=4.404µs next_access_last[of 10000 keys] 106ns
```

Recsplit
```
354000 searches, last took 1.209µs avg=3.081µs
355000 searches, last took 1.333µs avg=3.076µs
356000 searches, last took 1.541µs avg=3.071µs
357000 searches, last took 1.333µs avg=3.066µs
358000 searches, last took 1.333µs avg=3.062µs
359000 searches, last took 1.167µs avg=3.057µs
360000 searches, last took 1.791µs avg=3.052µs
361000 searches, last took 1.209µs avg=3.048µs
362000 searches, last took 1.375µs avg=3.043µs
363000 searches, last took 1.166µs avg=3.038µs
364000 searches, last took 1.625µs avg=3.033µs
365000 searches, last took 1.125µs avg=3.029µs
366000 searches, last took 1.208µs avg=3.024µs
367000 searches, last took 1.333µs avg=3.019µs
368000 searches, last took 1.791µs avg=3.015µs
369000 searches, last took 1.333µs avg=3.01µs
370000 searches, last took 1.166µs avg=3.006µs
371000 searches, last took 1.125µs avg=3.001µs
372000 searches, last took 1.333µs avg=2.997µs
```

https://github.com/ledgerwatch/erigon-lib/pull/891/files
2023-03-04 03:19:25 +00:00
Andrew Ashikhmin
1dab298db7
Add Prague fork support (#7005)
Prague is the next EL fork after
[Cancun](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/cancun.md).
This change picks up https://github.com/ledgerwatch/erigon-lib/pull/913,
adds support of the fork, and moves Verkle trees to it.
2023-03-02 17:51:36 +00:00
Alex Sharov
a4f69abb97
e3: less locks, less casts (#6991) 2023-03-01 07:59:56 +00:00
alex.sharov
4fd421a3f0 use correct mdbx version in go.mod 2023-03-01 13:13:57 +07:00
alex.sharov
85df06ca26 remove zstd dependency 2023-03-01 12:57:15 +07:00
Alex Sharov
8bffe4499b
e3: less wal locks (#6971) 2023-02-27 07:03:00 +00:00
Alex Sharov
1a141a6219
mdbx: more funcs (#6961) 2023-02-26 02:29:48 +00:00
Alex Sharov
fae889f815
e3: index readers pool (#6954) 2023-02-26 02:04:46 +00:00
Alex Sharov
2a8eec51bc
e3: eliasfano iterator.seek() (#6944) 2023-02-24 13:39:19 +00:00
Alex Sharov
58236d62bb
e3: optimize iterators, step1 (#6937) 2023-02-24 10:32:50 +07:00
Alex Sharov
97d6cd5c11
e3: more mainnet snapshots (#6942) 2023-02-24 10:29:50 +07:00
ledgerwatch
9c9449a059
Mock of Erigon support - step 1 (#6930)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-02-22 12:38:22 +00:00
Alex Sharov
a19d1b59c5
Pool and RPCBlocksCache: to use generic lru (#6928) 2023-02-22 07:50:00 +00:00
Alex Sharov
be4f5cf078
mdbx: to support empty key/values #6923 (#6927) 2023-02-22 07:04:48 +00:00
Alex Sharov
7f0abb8250
downloader: suspend cancelation warning #899 (#6926) 2023-02-22 11:32:19 +07:00
Alex Sharov
309b4812d3
mdbx: to support empty key/values (#6923) 2023-02-22 04:10:57 +00:00
Alex Sharov
510993f09f
libp2p pubsub panic fix (#6924)
see: https://github.com/libp2p/go-libp2p-pubsub/pull/522
2023-02-22 03:07:17 +00:00
Stéphane Loeuillet
195a72bf74
Initial GraphQL interface implementation (#6821)
That's an initial PR mostly for code review, not ready for production
use

Got basic GraphQL working when querying a single block

---------

Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
2023-02-20 11:23:06 +00:00
Alex Sharov
2b0d0d8811
EliasFano: Search to touch only UpperBits array (#6904) 2023-02-20 06:42:37 +00:00
Alex Sharov
c3325b989f
torrent lib version up (#6903) 2023-02-20 04:43:03 +00:00
dependabot[bot]
3b68ccd823
build(deps): bump golang.org/x/net from 0.6.0 to 0.7.0 (#6896)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.6.0 to
0.7.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8e2b117aee"><code>8e2b117</code></a>
http2/hpack: avoid quadratic complexity in hpack decoding</li>
<li><a
href="547e7edf38"><code>547e7ed</code></a>
http2: avoid referencing ResponseWrite.Write parameter after
returning</li>
<li><a
href="39940adcaa"><code>39940ad</code></a>
html: parse comments per HTML spec</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.6.0...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.6.0&new-version=0.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-18 10:29:20 +07:00
Giulio rebuffo
df1563b401
Passing all finality consensus-spec-tests for altair/bellatrix (#6894) 2023-02-17 17:57:05 +00:00
Alex Sharov
433f031db3
trackers up (#6890) 2023-02-17 02:38:25 +00:00
alex.sharov
349bdb7a4f e3: less mainnet snapshots 2023-02-15 19:23:18 +07:00
alex.sharov
ada7ab2da8 e3: remove some snaps 2023-02-14 09:45:20 +07:00
Alex Sharov
999899d66d
e3: read files list from db (#6833) 2023-02-13 05:17:01 +00:00
Alex Sharov
d5bb42453a
Update lib-p2p to support golang 1.20 (#6848) 2023-02-12 11:23:33 +00:00
Alex Sharov
de17e82375
e3: reduce buildIndex ctx scope (#6850) 2023-02-12 16:24:39 +07:00
Alex Sharov
acd76ea520
e3: temporary remove semaphore (#6849) 2023-02-12 16:06:28 +07:00
ledgerwatch
ee83447fa1
CL lightclient to create memdb in the tmpdir which gets cleaned up on… (#6829)
… startup

---------

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-02-11 20:44:51 +00:00
Alex Sharov
c6f6aab542
e3: more mainnet snapshots (#6839) 2023-02-11 09:38:04 +07:00
Giulio rebuffo
1b862a7c31
Added processing of sync committee updates (#6836) 2023-02-10 16:17:12 +00:00
Alex Sharov
7c9f30d25b
e3: return list of e3 files by grpc (#6827) 2023-02-10 13:45:33 +07:00
Alex Sharov
983db55dc5
Downloader: "incomplete" markers are more important than "complete", so commit them with fsync (#6825) 2023-02-10 13:20:19 +07:00
Alex Sharov
98a8ca50fe
erigon3 snapshots version 3 (#6811) 2023-02-09 13:51:51 +07:00
Alex Sharov
c31824ee17
less logs (#6813) 2023-02-09 13:27:45 +07:00
Alex Sharov
ec52990c43
e3: Thread safe reopen folder (#6810) 2023-02-09 12:11:40 +07:00
Alex Sharov
a75a948beb
e3: MakeContext must not see garbage files (deleted, overlapped, etc...). make it cheaper. (#6803) 2023-02-08 12:50:47 +07:00
Giulio rebuffo
a585ae4894
Added Optimizations to Beacon state transition. (#6792) 2023-02-07 17:44:37 +01:00
Alex Sharov
4cae1b94e9
switch sentinel to native go errors (#6782) 2023-02-06 10:15:52 +07:00
Alex Sharov
dede1a2c9e
e3: more mergeFiles tests, refcnt for LocalityIndex (#6765) 2023-02-06 09:05:29 +07:00