Alex Sharov
7dd678896a
downloader: move from snapshots/db to snapshots/downloader ( #8375 )
2023-10-05 14:25:00 +07:00
Alex Sharov
0165d00ee6
downloader: don't drop torrents after download (performance problem there solved) ( #8358 )
2023-10-04 12:34:45 +07:00
Alex Sharov
fa3b8c23b2
Downloader: step towards more complex datadir ( #8286 )
...
migration included - no manual actions required
2023-10-04 11:01:02 +07:00
Alex Sharov
ce47ad30e2
downloader: progress print better ( #8350 )
2023-10-04 09:57:37 +07:00
Alex Sharov
8dda88d866
downloader: check too big and too small content ( #8349 )
2023-10-03 14:24:08 +07:00
Alex Sharov
dbdb486dd3
downloader: download .torrent files from webseeds provider ( #8346 )
2023-10-03 11:21:41 +07:00
Alex Sharov
d0366ae34e
snapshots: remove concept of separated hist .toml file ( #8326 )
2023-09-29 11:30:02 +07:00
Alex Sharov
6cc2bd5751
downloader: non-readonly open db (so it can auto-recover if need) ( #8312 )
2023-09-28 11:38:29 +07:00
Alex Sharov
eb747c042a
Downloader: add tests to protect from re-downloading ( #8191 )
2023-09-15 13:46:59 +07:00
Alex Sharov
93fbda14a5
downloader: support --chain parameter ( #8184 )
2023-09-13 15:21:43 +07:00
Alex Sharov
2158b4916d
Downloader: correct logging when create .torrent files ( #8182 )
2023-09-13 11:49:45 +07:00
Alex Sharov
3cea1b9b9e
torrent: add --webseeds cli arg ( #8176 )
2023-09-12 12:18:47 +07:00
Alex Sharov
bf8ac9a38f
mdbx_to_mdbx: to use logger ( #7860 )
2023-07-09 08:04:20 +01:00
ledgerwatch
e75ea786c0
[devnet tool] separate logging ( #7526 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-17 07:36:06 +01:00
Alex Sharov
a2ad68d634
downloader: speedup "--downloader.verify" mode ( #7529 )
...
deduplicate logic
create more producer goroutines (torrent lib does limiting internally
amount of consumers/disk-readers/hashers by 2, and it's enough because
we can verify multiple files in parallel)
move flag from "downloader torrent_hashes --verify" to "downloader
--verify"
2023-05-17 06:33:59 +01:00
Alex Sharov
f23612bdfe
Enode logging broke when NAT Parameter set in 2.43.0 ( #7480 )
...
for https://github.com/ledgerwatch/erigon/issues/7472
2023-05-10 10:25:53 +07:00
ledgerwatch
fdd385cef1
[Devnet tool] Side-quest to improve logging - part 1 ( #7445 )
...
This is the beginning of the series of changes to make it possible to
run multiple instances of erigon inside a single process (as devnet tool
does), with the logging from these processes going to respective log
files correctly.
This is the first part where the initial infrastructure is being
established
---------
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-07 07:28:15 +01:00
Alex Sharov
e1f549ff01
cli: use SplitAndTrim instead of strgings.Split(",") ( #7369 )
2023-04-23 10:54:55 +07:00
ledgerwatch
9690228ede
[Diagnostics] Simplify logging settings, introduce correct log rotation with lumberjack ( #7273 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-04-07 21:08:44 +00:00
Alex Sharov
a44e0afbd6
Downloader main loop wait on close ( #7082 )
2023-03-13 02:12:30 +00:00
Alex Sharov
92c444bc1a
--torrent.staticpeers ( #7052 )
2023-03-08 08:30:00 +00:00
Alex Sharov
999899d66d
e3: read files list from db ( #6833 )
2023-02-13 05:17:01 +00:00
Alex Sharov
983db55dc5
Downloader: "incomplete" markers are more important than "complete", so commit them with fsync ( #6825 )
2023-02-10 13:20:19 +07:00
Enrique Jose Avila Asapche
e5ddccf212
Disable ipv6 flag ( #6295 )
...
This was a great idea #6261
Added two new flags to turn off downloader ipv6 and/or ipv4 usage
2022-12-15 16:40:59 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 ( #6076 )
2022-11-20 10:41:30 +07:00
Alex Sharov
93926646f8
e3: prepare downloader for extraction ( #6061 )
2022-11-16 15:48:23 +03:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live ( #5857 )
2022-10-25 09:58:25 +07:00
Alex Sharov
606ce5c99a
add logging cli flags to various cmd ( #5816 )
2022-10-21 12:36:17 +07:00
Max Revitt
07ffa36d44
File system logging ( #5812 )
...
- lives in internal/logging
- all log flags moved to internal/logging/flags
- allows continued use of root logger via log.Info etc.
- update logger to take change allowing string to lvl for 'trace'
Verbosity flag is overridden by log.console.verbosity. Logs will be
colocated if all run as one process, only split where progs are run as
separate processes, in a future update this will be addressed so for
example rpcdeamon will always log to it's own file
2022-10-20 19:25:06 +01:00
Alex Sharov
9fb8a190bc
erigon22: folder snapshots/history ( #5351 )
2022-09-18 17:41:01 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt ( #4988 )
2022-08-10 19:04:13 +07:00
Leonard Chinonso
43f5ab09f8
Fixed lint errors for gosec linter ( #4933 )
...
* Fixed lint errors for gosec linter
* Changed uint64 to int64 in common/GetRandInt
* Changed naming of function
2022-08-04 14:59:40 +01:00
Alex Sharov
2415fecb26
--downloader.verfiy flag to verify once on startup ( #4597 )
...
* save
* save
* save
* save
* save
* save
2022-07-01 16:52:43 +06:00
Alex Sharov
f613fcafd2
torrent verbosity allow debug #4569 Open
2022-06-29 08:42:11 +06:00
Enrique Jose Avila Asapche
64697a9647
torrent verbosity now uses int ( #4551 )
...
* torrent verbosity now uses int
* logging torrent verbosity
* moved crit correctly
2022-06-29 08:29:59 +06:00
Alex Sharov
b9cb6d953e
Fix nil td ( #4546 )
...
* save
* save
2022-06-26 17:27:14 +06:00
Alex Sharov
afd07e5dee
--no-downloader flag support ( #4545 )
2022-06-26 17:13:32 +06:00
Alex Sharov
a53642b4bf
datadir.Dirs configuration object to group dir config ( #4387 )
2022-06-07 10:24:50 +07:00
Alex Sharov
1eb9aec08a
Better validate --torrent.verbosity flag ( #4257 )
...
* lastDup no key
* lastDup no key
* lastDup no key
2022-05-25 15:24:11 +07:00
Alex Sharov
4e7807251a
Ensure snapshotDir not created by accident ( #4164 )
...
* save
* save
* save
2022-05-17 09:40:45 +07:00
Alex Sharov
45adf8fba9
Torrent: don't drop db on "torrent_hashes --rebuild" #4118 Open
2022-05-11 14:38:37 +07:00
Alex Sharov
4ba1f03efe
Snapshots: atomic dir, step 3 ( #4103 )
2022-05-10 09:29:44 +07:00
Alex Sharov
32d8ab34d9
atomic snapshot dir, step 2 ( #4093 )
...
* atomic snapshot dir
* atomic snapshot dir
2022-05-07 14:53:21 +07:00
Alex Sharov
f480865a26
Downloader atomic snapshot dir, step 1 ( #4085 )
...
* save
* save
* save
2022-05-06 09:59:23 +07:00
Alex Sharov
43ddea2adf
Torrent: maxpeers flag were used incorrectly
2022-04-28 11:38:16 +07:00
Alex Sharov
ea8a347ece
Downloader: calc stat inside, add --torrent.download.slots and limit downloads inside ( #3986 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-04-28 09:10:00 +07:00
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os ( #3946 )
2022-04-23 15:43:00 +01:00
Alex Sharov
7a226ef94c
Downloader: re-use flags defaults ( #3941 )
2022-04-22 17:30:48 +07:00
Alex Sharov
9051d95836
Downloader tables, mdbx-based torrent pieces completion store ( #3923 )
...
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-04-21 10:34:31 +07:00
Alex Sharov
6aa4999729
Parallel indexing ( #3921 )
2022-04-21 10:34:10 +07:00
