d71fd41249
[fragment] Read downloaded fragments only when needed ( #3069 )
...
Authored by: Lesmiscore
2022-03-15 12:27:41 +09:00
b3edc8068e
[downloader/mhtml] Fix fragments with absolute urls ( #3044 )
...
Authored-by: coletdjnz
2022-03-13 22:03:40 +00:00
592b748582
[cleanup] Minor cleanup
...
Closes #3006
2022-03-11 19:40:15 +05:30
cf4f42cb97
Protect stdout from unexpected progress and console-title
...
Closes #3023
2022-03-11 19:29:45 +05:30
08d30158ec
[cleanup, docs] Misc cleanup
...
Closes #2828 , closes #2734 , closes #2802 , closes #2937
2022-03-08 22:38:06 +05:30
45806d44a7
[downloader] Obey `--file-access-retries` when deleting/renaming ( #2224 )
...
Authored by: ehoogeveen-medweb
2022-03-03 06:33:32 -08:00
93c8410d33
[downloader/fragment] Fix bugs around resuming with Range ( #2901 )
...
Authored by: Lesmiscore
2022-02-28 13:10:54 +09:00
195c22840c
[downloader/fragment] Ignore `FileNotFoundError` when downloading livestreams
...
when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403).
this is causing lack of the file and `FileNotFoundError`
lacking segment doesn't have any data, so it's safe to ignore
2022-02-26 12:34:36 +09:00
f0734e1190
[downloader/fragment] Fix problem where multiple threads can share one iterator
...
which causes "ValueError: generator already executing" error
Closes #2881
2022-02-25 13:22:17 +09:00
15dfb3929c
[fc2:live] Add extractor ( #2418 )
...
Authored by: Lesmiscore
2022-02-25 11:16:23 +09:00
a539f06570
[downloader/fragment] Improve `--live-from-start` for YouTube livestreams ( #2870 )
2022-02-25 02:00:46 +09:00
dcd55f766d
[aria2c] Add `--http-accept-gzip=true`
...
Closes #1936 , #1236
2022-02-18 18:03:20 +05:30
079a7cfc71
[downloader] Do not use aria2c for non-native `m3u8`
...
Closes #2718
2022-02-11 12:09:03 +05:30
1e5d87beee
[websocket] Make syntax error in `websockets` module non-fatal
...
Closes #2633
2022-02-04 12:02:01 +05:30
1d3586d0d5
[aes] Add unpad_pkcs7
2022-02-01 00:29:36 +05:30
af6793f804
[downloader/ffmpeg] Handle unknown formats better
2022-01-24 01:15:54 +05:30
d57576b9d9
[httpie] Fix `available` method
...
Closes #2330
2022-01-13 22:20:59 +05:30
f44afb54ef
[aria2c] Don't show progress when `--no-progress`
2021-12-27 04:27:34 +05:30
185bf31070
[youtube] End `live-from-start` properly when stream ends with 403
...
Closes #2089
2021-12-26 16:14:00 +05:30
205a0654c0
Add option `--file-access-retries` ( #2066 )
...
Closes #517
Authored by: ehoogeveen-medweb
2021-12-23 07:59:03 +05:30
b69fd25c25
[cleanup] Misc cleanup
...
Closes #1942 #1976 #2020 #2058 #1984
2021-12-23 07:12:46 +05:30
4ac5b94807
[dash] Fix `--test`
...
Bug in adbc4ec4bb
2021-12-23 03:34:18 +05:30
4273cc776d
[dash] Fix aria2c dash downloads
...
Bug in adbc4ec4bb
2021-12-21 21:40:04 +05:30
adbc4ec4bb
[dash,youtube] Download live from start to end ( #888 )
...
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms
Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats
Closes #1521
Authored by: nao20010128nao, pukkandan
2021-12-20 11:36:46 +05:30
f304da8a29
[cleanup] Misc cleanup
...
Closes #1805 , closes #1800
2021-11-29 23:34:33 +05:30
7578d77d8c
[downloader] Add colors to download progress
2021-11-29 22:51:18 +05:30
234416e4bf
[downloader/ffmpeg] Fix for direct videos inside mpd manifests
...
Closes #1751
2021-11-23 01:29:10 +05:30
d76991ab07
Fix `--check-formats` for `mhtml`
...
Closes #1709
2021-11-20 08:33:55 +05:30
6b993ca765
[hls] Better FairPlay DRM detection ( #1661 )
...
Authored by: nyuszika7h
2021-11-19 07:19:51 +05:30
59a7a13ef9
[docs] Minor documentation improvements
...
Closes #1583 , #1599
2021-11-10 04:19:52 +05:30
a4211baff5
[cleanup] Minor cleanup
2021-11-04 03:53:15 +05:30
8913ef74d7
[ffmpeg] Detect libavformat version for `aac_adtstoasc`
...
and print available features in verbose head
Based on https://github.com/ytdl-org/youtube-dl/pull/29581
2021-11-04 03:13:37 +05:30
bd93fd5d45
[fragment] Fix progress display in fragmented downloads
...
Closes #1517
2021-11-03 16:45:58 +05:30
49a57e70a9
[cleanup] misc
2021-10-23 02:09:10 +05:30
d3c93ec2b7
Don't create console for subprocesses on Windows ( #1261 )
...
Closes #1251
2021-10-20 21:49:40 +05:30
b4b855ebc7
[fragment] Print error message when skipping fragment
2021-10-19 22:58:26 +05:30
2cda6b401d
Revert "[fragments] Pad fragments before decrypting ( #1298 )"
...
This reverts commit 373475f035
2021-10-19 22:58:25 +05:30
aa7785f860
[utils] Standardize timestamp formatting code
...
Closes #1285
2021-10-19 22:58:25 +05:30
9fab498fbf
[http] Retry on socket timeout
...
Closes #1222
2021-10-19 22:58:24 +05:30
373475f035
[fragments] Pad fragments before decrypting ( #1298 )
...
Closes #197 , #1297 , #1007
Authored by: shirt-dev
2021-10-18 08:14:20 +05:30
580d3274e5
[youtube] Expose different formats with same itag
2021-10-16 20:28:17 +05:30
03b4de722a
[downloader] Fix slow progress hooks
...
Closes #1301
2021-10-16 20:02:40 +05:30
d5a39f0bad
[http] Show the last encountered error
...
Closes #1262
2021-10-14 14:44:28 +05:30
fc5c8b6492
[eria2c] Fix --skip-unavailable fragment
2021-10-13 04:14:12 +05:30
c111cefa5d
[downloader/ffmpeg] Improve simultaneous download and merge
2021-10-13 04:14:11 +05:30
21186af70a
[downloader] Fix throttledratelimit
...
The timer should not reset at start of each block
2021-10-10 09:32:00 +05:30
d1d5c08f29
[minicurses] Fix when printing to file
...
Closes #1215
2021-10-10 09:31:59 +05:30
b5ae35ee6d
[cleanup] Misc cleanup
2021-10-09 22:32:00 +05:30
b922db9fe5
[http] Respect user-provided chunk size over extractor's
2021-10-09 20:49:07 +05:30
819e05319b
Improved progress reporting (See desc) ( #1125 )
...
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting
Closes : #906 , #901 , #1085 , #1170
2021-10-09 00:41:59 +05:30
49e7e9c3ce
[docs,build] Change all pycryptodome references to pycryptodomex
2021-10-06 06:45:45 +05:30
7687c8ac6e
[HLS] Fix decryption issues ( #1117 )
...
* Unpad HLS fragments with PKCS#7 according to datatracker.ietf.org/doc/html/rfc8216
* media_sequence should only be incremented in for media fragments
* The native decryption should only be used if ffmpeg is unavailable since it is significantly slower. Closes #1086
Authored by: shirt-dev, pukkandan
2021-09-29 00:23:24 +05:30
99e9e001de
[youtube] Cleanup authentication code ( #786 )
...
Authored by: coletdjnz
2021-09-24 06:22:17 +05:30
be867b03f5
bugfix for bd50a52b0d
2021-09-24 05:14:16 +05:30
ee2b3563f3
[downloader/niconico] Pass custom headers ( #1063 )
...
Closes #1057
Authored by: nao20010128nao
2021-09-23 14:36:48 +05:30
bd50a52b0d
Basic framework for simultaneous download of multiple formats ( #1036 )
...
Authored by: nao20010128nao
2021-09-22 19:42:04 +05:30
d806c9fd97
[docs,cleanup] Add deprecation warning in docs
...
for some counter intuitive behaviour that may be removed in future.
and fix linter
2021-09-22 05:50:11 +05:30
1009f67c2a
[fragment,aria2c] Generalize and refactor some code
2021-09-22 05:27:07 +05:30
d9d8b85747
[fragment] Fix range header when using `-N` and media sequence ( #1048 )
...
Authored by: shirt
2021-09-22 04:19:45 +05:30
daf7ac2b92
[fragment] Avoid repeated request for AES key
2021-09-22 01:15:16 +05:30
96933fc1b6
[aria2c] Fix IV for some AES-128 streams
...
Authored by: shirt
2021-09-22 00:20:41 +05:30
57aa7b8511
[hls] Byterange + AES128 is supported by native downloader
2021-09-19 14:20:54 +05:30
edf65256aa
[hls,aes] Fallback to native implementation for AES-CBC
...
and detect `Cryptodome` in addition to `Crypto`
Closes #935
Related: #938
2021-09-18 00:55:58 +05:30
d5fe04f5c7
Fix `--compat-option no-direct-merge`
2021-09-12 21:40:28 +05:30
d75201a873
Use `os.replace` where applicable ( #793 )
...
When using
```py
os.remove(encodeFilename(filename))
os.rename(encodeFilename(temp_filename), encodeFilename(filename))
```
the `os.remove` need not be atomic and so can be executed arbitrarily compared to the immediately following rename call. It is better to use `os.replace` instead
Authored by: paulwrubel
2021-08-27 07:57:20 +05:30
691d5823d6
[aria2c] Obey `--rate-limit`
2021-08-27 00:59:36 +05:30
45842107b9
fix bug in 6251555f1c
...
:ci skip
2021-08-24 06:23:21 +05:30
6251555f1c
[downloader/ffmpeg] Support for DASH manifests (experimental)
...
Closes #159
2021-08-24 05:52:00 +05:30
330690a214
[downloader/ffmpeg] Allow passing custom arguments before -i
...
Closes #686
2021-08-24 04:24:12 +05:30
7a6742b5f9
[webvtt] Fix timestamp overflow adjustment ( #698 )
...
In some streams, empty segments may appear with a bogus, non-monotone MPEG timestamp.
This should not be considered as an overflow
Authored by: fstirlitz
2021-08-15 21:03:06 +05:30
f775c83110
Fix `--force-overwrites` when using `-k`
...
For formats that need merge, the `.fxxx` files are not removed before
downloading the corresponding `.part` files. This causes the rename to fail
2021-08-15 00:28:49 +05:30
0715f7e19b
Revert erroneous use of the `Content-Length` header ( #637 )
...
This reverts commit 6c907eb33f
2021-08-11 21:09:17 +05:30
25a3f4f5d6
[webvtt] Merge daisy-chained duplicate cues ( #638 )
...
Fixes: https://github.com/yt-dlp/yt-dlp/issues/631#issuecomment-893338552
Previous deduplication algorithm only removed duplicate cues with
identical text, styles and timestamps. This change also merges
cues that come in ‘daisy chains’, where sequences of cues with
identical text and styles appear in which the ending timestamp of
one equals the starting timestamp of the next.
This deduplication algorithm has the somewhat unfortunate side effect
that NOTE blocks between cues, if found, will be emitted in a different
order relative to their original cues. This may be unwanted if perfect
fidelity is desired, but then so is daisy-chain deduplication itself.
NOTE blocks ought to be ignored by WebVTT players in any case.
Authored by: fstirlitz
2021-08-10 01:52:30 +05:30
ad3dc496bb
Misc fixes - See desc
...
* Remove unnecessary uses of _list_from_options_callback
* Fix download tests - Bug from 6e84b21559#517
2021-08-10 01:22:55 +05:30
3448870205
[docs] Fix some mistakes and improve doc
2021-08-07 21:41:48 +05:30
b868936cd6
[cleanup] Misc
2021-08-07 21:17:07 +05:30
9cc1a3130a
Fix resuming when using `--no-part`
...
Closes #576
2021-08-06 00:55:04 +05:30
bb36a55c41
[nbcolympics:stream] Fix extractor
...
PR: https://github.com/ytdl-org/youtube-dl/pull/29688
Closes : #617 , https://github.com/ytdl-org/youtube-dl/issues/29665
* Livestreams are untested
* If using ffmpeg as downloader, v4.3+ is needed since `-http_seekable` option is necessary
* Instead of making a seperate key for each arg that needs to be passed to ffmpeg, I made `_ffmpeg_args`
* This deprecates `_seekable`, but the option is kept for compatibility
Authored by: nchilada, pukkandan
2021-08-04 20:41:59 +05:30
e5611e8eda
[ffmpeg] Fix streaming `mp4` to `stdout`
2021-08-03 00:05:16 +05:30
a46a815b05
[cleanup] Fix linter in 96fccc101f
2021-08-01 12:52:09 +05:30
96fccc101f
[downloader] Allow streaming unmerged formats to stdout using ffmpeg
...
For this to work:
1. The downloader must be ffmpeg
2. The selected formats must have the same protocol
3. The formats must be downloadable by ffmpeg to stdout
Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146 , https://github.com/ytdl-org/youtube-dl/issues/27265
2021-08-01 12:38:06 +05:30
dbf5416a20
[cleanup] Refactor some code
2021-08-01 12:38:05 +05:30
f45e6c1126
[downloader] Pass same status object to all `progress_hooks`
2021-07-23 09:46:55 +05:30
3ba7740dd8
[downloader] Pass `info_dict` to `progress_hook`s
2021-07-22 04:30:11 +05:30
11f9be0912
[youtube] Extract data from multiple clients ( #536 )
...
* `player_client` accepts multiple clients
* default `player_client` = `android,web`
* music clients can be specifically requested
* Add IOS `player_client`
* Hide live dash since they can't be downloaded
Closes #501
Authored-by: pukkandan, colethedj
2021-07-21 09:22:34 +05:30
723d44b92b
[fragment] Handle errors in threads correctly
2021-07-07 01:55:54 +05:30
6b6c16ca6c
[downloader/ffmpeg] Fix `--ppa` when using simultaneous download
2021-07-02 08:17:30 +05:30
c2603313b1
[youtube_live_chat] use `clickTrackingParams` ( #449 )
...
Authored by: siikamiika
2021-06-27 04:52:32 +05:30
bd4d1ea398
[cleanup] Minor refactoring of `fragment`
2021-06-24 22:23:33 +05:30
8e897ed283
[fragment] Return status of download correctly
2021-06-24 22:04:23 +05:30
d534c4520b
[youtube_live_chat] Fix download with cookies ( #437 )
...
Closes #417
Authored by: siikamiika
2021-06-24 21:26:32 +05:30
c60ee3a218
[youtube_live_chat] Support ongoing live chat ( #422 )
...
Authored by: siikamiika
2021-06-23 05:42:39 +05:30
8a77e5e6bc
[cleanup] Revert unnecessary changes in 51d9739f80
2021-06-23 05:34:40 +05:30
51d9739f80
Add option `--throttled-rate` below which video data is re-extracted
...
Currently only for HTTP downloads
Closes #430 , workaround for https://github.com/ytdl-org/youtube-dl/issues/29326
2021-06-23 05:29:58 +05:30
4c7853de14
[fragment] Merge during download for `-N`, and refactor `hls`/`dash` ( #364 )
2021-06-22 00:29:50 +05:30
e36d50c5dd
[websockets] Add `WebSocketFragmentFD` ( #399 )
...
Necessary for #392
Co-authored by: nao20010128nao, pukkandan
2021-06-21 22:56:36 +05:30
ff0f78e1fe
[aria2c] Lower `--min-split-size` for HTTP downloads
...
This makes downloading smaller files much faster
2021-06-20 19:28:54 +05:30
cdb19aa4c2
[downloader/mhtml] Add new downloader ( #343 )
...
This downloader is intended to be used for streams that consist of a
timed sequence of stand-alone images, such as slideshows or thumbnail
streams
This can be used for implementing:
https://github.com/ytdl-org/youtube-dl/issues/4974#issue-58006762
https://github.com/ytdl-org/youtube-dl/issues/4540#issuecomment-69574231
https://github.com/ytdl-org/youtube-dl/pull/11185#issuecomment-335554239
https://github.com/ytdl-org/youtube-dl/issues/9868
https://github.com/ytdl-org/youtube-dl/pull/14951
Authored by: fstirlitz
2021-06-13 20:36:40 +05:30
2ec1759f9d
[downloader/ffmpeg] Hide FFmpeg banner unless in verbose mode ( #372 )
...
Authored by: fstirlitz
2021-06-06 00:59:04 +05:30
e06ca6ddac
[hls] Decrypt fragment when reading from disk
...
Closes #373
2021-06-05 18:51:15 +05:30
Lesmiscore (Naoya Ozaki)