Commit Graph

563 Commits

Author SHA1 Message Date
Simon Sawicki 61bdf15fc7
[core] Raise minimum recommended Python version to 3.8 (#8183)
Authored by: Grub4K
2023-09-24 02:24:47 +02:00
bashonly 5ca095cbcd
[cleanup] Misc (#8182)
Closes #7796, Closes #8028
Authored by: barsnick, sqrtNOT, gamer191, coletdjnz, Grub4K, bashonly
2023-09-23 20:00:31 +00:00
Simon Sawicki 836e06d246
[core] Fix support for upcoming Python 3.12 (#8130)
This also adds the following test runners:
- `3.12-dev` on `ubuntu-latest`
- `3.12-dev` on `windows-latest`
- `pypy-3.10` on `ubuntu-latest`

Authored by: Grub4K
2023-09-17 12:56:50 +02:00
coletdjnz db7b054a61
[networking] Add request handler preference framework (#7603)
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-08-04 22:17:48 +00:00
pukkandan f73c118035
`FFmpegFixupM3u8PP` may need to run with ffmpeg
Bug in 62b5c94cad
Closes #7725
2023-07-30 04:24:46 +05:30
pukkandan 6148833f5c
[cleanup] Misc 2023-07-30 04:06:18 +05:30
pukkandan 8cb7fc44db
Fix `--check-formats`
Bug in bc344cd456
2023-07-30 03:23:13 +05:30
pukkandan 62b5c94cad
[cleanup] Misc fixes
Closes #7528
2023-07-22 09:09:52 +05:30
pukkandan a264433c9f
[outtmpl] Fix replacement for `playlist_index` 2023-07-22 08:57:43 +05:30
bashonly 75dc8e673b
[networking] Fix `--legacy-server-connect` (#7645)
Bugfix for 227bf1a33b

Authored by: bashonly
2023-07-20 13:31:17 +00:00
bashonly 6c5211cebe
[core] Fix HTTP headers and cookie handling
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor

Closes #7558
Authored by: bashonly, pukkandan
2023-07-15 15:25:45 -05:00
Aaruni Kaushik 2b029ca0a9
[cleanup] Add color to `download-archive` message (#5138)
Authored by: aaruni96, Grub4K, pukkandan
Closes #4913
2023-07-16 00:45:08 +05:30
coletdjnz 3d2623a898
[compat, networking] Deprecate old functions (#2861)
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
coletdjnz 227bf1a33b
[networking] Rewrite architecture (#2861)
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
2023-07-15 16:18:35 +05:30
pukkandan c365dba843
[networking] Add module (#2861)
No actual changes - code is only moved around
2023-07-15 16:18:34 +05:30
Simon Sawicki 3121512228
[core] Change how `Cookie` headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
2023-07-06 23:14:39 +05:30
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
2023-07-06 21:40:01 +05:30
pukkandan 906c0bdcd8
[formats] Fix best fallback for storyboards
Partial fix for #7478
2023-07-06 21:39:58 +05:30
pukkandan 337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
pukkandan 47bcd43724
[outtmpl] Pad `playlist_index` etc even when with internal formatting
Closes #7501
2023-07-06 20:22:03 +05:30
pukkandan b4e0d75848
Improve `--download-sections`
* Support negative time-ranges
* Add `*from-url` to obey time-ranges in URL

Closes #7248
2023-06-22 13:03:07 +05:30
pukkandan 0dff8e4d1e
Indicate `filesize` approximated from `tbr` better 2023-06-22 01:37:55 +05:30
pukkandan 1619ab3e67
Bugfix for ebe1b4e34f 2023-06-21 09:21:22 +05:30
pukkandan 84078a8b38
[core] Fix `filepath` being copied to underlying format dict
Closes #6536
2023-06-21 09:21:21 +05:30
pukkandan ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
Nicolai Dagestad db3ad8a676
Add option `--netrc-cmd` (#6682)
Authored by: NDagestad, pukkandan
Closes #1706
2023-06-21 08:37:42 +05:30
pukkandan ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes #7164
2023-06-21 06:10:39 +05:30
pukkandan 93b39cdbd9
Add `--compat-option playlist-match-filter`
Closes #6073
2023-06-21 06:10:39 +05:30
Ivan Skodje 372a0f3b9d
Auto-select default format in `-f-` (#7101)
Authored by: ivanskodje, pukkandan
Closes #6720
2023-05-29 20:20:21 +05:30
coletdjnz 3f66b6fe50
[core] Workaround erroneous urllib Windows proxy parsing (#7092)
Convert proxies extracted from windows registry to http for older Python versions.
See: https://github.com/python/cpython/issues/86793

Authored by: coletdjnz
2023-05-27 07:17:27 +00:00
coletdjnz b87e01c123
[cookies] Move `YoutubeDLCookieJar` to cookies module (#7091)
Authored by: coletdjnz
2023-05-27 07:08:19 +00:00
Simon Sawicki 8417f26b8a
[core] Implement `--color` flag (#6904)
Authored by: Grub4K
2023-05-24 20:35:07 +02:00
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script 2023-05-24 23:29:30 +05:30
coletdjnz 955c89584b
[core] Deprecate internal `Youtubedl-no-compression` header (#6876)
Authored by: coletdjnz
2023-05-20 22:55:09 +00:00
coletdjnz 69bec6730e
[cleanup, utils] Split into submodules (#7090)
Closes https://github.com/yt-dlp/yt-dlp/pull/2173

Authored by: pukkandan, coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-05-20 21:56:23 +00:00
pukkandan 6f2287cb18
[cleanup] Misc
Closes #7030, closes #6967
2023-05-20 04:23:41 +05:30
pukkandan c8bc203fbf
[docs] Misc improvements
Closes #6814, closes #6940, closes #6733, closes #6923, closes #6566, closes #6726, closes #6728
2023-05-20 02:38:24 +05:30
pukkandan 17ba4343cf
Fix f005a35aa7
Printing inside `finally` causes the order of logging to change
when there is an error, which is undesirable. So this is reverted.

The issue of `--print` being blocked by pre-processors was an
unintentional side-effect of changing the operation orders in
170605840e, and this is also partially
reverted.
2023-04-29 03:06:42 +05:30
pukkandan f005a35aa7
Ensure pre-processor errors do not block `--print`
Closes #6937
2023-04-29 01:06:14 +05:30
pukkandan b5f61b69d4
Fix bug in 170605840e
and related refactor
2023-04-27 19:35:28 +05:30
pukkandan 170605840e
Populate `filename` and `urls` fields at all stages of `--print`
Closes https://github.com/yt-dlp/yt-dlp/issues/6920
2023-04-27 06:13:42 +05:30
pukkandan ec9311c41b
[outtmpl] Support `str.format` syntax inside replacements
Closes #6843
2023-04-24 18:43:54 +05:30
pukkandan 78fde6e339
[outtmpl] Allow `\n` in replacements and default.
Fixes: https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1510055357
Fixes: https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1510363645
2023-04-24 18:28:30 +05:30
pukkandan 9874e82b5a
Do not translate newlines in `--print-to-file`
Fixes https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1509361107
2023-04-16 08:55:44 +05:30
pukkandan c3f624ef0a
Relaxed validation for numeric format filters
Continued from f96bff99cb

Closes #6782
2023-04-12 05:05:15 +05:30
pukkandan 26010b5cec
[postprocessor/FixupDuplicateMoov] Fix bug in triggering 2023-04-11 21:43:33 +05:30
pukkandan ab1de9cb1e
Support loading info.json with a list at it's root 2023-03-10 14:15:13 +05:30
pukkandan 392389b7df
[cleanup] Misc 2023-03-05 03:34:55 +05:30
pukkandan fe2ce85aff
Add option `--break-match-filters`
* Deprecates `--break-on-reject`

Closes #5962
2023-03-04 01:18:54 +05:30
Simon Sawicki 29cb20bd56
[build] Automated builds and nightly releases (#6220)
Closes #1839
Authored by: Grub4K, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2023-03-03 22:54:23 +05:30
pukkandan 5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan a538772969
[cleanup] Misc
Closes #5897
2023-02-17 17:52:22 +05:30
Siddhartha Sahu 376aa24b15
Improve default subtitle language selection (#6240)
Authored by: sdht0
2023-02-17 01:25:01 +05:30
pukkandan 9ebac35577
Bugfix for 39f32f1715
when `--ignore-no-formats-error`
2023-02-16 17:06:54 +05:30
pukkandan c154302c58
Bugfix for 39f32f1715 2023-02-13 01:35:54 +05:30
pukkandan 39f32f1715
Sanitize formats before sorting
Closes #4501
2023-02-13 01:19:51 +05:30
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
pukkandan 7aefd19afe
Make `title` completely non-fatal
Ref: https://github.com/yt-dlp/yt-dlp/pull/6158#discussion_r1096984349
2023-02-07 01:18:04 +05:30
pukkandan 59d7de0da5
Fix `--concat-playlist`
Closes #6080
2023-01-24 03:43:48 +05:30
pukkandan f079514957
[utils] `windows_enable_vt_mode`: Better error handling
Closes #5927
2023-01-03 15:59:49 +05:30
pukkandan 08e29b9f1f
[cleanup] Misc
Closes #5576, closes #5887
2023-01-02 19:40:15 +05:30
Matthew 8300774c4a
Add `--enable-file-urls` (#5917)
Closes https://github.com/yt-dlp/yt-dlp/issues/3675

Authored by: coletdjnz
2023-01-02 06:05:13 +00:00
Matthew e756f45ba0
Improve handling for overriding extractors with plugins (#5916)
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
2023-01-02 04:55:11 +00:00
pukkandan 193fb150b7
Fix bug in 119e40ef64 2023-01-01 17:01:48 +05:30
pukkandan 88fb942577
Add message when there are no subtitles/thumbnails
Closes #5551
2023-01-01 14:11:15 +05:30
Matthew 8e40b9d1ec
Improve plugin architecture (#5553)
to make plugins easier to develop and use:
* Plugins are now loaded as namespace packages.
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.).
* Plugin packages can be installed and managed via pip, or dropped into any of the documented locations.
* Users do not need to edit any code files to install plugins.
* Backwards-compatible with previous plugin architecture.

As a side-effect, yt-dlp will now search in a few more locations for config files.

Closes https://github.com/yt-dlp/yt-dlp/issues/1389

Authored by: flashdagger, coletdjnz, pukkandan, Grub4K
Co-authored-by: Marcel <flashdagger@googlemail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-01-01 04:29:22 +00:00
pukkandan ec54bd43f3
Fix bug in writing playlist info-json
Closes #4889
2022-12-30 14:07:15 +05:30
pukkandan 119e40ef64
Add pre-processor stage `video`
Related: #456, #5808
2022-12-30 11:18:45 +05:30
Anant Murmu efa944f4bc
[cleanup] Use `random.choices` (#5800)
Authored by: freezboltz
2022-12-30 08:13:49 +05:30
pukkandan 8791e78ccc
Fix `original_url` in playlists 2022-12-23 01:44:20 +05:30
pukkandan 71df9b7fd5
[cleanup] Misc 2022-12-03 19:52:31 +05:30
pukkandan 784320c98c
Implement universal format sorting
Closes #5566
2022-11-17 11:05:49 +05:30
pukkandan 105bfd90f5
Add new field `aspect_ratio`
Closes #5402
2022-11-16 06:57:09 +05:30
pukkandan 6368e2e639
[cleanup] Misc
Closes #5541
2022-11-16 06:57:07 +05:30
pukkandan d7b460d0e5
Make early reject of `--match-filter` stricter
Closes #5509
2022-11-13 10:56:06 +05:30
pukkandan bc5c2f8a2c
Fix bugs in `PlaylistEntries` 2022-11-12 00:02:12 +05:30
bashonly e4221b700f
Fix `--list` options not implying `-s` in some cases (#5296)
Authored by: bashonly, Grub4K
2022-11-11 14:24:57 +05:30
pukkandan 497074f044
Write API params in debug head 2022-10-25 20:09:28 +05:30
pukkandan d5d1df8afd
[cleanup Misc
Closes #5162
2022-10-18 23:52:44 +05:30
cruel-efficiency 2576d53a31
Fix end time of clips (#5255)
Closes #5256
Authored by: cruel-efficiency
2022-10-18 18:21:43 +05:30
pukkandan 9b9dad119a
[outtmpl] Ensure ASCII in json and add option for Unicode
Closes #5236
2022-10-14 11:50:24 +05:30
pukkandan 94dc8604dd
Do more processing in `--flat-playlist` 2022-10-13 15:30:14 +05:30
pukkandan aebb4f4ba7
Fix for formats=None
Fixes: https://github.com/yt-dlp/yt-dlp/pull/4965#issuecomment-1267682512
2022-10-05 09:17:33 +05:30
pukkandan a057779d5e
[cleanup] Minor fixes
Closes #5129, Closes #4982
2022-10-04 01:48:14 +05:30
sam 177662e0f2
[extractor/MicrosoftEmbed] Add extractor (#5082)
Closes #2638
Authored by: DoubleCouponDay
2022-10-03 19:22:30 +05:30
Simon Sawicki dfea94f8f6
[extractor/crunchyroll:beta] Improve handling of hardsubs (#5041)
Closes #3397
Authored by: Grub4K
2022-09-30 06:35:44 +05:30
pukkandan 0500ee3d81
Don't download entire video when no matching `--download-sections` 2022-09-26 03:04:32 +05:30
pukkandan 0bd5a039ea
Playlists maynot always have webpage_url 2022-09-25 23:27:21 +05:30
Lesmiscore fc2ba496fd
Allow open ranges for time ranges (#4940)
Authored by: Lesmiscore
2022-09-17 01:04:23 +09:00
pukkandan 9c935fbc72
Fix bug in ae1035646a
Closes #4890
2022-09-11 05:12:52 +05:30
pukkandan deae7c1711
[cleanup] Misc 2022-09-10 03:46:54 +05:30
pukkandan 941e881e1f
Fix bug in ae1035646a
Closes #4881
2022-09-10 03:42:43 +05:30
pukkandan ae1035646a
Allow a `set` to be passed as `download_archive` 2022-09-08 15:09:30 +05:30
pukkandan 17ffed1842
[docs] Improvements
* Move detailed installation instructions to https://github.com/yt-dlp/yt-dlp/wiki/Installation
* Link to wiki where applicable
* Fix some mistakes. Closes #4853, Closes #4855, Closes #4852
* Improve some error messages
2022-09-07 17:38:05 +05:30
pukkandan 48c8424bd9
Fix bug in 07a1250e0e 2022-09-06 19:57:29 +05:30
pukkandan 07a1250e0e
[outtmpl] Curly braces to filter keys 2022-09-03 18:28:24 +05:30
pukkandan d2c8aadf79
[cleanup] Misc
Closes #4710, Closes #4754, Closes #4723
Authored by: pukkandan, MrRawes, DavidH-2022
2022-09-01 16:49:03 +05:30
pukkandan 1ac7f46184
Update to ytdl-commit-ed5c44e7
[compat] Replace deficient ChainMap class in Py3.3 and earlier
ed5c44e7b7
2022-09-01 16:46:32 +05:30
bashonly 9bd13fe5bb
[cookies] Support firefox container in `--cookies-from-browser` (#4753)
Authored by: bashonly
2022-08-30 22:24:46 +05:30
pukkandan da4db748fa
[utils] Add `deprecation_warning`
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
2022-08-30 21:03:07 +05:30