Commit Graph

17770 Commits

Author SHA1 Message Date
xarantolus c4a1d0eef8 Merge branch 'fixYTSearch' of https://github.com/xarantolus/youtube-dl into fixYTSearch 2020-09-06 09:25:02 +02:00
xarantolus bea9b00588 Fix regex for other variable declaration type
This now supports declarations like `window["ytInitialData"] = ...` and `var ytInitialData = ...`
2020-09-06 09:22:49 +02:00
xarantolus f442082a50 [youtube] More general title extraction
Seems like this attribute is moved every few weeks, so we just extract both and use the one that is present.
2020-09-06 09:22:48 +02:00
xarantolus 1f93faf60b [youtube] Make `ytcfg.set` config extraction non-fatal
If the markup of the page changes in the future, it might be possible that _FEED_DATA still works, but the other regex does not. SInce it is not necessary for the first page of videos, we make sure the program doesn't exit before extracting them.

TL;DR: Extract the first video page even if there are problems
2020-09-06 09:22:47 +02:00
xarantolus 299056ad52 Fix python2 compatibility and title extraction 2020-09-06 09:22:47 +02:00
xarantolus f536080701 Run formatter 2020-09-06 09:22:46 +02:00
xarantolus 5c430b67bd [youtube] Fix feed extraction
In order to extract videos from further pages, we need to get various variables that are in an argument to the `ytcfg.set` call in a script on the feed page.
2020-09-06 09:22:46 +02:00
xarantolus e03b4f3e05 [youtube] Make search extraction less dependent on json schema.
If an object looks like a video (it has a `videoId` key), assume that it is.
2020-09-06 09:22:30 +02:00
xarantolus 19f671f88b [youtube] Move search URL extraction to appropriate extractor 2020-09-06 09:21:26 +02:00
xarantolus b948643f9c [youtube] Fix extraction of search urls (closes ytdl-org/youtube-dl#25696) 2020-09-06 09:21:25 +02:00
Sergey M․ d51e23d9fc
release 2020.09.06 2020-09-06 13:00:41 +07:00
Sergey M․ 6cd452acff
[ChangeLog] Actualize
[ci skip]
2020-09-06 12:57:56 +07:00
Sergey M․ 50e9fcc1fd
[nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
random-nick 16ee69c1b7
[youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
Sergey M․ 67171ed7e9
[youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
Sergey M․ 1d9bf655e6
[utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
TheRealDude2 62ae19ff76
[xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
Sergey M․ 5ed05f26ad
[svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
Sergey M․ 841b683804
[twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
xarantolus 4c47858c05 Fix regex for other variable declaration type
This now supports declarations like `window["ytInitialData"] = ...` and `var ytInitialData = ...`
2020-09-03 20:41:45 +02:00
Remita Amine f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
xarantolus 94255fa0b1 [youtube] More general title extraction
Seems like this attribute is moved every few weeks, so we just extract both and use the one that is present.
2020-08-24 14:30:08 +02:00
xarantolus 7d743516b5 [youtube] Make `ytcfg.set` config extraction non-fatal
If the markup of the page changes in the future, it might be possible that _FEED_DATA still works, but the other regex does not. SInce it is not necessary for the first page of videos, we make sure the program doesn't exit before extracting them.

TL;DR: Extract the first video page even if there are problems
2020-08-24 14:29:16 +02:00
Sergey M․ 10709fc7c6
[xhamster] Extend _VALID_URL (closes #25927) 2020-08-12 21:51:50 +07:00
TheRealDude2 a7e348556a
[xhamster] Fix extraction (closes #26157) (#26254) 2020-08-12 21:42:17 +07:00
JChris246 6cb30ea5ed
[xhamster] Extend _VALID_URL (closes #25789) (#25804) 2020-08-12 21:37:22 +07:00
xarantolus 2ad7b6ace9 Merge branch 'fixYTSearch' of https://github.com/xarantolus/youtube-dl into fixYTSearch 2020-07-31 10:05:19 +02:00
xarantolus a5e386d9fe Fix python2 compatibility and title extraction 2020-07-31 10:05:11 +02:00
Philipp d39cf56210
Merge branch 'master' into fixYTSearch 2020-07-30 16:41:07 +02:00
xarantolus 4f37c60bf5 Run formatter 2020-07-30 16:38:56 +02:00
xarantolus c449f70965 [youtube] Fix feed extraction
In order to extract videos from further pages, we need to get various variables that are in an argument to the `ytcfg.set` call in a script on the feed page.
2020-07-30 16:34:48 +02:00
Sergey M․ de722d3cd7 release 2020.07.28 2020-07-28 15:34:32 +02:00
Sergey M․ f4492c4890 [ChangeLog] Actualize
[ci skip]
2020-07-28 15:34:32 +02:00
Sergey M․ 49004a6b59 [youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 15:34:32 +02:00
MRWITEK 54ffcbb8eb [youtube] Improve description extraction (closes #25937) (#25980) 2020-07-28 15:34:32 +02:00
Sergey M․ 9fa728f4e8 [wistia] Restrict embed regex (closes #25969) 2020-07-28 15:34:31 +02:00
Glenn Slayden bb2c950b8e [youtube] Prevent excess HTTP 301 (#25786) 2020-07-28 15:34:31 +02:00
Sergey M․ 255f31b5cb [youtube:playlists] Extend _VALID_URL (closes #25810) 2020-07-28 15:34:31 +02:00
Remita Amine 2bd94127a2 [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-07-28 15:34:30 +02:00
Sergey M․ a4ed50bb84
release 2020.07.28 2020-07-28 05:13:03 +07:00
Sergey M․ 570611955f
[ChangeLog] Actualize
[ci skip]
2020-07-28 05:07:54 +07:00
Sergey M․ e450f6cb63
[youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 05:05:38 +07:00
MRWITEK a115e07594
[youtube] Improve description extraction (closes #25937) (#25980) 2020-07-14 12:01:15 +01:00
Sergey M․ 718393c632
[wistia] Restrict embed regex (closes #25969) 2020-07-11 18:27:19 +07:00
xarantolus 7fa0a67cc1 Remove unused variable 2020-07-10 11:50:50 +02:00
xarantolus c37ca4732b [youtube] Remote download limit 2020-07-10 11:47:13 +02:00
Glenn Slayden 07af16b92e
[youtube] Prevent excess HTTP 301 (#25786) 2020-07-01 02:56:16 +07:00
Sergey M․ e942cfd1a7
[youtube:playlists] Extend _VALID_URL (closes #25810) 2020-06-28 10:30:03 +07:00
Remita Amine 9a7e5cb88a [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-06-23 15:09:13 +01:00
xarantolus 5cbe7563be [youtube] Return to old feed extraction code as it *seems* like that change was reverted
The old code now works again, but it downloads without limit. This is why a limit of 1000 videos is added, it can be overwritten with the `--max-downloads` option - that way, only so many ids will be extracted as videos downloaded
2020-06-23 11:27:02 +02:00