Commit graph

927 commits

Author SHA1 Message Date
Philipp Hagemeister df725153d2 Credit mc2avr for JukeboxIE (#924) 2013-06-25 17:57:47 +02:00
Philipp Hagemeister d662896090 [googleplus] Adapt to new detail URL format 2013-06-25 17:52:32 +02:00
Philipp Hagemeister db241e8645 Add encoding to jukebox IE and simplify it a little bit 2013-06-25 17:16:38 +02:00
Philipp Hagemeister 515d7a5e73 Add Jukebox IE 2013-06-25 17:12:35 +02:00
mc2avr 14fbdc9cdd [jukebox] call YoutubeIE if necessary 2013-06-25 16:51:09 +02:00
Filippo Valsorda 98bcd2834a improve generic and encrypted signature error messages 2013-06-25 16:47:16 +02:00
mc2avr 28ef06f7c2 add JukeboxIE 2013-06-25 13:28:59 +02:00
Philipp Hagemeister 577d02370d release 2013.06.30 2013-06-25 12:28:40 +02:00
Philipp Hagemeister 50be92c11c Handle video pages without vevo IDs (Fixes #923) 2013-06-25 12:28:17 +02:00
Johny Mo Swag d18596baf4 added Youtube embed detection to WorldstarIE 2013-06-24 18:58:49 -07:00
Jaime Marquínez Ferrándiz 7ce7e39476 YoutubeIE: Extend decryption of signatures to all videos that have the 's' field in the url_encoded_fmt_stream_map (related #920) 2013-06-24 21:25:12 +02:00
Filippo Valsorda 93eb15c573 clean up printing in __init__.py 2013-06-24 15:57:53 +02:00
Philipp Hagemeister 9f4d83e3b1 release 2013.06.29 2013-06-24 14:51:24 +02:00
Jaime Marquínez Ferrándiz 1c251cd948 MTVIE: add support for Vevo videos (related #913) 2013-06-24 13:54:19 +02:00
Jaime Marquínez Ferrándiz 70d1924f8b Add VevoIE 2013-06-24 12:31:41 +02:00
Philipp Hagemeister 7b4948b05f release 2013.06.28 2013-06-24 11:11:33 +02:00
Philipp Hagemeister 878b5d9f0d Merge remote-tracking branch 'jaimeMF/youtubedl_class' 2013-06-24 10:48:41 +02:00
Philipp Hagemeister 2bc1820660 release 2013.06.27 2013-06-24 10:32:08 +02:00
Jaime Marquínez Ferrándiz 8222d8de88 Split FileDownloader in two classes: FileDownloader and YoutubeDL
YoutubeDL is the class that coordinates everything
FileDownloader gets a filename and an info dict and downloads the video.
2013-06-24 10:21:43 +02:00
Jaime Marquínez Ferrándiz c7253e2e8c [youtube] fix condition always being evaluated to true 2013-06-24 09:42:46 +02:00
Philipp Hagemeister d69cf69a6a [youtube] Use mp4 as extension for format 38 (Fixes #892) 2013-06-24 01:22:59 +02:00
Philipp Hagemeister d02ecdefab release 2013.06.26 2013-06-24 01:01:53 +02:00
Philipp Hagemeister f8bf74575a release 2013.06.25 2013-06-24 00:20:36 +02:00
Philipp Hagemeister dabcaf3b06 release 2013.06.24 2013-06-24 00:02:20 +02:00
Philipp Hagemeister b07d9c23c5 release 2013.06.23 2013-06-23 23:42:21 +02:00
Philipp Hagemeister 825e0984e2 [break] adapt to new paths 2013-06-23 22:59:51 +02:00
Philipp Hagemeister 190717e31f [justin.tv] Clarify variable content 2013-06-23 22:52:43 +02:00
Philipp Hagemeister 0824c28c8b Remove mentions of old InfoExtractors module 2013-06-23 22:42:59 +02:00
Philipp Hagemeister c59b4aaeef Fix imports and restrict available legacy imports 2013-06-23 22:38:59 +02:00
Philipp Hagemeister f9c6cbf002 Move extractor imports and functions into extractor/__init__.py 2013-06-23 22:36:24 +02:00
Philipp Hagemeister b8fe71ab86 Remove unused imports from InfoExtractor 2013-06-23 22:34:23 +02:00
Philipp Hagemeister cb10cded2a [xhamster] Move into own file 2013-06-23 22:32:44 +02:00
Philipp Hagemeister cd8b830292 [Teamcoco] Move into own file 2013-06-23 22:31:50 +02:00
Philipp Hagemeister 1ac4004f3a [flickr] Move into own file 2013-06-23 22:31:12 +02:00
Philipp Hagemeister e17d368ae2 [howcast] Move into own file 2013-06-23 22:30:16 +02:00
Philipp Hagemeister 27110b0567 [hypem] Move into own file 2013-06-23 22:29:27 +02:00
Philipp Hagemeister 9fe4de3471 [ina] Move into own file 2013-06-23 22:28:19 +02:00
Philipp Hagemeister d26d440e19 [redtube] Simplify 2013-06-23 22:27:34 +02:00
Philipp Hagemeister 9f5daf0006 [redtube] move into own file 2013-06-23 22:27:16 +02:00
Philipp Hagemeister eb1634cbf8 [Vine] move into own file 2013-06-23 22:26:30 +02:00
Philipp Hagemeister 01c10ca26e [VBox7] move into own file 2013-06-23 22:25:46 +02:00
Philipp Hagemeister 45aef47281 [Bandcamp] move into own file 2013-06-23 22:24:58 +02:00
Philipp Hagemeister ae287755b7 [Tumblr] move into own file 2013-06-23 22:24:07 +02:00
Philipp Hagemeister a37f27ae99 [LiveLeak] move into own file 2013-06-23 22:23:19 +02:00
Philipp Hagemeister 49f5f315fd [Spiegel] move into own file 2013-06-23 22:22:08 +02:00
Philipp Hagemeister 97d2db017c [myspass] Move into own file and default to mp4 ext 2013-06-23 22:20:45 +02:00
Philipp Hagemeister 2c64df0399 [keek] move into own file 2013-06-23 22:16:41 +02:00
Philipp Hagemeister 828400422a [8tracks] Move into own file 2013-06-23 22:15:50 +02:00
Philipp Hagemeister c3c77cec30 [youjizz] move into own file 2013-06-23 22:14:22 +02:00
Philipp Hagemeister 1183b85f50 [pornotube] move into own file 2013-06-23 22:13:32 +02:00
Philipp Hagemeister 0143dc029c [YouPorn] move into own file 2013-06-23 22:12:14 +02:00
Philipp Hagemeister e10e576fed [RBMARadio] move into own file 2013-06-23 22:09:32 +02:00
Philipp Hagemeister 78af8eb1d1 [ustream] move into its own file 2013-06-23 22:08:28 +02:00
Philipp Hagemeister 79e93125d0 [justin.tv] move into own file 2013-06-23 22:07:27 +02:00
Philipp Hagemeister 48db0b1f4a [FunnyOrDie] Remove unused import 2013-06-23 22:07:17 +02:00
Philipp Hagemeister 8f0578f0fc Move FunnyOrDie into its own file 2013-06-23 22:05:23 +02:00
Philipp Hagemeister 250f557872 Move WorldStarHipHop into its own file 2013-06-23 22:04:08 +02:00
Philipp Hagemeister 462dc88b17 Move Steam IE into its own file 2013-06-23 22:02:56 +02:00
Philipp Hagemeister 570fa151fc Move XNXX into its own file 2013-06-23 22:01:57 +02:00
Philipp Hagemeister 9c286cfa00 Move Youku IE into its own file 2013-06-23 22:01:02 +02:00
Philipp Hagemeister 80cbb6ddbb Move MixCloud into its own file 2013-06-23 21:59:15 +02:00
Philipp Hagemeister 9fd5ce0cbe Move TED IE into its own file 2013-06-23 21:55:53 +02:00
Philipp Hagemeister 1736dec629 Mark MTV as broken for now (#913) 2013-06-23 21:52:41 +02:00
Philipp Hagemeister fc28721960 Add MTV IE file (oops) 2013-06-23 21:34:03 +02:00
Philipp Hagemeister 335056663a Move MTV IE into its own file 2013-06-23 21:27:38 +02:00
Philipp Hagemeister 5b286728de Move NBA IE into its own file 2013-06-23 21:18:00 +02:00
Philipp Hagemeister 291a168bcc Move StanfordOC IE into its own file 2013-06-23 21:16:32 +02:00
Philipp Hagemeister fda7d31aa0 Move infoq into its own file 2013-06-23 21:14:19 +02:00
Philipp Hagemeister cbf46c737c Move XVideos IE into its own file (and simplify it a bit) 2013-06-23 21:11:47 +02:00
Philipp Hagemeister 7beb36a529 Move Collegehumor IE into its own file 2013-06-23 21:10:21 +02:00
Philipp Hagemeister 153697660d Move Escapist into its own file 2013-06-23 21:08:17 +02:00
Philipp Hagemeister 60a72e8d45 Simplify EscapistIE 2013-06-23 21:06:49 +02:00
Philipp Hagemeister 426ff04282 Move DepositFiles into its own IE 2013-06-23 21:06:20 +02:00
Philipp Hagemeister a50e1b32e4 Add facebook import 2013-06-23 21:00:34 +02:00
Philipp Hagemeister 9eae41ddef Move Facebook into its own file 2013-06-23 20:59:45 +02:00
Philipp Hagemeister aad0d6d5ba Move Soundcloud into its own file 2013-06-23 20:57:44 +02:00
Philipp Hagemeister 7aca14a1ec Move G+ IE into its own file, and move google search into a more descriptive module 2013-06-23 20:55:15 +02:00
Philipp Hagemeister d1596ef439 Add import for google search 2013-06-23 20:51:42 +02:00
Philipp Hagemeister ea63e4998b Move comedycentral into its own file 2013-06-23 20:51:04 +02:00
Philipp Hagemeister a08dfd27a8 Move MyVideo into its own file 2013-06-23 20:48:32 +02:00
Philipp Hagemeister f58848011e Move blip.tv extractors into their own file 2013-06-23 20:44:48 +02:00
Philipp Hagemeister 934858ad86 Move YahooSearchIE to youtube_dl.extractor.yahoo 2013-06-23 20:41:54 +02:00
Philipp Hagemeister 3c25b9abae Remove useless headers 2013-06-23 20:35:50 +02:00
Philipp Hagemeister 3fc03845a1 Move GoogleSearchIE into its own file 2013-06-23 20:32:49 +02:00
Philipp Hagemeister 9b122384e9 Move GenericIE into its own file 2013-06-23 20:31:45 +02:00
Philipp Hagemeister 9f4e6bbaeb Move gametrailers IE into its own file 2013-06-23 20:29:56 +02:00
Philipp Hagemeister b05654f0e3 Move YoutubeSearchIE to the other youtube IEs 2013-06-23 20:28:15 +02:00
Philipp Hagemeister 9b3a760bbb [arte] Mark dead code as such 2013-06-23 20:26:35 +02:00
Philipp Hagemeister d5822b96b0 Move ARD, Arte, ZDF into their own files 2013-06-23 20:24:07 +02:00
Philipp Hagemeister b3d14cbfa7 Move Vimeo into its own file 2013-06-23 20:18:21 +02:00
Philipp Hagemeister d6039175e5 Move yahoo into its own file 2013-06-23 20:13:52 +02:00
Philipp Hagemeister 97d6faaced Move Photobucket into its own file 2013-06-23 20:12:18 +02:00
Philipp Hagemeister 219b8130df Move DailyMotion into its own file 2013-06-23 20:12:03 +02:00
Philipp Hagemeister 38cbc40a64 Move Metacafe and Statigram into their own files, and remove absolute import 2013-06-23 20:07:51 +02:00
Philipp Hagemeister 93d3a642a9 [youtube] remove dead code 2013-06-23 19:59:40 +02:00
Philipp Hagemeister c5e8d7af0e Move youtube extractors to youtube_dl.extractor.youtube 2013-06-23 19:58:33 +02:00
Philipp Hagemeister d6983cb460 Fix generic class move (add all files) 2013-06-23 19:57:38 +02:00
Philipp Hagemeister dd9829292e Improve vevo message 2013-06-23 19:45:42 +02:00
Philipp Hagemeister 89cb0eb0b6 Use new signature calculation method only if sig is not present 2013-06-23 19:43:18 +02:00
M.Yasoob Khalid 9b5fffb149 added an IE and test for break.com 2013-06-23 22:42:51 +05:00
Philipp Hagemeister 1f90438025 Merge remote-tracking branch 'jaimeMF/vevo_fix' 2013-06-23 19:42:27 +02:00
Philipp Hagemeister a130adb25b [Statigr.am] Correct uploader id 2013-06-23 19:41:28 +02:00
Philipp Hagemeister 8756c5fe7a Merge remote-tracking branch 'origin/vimeo_passworded_videos' 2013-06-23 19:00:16 +02:00
Philipp Hagemeister 6b3f5a329b Improve Statigr.am IE 2013-06-23 18:58:53 +02:00
M.Yasoob Khalid 01ba4b80a7 added StatigrIE 2013-06-23 18:02:55 +05:00
M.Yasoob Khalid de66764e4e added StatigrIE 2013-06-23 17:46:14 +05:00
Jaime Marquínez Ferrándiz 1037d53988 GenericIE: look for Open Graph info
Only if there is a direct link to the file, don't try if it points to a Flash player
2013-06-23 13:26:49 +02:00
Jaime Marquínez Ferrándiz c3ab8f866c Change metavar of "--sub-format" from LANG to FORMAT 2013-06-23 12:59:20 +02:00
Jaime Marquínez Ferrándiz 346b5ce8fd YoutubeIE: report warnings instead of errors if the subtitles are not found (related #901)
For example when downloading a playlist some videos may not have subtitles but the download shouldn't stop.
2013-06-22 14:15:33 +02:00
Jaime Marquínez Ferrándiz b37fbb990b Move the decrypting function to a static method 2013-06-22 13:20:06 +02:00
Jaime Marquínez Ferrándiz ef75f76f5c Detect more vevo videos 2013-06-22 13:13:40 +02:00
Jaime Marquínez Ferrándiz e296100005 Merge pull request #888 from rg3/youtube_playlists_fix_886
YoutubePlaylistIE: try to extract the url of the entries from the media$group dictionary (closes #886)
2013-06-22 03:35:32 -07:00
Jaime Marquínez Ferrándiz 953dd93a48 YoutubePlaylistIE: don't look into entry['content']['src'], accruing to the docs this can return live stream urls 2013-06-22 12:32:27 +02:00
Jaime Marquínez Ferrándiz e704f4d378 YoutubeIE: If not subtitles language is given default to English for automatic captions (related #901) 2013-06-22 12:14:24 +02:00
Jaime Marquínez Ferrándiz 77d0f05f71 YoutubeIE: Detect new Vevo style videos
The url_encoded_fmt_stream_map can be found in the video page, but the signature must be decrypted, we get it from the webpage instead of the `get_video_info` pages because we have only discover the algorithm for keys with both sub keys of size 43.
2013-06-21 21:51:10 +02:00
Philipp Hagemeister 50d2376769 Leave out sig if not present (#896) 2013-06-21 01:22:47 +02:00
Philipp Hagemeister 759d525301 release 2013.06.21 2013-06-21 00:33:44 +02:00
Jaime Marquínez Ferrándiz f4c8bbcfc2 TEDIE: download the best quality video and use the new _search_regex functions
Also extracts the description.
2013-06-20 20:51:20 +02:00
Jaime Marquínez Ferrándiz 31eead52e7 YoutubePlaylistIE: try to extract the url of the entries from the media$group dictionary
Extracting it from content can return rtsp urls.
2013-06-20 17:23:27 +02:00
Jaime Marquínez Ferrándiz 038a3a1a61 RBMARadioIE: fix the extraction of the JSON data 2013-06-20 14:37:43 +02:00
Jaime Marquínez Ferrándiz 587c68b2cd DailymotionIE: fix the extraction of the video uploader and use _search_regex for getting it 2013-06-20 14:15:29 +02:00
Jaime Marquínez Ferrándiz 5c67601931 Revert "Fix GooglePlusIE: the video_page url has changed of place"
The old method is working again.

This reverts commit 449d5c910c.
2013-06-20 13:53:04 +02:00
Jaime Marquínez Ferrándiz 68f54207a3 SteamIE: only verify the age if needed
Also use the _html_search_regex function
2013-06-20 13:43:44 +02:00
Philipp Hagemeister bb47437686 Ignore invalid dates (Fixes #894) 2013-06-19 22:13:16 +02:00
Jaime Marquínez Ferrándiz 213b715893 Merge pull request #887 from anisse/master
Fetch all entries that are in a youtube playlist

Also add a test.
2013-06-19 12:52:44 +02:00
Jaime Marquínez Ferrándiz 449d5c910c Fix GooglePlusIE: the video_page url has changed of place 2013-06-18 14:22:16 +02:00
Filippo Valsorda 0251f9c9c0 add _search_regex to the new IEs 2013-06-17 19:47:44 +02:00
Filippo Valsorda 8bc7c3d858 Merge branch 'search_regex' - PR #872 - closes #847 2013-06-17 19:28:18 +02:00
Filippo Valsorda af44c94862 use _search_regex in GenericIE 2013-06-17 19:25:35 +02:00
Jaime Marquínez Ferrándiz 32aa88bcae Add GametrailersIE 2013-06-16 20:34:45 +02:00
Jaime Marquínez Ferrándiz 51090d636b VimeoIE: allow to download password protected videos 2013-06-15 11:35:14 +02:00
Anisse Astier 88cebbd7b8 YoutubePlaylistIE: get *all* videos
For that, we add parameter safeSearch=none that asks youtube not filter
results before sending them to us.

Note: this parameter could be added to YoutubeSearchIE and YoutubeUserIE
as well, but I don't know what would be the impact in term of unwanted
results. Maybe expose that as a parameter? For a playlist it's different
since the user chose what she put in the playlist.
2013-06-13 23:45:32 +02:00
Jaime Marquínez Ferrándiz fb8f7280bc GenericIE: try to find videos from twitter cards info 2013-06-13 08:26:39 +02:00
Jaime Marquínez Ferrándiz f380401bbd YoutubeSearchIE: the query is a str, in python 3 it fails if decode is called 2013-06-11 19:15:07 +02:00
Philipp Hagemeister 8cd252f115 Use long rtmpdump options
Note that we accidentally called rtmpdump with -v (--live) instead of -V (--verbose) because we missed this.
2013-06-10 18:14:45 +02:00
Philipp Hagemeister 53f72b11e5 Allow unsetting the proxy with the --proxy option 2013-06-09 23:43:18 +02:00
Filippo Valsorda 78d3442b12 test: extend the reach of info_dict checking
* print the info_dict in a format suitable to easy adding to tests.json during tests if un-tested fields are detected
* make it possible to put the crc32 in tests.json if the field is too long
* complete the "info_dict" fields in existing tests
* fixed the bugs catched doing this
2013-06-09 14:21:42 +02:00
Filippo Valsorda 979a9dd4c4 _html_search_regex with clean_html superpowers 2013-06-09 11:57:13 +02:00
Filippo Valsorda d5979c5d55 do not ask the user to report network errors 2013-06-09 11:55:08 +02:00
Jaime Marquínez Ferrándiz 8027175600 Set the extractor key in playlists entries
If they were videos the extractor key wasn't being set anywhere else
Closes 877
2013-06-08 12:08:44 +02:00
Jaime Marquínez Ferrándiz 3054ff0cbe Merge pull request #853 from mc2avr/master
add ZDFIE
2013-06-08 11:44:01 +02:00
Jaime Marquínez Ferrándiz cd453d38bb Merge pull request #878 from yasoob/master
Added Vbox7.com InfoExtractor and tests.
2013-06-08 10:54:47 +02:00
Filippo Valsorda f5a290eed9 print "please report this issue on GitHub" on every ExtractorError 2013-06-08 09:56:34 +02:00
M.Yasoob Khalid ecb3e676a5 Added Vbox7 Infoextractor 2013-06-08 12:44:38 +05:00
Filippo Valsorda 8b59a98610 XHamster: Can't see the description anywhere in the UI 2013-06-07 12:47:12 +02:00
Filippo Valsorda 8409501206 use search_regex in new IEs 2013-06-07 12:47:12 +02:00
Filippo Valsorda be95cac157 raise exceptions on warnings during tests - and solve a couple of them 2013-06-07 12:46:23 +02:00
Filippo Valsorda 476203d025 print WARNINGs during test + minor fix to NBAIE 2013-06-06 15:07:05 +02:00
Filippo Valsorda 468e2e926b implement fallbacks and defaults in _search_regex 2013-06-06 14:35:08 +02:00
Anna Bernardi ac3e9394e7 Implement search_regex from #847 2013-06-06 14:01:44 +02:00