Release 2021.02.09

[embedsubtitle] Keep original subtitle after conversion if write_subtitles given
Closes: https://github.com/pukkandan/yt-dlp/issues/57#issuecomment-775227745 :ci skip dl
2025-12-16 22:25:40 +07:00 · 2021-02-10 01:26:56 +05:30 · 2021-02-10 00:12:42 +05:30 · 2021-02-09 23:18:20 +05:30 · 2021-02-09 23:12:41 +05:30 · 2021-02-09 22:01:34 +05:30
34 changed files with 461 additions and 285 deletions
--- a/.github/ISSUE_TEMPLATE/1_broken_site.md
+++ b/.github/ISSUE_TEMPLATE/1_broken_site.md
@@ -21,7 +21,7 @@ ## Checklist

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dlc:
- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.01.29. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.02.04. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/pukkandan/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/pukkandan/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ ## Checklist
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.01.29**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.04**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ ## Verbose log
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.01.29
+ [debug] yt-dlp version 2021.02.04
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/2_site_support_request.md
+++ b/.github/ISSUE_TEMPLATE/2_site_support_request.md
@@ -21,7 +21,7 @@ ## Checklist

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dlc:
- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.01.29. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.02.04. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/pukkandan/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/pukkandan/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ ## Checklist
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.01.29**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.04**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones
--- a/.github/ISSUE_TEMPLATE/3_site_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/3_site_feature_request.md
@@ -21,13 +21,13 @@ ## Checklist

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dlc:
- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.01.29. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.02.04. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/pukkandan/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.01.29**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.04**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


--- a/.github/ISSUE_TEMPLATE/4_bug_report.md
+++ b/.github/ISSUE_TEMPLATE/4_bug_report.md
@@ -21,7 +21,7 @@ ## Checklist

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dlc:
- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.01.29. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.02.04. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/pukkandan/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/pukkandan/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ ## Checklist
 -->

 - [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.01.29**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.04**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ ## Verbose log
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.01.29
+ [debug] yt-dlp version 2021.02.04
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/.github/ISSUE_TEMPLATE/5_feature_request.md
+++ b/.github/ISSUE_TEMPLATE/5_feature_request.md
@@ -21,13 +21,13 @@ ## Checklist

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dlc:
- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.01.29. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `youtube-dlc --version` and ensure your version is 2021.02.04. If it's not, see https://github.com/pukkandan/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/pukkandan/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.01.29**
+- [ ] I've verified that I'm running yt-dlp version **2021.02.04**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -84,14 +84,14 @@ jobs:
      with:
          python-version: '3.8'
    - name: Install Requirements
-      run: pip install pyinstaller mutagen
+      run: pip install pyinstaller mutagen Crypto
    - name: Bump version
      id: bump_version
      run: python devscripts/update-version.py
    - name: Print version
      run: echo "${{ steps.bump_version.outputs.ytdlc_version }}"
    - name: Run PyInstaller Script
-      run: python devscripts/pyinst.py 64
+      run: python pyinst.py 64
    - name: Upload youtube-dlc.exe Windows binary
      id: upload-release-windows
      uses: actions/upload-release-asset@v1
@@ -122,14 +122,14 @@ jobs:
          python-version: '3.4.4'
          architecture: 'x86'
    - name: Install Requirements for 32 Bit
-      run: pip install pyinstaller==3.5 mutagen
+      run: pip install pyinstaller==3.5 mutagen Crypto
    - name: Bump version
      id: bump_version
      run: python devscripts/update-version.py
    - name: Print version
      run: echo "${{ steps.bump_version.outputs.ytdlc_version }}"
    - name: Run PyInstaller Script for 32 Bit
-      run: python devscripts/pyinst.py 32
+      run: python pyinst.py 32
    - name: Upload Executable youtube-dlc_x86.exe
      id: upload-release-windows32
      uses: actions/upload-release-asset@v1
--- a/.gitignore
+++ b/.gitignore
@@ -17,6 +17,7 @@ MANIFEST
 test/local_parameters.json
 .coverage
 cover/
+secrets/
 updates_key.pem
 *.egg-info
 .tox
@@ -58,6 +59,10 @@ youtube-dlc
 *.ogg
 *.opus
 *.info.json
+*.live_chat.json
+*.jpg
+*.png
+*.webp
 *.annotations.xml
 *.description

--- a/1
+++ b/1
@@ -17,3 +17,4 @@ alxnull
 FelixFrog
 Zocker1999NET
 nao20010128nao
+shirt-dev
--- a/Changelog.md
+++ b/Changelog.md
@@ -17,12 +17,31 @@ # Instuctions for creating release
 -->


+### 2021.02.09
+* **aria2c support for DASH/HLS**: by [shirt](https://github.com/shirt-dev)
+* **Implement Updater** (`-U`) by [shirt](https://github.com/shirt-dev)
+* [youtube] Fix comment extraction
+* [youtube_live_chat] Improve extraction
+* [youtube] Fix for channel URLs sometimes not downloading all pages
+* [aria2c] Changed default arguments to `--console-log-level=warn --summary-interval=0 --file-allocation=none -x16 -j16 -s16`
+* Add fallback for thumbnails
+* [embedthumbnail] Keep original thumbnail after conversion if write_thumbnail given
+* [embedsubtitle] Keep original subtitle after conversion if write_subtitles given
+* [pyinst.py] Move back to root dir
+* [youtube] Simplified renderer parsing and bugfixes
+* [movefiles] Fix compatibility with python2
+* [remuxvideo] Fix validation of conditional remux
+* [sponskrub] Don't raise error when the video does not exist
+* [documentation] Crypto is an optional dependency
+
+
 ### 2021.02.04
 * **Merge youtube-dl:** Upto [2021.02.04.1](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.02.04.1)
-* **Date/time formatting in output template:** You can now use [`strftime`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) to format date/time fields. Example: `%(upload_date>%Y-%m-%d)s`
+* **Date/time formatting in output template:**
+    * You can use [`strftime`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) to format date/time fields. Example: `%(upload_date>%Y-%m-%d)s`
 * **Multiple output templates:**
-    * Seperate output templates can be given for the different metadata files by using `-o TYPE:TEMPLATE`
-    * The alowed types are: `subtitle|thumbnail|description|annotation|infojson|pl_description|pl_infojson`
+    * Separate output templates can be given for the different metadata files by using `-o TYPE:TEMPLATE`
+    * The allowed types are: `subtitle|thumbnail|description|annotation|infojson|pl_description|pl_infojson`
 * [youtube] More metadata extraction for channel/playlist URLs (channel, uploader, thumbnail, tags)
 * New option `--no-write-playlist-metafiles` to prevent writing playlist metadata files
 * [audius] Fix extractor
@@ -38,7 +57,7 @@ ### 2021.02.04


 ### 2021.01.29
-* **Features from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl)**: Co-authored by @animelover1984 and @bbepis
+* **Features from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl)**: Co-authored by [animelover1984](https://github.com/animelover1984) and [bbepis](https://github.com/bbepis)
    * Add `--get-comments`
    * [youtube] Extract comments
    * [billibilli] Added BiliBiliSearchIE, BilibiliChannelIE
@@ -78,7 +97,7 @@ ### 2021.01.24
    * Valid types are: home, temp, description, annotation, subtitle, infojson, thumbnail
    * Additionally, configuration file is taken from home directory or current directory ([documentation](https://github.com/pukkandan/yt-dlp#:~:text=Home%20Configuration))
 * Allow passing different arguments to different external downloaders ([documentation](https://github.com/pukkandan/yt-dlp#:~:text=--downloader-args%20NAME:ARGS))
-* [mildom] Add extractor by @nao20010128nao
+* [mildom] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
 * Warn when using old style `--external-downloader-args` and `--post-processor-args`
 * Fix `--no-overwrite` when using `--write-link`
 * [sponskrub] Output `unrecognized argument` error message correctly
@@ -109,7 +128,7 @@ ### 2021.01.16

 ### 2021.01.14
 * Added option `--break-on-reject`
-* [roosterteeth.com] Fix for bonus episodes by @Zocker1999NET
+* [roosterteeth.com] Fix for bonus episodes by [Zocker1999NET](https://github.com/Zocker1999NET)
 * [tiktok] Fix for when share_info is empty
 * [EmbedThumbnail] Fix bug due to incorrect function name
 * [documentation] Changed sponskrub links to point to [pukkandan/sponskrub](https://github.com/pukkandan/SponSkrub) since I am now providing both linux and windows releases
@@ -118,18 +137,18 @@ ### 2021.01.14


 ### 2021.01.12
-* [roosterteeth.com] Add subtitle support by @samiksome
-* Added `--force-overwrites`, `--no-force-overwrites` by @alxnull
+* [roosterteeth.com] Add subtitle support by [samiksome](https://github.com/samiksome)
+* Added `--force-overwrites`, `--no-force-overwrites` by [alxnull](https://github.com/alxnull)
 * Changed fork name to `yt-dlp`
-* Fix typos by @FelixFrog
+* Fix typos by [FelixFrog](https://github.com/FelixFrog)
 * [ci] Option to skip
 * [changelog] Added unreleased changes in blackjack4494/yt-dlc


 ### 2021.01.10
-* [archive.org] Fix extractor and add support for audio and playlists by @wporr
-* [Animelab] Added by @mariuszskon
-* [youtube:search] Fix view_count by @ohnonot
+* [archive.org] Fix extractor and add support for audio and playlists by [wporr](https://github.com/wporr)
+* [Animelab] Added by [mariuszskon](https://github.com/mariuszskon)
+* [youtube:search] Fix view_count by [ohnonot](https://github.com/ohnonot)
 * [youtube] Show if video is embeddable in info
 * Update version badge automatically in README
 * Enable `test_youtube_search_matching`
@@ -138,11 +157,11 @@ ### 2021.01.10

 ### 2021.01.09
 * [youtube] Fix bug in automatic caption extraction
-* Add `post_hooks` to YoutubeDL by @alexmerkel
-* Batch file enumeration improvements by @glenn-slayden
-* Stop immediately when reaching `--max-downloads` by @glenn-slayden
-* Fix incorrect ANSI sequence for restoring console-window title by @glenn-slayden
-* Kill child processes when yt-dlc is killed by @Unrud
+* Add `post_hooks` to YoutubeDL by [alexmerkel](https://github.com/alexmerkel)
+* Batch file enumeration improvements by [glenn-slayden](https://github.com/glenn-slayden)
+* Stop immediately when reaching `--max-downloads` by [glenn-slayden](https://github.com/glenn-slayden)
+* Fix incorrect ANSI sequence for restoring console-window title by [glenn-slayden](https://github.com/glenn-slayden)
+* Kill child processes when yt-dlc is killed by [Unrud](https://github.com/Unrud)


 ### 2021.01.08
@@ -152,11 +171,11 @@ ### 2021.01.08


 ### 2021.01.07-1
-* [Akamai] fix by @nixxo
-* [Tiktok] merge youtube-dl tiktok extractor by @GreyAlien502
-* [vlive] add support for playlists by @kyuyeunk
-* [youtube_live_chat] make sure playerOffsetMs is positive by @siikamiika
-* Ignore extra data streams in ffmpeg by @jbruchon
+* [Akamai] fix by [nixxo](https://github.com/nixxo)
+* [Tiktok] merge youtube-dl tiktok extractor by [GreyAlien502](https://github.com/GreyAlien502)
+* [vlive] add support for playlists by [kyuyeunk](https://github.com/kyuyeunk)
+* [youtube_live_chat] make sure playerOffsetMs is positive by [siikamiika](https://github.com/siikamiika)
+* Ignore extra data streams in ffmpeg by [jbruchon](https://github.com/jbruchon)
 * Allow passing different arguments to different postprocessors using `--postprocessor-args`
 * Deprecated `--sponskrub-args`. The same can now be done using `--postprocessor-args "sponskrub:<args>"`
 * [CI] Split tests into core-test and full-test
@@ -186,15 +205,15 @@ ### 2021.01.05
    * Changed video format sorting to show video only files and video+audio files together.
    * Added `--video-multistreams`, `--no-video-multistreams`, `--audio-multistreams`, `--no-audio-multistreams`
    * Added `b`,`w`,`v`,`a` as alias for `best`, `worst`, `video` and `audio` respectively
-* **Shortcut Options:** Added `--write-link`, `--write-url-link`, `--write-webloc-link`, `--write-desktop-link` by @h-h-h-h - See [Internet Shortcut Options]README.md(#internet-shortcut-options) for details
+* **Shortcut Options:** Added `--write-link`, `--write-url-link`, `--write-webloc-link`, `--write-desktop-link` by [h-h-h-h](https://github.com/h-h-h-h) - See [Internet Shortcut Options]README.md(#internet-shortcut-options) for details
 * **Sponskrub integration:** Added `--sponskrub`, `--sponskrub-cut`, `--sponskrub-force`, `--sponskrub-location`, `--sponskrub-args` - See [SponSkrub Options](README.md#sponskrub-options-sponsorblock) for details
-* Added `--force-download-archive` (`--force-write-archive`) by @h-h-h-h
+* Added `--force-download-archive` (`--force-write-archive`) by [h-h-h-h](https://github.com/h-h-h-h)
 * Added `--list-formats-as-table`,  `--list-formats-old`
 * **Negative Options:** Makes it possible to negate most boolean options by adding a `no-` to the switch. Usefull when you want to reverse an option that is defined in a config file
    * Added `--no-ignore-dynamic-mpd`, `--no-allow-dynamic-mpd`, `--allow-dynamic-mpd`, `--youtube-include-hls-manifest`, `--no-youtube-include-hls-manifest`, `--no-youtube-skip-hls-manifest`, `--no-download`, `--no-download-archive`, `--resize-buffer`, `--part`, `--mtime`, `--no-keep-fragments`, `--no-cookies`, `--no-write-annotations`, `--no-write-info-json`, `--no-write-description`, `--no-write-thumbnail`, `--youtube-include-dash-manifest`, `--post-overwrites`, `--no-keep-video`, `--no-embed-subs`, `--no-embed-thumbnail`, `--no-add-metadata`, `--no-include-ads`, `--no-write-sub`, `--no-write-auto-sub`, `--no-playlist-reverse`, `--no-restrict-filenames`, `--youtube-include-dash-manifest`, `--no-format-sort-force`, `--flat-videos`, `--no-list-formats-as-table`, `--no-sponskrub`, `--no-sponskrub-cut`, `--no-sponskrub-force`
    * Renamed: `--write-subs`, `--no-write-subs`, `--no-write-auto-subs`, `--write-auto-subs`. Note that these can still be used without the ending "s"
 * Relaxed validation for format filters so that any arbitrary field can be used
-* Fix for embedding thumbnail in mp3 by @pauldubois98 ([ytdl-org/youtube-dl#21569](https://github.com/ytdl-org/youtube-dl/pull/21569))
+* Fix for embedding thumbnail in mp3 by [pauldubois98](https://github.com/pauldubois98) ([ytdl-org/youtube-dl#21569](https://github.com/ytdl-org/youtube-dl/pull/21569))
 * Make Twitch Video ID output from Playlist and VOD extractor same. This is only a temporary fix
 * **Merge youtube-dl:** Upto [2021.01.03](https://github.com/ytdl-org/youtube-dl/commit/8e953dcbb10a1a42f4e12e4e132657cb0100a1f8) - See [blackjack4494/yt-dlc#280](https://github.com/blackjack4494/yt-dlc/pull/280) for details
    * Extractors [tiktok](https://github.com/ytdl-org/youtube-dl/commit/fb626c05867deab04425bad0c0b16b55473841a2) and [hotstar](https://github.com/ytdl-org/youtube-dl/commit/bb38a1215718cdf36d73ff0a7830a64cd9fa37cc) have not been merged
@@ -211,7 +230,7 @@ ### Unreleased changes in [blackjack4494/yt-dlc](https://github.com/blackjack449
    * Redirect channel home to /video
    * Print youtube's warning message
    * Multiple pages are handled better for feeds
-* Add --break-on-existing by @gergesh
+* Add --break-on-existing by [gergesh](https://github.com/gergesh)
 * Pre-check video IDs in the archive before downloading
 * [bitwave.tv] New extractor
 * [Gedi] Add extractor
--- a/README.md
+++ b/README.md
@@ -2,7 +2,9 @@ # YT-DLP

 [![Release version](https://img.shields.io/github/v/release/pukkandan/yt-dlp?color=brightgreen&label=Release)](https://github.com/pukkandan/yt-dlp/releases/latest)
 [![License: Unlicense](https://img.shields.io/badge/License-Unlicense-blue.svg)](LICENSE)
-[![CI Status](https://github.com/pukkandan/yt-dlp/workflows/Core%20Tests/badge.svg?branch=master)](https://github.com/pukkandan/yt-dlp/actions)  
+[![CI Status](https://github.com/pukkandan/yt-dlp/workflows/Core%20Tests/badge.svg?branch=master)](https://github.com/pukkandan/yt-dlp/actions)
+[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord)](https://discord.gg/S75JaBna)  
+
 [![Commits](https://img.shields.io/github/commit-activity/m/pukkandan/yt-dlp?label=commits)](https://github.com/pukkandan/yt-dlp/commits)
 [![Last Commit](https://img.shields.io/github/last-commit/pukkandan/yt-dlp/master)](https://github.com/pukkandan/yt-dlp/commits)
 [![Downloads](https://img.shields.io/github/downloads/pukkandan/yt-dlp/total)](https://github.com/pukkandan/yt-dlp/releases/latest)
@@ -102,11 +104,11 @@ ### UPDATE
 ### COMPILE

 **For Windows**:
-To build the Windows executable, you must have pyinstaller (and optionally mutagen for embedding thumbnail in opus/ogg files)
+To build the Windows executable, you must have pyinstaller (and optionally mutagen and Crypto)

-    python -m pip install --upgrade pyinstaller mutagen
+    python -m pip install --upgrade pyinstaller mutagen Crypto

-Once you have all the necessary dependancies installed, just run `py devscripts\pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it. It is strongly reccomended to use python3 although python2.6+ is supported.
+Once you have all the necessary dependancies installed, just run `py pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it. It is strongly reccomended to use python3 although python2.6+ is supported.

 You can also build the executable without any version info or metadata by using:

--- a/devscripts/update-version.py
+++ b/devscripts/update-version.py
@@ -8,7 +8,7 @@
 exec(compile(open('youtube_dlc/version.py').read(), 'youtube_dlc/version.py', 'exec'))
 old_version = locals()['__version__']

-old_version_list = old_version.replace('-', '.').split(".", 4)
+old_version_list = old_version.split(".", 4)

 old_ver = '.'.join(old_version_list[:3])
 old_rev = old_version_list[3] if len(old_version_list) > 3 else ''
--- a/docs/supportedsites.md
+++ b/docs/supportedsites.md
@@ -1218,9 +1218,9 @@ # Supported sites
 - **youtube:history**: Youtube watch history, ":ythistory" for short (requires authentication)
 - **youtube:playlist**: YouTube.com playlists
 - **youtube:recommended**: YouTube.com recommended videos, ":ytrec" for short (requires authentication)
- - **youtube:search**: YouTube.com searches
+ - **youtube:search**: YouTube.com searches, "ytsearch" keyword
 - **youtube:search:date**: YouTube.com searches, newest videos first, "ytsearchdate" keyword
- - **youtube:search_url**: YouTube.com searches, "ytsearch" keyword
+ - **youtube:search_url**: YouTube.com search URLs
 - **youtube:subscriptions**: YouTube.com subscriptions feed, ":ytsubs" for short (requires authentication)
 - **youtube:tab**: YouTube.com tab
 - **youtube:watchlater**: Youtube watch later list, ":ytwatchlater" for short (requires authentication)
--- a/devscripts/pyinst.py
+++ b/devscripts/pyinst.py
@@ -3,7 +3,7 @@

 from __future__ import unicode_literals
 import sys
-import os
+# import os
 import platform

 from PyInstaller.utils.win32.versioninfo import (
@@ -18,16 +18,15 @@
 _x86 = '_x86' if arch == '32' else ''

 FILE_DESCRIPTION = 'Media Downloader%s' % (' (32 Bit)' if _x86 else '')
-SHORT_URLS = {'32': 'git.io/JUGsM', '64': 'git.io/JLh7K'}

-root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
-print('Changing working directory to %s' % root_dir)
-os.chdir(root_dir)
+# root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+# print('Changing working directory to %s' % root_dir)
+# os.chdir(root_dir)

 exec(compile(open('youtube_dlc/version.py').read(), 'youtube_dlc/version.py', 'exec'))
 VERSION = locals()['__version__']

-VERSION_LIST = VERSION.replace('-', '.').split('.')
+VERSION_LIST = VERSION.split('.')
 VERSION_LIST = list(map(int, VERSION_LIST)) + [0] * (4 - len(VERSION_LIST))

 print('Version: %s%s' % (VERSION, _x86))
@@ -49,7 +48,7 @@
            StringTable(
                '040904B0', [
                    StringStruct('Comments', 'Youtube-dlc%s Command Line Interface.' % _x86),
-                    StringStruct('CompanyName', 'pukkandan@gmail.com'),
+                    StringStruct('CompanyName', 'https://github.com/pukkandan/yt-dlp'),
                    StringStruct('FileDescription', FILE_DESCRIPTION),
                    StringStruct('FileVersion', VERSION),
                    StringStruct('InternalName', 'youtube-dlc%s' % _x86),
@@ -59,7 +58,7 @@
                    ),
                    StringStruct('OriginalFilename', 'youtube-dlc%s.exe' % _x86),
                    StringStruct('ProductName', 'Youtube-dlc%s' % _x86),
-                    StringStruct('ProductVersion', '%s%s | %s' % (VERSION, _x86, SHORT_URLS[arch])),
+                    StringStruct('ProductVersion', '%s%s' % (VERSION, _x86)),
                ])]),
        VarFileInfo([VarStruct('Translation', [0, 1200])])
    ]
@@ -73,6 +72,7 @@
    '--exclude-module=test',
    '--exclude-module=ytdlp_plugins',
    '--hidden-import=mutagen',
+    '--hidden-import=Crypto',
    'youtube_dlc/__main__.py',
 ])
 SetVersion('dist/youtube-dlc%s.exe' % _x86, VERSION_FILE)
--- a/requirements.txt
+++ b/requirements.txt
@@ -1 +1,2 @@
 mutagen
+Crypto
--- a/setup.py
+++ b/setup.py
@@ -20,7 +20,7 @@
    '**PS**: Many links in this document will not work since this is a copy of the README.md from Github',
    open("README.md", "r", encoding="utf-8").read()))

-REQUIREMENTS = ['mutagen']
+REQUIREMENTS = ['mutagen', 'Crypto']


 if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
--- a/test/test_update.py.disabled
+++ b/test/test_update.py.disabled
--- a/youtube_dlc/YoutubeDL.py
+++ b/youtube_dlc/YoutubeDL.py
@@ -2131,7 +2131,7 @@ def dl(name, info, subtitle=False):
                sub_format = sub_info['ext']
                sub_fn = self.prepare_filename(info_dict, 'subtitle')
                sub_filename = subtitles_filename(
-                     temp_filename if not skip_dl else sub_fn,
+                    temp_filename if not skip_dl else sub_fn,
                    sub_lang, sub_format, info_dict.get('ext'))
                sub_filename_final = subtitles_filename(sub_fn, sub_lang, sub_format, info_dict.get('ext'))
                if not self.params.get('overwrites', True) and os.path.exists(encodeFilename(sub_filename)):
@@ -2292,12 +2292,9 @@ def existing_file(*filepaths):
                    downloaded = []
                    merger = FFmpegMergerPP(self)
                    if not merger.available:
-                        postprocessors = []
                        self.report_warning('You have requested multiple '
                                            'formats but ffmpeg is not installed.'
                                            ' The formats won\'t be merged.')
-                    else:
-                        postprocessors = [merger]

                    def compatible_formats(formats):
                        # TODO: some formats actually allow this (mkv, webm, ogg, mp4), but not all of them.
@@ -2349,7 +2346,8 @@ def correct_ext(filename):
                            downloaded.append(fname)
                            partial_success, real_download = dl(fname, new_info)
                            success = success and partial_success
-                        info_dict['__postprocessors'] = postprocessors
+                        if merger.available:
+                            info_dict['__postprocessors'].append(merger)
                        info_dict['__files_to_merge'] = downloaded
                        # Even if there were no downloads, it is being merged only now
                        info_dict['__real_download'] = True
@@ -2895,20 +2893,17 @@ def get_encoding(self):
        return encoding

    def _write_thumbnails(self, info_dict, filename):  # return the extensions
-        if self.params.get('writethumbnail', False):
-            thumbnails = info_dict.get('thumbnails')
-            if thumbnails:
-                thumbnails = [thumbnails[-1]]
-        elif self.params.get('write_all_thumbnails', False):
+        write_all = self.params.get('write_all_thumbnails', False)
+        thumbnails = []
+        if write_all or self.params.get('writethumbnail', False):
            thumbnails = info_dict.get('thumbnails') or []
-        else:
-            thumbnails = []
+        multiple = write_all and len(thumbnails) > 1

        ret = []
-        for t in thumbnails:
+        for t in thumbnails[::1 if write_all else -1]:
            thumb_ext = determine_ext(t['url'], 'jpg')
-            suffix = '%s.' % t['id'] if len(thumbnails) > 1 else ''
-            thumb_display_id = '%s ' % t['id'] if len(thumbnails) > 1 else ''
+            suffix = '%s.' % t['id'] if multiple else ''
+            thumb_display_id = '%s ' % t['id'] if multiple else ''
            t['filename'] = thumb_filename = replace_extension(filename, suffix + thumb_ext, info_dict.get('ext'))

            if not self.params.get('overwrites', True) and os.path.exists(encodeFilename(thumb_filename)):
@@ -2928,4 +2923,6 @@ def _write_thumbnails(self, info_dict, filename):  # return the extensions
                except (compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
                    self.report_warning('Unable to download thumbnail "%s": %s' %
                                        (t['url'], error_to_compat_str(err)))
+            if ret and not write_all:
+                break
        return ret
--- a/youtube_dlc/init.py
+++ b/youtube_dlc/init.py
@@ -15,7 +15,6 @@

 from .options import (
    parseOpts,
-    _remux_formats,
 )
 from .compat import (
    compat_getpass,
@@ -24,7 +23,6 @@
 from .utils import (
    DateRange,
    decodeOption,
-    DEFAULT_OUTTMPL,
    DownloadError,
    ExistingVideoReached,
    expand_path,
@@ -33,11 +31,12 @@
    preferredencoding,
    read_batch_urls,
    RejectedVideoReached,
+    REMUX_EXTENSIONS,
+    render_table,
    SameFileError,
    setproctitle,
    std_headers,
    write_string,
-    render_table,
 )
 from .update import update_self
 from .downloader import (
@@ -211,13 +210,15 @@ def parse_retries(retries):
        if not opts.audioquality.isdigit():
            parser.error('invalid audio quality specified')
    if opts.recodevideo is not None:
-        if opts.recodevideo not in _remux_formats:
+        if opts.recodevideo not in REMUX_EXTENSIONS:
            parser.error('invalid video recode format specified')
    if opts.remuxvideo and opts.recodevideo:
        opts.remuxvideo = None
        write_string('WARNING: --remux-video is ignored since --recode-video was given\n', out=sys.stderr)
    if opts.remuxvideo is not None:
-        if opts.remuxvideo not in _remux_formats:
+        opts.remuxvideo = opts.remuxvideo.replace(' ', '')
+        remux_regex = r'{0}(?:/{0})*$'.format(r'(?:\w+>)?(?:%s)' % '|'.join(REMUX_EXTENSIONS))
+        if not re.match(remux_regex, opts.remuxvideo):
            parser.error('invalid video remux format specified')
    if opts.convertsubtitles is not None:
        if opts.convertsubtitles not in ['srt', 'vtt', 'ass', 'lrc']:
@@ -232,11 +233,6 @@ def parse_retries(retries):
    if opts.extractaudio and not opts.keepvideo and opts.format is None:
        opts.format = 'bestaudio/best'

-    # --all-sub automatically sets --write-sub if --write-auto-sub is not given
-    # this was the old behaviour if only --all-sub was given.
-    if opts.allsubtitles and not opts.writeautomaticsub:
-        opts.writesubtitles = True
-
    outtmpl = opts.outtmpl
    if not outtmpl:
        outtmpl = {'default': (
@@ -310,9 +306,17 @@ def parse_retries(retries):
            'format': opts.convertsubtitles,
        })
    if opts.embedsubtitles:
+        already_have_subtitle = opts.writesubtitles
        postprocessors.append({
            'key': 'FFmpegEmbedSubtitle',
+            'already_have_subtitle': already_have_subtitle
        })
+        if not already_have_subtitle:
+            opts.writesubtitles = True
+    # --all-sub automatically sets --write-sub if --write-auto-sub is not given
+    # this was the old behaviour if only --all-sub was given.
+    if opts.allsubtitles and not opts.writeautomaticsub:
+        opts.writesubtitles = True
    if opts.embedthumbnail:
        already_have_thumbnail = opts.writethumbnail or opts.write_all_thumbnails
        postprocessors.append({
@@ -353,7 +357,11 @@ def parse_retries(retries):
        opts.postprocessor_args.setdefault('sponskrub', [])
        opts.postprocessor_args['default'] = opts.postprocessor_args['default-compat']

-    audio_ext = opts.audioformat if (opts.extractaudio and opts.audioformat != 'best') else None
+    final_ext = (
+        opts.recodevideo
+        or (opts.remuxvideo in REMUX_EXTENSIONS) and opts.remuxvideo
+        or (opts.extractaudio and opts.audioformat != 'best') and opts.audioformat
+        or None)

    match_filter = (
        None if opts.match_filter is None
@@ -474,7 +482,7 @@ def parse_retries(retries):
        'extract_flat': opts.extract_flat,
        'mark_watched': opts.mark_watched,
        'merge_output_format': opts.merge_output_format,
-        'final_ext': opts.recodevideo or opts.remuxvideo or audio_ext,
+        'final_ext': final_ext,
        'postprocessors': postprocessors,
        'fixup': opts.fixup,
        'source_address': opts.source_address,
--- a/youtube_dlc/downloader/init.py
+++ b/youtube_dlc/downloader/init.py
@@ -1,11 +1,24 @@
 from __future__ import unicode_literals

+from ..utils import (
+    determine_protocol,
+)
+
+
+def _get_real_downloader(info_dict, protocol=None, *args, **kwargs):
+    info_copy = info_dict.copy()
+    if protocol:
+        info_copy['protocol'] = protocol
+    return get_suitable_downloader(info_copy, *args, **kwargs)
+
+
+# Some of these require _get_real_downloader
 from .common import FileDownloader
+from .dash import DashSegmentsFD
 from .f4m import F4mFD
 from .hls import HlsFD
 from .http import HttpFD
 from .rtmp import RtmpFD
-from .dash import DashSegmentsFD
 from .rtsp import RtspFD
 from .ism import IsmFD
 from .youtube_live_chat import YoutubeLiveChatReplayFD
@@ -14,10 +27,6 @@
    FFmpegFD,
 )

-from ..utils import (
-    determine_protocol,
-)
-
 PROTOCOL_MAP = {
    'rtmp': RtmpFD,
    'm3u8_native': HlsFD,
@@ -31,7 +40,7 @@
 }


-def get_suitable_downloader(info_dict, params={}):
+def get_suitable_downloader(info_dict, params={}, default=HttpFD):
    """Get the downloader class that can handle the info dict."""
    protocol = determine_protocol(info_dict)
    info_dict['protocol'] = protocol
@@ -45,16 +54,17 @@ def get_suitable_downloader(info_dict, params={}):
        if ed.can_download(info_dict):
            return ed

-    if protocol.startswith('m3u8') and info_dict.get('is_live'):
-        return FFmpegFD
+    if protocol.startswith('m3u8'):
+        if info_dict.get('is_live'):
+            return FFmpegFD
+        elif _get_real_downloader(info_dict, 'frag_urls', params, None):
+            return HlsFD
+        elif params.get('hls_prefer_native') is True:
+            return HlsFD
+        elif params.get('hls_prefer_native') is False:
+            return FFmpegFD

-    if protocol == 'm3u8' and params.get('hls_prefer_native') is True:
-        return HlsFD
-
-    if protocol == 'm3u8_native' and params.get('hls_prefer_native') is False:
-        return FFmpegFD
-
-    return PROTOCOL_MAP.get(protocol, HttpFD)
+    return PROTOCOL_MAP.get(protocol, default)


 __all__ = [
--- a/youtube_dlc/downloader/dash.py
+++ b/youtube_dlc/downloader/dash.py
@@ -1,6 +1,8 @@
 from __future__ import unicode_literals

+from ..downloader import _get_real_downloader
 from .fragment import FragmentFD
+
 from ..compat import compat_urllib_error
 from ..utils import (
    DownloadError,
@@ -20,31 +22,42 @@ def real_download(self, filename, info_dict):
        fragments = info_dict['fragments'][:1] if self.params.get(
            'test', False) else info_dict['fragments']

+        real_downloader = _get_real_downloader(info_dict, 'frag_urls', self.params, None)
+
        ctx = {
            'filename': filename,
            'total_frags': len(fragments),
        }

-        self._prepare_and_start_frag_download(ctx)
+        if real_downloader:
+            self._prepare_external_frag_download(ctx)
+        else:
+            self._prepare_and_start_frag_download(ctx)

        fragment_retries = self.params.get('fragment_retries', 0)
        skip_unavailable_fragments = self.params.get('skip_unavailable_fragments', True)

+        fragment_urls = []
        frag_index = 0
        for i, fragment in enumerate(fragments):
            frag_index += 1
            if frag_index <= ctx['fragment_index']:
                continue
+            fragment_url = fragment.get('url')
+            if not fragment_url:
+                assert fragment_base_url
+                fragment_url = urljoin(fragment_base_url, fragment['path'])
+
+            if real_downloader:
+                fragment_urls.append(fragment_url)
+                continue
+
            # In DASH, the first segment contains necessary headers to
            # generate a valid MP4 file, so always abort for the first segment
            fatal = i == 0 or not skip_unavailable_fragments
            count = 0
            while count <= fragment_retries:
                try:
-                    fragment_url = fragment.get('url')
-                    if not fragment_url:
-                        assert fragment_base_url
-                        fragment_url = urljoin(fragment_base_url, fragment['path'])
                    success, frag_content = self._download_fragment(ctx, fragment_url, info_dict)
                    if not success:
                        return False
@@ -75,6 +88,16 @@ def real_download(self, filename, info_dict):
                self.report_error('giving up after %s fragment retries' % fragment_retries)
                return False

-        self._finish_frag_download(ctx)
-
+        if real_downloader:
+            info_copy = info_dict.copy()
+            info_copy['url_list'] = fragment_urls
+            fd = real_downloader(self.ydl, self.params)
+            # TODO: Make progress updates work without hooking twice
+            # for ph in self._progress_hooks:
+            #     fd.add_progress_hook(ph)
+            success = fd.real_download(filename, info_copy)
+            if not success:
+                return False
+        else:
+            self._finish_frag_download(ctx)
        return True
--- a/youtube_dlc/downloader/external.py
+++ b/youtube_dlc/downloader/external.py
@@ -5,6 +5,13 @@
 import subprocess
 import sys
 import time
+import shutil
+
+try:
+    from Crypto.Cipher import AES
+    can_decrypt_frag = True
+except ImportError:
+    can_decrypt_frag = False

 from .common import FileDownloader
 from ..compat import (
@@ -18,15 +25,19 @@
    cli_bool_option,
    cli_configuration_args,
    encodeFilename,
+    error_to_compat_str,
    encodeArgument,
    handle_youtubedl_headers,
    check_executable,
    is_outdated_version,
    process_communicate_or_kill,
+    sanitized_Request,
 )


 class ExternalFD(FileDownloader):
+    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps')
+
    def real_download(self, filename, info_dict):
        self.report_destination(filename)
        tmpfilename = self.temp_name(filename)
@@ -79,7 +90,7 @@ def available(cls):

    @classmethod
    def supports(cls, info_dict):
-        return info_dict['protocol'] in ('http', 'https', 'ftp', 'ftps')
+        return info_dict['protocol'] in cls.SUPPORTED_PROTOCOLS

    @classmethod
    def can_download(cls, info_dict):
@@ -109,8 +120,47 @@ def _call_downloader(self, tmpfilename, info_dict):
        _, stderr = process_communicate_or_kill(p)
        if p.returncode != 0:
            self.to_stderr(stderr.decode('utf-8', 'replace'))
+
+        if 'url_list' in info_dict:
+            file_list = []
+            for [i, url] in enumerate(info_dict['url_list']):
+                tmpsegmentname = '%s_%s.frag' % (tmpfilename, i)
+                file_list.append(tmpsegmentname)
+            with open(tmpfilename, 'wb') as dest:
+                for i in file_list:
+                    if 'decrypt_info' in info_dict:
+                        decrypt_info = info_dict['decrypt_info']
+                        with open(i, 'rb') as src:
+                            if decrypt_info['METHOD'] == 'AES-128':
+                                iv = decrypt_info.get('IV')
+                                decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
+                                    self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
+                                encrypted_data = src.read()
+                                decrypted_data = AES.new(
+                                    decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(encrypted_data)
+                                dest.write(decrypted_data)
+                            else:
+                                shutil.copyfileobj(open(i, 'rb'), dest)
+                    else:
+                        shutil.copyfileobj(open(i, 'rb'), dest)
+            if not self.params.get('keep_fragments', False):
+                for file_path in file_list:
+                    try:
+                        os.remove(file_path)
+                    except OSError as ose:
+                        self.report_error("Unable to delete file %s; %s" % (file_path, error_to_compat_str(ose)))
+                try:
+                    file_path = '%s.frag.urls' % tmpfilename
+                    os.remove(file_path)
+                except OSError as ose:
+                    self.report_error("Unable to delete file %s; %s" % (file_path, error_to_compat_str(ose)))
+
        return p.returncode

+    def _prepare_url(self, info_dict, url):
+        headers = info_dict.get('http_headers')
+        return sanitized_Request(url, None, headers) if headers else url
+

 class CurlFD(ExternalFD):
    AVAILABLE_OPT = '-V'
@@ -186,15 +236,17 @@ def _make_cmd(self, tmpfilename, info_dict):

 class Aria2cFD(ExternalFD):
    AVAILABLE_OPT = '-v'
+    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'frag_urls')

    def _make_cmd(self, tmpfilename, info_dict):
        cmd = [self.exe, '-c']
-        cmd += self._configuration_args([
-            '--min-split-size', '1M', '--max-connection-per-server', '4'])
        dn = os.path.dirname(tmpfilename)
+        if 'url_list' not in info_dict:
+            cmd += ['--out', os.path.basename(tmpfilename)]
+        verbose_level_args = ['--console-log-level=warn', '--summary-interval=0']
+        cmd += self._configuration_args(['--file-allocation=none', '-x16', '-j16', '-s16'] + verbose_level_args)
        if dn:
            cmd += ['--dir', dn]
-        cmd += ['--out', os.path.basename(tmpfilename)]
        if info_dict.get('http_headers') is not None:
            for key, val in info_dict['http_headers'].items():
                cmd += ['--header', '%s: %s' % (key, val)]
@@ -202,7 +254,21 @@ def _make_cmd(self, tmpfilename, info_dict):
        cmd += self._option('--all-proxy', 'proxy')
        cmd += self._bool_option('--check-certificate', 'nocheckcertificate', 'false', 'true', '=')
        cmd += self._bool_option('--remote-time', 'updatetime', 'true', 'false', '=')
-        cmd += ['--', info_dict['url']]
+        cmd += ['--auto-file-renaming=false']
+        if 'url_list' in info_dict:
+            cmd += verbose_level_args
+            cmd += ['--uri-selector', 'inorder', '--download-result=hide']
+            url_list_file = '%s.frag.urls' % tmpfilename
+            url_list = []
+            for [i, url] in enumerate(info_dict['url_list']):
+                tmpsegmentname = '%s_%s.frag' % (os.path.basename(tmpfilename), i)
+                url_list.append('%s\n\tout=%s' % (url, tmpsegmentname))
+            with open(url_list_file, 'w') as f:
+                f.write('\n'.join(url_list))
+
+            cmd += ['-i', url_list_file]
+        else:
+            cmd += ['--', info_dict['url']]
        return cmd


@@ -221,9 +287,7 @@ def _make_cmd(self, tmpfilename, info_dict):


 class FFmpegFD(ExternalFD):
-    @classmethod
-    def supports(cls, info_dict):
-        return info_dict['protocol'] in ('http', 'https', 'ftp', 'ftps', 'm3u8', 'rtsp', 'rtmp', 'mms')
+    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'rtsp', 'rtmp', 'mms')

    @classmethod
    def available(cls):
--- a/youtube_dlc/downloader/fragment.py
+++ b/youtube_dlc/downloader/fragment.py
@@ -277,3 +277,24 @@ def _finish_frag_download(self, ctx):
            'status': 'finished',
            'elapsed': elapsed,
        })
+
+    def _prepare_external_frag_download(self, ctx):
+        if 'live' not in ctx:
+            ctx['live'] = False
+        if not ctx['live']:
+            total_frags_str = '%d' % ctx['total_frags']
+            ad_frags = ctx.get('ad_frags', 0)
+            if ad_frags:
+                total_frags_str += ' (not including %d ad)' % ad_frags
+        else:
+            total_frags_str = 'unknown (live)'
+        self.to_screen(
+            '[%s] Total fragments: %s' % (self.FD_NAME, total_frags_str))
+
+        tmpfilename = self.temp_name(ctx['filename'])
+
+        # Should be initialized before ytdl file check
+        ctx.update({
+            'tmpfilename': tmpfilename,
+            'fragment_index': 0,
+        })
--- a/youtube_dlc/downloader/hls.py
+++ b/youtube_dlc/downloader/hls.py
@@ -8,6 +8,7 @@
 except ImportError:
    can_decrypt_frag = False

+from ..downloader import _get_real_downloader
 from .fragment import FragmentFD
 from .external import FFmpegFD

@@ -73,10 +74,13 @@ def real_download(self, filename, info_dict):
                'hlsnative has detected features it does not support, '
                'extraction will be delegated to ffmpeg')
            fd = FFmpegFD(self.ydl, self.params)
-            for ph in self._progress_hooks:
-                fd.add_progress_hook(ph)
+            # TODO: Make progress updates work without hooking twice
+            # for ph in self._progress_hooks:
+            #     fd.add_progress_hook(ph)
            return fd.real_download(filename, info_dict)

+        real_downloader = _get_real_downloader(info_dict, 'frag_urls', self.params, None)
+
        def is_ad_fragment_start(s):
            return (s.startswith('#ANVATO-SEGMENT-INFO') and 'type=ad' in s
                    or s.startswith('#UPLYNK-SEGMENT') and s.endswith(',ad'))
@@ -85,6 +89,8 @@ def is_ad_fragment_end(s):
            return (s.startswith('#ANVATO-SEGMENT-INFO') and 'type=master' in s
                    or s.startswith('#UPLYNK-SEGMENT') and s.endswith(',segment'))

+        fragment_urls = []
+
        media_frags = 0
        ad_frags = 0
        ad_frag_next = False
@@ -109,7 +115,10 @@ def is_ad_fragment_end(s):
            'ad_frags': ad_frags,
        }

-        self._prepare_and_start_frag_download(ctx)
+        if real_downloader:
+            self._prepare_external_frag_download(ctx)
+        else:
+            self._prepare_and_start_frag_download(ctx)

        fragment_retries = self.params.get('fragment_retries', 0)
        skip_unavailable_fragments = self.params.get('skip_unavailable_fragments', True)
@@ -140,6 +149,11 @@ def is_ad_fragment_end(s):
                        else compat_urlparse.urljoin(man_url, line))
                    if extra_query:
                        frag_url = update_url_query(frag_url, extra_query)
+
+                    if real_downloader:
+                        fragment_urls.append(frag_url)
+                        continue
+
                    count = 0
                    headers = info_dict.get('http_headers', {})
                    if byte_range:
@@ -168,6 +182,7 @@ def is_ad_fragment_end(s):
                        self.report_error(
                            'giving up after %s fragment retries' % fragment_retries)
                        return False
+
                    if decrypt_info['METHOD'] == 'AES-128':
                        iv = decrypt_info.get('IV') or compat_struct_pack('>8xq', media_sequence)
                        decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
@@ -211,6 +226,17 @@ def is_ad_fragment_end(s):
                elif is_ad_fragment_end(line):
                    ad_frag_next = False

-        self._finish_frag_download(ctx)
-
+        if real_downloader:
+            info_copy = info_dict.copy()
+            info_copy['url_list'] = fragment_urls
+            info_copy['decrypt_info'] = decrypt_info
+            fd = real_downloader(self.ydl, self.params)
+            # TODO: Make progress updates work without hooking twice
+            # for ph in self._progress_hooks:
+            #     fd.add_progress_hook(ph)
+            success = fd.real_download(filename, info_copy)
+            if not success:
+                return False
+        else:
+            self._finish_frag_download(ctx)
        return True
--- a/youtube_dlc/downloader/youtube_live_chat.py
+++ b/youtube_dlc/downloader/youtube_live_chat.py
@@ -50,7 +50,16 @@ def download_and_parse_fragment(url, frag_index):
                    success, raw_fragment = dl_fragment(url)
                    if not success:
                        return False, None, None
-                    data = parse_yt_initial_data(raw_fragment) or json.loads(raw_fragment)['response']
+                    data = parse_yt_initial_data(raw_fragment)
+                    if not data:
+                        raw_data = json.loads(raw_fragment)
+                        # sometimes youtube replies with a list
+                        if not isinstance(raw_data, list):
+                            raw_data = [raw_data]
+                        try:
+                            data = next(item['response'] for item in raw_data if 'response' in item)
+                        except StopIteration:
+                            data = {}

                    live_chat_continuation = try_get(
                        data,
--- a/youtube_dlc/extractor/common.py
+++ b/youtube_dlc/extractor/common.py
@@ -975,7 +975,7 @@ def playlist_result(entries, playlist_id=None, playlist_title=None, playlist_des
            video_info['id'] = playlist_id
        if playlist_title:
            video_info['title'] = playlist_title
-        if playlist_description:
+        if playlist_description is not None:
            video_info['description'] = playlist_description
        return video_info

--- a/youtube_dlc/extractor/youtube.py
+++ b/youtube_dlc/extractor/youtube.py
@@ -1999,8 +1999,10 @@ def get_continuation(continuation, session_token, replies=False):
                    raise ExtractorError('Unexpected HTTP error code: %s' % response_code)

            first_continuation = True
+            chain_msg = ''
+            self.to_screen('Downloading comments')
            while continuations:
-                continuation, itct = continuations.pop()
+                continuation = continuations.pop()
                comment_response = get_continuation(continuation, xsrf_token)
                if not comment_response:
                    continue
@@ -2046,9 +2048,10 @@ def get_continuation(continuation, session_token, replies=False):
                            continue

                        if self._downloader.params.get('verbose', False):
-                            self.to_screen('[debug] Comments downloaded (chain %s) %s of ~%s' % (comment['commentId'], len(video_comments), expected_video_comment_count))
+                            chain_msg = ' (chain %s)' % comment['commentId']
+                        self.to_screen('Comments downloaded: %d of ~%d%s' % (len(video_comments), expected_video_comment_count, chain_msg))
                        reply_comment_meta = replies_data[1]['response']['continuationContents']['commentRepliesContinuation']
-                        for reply_meta in replies_data[1]['response']['continuationContents']['commentRepliesContinuation']['contents']:
+                        for reply_meta in reply_comment_meta.get('contents', {}):
                            reply_comment = reply_meta['commentRenderer']
                            video_comments.append({
                                'id': reply_comment['commentId'],
@@ -2063,12 +2066,12 @@ def get_continuation(continuation, session_token, replies=False):
                            continue
                        reply_continuations += [rcn['nextContinuationData']['continuation'] for rcn in reply_comment_meta['continuations']]

-                self.to_screen('Comments downloaded %s of ~%s' % (len(video_comments), expected_video_comment_count))
+                self.to_screen('Comments downloaded: %d of ~%d' % (len(video_comments), expected_video_comment_count))
                if 'continuations' in item_section:
                    continuations += [ncd['nextContinuationData']['continuation'] for ncd in item_section['continuations']]
                time.sleep(1)

-            self.to_screen('Total comments downloaded %s of ~%s' % (len(video_comments), expected_video_comment_count))
+            self.to_screen('Total comments downloaded: %d of ~%d' % (len(video_comments), expected_video_comment_count))
            info.update({
                'comments': video_comments,
                'comment_count': expected_video_comment_count
@@ -2108,6 +2111,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCqj7Cz7revf5maW9g5pgNcg',
            'title': 'Игорь Клейнер - Playlists',
            'description': 'md5:be97ee0f14ee314f1f002cf187166ee2',
+            'uploader': 'Игорь Клейнер',
+            'uploader_id': 'UCqj7Cz7revf5maW9g5pgNcg',
        },
    }, {
        # playlists, multipage, different order
@@ -2117,6 +2122,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCqj7Cz7revf5maW9g5pgNcg',
            'title': 'Игорь Клейнер - Playlists',
            'description': 'md5:be97ee0f14ee314f1f002cf187166ee2',
+            'uploader_id': 'UCqj7Cz7revf5maW9g5pgNcg',
+            'uploader': 'Игорь Клейнер',
        },
    }, {
        # playlists, singlepage
@@ -2126,6 +2133,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCAEtajcuhQ6an9WEzY9LEMQ',
            'title': 'ThirstForScience - Playlists',
            'description': 'md5:609399d937ea957b0f53cbffb747a14c',
+            'uploader': 'ThirstForScience',
+            'uploader_id': 'UCAEtajcuhQ6an9WEzY9LEMQ',
        }
    }, {
        'url': 'https://www.youtube.com/c/ChristophLaimer/playlists',
@@ -2157,6 +2166,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Home',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
        'playlist_mincount': 2,
    }, {
@@ -2166,6 +2177,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Videos',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
        'playlist_mincount': 975,
    }, {
@@ -2175,6 +2188,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Videos',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
        'playlist_mincount': 199,
    }, {
@@ -2184,6 +2199,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Playlists',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
        'playlist_mincount': 17,
    }, {
@@ -2193,6 +2210,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Community',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
        'playlist_mincount': 18,
    }, {
@@ -2202,8 +2221,10 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
            'title': 'lex will - Channels',
            'description': 'md5:2163c5d0ff54ed5f598d6a7e6211e488',
+            'uploader': 'lex will',
+            'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
        },
-        'playlist_mincount': 138,
+        'playlist_mincount': 12,
    }, {
        'url': 'https://invidio.us/channel/UCmlqkdCBesrv2Lak1mF_MxA',
        'only_matching': True,
@@ -2221,6 +2242,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC',
            'uploader': 'Christiaan008',
            'uploader_id': 'UCEPzS1rYsrkqzSLNp76nrcg',
+            'description': 'md5:a14dc1a8ef8307a9807fe136a0660268',
        },
        'playlist_count': 96,
    }, {
@@ -2255,6 +2277,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'id': 'PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
            'uploader_id': 'UC9-y-6csu5WGm29I7JiwpnA',
            'uploader': 'Computerphile',
+            'description': 'md5:7f567c574d13d3f8c0954d9ffee4e487',
        },
        'playlist_mincount': 11,
    }, {
@@ -2295,12 +2318,12 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
        'info_dict': {
            'id': '9Auq9mYxFEE',
            'ext': 'mp4',
-            'title': 'Watch Sky News live',
+            'title': compat_str,
            'uploader': 'Sky News',
            'uploader_id': 'skynews',
            'uploader_url': r're:https?://(?:www\.)?youtube\.com/user/skynews',
            'upload_date': '20191102',
-            'description': 'md5:78de4e1c2359d0ea3ed829678e38b662',
+            'description': 'md5:85ddd75d888674631aaf9599a9a0b0ae',
            'categories': ['News & Politics'],
            'tags': list,
            'like_count': int,
@@ -2574,7 +2597,7 @@ def _extract_continuation(cls, renderer):
        next_continuation = cls._extract_next_continuation_data(renderer)
        if next_continuation:
            return next_continuation
-        contents = renderer.get('contents')
+        contents = renderer.get('contents') or renderer.get('items')
        if not isinstance(contents, list):
            return
        for content in contents:
@@ -2611,35 +2634,22 @@ def extract_entries(parent_renderer):  # this needs to called again for continua
                for isr_content in isr_contents:
                    if not isinstance(isr_content, dict):
                        continue
-                    renderer = isr_content.get('playlistVideoListRenderer')
-                    if renderer:
-                        for entry in self._playlist_entries(renderer):
-                            yield entry
+
+                    known_renderers = {
+                        'playlistVideoListRenderer': self._playlist_entries,
+                        'gridRenderer': self._grid_entries,
+                        'shelfRenderer': lambda x: self._shelf_entries(x, tab.get('title') != 'Channels'),
+                        'backstagePostThreadRenderer': self._post_thread_entries,
+                        'videoRenderer': lambda x: [self._video_entry(x)],
+                    }
+                    for key, renderer in isr_content.items():
+                        if key not in known_renderers:
+                            continue
+                        for entry in known_renderers[key](renderer):
+                            if entry:
+                                yield entry
                        continuation_list[0] = self._extract_continuation(renderer)
-                        continue
-                    renderer = isr_content.get('gridRenderer')
-                    if renderer:
-                        for entry in self._grid_entries(renderer):
-                            yield entry
-                        continuation_list[0] = self._extract_continuation(renderer)
-                        continue
-                    renderer = isr_content.get('shelfRenderer')
-                    if renderer:
-                        is_channels_tab = tab.get('title') == 'Channels'
-                        for entry in self._shelf_entries(renderer, not is_channels_tab):
-                            yield entry
-                        continue
-                    renderer = isr_content.get('backstagePostThreadRenderer')
-                    if renderer:
-                        for entry in self._post_thread_entries(renderer):
-                            yield entry
-                        continuation_list[0] = self._extract_continuation(renderer)
-                        continue
-                    renderer = isr_content.get('videoRenderer')
-                    if renderer:
-                        entry = self._video_entry(renderer)
-                        if entry:
-                            yield entry
+                        break

                if not continuation_list[0]:
                    continuation_list[0] = self._extract_continuation(is_renderer)
@@ -2692,48 +2702,47 @@ def extract_entries(parent_renderer):  # this needs to called again for continua
            if not response:
                break

+            known_continuation_renderers = {
+                'playlistVideoListContinuation': self._playlist_entries,
+                'gridContinuation': self._grid_entries,
+                'itemSectionContinuation': self._post_thread_continuation_entries,
+                'sectionListContinuation': extract_entries,  # for feeds
+            }
            continuation_contents = try_get(
-                response, lambda x: x['continuationContents'], dict)
-            if continuation_contents:
-                continuation_renderer = continuation_contents.get('playlistVideoListContinuation')
-                if continuation_renderer:
-                    for entry in self._playlist_entries(continuation_renderer):
-                        yield entry
-                    continuation = self._extract_continuation(continuation_renderer)
-                    continue
-                continuation_renderer = continuation_contents.get('gridContinuation')
-                if continuation_renderer:
-                    for entry in self._grid_entries(continuation_renderer):
-                        yield entry
-                    continuation = self._extract_continuation(continuation_renderer)
-                    continue
-                continuation_renderer = continuation_contents.get('itemSectionContinuation')
-                if continuation_renderer:
-                    for entry in self._post_thread_continuation_entries(continuation_renderer):
-                        yield entry
-                    continuation = self._extract_continuation(continuation_renderer)
-                    continue
-                continuation_renderer = continuation_contents.get('sectionListContinuation')  # for feeds
-                if continuation_renderer:
-                    continuation_list = [None]
-                    for entry in extract_entries(continuation_renderer):
-                        yield entry
-                    continuation = continuation_list[0]
+                response, lambda x: x['continuationContents'], dict) or {}
+            continuation_renderer = None
+            for key, value in continuation_contents.items():
+                if key not in known_continuation_renderers:
                    continue
+                continuation_renderer = value
+                continuation_list = [None]
+                for entry in known_continuation_renderers[key](continuation_renderer):
+                    yield entry
+                continuation = continuation_list[0] or self._extract_continuation(continuation_renderer)
+                break
+            if continuation_renderer:
+                continue

+            known_renderers = {
+                'gridPlaylistRenderer': (self._grid_entries, 'items'),
+                'gridVideoRenderer': (self._grid_entries, 'items'),
+                'playlistVideoRenderer': (self._playlist_entries, 'contents'),
+                'itemSectionRenderer': (self._playlist_entries, 'contents'),
+            }
            continuation_items = try_get(
                response, lambda x: x['onResponseReceivedActions'][0]['appendContinuationItemsAction']['continuationItems'], list)
-            if continuation_items:
-                continuation_item = continuation_items[0]
-                if not isinstance(continuation_item, dict):
-                    continue
-                renderer = continuation_item.get('playlistVideoRenderer') or continuation_item.get('itemSectionRenderer')
-                if renderer:
-                    video_list_renderer = {'contents': continuation_items}
-                    for entry in self._playlist_entries(video_list_renderer):
-                        yield entry
-                    continuation = self._extract_continuation(video_list_renderer)
+            continuation_item = try_get(continuation_items, lambda x: x[0], dict) or {}
+            video_items_renderer = None
+            for key, value in continuation_item.items():
+                if key not in known_renderers:
                    continue
+                video_items_renderer = {known_renderers[key][1]: continuation_items}
+                for entry in known_renderers[key][0](video_items_renderer):
+                    yield entry
+                continuation = self._extract_continuation(video_items_renderer)
+                break
+            if video_items_renderer:
+                continue
            break

    @staticmethod
@@ -2784,12 +2793,15 @@ def _extract_from_tabs(self, item_id, webpage, data, tabs, identity_token):
                data, lambda x: x['metadata']['playlistMetadataRenderer'], dict)
        if renderer:
            title = renderer.get('title')
-            description = renderer.get('description')
+            description = renderer.get('description', '')
            playlist_id = channel_id
            tags = renderer.get('keywords', '').split()
            thumbnails_list = (
                try_get(renderer, lambda x: x['avatar']['thumbnails'], list)
-                or data['sidebar']['playlistSidebarRenderer']['items'][0]['playlistSidebarPrimaryInfoRenderer']['thumbnailRenderer']['playlistVideoThumbnailRenderer']['thumbnail']['thumbnails']
+                or try_get(
+                    data,
+                    lambda x: x['sidebar']['playlistSidebarRenderer']['items'][0]['playlistSidebarPrimaryInfoRenderer']['thumbnailRenderer']['playlistVideoThumbnailRenderer']['thumbnail']['thumbnails'],
+                    list)
                or [])

        thumbnails = []
@@ -3089,7 +3101,7 @@ def _real_extract(self, url):


 class YoutubeSearchIE(SearchInfoExtractor, YoutubeBaseInfoExtractor):
-    IE_DESC = 'YouTube.com searches'
+    IE_DESC = 'YouTube.com searches, "ytsearch" keyword'
    # there doesn't appear to be a real limit, for example if you search for
    # 'python' you get more than 8.000.000 results
    _MAX_RESULTS = float('inf')
@@ -3178,7 +3190,7 @@ class YoutubeSearchDateIE(YoutubeSearchIE):


 class YoutubeSearchURLIE(YoutubeSearchIE):
-    IE_DESC = 'YouTube.com searches, "ytsearch" keyword'
+    IE_DESC = 'YouTube.com search URLs'
    IE_NAME = YoutubeSearchIE.IE_NAME + '_url'
    _VALID_URL = r'https?://(?:www\.)?youtube\.com/results\?(.*?&)?(?:search_query|q)=(?:[^&]+)(?:[&]|$)'
    # _MAX_RESULTS = 100
--- a/youtube_dlc/options.py
+++ b/youtube_dlc/options.py
@@ -18,14 +18,12 @@
    get_executable_path,
    OUTTMPL_TYPES,
    preferredencoding,
+    REMUX_EXTENSIONS,
    write_string,
 )
 from .version import __version__


-_remux_formats = ('mp4', 'mkv', 'flv', 'webm', 'mov', 'avi', 'mp3', 'mka', 'm4a', 'ogg', 'opus')
-
-
 def _hide_login_info(opts):
    PRIVATE_OPTS = set(['-p', '--password', '-u', '--username', '--video-password', '--ap-password', '--ap-username'])
    eqre = re.compile('^(?P<key>' + ('|'.join(re.escape(po) for po in PRIVATE_OPTS)) + ')=.+$')
@@ -1042,7 +1040,7 @@ def _dict_from_multiple_values_options_callback(
            'Remux the video into another container if necessary (currently supported: %s). '
            'If target container does not support the video/audio codec, remuxing will fail. '
            'You can specify multiple rules; eg. "aac>m4a/mov>mp4/mkv" will remux aac to m4a, mov to mp4 '
-            'and anything else to mkv.' % '|'.join(_remux_formats)))
+            'and anything else to mkv.' % '|'.join(REMUX_EXTENSIONS)))
    postproc.add_option(
        '--recode-video',
        metavar='FORMAT', dest='recodevideo', default=None,
--- a/youtube_dlc/postprocessor/embedthumbnail.py
+++ b/youtube_dlc/postprocessor/embedthumbnail.py
@@ -10,9 +10,9 @@

 try:
    import mutagen
-    _has_mutagen = True
+    has_mutagen = True
 except ImportError:
-    _has_mutagen = False
+    has_mutagen = False

 from .ffmpeg import FFmpegPostProcessor

@@ -42,13 +42,12 @@ def __init__(self, downloader=None, already_have_thumbnail=False):
    def run(self, info):
        filename = info['filepath']
        temp_filename = prepend_extension(filename, 'temp')
-        files_to_delete = []

        if not info.get('thumbnails'):
            self.to_screen('There aren\'t any thumbnails to embed')
            return [], info

-        thumbnail_filename = info['thumbnails'][-1]['filename']
+        original_thumbnail = thumbnail_filename = info['thumbnails'][-1]['filename']

        if not os.path.exists(encodeFilename(thumbnail_filename)):
            self.report_warning('Skipping embedding the thumbnail because the file is missing.')
@@ -67,7 +66,7 @@ def is_webp(path):
                self.to_screen('Correcting extension to webp and escaping path for thumbnail "%s"' % thumbnail_filename)
                thumbnail_webp_filename = replace_extension(thumbnail_filename, 'webp')
                os.rename(encodeFilename(thumbnail_filename), encodeFilename(thumbnail_webp_filename))
-                thumbnail_filename = thumbnail_webp_filename
+                original_thumbnail = thumbnail_filename = thumbnail_webp_filename
                thumbnail_ext = 'webp'

        # Convert unsupported thumbnail formats to JPEG (see #25687, #25717)
@@ -79,9 +78,9 @@ def is_webp(path):
            escaped_thumbnail_jpg_filename = replace_extension(escaped_thumbnail_filename, 'jpg')
            self.to_screen('Converting thumbnail "%s" to JPEG' % escaped_thumbnail_filename)
            self.run_ffmpeg(escaped_thumbnail_filename, escaped_thumbnail_jpg_filename, ['-bsf:v', 'mjpeg2jpeg'])
-            files_to_delete.append(escaped_thumbnail_filename)
            thumbnail_jpg_filename = replace_extension(thumbnail_filename, 'jpg')
            # Rename back to unescaped for further processing
+            os.rename(encodeFilename(escaped_thumbnail_filename), encodeFilename(thumbnail_filename))
            os.rename(encodeFilename(escaped_thumbnail_jpg_filename), encodeFilename(thumbnail_jpg_filename))
            thumbnail_filename = thumbnail_jpg_filename
            thumbnail_ext = 'jpg'
@@ -153,7 +152,7 @@ def is_webp(path):
                    success = False

        elif info['ext'] in ['ogg', 'opus']:
-            if not _has_mutagen:
+            if not has_mutagen:
                raise EmbedThumbnailPPError('module mutagen was not found. Please install using `python -m pip install mutagen`')
            self.to_screen('Adding thumbnail to "%s"' % filename)

@@ -184,9 +183,11 @@ def is_webp(path):
        if success and temp_filename != filename:
            os.remove(encodeFilename(filename))
            os.rename(encodeFilename(temp_filename), encodeFilename(filename))
+
+        files_to_delete = [thumbnail_filename]
        if self._already_have_thumbnail:
-            info['__files_to_move'][thumbnail_filename] = replace_extension(
-                info['__thumbnail_filename'], os.path.splitext(thumbnail_filename)[1][1:])
-        else:
-            files_to_delete.append(thumbnail_filename)
+            info['__files_to_move'][original_thumbnail] = replace_extension(
+                info['__thumbnail_filename'], os.path.splitext(original_thumbnail)[1][1:])
+            if original_thumbnail == thumbnail_filename:
+                files_to_delete = []
        return files_to_delete, info
--- a/youtube_dlc/postprocessor/ffmpeg.py
+++ b/youtube_dlc/postprocessor/ffmpeg.py
@@ -442,6 +442,10 @@ def run(self, information):


 class FFmpegEmbedSubtitlePP(FFmpegPostProcessor):
+    def __init__(self, downloader=None, already_have_subtitle=False):
+        super(FFmpegEmbedSubtitlePP, self).__init__(downloader)
+        self._already_have_subtitle = already_have_subtitle
+
    def run(self, information):
        if information['ext'] not in ('mp4', 'webm', 'mkv'):
            self.to_screen('Subtitles can only be embedded in mp4, webm or mkv files')
@@ -501,7 +505,8 @@ def run(self, information):
        os.remove(encodeFilename(filename))
        os.rename(encodeFilename(temp_filename), encodeFilename(filename))

-        return sub_filenames, information
+        files_to_delete = [] if self._already_have_subtitle else sub_filenames
+        return files_to_delete, information


 class FFmpegMetadataPP(FFmpegPostProcessor):
--- a/youtube_dlc/postprocessor/movefilesafterdownload.py
+++ b/youtube_dlc/postprocessor/movefilesafterdownload.py
@@ -4,11 +4,11 @@

 from .common import PostProcessor
 from ..utils import (
+    decodeFilename,
    encodeFilename,
    make_dir,
    PostProcessingError,
 )
-from ..compat import compat_str


 class MoveFilesAfterDownloadPP(PostProcessor):
@@ -26,12 +26,12 @@ def run(self, info):
        finaldir = info.get('__finaldir', dl_path)
        finalpath = os.path.join(finaldir, dl_name)
        self.files_to_move.update(info['__files_to_move'])
-        self.files_to_move[info['filepath']] = finalpath
+        self.files_to_move[info['filepath']] = decodeFilename(finalpath)

+        make_newfilename = lambda old: decodeFilename(os.path.join(finaldir, os.path.basename(encodeFilename(old))))
        for oldfile, newfile in self.files_to_move.items():
            if not newfile:
-                newfile = os.path.join(finaldir, os.path.basename(encodeFilename(oldfile)))
-            oldfile, newfile = compat_str(oldfile), compat_str(newfile)
+                newfile = make_newfilename(oldfile)
            if os.path.abspath(encodeFilename(oldfile)) == os.path.abspath(encodeFilename(newfile)):
                continue
            if not os.path.exists(encodeFilename(oldfile)):
@@ -50,5 +50,5 @@ def run(self, info):
            self.to_screen('Moving file "%s" to "%s"' % (oldfile, newfile))
            shutil.move(oldfile, newfile)  # os.rename cannot move between volumes

-        info['filepath'] = compat_str(finalpath)
+        info['filepath'] = finalpath
        return [], info
--- a/youtube_dlc/postprocessor/sponskrub.py
+++ b/youtube_dlc/postprocessor/sponskrub.py
@@ -43,6 +43,10 @@ def run(self, information):
        if self.path is None:
            return [], information

+        filename = information['filepath']
+        if not os.path.exists(encodeFilename(filename)):  # no download
+            return [], information
+
        if information['extractor_key'].lower() != 'youtube':
            self.to_screen('Skipping sponskrub since it is not a YouTube video')
            return [], information
@@ -58,7 +62,6 @@ def run(self, information):
            if not information.get('__real_download', False):
                self.report_warning('If sponskrub is run multiple times, unintended parts of the video could be cut out.')

-        filename = information['filepath']
        temp_filename = prepend_extension(filename, self._temp_ext)
        if os.path.exists(encodeFilename(temp_filename)):
            os.remove(encodeFilename(temp_filename))
--- a/youtube_dlc/update.py
+++ b/youtube_dlc/update.py
@@ -15,6 +15,7 @@
 from .version import __version__


+'''  # Not signed
 def rsa_verify(message, signature, key):
    from hashlib import sha256
    assert isinstance(message, bytes)
@@ -27,17 +28,13 @@ def rsa_verify(message, signature, key):
        return False
    expected = b'0001' + (byte_size - len(asn1) // 2 - 3) * b'ff' + b'00' + asn1
    return expected == signature
+'''


 def update_self(to_screen, verbose, opener):
    """Update the program file with the latest version from the repository"""

-    return to_screen('Update is currently broken.\nVisit  https://github.com/pukkandan/yt-dlp/releases/latest  to get the latest version')
-
-    UPDATE_URL = 'https://blackjack4494.github.io//update/'
-    VERSION_URL = UPDATE_URL + 'LATEST_VERSION'
-    JSON_URL = UPDATE_URL + 'versions.json'
-    UPDATES_RSA_KEY = (0x9d60ee4d8f805312fdb15a62f87b95bd66177b91df176765d13514a0f1754bcd2057295c5b6f1d35daa6742c3ffc9a82d3e118861c207995a8031e151d863c9927e304576bc80692bc8e094896fcf11b66f3e29e04e3a71e9a11558558acea1840aec37fc396fb6b65dc81a1c4144e03bd1c011de62e3f1357b327d08426fe93, 65537)
+    JSON_URL = 'https://api.github.com/repos/pukkandan/yt-dlp/releases/latest'

    def sha256sum():
        h = hashlib.sha256()
@@ -54,55 +51,36 @@ def sha256sum():
        to_screen('It looks like you installed youtube-dlc with a package manager, pip, setup.py or a tarball. Please use that to update.')
        return

-    # compiled file.exe can find itself by
-    # to_screen(os.path.basename(sys.executable))
-    # and path to py or exe
-    # to_screen(os.path.realpath(sys.executable))
-
-    # Check if there is a new version
-    try:
-        newversion = opener.open(VERSION_URL).read().decode('utf-8').strip()
-    except Exception:
-        if verbose:
-            to_screen(encode_compat_str(traceback.format_exc()))
-        to_screen('ERROR: can\'t find the current version. Please try again later.')
-        to_screen('Visit https://github.com/blackjack4494/yt-dlc/releases/latest')
-        return
-    if newversion == __version__:
-        to_screen('youtube-dlc is up-to-date (' + __version__ + ')')
-        return
-
    # Download and check versions info
    try:
-        versions_info = opener.open(JSON_URL).read().decode('utf-8')
-        versions_info = json.loads(versions_info)
+        version_info = opener.open(JSON_URL).read().decode('utf-8')
+        version_info = json.loads(version_info)
    except Exception:
        if verbose:
            to_screen(encode_compat_str(traceback.format_exc()))
        to_screen('ERROR: can\'t obtain versions info. Please try again later.')
-        to_screen('Visit https://github.com/blackjack4494/yt-dlc/releases/latest')
-        return
-    if 'signature' not in versions_info:
-        to_screen('ERROR: the versions file is not signed or corrupted. Aborting.')
-        return
-    signature = versions_info['signature']
-    del versions_info['signature']
-    if not rsa_verify(json.dumps(versions_info, sort_keys=True).encode('utf-8'), signature, UPDATES_RSA_KEY):
-        to_screen('ERROR: the versions file signature is invalid. Aborting.')
+        to_screen('Visit https://github.com/pukkandan/yt-dlp/releases/lastest')
        return

-    version_id = versions_info['latest']
+    version_id = version_info['tag_name']
+    if version_id == __version__:
+        to_screen('youtube-dlc is up-to-date (' + __version__ + ')')
+        return

    def version_tuple(version_str):
        return tuple(map(int, version_str.split('.')))
+
    if version_tuple(__version__) >= version_tuple(version_id):
        to_screen('youtube-dlc is up to date (%s)' % __version__)
        return

    to_screen('Updating to version ' + version_id + ' ...')
-    version = versions_info['versions'][version_id]

-    print_notes(to_screen, versions_info['versions'])
+    version = {
+        'bin': next(i for i in version_info['assets'] if i['name'] == 'youtube-dlc'),
+        'exe': next(i for i in version_info['assets'] if i['name'] == 'youtube-dlc.exe'),
+        'exe_x86': next(i for i in version_info['assets'] if i['name'] == 'youtube-dlc_x86.exe'),
+    }

    # sys.executable is set to the full pathname of the exe-file for py2exe
    # though symlinks are not followed so that we need to do this manually
@@ -113,7 +91,7 @@ def version_tuple(version_str):
        to_screen('ERROR: no write permissions on %s' % filename)
        return

-    # Py2EXE
+    # PyInstaller
    if hasattr(sys, 'frozen'):
        exe = filename
        directory = os.path.dirname(exe)
@@ -122,19 +100,14 @@ def version_tuple(version_str):
            return

        try:
-            urlh = opener.open(version['exe'][0])
+            urlh = opener.open(version['exe']['browser_download_url'])
            newcontent = urlh.read()
            urlh.close()
        except (IOError, OSError):
            if verbose:
                to_screen(encode_compat_str(traceback.format_exc()))
            to_screen('ERROR: unable to download latest version')
-            to_screen('Visit https://github.com/blackjack4494/yt-dlc/releases/latest')
-            return
-
-        newcontent_hash = hashlib.sha256(newcontent).hexdigest()
-        if newcontent_hash != version['exe'][1]:
-            to_screen('ERROR: the downloaded file hash does not match. Aborting.')
+            to_screen('Visit https://github.com/pukkandan/yt-dlp/releases/lastest')
            return

        try:
@@ -147,16 +120,17 @@ def version_tuple(version_str):
            return

        try:
-            bat = os.path.join(directory, 'youtube-dlc-updater.bat')
+            bat = os.path.join(directory, 'yt-dlp-updater.cmd')
            with io.open(bat, 'w') as batfile:
                batfile.write('''
-@echo off
-echo Waiting for file handle to be closed ...
-ping 127.0.0.1 -n 5 -w 1000 > NUL
-move /Y "%s.new" "%s" > NUL
-echo Updated youtube-dlc to version %s.
-start /b "" cmd /c del "%%~f0"&exit /b"
-                \n''' % (exe, exe, version_id))
+@(
+    echo.Waiting for file handle to be closed ...
+    ping 127.0.0.1 -n 5 -w 1000 > NUL
+    move /Y "%s.new" "%s" > NUL
+    echo.Updated youtube-dlc to version %s.
+)
+@start /b "" cmd /c del "%%~f0"&exit /b
+                ''' % (exe, exe, version_id))

            subprocess.Popen([bat])  # Continues to run in the background
            return  # Do not show premature success messages
@@ -169,19 +143,14 @@ def version_tuple(version_str):
    # Zip unix package
    elif isinstance(globals().get('__loader__'), zipimporter):
        try:
-            urlh = opener.open(version['bin'][0])
+            urlh = opener.open(version['bin']['browser_download_url'])
            newcontent = urlh.read()
            urlh.close()
        except (IOError, OSError):
            if verbose:
                to_screen(encode_compat_str(traceback.format_exc()))
            to_screen('ERROR: unable to download latest version')
-            to_screen('Visit https://github.com/blackjack4494/yt-dlc/releases/latest')
-            return
-
-        newcontent_hash = hashlib.sha256(newcontent).hexdigest()
-        if newcontent_hash != version['bin'][1]:
-            to_screen('ERROR: the downloaded file hash does not match. Aborting.')
+            to_screen('Visit https://github.com/pukkandan/yt-dlp/releases/lastest')
            return

        try:
@@ -196,6 +165,7 @@ def version_tuple(version_str):
    to_screen('Updated youtube-dlc. Restart youtube-dlc to use the new version.')


+'''  # UNUSED
 def get_notes(versions, fromVersion):
    notes = []
    for v, vdata in sorted(versions.items()):
@@ -210,3 +180,4 @@ def print_notes(to_screen, versions, fromVersion=__version__):
        to_screen('PLEASE NOTE:')
        for note in notes:
            to_screen(note)
+'''
--- a/youtube_dlc/utils.py
+++ b/youtube_dlc/utils.py
@@ -1715,6 +1715,8 @@ def random_user_agent():
    'wav',
    'f4f', 'f4m', 'm3u8', 'smil')

+REMUX_EXTENSIONS = ('mp4', 'mkv', 'flv', 'webm', 'mov', 'avi', 'mp3', 'mka', 'm4a', 'ogg', 'opus')
+
 # needed for sanitizing filenames in restricted mode
 ACCENT_CHARS = dict(zip('ÂÃÄÀÁÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖŐØŒÙÚÛÜŰÝÞßàáâãäåæçèéêëìíîïðñòóôõöőøœùúûüűýþÿ',
                        itertools.chain('AAAAAA', ['AE'], 'CEEEEIIIIDNOOOOOOO', ['OE'], 'UUUUUY', ['TH', 'ss'],
@@ -4690,9 +4692,7 @@ def cli_configuration_args(params, arg_name, key, default=[], exe=None):  # retu
        return default, False
    assert isinstance(argdict, dict)

-    assert isinstance(key, compat_str)
    key = key.lower()
-
    args = exe_args = None
    if exe is not None:
        assert isinstance(exe, compat_str)
--- a/youtube_dlc/version.py
+++ b/youtube_dlc/version.py
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2021.01.29'
+__version__ = '2021.02.04'
Author	SHA1	Message	Date
pukkandan	ba9f36d732	Release 2021.02.09	2021-02-10 01:26:56 +05:30
pukkandan	cffab0eefc	[embedsubtitle] Keep original subtitle after conversion if write_subtitles given Closes: https://github.com/pukkandan/yt-dlp/issues/57#issuecomment-775227745 :ci skip dl	2021-02-10 00:12:42 +05:30
pukkandan	2e339f59c3	[embedthumbnail] Keep original thumbnail after conversion if write_thumbnail given (Closes #67 ) Closes https://github.com/ytdl-org/youtube-dl/issues/27041 :ci skip dl	2021-02-09 23:18:20 +05:30
pukkandan	6c4fd172de	Add fallback for thumbnails Workaround for: https://github.com/ytdl-org/youtube-dl/issues/28023 Related: https://github.com/ytdl-org/youtube-dl/pull/28031 Also fixes https://www.reddit.com/r/youtubedl/comments/lfslw1/youtubedlp_with_aria2c_for_dash_support_is/gmolt0r?context=3	2021-02-09 23:12:41 +05:30
pukkandan	deaec5afc2	[youtube] Fix tests	2021-02-09 22:01:34 +05:30
pukkandan	69184e4152	[youtube] Simplified renderer parsing	2021-02-09 21:37:59 +05:30
pukkandan	a1b535bd75	[youtube] Support gridPlaylistRenderer and gridVideoRenderer (Closes #65 )	2021-02-09 20:40:37 +05:30
pukkandan	b3943b2f33	[pyinst.py] Move back to root dir (Closes #63 )	2021-02-09 18:04:27 +05:30
shirt-dev	3dd264bf42	#64 Implement self updater Co-authored-by: shirtjs <2660574+shirtjs@users.noreply.github.com> (shirt-dev) Co-authored-by: pukkandan <pukkandan@gmail.com>	2021-02-09 18:04:00 +05:30
pukkandan	efabc16165	[postprocessor] Fix bug (Closes #62 ) introduced by: `1bf540d28b` :ci skip dl	2021-02-09 00:27:39 +05:30
shirt-dev	5219cb3e75	#55 Add aria2c support for DASH (mpd) and HLS (m3u8) Co-authored-by: Dan <2660574+shirtjs@users.noreply.github.com> Co-authored-by: pukkandan <pukkandan@gmail.com>	2021-02-08 22:16:01 +05:30
pukkandan	ff84930c86	[youtube] Bugfix (Closes #60 )	2021-02-08 19:20:19 +05:30
pukkandan	06ff212d64	[documentation] Crypto is an optional dependency	2021-02-08 18:05:22 +05:30
pukkandan	1bf540d28b	[sponskrub] Don't raise error when the video does not exist Eg: `--convert-sub srt --no-download --sponskrub` gave error before :ci skip dl	2021-02-08 15:48:12 +05:30
pukkandan	df692c5a7a	[remuxvideo] Fix validation of conditional remux	2021-02-08 15:29:02 +05:30
pukkandan	ecc97af344	[youtube] Don't show warning for empty playlist description (Closes #54 ) :ci skip dl	2021-02-07 20:15:02 +05:30
pukkandan	8a0b932258	[movefiles] Fix compatibility with python2 :ci skip dl	2021-02-07 17:41:41 +05:30
pukkandan	4d608b522f	[youtube_live_chat] Improve extraction :ci skip dl	2021-02-07 15:22:36 +05:30
pukkandan	885d36d4e4	[youtube] Fix comment extraction (Closes #53 ) :ci skip dl	2021-02-05 16:47:44 +05:30
pukkandan	0fd1a2b0bf	[version] update (and linter)	2021-02-05 05:02:41 +05:30