# automated-youtube-dl

_Automated YouTube Archival._

A wrapper for youtube-dl used for keeping very large amounts of data from YouTube in sync. It's designed to be simple and easy to use.

I have a single, very large playlist that I add any videos I like to. This runs as a service on my NAS (see [Example systemd Service.md]).

---

## Project Status

This project is archived. I was working on a web interface for this project but decided to just use [tubearchivist](https://github.com/tubearchivist/tubearchivist) rather than write my own. If tubearchivist does not meet my needs then I will restart work on this project.

---


### Features

- Uses yt-dlp instead of youtube-dl.
- Skips videos that are already downloaded.
- Automatically update yt-dlp on launch.
- Download the videos in a format suitable for archiving:
    - Complex `format` that balances video quality and file size.
    - Embedding of metadata: chapters, thumbnail, english subtitles (automatic too), and YouTube metadata.
- Log progress to a file.
- Simple display using `tqdm`.
- Limit the size of the downloaded videos.
- Parallel downloads.
- Daemon mode for running as a system service.

### Installation

```bash
sudo apt update && sudo apt install ffmpeg atomicparsley phantomjs
pip install -r requirements.txt
```

### Usage

This program has 3 modes:

<br>

**Direct-Download Mode:**

In this mode, you give the downloader a URL to the media you want to download.

`./downloader.py <video URL to download> --output <output directory>`

<br>

**Config-File Mode:**

In this mode, you give the downloader the path to a config file that contains the URLs of the media and where to download them to.

The config file can be a YAML file or a TXT file with the URL to download on each line.

When using the YAML file (see [targets.sample.yml]): `./downloader.py <path to the config file>`

When using a TXT file: `./downloader.py <path to the config file> --output <output directory>`

<br>

**Daemon Mode:**

In this mode, the downloader will loop over the media you give it and sleep for a certain number of minutes. It takes

To run as a daemon, do:

`/usr/bin/python3 /home/user/automated-youtube-dl/downloader.py --daemon --sleep 60 <video URL or config file path>`

`--sleep` is how many minutes to sleep after completing all downloads.

Daemon mode can take a URL (like direct-download mode) or a path to a config file (like config-file mode).

<br>

#### Folder Structure

```
Output Directory/
├─ logs/
│  ├─ youtube_dl-<UNIX timestamp>.log
│  ├─ youtube_dl-errors-<UNIX timestamp>.log
├─ download-archive.log
├─ Example Video.mkv
```

`download-archive.log` contains the videos that have already been downloaded. You can import videos you've already downloaded by adding their ID to this file.

Videos will be saved using this name format:

```
[%(id)s] [%(title)s] [%(uploader)s] [%(uploader_id)s]
```

<br>

#### Arguments

| Argument              | Flag | Help                                                         |
| --------------------- | ---- | ------------------------------------------------------------ |
| `--no-update`         | `-n` | Don\'t update yt-dlp at launch.                              |
| `--max-size`          |      | Max allowed size of a video in MB. Default: 1100.            |
| `--rm-cache`          | `-r` | Delete the yt-dlp cache on start.                            |
| `--threads`           |      | How many download processes to use (threads). Default is how many CPU cores you have. You will want to find a good value that doesn't overload your connection. |
| `--daemon`            | `-d` | Run in daemon mode. Disables progress bars and sleeps for the amount of time specified in `--sleep`. |
| `--sleep`             |      | How many minutes to sleep when in daemon mode.               |
| `--silent`            | `-s` | Don't print any error messages to the console. Errors will still be logged in the log files.               |
| `--ignore-downloaded` | `-i` | Ignore videos that have been already downloaded and let youtube-dl handle everything. Videos will not be re-downloaded, but metadata will be updated. |