Repository navigation
feat(175): channels as tv shows - #915
Draft
dialmaster wants to merge 8 commits into
Draft
dialmaster wants to merge 8 commits into
dialmaster wants to merge 8 commits into
Conversation
First phase of saving channels as TV shows. Nothing changes for users yet: every library folder still reports the videos layout, so the TV code paths only run in tests. - Video deletion and the orphan sweep choose their cleanup by the library folder's layout. In a TV folder they remove an emptied season folder, then the show folder once only metadata and art remain, and skip a folder they cannot list. - The post-processor writes the channel poster and backdrop to the folder the move wrote to, instead of working it out from folder depth. - Add sanitizeFilenameLikeYtDlp, a port of yt-dlp's title sanitization that cuts to a byte limit between whole characters, with fixtures generated from the dev image's yt-dlp. - Add tvShows modules for library layouts, episode file names, date numbering (UTC MMDDHHMM, bumped on collision, adopting "Plex TV Series" preset names) and routing, plus a corpus of real channel titles for title-pattern shows. Refs: #175
Plex playlist sync could not find videos stored in a TV (show) library section, such as files named with the Plex TV Series preset, so those videos were left out of the synced playlists. The file lookup listed show sections without a type filter, which appears to return only show-level items with no file paths. Watch status sync already lists show sections with type=4 (episodes). Request type=4 for show sections when resolving playlist files too. An episode listing covers every show on the server and is fetched again on each resolve round, so show sections are only listed when a file in the batch is named like an episode (SxxEyy). Batches of movie-style files skip them, which keeps a large unrelated TV library from being listed episode by episode. Refs: #175
Each library folder (the main downloads folder and every __subfolder) now has a layout, videos or tv. A channel that downloads to a TV folder is saved as one show, <show>/Season <year>/SxxEyy - Title [id].ext, numbered by upload time, with an episode NFO and a tvshow.nfo, so a Plex, Jellyfin or Emby TV library can index it. - Store subfolder layouts on the subfolders table and the main folder's in config (mainFolderLayout); store shows and episode numbers in the new tv_shows and video_classifications tables. A show's folder is pinned when it is created, so an uploader rename or a default-folder change does not start a second show. - Guard layout changes: a folder or channel that already holds downloaded videos cannot switch between videos and TV, nothing switches while a download runs, and TV folders are video-only. Saved MP3 settings are refused; a download-time MP3 type is downgraded to video for videos whose predicted folder is TV, and an MP3 that still reaches a TV folder is placed as the episode, with a warning, rather than movie-style. - Add GET/PUT /api/library-folders, GET /api/channels/:id/tv and PUT /api/channels/:id/tv/layout, a layout filter on /getchannels, and episode details on the video listings and video metadata. - Settings gains a library folder list, Channel Settings a TV Show section that switches a channel (creating a TV folder when none exists), and the subscription, channel and video pages show TV and episode chips. - Replace episode files through a .part staging file and an atomic rename. fs-extra's overwrite removes the destination before a cross-device copy, so a copy that failed on a full disk could have left no copy of the episode at all. Refs: #175
Switching a channel, a library folder or the default subfolder between Videos and TV no longer refuses when downloads exist. The change opens a dry run of every file move (paths, episode numbers, new shows, missing files, collisions, watch state at risk), and confirming it moves the files in the background, so existing downloads become episodes, or movie-style videos again, without downloading them again. - Record each reorganize in tv_reorganize_operations and tv_reorganize_items, so a restart resumes it and failed videos can be retried. A revision token over the planned moves refuses a stale preview. The settings change applies when the move starts and is undone only if no video could be moved. - Hold download jobs in the queue while files move, and refuse the rescan, library repair, auto-removal, watch status sync and yt-dlp update, scheduled runs included. Deletes and channel settings saves of the videos and channels being moved get a 409. - Move files without replacing anything at the destination, through a staging copy when it is on another filesystem, and move earlier files back when a later one fails. NFOs, art, tvshow.nfo, the paths stored in jobs/info, channel and playlist m3u files and media server playlists follow the moved files. - Keep Youtarr's watch state while media servers show the moved files as new, unwatched items: watch_status_holds snapshots it before the move, watch status sync does not lower a held row, and the state is pushed back to the Plex owner account, Jellyfin and Emby once they index the new path. Restores that don't land within 14 days are listed under Watch Status with Retry and Dismiss. - Name videos leaving a TV folder the way a new download would be named, rendered in batches by yt-dlp from their stored info.json. - Add POST /api/tv/reorganize/preview, POST /api/tv/reorganize and the /api/tv/operations and /api/tv/holds endpoints. The library folder, channel layout, channel settings and default subfolder routes answer a reorganizeRequired 409 that names the change to preview. - Add the review and progress dialog to Channel Settings, the library folder list and Settings, and a banner on every page while a reorganize runs. Refs: #175
Setting up a TV folder meant guessing whether the media server library for it was right. Settings and Channel Settings now show, per library folder and per configured server, which libraries hold the folder and what in their setup works against Youtarr's files: no TV library yet, a library of the wrong type, the Plex Series or a legacy agent, a Jellyfin or Emby library that saves NFO files or looks items up online, another library showing the same files again, a library nested inside another on Jellyfin or Emby (which leaves it empty), a mount under another name, or a missing Plex refresh mapping. Server paths differ from the container's, so folders are matched by a location ending in __name and by sampling a few files from each library and pairing them with Youtarr's downloads by YouTube id. - Add GET /api/library-folders/check (optionally for given folders, checked as a given layout) and PUT /api/library-folders/plex-mapping. Channel Settings adds the Plex refresh mapping itself when exactly one Plex TV library holds the channel's folder; the folder list offers it as a button. The saved mapping reaches an open Settings page without discarding its unsaved edits, including pending edits to other mappings. - Show the check in the move review for the TV folders videos move into, evaluated as TV folders even while they are still saved as Videos folders, and only for the move being previewed. - List only the libraries that may hold Youtarr's files when Plex watch sync and playlist sync and Jellyfin/Emby push-back read a server, so an unrelated TV library is no longer listed episode by episode. A library whose sample says nothing (empty, unreadable, or YouTube downloads Youtarr can't place) stays in scope, as do the configured and mapped Plex sections. - Re-sync media server playlists 5 and 15 minutes after a reorganize: a move that keeps file names first resolved to the item at the old path. Remove Youtarr's tvshow.nfo from a channel folder that is movie-style again; a Jellyfin Mixed library kept showing the channel as a series. - Recheck a move review blocked by a running task every 5 seconds, after a failed recheck too, so Move enables without reopening the dialog. Word the default-subfolder confirmation for a layout change, which moves the channels' videos. - Document the TV library settings for Plex, Jellyfin and Emby with the servers' own labels, one library per folder on Jellyfin and Emby, and advise against Mixed libraries. Refs: #175
A channel that uploads several series (Hermitcraft seasons, one
cartoon in four languages) could only be saved as a single show
numbered by upload date. Title shows are series inside a channel,
defined by title patterns and numbered from the titles: season and
episode taken from the title, a fixed season, the next number in the
season, or the upload year. Each show has its own folder in a TV
folder; videos no show takes follow the channel's layout.
- Edit shows in Channel Settings > TV Show: patterns in a simple
syntax ({season}, {episode}, {title}, *) or as Python regular
expressions, matched by Python's re as yt-dlp does, exclude terms,
season names and the TV folder, with a live preview of episodes,
duplicates, gaps, unmatched titles and compilations or parts not
supported yet.
- When two uploads claim one number, the stored holder keeps it, else
the earliest available upload. A duplicate that isn't downloaded is
ignored and its complete.list line written, deferred while a
download runs; episode_conflicts records what Youtarr did, so a
release undoes only that. A downloaded duplicate stays where it is.
"Use this copy instead", a manual episode and "Not an episode"
override the classification.
- A save that changes a downloaded episode's show, number or file
name opens the move review and runs as a reorganize, which restores
the previous shows and rows when no video moves. Channel and folder
layout switches leave title show episodes in place, and a folder
holding title shows can't switch to Videos.
- Classify new downloads at finalize and new uploads on listing
refresh. An optional switch limits channel downloads and Download
All to show episodes, with one --match-filter per show.
- Write a youtarr uniqueid and <namedseason> to tvshow.nfo, and
season.nfo to named season folders.
- Add a show filter, a Missing episodes view and planned episode
chips to the channel page, and an "N shows" chip to Subscriptions
and the channel header.
- Show a file sitting where a destination folder belongs as a
collision in the move review instead of failing the preview, and
say when a video's files moved but finishing it failed.
- Add tv_show_patterns, tv_show_seasons, episode_conflicts and
channels.tv_show_only_downloads; the routes under
/api/channels/:channelId/tv/shows, preview, show-only, conflicts
and recheck; and GET/PUT /api/videos/:youtubeId/episode. Document
title shows in the usage and folder structure guides.
Refs: #175
Contributor
📊 Test Coverage ReportBackend Coverage
Frontend Coverage
Coverage Requirements
Coverage report generated for commit 23fd4ac |
Follow-up fixes from the local checks of title shows and the reorganize: - The per-page file check showed videos as missing when their file had been renamed in place (for example to a TV episode name) but kept its [id]. It now also looks in the stored folder for a media file carrying the video's id, skips macOS AppleDouble copies, and lists each folder once per batch. - Gaps and Missing episodes listed long runs of missing numbers for seasons numbered by upload year. Those seasons are skipped, as date-numbered seasons already were. - A show's duplicate count comes from its recorded conflicts, so a TV channel's episode that lost a title claim, and kept its channel-show row, is counted. - The reorganize preview notes MP3 files moving from a Videos folder into a TV folder, where TV libraries don't show them. - After a move applies a default subfolder change, Settings asks to save again only when the form still has other unsaved changes. Refs: #175
Follow-up fixes from the review of title shows and the reorganize: - Renaming a show folder only in case or accents saved without moving its episodes, so later downloads could land in a second folder. Such renames now move the files. On case-insensitive filesystems the move no longer reports the source as a collision or deletes the episode NFO it just wrote, and a destination that differs only in case or accents is never taken for a finished copy of the source. - A restart between creating a title show change's shows and recording the reorganize as applied could leave recovery and Retry refused on every attempt. Both are now written in one transaction. - Regex patterns with numbered backreferences or conditional groups could match differently in channel downloads than in classification. They are refused; three-digit octal escapes such as \123 still work. - A job started from an idle queue could begin while complete.list was being updated for duplicate episodes. Every job start now waits for that work, and a failed read of the pending updates no longer fails a save that has already committed. - Listing refreshes could record a new video as a duplicate of a video they left out, spend order numbers on rows they did not write, and clear classification errors of videos they did not check. - The subfolder delete guard did not count title shows of channels that download to a Videos folder. - Undoing a title show change dropped the pattern link of rows in retired shows. Drafts naming another channel's show, the same show twice, or a folder another show gives up in the same change are refused. - Moved title episodes lost their stored time source and were missing from the review's download-time and upload-day notes, and videos without a date were numbered before dated ones. - The reorganize API docs now list the titleShows change. Refs: #175
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work in progress: NOT ready for merge or testing yet :)