YouTube-Kanäle als Podcasts abonnieren
All checks were successful
Build and push Docker image / build (push) Successful in 15s
Test / test (push) Successful in 8m34s
Test / e2e (push) Successful in 1m52s

Eine YouTube-Adresse (Kanal, Playlist, Video, Feed) lässt sich wie ein
Podcast abonnieren und in der Suche einfügen. Der Server liest dafür nur
die Kanalseite (einmal) und danach den RSS-Feed des Kanals; Ton oder Video
holt er nie: Die Adresse einer Folge ist die Videoseite (watch?v=…), und
jede App sucht sich den reinen Audio-Stream selbst (docs/youtube.md).
Shorts bleiben draußen, Stille-Analyse und ID3-Kapitel gibt es für solche
Folgen nicht.

Der Player im Browser verweist auf die Apps; die Desktop-App fragt ihren
Hauptprozess (yt-dlp) nach dem Stream. Eine Dauer, die der Feed nicht hat,
meldet der Player mit dem Fortschritt (progress/save: duration_seconds).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mrwnslz 2026-10-16 10:33:26 +02:00
parent 1d6ddabb7b
commit cde90bed8e
6 changed files with 504 additions and 8 deletions

46
docs/youtube.md Normal file
View file

@ -0,0 +1,46 @@
# YouTube channels as podcasts
A YouTube channel (or playlist) can be subscribed to like a podcast. Only its
sound is played, and **the server never downloads or streams a video or its
sound**: each app finds the audio stream itself.
## Server (diora-web, `podcasts/youtube.py`)
- `POST /podcasts/feeds/add/` takes a YouTube address as `rss_url`: a channel
(`/@name`, `/channel/UC…`, `/c/…`, `/user/…`), a playlist (`?list=…`), a video
(subscribes its channel) or the feed itself. The server reads the channel page
once (feed address, name, picture) and stores the feed
`https://www.youtube.com/feeds/videos.xml?channel_id=UC…` (or `?playlist_id=…`).
`GET /podcasts/search/?q=<YouTube address>` answers with that one channel.
- From then on the feed is refreshed like any other. It carries the latest 15
videos. **Shorts are left out.** There is no duration (`duration_seconds` 0)
until a player knows it: `POST /podcasts/progress/save/` takes an optional
`duration_seconds` (number, 1 s … 7 days), stored only while the episode has none.
- An episode from such a feed has
- `audio_url` = `https://www.youtube.com/watch?v=<11-character id>`. This is
**the** marker: a client recognises a YouTube episode by this prefix;
- `guid` = `yt:video:<id>`, `artwork_url` = the video's thumbnail,
`description` = the video description as HTML (links, line breaks; timestamps
like `00:00 Intro` work as usual).
- Nothing on the server touches the sound of such an episode: no ID3 chapters,
no transcripts, and `/podcasts/episodes/<id>/silences/` answers
`{"ready": true, "spans": [], "error": "youtube"}` (skip silence is up to the
app, as on Android anyway).
## Apps
A client that sees a `watch?v=` address resolves it **itself, on the device,
to an audio-only stream** (no video track), and plays or downloads that.
| Client | How |
|---------|-----|
| Android | NewPipeExtractor: the original audio track, progressive, opus before m4a, ≤ 160 kbit/s; resolved when the stream is opened and once more on a 403; downloads in 10 MB ranges (in one go YouTube throttles to ~50 KB/s) |
| Desktop | yt-dlp in the main process (`-f bestaudio`), Electron itself as its JavaScript runtime; yt-dlp is fetched from its GitHub release on first use and kept up to date (`src/main/youtube.js`) |
| Browser | can't (YouTube doesn't let a web page read its pages, and a stream address only works for the device that asked for it): the episode is listed but says it plays in the app |
Stream addresses are bound to the device's IP address and run out after a few
hours: resolve shortly before playing, and again when the player reports an
error (podcasts-web.js reloads after a media error anyway).
Progress, queue, played state etc. work as for any episode: they belong to the
episode on the server, not to the stream address.

194
podcasts/test_youtube.py Normal file
View file

@ -0,0 +1,194 @@
import json
from unittest.mock import patch
import feedparser
from django.contrib.auth.models import User
from django.test import TestCase
from . import youtube
from .models import PodcastEpisode, PodcastFeed
from .views import FeedFetch, _load_chapters
CHANNEL = 'UC5NOEUbkLheQcaaRldYW5GA'
FEATURED = 'UC1_uAIS3r8Vu6JjXWvastJg' # another channel the page shows
PAGE = (
f'<html><script>{{"channelId":"{FEATURED}"}}</script><head><meta property="og:title" content="Tages &amp; Schau">'
'<meta property="og:image" content="https://yt3.example/pic.jpg">'
f'<link rel="alternate" type="application/rss+xml" title="RSS" href="{youtube.FEED}?channel_id={CHANNEL}">'
f'</head><body><script>var x = {{"externalId":"{CHANNEL}"}};</script></body></html>'
)
def _entry(vid, title, kind='watch?v=', when='2026-10-14T10:00:00+00:00', text='Hallo\nhttps://example.org/x\n\n00:00 Anfang'):
link = f'https://www.youtube.com/{kind}{vid}'
return (
f'<entry><id>yt:video:{vid}</id><yt:videoId>{vid}</yt:videoId><yt:channelId>{CHANNEL}</yt:channelId>'
f'<title>{title}</title><link rel="alternate" href="{link}"/>'
f'<author><name>Tages Schau</name></author><published>{when}</published><updated>{when}</updated>'
f'<media:group><media:title>{title}</media:title>'
f'<media:content url="https://www.youtube.com/v/{vid}?version=3" type="application/x-shockwave-flash" width="640" height="390"/>'
f'<media:thumbnail url="https://i1.ytimg.com/vi/{vid}/hqdefault.jpg" width="480" height="360"/>'
f'<media:description>{text}</media:description></media:group></entry>'
)
FEED_XML = (
'<?xml version="1.0" encoding="UTF-8"?>'
'<feed xmlns:yt="http://www.youtube.com/xml/schemas/2015" xmlns:media="http://search.yahoo.com/mrss/" '
'xmlns="http://www.w3.org/2005/Atom">'
f'<link rel="self" href="{youtube.FEED}?channel_id={CHANNEL}"/><id>yt:channel:{CHANNEL[2:]}</id>'
f'<yt:channelId>{CHANNEL[2:]}</yt:channelId><title>Tages Schau</title>'
f'<link rel="alternate" href="https://www.youtube.com/channel/{CHANNEL}"/>'
'<author><name>Tages Schau</name></author><published>2010-01-01T00:00:00+00:00</published>'
+ _entry('aaaaaaaaaaa', 'Video eins')
+ _entry('bbbbbbbbbbb', 'Ein Short', kind='shorts/')
+ _entry('ccccccccccc', 'Video zwei', when='2026-10-13T10:00:00+00:00')
+ '</feed>'
)
class _Page:
def __init__(self, text, status=200):
self.status_code = status
self.raw = self
self._body = text.encode()
def read(self, n, decode_content=True):
return self._body[:n]
def close(self):
pass
class ResolveTests(TestCase):
def resolve(self, url, page=PAGE):
seen = []
def get(u, **kw):
seen.append(u)
return _Page(page)
with patch('podcasts.youtube.requests.get', get):
return youtube.resolve(url), seen
def test_a_channel_page_is_its_own_channel_not_one_it_features(self):
page = (f'<script>{{"channelId":"{FEATURED}"}}</script>'
f'<link rel="canonical" href="https://www.youtube.com/channel/{CHANNEL}">')
found, _ = self.resolve('https://www.youtube.com/@3blue1brown', page)
self.assertEqual(found['feed'], f'{youtube.FEED}?channel_id={CHANNEL}')
def test_a_handle_leads_to_its_channel_feed(self):
found, seen = self.resolve('https://www.youtube.com/@tagesschau/videos')
self.assertEqual(found, {'feed': f'{youtube.FEED}?channel_id={CHANNEL}', 'title': 'Tages & Schau',
'artwork': 'https://yt3.example/pic.jpg'})
self.assertEqual(seen, ['https://www.youtube.com/@tagesschau/videos'])
def test_a_video_leads_to_its_channel_without_the_videos_name(self):
page = (f'<meta property="og:title" content="Ein Video"><script>{{"channelId":"{FEATURED}"}};'
f'{{"videoDetails":{{"videoId":"aaaaaaaaaaa","channelId":"{CHANNEL}"}}}}</script>')
found, seen = self.resolve('https://youtu.be/aaaaaaaaaaa', page)
self.assertEqual(found['feed'], f'{youtube.FEED}?channel_id={CHANNEL}')
self.assertEqual((found['title'], found['artwork']), ('', ''))
self.assertEqual(seen, [youtube.WATCH + 'aaaaaaaaaaa'])
def test_a_playlist_has_its_own_feed(self):
found, _ = self.resolve('https://www.youtube.com/playlist?list=PLabcdefghijk')
self.assertEqual(found['feed'], f'{youtube.FEED}?playlist_id=PLabcdefghijk')
def test_other_youtube_pages_are_no_channel(self):
with self.assertRaises(youtube.NotAChannel):
youtube.resolve('https://www.youtube.com/feed/subscriptions')
def test_a_page_without_a_channel_is_none(self):
with self.assertRaises(youtube.NotAChannel):
self.resolve('https://www.youtube.com/@nobody', '<html>consent</html>')
class YouTubeFeedTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
def subscribe(self, url):
pages = []
def get(u, **kw):
pages.append(u)
return _Page(PAGE)
fetched = []
def fetch(u, **kw):
fetched.append(u)
return FeedFetch(200, feedparser.parse(FEED_XML))
with patch('podcasts.youtube.requests.get', get), patch('podcasts.views._fetch_feed', fetch):
r = self.client.post('/podcasts/feeds/add/', json.dumps({'rss_url': url, 'title': url}),
content_type='application/json')
return r, pages, fetched
def test_subscribing_to_a_channel_reads_its_feed_never_a_video(self):
r, pages, fetched = self.subscribe('https://www.youtube.com/@tagesschau')
self.assertEqual(r.status_code, 200, r.content)
feed = PodcastFeed.objects.get()
self.assertEqual(feed.rss_url, f'{youtube.FEED}?channel_id={CHANNEL}')
self.assertEqual(feed.title, 'Tages Schau')
self.assertEqual(feed.artwork_url, 'https://yt3.example/pic.jpg') # kept: the feed has none
self.assertEqual(pages, ['https://www.youtube.com/@tagesschau'])
self.assertEqual(fetched, [feed.rss_url])
eps = {e.title: e for e in PodcastEpisode.objects.all()}
self.assertEqual(set(eps), {'Video eins', 'Video zwei'}) # no Short
one = eps['Video eins']
self.assertEqual(one.audio_url, 'https://www.youtube.com/watch?v=aaaaaaaaaaa')
self.assertEqual(one.guid, 'yt:video:aaaaaaaaaaa')
self.assertEqual(one.artwork_url, 'https://i1.ytimg.com/vi/aaaaaaaaaaa/hqdefault.jpg')
self.assertEqual(one.description,
'<p>Hallo<br><a href="https://example.org/x">https://example.org/x</a></p><p>00:00 Anfang</p>')
def test_the_same_channel_twice_is_one_subscription(self):
self.subscribe('https://www.youtube.com/@tagesschau')
r, _, _ = self.subscribe(f'https://www.youtube.com/channel/{CHANNEL}')
self.assertFalse(r.json()['created'])
self.assertEqual(PodcastFeed.objects.count(), 1)
def test_a_feed_subscribed_directly_gets_the_channels_picture(self):
r, pages, _ = self.subscribe(f'{youtube.FEED}?channel_id={CHANNEL}')
self.assertEqual(r.status_code, 200)
self.assertEqual(PodcastFeed.objects.get().artwork_url, 'https://yt3.example/pic.jpg')
self.assertEqual(pages, [f'https://www.youtube.com/channel/{CHANNEL}'])
def test_no_channel_there_is_an_error(self):
with patch('podcasts.youtube.requests.get', lambda u, **kw: _Page('<html></html>')):
r = self.client.post('/podcasts/feeds/add/', json.dumps({'rss_url': 'https://www.youtube.com/@x'}),
content_type='application/json')
self.assertEqual(r.status_code, 400)
self.assertFalse(PodcastFeed.objects.exists())
def test_search_finds_a_pasted_channel(self):
with patch('podcasts.youtube.requests.get', lambda u, **kw: _Page(PAGE)):
r = self.client.get('/podcasts/search/', {'q': 'https://www.youtube.com/@tagesschau'})
self.assertEqual(r.json()['results'][0]['rss_url'], f'{youtube.FEED}?channel_id={CHANNEL}')
self.assertEqual(r.json()['results'][0]['title'], 'Tages & Schau')
def test_the_server_never_fetches_a_videos_sound(self):
self.subscribe('https://www.youtube.com/@tagesschau')
ep = PodcastEpisode.objects.get(title='Video eins')
with patch('podcasts.silence.ensure') as ensure:
d = self.client.get(f'/podcasts/episodes/{ep.pk}/silences/').json()
ensure.assert_not_called()
self.assertEqual((d['ready'], d['spans'], d['error']), (True, [], 'youtube'))
with patch('podcasts.chapters.fetch_id3') as id3:
self.assertEqual(_load_chapters(ep), [])
id3.assert_not_called()
def test_the_players_duration_is_kept_when_the_feed_has_none(self):
self.subscribe('https://www.youtube.com/@tagesschau')
ep = PodcastEpisode.objects.get(title='Video eins')
save = lambda body: self.client.post('/podcasts/progress/save/', json.dumps({'episode_id': ep.pk, **body}),
content_type='application/json')
save({'position_seconds': 10, 'duration_seconds': 'x'})
save({'position_seconds': 10, 'duration_seconds': 10 ** 9})
ep.refresh_from_db()
self.assertEqual(ep.duration_seconds, 0)
save({'position_seconds': 10, 'duration_seconds': 929.3})
save({'position_seconds': 20, 'duration_seconds': 5}) # known now: not changed again
ep.refresh_from_db()
self.assertEqual(ep.duration_seconds, 929)

View file

@ -28,6 +28,7 @@ from . import chapters as chapters_mod
from . import silence as silence_mod from . import silence as silence_mod
from . import feedext from . import feedext
from . import transcripts as transcripts_mod from . import transcripts as transcripts_mod
from . import youtube as youtube_mod
from .models import ( from .models import (
EpisodeProgress, EpisodeTranscript, PlayerDevice, PlayerState, PodcastEpisode, PodcastFeed, PodcastQueue, EpisodeProgress, EpisodeTranscript, PlayerDevice, PlayerState, PodcastEpisode, PodcastFeed, PodcastQueue,
MAX_SPEED, MIN_SPEED, PodcastSettings, counts_as_played, MAX_SPEED, MIN_SPEED, PodcastSettings, counts_as_played,
@ -44,6 +45,7 @@ FEED_WORKERS = 16 # feeds downloaded at the same time
REFRESH_WAIT = 40 # seconds a refresh-all (⟳) waits; feeds not done by then wait for the hourly job REFRESH_WAIT = 40 # seconds a refresh-all (⟳) waits; feeds not done by then wait for the hourly job
MAX_DESCRIPTION = 50_000 # characters of shownotes HTML kept per episode MAX_DESCRIPTION = 50_000 # characters of shownotes HTML kept per episode
MAX_TRANSCRIPTS = 10 MAX_TRANSCRIPTS = 10
MAX_DURATION = 7 * 24 * 3600 # seconds: a longer one reported is nonsense
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -537,7 +539,8 @@ def _entry_media_url(entry):
return url return url
if not video and (mime.startswith('video/') or path.endswith(VIDEO_EXTENSIONS)): if not video and (mime.startswith('video/') or path.endswith(VIDEO_EXTENSIONS)):
video = url video = url
return video # A YouTube feed has no files: the video's page, whose sound the app finds itself.
return video or youtube_mod.video_url(entry)
def _apply_feed(feed_obj, fetched, full=False): def _apply_feed(feed_obj, fetched, full=False):
@ -569,7 +572,10 @@ def _apply_feed(feed_obj, fetched, full=False):
channel.get('itunes_image', {}).get('href', '') channel.get('itunes_image', {}).get('href', '')
or image.get('href', '') or image.get('href', '')
or image.get('url', '') or image.get('url', '')
or feed_obj.artwork_url # a YouTube feed has none: the channel's, found when subscribing
)[:1000] )[:1000]
if not feed_obj.artwork_url and youtube_mod.feed_channel_id(channel):
feed_obj.artwork_url = youtube_mod.channel_artwork(youtube_mod.feed_channel_id(channel))[:1000]
feed_obj.last_refreshed_at = now feed_obj.last_refreshed_at = now
feed_obj.save() feed_obj.save()
@ -596,7 +602,8 @@ def _apply_feed(feed_obj, fetched, full=False):
guid = entry.get('id') or entry.get('guid') or audio_url guid = entry.get('id') or entry.get('guid') or audio_url
guid = str(guid)[:1000] guid = str(guid)[:1000]
title = entry.get('title', 'Untitled')[:500] title = entry.get('title', 'Untitled')[:500]
description = _entry_description(entry) video = youtube_mod.is_video(audio_url)
description = youtube_mod.description_html(entry.get('summary')) if video else _entry_description(entry)
chapters_url = _entry_chapters_url(entry) chapters_url = _entry_chapters_url(entry)
psc_chapters = chapters_mod.from_psc(entry) psc_chapters = chapters_mod.from_psc(entry)
transcripts = _entry_transcripts(entry) transcripts = _entry_transcripts(entry)
@ -628,7 +635,7 @@ def _apply_feed(feed_obj, fetched, full=False):
except (ValueError, TypeError): except (ValueError, TypeError):
pass pass
artwork_url = entry.get('itunes_image', {}).get('href', '')[:1000] artwork_url = (youtube_mod.entry_artwork(entry) if video else entry.get('itunes_image', {}).get('href', ''))[:1000]
episode_obj, created = PodcastEpisode.objects.get_or_create( episode_obj, created = PodcastEpisode.objects.get_or_create(
feed=feed_obj, feed=feed_obj,
@ -723,6 +730,15 @@ def podcast_search(request):
q = request.GET.get('q', '').strip() q = request.GET.get('q', '').strip()
if not q: if not q:
return JsonResponse({'results': []}) return JsonResponse({'results': []})
if youtube_mod.is_youtube(q):
try:
found = youtube_mod.resolve(q)
except youtube_mod.NotAChannel:
return JsonResponse({'results': []})
return JsonResponse({'results': [{
'id': found['feed'], 'title': found['title'] or 'YouTube', 'author': 'YouTube',
'artwork_url': found['artwork'], 'rss_url': found['feed'], 'genre': 'YouTube',
}]})
try: try:
url = f'https://itunes.apple.com/search?term={urllib.parse.quote(q)}&media=podcast&limit=20' url = f'https://itunes.apple.com/search?term={urllib.parse.quote(q)}&media=podcast&limit=20'
@ -830,11 +846,22 @@ def add_feed(request):
rss_url = body.get('rss_url', '').strip() rss_url = body.get('rss_url', '').strip()
if not rss_url: if not rss_url:
return JsonResponse({'error': 'rss_url required'}, status=400) return JsonResponse({'error': 'rss_url required'}, status=400)
title = body.get('title', rss_url)
artwork = ''
if youtube_mod.is_youtube(rss_url) and not youtube_mod.is_feed(rss_url):
# A channel, playlist or video address: its channel's feed (the server reads only that).
try:
found = youtube_mod.resolve(rss_url)
except youtube_mod.NotAChannel as e:
return JsonResponse({'error': f'No YouTube channel there ({e})'}, status=400)
rss_url, artwork = found['feed'], found['artwork']
title = found['title'] or title
feed, created = PodcastFeed.objects.get_or_create( feed, created = PodcastFeed.objects.get_or_create(
user=request.user, user=request.user,
rss_url=rss_url, rss_url=rss_url,
defaults={'title': body.get('title', rss_url)[:300], **PodcastSettings.new_feed_defaults(request.user)}, defaults={'title': title[:300], 'artwork_url': artwork[:1000],
**PodcastSettings.new_feed_defaults(request.user)},
) )
new_episodes = 0 new_episodes = 0
@ -1101,7 +1128,7 @@ def _load_chapters(episode):
chapters = chapters_mod.fetch(url) chapters = chapters_mod.fetch(url)
except chapters_mod.ChaptersInvalid as e: except chapters_mod.ChaptersInvalid as e:
log.info('chapters of episode %s unusable: %s', episode.id, e) log.info('chapters of episode %s unusable: %s', episode.id, e)
if not chapters and episode.audio_url: if not chapters and episode.audio_url and not youtube_mod.is_video(episode.audio_url):
try: try:
chapters = chapters_mod.fetch_id3(episode.audio_url) chapters = chapters_mod.fetch_id3(episode.audio_url)
except chapters_mod.ChaptersInvalid as e: except chapters_mod.ChaptersInvalid as e:
@ -1179,9 +1206,11 @@ def episode_silences(request, pk):
on the server ({"ready": false}, ask again later).""" on the server ({"ready": false}, ask again later)."""
if not request.user.is_authenticated: if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401) return JsonResponse({'error': 'authentication required'}, status=401)
ep = PodcastEpisode.objects.filter(pk=pk, feed__user=request.user).only('id', 'silences').first() ep = PodcastEpisode.objects.filter(pk=pk, feed__user=request.user).only('id', 'silences', 'audio_url').first()
if ep is None: if ep is None:
return JsonResponse({'error': 'not found'}, status=404) return JsonResponse({'error': 'not found'}, status=404)
if youtube_mod.is_video(ep.audio_url): # the server never fetches a video's sound
return JsonResponse({'ready': True, 'v': silence_mod.VERSION, 'spans': [], 'error': 'youtube'})
if not silence_mod.current(ep.silences): # none yet, or found by older rules if not silence_mod.current(ep.silences): # none yet, or found by older rules
silence_mod.ensure(ep.pk) silence_mod.ensure(ep.pk)
return JsonResponse({'ready': False}) return JsonResponse({'ready': False})
@ -1917,6 +1946,12 @@ def save_progress(request):
except PodcastEpisode.DoesNotExist: except PodcastEpisode.DoesNotExist:
return JsonResponse({'error': 'not found'}, status=404) return JsonResponse({'error': 'not found'}, status=404)
# A feed without durations (YouTube): the one the player found, once.
duration = body.get('duration_seconds')
if not episode.duration_seconds and _number(duration) and math.isfinite(duration) and 1 <= duration <= MAX_DURATION:
episode.duration_seconds = int(duration)
PodcastEpisode.objects.filter(pk=episode.pk, duration_seconds=0).update(duration_seconds=int(duration))
progress, _ = EpisodeProgress.objects.get_or_create( progress, _ = EpisodeProgress.objects.get_or_create(
user=request.user, user=request.user,
episode=episode, episode=episode,

176
podcasts/youtube.py Normal file
View file

@ -0,0 +1,176 @@
"""YouTube channels (and playlists) as podcasts.
The server only reads what YouTube publishes about a channel: its page once,
when it's subscribed (feed address, name, picture), and from then on its RSS
feed like any podcast's. It never fetches a video or its sound: an episode's
address is the video's page (https://www.youtube.com/watch?v=…), and each app
finds the audio stream for it itself and plays only that (docs/youtube.md).
Hence no skipping of silence and no ID3 chapters found on the server for them.
Shorts are left out: a minute of sound cut from a vertical video isn't an
episode.
"""
import html
import re
import urllib.parse
import requests
WATCH = 'https://www.youtube.com/watch?v='
FEED = 'https://www.youtube.com/feeds/videos.xml'
HOSTS = {'youtube.com', 'www.youtube.com', 'm.youtube.com', 'music.youtube.com', 'youtu.be'}
TIMEOUT = 15
MAX_PAGE = 4 * 1024 * 1024
# Without this, a server in the EU gets the cookie consent page instead (SOCS=CAI: declined).
COOKIES = {'SOCS': 'CAI'}
HEADERS = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0',
'Accept-Language': 'de,en;q=0.8'}
_CHANNEL_ID = re.compile(r'^UC[\w-]{22}$')
_VIDEO_ID = re.compile(r'^[\w-]{11}$')
# Where a page names its own channel, surest first: a channel page in its head, a video's in
# its videoDetails.
_OWN_CHANNEL = (
r'<link rel="canonical" href="https://www\.youtube\.com/channel/(UC[\w-]{22})"',
r'type="application/rss\+xml" title="RSS" href="[^"]*channel_id=(UC[\w-]{22})"',
r'"externalId":"(UC[\w-]{22})"',
r'<meta itemprop="identifier" content="(UC[\w-]{22})"',
r'"videoDetails":\{[^{}]*?"channelId":"(UC[\w-]{22})"',
)
class NotAChannel(Exception):
pass
def _host(url):
try:
return (urllib.parse.urlsplit(url).hostname or '').lower()
except ValueError:
return ''
def is_youtube(url):
"""An address on YouTube (a channel, a video, a playlist, a feed)."""
return _host(url) in HOSTS
def is_feed(url):
return is_youtube(url) and urllib.parse.urlsplit(url).path == '/feeds/videos.xml'
def is_video(audio_url):
"""An episode that is a YouTube video (its sound is found by the app)."""
return audio_url.startswith(WATCH)
def channel_feed(channel_id):
return f'{FEED}?channel_id={channel_id}'
def video_url(entry):
"""The episode address for a YouTube feed entry, '' for a Short (or not a video)."""
vid = entry.get('yt_videoid') or ''
if not _VIDEO_ID.match(vid) or '/shorts/' in (entry.get('link') or ''):
return ''
return WATCH + vid
def entry_artwork(entry):
thumbs = entry.get('media_thumbnail') or []
return (thumbs[0].get('url') or '') if thumbs else ''
_LINK = re.compile(r'(https?://[^\s<>"]+)')
def description_html(text):
"""A video description (plain text) as shownotes: links clickable, lines kept."""
escaped = html.escape(text or '', quote=False)
linked = _LINK.sub(lambda m: f'<a href="{m.group(1)}">{m.group(1)}</a>', escaped)
return '<p>' + linked.strip().replace('\n\n', '</p><p>').replace('\n', '<br>') + '</p>' if linked.strip() else ''
def _page(url):
r = requests.get(url, headers=HEADERS, cookies=COOKIES, timeout=TIMEOUT, stream=True)
try:
if r.status_code >= 400:
raise NotAChannel(f'HTTP {r.status_code}')
body = r.raw.read(MAX_PAGE, decode_content=True)
finally:
r.close()
return body.decode('utf-8', 'replace')
def _meta(page, prop):
m = re.search(r'<meta property="%s" content="([^"]*)"' % re.escape(prop), page)
return html.unescape(m.group(1)) if m else ''
def resolve(url):
"""{'feed', 'title', 'artwork'} for a YouTube address someone wants to subscribe to:
a channel (/@name, /channel/…, /c/…, /user/…), a playlist, or a video (its channel).
Raises NotAChannel when it leads to none."""
parts = urllib.parse.urlsplit(url.strip())
query = urllib.parse.parse_qs(parts.query)
path = parts.path.rstrip('/')
if path == '/feeds/videos.xml':
return {'feed': url.strip(), 'title': '', 'artwork': ''}
if 'list' in query and path in ('/playlist', '/watch'):
playlist = query['list'][0]
if not re.match(r'^[\w-]{10,64}$', playlist):
raise NotAChannel('playlist id')
page = ''
try:
page = _page(f'https://www.youtube.com/playlist?list={playlist}')
except (requests.RequestException, NotAChannel):
pass
return {'feed': f'{FEED}?playlist_id={playlist}', 'title': _meta(page, 'og:title'),
'artwork': _meta(page, 'og:image')}
segs = [s for s in path.split('/') if s]
if segs[:1] == ['channel'] and len(segs) > 1 and _CHANNEL_ID.match(segs[1]):
page_url = f'https://www.youtube.com/channel/{segs[1]}'
elif parts.hostname == 'youtu.be' and segs:
page_url = WATCH + segs[0]
elif segs and (segs[0].startswith('@') or segs[0] in ('c', 'user', 'watch', 'shorts', 'live')):
page_url = urllib.parse.urlunsplit(('https', 'www.youtube.com', parts.path, parts.query, ''))
else:
raise NotAChannel('not a channel address')
try:
page = _page(page_url)
except requests.RequestException as e:
raise NotAChannel(type(e).__name__)
# The page's own channel, never the first one named (featured channels come earlier).
m = None
for pattern in _OWN_CHANNEL:
m = re.search(pattern, page)
if m:
break
if not m:
raise NotAChannel('no channel found')
is_channel_page = not (segs and segs[0] in ('watch', 'shorts', 'live')) and parts.hostname != 'youtu.be'
return {
'feed': channel_feed(m.group(1)),
# A video's page is about the video: its channel's name and picture come with the feed.
'title': _meta(page, 'og:title') if is_channel_page else '',
'artwork': _meta(page, 'og:image') if is_channel_page else '',
}
def feed_channel_id(channel):
"""The channel a YouTube feed (feedparser's .feed) belongs to, '' for another feed."""
for url in (channel.get('link') or '', (channel.get('author_detail') or {}).get('href') or ''):
m = re.match(r'^https://www\.youtube\.com/channel/(UC[\w-]{22})$', url)
if m:
return m.group(1)
return ''
def channel_artwork(channel_id):
"""The channel's picture (the feed has none), '' if it can't be had."""
try:
return _meta(_page(f'https://www.youtube.com/channel/{channel_id}'), 'og:image')
except (requests.RequestException, NotAChannel):
return ''

View file

@ -330,7 +330,10 @@
function saveProgress(id, seconds) { function saveProgress(id, seconds) {
forEpisode(state, id, o => { o.position_seconds = seconds; }); forEpisode(state, id, o => { o.position_seconds = seconds; });
if (seconds > 0) removeFromInbox(state, new Set([id])); // started: no longer new if (seconds > 0) removeFromInbox(state, new Set([id])); // started: no longer new
queueOp('/podcasts/progress/save/', {episode_id: id, position_seconds: seconds}, 'progress:' + id); const body = {episode_id: id, position_seconds: seconds};
// A feed without durations (YouTube): the player's, kept by the server if it has none.
if (curId() === id && !(cur.episode__duration_seconds > 0) && lastDur > 0) body.duration_seconds = Math.round(lastDur / 1000);
queueOp('/podcasts/progress/save/', body, 'progress:' + id);
save(); save();
// No re-render: progress changes every few seconds while playing. // No re-render: progress changes every few seconds while playing.
} }
@ -708,12 +711,52 @@
loaded = true; loaded = true;
error = null; error = null;
buffering = true; buffering = true;
if (YOUTUBE.test(cur.episode__audio_url)) { prepareYouTube(); return; }
audio.src = cur.episode__audio_url; audio.src = cur.episode__audio_url;
applyKind(); applyKind();
} }
// A YouTube episode's address is the video's page: the server never touches its sound
// (diora-web docs/youtube.md). The desktop app finds the audio-only stream on this
// computer; a browser can't (YouTube doesn't let a page read it).
const YOUTUBE = /^https:\/\/www\.youtube\.com\/watch\?v=[\w-]{11}$/;
let youtubeStale = false; // the last stream failed: find it anew
let youtubeAsk = 0;
function prepareYouTube() {
const ask = ++youtubeAsk;
audio.removeAttribute('src'); // not the stream that failed, while the new one is found
try { audio.load(); } catch (e) {}
const desk = window.dioraDesktop;
const fail = text => {
loaded = false;
error = 'youtube';
wantPlay = false;
buffering = false;
toast(text);
report();
syncSoon();
};
if (!desk || !desk.youtubeAudio) { fail('YouTube-Folgen spielen in der diora-App (Android, Desktop)'); return; }
const fresh = youtubeStale;
youtubeStale = false;
desk.youtubeAudio(cur.episode__audio_url, fresh).then(r => {
if (ask !== youtubeAsk || !loaded) return; // moved on meanwhile
if (!r || r.error || !r.url) { fail('Diese YouTube-Folge lässt sich gerade nicht abspielen'); return; }
if (r.duration > 0) lastDur = r.duration * 1000;
audio.src = r.url;
applyKind();
if (wantPlay) {
const p = audio.play();
if (p && p.catch) p.catch(() => {});
}
report();
});
}
function unload() { function unload() {
loaded = false; loaded = false;
youtubeAsk++;
mustReach = null; mustReach = null;
audio.pause(); audio.pause();
audio.removeAttribute('src'); audio.removeAttribute('src');
@ -764,6 +807,7 @@
sync.start = true; sync.start = true;
applyKind(); applyKind();
setSession(); setSession();
if (!audio.getAttribute('src')) { report(); syncSoon(); return; } // a stream still being found
const p = audio.play(); const p = audio.play();
if (p && p.catch) { if (p && p.catch) {
p.catch(err => { p.catch(err => {
@ -935,6 +979,7 @@
loaded = false; loaded = false;
startAt = at; startAt = at;
mustReach = null; mustReach = null;
youtubeStale = true;
if (play && ++failures <= 3) { resume(); return; } if (play && ++failures <= 3) { resume(); return; }
error = 'MEDIA_ERR_' + (audio.error ? audio.error.code : 0); error = 'MEDIA_ERR_' + (audio.error ? audio.error.code : 0);
wantPlay = false; wantPlay = false;

View file

@ -2,7 +2,7 @@
* diora service worker — caches the app shell for offline use. * diora service worker — caches the app shell for offline use.
*/ */
const CACHE = 'diora-v111'; const CACHE = 'diora-v112';
const SHELL = [ const SHELL = [
'/static/css/app.css', '/static/css/app.css',
'/static/js/app/01-core.js', '/static/js/app/01-core.js',