Compare commits

...

7 commits

Author SHA1 Message Date
f9b3a0ce04 Podcasts: Inbox und Gehört wie in AntennaPod
Some checks failed
Build and push Docker image / build (push) Successful in 14s
Test / test (push) Failing after 28s
- Eingereihte und angefangene Folgen verlassen die Inbox (ausgeblendet,
  nicht gehört), statt dort neben der Warteschlange zu stehen.
- Gehört heißt: höchstens 30 s vor dem Ende, wie in den Apps, statt bei
  90 % (bei 2 h waren das 12 Minuten vor Schluss). Gilt auch für den
  gPodder-Sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:26:28 +02:00
c335ab4094 gPodder-Sync: Hörstelle auf die eigene Folge schreiben
Some checks failed
Build and push Docker image / build (push) Successful in 16s
Test / test (push) Failing after 28s
Die Folge wurde nur über die Audio-Adresse gesucht, über alle Nutzer
hinweg. Haben zwei Nutzer denselben Podcast, landete die Stelle bei der
Folgen-Zeile des anderen (der eigene Fortschritt, aber an der falschen
Folge) und kam beim eigenen Abo nie an. Jetzt nur unter den eigenen
Abos gesucht; mit Test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:18:23 +02:00
f30bf2fcdc Podcasts: Favoriten, Folgenliste mit Filtern und Verlauf, Hörstatistik, OPML-Export
Some checks failed
Build and push Docker image / build (push) Successful in 15s
Test / test (push) Failing after 27s
- EpisodeProgress.favorite (Migration 0006) und POST episodes/<id>/favorite/
  (setzen oder umschalten; ändert updated_at nicht, damit Verlauf und
  Statistik nicht verfälscht werden)
- GET episodes/: alle Folgen aller Abos, Filter all/unplayed/in_progress/
  played/favorites/history, feed_id, q, offset/limit/total; zwei Abfragen
  pro Seite, ohne Shownotes
- Folgendetails liefern zusätzlich favorite und last_played_at
- GET stats/: Hörzeit gesamt und pro Podcast, optional nur die letzten N Tage
- GET feeds/export.opml: Abos als OPML 2.0 (lässt sich wieder importieren)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:12:27 +02:00
fd0a103fb2 Podcasts: state/ ohne Shownotes in der Inbox
Some checks failed
Build and push Docker image / build (push) Successful in 16s
Test / test (push) Failing after 28s
Seit die Shownotes content:encoded nehmen (bis 50 KB pro Folge), würde
die eine Anfrage der Apps mit 200 Inbox-Folgen sehr groß. state/ lässt
sie weg; die Apps holen sie pro Folge über episodes/<id>/. inbox/ für
das Web liefert sie weiter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:01:34 +02:00
f4b8bcc891 Podcasts: Kapitel, Shownotes und Transkripte für die Apps
- Feeds liefern jetzt Podlove-Kapitel (psc:chapters), Podcasting-2.0-Kapitel
  (podcast:chapters, JSON-URL) und Transkripte (podcast:transcript, auch
  mehrere); als Beschreibung wird content:encoded bevorzugt.
- Bekannte Folgen werden beim Aktualisieren nachgezogen, aber nur die Zeilen,
  die sich wirklich geändert haben.
- Neuer Endpunkt GET /podcasts/episodes/<id>/ mit bereinigten Shownotes,
  Zeitmarken, Kapiteln, Transkripten und Fortschritt. JSON-Kapitel werden
  einmal serverseitig geholt (nur öffentliche Adressen, max. 1 MB) und an
  der Folge gespeichert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 07:56:44 +02:00
a1e17081c5 Podcasts: neues Abo bringt nur die neueste Folge in die Inbox
All checks were successful
Build and push Docker image / build (push) Successful in 16s
Test / test (push) Successful in 2m3s
Beim ersten Abruf eines Feeds (Abonnieren, OPML-Import, gpodder) kam der
ganze Rückkatalog als „neu“ in die Inbox (bei Lage der Nation 200
Folgen). Jetzt zählt nur die neueste; die älteren sind ausgeblendet,
nicht gehört, und stehen weiter in der Folgenliste des Podcasts. Auch
automatisches Einreihen nimmt beim ersten Abruf nur diese eine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 07:48:45 +02:00
176771a407 Podcasts: Position in der Warteschlange einstellbar; Feeds nur noch auf dem Server holen (SW v51)
All checks were successful
Build and push Docker image / build (push) Successful in 19s
Test / test (push) Successful in 2m0s
Warteschlange
- PodcastSettings: wohin neue Folgen kommen, getrennt für Hinzufügen von
  Hand (auch beim Abspielen einer Folge außerhalb der Warteschlange) und
  für automatisch eingereihte neue Folgen: ans Ende, an den Anfang oder
  nach die laufende Folge. Voreinstellung wie bisher (Hand: Ende,
  automatisch: Anfang). GET/POST /podcasts/settings/.
- queue/add nimmt auch mehrere Folgen (episode_ids), eine Position
  (where) und die laufende Folge (current_episode_id); ohne Angabe gilt
  die zuletzt angehörte, nicht fertige Folge als laufende.
- Positionen werden lückenlos neu durchnummeriert.

Feeds
- POST /podcasts/feeds/refresh-all/ aktualisiert alle Feeds des Nutzers
  auf dem Server: Downloads parallel, jede Feed-Adresse nur einmal (auch
  über Nutzer hinweg), mit Timeout; kürzlich aktualisierte werden
  übersprungen. Der stündliche Cron-Job nutzt dasselbe.
- GET /podcasts/state/ liefert Abos, Warteschlange, Inbox und
  Einstellungen in einer Antwort (für die Apps).
- Web: „↻ All“ ist eine Anfrage statt einer pro Feed; die stündliche
  Schleife im Browser lädt nur noch die Listen neu, statt jeden Feed
  selbst anzustoßen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 07:26:16 +02:00
15 changed files with 2202 additions and 92 deletions

36
gpodder/tests.py Normal file
View file

@ -0,0 +1,36 @@
import base64
import json
from django.contrib.auth.models import User
from django.test import TestCase
from podcasts.models import EpisodeProgress, PodcastEpisode, PodcastFeed
class EpisodeActionTests(TestCase):
"""Play positions from a gPodder client (AntennaPod sync) land on the user's own episode."""
def setUp(self):
self.alice = User.objects.create_user(username='alice', password='pw12345678')
bob = User.objects.create_user(username='bob', password='pw12345678')
url = 'https://example.org/feed'
audio = 'https://example.org/e1.mp3'
# Bob subscribed first: his copy of the episode has the lower id.
bobs_feed = PodcastFeed.objects.create(user=bob, rss_url=url, title='Feed')
self.bobs_ep = PodcastEpisode.objects.create(feed=bobs_feed, guid='e1', title='E1', audio_url=audio,
duration_seconds=1000)
alices_feed = PodcastFeed.objects.create(user=self.alice, rss_url=url, title='Feed')
self.alices_ep = PodcastEpisode.objects.create(feed=alices_feed, guid='e1', title='E1', audio_url=audio,
duration_seconds=1000)
self.auth = 'Basic ' + base64.b64encode(b'alice:pw12345678').decode()
def test_position_goes_to_own_episode(self):
actions = [{'podcast': 'https://example.org/feed', 'episode': 'https://example.org/e1.mp3',
'action': 'play', 'position': 300, 'total': 1000}]
r = self.client.post('/api/2/episodes/alice.json', json.dumps(actions), content_type='application/json',
HTTP_AUTHORIZATION=self.auth)
self.assertEqual(r.status_code, 200, r.content)
progress = EpisodeProgress.objects.get(user=self.alice)
self.assertEqual(progress.episode, self.alices_ep)
self.assertEqual(progress.position_seconds, 300)
self.assertFalse(EpisodeProgress.objects.filter(episode=self.bobs_ep).exists())

View file

@ -8,7 +8,7 @@ from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods
from podcasts.models import PodcastFeed, PodcastEpisode, EpisodeProgress
from podcasts.models import PodcastFeed, PodcastEpisode, EpisodeProgress, counts_as_played
from podcasts.views import _refresh_feed
from .models import GpodderDevice, GpodderSubscriptionChange, GpodderEpisodeAction
@ -230,11 +230,12 @@ def episode_actions(request, username):
# Sync playback position to EpisodeProgress
if action == 'play' and a.get('position') is not None:
try:
ep = PodcastEpisode.objects.filter(audio_url=episode_url).first()
# Only this user's subscriptions: several users can have the same feed.
ep = PodcastEpisode.objects.filter(audio_url=episode_url, feed__user=request.user).first()
if ep:
pos = int(a['position'])
dur = ep.duration_seconds or 0
played = dur > 0 and pos >= dur * 0.9
played = counts_as_played(pos, dur)
EpisodeProgress.objects.update_or_create(
user=request.user, episode=ep,
defaults={'position_seconds': pos, 'played': played}

166
podcasts/chapters.py Normal file
View file

@ -0,0 +1,166 @@
"""Podcast chapters: Podlove Simple Chapters from the feed, and Podcasting 2.0
JSON chapters fetched from the URL a feed points to.
Both end up in one shape, a list sorted by start time:
[{"start": 62.5, "title": "…", "url": "https://…", "img": "https://…"}]
url and img only when present.
The JSON file lives on a server the feed's author chose, so fetching it is an
SSRF surface like the WebDAV import, and it reuses that module's guards:
public addresses only (checked before connecting and again on the connected
socket), redirects followed by hand with each hop re-checked, and a size cap.
"""
import json
import re
from urllib.parse import urljoin
import requests
from accounts.webdav import WebDAVError, WebDAVInputError, _assert_peer_is_safe, _read_capped, assert_safe_url
MAX_CHAPTERS = 500
MAX_TITLE = 500
MAX_URL = 1000
CHAPTERS_TIMEOUT = 10 # seconds
CHAPTERS_MAX_BYTES = 1024 * 1024
CHAPTERS_MAX_REDIRECTS = 3
_SCHEMES = ('http://', 'https://')
def _clean_url(value):
value = (value or '').strip() if isinstance(value, str) else ''
if value.lower().startswith(_SCHEMES) and len(value) <= MAX_URL:
return value
return ''
def _chapter(start, title, url='', img=''):
"""One normalised chapter, or None if the start isn't a usable time."""
try:
start = float(start)
except (TypeError, ValueError):
return None
if start != start or start < 0 or start == float('inf'): # NaN, negative, inf
return None
ch = {'start': round(start, 3), 'title': ' '.join(str(title or '').split())[:MAX_TITLE]}
url, img = _clean_url(url), _clean_url(img)
if url:
ch['url'] = url
if img:
ch['img'] = img
return ch
def _finish(chapters):
chapters = [c for c in chapters if c is not None]
chapters.sort(key=lambda c: c['start'])
return chapters[:MAX_CHAPTERS]
# ---------------------------------------------------------------------------
# Podlove Simple Chapters (inline in the feed)
# ---------------------------------------------------------------------------
# "1:02:03.500", "01:02:03", "02:03", "123.5" (normal play time, as PSC uses)
_NPT = re.compile(r'^\s*(?:(?:(\d+):)?(\d{1,2}):)?(\d+(?:\.\d+)?)\s*$')
def parse_npt(value):
"""Seconds (float) from a PSC start time, or None."""
m = _NPT.match(str(value or ''))
if not m:
return None
hours, minutes, seconds = m.groups()
return int(hours or 0) * 3600 + int(minutes or 0) * 60 + float(seconds)
def from_psc(entry):
"""Chapters from feedparser's entry["psc_chapters"], [] if there are none."""
raw = (entry.get('psc_chapters') or {}).get('chapters') or []
return _finish(
_chapter(parse_npt(c.get('start')), c.get('title'), c.get('href'), c.get('image'))
for c in raw
)
# ---------------------------------------------------------------------------
# Podcasting 2.0 JSON chapters (fetched)
# ---------------------------------------------------------------------------
def from_json(data):
"""Normalise a JSON chapters document (https://github.com/Podcastindex-org/
podcast-namespace/blob/main/chapters/jsonChapters.md)."""
raw = data.get('chapters') if isinstance(data, dict) else None
if not isinstance(raw, list):
raise ValueError('no chapters list')
chapters = []
for c in raw:
if not isinstance(c, dict):
continue
# toc: false = "don't show this one to the user" (e.g. art changes).
if c.get('toc') is False:
continue
chapters.append(_chapter(c.get('startTime'), c.get('title'), c.get('url'), c.get('img')))
return _finish(chapters)
class ChaptersUnavailable(Exception):
"""The chapters can't be loaded right now; worth trying again later."""
class ChaptersInvalid(Exception):
"""The chapters URL is unusable for good (blocked, missing, not chapters)."""
def fetch_json(url):
"""Download and parse the JSON at `url`.
Raises ChaptersInvalid for permanent problems and ChaptersUnavailable for
ones that may go away (network trouble, server errors).
"""
for _ in range(CHAPTERS_MAX_REDIRECTS + 1):
try:
assert_safe_url(url)
except WebDAVInputError as e:
raise ChaptersInvalid(str(e))
try:
r = requests.get(
url, timeout=CHAPTERS_TIMEOUT, allow_redirects=False, stream=True,
headers={'User-Agent': 'diora podcast chapters', 'Accept': 'application/json'},
)
except requests.RequestException as e:
raise ChaptersUnavailable(e.__class__.__name__)
try:
_assert_peer_is_safe(r)
if r.status_code in (301, 302, 303, 307, 308):
location = r.headers.get('Location')
if not location:
raise ChaptersInvalid('redirect without a location')
url = urljoin(url, location)
continue
if r.status_code >= 500 or r.status_code == 429:
raise ChaptersUnavailable(f'HTTP {r.status_code}')
if r.status_code >= 400:
raise ChaptersInvalid(f'HTTP {r.status_code}')
body = _read_capped(r, CHAPTERS_MAX_BYTES, lambda: ChaptersInvalid('too large'))
except WebDAVInputError as e:
raise ChaptersInvalid(str(e))
except WebDAVError as e: # transfer broke off in _read_capped
raise ChaptersUnavailable(str(e))
finally:
r.close()
try:
return json.loads(body)
except ValueError:
raise ChaptersInvalid('not JSON')
raise ChaptersInvalid('too many redirects')
def fetch(url):
"""Normalised chapters from a Podcasting 2.0 chapters URL."""
try:
return from_json(fetch_json(url))
except ValueError:
raise ChaptersInvalid('not a chapters document')

43
podcasts/feedext.py Normal file
View file

@ -0,0 +1,43 @@
"""Teach feedparser the Podcasting 2.0 tags we read.
feedparser 6 doesn't know the podcast namespace. Unknown tags still come
through, but only under whatever prefix the feed happened to declare, and an
unknown tag with attributes is stored as a single dict, so a second
<podcast:transcript> overwrites the first. Registering the namespace fixes the
prefix ("podcast_…" for every feed); the handler below keeps every
transcript in a list, entry["podcast_transcripts"].
podcast:chapters is left to feedparser's default handling, which gives
entry["podcast_chapters"] = {"url": …, "type": …} (one per item, as the spec
says).
"""
from feedparser.mixin import _FeedParserMixin as FeedParserMixin
from feedparser.util import FeedParserDict
PODCAST_NAMESPACES = (
'https://podcastindex.org/namespace/1.0',
# The URL the namespace was announced under; some early feeds still use it.
'https://github.com/podcastindex-org/podcast-namespace/blob/main/docs/1.0.md',
)
def _start_podcast_transcript(self, attrs_d):
context = self._get_context()
context.setdefault('podcast_transcripts', []).append(FeedParserDict(attrs_d))
def _end_podcast_transcript(self):
pass
def install():
"""Idempotent; called when podcasts.views is imported."""
for uri in PODCAST_NAMESPACES:
FeedParserMixin.namespaces.setdefault(uri, 'podcast')
# Filled lazily from `namespaces` by the first parser; once it has
# been, later additions have to go in here directly.
if FeedParserMixin._matchnamespaces:
FeedParserMixin._matchnamespaces.setdefault(uri.lower(), 'podcast')
FeedParserMixin._start_podcast_transcript = _start_podcast_transcript
FeedParserMixin._end_podcast_transcript = _end_podcast_transcript

View file

@ -3,7 +3,7 @@ import sys
from django.core.management.base import BaseCommand
from podcasts.models import PodcastFeed
from podcasts.views import _refresh_feed
from podcasts.views import _refresh_feeds
class Command(BaseCommand):
@ -26,9 +26,6 @@ class Command(BaseCommand):
self.stdout.write('No feeds to refresh.')
return
for feed in qs:
try:
new_ep = _refresh_feed(feed)
self.stdout.write(f'{feed.title}: +{new_ep} episodes')
except Exception as e:
self.stderr.write(f'ERROR refreshing "{feed.title}" ({feed.rss_url}): {e}')
# Parallel downloads, one per distinct feed URL (see _refresh_feeds).
refreshed, failed, new = _refresh_feeds(qs)
self.stdout.write(f'{refreshed} feeds refreshed, {failed} failed, +{new} episodes')

View file

@ -0,0 +1,27 @@
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('podcasts', '0003_episodeprogress_dismissed'),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='PodcastSettings',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('enqueue_position', models.CharField(
choices=[('back', 'Ans Ende'), ('front', 'An den Anfang'), ('after_current', 'Nach der laufenden Folge')],
default='back', max_length=20)),
('auto_queue_position', models.CharField(
choices=[('back', 'Ans Ende'), ('front', 'An den Anfang'), ('after_current', 'Nach der laufenden Folge')],
default='front', max_length=20)),
('user', models.OneToOneField(
on_delete=django.db.models.deletion.CASCADE, related_name='podcast_settings',
to=settings.AUTH_USER_MODEL)),
],
),
]

View file

@ -0,0 +1,33 @@
# Generated by Django 6.1 on 2026-10-02 05:53
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('podcasts', '0004_podcastsettings'),
]
operations = [
migrations.AddField(
model_name='podcastepisode',
name='chapters',
field=models.JSONField(blank=True, default=list),
),
migrations.AddField(
model_name='podcastepisode',
name='chapters_fetched_at',
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name='podcastepisode',
name='chapters_url',
field=models.URLField(blank=True, max_length=1000),
),
migrations.AddField(
model_name='podcastepisode',
name='transcripts',
field=models.JSONField(blank=True, default=list),
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 6.1 on 2026-10-02 12:00
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('podcasts', '0005_episode_chapters_transcripts'),
]
operations = [
migrations.AddField(
model_name='episodeprogress',
name='favorite',
field=models.BooleanField(default=False),
),
]

View file

@ -34,6 +34,15 @@ class PodcastEpisode(models.Model):
season_number = models.IntegerField(null=True, blank=True)
artwork_url = models.URLField(max_length=1000, blank=True)
discovered_at = models.DateTimeField(auto_now_add=True)
# Podcasting 2.0 <podcast:chapters url=…>: a JSON file, loaded on demand.
chapters_url = models.URLField(max_length=1000, blank=True)
# [{start: seconds, title, url?, img?}], from Podlove Simple Chapters in
# the feed, or the JSON at chapters_url once it has been fetched.
chapters = models.JSONField(default=list, blank=True)
# When chapters_url was fetched into `chapters`; null = not yet.
chapters_fetched_at = models.DateTimeField(null=True, blank=True)
# [{url, type, language}] from <podcast:transcript>.
transcripts = models.JSONField(default=list, blank=True)
class Meta:
unique_together = ('feed', 'guid')
@ -43,12 +52,23 @@ class PodcastEpisode(models.Model):
return self.title
# An episode counts as played this close to its end (AntennaPod's default),
# the same rule as the apps' "als gehört zählen".
PLAYED_BEFORE_END_SECONDS = 30
def counts_as_played(position_seconds, duration_seconds):
return bool(duration_seconds) and duration_seconds > 0 and \
position_seconds >= max(duration_seconds - PLAYED_BEFORE_END_SECONDS, 1)
class EpisodeProgress(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='episode_progress')
episode = models.ForeignKey(PodcastEpisode, on_delete=models.CASCADE, related_name='progress')
position_seconds = models.IntegerField(default=0)
played = models.BooleanField(default=False)
dismissed = models.BooleanField(default=False)
favorite = models.BooleanField(default=False)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
@ -71,3 +91,29 @@ class PodcastQueue(models.Model):
def __str__(self):
return f'{self.user} queue pos {self.position}: {self.episode}'
class PodcastSettings(models.Model):
"""A user's podcast preferences. Where episodes go in the queue: when added
by hand (also when played from outside the queue), and when a feed with
auto_queue brings new ones."""
BACK = 'back'
FRONT = 'front'
AFTER_CURRENT = 'after_current'
POSITIONS = [
(BACK, 'Ans Ende'),
(FRONT, 'An den Anfang'),
(AFTER_CURRENT, 'Nach der laufenden Folge'),
]
user = models.OneToOneField(User, on_delete=models.CASCADE, related_name='podcast_settings')
enqueue_position = models.CharField(max_length=20, choices=POSITIONS, default=BACK)
# front = how auto-queue always worked before this setting existed
auto_queue_position = models.CharField(max_length=20, choices=POSITIONS, default=FRONT)
@classmethod
def for_user(cls, user):
obj, _ = cls.objects.get_or_create(user=user)
return obj
def __str__(self):
return f'Podcast settings of {self.user}'

209
podcasts/shownotes.py Normal file
View file

@ -0,0 +1,209 @@
"""Shownotes HTML: an allowlist sanitizer and timestamp extraction.
Feed HTML is written by whoever runs the podcast, so it is untrusted. feedparser
already cleans it once while parsing, but the apps render what this server hands
out in web views, so we clean it again on the way out with a deliberately small
allowlist: unknown tags are dropped (their text kept), script-like tags are
dropped with their content, and only a handful of attributes survive, with
links limited to http(s)/mailto.
"""
import html
import re
from html.parser import HTMLParser
ALLOWED_TAGS = {
'a', 'abbr', 'b', 'blockquote', 'br', 'cite', 'code', 'dd', 'del', 'div', 'dl', 'dt',
'em', 'figcaption', 'figure', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'hr', 'i', 'img',
'ins', 'li', 'mark', 'ol', 'p', 'pre', 'q', 's', 'small', 'span', 'strong', 'sub',
'sup', 'table', 'tbody', 'td', 'tfoot', 'th', 'thead', 'time', 'tr', 'u', 'ul',
}
VOID_TAGS = {'br', 'hr', 'img'}
# Dropped together with everything inside them.
DROP_CONTENT_TAGS = {
'script', 'style', 'iframe', 'object', 'embed', 'noscript', 'template', 'svg',
'math', 'head', 'title', 'frameset', 'frame', 'applet', 'audio', 'video', 'canvas',
'select', 'textarea', 'button',
}
ALLOWED_ATTRS = {
'a': {'href', 'title'},
'img': {'src', 'alt', 'title', 'width', 'height'},
'td': {'colspan', 'rowspan'},
'th': {'colspan', 'rowspan'},
'abbr': {'title'},
'time': {'datetime'},
'ol': {'start'},
}
URL_ATTRS = {'href', 'src'}
LINK_SCHEMES = {'http', 'https', 'mailto'}
IMAGE_SCHEMES = {'http', 'https'}
_CONTROL_AND_SPACE = re.compile(r'[\x00-\x20\x7f]+')
_SCHEME = re.compile(r'^([a-z][a-z0-9+.\-]*):')
def _safe_url(value, schemes):
"""The URL if it is absolute with an allowed scheme, else None.
Browsers ignore whitespace and control characters inside a scheme
("java\\tscript:"), so those are removed before looking at it.
"""
compact = _CONTROL_AND_SPACE.sub('', value or '')
m = _SCHEME.match(compact.lower())
if not m or m.group(1) not in schemes:
return None
return value.strip()
class _Sanitizer(HTMLParser):
def __init__(self):
super().__init__(convert_charrefs=True)
self.out = []
self.open = [] # allowed tags currently open, outermost first
self.dropping = 0 # depth inside DROP_CONTENT_TAGS
def handle_starttag(self, tag, attrs):
self._start(tag, attrs, closed=False)
def handle_startendtag(self, tag, attrs):
self._start(tag, attrs, closed=True)
def _start(self, tag, attrs, closed):
if tag in DROP_CONTENT_TAGS:
if not closed:
self.dropping += 1
return
if self.dropping or tag not in ALLOWED_TAGS:
return
kept = []
allowed = ALLOWED_ATTRS.get(tag, set())
for name, value in attrs:
name = name.lower()
if name not in allowed or value is None:
continue
if name in URL_ATTRS:
value = _safe_url(value, IMAGE_SCHEMES if tag == 'img' else LINK_SCHEMES)
if value is None:
continue
kept.append((name, value))
if tag == 'img' and not any(n == 'src' for n, _ in kept):
return
if tag == 'a':
kept.append(('rel', 'nofollow noopener noreferrer'))
attr_text = ''.join(f' {n}="{html.escape(v, quote=True)}"' for n, v in kept)
self.out.append(f'<{tag}{attr_text}>')
if tag not in VOID_TAGS and not closed:
self.open.append(tag)
def handle_endtag(self, tag):
if tag in DROP_CONTENT_TAGS:
if self.dropping:
self.dropping -= 1
return
if self.dropping or tag not in self.open:
return
# Close everything opened inside it too, so the output stays nested.
while self.open:
t = self.open.pop()
self.out.append(f'</{t}>')
if t == tag:
break
def handle_data(self, data):
if not self.dropping:
self.out.append(html.escape(data, quote=False))
def result(self):
self.close()
while self.open:
self.out.append(f'</{self.open.pop()}>')
return ''.join(self.out)
def sanitize_html(text):
"""Return `text` reduced to the allowlist above."""
if not text:
return ''
s = _Sanitizer()
s.feed(text)
return s.result()
# ---------------------------------------------------------------------------
# Plain text and timestamps
# ---------------------------------------------------------------------------
_BLOCK_TAGS = {
'br', 'p', 'div', 'li', 'tr', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'blockquote',
'pre', 'dd', 'dt', 'hr', 'table', 'ul', 'ol', 'figure', 'figcaption',
}
class _TextExtractor(HTMLParser):
def __init__(self):
super().__init__(convert_charrefs=True)
self.out = []
self.dropping = 0
def handle_starttag(self, tag, attrs):
if tag in DROP_CONTENT_TAGS:
self.dropping += 1
elif tag in _BLOCK_TAGS:
self.out.append('\n')
def handle_startendtag(self, tag, attrs):
if tag in _BLOCK_TAGS:
self.out.append('\n')
def handle_endtag(self, tag):
if tag in DROP_CONTENT_TAGS:
self.dropping = max(self.dropping - 1, 0)
elif tag in _BLOCK_TAGS:
self.out.append('\n')
def handle_data(self, data):
if not self.dropping:
self.out.append(data)
def html_to_text(text):
"""Shownotes as plain text, one line per block/line break."""
if not text:
return ''
p = _TextExtractor()
p.feed(text)
p.close()
return ''.join(p.out)
# A time at the start of a line, optionally in brackets, then the label:
# "12:34 Intro", "(1:02:03) - Fragen", "[00:05] Begrüßung".
_TIMESTAMP_LINE = re.compile(
r'^[\s\-*•·]*[(\[]?(?:(\d{1,2}):)?(\d{1,2}):(\d{2})(?:\.\d+)?[)\]]?(?=\s|$|[-–—:|])'
r'\s*[-–—:|]?\s*(.*)$'
)
MAX_TIMESTAMPS = 300
def extract_timestamps(text, duration_seconds=0):
"""[{seconds, label}] for lines of the shownotes that start with a time.
Only line starts are looked at: a time in the middle of a sentence is just
as likely to be a time of day.
"""
result = []
for line in html_to_text(text).splitlines():
m = _TIMESTAMP_LINE.match(line)
if not m:
continue
hours, minutes, seconds, label = m.groups()
minutes, seconds = int(minutes), int(seconds)
if seconds >= 60 or (hours is not None and minutes >= 60):
continue
total = int(hours or 0) * 3600 + minutes * 60 + seconds
if duration_seconds and total > duration_seconds:
continue
result.append({'seconds': total, 'label': ' '.join(label.split())[:300]})
if len(result) >= MAX_TIMESTAMPS:
break
return result

896
podcasts/tests.py Normal file
View file

@ -0,0 +1,896 @@
import json
from datetime import datetime, timedelta, timezone
from unittest.mock import patch
import feedparser
from django.contrib.auth.models import User
from django.test import TestCase
from .models import EpisodeProgress, PodcastEpisode, PodcastFeed, PodcastQueue, PodcastSettings
from .views import _refresh_feed
_parse = feedparser.parse
def _rss(items):
entries = ''.join(
f'<item><guid>{guid}</guid><title>{guid}</title><pubDate>{date}</pubDate>'
f'<enclosure url="https://example.org/{guid}.mp3" type="audio/mpeg" length="1"/></item>'
for guid, date in items
)
return f'<?xml version="1.0"?><rss version="2.0"><channel><title>Feed</title>{entries}</channel></rss>'
class QueuePositionTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.feed = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/feed', title='Feed')
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
self.eps = [
PodcastEpisode.objects.create(
feed=self.feed, guid=f'e{i}', title=f'E{i}',
audio_url=f'https://example.org/e{i}.mp3', pub_date=base + timedelta(days=i),
)
for i in range(6)
]
def queue(self):
return list(PodcastQueue.objects.filter(user=self.user).order_by('position').values_list('episode_id', flat=True))
def add(self, *episodes, **extra):
body = {'episode_ids': [e.id for e in episodes], **extra}
return self.client.post('/podcasts/queue/add/', json.dumps(body), content_type='application/json')
def test_default_adds_to_the_back(self):
self.add(self.eps[0])
self.add(self.eps[1])
self.assertEqual(self.queue(), [self.eps[0].id, self.eps[1].id])
def test_front_from_setting(self):
self.add(self.eps[0], self.eps[1])
PodcastSettings.objects.update_or_create(user=self.user, defaults={'enqueue_position': 'front'})
self.add(self.eps[2])
self.assertEqual(self.queue(), [self.eps[2].id, self.eps[0].id, self.eps[1].id])
def test_after_current_uses_client_hint(self):
self.add(self.eps[0], self.eps[1], self.eps[2])
self.add(self.eps[3], where='after_current', current_episode_id=self.eps[1].id)
self.assertEqual(self.queue(), [e.id for e in (self.eps[0], self.eps[1], self.eps[3], self.eps[2])])
def test_after_current_falls_back_to_last_listened(self):
self.add(self.eps[0], self.eps[1], self.eps[2])
EpisodeProgress.objects.update_or_create(user=self.user, episode=self.eps[2], defaults={'position_seconds': 30})
self.add(self.eps[3], where='after_current')
self.assertEqual(self.queue(), [e.id for e in (self.eps[0], self.eps[1], self.eps[2], self.eps[3])])
def test_after_current_without_current_goes_to_front(self):
self.add(self.eps[0])
self.add(self.eps[1], where='after_current')
self.assertEqual(self.queue(), [self.eps[1].id, self.eps[0].id])
def test_already_queued_stays_where_it_is(self):
self.add(self.eps[0], self.eps[1])
r = self.add(self.eps[0], where='front')
self.assertFalse(r.json()['created'])
self.assertEqual(self.queue(), [self.eps[0].id, self.eps[1].id])
def test_single_episode_id_still_works(self):
r = self.client.post('/podcasts/queue/add/', json.dumps({'episode_id': self.eps[4].id}),
content_type='application/json')
self.assertTrue(r.json()['created'])
self.assertEqual(self.queue(), [self.eps[4].id])
def test_other_users_episode_is_not_found(self):
bob = User.objects.create_user(username='bob', password='pw12345678')
feed = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/b', title='B')
ep = PodcastEpisode.objects.create(feed=feed, guid='x', title='X', audio_url='https://example.org/x.mp3')
r = self.add(ep)
self.assertEqual(r.status_code, 404)
def test_unknown_position_is_rejected(self):
self.assertEqual(self.add(self.eps[0], where='middle').status_code, 400)
def test_settings_roundtrip(self):
r = self.client.get('/podcasts/settings/')
self.assertEqual(r.json()['enqueue_position'], 'back')
self.assertEqual(r.json()['auto_queue_position'], 'front')
r = self.client.post('/podcasts/settings/', json.dumps({'auto_queue_position': 'after_current'}),
content_type='application/json')
self.assertEqual(r.json()['auto_queue_position'], 'after_current')
self.assertEqual(PodcastSettings.objects.get(user=self.user).auto_queue_position, 'after_current')
bad = self.client.post('/podcasts/settings/', json.dumps({'enqueue_position': 'nowhere'}),
content_type='application/json')
self.assertEqual(bad.status_code, 400)
class InboxRuleTests(TestCase):
"""Queued or started episodes leave the inbox; played means the last 30 s."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
feed = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/f', title='F')
self.eps = [PodcastEpisode.objects.create(feed=feed, guid=f'g{i}', title=f'E{i}', duration_seconds=3600,
audio_url=f'https://example.org/{i}.mp3') for i in range(3)]
def inbox(self):
return {e['id'] for e in self.client.get('/podcasts/inbox/').json()['episodes']}
def post(self, url, body):
return self.client.post(url, json.dumps(body), content_type='application/json')
def test_queued_and_started_leave_the_inbox(self):
self.assertEqual(self.inbox(), {e.id for e in self.eps})
self.post('/podcasts/queue/add/', {'episode_ids': [self.eps[0].id]})
self.post('/podcasts/progress/save/', {'episode_id': self.eps[1].id, 'position_seconds': 12})
self.assertEqual(self.inbox(), {self.eps[2].id})
p = EpisodeProgress.objects.get(user=self.user, episode=self.eps[0])
self.assertFalse(p.played)
def test_played_within_the_last_30_seconds(self):
r = self.post('/podcasts/progress/save/', {'episode_id': self.eps[0].id, 'position_seconds': 3500})
self.assertFalse(r.json()['played']) # 90 % but 100 s left: not yet
r = self.post('/podcasts/progress/save/', {'episode_id': self.eps[0].id, 'position_seconds': 3575})
self.assertTrue(r.json()['played'])
class AutoQueueTests(TestCase):
"""New episodes of an auto_queue feed land where the user's setting says."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.feed = PodcastFeed.objects.create(
user=self.user, rss_url='https://example.org/feed', title='Feed', auto_queue=True,
last_refreshed_at=datetime(2026, 1, 1, tzinfo=timezone.utc), # not the first fetch
)
other = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/o', title='O')
self.old = [
PodcastEpisode.objects.create(feed=other, guid=f'o{i}', title=f'O{i}', audio_url=f'https://example.org/o{i}.mp3')
for i in range(3)
]
for i, ep in enumerate(self.old):
PodcastQueue.objects.create(user=self.user, episode=ep, position=i)
self.rss = _rss([
('new1', 'Mon, 05 Jan 2026 10:00:00 GMT'),
('new2', 'Tue, 06 Jan 2026 10:00:00 GMT'),
])
def refresh(self):
with patch('podcasts.views._fetch_feed', lambda url: _parse(self.rss)):
_refresh_feed(self.feed)
order = PodcastQueue.objects.filter(user=self.user).order_by('position')
return [q.episode.guid for q in order]
def test_front_is_the_default(self):
self.assertEqual(self.refresh(), ['new2', 'new1', 'o0', 'o1', 'o2'])
def test_back_appends_oldest_first(self):
PodcastSettings.objects.create(user=self.user, auto_queue_position='back')
self.assertEqual(self.refresh(), ['o0', 'o1', 'o2', 'new1', 'new2'])
def test_after_current(self):
PodcastSettings.objects.create(user=self.user, auto_queue_position='after_current')
EpisodeProgress.objects.create(user=self.user, episode=self.old[1], position_seconds=12)
self.assertEqual(self.refresh(), ['o0', 'o1', 'new2', 'new1', 'o2'])
def test_feed_without_auto_queue_queues_nothing(self):
self.feed.auto_queue = False
self.feed.save()
self.assertEqual(self.refresh(), ['o0', 'o1', 'o2'])
class FirstFetchTests(TestCase):
"""A new subscription doesn't flood the inbox with its back catalogue."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.rss = _rss([(f'e{i}', f'Mon, {i + 1:02d} Jan 2026 10:00:00 GMT') for i in range(5)])
def subscribe(self):
with patch('podcasts.views._fetch_feed', lambda url: _parse(self.rss)):
return self.client.post('/podcasts/feeds/add/', json.dumps({'rss_url': 'https://example.org/f'}),
content_type='application/json')
def test_only_the_newest_is_new(self):
self.assertEqual(self.subscribe().json()['new_episodes'], 5)
inbox = self.client.get('/podcasts/inbox/').json()['episodes']
self.assertEqual([e['title'] for e in inbox], ['e4'])
# The others are still listed for the podcast, not marked played.
feed = PodcastFeed.objects.get(user=self.user)
eps = self.client.get(f'/podcasts/feeds/{feed.id}/episodes/').json()['episodes']
self.assertEqual(len(eps), 5)
self.assertFalse(any(e['played'] for e in eps))
def test_later_fetches_keep_every_new_episode(self):
self.subscribe()
feed = PodcastFeed.objects.get(user=self.user)
self.rss = _rss([(f'e{i}', f'Mon, {i + 1:02d} Jan 2026 10:00:00 GMT') for i in range(8)])
with patch('podcasts.views._fetch_feed', lambda url: _parse(self.rss)):
_refresh_feed(feed)
inbox = self.client.get('/podcasts/inbox/').json()['episodes']
self.assertEqual([e['title'] for e in inbox], ['e7', 'e6', 'e5', 'e4'])
class RefreshAllTests(TestCase):
"""One request refreshes all feeds on the server; one request returns the state."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.a = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/a', title='A')
self.b = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/b', title='B')
bob = User.objects.create_user(username='bob', password='pw12345678')
self.bobs = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/a', title='A')
self.fetched = []
def fake_fetch(self, url):
self.fetched.append(url)
if url.endswith('/b'):
raise ValueError('broken feed')
return _parse(_rss([(url[-1] + '1', 'Mon, 05 Jan 2026 10:00:00 GMT')]))
def test_refresh_all_reports_counts_and_skips_recent(self):
with patch('podcasts.views._fetch_feed', self.fake_fetch):
r = self.client.post('/podcasts/feeds/refresh-all/', '{}', content_type='application/json')
self.assertEqual(r.json(), {'ok': True, 'refreshed': 1, 'failed': 1, 'new_episodes': 1})
self.assertEqual(sorted(self.fetched), ['https://example.org/a', 'https://example.org/b'])
# A was just refreshed: a second call within max_age only retries B.
self.fetched.clear()
with patch('podcasts.views._fetch_feed', self.fake_fetch):
self.client.post('/podcasts/feeds/refresh-all/', '{}', content_type='application/json')
self.assertEqual(self.fetched, ['https://example.org/b'])
def test_a_shared_feed_url_is_fetched_once(self):
from .views import _refresh_feeds
with patch('podcasts.views._fetch_feed', self.fake_fetch):
refreshed, failed, new = _refresh_feeds([self.a, self.bobs])
self.assertEqual(self.fetched, ['https://example.org/a'])
self.assertEqual((refreshed, failed, new), (2, 0, 2))
def test_state_in_one_response(self):
ep = PodcastEpisode.objects.create(feed=self.a, guid='x', title='X', audio_url='https://example.org/x.mp3')
PodcastQueue.objects.create(user=self.user, episode=ep, position=0)
r = self.client.get('/podcasts/state/')
data = r.json()
self.assertEqual({f['title'] for f in data['feeds']}, {'A', 'B'})
self.assertEqual([q['episode__id'] for q in data['queue']], [ep.id])
self.assertEqual([e['id'] for e in data['inbox']], [ep.id])
self.assertTrue(data['inbox'][0]['in_queue'])
# Shownotes stay out of the one-request state (they can be large).
self.assertNotIn('description', data['inbox'][0])
self.assertEqual(data['settings'], {'enqueue_position': 'back', 'auto_queue_position': 'front'})
# ---------------------------------------------------------------------------
# Shownotes, chapters, transcripts
# ---------------------------------------------------------------------------
NS = (
'xmlns:podcast="https://podcastindex.org/namespace/1.0" '
'xmlns:psc="http://podlove.org/simple-chapters" '
'xmlns:content="http://purl.org/rss/1.0/modules/content/" '
'xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"'
)
def _rich_rss(items, ns=NS):
"""Like _rss, but each item is (guid, extra XML inside the <item>)."""
entries = ''.join(
f'<item><guid>{guid}</guid><title>{guid}</title><pubDate>Mon, 05 Jan 2026 10:00:00 GMT</pubDate>'
f'<enclosure url="https://example.org/{guid}.mp3" type="audio/mpeg" length="1"/>{extra}</item>'
for guid, extra in items
)
return f'<?xml version="1.0"?><rss version="2.0" {ns}><channel><title>Feed</title>{entries}</channel></rss>'
PSC = (
'<psc:chapters version="1.2">'
'<psc:chapter start="00:00:00" title="Intro"/>'
'<psc:chapter start="00:01:02.500" title="Teil &amp; zwei" href="https://x.org/2" image="https://x.org/2.jpg"/>'
'<psc:chapter start="1:02:03" title="Drei"/>'
'<psc:chapter start="12:05" title="Kurz"/>'
'</psc:chapters>'
)
TRANSCRIPTS = (
'<podcast:transcript url="https://example.org/a.vtt" type="text/vtt" language="de"/>'
'<podcast:transcript url="https://example.org/a.srt" type="application/srt" rel="captions"/>'
'<podcast:transcript url="javascript:alert(1)" type="text/html"/>'
)
CHAPTERS_JSON = {
'version': '1.2.0',
'chapters': [
{'startTime': 95.5, 'title': 'Zweites', 'url': 'https://x.org/b', 'img': 'https://x.org/b.jpg'},
{'startTime': 0, 'title': 'Erstes'},
{'startTime': 50, 'title': 'Nur Bild', 'toc': False},
{'startTime': 'kaputt', 'title': 'Ungültig'},
{'startTime': 120, 'title': 'Link', 'url': 'javascript:alert(1)'},
],
}
class FeedExtrasParsingTests(TestCase):
"""What feedparser (6.0.x) hands us for these tags, and what we store."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.feed = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/feed', title='Feed')
def refresh(self, rss):
with patch('podcasts.views._fetch_feed', lambda url: _parse(rss)):
_refresh_feed(self.feed)
def test_feedparser_keys(self):
import podcasts.views # noqa: F401 -- installs the feedparser extension
rss = _rich_rss([('a', PSC + TRANSCRIPTS + '<podcast:chapters url="https://x.org/c.json" '
'type="application/json+chapters"/>')])
entry = _parse(rss).entries[0]
self.assertEqual(entry['podcast_chapters']['url'], 'https://x.org/c.json')
self.assertEqual([t['url'] for t in entry['podcast_transcripts']],
['https://example.org/a.vtt', 'https://example.org/a.srt', 'javascript:alert(1)'])
self.assertEqual(entry['psc_chapters']['chapters'][1]['start'], '00:01:02.500')
def test_other_prefix_for_the_podcast_namespace(self):
ns = NS.replace('xmlns:podcast=', 'xmlns:pc=')
rss = _rich_rss([('a', '<pc:transcript url="https://example.org/a.vtt" type="text/vtt"/>'
'<pc:chapters url="https://x.org/c.json" type="application/json+chapters"/>')], ns)
self.refresh(rss)
ep = PodcastEpisode.objects.get()
self.assertEqual(ep.chapters_url, 'https://x.org/c.json')
self.assertEqual(ep.transcripts, [{'url': 'https://example.org/a.vtt', 'type': 'text/vtt', 'language': ''}])
def test_psc_chapters(self):
self.refresh(_rich_rss([('a', PSC)]))
ep = PodcastEpisode.objects.get()
self.assertEqual(ep.chapters, [
{'start': 0.0, 'title': 'Intro'},
{'start': 62.5, 'title': 'Teil & zwei', 'url': 'https://x.org/2', 'img': 'https://x.org/2.jpg'},
{'start': 725.0, 'title': 'Kurz'},
{'start': 3723.0, 'title': 'Drei'},
])
self.assertEqual(ep.chapters_url, '')
def test_transcripts_and_chapters_url(self):
self.refresh(_rich_rss([('a', TRANSCRIPTS + '<podcast:chapters url="https://x.org/c.json" '
'type="application/json+chapters"/>')]))
ep = PodcastEpisode.objects.get()
self.assertEqual(ep.transcripts, [
{'url': 'https://example.org/a.vtt', 'type': 'text/vtt', 'language': 'de'},
{'url': 'https://example.org/a.srt', 'type': 'application/srt', 'language': ''},
])
self.assertEqual(ep.chapters_url, 'https://x.org/c.json')
self.assertEqual(ep.chapters, [])
self.assertIsNone(ep.chapters_fetched_at)
def test_content_encoded_is_preferred(self):
self.refresh(_rich_rss([
('a', '<description>Kurz</description>'
'<content:encoded><![CDATA[<p>Lang <a href="https://x.org">Link</a></p>]]></content:encoded>'),
('b', '<description>Nur kurz</description>'),
]))
self.assertIn('Lang', PodcastEpisode.objects.get(guid='a').description)
self.assertEqual(PodcastEpisode.objects.get(guid='b').description, 'Nur kurz')
def test_existing_episodes_are_updated(self):
self.refresh(_rich_rss([('a', '<description>Alt</description>'), ('b', '<description>Gleich</description>')]))
a, b = PodcastEpisode.objects.get(guid='a'), PodcastEpisode.objects.get(guid='b')
rss = _rich_rss([
('a', '<description>Neu</description>' + PSC + TRANSCRIPTS),
('b', '<description>Gleich</description>'),
])
with patch('podcasts.views.PodcastEpisode.objects.bulk_update',
wraps=PodcastEpisode.objects.bulk_update) as bulk:
self.refresh(rss)
# Only the episode that changed is written.
self.assertEqual([e.guid for e in bulk.call_args.args[0]], ['a'])
a.refresh_from_db()
self.assertEqual(a.description, 'Neu')
self.assertEqual(len(a.chapters), 4)
self.assertEqual(len(a.transcripts), 2)
self.assertEqual(PodcastEpisode.objects.count(), 2)
# Nothing changed: nothing written.
with patch('podcasts.views.PodcastEpisode.objects.bulk_update') as bulk:
self.refresh(rss)
bulk.assert_not_called()
def test_fetched_chapters_survive_a_refresh_until_the_url_changes(self):
tag = '<podcast:chapters url="https://x.org/{}.json" type="application/json+chapters"/>'
self.refresh(_rich_rss([('a', tag.format('c1'))]))
fetched = datetime(2026, 1, 2, tzinfo=timezone.utc)
PodcastEpisode.objects.update(chapters=[{'start': 0.0, 'title': 'X'}], chapters_fetched_at=fetched)
self.refresh(_rich_rss([('a', tag.format('c1'))]))
ep = PodcastEpisode.objects.get()
self.assertEqual(ep.chapters, [{'start': 0.0, 'title': 'X'}])
self.assertEqual(ep.chapters_fetched_at, fetched)
self.refresh(_rich_rss([('a', tag.format('c2'))]))
ep.refresh_from_db()
self.assertEqual((ep.chapters_url, ep.chapters, ep.chapters_fetched_at), ('https://x.org/c2.json', [], None))
class ShownotesTests(TestCase):
def test_sanitizer(self):
from .shownotes import sanitize_html
dirty = (
'<p onclick="steal()">Hallo <a href="https://x.org/a?b=1&amp;c=2" onmouseover="x()" target="_top">Link</a>'
'<script>alert(1)</script><style>p{}</style><iframe src="https://evil.org">drin</iframe>'
'<a href="javascript:alert(1)">böse</a><a href=" JaVa\tScript:alert(1)">auch</a>'
'<img src="data:image/png;base64,AAA"><img src="https://x.org/i.png" onerror="x()">'
'<unknown>Text bleibt</unknown><b>offen'
)
self.assertEqual(
sanitize_html(dirty),
'<p>Hallo <a href="https://x.org/a?b=1&amp;c=2" rel="nofollow noopener noreferrer">Link</a>'
'<a rel="nofollow noopener noreferrer">böse</a><a rel="nofollow noopener noreferrer">auch</a>'
'<img src="https://x.org/i.png">Text bleibt<b>offen</b></p>',
)
self.assertEqual(sanitize_html('1 &lt; 2 &amp;&amp; <i>3</i>'), '1 &lt; 2 &amp;&amp; <i>3</i>')
self.assertEqual(sanitize_html('<a href="mailto:a@b.org">Mail</a>'),
'<a href="mailto:a@b.org" rel="nofollow noopener noreferrer">Mail</a>')
def test_timestamps(self):
from .shownotes import extract_timestamps
notes = (
'<p>Kapitel:<br>00:00 Intro<br>(12:34) - Thema eins<br>1:02:03 Zum Schluss</p>'
'<ul><li>[05:07] Liste</li><li>Um 20:15 geht es los</li><li>99:99 kaputt</li></ul>'
)
self.assertEqual(extract_timestamps(notes), [
{'seconds': 0, 'label': 'Intro'},
{'seconds': 754, 'label': 'Thema eins'},
{'seconds': 3723, 'label': 'Zum Schluss'},
{'seconds': 307, 'label': 'Liste'},
])
self.assertEqual(len(extract_timestamps(notes, duration_seconds=1000)), 3)
class _FakeResponse:
def __init__(self, body=b'', status=200, headers=None):
self.body, self.status_code, self.headers = body, status, headers or {}
self.raw = None
def iter_content(self, chunk_size=1):
yield self.body
def close(self):
pass
class EpisodeDetailTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.feed = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/feed', title='Feed',
artwork_url='https://example.org/art.jpg')
self.ep = PodcastEpisode.objects.create(
feed=self.feed, guid='a', title='A', audio_url='https://example.org/a.mp3', duration_seconds=3600,
pub_date=datetime(2026, 1, 5, 10, tzinfo=timezone.utc),
description='<p>Hallo<script>alert(1)</script><br>01:02 Thema <a href="https://x.org">x</a></p>',
transcripts=[{'url': 'https://example.org/a.vtt', 'type': 'text/vtt', 'language': 'de'}],
chapters_url='https://x.org/c.json',
)
self.requests = []
def fake_get(self, url, **kwargs):
self.requests.append(url)
return _FakeResponse(json.dumps(CHAPTERS_JSON).encode())
def get(self, ep=None):
with patch('podcasts.chapters.assert_safe_url'), patch('podcasts.chapters.requests.get', self.fake_get):
return self.client.get(f'/podcasts/episodes/{(ep or self.ep).id}/')
def test_detail(self):
EpisodeProgress.objects.create(user=self.user, episode=self.ep, position_seconds=42)
r = self.get()
self.assertEqual(r.status_code, 200)
data = r.json()
self.assertEqual(data['description'], '<p>Hallo<br>01:02 Thema <a href="https://x.org" '
'rel="nofollow noopener noreferrer">x</a></p>')
self.assertEqual(data['timestamps'], [{'seconds': 62, 'label': 'Thema x'}])
self.assertEqual(data['transcripts'], self.ep.transcripts)
self.assertEqual(data['feed'], {'id': self.feed.id, 'title': 'Feed',
'artwork_url': 'https://example.org/art.jpg'})
self.assertEqual((data['position_seconds'], data['played'], data['in_queue']), (42, False, False))
self.assertEqual(data['pub_date'], '2026-01-05T10:00:00+00:00')
self.assertEqual(data['chapters'], [
{'start': 0.0, 'title': 'Erstes'},
{'start': 95.5, 'title': 'Zweites', 'url': 'https://x.org/b', 'img': 'https://x.org/b.jpg'},
{'start': 120.0, 'title': 'Link'},
])
def test_chapters_are_fetched_once(self):
self.get()
self.get()
self.assertEqual(self.requests, ['https://x.org/c.json'])
self.ep.refresh_from_db()
self.assertEqual(len(self.ep.chapters), 3)
self.assertIsNotNone(self.ep.chapters_fetched_at)
def test_inline_chapters_need_no_fetch(self):
self.ep.chapters = [{'start': 0.0, 'title': 'Inline'}]
self.ep.save()
self.assertEqual(self.get().json()['chapters'], [{'start': 0.0, 'title': 'Inline'}])
self.assertEqual(self.requests, [])
def test_redirect_is_followed_and_rechecked(self):
def fake_get(url, **kwargs):
self.requests.append(url)
if url.endswith('c.json'):
return _FakeResponse(status=302, headers={'Location': '/real.json'})
return _FakeResponse(json.dumps(CHAPTERS_JSON).encode())
with patch('podcasts.chapters.assert_safe_url') as check, patch('podcasts.chapters.requests.get', fake_get):
r = self.client.get(f'/podcasts/episodes/{self.ep.id}/')
self.assertEqual(len(r.json()['chapters']), 3)
self.assertEqual([c.args[0] for c in check.call_args_list], ['https://x.org/c.json', 'https://x.org/real.json'])
def test_private_chapters_url_is_refused_and_remembered(self):
self.ep.chapters_url = 'http://127.0.0.1/c.json'
self.ep.save()
with patch('podcasts.chapters.requests.get', self.fake_get):
r = self.client.get(f'/podcasts/episodes/{self.ep.id}/')
self.assertEqual(r.json()['chapters'], [])
self.assertEqual(self.requests, [])
self.ep.refresh_from_db()
self.assertIsNotNone(self.ep.chapters_fetched_at)
def test_oversized_or_broken_chapters(self):
from . import chapters
with patch('podcasts.chapters.assert_safe_url'), patch(
'podcasts.chapters.requests.get',
lambda url, **kw: _FakeResponse(b'x' * (chapters.CHAPTERS_MAX_BYTES + 1)),
):
with self.assertRaises(chapters.ChaptersInvalid):
chapters.fetch('https://x.org/big.json')
with patch('podcasts.chapters.assert_safe_url'), patch(
'podcasts.chapters.requests.get', lambda url, **kw: _FakeResponse(b'{"chapters": 3}'),
):
with self.assertRaises(chapters.ChaptersInvalid):
chapters.fetch('https://x.org/odd.json')
def test_server_error_is_retried_later(self):
with patch('podcasts.chapters.assert_safe_url'), patch(
'podcasts.chapters.requests.get', lambda url, **kw: _FakeResponse(status=503),
):
r = self.client.get(f'/podcasts/episodes/{self.ep.id}/')
self.assertEqual(r.json()['chapters'], [])
self.ep.refresh_from_db()
self.assertIsNone(self.ep.chapters_fetched_at)
def test_other_users_episode_is_not_found(self):
bob = User.objects.create_user(username='bob', password='pw12345678')
feed = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/b', title='B')
ep = PodcastEpisode.objects.create(feed=feed, guid='x', title='X', audio_url='https://example.org/x.mp3',
chapters_url='https://x.org/c.json')
self.assertEqual(self.get(ep).status_code, 404)
self.assertEqual(self.requests, [])
def test_login_required(self):
self.client.logout()
self.assertEqual(self.get().status_code, 401)
def test_favorite_and_last_played_at(self):
r = self.get().json()
self.assertEqual((r['favorite'], r['last_played_at']), (False, None))
p = EpisodeProgress.objects.create(user=self.user, episode=self.ep, position_seconds=5, favorite=True)
EpisodeProgress.objects.filter(pk=p.pk).update(updated_at=datetime(2026, 2, 3, 8, tzinfo=timezone.utc))
r = self.get().json()
self.assertEqual((r['favorite'], r['last_played_at']), (True, '2026-02-03T08:00:00+00:00'))
# ---------------------------------------------------------------------------
# Favorites, episode list, statistics, OPML export
# ---------------------------------------------------------------------------
def _at(month, day):
return datetime(2026, month, day, 9, tzinfo=timezone.utc)
def _progress(user, episode, updated_at, **fields):
p = EpisodeProgress.objects.create(user=user, episode=episode, **fields)
# auto_now would overwrite updated_at on save(); update() doesn't.
EpisodeProgress.objects.filter(pk=p.pk).update(updated_at=updated_at)
return p
class FavoriteTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
feed = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/feed', title='Feed')
self.ep = PodcastEpisode.objects.create(feed=feed, guid='a', title='A', audio_url='https://example.org/a.mp3')
def post(self, body=None, ep=None):
return self.client.post(f'/podcasts/episodes/{(ep or self.ep).id}/favorite/',
'' if body is None else json.dumps(body), content_type='application/json')
def test_toggle_creates_the_progress_row(self):
self.assertFalse(EpisodeProgress.objects.exists())
self.assertEqual(self.post().json(), {'ok': True, 'favorite': True})
p = EpisodeProgress.objects.get(user=self.user, episode=self.ep)
self.assertTrue(p.favorite)
self.assertEqual((p.played, p.dismissed, p.position_seconds), (False, False, 0))
self.assertEqual(self.post({}).json(), {'ok': True, 'favorite': False})
self.assertFalse(EpisodeProgress.objects.get(pk=p.pk).favorite)
def test_set_explicitly(self):
self.assertEqual(self.post({'favorite': True}).json()['favorite'], True)
self.assertEqual(self.post({'favorite': True}).json()['favorite'], True)
self.assertTrue(EpisodeProgress.objects.get().favorite)
self.assertEqual(self.post({'favorite': False}).json()['favorite'], False)
self.assertEqual(self.post({'favorite': False}).json()['favorite'], False)
self.assertFalse(EpisodeProgress.objects.get().favorite)
def test_starring_keeps_the_listening_time(self):
p = _progress(self.user, self.ep, _at(2, 1), position_seconds=30)
self.post({'favorite': True})
p.refresh_from_db()
self.assertTrue(p.favorite)
self.assertEqual((p.position_seconds, p.updated_at), (30, _at(2, 1)))
def test_owner_only(self):
bob = User.objects.create_user(username='bob', password='pw12345678')
feed = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/b', title='B')
ep = PodcastEpisode.objects.create(feed=feed, guid='x', title='X', audio_url='https://example.org/x.mp3')
self.assertEqual(self.post({'favorite': True}, ep).status_code, 404)
self.assertFalse(EpisodeProgress.objects.exists())
self.assertEqual(self.client.post('/podcasts/episodes/999999/favorite/', '{}',
content_type='application/json').status_code, 404)
def test_login_and_post_required(self):
self.assertEqual(self.client.get(f'/podcasts/episodes/{self.ep.id}/favorite/').status_code, 405)
self.client.logout()
self.assertEqual(self.post({'favorite': True}).status_code, 401)
class EpisodeListTests(TestCase):
"""GET /podcasts/episodes/: all feeds, filters, history, search, pages."""
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.fa = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/a', title='Alpha',
artwork_url='https://example.org/a.jpg')
self.fb = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/b', title='Beta')
def ep(feed, guid, title, pub_date, **kw):
return PodcastEpisode.objects.create(feed=feed, guid=guid, title=title, pub_date=pub_date,
audio_url=f'https://example.org/{guid}.mp3',
description='x' * 1000, **kw)
self.a1 = ep(self.fa, 'a1', 'A1', _at(1, 1), duration_seconds=600)
self.a2 = ep(self.fa, 'a2', 'A2', _at(1, 2))
self.a3 = ep(self.fa, 'a3', 'A3', _at(1, 3))
self.a4 = ep(self.fa, 'a4', 'A4 ohne Datum', None)
self.b1 = ep(self.fb, 'b1', 'Bonus Talk', _at(1, 4))
self.b2 = ep(self.fb, 'b2', 'B2', _at(1, 5))
_progress(self.user, self.a1, _at(2, 1), played=True)
_progress(self.user, self.a2, _at(2, 3), position_seconds=100)
_progress(self.user, self.b1, _at(2, 2), position_seconds=50)
_progress(self.user, self.a3, _at(2, 5), favorite=True)
_progress(self.user, self.b2, _at(2, 4), dismissed=True) # e.g. back catalogue of a new subscription
PodcastQueue.objects.create(user=self.user, episode=self.a2, position=0)
# Someone else's listening must not show up.
bob = User.objects.create_user(username='bob', password='pw12345678')
fbob = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/a', title='Alpha')
xb = PodcastEpisode.objects.create(feed=fbob, guid='a1', title='A1', audio_url='https://example.org/a1.mp3',
pub_date=_at(1, 9))
_progress(bob, xb, _at(3, 1), played=True, favorite=True)
_progress(bob, self.a3, _at(3, 1), played=True, position_seconds=10) # can't happen, but must not leak
def get(self, **params):
r = self.client.get('/podcasts/episodes/', params)
self.assertEqual(r.status_code, 200)
return r.json()
def ids(self, **params):
return [e['id'] for e in self.get(**params)['episodes']]
def test_all_newest_first_without_date_last(self):
data = self.get()
self.assertEqual([e['id'] for e in data['episodes']],
[e.id for e in (self.b2, self.b1, self.a3, self.a2, self.a1, self.a4)])
self.assertEqual((data['offset'], data['limit'], data['total']), (0, 100, 6))
self.assertEqual(self.ids(filter='all'), [e['id'] for e in data['episodes']])
def test_episode_fields(self):
eps = {e['id']: e for e in self.get()['episodes']}
self.assertEqual(eps[self.a2.id], {
'id': self.a2.id, 'title': 'A2', 'audio_url': 'https://example.org/a2.mp3', 'duration_seconds': 0,
'pub_date': '2026-01-02T09:00:00+00:00', 'artwork_url': '',
'feed__id': self.fa.id, 'feed__title': 'Alpha', 'feed__artwork_url': 'https://example.org/a.jpg',
'position_seconds': 100, 'played': False, 'favorite': False, 'in_queue': True,
'last_played_at': '2026-02-03T09:00:00+00:00',
})
a4 = eps[self.a4.id]
self.assertEqual((a4['pub_date'], a4['last_played_at'], a4['position_seconds'], a4['played'],
a4['favorite'], a4['in_queue']), (None, None, 0, False, False, False))
self.assertEqual(eps[self.a3.id]['favorite'], True)
self.assertEqual(eps[self.a3.id]['played'], False) # bob's row doesn't count
self.assertEqual(eps[self.a1.id]['played'], True)
def test_unplayed(self):
# Dismissed episodes (b2) are still listed.
self.assertEqual(self.ids(filter='unplayed'), [e.id for e in (self.b2, self.b1, self.a3, self.a2, self.a4)])
def test_in_progress_by_publication_date(self):
self.assertEqual(self.ids(filter='in_progress'), [self.b1.id, self.a2.id])
def test_played(self):
self.assertEqual(self.ids(filter='played'), [self.a1.id])
def test_favorites(self):
self.assertEqual(self.ids(filter='favorites'), [self.a3.id])
def test_history_last_listened_first(self):
data = self.get(filter='history')
self.assertEqual([e['id'] for e in data['episodes']], [self.a2.id, self.b1.id, self.a1.id])
self.assertEqual([e['last_played_at'] for e in data['episodes']],
['2026-02-03T09:00:00+00:00', '2026-02-02T09:00:00+00:00', '2026-02-01T09:00:00+00:00'])
self.assertEqual(data['total'], 3)
# Listening again moves an episode to the top.
self.client.post('/podcasts/progress/save/', json.dumps({'episode_id': self.a1.id, 'position_seconds': 20}),
content_type='application/json')
self.assertEqual(self.ids(filter='history'), [self.a1.id, self.a2.id, self.b1.id])
def test_feed_id_and_q(self):
self.assertEqual(self.ids(feed_id=self.fb.id), [self.b2.id, self.b1.id])
self.assertEqual(self.ids(q='BONUS'), [self.b1.id])
self.assertEqual(self.ids(q='ohne dat'), [self.a4.id])
self.assertEqual(self.ids(filter='history', feed_id=self.fa.id), [self.a2.id, self.a1.id])
self.assertEqual(self.ids(filter='in_progress', q='a2'), [self.a2.id])
data = self.get(feed_id=self.fb.id, q='nothing')
self.assertEqual((data['episodes'], data['total']), ([], 0))
bobs_feed = PodcastFeed.objects.get(user__username='bob')
self.assertEqual(self.ids(feed_id=bobs_feed.id), [])
def test_pages(self):
data = self.get(offset=1, limit=2)
self.assertEqual([e['id'] for e in data['episodes']], [self.b1.id, self.a3.id])
self.assertEqual((data['offset'], data['limit'], data['total']), (1, 2, 6))
data = self.get(offset=5, limit=2)
self.assertEqual(([e['id'] for e in data['episodes']], data['total']), ([self.a4.id], 6))
data = self.get(offset=10)
self.assertEqual((data['episodes'], data['total']), ([], 6))
self.assertEqual(self.get(limit=10000)['limit'], 500)
self.assertEqual(self.get(offset=-3)['offset'], 0)
data = self.get(filter='history', limit=1, offset=1)
self.assertEqual(([e['id'] for e in data['episodes']], data['total']), ([self.b1.id], 3))
def test_query_count_does_not_grow_with_the_page(self):
from .views import _episode_list_data
for i in range(20):
e = PodcastEpisode.objects.create(feed=self.fb, guid=f'n{i}', title=f'N{i}',
audio_url=f'https://example.org/n{i}.mp3', pub_date=_at(1, 10))
_progress(self.user, e, _at(2, 10), position_seconds=i + 1)
PodcastQueue.objects.create(user=self.user, episode=e, position=i + 1)
with self.assertNumQueries(2):
episodes, total = _episode_list_data(self.user, 'history')
self.assertEqual((len(episodes), total), (23, 23))
def test_bad_parameters(self):
self.assertEqual(self.client.get('/podcasts/episodes/', {'filter': 'new'}).status_code, 400)
self.assertEqual(self.client.get('/podcasts/episodes/', {'limit': 'ten'}).status_code, 400)
self.assertEqual(self.client.get('/podcasts/episodes/', {'feed_id': 'x'}).status_code, 400)
def test_login_required(self):
self.client.logout()
self.assertEqual(self.client.get('/podcasts/episodes/').status_code, 401)
class StatsTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
self.fa = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/a', title='Alpha',
artwork_url='https://example.org/a.jpg')
self.fb = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/b', title='Beta')
self.fc = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/c', title='Gamma')
def ep(feed, guid, duration):
return PodcastEpisode.objects.create(feed=feed, guid=guid, title=guid, duration_seconds=duration,
audio_url=f'https://example.org/{guid}.mp3')
now = datetime.now(timezone.utc)
_progress(self.user, ep(self.fa, 'a1', 3600), now, played=True, position_seconds=50) # 3600
_progress(self.user, ep(self.fa, 'a2', 1000), now, position_seconds=300) # 300, started
_progress(self.user, ep(self.fa, 'a3', 0), now, played=True, position_seconds=120) # 120: no length
_progress(self.user, ep(self.fb, 'b1', 900), now - timedelta(days=30), position_seconds=200) # 200, started
_progress(self.user, ep(self.fc, 'c1', 500), now, dismissed=True, favorite=True) # 0
bob = User.objects.create_user(username='bob', password='pw12345678')
fbob = PodcastFeed.objects.create(user=bob, rss_url='https://example.org/a', title='Alpha')
_progress(bob, ep(fbob, 'x', 999), now, played=True)
def test_all_time(self):
r = self.client.get('/podcasts/stats/')
self.assertEqual(r.status_code, 200)
self.assertEqual(r.json(), {
'total_seconds': 4220,
'episodes_played': 2,
'episodes_started': 2,
'since': None,
'feeds': [
{'feed_id': self.fa.id, 'title': 'Alpha', 'artwork_url': 'https://example.org/a.jpg',
'seconds': 4020, 'episodes_played': 2},
{'feed_id': self.fb.id, 'title': 'Beta', 'artwork_url': '', 'seconds': 200, 'episodes_played': 0},
],
})
def test_last_days(self):
before = datetime.now(timezone.utc)
data = self.client.get('/podcasts/stats/', {'days': 7}).json()
self.assertEqual((data['total_seconds'], data['episodes_played'], data['episodes_started']), (4020, 2, 1))
self.assertEqual([f['feed_id'] for f in data['feeds']], [self.fa.id])
since = datetime.fromisoformat(data['since'])
self.assertAlmostEqual((before - since).total_seconds(), 7 * 86400, delta=60)
data = self.client.get('/podcasts/stats/', {'days': 60}).json()
self.assertEqual(data['total_seconds'], 4220)
self.assertEqual([f['feed_id'] for f in data['feeds']], [self.fa.id, self.fb.id])
def test_empty(self):
EpisodeProgress.objects.filter(user=self.user).delete()
self.assertEqual(self.client.get('/podcasts/stats/').json(), {
'total_seconds': 0, 'episodes_played': 0, 'episodes_started': 0, 'since': None, 'feeds': [],
})
def test_bad_days_and_login(self):
self.assertEqual(self.client.get('/podcasts/stats/', {'days': 'week'}).status_code, 400)
self.assertEqual(self.client.get('/podcasts/stats/', {'days': -1}).status_code, 400)
self.client.logout()
self.assertEqual(self.client.get('/podcasts/stats/').status_code, 401)
class OpmlExportTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(username='alice', password='pw12345678')
self.client.force_login(self.user)
PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/feed?a=1&b=2',
title='Tom & Jerry "Live" <Folge>', link='https://example.org/?x=1&y=2')
PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/plain', title="Bob's Podcast")
bob = User.objects.create_user(username='bob', password='pw12345678')
PodcastFeed.objects.create(user=bob, rss_url='https://example.org/secret', title='Secret')
def test_export(self):
import xml.etree.ElementTree as ET
r = self.client.get('/podcasts/feeds/export.opml')
self.assertEqual(r.status_code, 200)
self.assertEqual(r['Content-Type'], 'text/x-opml; charset=utf-8')
self.assertEqual(r['Content-Disposition'], 'attachment; filename="diora-podcasts.opml"')
raw = r.content.decode('utf-8')
self.assertIn('Tom &amp; Jerry &quot;Live&quot; &lt;Folge&gt;', raw)
self.assertIn('xmlUrl="https://example.org/feed?a=1&amp;b=2"', raw)
self.assertNotIn('Secret', raw)
root = ET.fromstring(r.content)
self.assertEqual(root.tag, 'opml')
self.assertEqual(root.get('version'), '2.0')
self.assertEqual(root.findtext('head/title'), 'diora Podcasts')
outlines = [o.attrib for o in root.iter('outline')]
self.assertEqual(outlines, [
{'type': 'rss', 'text': "Bob's Podcast", 'title': "Bob's Podcast",
'xmlUrl': 'https://example.org/plain', 'htmlUrl': ''},
{'type': 'rss', 'text': 'Tom & Jerry "Live" <Folge>', 'title': 'Tom & Jerry "Live" <Folge>',
'xmlUrl': 'https://example.org/feed?a=1&b=2', 'htmlUrl': 'https://example.org/?x=1&y=2'},
])
def test_round_trip_through_import(self):
from django.core.files.uploadedfile import SimpleUploadedFile
opml = self.client.get('/podcasts/feeds/export.opml').content
carol = User.objects.create_user(username='carol', password='pw12345678')
self.client.force_login(carol)
r = self.client.post('/podcasts/feeds/import/',
{'file': SimpleUploadedFile('diora-podcasts.opml', opml, 'text/x-opml')})
self.assertEqual(r.json(), {'ok': True, 'added': 2, 'skipped': 0})
mine = lambda u: set(PodcastFeed.objects.filter(user=u).values_list('title', 'rss_url'))
self.assertEqual(mine(carol), mine(self.user))
# Importing the same file again adds nothing.
r = self.client.post('/podcasts/feeds/import/',
{'file': SimpleUploadedFile('diora-podcasts.opml', opml, 'text/x-opml')})
self.assertEqual(r.json(), {'ok': True, 'added': 0, 'skipped': 2})
def test_no_feeds(self):
import xml.etree.ElementTree as ET
PodcastFeed.objects.filter(user=self.user).delete()
root = ET.fromstring(self.client.get('/podcasts/feeds/export.opml').content)
self.assertEqual(list(root.iter('outline')), [])
def test_login_required(self):
self.client.logout()
self.assertEqual(self.client.get('/podcasts/feeds/export.opml').status_code, 401)

View file

@ -6,14 +6,22 @@ urlpatterns = [
path('feeds/', views.feed_list, name='podcast_feed_list'),
path('feeds/add/', views.add_feed, name='podcast_add_feed'),
path('feeds/import/', views.import_opml, name='podcast_import_opml'),
path('feeds/export.opml', views.export_opml, name='podcast_export_opml'),
path('feeds/refresh/', views.refresh_feed_now, name='podcast_refresh_feed'),
path('feeds/refresh-all/', views.refresh_all_feeds, name='podcast_refresh_all_feeds'),
path('state/', views.podcast_state, name='podcast_state'),
path('feeds/<int:pk>/remove/', views.remove_feed, name='podcast_remove_feed'),
path('feeds/<int:pk>/set-auto-queue/', views.set_auto_queue, name='podcast_set_auto_queue'),
path('feeds/<int:pk>/episodes/', views.feed_episodes, name='podcast_feed_episodes'),
path('episodes/', views.episode_list, name='podcast_episode_list'),
path('episodes/<int:pk>/', views.episode_detail, name='podcast_episode_detail'),
path('episodes/<int:pk>/favorite/', views.set_favorite, name='podcast_set_favorite'),
path('stats/', views.podcast_stats, name='podcast_stats'),
path('queue/', views.queue_get, name='podcast_queue_get'),
path('queue/add/', views.queue_add, name='podcast_queue_add'),
path('queue/remove/', views.queue_remove, name='podcast_queue_remove'),
path('queue/reorder/', views.queue_reorder, name='podcast_queue_reorder'),
path('settings/', views.podcast_settings, name='podcast_settings'),
path('progress/save/', views.save_progress, name='podcast_save_progress'),
path('progress/mark-played/', views.mark_played, name='podcast_mark_played'),
path('progress/dismiss/', views.dismiss_episodes, name='podcast_dismiss_episodes'),

View file

@ -1,17 +1,119 @@
import json
import logging
import urllib.parse
from concurrent.futures import ThreadPoolExecutor
from datetime import timedelta
import xml.etree.ElementTree as ET
import feedparser
import requests
from django.conf import settings
from django.db.models import Count, F, Max, Q
from django.http import JsonResponse
from django.db.models import (
Case, Count, Exists, F, FilteredRelation, IntegerField, Max, OuterRef, Q, Sum, Value, When,
)
from django.db.models.functions import Coalesce
from django.http import HttpResponse, JsonResponse
from django.utils import timezone
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods
from .models import EpisodeProgress, PodcastEpisode, PodcastFeed, PodcastQueue
from . import chapters as chapters_mod
from . import feedext
from .models import EpisodeProgress, PodcastEpisode, PodcastFeed, PodcastQueue, PodcastSettings, counts_as_played
from .shownotes import extract_timestamps, sanitize_html
feedext.install()
log = logging.getLogger(__name__)
FEED_TIMEOUT = 20 # seconds per feed download
NEW_ON_SUBSCRIBE = 1 # episodes of a new subscription that count as new
FEED_WORKERS = 8 # feeds downloaded at the same time
MAX_DESCRIPTION = 50_000 # characters of shownotes HTML kept per episode
MAX_TRANSCRIPTS = 10
# ---------------------------------------------------------------------------
# Queue positions
# ---------------------------------------------------------------------------
QUEUE_POSITIONS = {key for key, _ in PodcastSettings.POSITIONS}
def _queue_order(user):
"""Episode ids in the user's queue, first to last."""
return list(
PodcastQueue.objects.filter(user=user)
.order_by('position', 'added_at', 'id')
.values_list('episode_id', flat=True)
)
def _write_queue_order(user, order):
"""Renumber the queue 0..n-1 in the given order."""
rows = {q.episode_id: q for q in PodcastQueue.objects.filter(user=user)}
changed = []
for pos, ep_id in enumerate(order):
row = rows.get(ep_id)
if row is not None and row.position != pos:
row.position = pos
changed.append(row)
if changed:
PodcastQueue.objects.bulk_update(changed, ['position'])
def _current_episode_id(user, queued, hint=None):
"""The episode being listened to: the client's word for it, else the
queued episode most recently listened to and not finished."""
if hint in queued:
return hint
p = (
EpisodeProgress.objects
.filter(user=user, episode_id__in=queued, played=False, position_seconds__gt=0)
.order_by('-updated_at')
.values_list('episode_id', flat=True)
.first()
)
return p
def _enqueue(user, episode_ids, where, current_hint=None):
"""Put episodes that aren't queued yet into the queue at `where`
(back, front or after_current), keeping their given order.
Returns the ids that were added."""
queued = _queue_order(user)
present = set(queued)
new = []
for ep_id in episode_ids:
if ep_id not in present and ep_id not in new:
new.append(ep_id)
if not new:
return []
if where == PodcastSettings.FRONT:
at = 0
elif where == PodcastSettings.AFTER_CURRENT:
current = _current_episode_id(user, queued, current_hint)
at = queued.index(current) + 1 if current in queued else 0
else:
at = len(queued)
PodcastQueue.objects.bulk_create(
[PodcastQueue(user=user, episode_id=ep_id, position=len(queued) + i) for i, ep_id in enumerate(new)],
ignore_conflicts=True,
)
_write_queue_order(user, queued[:at] + new + queued[at:])
_leave_inbox(user, new)
return new
def _leave_inbox(user, episode_ids):
"""Queued or started episodes are no longer new (AntennaPod's rule): out
of the inbox, without counting as played."""
EpisodeProgress.objects.filter(user=user, episode_id__in=episode_ids, dismissed=False).update(dismissed=True)
have = set(EpisodeProgress.objects.filter(user=user, episode_id__in=episode_ids).values_list('episode_id', flat=True))
EpisodeProgress.objects.bulk_create(
[EpisodeProgress(user=user, episode_id=ep_id, dismissed=True) for ep_id in episode_ids if ep_id not in have],
ignore_conflicts=True,
)
# ---------------------------------------------------------------------------
@ -36,14 +138,132 @@ def _parse_duration(raw):
return 0
# ---------------------------------------------------------------------------
# Shownotes, chapters and transcripts of a feed entry
# ---------------------------------------------------------------------------
def _entry_description(entry):
"""Shownotes HTML: content:encoded when the feed has it (that's where the
full notes usually are), else the description/summary."""
for c in entry.get('content') or []:
value = (c.get('value') or '').strip()
if value:
return value[:MAX_DESCRIPTION]
return (entry.get('summary') or entry.get('description') or '')[:MAX_DESCRIPTION]
def _entry_chapters_url(entry):
url = ((entry.get('podcast_chapters') or {}).get('url') or '').strip()
return url[:1000] if url.lower().startswith(('http://', 'https://')) else ''
def _entry_transcripts(entry):
raw = entry.get('podcast_transcripts')
if raw is None and entry.get('podcast_transcript'):
raw = [entry['podcast_transcript']] # feedparser without feedext
transcripts = []
for t in raw or []:
url = (t.get('url') or '').strip()
if not url.lower().startswith(('http://', 'https://')) or len(url) > 1000:
continue
transcripts.append({
'url': url,
'type': (t.get('type') or '')[:100],
'language': (t.get('language') or '')[:35],
})
if len(transcripts) >= MAX_TRANSCRIPTS:
break
return transcripts
def _apply_extras(episode, description, chapters_url, psc_chapters, transcripts):
"""Copy what the feed says now onto `episode`; True if anything changed.
Inline (Podlove) chapters win. Without them, chapters fetched from
chapters_url stay cached for as long as the URL stays the same.
"""
changed = []
if episode.description != description:
episode.description = description
changed.append('description')
if episode.transcripts != transcripts:
episode.transcripts = transcripts
changed.append('transcripts')
if psc_chapters:
chapters, fetched_at = psc_chapters, episode.chapters_fetched_at
elif chapters_url and chapters_url == episode.chapters_url and episode.chapters_fetched_at:
chapters, fetched_at = episode.chapters, episode.chapters_fetched_at
else:
chapters, fetched_at = [], None
if episode.chapters_url != chapters_url:
episode.chapters_url = chapters_url
changed.append('chapters_url')
if episode.chapters != chapters:
episode.chapters = chapters
changed.append('chapters')
if episode.chapters_fetched_at != fetched_at:
episode.chapters_fetched_at = fetched_at
changed.append('chapters_fetched_at')
return bool(changed)
EXTRA_FIELDS = ['description', 'transcripts', 'chapters_url', 'chapters', 'chapters_fetched_at']
# ---------------------------------------------------------------------------
# Shared feed refresh helper
# ---------------------------------------------------------------------------
def _refresh_feed(feed_obj):
def _fetch_feed(url):
"""Download and parse a feed (with a timeout, unlike feedparser's own fetch)."""
r = requests.get(url, timeout=FEED_TIMEOUT, headers={'User-Agent': 'diora podcast refresh'})
r.raise_for_status()
return feedparser.parse(r.content, response_headers={'content-location': r.url, **r.headers})
def _refresh_feeds(feeds, chunk=32):
"""Refresh many feeds at once: the downloads run in parallel and each
feed URL is fetched once, even when several users subscribed to it; the
database writes then happen one by one. Returns (refreshed, failed, new)."""
feeds = list(feeds)
refreshed = failed = new = 0
for start in range(0, len(feeds), chunk):
part = feeds[start:start + chunk]
urls = sorted({f.rss_url for f in part})
results = {}
def fetch(url):
try:
return url, _fetch_feed(url), None
except Exception as e: # network, HTTP or parser trouble: report and go on
return url, None, e
with ThreadPoolExecutor(max_workers=min(FEED_WORKERS, len(urls) or 1)) as pool:
for url, parsed, err in pool.map(fetch, urls):
results[url] = (parsed, err)
for feed in part:
parsed, err = results[feed.rss_url]
if err is not None:
failed += 1
log.warning('refreshing %s failed: %s', feed.rss_url, err)
continue
try:
new += _refresh_feed(feed, parsed)
refreshed += 1
except Exception as e:
failed += 1
log.warning('updating %s failed: %s', feed.rss_url, e)
return refreshed, failed, new
def _refresh_feed(feed_obj, parsed=None):
"""Parse rss_url and upsert episodes. Returns count of new episodes."""
parsed = feedparser.parse(feed_obj.rss_url)
if parsed is None:
parsed = _fetch_feed(feed_obj.rss_url)
channel = parsed.feed
# The first fetch of a new subscription brings the whole back catalogue.
first_fetch = feed_obj.last_refreshed_at is None
feed_obj.title = channel.get('title', feed_obj.title or feed_obj.rss_url)[:300]
feed_obj.description = channel.get('subtitle', channel.get('description', ''))[:2000]
@ -64,8 +284,18 @@ def _refresh_feed(feed_obj):
max_ep = getattr(settings, 'PODCAST_MAX_EPISODES_PER_FEED', 200)
new_count = 0
new_episodes_list = []
entries = parsed.entries[:max_ep]
for entry in parsed.entries[:max_ep]:
# Episodes we already have, in one query, so shownotes/chapters/transcripts
# that changed in the feed can be brought up to date (only those rows).
guids = [str(e.get('id') or e.get('guid') or '')[:1000] for e in entries]
existing = {
ep.guid: ep
for ep in feed_obj.episodes.filter(guid__in=[g for g in guids if g]).only('id', 'guid', *EXTRA_FIELDS)
}
changed_episodes = {}
for entry in entries:
# Find audio enclosure
audio_url = ''
for enc in entry.get('enclosures', []):
@ -79,7 +309,16 @@ def _refresh_feed(feed_obj):
guid = entry.get('id') or entry.get('guid') or audio_url
guid = str(guid)[:1000]
title = entry.get('title', 'Untitled')[:500]
description = entry.get('summary', entry.get('description', ''))
description = _entry_description(entry)
chapters_url = _entry_chapters_url(entry)
psc_chapters = chapters_mod.from_psc(entry)
transcripts = _entry_transcripts(entry)
known = existing.get(guid)
if known is not None:
if _apply_extras(known, description, chapters_url, psc_chapters, transcripts):
changed_episodes[known.id] = known
continue
duration_raw = entry.get('itunes_duration', 0)
duration_seconds = _parse_duration(duration_raw)
@ -116,20 +355,43 @@ def _refresh_feed(feed_obj):
'episode_number': ep_number,
'season_number': season_number,
'artwork_url': artwork_url,
'chapters_url': chapters_url,
'chapters': psc_chapters,
'transcripts': transcripts,
},
)
existing[guid] = episode_obj # the same guid twice in one feed
if created:
new_count += 1
new_episodes_list.append(episode_obj)
elif _apply_extras(episode_obj, description, chapters_url, psc_chapters, transcripts):
changed_episodes[episode_obj.id] = episode_obj
if changed_episodes:
PodcastEpisode.objects.bulk_update(list(changed_episodes.values()), EXTRA_FIELDS, batch_size=100)
if first_fetch and len(new_episodes_list) > NEW_ON_SUBSCRIBE:
# Only the newest episodes count as new; the rest stay in the
# podcast's episode list, out of the inbox (dismissed, not played).
newest_first = sorted(new_episodes_list, key=lambda e: e.pub_date or e.discovered_at, reverse=True)
older = newest_first[NEW_ON_SUBSCRIBE:]
EpisodeProgress.objects.bulk_create(
[EpisodeProgress(user=feed_obj.user, episode=e, dismissed=True) for e in older],
ignore_conflicts=True,
)
new_episodes_list = newest_first[:NEW_ON_SUBSCRIBE]
if feed_obj.auto_queue and new_episodes_list:
count = len(new_episodes_list)
PodcastQueue.objects.filter(user=feed_obj.user).update(position=F('position') + count)
for idx, ep_obj in enumerate(new_episodes_list):
PodcastQueue.objects.get_or_create(
user=feed_obj.user, episode=ep_obj,
defaults={'position': idx},
where = PodcastSettings.for_user(feed_obj.user).auto_queue_position
# Newest first in front of what's there; oldest first when appended,
# so the end of the queue plays in the order the episodes came out.
newest_first = sorted(
new_episodes_list,
key=lambda e: e.pub_date or e.discovered_at,
reverse=True,
)
ordered = newest_first if where != PodcastSettings.BACK else newest_first[::-1]
_enqueue(feed_obj.user, [e.id for e in ordered], where)
return new_count
@ -174,8 +436,12 @@ def feed_list(request):
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
return JsonResponse({'feeds': _feeds_data(request.user)})
def _feeds_data(user):
feeds = list(
request.user.podcast_feeds
user.podcast_feeds
.annotate(latest_episode_at=Max('episodes__pub_date'))
.values('id', 'title', 'artwork_url', 'rss_url', 'last_refreshed_at', 'author', 'added_at', 'auto_queue', 'latest_episode_at')
)
@ -187,8 +453,7 @@ def feed_list(request):
f['added_at'] = f['added_at'].isoformat()
if f['latest_episode_at']:
f['latest_episode_at'] = f['latest_episode_at'].isoformat()
return JsonResponse({'feeds': feeds})
return feeds
# ---------------------------------------------------------------------------
@ -332,6 +597,251 @@ def feed_episodes(request, pk):
})
# ---------------------------------------------------------------------------
# One episode in detail: shownotes, chapters, transcripts
# ---------------------------------------------------------------------------
def _load_chapters(episode):
"""Fetch the JSON chapters of `episode` once and cache them on it.
Problems that may go away (timeouts, server errors) aren't cached, so the
next request tries again; anything else is remembered as "no chapters".
"""
url = episode.chapters_url
try:
chapters = chapters_mod.fetch(url)
except chapters_mod.ChaptersUnavailable as e:
log.info('chapters of episode %s not available: %s', episode.id, e)
return []
except chapters_mod.ChaptersInvalid as e:
log.info('chapters of episode %s unusable: %s', episode.id, e)
chapters = []
now = timezone.now()
# Only if a refresh hasn't changed the URL in the meantime.
PodcastEpisode.objects.filter(pk=episode.pk, chapters_url=url).update(
chapters=chapters, chapters_fetched_at=now,
)
episode.chapters, episode.chapters_fetched_at = chapters, now
return chapters
@csrf_exempt
@require_http_methods(['GET'])
def episode_detail(request, pk):
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
try:
ep = PodcastEpisode.objects.select_related('feed').get(pk=pk, feed__user=request.user)
except PodcastEpisode.DoesNotExist:
return JsonResponse({'error': 'not found'}, status=404)
chapters = ep.chapters
if not chapters and ep.chapters_url and ep.chapters_fetched_at is None:
chapters = _load_chapters(ep)
progress = EpisodeProgress.objects.filter(user=request.user, episode=ep).first()
return JsonResponse({
'id': ep.id,
'title': ep.title,
'description': sanitize_html(ep.description),
'timestamps': extract_timestamps(ep.description, ep.duration_seconds),
'chapters': chapters,
'transcripts': ep.transcripts,
'feed': {'id': ep.feed.id, 'title': ep.feed.title, 'artwork_url': ep.feed.artwork_url},
'audio_url': ep.audio_url,
'artwork_url': ep.artwork_url,
'duration_seconds': ep.duration_seconds,
'pub_date': ep.pub_date.isoformat() if ep.pub_date else None,
'episode_number': ep.episode_number,
'season_number': ep.season_number,
'position_seconds': progress.position_seconds if progress else 0,
'played': progress.played if progress else False,
'favorite': progress.favorite if progress else False,
'last_played_at': progress.updated_at.isoformat() if progress else None,
'in_queue': PodcastQueue.objects.filter(user=request.user, episode=ep).exists(),
})
# ---------------------------------------------------------------------------
# Favorites
# ---------------------------------------------------------------------------
@csrf_exempt
@require_http_methods(['POST'])
def set_favorite(request, pk):
"""Star or unstar an episode: {"favorite": true|false}, no value toggles."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
try:
body = json.loads(request.body or b'{}')
except (json.JSONDecodeError, ValueError):
return JsonResponse({'error': 'invalid JSON'}, status=400)
if not isinstance(body, dict):
return JsonResponse({'error': 'invalid JSON'}, status=400)
try:
episode = PodcastEpisode.objects.get(pk=pk, feed__user=request.user)
except PodcastEpisode.DoesNotExist:
return JsonResponse({'error': 'not found'}, status=404)
progress, _ = EpisodeProgress.objects.get_or_create(user=request.user, episode=episode)
wanted = body.get('favorite')
favorite = (not progress.favorite) if wanted is None else bool(wanted)
if favorite != progress.favorite:
# update() leaves updated_at alone: a star isn't listening, and the
# history and the statistics go by updated_at.
EpisodeProgress.objects.filter(pk=progress.pk).update(favorite=favorite)
return JsonResponse({'ok': True, 'favorite': favorite})
# ---------------------------------------------------------------------------
# Episodes of all feeds: filters, history, search
# ---------------------------------------------------------------------------
EPISODE_FILTERS = {
'all': Q(),
'unplayed': Q(played=False),
'in_progress': Q(position_seconds__gt=0, played=False),
'played': Q(played=True),
'favorites': Q(favorite=True),
'history': Q(played=True) | Q(position_seconds__gt=0),
}
EPISODE_LIST_LIMIT = 100
EPISODE_LIST_MAX = 500
EPISODE_LIST_FIELDS = (
'id', 'title', 'audio_url', 'duration_seconds', 'pub_date', 'artwork_url',
'feed__id', 'feed__title', 'feed__artwork_url',
'position_seconds', 'played', 'favorite', 'in_queue', 'last_played_at',
)
def _int_param(request, name, default):
raw = request.GET.get(name, '').strip()
return default if raw == '' else int(raw)
def _episode_list_data(user, filter_name='all', feed_id=None, q='', offset=0, limit=EPISODE_LIST_LIMIT):
"""(episodes, total) for the episode list. Two queries, whatever the
page size: the count and the page. The user's progress row comes in by
a join, queue membership by a subquery. No shownotes (up to 50 KB each)."""
qs = (
PodcastEpisode.objects.filter(feed__user=user)
.annotate(mine=FilteredRelation('progress', condition=Q(progress__user=user)))
.annotate(
position_seconds=Coalesce('mine__position_seconds', Value(0)),
played=Coalesce('mine__played', Value(False)),
favorite=Coalesce('mine__favorite', Value(False)),
last_played_at=F('mine__updated_at'),
in_queue=Exists(PodcastQueue.objects.filter(user=user, episode=OuterRef('pk'))),
)
.filter(EPISODE_FILTERS[filter_name])
)
if feed_id is not None:
qs = qs.filter(feed_id=feed_id)
if q:
qs = qs.filter(title__icontains=q)
if filter_name == 'history':
qs = qs.order_by(F('last_played_at').desc(nulls_last=True), '-id')
else:
qs = qs.order_by(F('pub_date').desc(nulls_last=True), '-id')
total = qs.count()
episodes = list(qs.values(*EPISODE_LIST_FIELDS)[offset:offset + limit]) if limit else []
for ep in episodes:
ep['pub_date'] = ep['pub_date'].isoformat() if ep['pub_date'] else None
ep['last_played_at'] = ep['last_played_at'].isoformat() if ep['last_played_at'] else None
ep['played'] = bool(ep['played'])
ep['favorite'] = bool(ep['favorite'])
ep['in_queue'] = bool(ep['in_queue'])
return episodes, total
@csrf_exempt
@require_http_methods(['GET'])
def episode_list(request):
"""Episodes of all the user's feeds, filtered (all, unplayed, in_progress,
played, favorites, history), optionally of one feed or matching `q`."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
filter_name = request.GET.get('filter', '').strip() or 'all'
if filter_name not in EPISODE_FILTERS:
return JsonResponse({'error': 'unknown filter'}, status=400)
try:
offset = max(_int_param(request, 'offset', 0), 0)
limit = min(max(_int_param(request, 'limit', EPISODE_LIST_LIMIT), 0), EPISODE_LIST_MAX)
feed_id = _int_param(request, 'feed_id', None)
except ValueError:
return JsonResponse({'error': 'offset, limit and feed_id must be numbers'}, status=400)
episodes, total = _episode_list_data(
request.user, filter_name, feed_id, request.GET.get('q', '').strip(), offset, limit,
)
return JsonResponse({'episodes': episodes, 'offset': offset, 'limit': limit, 'total': total})
# ---------------------------------------------------------------------------
# Listening statistics
# ---------------------------------------------------------------------------
@csrf_exempt
@require_http_methods(['GET'])
def podcast_stats(request):
"""How much the user listened, in total and per feed; `days` limits it to
progress rows updated within the last N days."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
rows = EpisodeProgress.objects.filter(user=request.user, episode__feed__user=request.user)
since = None
raw_days = request.GET.get('days', '').strip()
if raw_days:
try:
days = int(raw_days)
except ValueError:
return JsonResponse({'error': 'days must be a number'}, status=400)
if days < 0:
return JsonResponse({'error': 'days must not be negative'}, status=400)
since = timezone.now() - timedelta(days=days)
rows = rows.filter(updated_at__gte=since)
# A played episode counts with its full length (or as far as it got, if
# the feed gave no length); anything else as far as it got.
listened = Case(
When(played=True, episode__duration_seconds__gt=0, then=F('episode__duration_seconds')),
default=F('position_seconds'),
output_field=IntegerField(),
)
per_feed = (
rows.order_by()
.values('episode__feed_id', 'episode__feed__title', 'episode__feed__artwork_url')
.annotate(
seconds=Sum(listened),
played_count=Count('id', filter=Q(played=True)),
started_count=Count('id', filter=Q(played=False, position_seconds__gt=0)),
)
)
total_seconds = episodes_played = episodes_started = 0
feeds = []
for row in per_feed:
seconds = row['seconds'] or 0
total_seconds += seconds
episodes_played += row['played_count']
episodes_started += row['started_count']
if seconds > 0:
feeds.append({
'feed_id': row['episode__feed_id'],
'title': row['episode__feed__title'],
'artwork_url': row['episode__feed__artwork_url'],
'seconds': seconds,
'episodes_played': row['played_count'],
})
feeds.sort(key=lambda f: (-f['seconds'], f['title'].lower(), f['feed_id']))
return JsonResponse({
'total_seconds': total_seconds,
'episodes_played': episodes_played,
'episodes_started': episodes_started,
'since': since.isoformat() if since else None,
'feeds': feeds,
})
# ---------------------------------------------------------------------------
# Import OPML
# ---------------------------------------------------------------------------
@ -372,6 +882,44 @@ def import_opml(request):
return JsonResponse({'ok': True, 'added': added, 'skipped': skipped})
# ---------------------------------------------------------------------------
# Export OPML
# ---------------------------------------------------------------------------
def _xml_text(s):
"""Drop characters XML 1.0 can't carry at all (escaping is ElementTree's job)."""
return ''.join(c for c in s if c in '\t\n\r' or (' ' <= c <= '퟿') or ('' <= c <= '<EFBFBD>')
or ord(c) >= 0x10000)
@csrf_exempt
@require_http_methods(['GET'])
def export_opml(request):
"""The user's subscriptions as an OPML 2.0 file, for other podcast apps
(and for import_opml)."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
root = ET.Element('opml', version='2.0')
head = ET.SubElement(root, 'head')
ET.SubElement(head, 'title').text = 'diora Podcasts'
body = ET.SubElement(root, 'body')
for feed in request.user.podcast_feeds.order_by('title', 'id').only('title', 'rss_url', 'link'):
title = _xml_text(feed.title or feed.rss_url)
ET.SubElement(body, 'outline', {
'type': 'rss',
'text': title,
'title': title,
'xmlUrl': _xml_text(feed.rss_url),
'htmlUrl': _xml_text(feed.link),
})
ET.indent(root)
xml = '<?xml version="1.0" encoding="UTF-8"?>\n' + ET.tostring(root, encoding='unicode') + '\n'
response = HttpResponse(xml.encode('utf-8'), content_type='text/x-opml; charset=utf-8')
response['Content-Disposition'] = 'attachment; filename="diora-podcasts.opml"'
return response
# ---------------------------------------------------------------------------
# Refresh feed now
# ---------------------------------------------------------------------------
@ -404,6 +952,48 @@ def refresh_feed_now(request):
return JsonResponse({'ok': True, 'new_episodes': new_episodes})
@csrf_exempt
@require_http_methods(['POST'])
def refresh_all_feeds(request):
"""Refresh all of the user's feeds on the server in one go. Feeds the
hourly job (or an earlier call) refreshed within `max_age` seconds are
skipped. Clients then fetch the result once (state/, inbox/ …)."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
try:
body = json.loads(request.body or b'{}')
except (json.JSONDecodeError, ValueError):
return JsonResponse({'error': 'invalid JSON'}, status=400)
try:
max_age = max(int(body.get('max_age', 300)), 0)
except (TypeError, ValueError):
return JsonResponse({'error': 'max_age must be a number'}, status=400)
cutoff = timezone.now() - timedelta(seconds=max_age)
feeds = request.user.podcast_feeds.filter(Q(last_refreshed_at__isnull=True) | Q(last_refreshed_at__lt=cutoff))
refreshed, failed, new = _refresh_feeds(feeds)
return JsonResponse({'ok': True, 'refreshed': refreshed, 'failed': failed, 'new_episodes': new})
@csrf_exempt
def podcast_state(request):
"""Everything a client shows first, in one response: feeds, queue, the
first page of the inbox and the settings."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
try:
limit = min(max(int(request.GET.get('inbox_limit', 100)), 0), 1000)
except ValueError:
return JsonResponse({'error': 'inbox_limit must be a number'}, status=400)
s = PodcastSettings.for_user(request.user)
return JsonResponse({
'feeds': _feeds_data(request.user),
'queue': _queue_data(request.user),
'inbox': _inbox_data(request.user, 0, limit, with_description=False),
'settings': {'enqueue_position': s.enqueue_position, 'auto_queue_position': s.auto_queue_position},
'server_time': timezone.now().isoformat(),
})
# ---------------------------------------------------------------------------
# Queue
# ---------------------------------------------------------------------------
@ -413,15 +1003,21 @@ def queue_get(request):
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
return JsonResponse({'queue': _queue_data(request.user)})
def _queue_data(user):
items = list(
PodcastQueue.objects
.filter(user=request.user)
.filter(user=user)
.order_by('position', 'added_at', 'id')
.select_related('episode__feed')
.values(
'id', 'position',
'episode__id', 'episode__title', 'episode__audio_url',
'episode__duration_seconds', 'episode__artwork_url',
'episode__feed__id', 'episode__feed__title',
'episode__feed__id', 'episode__feed__title', 'episode__feed__artwork_url',
'episode__pub_date',
)
)
@ -429,13 +1025,14 @@ def queue_get(request):
progress_map = {
p['episode_id']: p['position_seconds']
for p in EpisodeProgress.objects.filter(
user=request.user, episode_id__in=ep_ids,
user=user, episode_id__in=ep_ids,
).values('episode_id', 'position_seconds')
}
for item in items:
item['position_seconds'] = progress_map.get(item['episode__id'], 0)
return JsonResponse({'queue': items})
if item['episode__pub_date']:
item['episode__pub_date'] = item['episode__pub_date'].isoformat()
return items
@csrf_exempt
@ -449,19 +1046,29 @@ def queue_add(request):
except (json.JSONDecodeError, ValueError):
return JsonResponse({'error': 'invalid JSON'}, status=400)
episode_id = body.get('episode_id')
try:
episode = PodcastEpisode.objects.get(pk=episode_id, feed__user=request.user)
except PodcastEpisode.DoesNotExist:
return JsonResponse({'error': 'not found'}, status=404)
# One episode (episode_id) or several in order (episode_ids). Where they
# go: `where` if given, else the user's setting. `current_episode_id` is
# what the client is playing, for after_current.
ids = body.get('episode_ids')
if ids is None:
ids = [body.get('episode_id')]
if not isinstance(ids, list):
return JsonResponse({'error': 'episode_ids must be a list'}, status=400)
where = body.get('where') or PodcastSettings.for_user(request.user).enqueue_position
if where not in QUEUE_POSITIONS:
return JsonResponse({'error': 'unknown position'}, status=400)
max_pos = PodcastQueue.objects.filter(user=request.user).count()
_, created = PodcastQueue.objects.get_or_create(
user=request.user,
episode=episode,
defaults={'position': max_pos},
found = set(
PodcastEpisode.objects.filter(pk__in=[i for i in ids if isinstance(i, int)], feed__user=request.user)
.values_list('id', flat=True)
)
return JsonResponse({'ok': True, 'created': created})
if not found:
return JsonResponse({'error': 'not found'}, status=404)
added = _enqueue(
request.user, [i for i in ids if i in found], where,
current_hint=body.get('current_episode_id'),
)
return JsonResponse({'ok': True, 'created': bool(added), 'added': added})
@csrf_exempt
@ -498,6 +1105,35 @@ def queue_reorder(request):
return JsonResponse({'ok': True})
@csrf_exempt
def podcast_settings(request):
"""GET the user's podcast settings; POST changes some of them."""
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
s = PodcastSettings.for_user(request.user)
if request.method == 'POST':
try:
body = json.loads(request.body)
except (json.JSONDecodeError, ValueError):
return JsonResponse({'error': 'invalid JSON'}, status=400)
fields = []
for name in ('enqueue_position', 'auto_queue_position'):
if name in body:
if body[name] not in QUEUE_POSITIONS:
return JsonResponse({'error': f'unknown {name}'}, status=400)
setattr(s, name, body[name])
fields.append(name)
if fields:
s.save(update_fields=fields)
elif request.method != 'GET':
return JsonResponse({'error': 'method not allowed'}, status=405)
return JsonResponse({
'enqueue_position': s.enqueue_position,
'auto_queue_position': s.auto_queue_position,
'positions': [{'value': k, 'label': v} for k, v in PodcastSettings.POSITIONS],
})
# ---------------------------------------------------------------------------
# Progress
# ---------------------------------------------------------------------------
@ -526,10 +1162,11 @@ def save_progress(request):
episode=episode,
)
progress.position_seconds = position
# Auto-mark played at 90%
if episode.duration_seconds and episode.duration_seconds > 0:
if position >= episode.duration_seconds * 0.9:
# Started: no longer new.
if position > 0:
progress.dismissed = True
# Within the last seconds: played (same rule as the apps).
if counts_as_played(position, episode.duration_seconds):
progress.played = True
progress.save()
@ -612,39 +1249,44 @@ def inbox(request):
if not request.user.is_authenticated:
return JsonResponse({'error': 'authentication required'}, status=401)
limit = min(int(request.GET.get('limit', getattr(settings, 'PODCAST_INBOX_PAGE_SIZE', 200))), 1000)
offset = max(int(request.GET.get('offset', 0)), 0)
return JsonResponse({'episodes': _inbox_data(request.user, offset, limit), 'offset': offset, 'limit': limit})
def _inbox_data(user, offset, limit, with_description=True):
"""Inbox episodes. Shownotes can be large (up to 50 KB each); the apps'
state/ leaves them out and loads one episode's at a time."""
fields = ['id', 'title', 'audio_url', 'duration_seconds', 'pub_date', 'artwork_url',
'feed__id', 'feed__title', 'feed__artwork_url']
if with_description:
fields.insert(2, 'description')
hidden_ids = set(
EpisodeProgress.objects.filter(
user=request.user,
user=user,
).filter(
Q(played=True) | Q(dismissed=True)
).values_list('episode_id', flat=True)
)
limit = min(int(request.GET.get('limit', getattr(settings, 'PODCAST_INBOX_PAGE_SIZE', 200))), 1000)
offset = max(int(request.GET.get('offset', 0)), 0)
episodes = list(
PodcastEpisode.objects.filter(feed__user=request.user)
PodcastEpisode.objects.filter(feed__user=user)
.exclude(id__in=hidden_ids)
.select_related('feed')
.order_by('-pub_date')
.values(
'id', 'title', 'description', 'audio_url', 'duration_seconds',
'pub_date', 'artwork_url',
'feed__id', 'feed__title', 'feed__artwork_url',
)[offset:offset + limit]
.values(*fields)[offset:offset + limit]
)
progress_map = {
ep['episode_id']: ep
for ep in EpisodeProgress.objects.filter(
user=request.user,
user=user,
episode_id__in=[e['id'] for e in episodes],
).values('episode_id', 'position_seconds')
}
queued_ids = set(
PodcastQueue.objects.filter(
user=request.user,
user=user,
episode_id__in=[e['id'] for e in episodes],
).values_list('episode_id', flat=True)
)
@ -654,5 +1296,4 @@ def inbox(request):
prog = progress_map.get(ep['id'], {})
ep['position_seconds'] = prog.get('position_seconds', 0)
ep['in_queue'] = ep['id'] in queued_ids
return JsonResponse({'episodes': episodes, 'offset': offset, 'limit': limit})
return episodes

View file

@ -2181,44 +2181,33 @@ async function refreshOpenFeed(btn) {
}
async function refreshAllFeedsBtn(btn) {
if (btn) { btn.disabled = true; btn.textContent = '↻ 0/' + podcastFeeds.length; }
let done = 0;
for (const feed of podcastFeeds) {
// One request: the server fetches every feed itself (in parallel, each
// feed address once) and the lists are then loaded once.
if (btn) { btn.disabled = true; btn.textContent = '↻ …'; }
try {
await fetch('/podcasts/feeds/refresh/', {
await fetch('/podcasts/feeds/refresh-all/', {
method: 'POST',
headers: {'Content-Type': 'application/json', 'X-CSRFToken': getCsrfToken()},
body: JSON.stringify({feed_id: feed.id}),
body: JSON.stringify({max_age: 60}),
});
} catch (e) {}
done++;
if (btn) btn.textContent = `↻ ${done}/${podcastFeeds.length}`;
}
await loadFeedList();
if (podcastCurrentView === 'feeds') renderFeedList();
if (podcastCurrentView === 'inbox') loadAndRenderInbox();
if (podcastCurrentView === 'episodes' && podcastCurrentFeedId) openFeed(podcastCurrentFeedId);
await reloadPodcastViews();
if (btn) { btn.disabled = false; btn.textContent = '↻ All'; }
}
async function refreshAllFeeds() {
if (!IS_AUTHENTICATED || !podcastFeeds.length) return;
for (const feed of podcastFeeds) {
try {
await fetch('/podcasts/feeds/refresh/', {
method: 'POST',
headers: {'Content-Type': 'application/json', 'X-CSRFToken': getCsrfToken()},
body: JSON.stringify({feed_id: feed.id}),
});
} catch (e) {}
}
// Reload feed metadata and refresh the currently open view
async function reloadPodcastViews() {
await loadFeedList();
if (podcastCurrentView === 'feeds') renderFeedList();
if (podcastCurrentView === 'inbox') loadAndRenderInbox();
if (podcastCurrentView === 'episodes' && podcastCurrentFeedId) openFeed(podcastCurrentFeedId);
}
async function refreshAllFeeds() {
// The server refreshes all feeds every hour; just pick up what it found.
if (!IS_AUTHENTICATED || !podcastFeeds.length) return;
await reloadPodcastViews();
}
async function removeFeed(feedId) {
if (!await customConfirm('Remove this podcast?', { danger: true })) return;
try {

View file

@ -2,7 +2,7 @@
* diora service worker — caches the app shell for offline use.
*/
const CACHE = 'diora-v50';
const CACHE = 'diora-v51';
const PODCAST_CACHE = 'diora-podcast-v1';
const SHELL = [
'/static/css/app.css',