Podcasts: Merkliste und Wiedergabeliste (hinter lists_v2), Abo-Stufen inkl. Stamm direkt in die Wiedergabeliste, Abrufintervall pro Abo, Strom-Daten (heard_365, stream/seen, filter=stream)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/podcasts.md

@@ -52,7 +52,7 @@ silent subscriptions become Strom with *täglich*.
 All episodes of all Strom and Stamm subscriptions, by period (Heute, Gestern, Diese Woche,
 …). Within a period the podcasts are ordered by **how much the user has listened to them**
 (episodes heard): every podcast with at least one heard episode before every one without —
-those count for infinitely more; among the never-heard ones, rarely publishing first.
+those count for infinitely more (heard in the last 12 months, `heard_365`); among the never-heard ones, rarely publishing first.
 Each podcast appears **once per period, always bundled**: its newest episode, and "+n ›" for the others. Episodes new since
 they were last seen carry a **small mark** (no counts). Per episode: ▶ (Wiedergabeliste,
 first), ＋ (Merkliste), ★. Filters stay: Favoriten, Verlauf, Angefangen, Gehört.
@@ -85,3 +85,30 @@ thought through.
 2. The three levels and the fetch interval; downloads following the lists.
 3. The Strom replacing "Neu" and "Folgen".
 4. Browsing — after more thought.
+
+## API (server, since 2026-10-19)
+
+- `PodcastSettings.lists_v2` turns the two lists on. `POST /podcasts/lists/ {"on": true}`
+  moves the queue into the Merkliste (the Wiedergabeliste starts empty); `{"on": false}` puts
+  the Merkliste back at the end of the queue. Off, everything works as before (apps not
+  updated yet).
+- **Wiedergabeliste** = the queue as it was: `state.queue`, `/podcasts/queue/add|remove|reorder/`.
+  The player goes through it and stops at its end.
+- **Merkliste** = `PodcastLater`: `state.later` (rows like the queue's, newest first, plus
+  `added_at`, `auto` — added by a Stamm feed — and `in_queue`), `POST /podcasts/later/add/` and
+  `/podcasts/later/remove/` `{"episode_ids": [...]}`. Heard (progress at the end, mark-played)
+  and "Folge löschen" take an episode off it. Episode lists carry `in_later`; filter `later`.
+- **Levels**: `PodcastFeed.new_episodes` — `queue` = Stamm into the Merkliste (with lists_v2;
+  without it: the queue), `play` = Stamm into the Wiedergabeliste, `inbox` = Strom,
+  `ignore` = Auf Abruf. `auto_queue` stays true for queue and play.
+- **Fetch interval**: `PodcastFeed.refresh_interval` — `''` (by rhythm: every `interval_days`
+  hours, within 15 minutes and 24 hours), `hourly`, `daily`, `weekly`; the cron's
+  `refresh_feeds` only fetches feeds that are due (`views.refresh_due`).
+- **Strom**: `GET /podcasts/episodes/?filter=stream` (all but Auf Abruf; with `folder=`, `sort=`);
+  episodes carry `discovered_at`. Per feed in `state.feeds`: `heard_365` (episodes heard in the
+  last 12 months: played, or listened into for more than a minute) and `interval_days`.
+  `POST /podcasts/stream/seen/` → `{before, now}`; `state.stream_seen_at`. An episode is marked
+  when its `discovered_at` is after the time before the current look.
+- Web host (`static/js/podcasts-web.js`, `DioraPodcastHost`): `podcastLaterAdd(json episodes)`,
+  `podcastLaterRemove(json ids)`, `podcastListsSwitch(on)`, `podcastStreamSeen()` (keeps the
+  previous time as `state.stream_seen_before` for the marks).

podcasts/management/commands/refresh_feeds.py

@@ -2,7 +2,7 @@ from django.core.management.base import BaseCommand
 from django.utils import timezone
 
 from podcasts.models import PodcastFeed
-from podcasts.views import _in_backoff_q, _refresh_feeds
+from podcasts.views import _in_backoff_q, _refresh_feeds, refresh_due
 
 
 class Command(BaseCommand):
@@ -26,6 +26,10 @@ class Command(BaseCommand):
             # A feed failing 3+ times in a row is tried every 2, 4, 8, 16,
             # at most 24 hours instead of every hour.
             qs = qs.exclude(_in_backoff_q(timezone.now()))
+            # Each feed as often as its interval says (docs/podcasts.md): by its rhythm,
+            # or hourly, daily, weekly. --feed-id and --full take every one.
+            if not options['full']:
+                qs = qs.filter(pk__in=refresh_due(list(qs.select_related('user'))))
 
         if options['limit']:
             qs = qs[: options['limit']]

podcasts/migrations/0022_lists_and_levels.py

@@ -0,0 +1,49 @@
+# Generated by Django 6.1 on 2026-10-19 15:53
+
+import django.db.models.deletion
+from django.conf import settings
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('podcasts', '0021_folders_list'),
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='podcastfeed',
+            name='refresh_interval',
+            field=models.CharField(blank=True, choices=[('', 'Automatisch'), ('hourly', 'Stündlich'), ('daily', 'Täglich'), ('weekly', 'Wöchentlich')], default='', max_length=10),
+        ),
+        migrations.AddField(
+            model_name='podcastsettings',
+            name='lists_v2',
+            field=models.BooleanField(default=False),
+        ),
+        migrations.AddField(
+            model_name='podcastsettings',
+            name='stream_seen_at',
+            field=models.DateTimeField(blank=True, null=True),
+        ),
+        migrations.AlterField(
+            model_name='podcastfeed',
+            name='new_episodes',
+            field=models.CharField(choices=[('inbox', 'Im Strom'), ('queue', 'Stamm: in die Merkliste'), ('play', 'Stamm: in die Wiedergabeliste'), ('ignore', 'Auf Abruf')], default='inbox', max_length=10),
+        ),
+        migrations.CreateModel(
+            name='PodcastLater',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('added_at', models.DateTimeField(auto_now_add=True)),
+                ('auto', models.BooleanField(default=False)),
+                ('episode', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='later_by', to='podcasts.podcastepisode')),
+                ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='podcast_later', to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'unique_together': {('user', 'episode')},
+            },
+        ),
+    ]

podcasts/models.py

@@ -8,15 +8,23 @@ MAX_SPEED = 3.5
 
 
 class PodcastFeed(models.Model):
-    # What happens to a new episode of this feed (one that passed the filter).
+    # What happens to a new episode of this feed (one that passed the filter); with the
+    # lists of docs/podcasts.md (PodcastSettings.lists_v2) these are the subscription levels:
+    # inbox = Strom, queue = Stamm (into the Merkliste), play = Stamm straight into the
+    # Wiedergabeliste, ignore = Auf Abruf. Without lists_v2, queue and play both queue.
     NEW_INBOX = 'inbox'
     NEW_QUEUE = 'queue'
+    NEW_PLAY = 'play'
     NEW_IGNORE = 'ignore'
     NEW_EPISODE_ACTIONS = [
-        (NEW_INBOX, 'In „Neu“'),
-        (NEW_QUEUE, 'In die Warteschlange'),
-        (NEW_IGNORE, 'Nicht anzeigen'),
+        (NEW_INBOX, 'Im Strom'),
+        (NEW_QUEUE, 'Stamm: in die Merkliste'),
+        (NEW_PLAY, 'Stamm: in die Wiedergabeliste'),
+        (NEW_IGNORE, 'Auf Abruf'),
     ]
+    # How often the feed is fetched: '' = by its rhythm (refresh_due()), or fixed.
+    REFRESH_INTERVALS = [('', 'Automatisch'), ('hourly', 'Stündlich'), ('daily', 'Täglich'),
+                         ('weekly', 'Wöchentlich')]
     # Volume factors: below 1 quieter, above 1 louder (the apps boost those).
     VOLUMES = (0.2, 0.5, 1.0, 1.5, 2.0, 2.5)
     # What it is: a podcast, or an audiobook (Hörbuch, Hörspiel), which lives in the
@@ -58,6 +66,7 @@ class PodcastFeed(models.Model):
     volume = models.FloatField(default=1.0)               # one of VOLUMES
     # New episodes: what happens to them, and which ones count at all.
     new_episodes = models.CharField(max_length=10, choices=NEW_EPISODE_ACTIONS, default=NEW_INBOX)
+    refresh_interval = models.CharField(max_length=10, choices=REFRESH_INTERVALS, default='', blank=True)
     # Comma-separated words, case-insensitive, matched against the title.
     include_words = models.CharField(max_length=500, blank=True)
     exclude_words = models.CharField(max_length=500, blank=True)
@@ -106,15 +115,16 @@ class PodcastFeed(models.Model):
         self._synced = (self.__dict__.get('auto_queue'), self.__dict__.get('new_episodes'))
 
     def _sync_auto_queue(self):
-        """Keep auto_queue == (new_episodes == 'queue'). Whichever of the two
+        """Keep auto_queue == (new_episodes is 'queue' or 'play'). Whichever of the two
         was changed since loading (or, for a new feed, set away from its
         default) wins; new_episodes if both were."""
         auto_before, new_before = getattr(self, '_synced', (False, self.NEW_INBOX))
+        queues = (self.NEW_QUEUE, self.NEW_PLAY)
         if self.new_episodes != new_before or self.auto_queue == auto_before:
-            self.auto_queue = self.new_episodes == self.NEW_QUEUE
+            self.auto_queue = self.new_episodes in queues
         elif self.auto_queue:
             self.new_episodes = self.NEW_QUEUE
-        elif self.new_episodes == self.NEW_QUEUE:
+        elif self.new_episodes in queues:
             self.new_episodes = self.NEW_INBOX
 
     def save(self, *args, **kwargs):
@@ -260,6 +270,23 @@ class PodcastQueue(models.Model):
     def __str__(self):
         return f'{self.user} queue pos {self.position}: {self.episode}'
 
+class PodcastLater(models.Model):
+    """The Merkliste (docs/podcasts.md): episodes the user wants to hear some day. It may
+    grow and never expires; shown grouped by podcast, so it has no order of its own. An
+    episode can be in it and in the Wiedergabeliste (PodcastQueue) at once; heard, it leaves
+    both. auto: put there by a Stamm subscription, not by hand."""
+    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='podcast_later')
+    episode = models.ForeignKey(PodcastEpisode, on_delete=models.CASCADE, related_name='later_by')
+    added_at = models.DateTimeField(auto_now_add=True)
+    auto = models.BooleanField(default=False)
+
+    class Meta:
+        unique_together = ('user', 'episode')
+
+    def __str__(self):
+        return f'{self.user} Merkliste: {self.episode}'
+
+
 class PodcastSettings(models.Model):
     """A user's podcast preferences. Where episodes go in the queue: when added
     by hand (also when played from outside the queue), and when a feed that
@@ -297,6 +324,12 @@ class PodcastSettings(models.Model):
     import_downloads = models.JSONField(default=list, blank=True)
     # The order of the podcast/audiobook folders (their chips), names.
     folder_order = models.JSONField(default=list, blank=True)
+    # The two lists of docs/podcasts.md: PodcastQueue is the Wiedergabeliste (the player goes
+    # through it and stops at its end), PodcastLater the Merkliste, and Stamm subscriptions
+    # fill the Merkliste. Off: the one queue as before (apps not updated yet).
+    lists_v2 = models.BooleanField(default=False)
+    # When the user last looked at the Strom: episodes found after it carry a mark.
+    stream_seen_at = models.DateTimeField(null=True, blank=True)
 
     # Playback, the same on every device (AntennaPod's playback settings).
     # Null = not set yet: each device keeps its own value; the first app that

podcasts/test_lists.py

@@ -0,0 +1,176 @@
+"""The two lists, the subscription levels, the fetch interval and the Strom (docs/podcasts.md)."""
+import json
+from datetime import datetime, timedelta, timezone
+from unittest.mock import patch
+
+from django.contrib.auth.models import User
+from django.test import TestCase
+from django.utils import timezone as dj_tz
+
+from .models import EpisodeProgress, PodcastEpisode, PodcastFeed, PodcastLater, PodcastQueue, PodcastSettings
+from .tests import _ok, _rss
+from .views import _refresh_feed, refresh_due, refresh_hours
+
+
+class Base(TestCase):
+    def setUp(self):
+        self.user = User.objects.create_user(username='lists', password='pw12345678')
+        self.client.force_login(self.user)
+        self.feed = PodcastFeed.objects.create(
+            user=self.user, rss_url='https://example.org/feed', title='Feed',
+            last_refreshed_at=datetime(2026, 1, 1, tzinfo=timezone.utc))
+        self.eps = [PodcastEpisode.objects.create(feed=self.feed, guid=f'e{i}', title=f'E{i}', duration_seconds=600,
+                                                  audio_url=f'https://example.org/e{i}.mp3',
+                                                  pub_date=datetime(2026, 1, 1 + i, tzinfo=timezone.utc))
+                    for i in range(4)]
+
+    def post(self, url, body=None):
+        return self.client.post(url, json.dumps(body or {}), content_type='application/json')
+
+    def state(self):
+        return self.client.get('/podcasts/state/').json()
+
+    def queue(self):
+        return list(PodcastQueue.objects.filter(user=self.user).order_by('position').values_list('episode_id', flat=True))
+
+    def later(self):
+        return set(PodcastLater.objects.filter(user=self.user).values_list('episode_id', flat=True))
+
+
+class SwitchTests(Base):
+    def test_switching_on_makes_the_queue_the_merkliste(self):
+        for i, ep in enumerate(self.eps[:3]):
+            PodcastQueue.objects.create(user=self.user, episode=ep, position=i)
+        r = self.post('/podcasts/lists/', {'on': True}).json()
+        self.assertEqual((r['lists_v2'], r['queue'], r['later']), (True, 0, 3))
+        self.assertEqual(self.later(), {e.id for e in self.eps[:3]})
+        self.assertEqual(self.queue(), [])
+        st = self.state()
+        self.assertTrue(st['lists_v2'])
+        self.assertEqual([i['episode__id'] for i in st['later']], [self.eps[2].id, self.eps[1].id, self.eps[0].id])
+        # And back: the Merkliste joins the queue, oldest first.
+        self.post('/podcasts/lists/', {'on': False})
+        self.assertEqual(self.queue(), [e.id for e in self.eps[:3]])
+        self.assertEqual(self.later(), set())
+
+    def test_switching_twice_changes_nothing(self):
+        PodcastQueue.objects.create(user=self.user, episode=self.eps[0], position=0)
+        self.post('/podcasts/lists/', {'on': True})
+        PodcastQueue.objects.create(user=self.user, episode=self.eps[1], position=0)
+        self.post('/podcasts/lists/', {'on': True})
+        self.assertEqual(self.queue(), [self.eps[1].id])
+        self.assertEqual(self.later(), {self.eps[0].id})
+
+
+class MerklisteTests(Base):
+    def test_add_remove_and_both_lists(self):
+        r = self.post('/podcasts/later/add/', {'episode_ids': [self.eps[0].id, self.eps[1].id, 999999]}).json()
+        self.assertEqual(r['added'], [self.eps[0].id, self.eps[1].id])
+        self.post('/podcasts/queue/add/', {'episode_ids': [self.eps[0].id], 'where': 'front'})
+        rows = {i['episode__id']: i for i in self.state()['later']}
+        self.assertTrue(rows[self.eps[0].id]['in_queue'])
+        self.assertFalse(rows[self.eps[1].id]['in_queue'])
+        self.post('/podcasts/later/remove/', {'episode_ids': [self.eps[1].id]})
+        self.assertEqual(self.later(), {self.eps[0].id})
+        self.assertEqual(self.queue(), [self.eps[0].id], 'the Wiedergabeliste is left alone')
+
+    def test_heard_leaves_the_merkliste(self):
+        self.post('/podcasts/later/add/', {'episode_ids': [e.id for e in self.eps[:3]]})
+        self.post('/podcasts/progress/save/', {'episode_id': self.eps[0].id, 'position_seconds': 590})
+        self.post('/podcasts/progress/mark-played/', {'episode_id': self.eps[1].id})
+        self.post('/podcasts/progress/save/', {'episode_id': self.eps[2].id, 'position_seconds': 100})
+        self.assertEqual(self.later(), {self.eps[2].id}, 'started stays, heard goes')
+
+    def test_discard_leaves_both_lists(self):
+        self.post('/podcasts/later/add/', {'episode_ids': [self.eps[0].id]})
+        self.post('/podcasts/queue/add/', {'episode_ids': [self.eps[0].id]})
+        self.post('/podcasts/episodes/discard/', {'episode_ids': [self.eps[0].id]})
+        self.assertEqual((self.later(), self.queue()), (set(), []))
+
+    def test_episode_lists_tell_and_filter(self):
+        self.post('/podcasts/later/add/', {'episode_ids': [self.eps[3].id]})
+        r = self.client.get('/podcasts/episodes/?filter=later').json()
+        self.assertEqual([e['id'] for e in r['episodes']], [self.eps[3].id])
+        self.assertTrue(r['episodes'][0]['in_later'])
+        self.assertIn('discovered_at', r['episodes'][0])
+
+
+class LevelTests(Base):
+    """New episodes by level: Stamm into the Merkliste (or the Wiedergabeliste), Strom
+    nowhere, and the old one queue without lists_v2."""
+
+    RSS = _rss([('n1', 'Mon, 05 Jan 2026 10:00:00 GMT'), ('n2', 'Tue, 06 Jan 2026 10:00:00 GMT')])
+
+    def refresh(self, level, v2=True):
+        PodcastSettings.objects.update_or_create(user=self.user, defaults={'lists_v2': v2})
+        self.feed.new_episodes = level
+        self.feed.save()
+        with patch('podcasts.views._fetch_feed', lambda url, **kw: _ok(self.RSS)):
+            _refresh_feed(self.feed)
+        guid = dict(PodcastEpisode.objects.filter(feed=self.feed).values_list('id', 'guid'))
+        return ([guid[i] for i in self.queue()], sorted(guid[i] for i in self.later()))
+
+    def test_stamm_goes_to_the_merkliste_marked_auto(self):
+        self.assertEqual(self.refresh('queue'), ([], ['n1', 'n2']))
+        self.assertTrue(all(PodcastLater.objects.filter(user=self.user).values_list('auto', flat=True)))
+
+    def test_stamm_play_goes_to_the_wiedergabeliste(self):
+        self.assertEqual(self.refresh('play'), (['n2', 'n1'], []))
+        self.assertTrue(PodcastFeed.objects.get(pk=self.feed.pk).auto_queue)
+
+    def test_strom_and_auf_abruf_go_nowhere(self):
+        self.assertEqual(self.refresh('inbox'), ([], []))
+
+    def test_without_lists_v2_stamm_queues_as_before(self):
+        self.assertEqual(self.refresh('queue', v2=False), (['n2', 'n1'], []))
+
+    def test_the_strom_leaves_out_auf_abruf(self):
+        other = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/o', title='O', new_episodes='ignore')
+        PodcastEpisode.objects.create(feed=other, guid='x', title='X', audio_url='https://example.org/x.mp3')
+        r = self.client.get('/podcasts/episodes/?filter=stream&limit=50').json()
+        self.assertNotIn('X', [e['title'] for e in r['episodes']])
+        self.assertEqual(r['total'], 4)
+
+
+class IntervalTests(Base):
+    def test_fixed_and_by_rhythm(self):
+        self.assertEqual(refresh_hours({'refresh_interval': 'daily'}, 0.5), 24)
+        self.assertEqual(refresh_hours({'refresh_interval': 'weekly'}, 0.5), 168)
+        self.assertEqual(refresh_hours({'refresh_interval': ''}, 0.1), 0.25)
+        self.assertEqual(refresh_hours({'refresh_interval': ''}, 7), 7)
+        self.assertEqual(refresh_hours({'refresh_interval': ''}, 90), 24)
+        self.assertEqual(refresh_hours({'refresh_interval': ''}, None), 1)
+
+    def test_due(self):
+        now = dj_tz.now()
+        daily = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/d', title='D',
+                                           refresh_interval='daily', last_refreshed_at=now - timedelta(hours=5))
+        stale = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/s', title='S',
+                                           refresh_interval='daily', last_refreshed_at=now - timedelta(hours=25))
+        new = PodcastFeed.objects.create(user=self.user, rss_url='https://example.org/n', title='N')
+        due = refresh_due(list(PodcastFeed.objects.filter(pk__in=[daily.pk, stale.pk, new.pk])), now)
+        self.assertEqual(set(due), {stale.pk, new.pk})
+
+    def test_setting_it(self):
+        r = self.post(f'/podcasts/feeds/{self.feed.pk}/settings/', {'refresh_interval': 'weekly'}).json()
+        self.assertEqual(r['refresh_interval'], 'weekly')
+        self.assertEqual(self.post(f'/podcasts/feeds/{self.feed.pk}/settings/', {'refresh_interval': 'often'}).status_code, 400)
+
+
+class StromTests(Base):
+    def test_heard_in_twelve_months_per_feed(self):
+        EpisodeProgress.objects.create(user=self.user, episode=self.eps[0], played=True)
+        EpisodeProgress.objects.create(user=self.user, episode=self.eps[1], position_seconds=120)
+        EpisodeProgress.objects.create(user=self.user, episode=self.eps[2], position_seconds=10)   # just a taste
+        old = EpisodeProgress.objects.create(user=self.user, episode=self.eps[3], played=True)
+        EpisodeProgress.objects.filter(pk=old.pk).update(updated_at=dj_tz.now() - timedelta(days=400))
+        feed = [f for f in self.state()['feeds'] if f['id'] == self.feed.id][0]
+        self.assertEqual(feed['heard_365'], 2)
+
+    def test_seen(self):
+        self.assertIsNone(self.state()['stream_seen_at'])
+        r = self.post('/podcasts/stream/seen/').json()
+        self.assertIsNone(r['before'])
+        r2 = self.post('/podcasts/stream/seen/').json()
+        self.assertEqual(r2['before'], r['now'])
+        self.assertEqual(self.state()['stream_seen_at'], r2['now'])

podcasts/tests.py

@@ -834,11 +834,13 @@ class EpisodeListTests(TestCase):
 
     def test_episode_fields(self):
         eps = {e['id']: e for e in self.get()['episodes']}
-        self.assertEqual(eps[self.a2.id], {
+        a2 = dict(eps[self.a2.id])
+        self.assertTrue(a2.pop('discovered_at'))   # when diora found it (the Strom's mark)
+        self.assertEqual(a2, {
             'id': self.a2.id, 'title': 'A2', 'audio_url': 'https://example.org/a2.mp3', 'duration_seconds': 0,
             'pub_date': '2026-01-02T09:00:00+00:00', 'artwork_url': '',
             'feed__id': self.fa.id, 'feed__title': 'Alpha', 'feed__artwork_url': 'https://example.org/a.jpg',
-            'position_seconds': 100, 'played': False, 'favorite': False, 'in_queue': True,
+            'position_seconds': 100, 'played': False, 'favorite': False, 'in_queue': True, 'in_later': False,
             'last_played_at': '2026-02-03T09:00:00+00:00',
         })
         a4 = eps[self.a4.id]
@@ -1104,7 +1106,7 @@ FEED_DEFAULTS = {
     'playback_speed': None, 'skip_intro': 0, 'skip_outro': 0, 'volume': 1.0, 'new_episodes': 'inbox',
     'include_words': '', 'exclude_words': '', 'min_duration': 0, 'auto_queue': False, 'auto_download': False,
     'notify': False, 'episode_sort': '', 'episode_filter': [], 'skip_silence': None, 'kind': 'podcast',
-    'folder': '', 'folders': [],
+    'folder': '', 'folders': [], 'refresh_interval': '',
 }
 
 
@@ -1132,6 +1134,7 @@ class FeedSettingsTests(TestCase):
             'new_episodes': 'inbox', 'include_words': 'News, Talk', 'exclude_words': 'Teaser', 'min_duration': 10,
             'auto_queue': False, 'auto_download': False, 'notify': False,
             'episode_sort': '', 'episode_filter': [], 'skip_silence': None, 'kind': 'podcast', 'folder': '', 'folders': [],
+            'refresh_interval': '',
         })
         self.feed.refresh_from_db()
         self.assertEqual((self.feed.playback_speed, self.feed.skip_intro, self.feed.volume, self.feed.min_duration),

podcasts/urls.py

@@ -35,6 +35,10 @@ urlpatterns = [
     path('queue/add/', views.queue_add, name='podcast_queue_add'),
     path('queue/remove/', views.queue_remove, name='podcast_queue_remove'),
     path('queue/reorder/', views.queue_reorder, name='podcast_queue_reorder'),
+    path('later/add/', views.later_add, name='podcast_later_add'),
+    path('later/remove/', views.later_remove, name='podcast_later_remove'),
+    path('lists/', views.lists_switch, name='podcast_lists_switch'),
+    path('stream/seen/', views.stream_seen, name='podcast_stream_seen'),
     path('settings/', views.podcast_settings, name='podcast_settings'),
     path('progress/save/', views.save_progress, name='podcast_save_progress'),
     path('progress/mark-played/', views.mark_played, name='podcast_mark_played'),

podcasts/views.py

@@ -32,8 +32,8 @@ from . import transcripts as transcripts_mod
 from . import youtube as youtube_mod
 from . import folders as folders_mod
 from .models import (
-    EpisodeProgress, EpisodeTranscript, PlayerDevice, PlayerState, PodcastEpisode, PodcastFeed, PodcastQueue,
-    MAX_SPEED, MIN_SPEED, PodcastSettings, counts_as_played,
+    EpisodeProgress, EpisodeTranscript, PlayerDevice, PlayerState, PodcastEpisode, PodcastFeed, PodcastLater,
+    PodcastQueue, MAX_SPEED, MIN_SPEED, PodcastSettings, counts_as_played,
 )
 from .shownotes import extract_timestamps, sanitize_html
 
@@ -191,6 +191,89 @@ def _leave_inbox(user, episode_ids):
     )
 
 
+# ---------------------------------------------------------------------------
+# Merkliste (docs/podcasts.md)
+# ---------------------------------------------------------------------------
+
+def _later_add(user, episode_ids, auto=False):
+    """Put episodes on the user's Merkliste (those not on it yet). Like queueing they are no
+    longer new; wanted again after "Folge löschen". Returns the ids added."""
+    have = set(PodcastLater.objects.filter(user=user, episode_id__in=episode_ids).values_list('episode_id', flat=True))
+    new = [i for i in dict.fromkeys(episode_ids) if i not in have]
+    if not new:
+        return []
+    PodcastLater.objects.bulk_create([PodcastLater(user=user, episode_id=i, auto=auto) for i in new], ignore_conflicts=True)
+    _leave_inbox(user, new)
+    EpisodeProgress.objects.filter(user=user, episode_id__in=new, discarded_at__isnull=False).update(discarded_at=None)
+    return new
+
+
+def _later_remove(user, episode_ids):
+    PodcastLater.objects.filter(user=user, episode_id__in=episode_ids).delete()
+
+
+def _later_data(user):
+    """The Merkliste: like the queue's rows (episode__… fields, position_seconds), plus
+    added_at, auto and in_queue (also in the Wiedergabeliste). Newest episode first; the
+    apps group it by podcast."""
+    items = list(
+        PodcastLater.objects.filter(user=user)
+        .order_by(F('episode__pub_date').desc(nulls_last=True), '-id')
+        .values(
+            'id', 'added_at', 'auto',
+            'episode__id', 'episode__title', 'episode__audio_url',
+            'episode__duration_seconds', 'episode__artwork_url',
+            'episode__feed__id', 'episode__feed__title', 'episode__feed__artwork_url',
+            'episode__pub_date',
+        )
+    )
+    ids = [i['episode__id'] for i in items]
+    positions = dict(EpisodeProgress.objects.filter(user=user, episode_id__in=ids).values_list('episode_id', 'position_seconds'))
+    queued = set(PodcastQueue.objects.filter(user=user, episode_id__in=ids).values_list('episode_id', flat=True))
+    for i in items:
+        i['position_seconds'] = positions.get(i['episode__id'], 0)
+        i['in_queue'] = i['episode__id'] in queued
+        i['added_at'] = i['added_at'].isoformat()
+        if i['episode__pub_date']:
+            i['episode__pub_date'] = i['episode__pub_date'].isoformat()
+    return items
+
+
+# How often a feed is fetched (docs/podcasts.md): fixed, or by its rhythm — a feed that
+# brings an episode every n days is fetched every n hours, within 15 minutes and a day.
+REFRESH_HOURS = {'hourly': 1, 'daily': 24, 'weekly': 24 * 7}
+REFRESH_AUTO_MIN_HOURS = 0.25
+REFRESH_AUTO_MAX_HOURS = 24
+
+
+def refresh_hours(feed, interval_days):
+    """Hours between two fetches of `feed` (a PodcastFeed or a dict with refresh_interval)."""
+    fixed = REFRESH_HOURS.get(getattr(feed, 'refresh_interval', None) if not isinstance(feed, dict) else feed.get('refresh_interval'))
+    if fixed:
+        return fixed
+    if interval_days is None:
+        return 1
+    return min(max(interval_days, REFRESH_AUTO_MIN_HOURS), REFRESH_AUTO_MAX_HOURS)
+
+
+def refresh_due(feeds, now=None):
+    """The ids of the feeds due for a fetch: never fetched, or longer ago than their
+    refresh_hours (with a few minutes' slack for the 15-minute cron)."""
+    now = now or timezone.now()
+    intervals = {}
+    due = []
+    for f in feeds:
+        if f.user_id not in intervals:
+            intervals[f.user_id] = _intervals(f.user)
+        if f.last_refreshed_at is None:
+            due.append(f.id)
+            continue
+        hours = refresh_hours(f, intervals[f.user_id].get(f.id))
+        if now - f.last_refreshed_at >= timedelta(hours=hours) - timedelta(minutes=5):
+            due.append(f.id)
+    return due
+
+
 # ---------------------------------------------------------------------------
 # Duration helper
 # ---------------------------------------------------------------------------
@@ -724,15 +807,21 @@ def _apply_feed(feed_obj, fetched, full=False):
             _leave_inbox(feed_obj.user, list(gone))
             new_episodes_list = [e for e in new_episodes_list if e.id not in gone]
 
-    if feed_obj.new_episodes == PodcastFeed.NEW_QUEUE and new_episodes_list and feed_obj.kind != PodcastFeed.AUDIOBOOK:
-        where = PodcastSettings.for_user(feed_obj.user).auto_queue_position
-        # Newest first in front of what's there; oldest first when appended,
-        # so the end of the queue plays in the order the episodes came out.
+    stamm = (PodcastFeed.NEW_QUEUE, PodcastFeed.NEW_PLAY)
+    if feed_obj.new_episodes in stamm and new_episodes_list and feed_obj.kind != PodcastFeed.AUDIOBOOK:
+        s = PodcastSettings.for_user(feed_obj.user)
         newest_first = sorted(
             new_episodes_list,
             key=lambda e: e.pub_date or e.discovered_at,
             reverse=True,
         )
+        if s.lists_v2 and feed_obj.new_episodes == PodcastFeed.NEW_QUEUE:
+            # Stamm: into the Merkliste (docs/podcasts.md).
+            _later_add(feed_obj.user, [e.id for e in newest_first], auto=True)
+        else:
+            where = s.auto_queue_position
+            # Newest first in front of what's there; oldest first when appended,
+            # so the end of the queue plays in the order the episodes came out.
             ordered = newest_first if where != PodcastSettings.BACK else newest_first[::-1]
             _enqueue(feed_obj.user, [e.id for e in ordered], where)
 
@@ -803,6 +892,7 @@ FEED_SETTING_FIELDS = (
     'playback_speed', 'skip_intro', 'skip_outro', 'volume',
     'new_episodes', 'include_words', 'exclude_words', 'min_duration', 'auto_queue', 'auto_download',
     'notify', 'episode_sort', 'episode_filter', 'skip_silence', 'kind', 'folder', 'folders',
+    'refresh_interval',
 )
 
 
@@ -811,6 +901,7 @@ def _feed_settings(feed):
 
 
 INTERVAL_EPISODES = 10   # a feed's newest episodes that tell how often it comes
+HEARD_MIN_SECONDS = 60   # listened into this far, an episode counts as heard (with played)
 INTERVAL_SINGLE_DAYS = 30
 
 
@@ -870,6 +961,8 @@ def _feeds_data(user):
             played_count=Count('id', filter=Q(played=True)),
             hidden_count=Count('id', filter=Q(played=True) | Q(dismissed=True)),
             last_played_at=Max('updated_at', filter=Q(played=True) | Q(position_seconds__gt=0)),
+            heard_365=Count('id', filter=(Q(played=True) | Q(position_seconds__gt=HEARD_MIN_SECONDS))
+                            & Q(updated_at__gte=timezone.now() - timedelta(days=365))),
         )
     }
 
@@ -880,6 +973,9 @@ def _feeds_data(user):
         mine = listened.get(f['id'], {})
         f['unplayed_count'] = max(f['episode_count'] - mine.get('played_count', 0), 0)
         f['new_count'] = max(f['episode_count'] - mine.get('hidden_count', 0), 0)
+        # Episodes heard in the last 12 months: the Strom puts the most listened first,
+        # any heard one before every never-heard one (docs/podcasts.md).
+        f['heard_365'] = mine.get('heard_365', 0)
         last = mine.get('last_played_at')
         f['last_played_at'] = last.isoformat() if last else None
         if f['last_refreshed_at']:
@@ -1052,6 +1148,12 @@ def _new_episodes(value):
     return value
 
 
+def _refresh_interval(value):
+    if value not in {key for key, _ in PodcastFeed.REFRESH_INTERVALS}:
+        raise ValueError
+    return value
+
+
 def _boolean(value):
     if not isinstance(value, bool):
         raise ValueError
@@ -1083,6 +1185,7 @@ FEED_SETTING_PARSERS = {
     'folder': lambda v: folders_mod.clean_name(v),
     'folders': folders_mod.clean_list,
     'skip_silence': _optional_boolean,
+    'refresh_interval': lambda v: _refresh_interval(v),
 }
 
 
@@ -1493,6 +1596,9 @@ EPISODE_FILTERS = {
     'favorites': Q(favorite=True),
     'history': Q(played=True) | Q(position_seconds__gt=0),
     'queued': Q(in_queue=True),
+    'later': Q(in_later=True),
+    # The Strom (docs/podcasts.md): every subscription but the "Auf Abruf" ones.
+    'stream': ~Q(feed__new_episodes=PodcastFeed.NEW_IGNORE),
 }
 # Episodes of unknown length (0) come last when sorting by length.
 _UNKNOWN_LENGTH_LAST = Case(When(duration_seconds__gt=0, then=Value(0)), default=Value(1), output_field=IntegerField())
@@ -1510,7 +1616,7 @@ EPISODE_LIST_MAX = 500
 EPISODE_LIST_FIELDS = (
     'id', 'title', 'audio_url', 'duration_seconds', 'pub_date', 'artwork_url',
     'feed__id', 'feed__title', 'feed__artwork_url',
-    'position_seconds', 'played', 'favorite', 'in_queue', 'last_played_at',
+    'position_seconds', 'played', 'favorite', 'in_queue', 'in_later', 'last_played_at', 'discovered_at',
 )
 
 
@@ -1542,6 +1648,7 @@ def _episode_list_data(user, filter_name='all', feed_id=None, q='', offset=0, li
             favorite=Coalesce('mine__favorite', Value(False)),
             last_played_at=F('mine__updated_at'),
             in_queue=Exists(PodcastQueue.objects.filter(user=user, episode=OuterRef('pk'))),
+            in_later=Exists(PodcastLater.objects.filter(user=user, episode=OuterRef('pk'))),
         )
     )
     for name in names:
@@ -1574,6 +1681,8 @@ def _episode_list_data(user, filter_name='all', feed_id=None, q='', offset=0, li
         ep['played'] = bool(ep['played'])
         ep['favorite'] = bool(ep['favorite'])
         ep['in_queue'] = bool(ep['in_queue'])
+        ep['in_later'] = bool(ep['in_later'])
+        ep['discovered_at'] = ep['discovered_at'].isoformat() if ep['discovered_at'] else None
     return episodes, total
 
 
@@ -1838,6 +1947,10 @@ def podcast_state(request):
         'inbox_total': _inbox_episodes(request.user).count(),
         'inbox_limit': limit,
         'settings': _settings_data(s),
+        # docs/podcasts.md: the Merkliste, whether the two lists are on, the Strom's mark.
+        'later': _later_data(request.user),
+        'lists_v2': s.lists_v2,
+        'stream_seen_at': s.stream_seen_at.isoformat() if s.stream_seen_at else None,
         'import_downloads': {'count': len(downloads), 'bytes': sum(e['size'] for e in downloads)},
         'discarded': _discarded_data(request.user),
         'server_time': timezone.now().isoformat(),
@@ -1956,6 +2069,90 @@ def queue_reorder(request):
     return JsonResponse({'ok': True})
 
 
+def _episode_ids(request):
+    """The user's episodes among body['episode_ids'] (in order), or a JsonResponse error."""
+    try:
+        body = json.loads(request.body or b'{}')
+    except (json.JSONDecodeError, ValueError):
+        return None, JsonResponse({'error': 'invalid JSON'}, status=400)
+    ids = body.get('episode_ids') if isinstance(body, dict) else None
+    if not isinstance(ids, list) or not all(isinstance(i, int) and not isinstance(i, bool) for i in ids):
+        return None, JsonResponse({'error': 'episode_ids (a list of numbers) required'}, status=400)
+    found = set(PodcastEpisode.objects.filter(pk__in=ids[:EPISODE_IDS_MAX], feed__user=request.user)
+                .values_list('id', flat=True))
+    return [i for i in ids if i in found], None
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def later_add(request):
+    """{"episode_ids": [...]}: onto the Merkliste (docs/podcasts.md)."""
+    if not request.user.is_authenticated:
+        return JsonResponse({'error': 'authentication required'}, status=401)
+    ids, err = _episode_ids(request)
+    if err:
+        return err
+    return JsonResponse({'ok': True, 'added': _later_add(request.user, ids)})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def later_remove(request):
+    """{"episode_ids": [...]}: off the Merkliste (the Wiedergabeliste is left alone)."""
+    if not request.user.is_authenticated:
+        return JsonResponse({'error': 'authentication required'}, status=401)
+    ids, err = _episode_ids(request)
+    if err:
+        return err
+    _later_remove(request.user, ids)
+    return JsonResponse({'ok': True})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def lists_switch(request):
+    """{"on": true}: the two lists of docs/podcasts.md — what is in the queue now becomes
+    the Merkliste, the Wiedergabeliste starts empty. {"on": false}: back, the Merkliste
+    joins the end of the queue."""
+    if not request.user.is_authenticated:
+        return JsonResponse({'error': 'authentication required'}, status=401)
+    try:
+        body = json.loads(request.body or b'{}')
+    except (json.JSONDecodeError, ValueError):
+        return JsonResponse({'error': 'invalid JSON'}, status=400)
+    on = bool(body.get('on')) if isinstance(body, dict) else False
+    user = request.user
+    with transaction.atomic():
+        s = PodcastSettings.for_user(user)
+        if on and not s.lists_v2:
+            _later_add(user, _queue_order(user))
+            PodcastQueue.objects.filter(user=user).delete()
+        elif not on and s.lists_v2:
+            later = list(PodcastLater.objects.filter(user=user)
+                         .order_by(F('episode__pub_date').asc(nulls_last=True)).values_list('episode_id', flat=True))
+            _enqueue(user, later, PodcastSettings.BACK)
+            PodcastLater.objects.filter(user=user).delete()
+        s.lists_v2 = on
+        s.save(update_fields=['lists_v2'])
+    return JsonResponse({'ok': True, 'lists_v2': on, 'queue': len(_queue_order(user)),
+                         'later': PodcastLater.objects.filter(user=user).count()})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def stream_seen(request):
+    """The user looked at the Strom now: returns the time before (episodes found after it
+    carry the mark) and keeps now."""
+    if not request.user.is_authenticated:
+        return JsonResponse({'error': 'authentication required'}, status=401)
+    s = PodcastSettings.for_user(request.user)
+    before = s.stream_seen_at
+    s.stream_seen_at = timezone.now()
+    s.save(update_fields=['stream_seen_at'])
+    return JsonResponse({'ok': True, 'before': before.isoformat() if before else None,
+                         'now': s.stream_seen_at.isoformat()})
+
+
 def _choice(choices, what):
     def parse(value):
         if value not in choices:
@@ -2125,6 +2322,7 @@ def save_progress(request):
     # Within the last seconds: played (same rule as the apps).
     if counts_as_played(position, episode.duration_seconds):
         progress.played = True
+        _later_remove(request.user, [episode.id])   # heard: off the Merkliste
 
     progress.save()
     return JsonResponse({'ok': True, 'played': progress.played})
@@ -2159,6 +2357,8 @@ def mark_played(request):
         progress.position_seconds = 0   # heard to the end, now "not heard": from the start again
         fields.append('position_seconds')
     progress.save(update_fields=fields)
+    if played:
+        _later_remove(request.user, [episode.id])
     return JsonResponse({'ok': True, 'played': played})
 
 
@@ -2245,6 +2445,7 @@ def discard_episodes(request):
     now = timezone.now()
     with transaction.atomic():
         _dequeue(request.user, valid)
+        _later_remove(request.user, valid)
         # update() leaves updated_at alone: throwing an episode away isn't listening to it.
         rows = EpisodeProgress.objects.filter(user=request.user, episode_id__in=valid)
         rows.update(dismissed=True, discarded_at=now)

static/js/podcasts-web.js

@@ -96,7 +96,8 @@
   };
 
   function empty() {
-    return {feeds: [], queue: [], inbox: [], settings: {enqueue_position: 'back', auto_queue_position: 'front'}};
+    return {feeds: [], queue: [], inbox: [], later: [], lists_v2: false,
+      settings: {enqueue_position: 'back', auto_queue_position: 'front'}};
   }
 
   let state = load('state', null) || empty();
@@ -182,6 +183,8 @@
         if (r.status !== 200) return false;
         const fresh = await r.json();
         if (outbox.length) continue;
+        // The Strom's marks of this visit stay until the next look (podcastStreamSeen).
+        if (state && state.stream_seen_before !== undefined) fresh.stream_seen_before = state.stream_seen_before;
         state = fresh;
         save();
         changed();
@@ -260,6 +263,7 @@
 
   function markInQueue(s, ids, inQueue) {
     for (const o of s.inbox || []) if (ids.has(o.id)) o.in_queue = inQueue;
+    for (const o of s.later || []) if (ids.has(o.episode__id)) o.in_queue = inQueue;
   }
 
   function forEpisode(s, id, block) {
@@ -288,12 +292,45 @@
       const ids = fresh.map(e => e.episode__id);
       setQueueOrder(s, order.slice(0, at).concat(ids, order.slice(at)));
       removeFromInbox(s, new Set(ids));   // queued means no longer new
+      markInQueue(s, new Set(ids), true);
       const body = {episode_ids: ids, where: pos};
       if (currentId) body.current_episode_id = currentId;
       queueOp('/podcasts/queue/add/', body);
     });
   }
 
+  // ---- the Merkliste (docs/podcasts.md): state.later, entries like the queue's ------------------
+
+  function laterOf(s) {
+    if (!Array.isArray(s.later)) s.later = [];
+    return s.later;
+  }
+
+  function laterAdd(episodes) {
+    edit(s => {
+      const have = new Set(laterOf(s).map(e => e.episode__id));
+      const fresh = episodes.map(asQueueEntry).filter(e => !have.has(e.episode__id) && have.add(e.episode__id));
+      if (!fresh.length) return;
+      const queued = new Set(queueIds(s));
+      for (const e of fresh) {
+        laterOf(s).unshift(Object.assign({}, e, {added_at: new Date().toISOString(), auto: false, in_queue: queued.has(e.episode__id)}));
+      }
+      removeFromInbox(s, new Set(fresh.map(e => e.episode__id)));
+      queueOp('/podcasts/later/add/', {episode_ids: fresh.map(e => e.episode__id)});
+    });
+  }
+
+  function dropLater(s, set) {
+    if (Array.isArray(s.later)) s.later = s.later.filter(e => !set.has(e.episode__id));
+  }
+
+  function laterRemove(ids) {
+    edit(s => {
+      dropLater(s, new Set(ids));
+      queueOp('/podcasts/later/remove/', {episode_ids: ids});
+    });
+  }
+
   function dequeueAll(ids) {
     edit(s => {
       const gone = new Set(ids);
@@ -348,6 +385,7 @@
       if (!played) return;
       const set = new Set(ids);
       removeFromInbox(s, set);
+      dropLater(s, set);   // heard: off the Merkliste (the server does the same)
       for (const id of ids) forEpisode(s, id, o => { o.position_seconds = 0; });
       if (leaveQueue) {
         const inQueue = queueIds(s).filter(id => set.has(id));
@@ -378,6 +416,7 @@
       if (order.some(id => set.has(id))) setQueueOrder(s, order.filter(id => !set.has(id)));
       markInQueue(s, set, false);
       removeFromInbox(s, set);
+      dropLater(s, set);
       queueOp('/podcasts/episodes/discard/', {episode_ids: ids});
     });
     dropDiscarded(ids);
@@ -1832,6 +1871,24 @@
     },
     podcastMarkPlayedAll: sent((json, played) => markPlayedAll(ids(json), !!played, prefs.remove_played)),
     podcastDismiss: sent(json => dismiss(ids(json))),
+    // The Merkliste and the two lists (docs/podcasts.md).
+    podcastLaterAdd: sent(json => laterAdd(parse(json, []) || [])),
+    podcastLaterRemove: sent(json => laterRemove(ids(json))),
+    podcastListsSwitch(on) {
+      queueOp('/podcasts/lists/', {on: !!on});
+      save();
+      flush().then(() => refresh(false)).then(ok => { if (window.__dioraPodcastsRefreshed) window.__dioraPodcastsRefreshed(ok); });
+    },
+    // The Strom was looked at: the time before (its marks) stays in the state as
+    // stream_seen_before; the server keeps now.
+    podcastStreamSeen() {
+      edit(s => {
+        s.stream_seen_before = s.stream_seen_at || null;
+        s.stream_seen_at = new Date().toISOString();
+        queueOp('/podcasts/stream/seen/', {}, 'stream-seen');
+      });
+      flush();
+    },
     podcastDiscard: sent(json => discard(ids(json))),
     podcastSettings: sent(json => setSettings(parse(json, {}) || {})),
     podcastAutoQueue: sent((feedId, on) => setAutoQueue(feedId, !!on)),

static/js/sw.js

@@ -2,7 +2,7 @@
  * diora service worker — caches the app shell for offline use.
  */
 
-const CACHE = 'diora-v135';
+const CACHE = 'diora-v136';
 const SHELL = [
   '/static/css/app.css',
   '/static/js/app/01-core.js',