From f242605a9785777ea61614a2bbb9f71858204510 Mon Sep 17 00:00:00 2001 From: mrwnslz Date: Mon, 9 Nov 2026 09:16:16 +0100 Subject: [PATCH] Smarte Transkription: Vertrag und Prompt (docs/transcription.md), Abschnitt Transkripte in docs/podcasts.md, Podcast-Artikel in docs/ai.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Transkript und Artikel nur auf Knopfdruck; der Server transkribiert mit Whisper (eingebaut oder eigene Instanz), das Gerät schreibt den Artikel mit dem eigenen Schlüssel, sofort oder beim nächsten Öffnen; Benachrichtigung über /feeds/notifications/. Co-Authored-By: Claude Opus 5.5 --- docs/ai.md | 3 + docs/podcasts.md | 9 ++ docs/transcription.md | 216 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 228 insertions(+) create mode 100644 docs/transcription.md diff --git a/docs/ai.md b/docs/ai.md index 3ed9ece..e488831 100644 --- a/docs/ai.md +++ b/docs/ai.md @@ -29,6 +29,8 @@ kept by the server as a blob it can't read: the key can't be read: the settings ask for it again. Not on diora Reader (e-readers). - Server-side jobs (in the background, without an app open) were left out on purpose: they would need the key on the server. The project Radar has its own server key (docs/projects.md). + Podcast articles wait on the server as a wish until a device with the key is open + (docs/transcription.md). ## Building blocks (`static/js/ai.js`, an ES module) @@ -53,5 +55,6 @@ kept by the server as a blob it can't read: | Bücher, Zeitschriften | ✦ in the library's head (`static/js/app/55-library-tools.js`) | per book: id, title, author, folders, series, magazine mark; the existing folders | folders to add (existing first; new ones only for ≥ 3 books), series with volume, cleaned title/author where broken, magazine mark | | Podcasts | ✦ in Abos (diora-android `web-native/static/native-podcast.js`, copied to `static/podcasts/`) | per subscription: title, author, folders, level, heard in 12 months, publishing interval, three newest episode titles | folders to add (≤ 2), level Stamm / Strom / Auf Abruf (only with a clear reason) | | Nachrichten, Blogs | ✦ in the feeds' bar (`static/js/feeds.js`; the apps' section bar presses it) | per source: title, address, kind, Ressort or folders, recent titles shown | Ressort for news, folders for blogs, news ↔ blog when clearly wrong | +| Podcast-Artikel | ✦ Artikel in an episode (`static/js/podcast-article.js`, docs/transcription.md) | the episode's transcript (feed or Whisper), title, author, shownotes, chapters | an article from the speakers' standpoint with summary and sections, saved to the episode and to Blogs (no review sheet: it is new text, removable) | Applied through `_updateBookMeta` (the encrypted meta, as when editing by hand). diff --git a/docs/podcasts.md b/docs/podcasts.md index 566f414..e8b3145 100644 --- a/docs/podcasts.md +++ b/docs/podcasts.md @@ -182,3 +182,12 @@ Listening without committing: one button plays the Strom like a station. - Web host (`static/js/podcasts-web.js`, `DioraPodcastHost`): `podcastLaterAdd(json episodes)`, `podcastLaterRemove(json ids)`, `podcastListsSwitch(on)`, `podcastStreamSeen()` (keeps the previous time as `state.stream_seen_before` for the marks). + +## Transcripts + +An episode's Transkript tab shows one of two kinds. **From the feed**: `` +(SRT, VTT, JSON, Podlove), fetched and parsed on first use and cached in `EpisodeTranscript` +(`podcasts/transcripts.py`, `GET /podcasts/episodes//transcript/`). **Made by Whisper** +(since 2026-11): only when the user asks, kept on `TranscriptionJob`; the feed's wins when both +exist. From a transcript the device can write an article that is pinned to the episode and shows +in Blogs. All of that is in docs/transcription.md. diff --git a/docs/transcription.md b/docs/transcription.md new file mode 100644 index 0000000..af270df --- /dev/null +++ b/docs/transcription.md @@ -0,0 +1,216 @@ +# Smarte Transkription: Transkript und Artikel zu einer Folge + +Decided with the user on 2026-11-09. Plan and order of work: +diora-android `plans/smarte-transkription.md`. + +## What it is for + +Sometimes the shownotes don't tell whether an episode still has something to teach. Then the +user presses a button and, a few hours later at most, gets a readable article written from the +speakers' own standpoint. It is meant to let them judge the episode for themselves, so it keeps +arguments, reasons and evidence and does not just summarise them. The article sits on the +episode and in Blogs. + +Nothing happens on its own. The server never transcribes ahead, by cron or on subscribing; the +only way to start a transcription is the `POST` below, sent when the user presses a button. + +## Who does what + +The **server** transcribes. It already downloads and decodes episodes for skipping silence +(`podcasts/silence.py`), and browsers and the WebView can't fetch podcast audio (CORS). It cuts +the audio at pauses and sends the pieces in parallel to a Whisper instance with an +OpenAI-compatible API (`POST /v1/audio/transcriptions`): the built-in one by default (a container +beside diora-web, `DIORA_WHISPER_URL`), or any instance the user enters. YouTube episodes are +not transcribed: the server never fetches their sound (docs/youtube.md). + +The **device** writes the article with the user's own key (docs/ai.md). The key never reaches the +server. It is one LLM call of a few minutes. If diora is open when the transcript is done, it +happens right away; otherwise the next time diora is opened on any of the user's devices. The +server only remembers that an article was wanted. + +Transcript and article are stored in plain text on the server, like all podcast and feed data. + +## Flow + +1. In the episode the user presses "Transkript erstellen" (the Transkript tab) or + "✦ Artikel erstellen" / "✦ Transkribieren und Artikel erstellen". The second sends + `article: true`. With a transcript already there (from the feed or an earlier run) only the + article is written. +2. The server works through the episode (minutes to hours, no GPU on Strato). The UI shows + "Transkribiere 7/38 …" while it is open. +3. Done: a notice for `GET /feeds/notifications/`. Android's BackgroundSync and the desktop show + "Transkript fertig: ", with "Artikel wird beim Öffnen geschrieben" when one is wanted. + Tapping it opens diora at the episode. +4. A device with a key (`static/js/podcast-article.js`) picks up the open wishes, reserves one, + writes the article and saves it. It then says "Artikel fertig" (on Android as a system + notification through the bridge) and opens it on a tap. +5. The article is the first tab of the episode ("Artikel") and an item of the favourite blog + "Podcast-Artikel", linking back to the episode. + +## API + +**Settings.** `GET /podcasts/transcription/settings/` returns +`{engine: "builtin"|"custom", url, model, language, parallel, has_token, builtin_available}`. +`POST` takes the same fields and `token` (write-only; `""` removes it). The token is kept +encrypted with Fernet like `feeds.SiteLogin.secret`. `language` is `""` for automatic or an ISO +code (`de`). `parallel` is 1 to 8, default 3. `builtin_available` is whether `DIORA_WHISPER_URL` +is set. `url` is the instance's base, e.g. `https://whisper.example.org/v1`; the server appends +`/audio/transcriptions`. + +`POST /podcasts/transcription/test/` sends one second of silence to the configured instance and +returns `{ok, model, error}`. A custom URL goes through the same SSRF check as chapters +(`podcasts/chapters.py` `_open`): no private, loopback or link-local addresses. Only the built-in +URL from `DIORA_WHISPER_URL` is exempt. + +**Job.** `POST /podcasts/episodes//transcribe/` with `{article: true|false}` starts a job, or +updates `want_article` on a running one. With a transcript already there it returns +`{state: "done"}` straight away (and keeps `want_article`). `GET` on the same address returns +`{state, chunks_done, chunks_total, error, want_article, started_at, finished_at}`, with `state` +one of `none`, `queued`, `downloading`, `transcribing`, `done`, `error`. `DELETE` cancels a +queued or running job. The server runs one job at a time; the others wait as `queued`. A YouTube +episode returns 400 `{error: "youtube"}`, an episode over 800 MB 400 `{error: "too large"}`. + +**Transcript.** The result is kept on the job (`TranscriptionJob.data`) in the format of +`EpisodeTranscript.data`: `{segments: [{start, end, speaker, text}], source: {type: "whisper", +model, language}}`. `speaker` is `""`. It is kept apart from `EpisodeTranscript`, which is a cache +of the feed's transcripts and is rewritten when the feed changes them. +`GET /podcasts/episodes//transcript/` returns the feed's transcript when the feed has a +usable one, else the Whisper one, else 404 as before. `episode_detail` adds +`transcript_origin` (`"feed"`, `"whisper"` or `null`), `has_transcript` is true for both, and it +adds `article_item_id` (or `null`) and `transcription` (the job's state as above, or `null`). + +**Article.** `POST /podcasts/episodes//article/` takes `{title, summary, body_html, model, +kind}` and returns `{item_id}`. The server cleans `body_html` with `feeds/fetch.py clean_html`. +It creates the user's source "Podcast-Artikel" when missing (`kind` blog, `fav` true, +`synthetic` true, `url` `diora://podcast-artikel`) and adds a `feeds.Item` there, or replaces the +episode's earlier one. Item fields: the episode's link (else its audio URL) as `url`, the feed's +author as `author`, the episode's artwork as `image_url`, now as `published` (so it is on top of +the blog stream under "heute"), `words` counted. Saving clears the wish. `DELETE` on the same +address removes the article and its item. `GET /feeds/items//` and `_item_data` carry +`episode_id` for such items. + +Synthetic sources are skipped by the feed refresh (`refresh_newsfeeds`, `refresh_now`) and by the +pruning of old blog items (`feeds/refresh.py`). They can be hidden or have their bell turned off +like any blog; they can't be refreshed, renamed to another URL or turned into news. + +**Open wishes.** `GET /podcasts/transcription/pending/` returns +`[{episode_id, title, podcast, duration_seconds}]`: the user's episodes whose transcript is done, +that want an article, have none and aren't reserved. `POST +/podcasts/episodes//article/claim/` reserves one for 15 minutes for the calling device +(200), or answers 409 when another device holds it, so phone and desktop don't both write it. +`DELETE` on the claim releases it (the device gave up, e.g. no key or an error the user saw). + +**Notice.** `GET /feeds/notifications/` additionally takes `t_after` and returns +`transcripts: [{job_id, episode_id, title, podcast, want_article}]` (jobs done after that id) and +`t_last`. Without `t_after` (a device's first time) the list is empty and only `t_last` comes, +like `after`/`last`. Devices that don't know the fields ignore them. + +## Transcribing (`podcasts/transcribe.py`) + +Download with the same limits as `silence.py`, decode with imageio-ffmpeg to 16 kHz mono. Cut +into pieces of about eight minutes; each cut moves to the longest pause within ±30 seconds +(`silence.py`'s detection), so no word is cut. Pieces go out as FLAC to the built-in instance +and as Opus 32 kbit/s to custom ones, to keep uploads small. + +Each piece is sent with `model`, `language` (when set), `response_format=verbose_json` and a +`prompt` of podcast title, episode title and author names, so Whisper spells names right. Pieces +run in a thread pool of `parallel` workers; each is retried up to three times with a growing +pause; progress is written after each. Segment times are shifted by the piece's start; the +segments are joined and smoothed with `transcripts.merge`. A job runs in a daemon thread, like +`silence.ensure()`; one per server. A job found `downloading` or `transcribing` after a restart +is started again. + +## The article (`static/js/podcast-article.js`) + +Input: the transcript as plain text, one line per segment group of about 30 seconds, +`[mm:ss] Sprecher: text` (or `[mm:ss] text` without speakers), plus the episode's metadata: +podcast title, author, episode title, date, duration, the shownotes as plain text and the +chapters. Over about 120,000 tokens (more than about nine hours) the device stops with a clear +message for now. A five-hour episode is about 70,000 tokens. + +`DioraAI.ask({system, prompt, schema, effort: "medium", maxTokens: 32000, onProgress, signal})`. +`onProgress(words)` reports the words written so far while streaming ("Schreibe Artikel … 2.300 +Wörter"); `signal` aborts. Both are optional and new in ai.js; the existing callers don't change. + +Schema: + +```json +{ + "type": "object", + "additionalProperties": false, + "required": ["title", "summary", "kind", "speakers", "sections"], + "properties": { + "title": {"type": "string"}, + "summary": {"type": "string"}, + "kind": {"type": "string", "enum": ["Interview", "Gespräch", "Monolog", "Diskussion"]}, + "speakers": {"type": "array", "items": {"type": "string"}}, + "sections": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["heading", "start", "body_markdown"], + "properties": { + "heading": {"type": "string"}, + "start": {"type": "integer"}, + "body_markdown": {"type": "string"} + } + } + } + } +} +``` + +The HTML: the summary as a lead paragraph, then per section an `

`, below it a link +"▶ ab 12:34" (``; `clean_html` keeps fragment links, and feeds.js plays the +item's `episode_id` from there when such a link is tapped) and the body. Markdown is limited to paragraphs, emphasis, lists and block +quotes, converted on the device; the server cleans the result anyway. + +System prompt (the user's words decide the perspective; keep its substance when changing it): + +```text +You turn the transcript of a podcast episode into an article that reads as if the people +speaking had written it themselves instead of recording the episode. + +The reader wants to decide whether this episode still has something to teach them. They do not +want a summary; they want to understand what is argued here well enough to judge it for +themselves. Write for that reader. + +Perspective. Write from the speakers' standpoint and state their positions directly, as their +own claims. Never distance yourself from them: no "im Podcast wird die These aufgestellt", "der +Sprecher meint", "in dieser Folge geht es um", "die Gäste diskutieren", "laut X". If the episode +is one person speaking, write as that person argues, in the third person or impersonal voice, +but without reporting verbs. Do not add your own opinion, criticism or fact checks, and do not +soften what is said. + +Conversations. Interviews, conversations and discussions are condensed into their core theses; +the form of question and answer disappears. Where people hold different positions, keep them +apart and name who holds which ("Zuckermann hält dagegen, dass …"), but do not turn it into +minutes. Questions from a host matter only where they shape the argument. When the transcript +has no speaker names, infer them from the context (introductions, the shownotes, how people +address each other); if you can't, do not invent names. + +Substance over summary. Give the arguments with their reasons, examples and evidence, not just +the conclusions. Keep figures, dates, quotations that carry an argument (short, in quotation +marks, attributed), and every work mentioned (books, articles, studies, films) with author and +title. Make clear what is new, distinctive or surprising in the episode and where it restates +what is commonly said, by how you weight and order it, without grading it. + +Leave out: advertising and sponsors, appeals for donations, subscriptions or ratings, notes on +the podcast itself (schedules, technology, community), greetings, small talk and digressions that +lead nowhere. + +Form. Write in the language of the episode. `title`: a title for the article, not the episode +title repeated. `summary`: three to six sentences up front that state the central thesis and how +the episode gets there. `sections`: as many as the content needs (usually four to ten), each +with a heading that says something (not "Einleitung"), `start` = the second in the episode where +its content begins (from the [mm:ss] marks), and `body_markdown` in plain paragraphs; lists only +where the speakers list things themselves. Follow the order of the episode unless a different +order makes the argument clearer. Length: as long as needed for the reader to judge the episode, +roughly a tenth of the transcript, at most about 5,000 words. + +The transcript was made by speech recognition: fix obvious mishearings, especially of names, +using the metadata, and don't quote garbled passages. +``` + +Prompt (user turn): the metadata block, then `Transkript:` and the text.