diora-web/docs/transcription.md

Smarte Transkription: Transkript und Artikel zu einer Folge

Decided with the user on 2026-11-09. Plan and order of work: diora-android plans/smarte-transkription.md.

What it is for

Sometimes the shownotes don't tell whether an episode still has something to teach. Then the user presses a button and, a few hours later at most, gets a readable article written from the speakers' own standpoint. It is meant to let them judge the episode for themselves, so it keeps arguments, reasons and evidence and does not just summarise them. The article sits on the episode and in Blogs.

Nothing happens on its own. The server never transcribes ahead, by cron or on subscribing; the only way to start a transcription is the POST below, sent when the user presses a button.

Who does what

The server transcribes. It already downloads and decodes episodes for skipping silence (podcasts/silence.py), and browsers and the WebView can't fetch podcast audio (CORS). It cuts the audio at pauses and sends the pieces in parallel to a Whisper instance with an OpenAI-compatible API (POST /v1/audio/transcriptions): the built-in one by default (DIORA_WHISPER_URL), or any instance the user enters. YouTube episodes are not transcribed: the server never fetches their sound (docs/youtube.md).

The device writes the article with the user's own key (docs/ai.md). The key never reaches the server. It is one LLM call of a few minutes. If diora is open when the transcript is done, it happens right away; otherwise the next time diora is opened on any of the user's devices. The server only remembers that an article was wanted.

Transcript and article are stored in plain text on the server, like all podcast and feed data.

Flow

1. In the episode the user presses "Transkript erstellen" (the Transkript tab) or "✦ Artikel erstellen" / "✦ Transkribieren und Artikel erstellen". The second sends article: true. With a transcript already there (from the feed or an earlier run) only the article is written.
2. The server works through the episode (minutes to hours, no GPU on Strato). The UI shows "Transkribiere 7/38 …" while it is open.
3. Done: a notice for GET /feeds/notifications/. Android's BackgroundSync and the desktop show "Transkript fertig: ", with "Artikel wird beim Öffnen geschrieben" when one is wanted. Tapping it opens diora at the episode.
4. A device with a key (static/js/podcast-article.js) picks up the open wishes, reserves one, writes the article and saves it. It then says "Artikel fertig" (on Android as a system notification through the bridge, DioraNative.notify(title, text, target) with target episode:<id> or item:<id>; elsewhere a toast) and opens it on a tap.
5. The article is the first tab of the episode ("Artikel") and an item of the favourite blog "Podcast-Artikel", linking back to the episode.

API

Settings. GET /podcasts/transcription/settings/ returns {engine: "builtin"|"custom", url, model, language, parallel, has_token, builtin_available}. POST takes the same fields and token (write-only; "" removes it). The token is kept encrypted with Fernet like feeds.SiteLogin.secret. language is "" for automatic or an ISO code (de). parallel is 1 to 8, default 3. builtin_available is whether DIORA_WHISPER_URL is set. url is the instance's base, e.g. https://whisper.example.org/v1; the server appends /audio/transcriptions.

An empty model is sent as whisper-1. The built-in URL is read into settings.WHISPER_URL from DIORA_WHISPER_URL. A custom engine needs a url; the token is only sent to custom instances.

POST /podcasts/transcription/test/ sends one second of silence to the configured instance and returns {ok, model, error}. A custom URL goes through the same SSRF check as chapters (podcasts/chapters.py _open): no private, loopback or link-local addresses. Only the built-in URL from DIORA_WHISPER_URL is exempt.

Job. POST /podcasts/episodes/<id>/transcribe/ with {article: true|false} starts a job, or updates want_article on a running one. With a transcript already there it returns {state: "done"} straight away (and keeps want_article). GET on the same address returns {state, chunks_done, chunks_total, error, want_article, started_at, finished_at}, with state one of none, queued, downloading, transcribing, done, error. DELETE cancels a queued or running job (its row is deleted, so GET then says none; a finished one stays). The server runs one job at a time; the others wait as queued. A YouTube episode returns 400 {error: "youtube"}, an episode over 800 MB 400 {error: "too large"}. The size is asked with a HEAD request (Content-Length; if that fails the job starts anyway) and checked again while downloading, where it ends the job with the error too large. article: true is only ever raised, never lowered, by a repeated POST; a POST after an error starts a new job (at the back of the queue). Jobs run in id order, which is why the notice below can use ids as its cursor. A feed transcript already there gives a job that is done with no data: it only holds the wish for an article and causes no transcripts notice.

Transcript. The result is kept on the job (TranscriptionJob.data) in the format of EpisodeTranscript.data: {segments: [{start, end, speaker, text}], source: {type: "whisper", model, language}}. speaker is "". It is kept apart from EpisodeTranscript, which is a cache of the feed's transcripts and is rewritten when the feed changes them. GET /podcasts/episodes/<id>/transcript/ returns the feed's transcript when the feed has a usable one, else the Whisper one, else 404 as before. episode_detail adds transcript_origin ("feed", "whisper" or null), has_transcript is true for both, and it adds article_item_id (or null) and transcription (the job's state as above, or null).

Article. POST /podcasts/episodes/<id>/article/ takes {title, summary, body_html, model, kind} and returns {item_id}. The server cleans body_html with feeds/fetch.py clean_html. It creates the user's source "Podcast-Artikel" when missing (kind blog, fav true, synthetic true, url diora://podcast-artikel) and adds a feeds.Item there, or replaces the episode's earlier one. Item fields: the episode's link (else its audio URL) as url, the feed's author as author, the episode's artwork as image_url, now as published (so it is on top of the blog stream under "heute"), words counted. Saving clears the wish. DELETE on the same address removes the article and its item. GET /feeds/items/<id>/ and _item_data carry episode_id for such items. body_html may contain href="#t=<seconds>" links; clean_html keeps exactly this fragment form (podcasts/shownotes.py TIME_LINK) and no other # links. Validation: title 1 to 300 characters, body_html non-empty after cleaning and at most 1,000,000 characters, summary at most 2000, model at most 100, kind at most 20 (400 otherwise). The item has content_full true (no page is ever fetched) and, as the reader's furniture filter would strip repeated "▶ ab" blocks, GET /feeds/items/<id>/ skips that filter for synthetic sources. The endpoint is podcasts/article.py.

Synthetic sources are skipped by the feed refresh (refresh_newsfeeds, refresh_now) and by the pruning of old blog items (feeds/refresh.py). They can be hidden or have their bell turned off like any blog; they can't be refreshed, renamed to another URL or turned into news. refresh.refresh() itself drops them, the OPML export leaves them out, and _source_data carries synthetic.

Open wishes. GET /podcasts/transcription/pending/ returns [{episode_id, title, podcast, duration_seconds}]: the user's episodes whose transcript is done, that want an article, have none and aren't reserved. POST /podcasts/episodes/<id>/article/claim/ reserves one for 15 minutes for the calling device (200), or answers 409 when another device holds it, so phone and desktop don't both write it. DELETE on the claim releases it (the device gave up, e.g. no key or an error the user saw). The device names itself with device in the JSON body (or, if that is missing, the X-Diora-Device header): a short stable name such as android, desktop or browser, at most 60 characters; a claim without one is 400. The same device claiming again renews its 15 minutes. A claim on an episode with no open wish (no job done, no article wanted, or an article already there) is 404. DELETE by a device that does not hold the claim is 409.

Notice. GET /feeds/notifications/ additionally takes t_after and returns transcripts: [{job_id, episode_id, title, podcast, want_article}] (jobs done after that id) and t_last. Without t_after (a device's first time) the list is empty and only t_last comes, like after/last. Devices that don't know the fields ignore them.

Transcribing (podcasts/transcribe.py)

Download with the same limits as silence.py, decode with imageio-ffmpeg to 16 kHz mono. Cut into pieces of about eight minutes; each cut moves to the longest pause within ±30 seconds (silence.py's detection), so no word is cut. Pieces go out as FLAC to the built-in instance and as Opus 32 kbit/s to custom ones, to keep uploads small.

Each piece is sent with model, language (when set), response_format=verbose_json and a prompt of podcast title, episode title and author names, so Whisper spells names right. Pieces run in a thread pool of parallel workers; each is retried up to three times with a growing pause; progress is written after each. Segment times are shifted by the piece's start; the segments are joined and smoothed with transcripts.merge. A job runs in a daemon thread, like silence.ensure(); one per server. A job found downloading or transcribing after a restart is started again. This happens when the first request (GET/POST on /transcribe/ or /transcription/pending/) after a start reaches the server. Cancelling deletes the job's row; the worker notices at the latest after five seconds, stops sending pieces and removes its temporary files (they live in a temporary directory that is always removed). Only the worker's main thread writes to the database. The settings used are those of the episode's owner, read when the job starts. A refusal by the instance with a 4xx status other than 408 and 429 is not retried and ends the job with Whisper: HTTP <code>.

The article (static/js/podcast-article.js)

Input: the transcript as plain text, one line per segment group of about 30 seconds, [mm:ss] Sprecher: text (or [mm:ss] text without speakers), plus the episode's metadata: podcast title, author, episode title, date, duration, the shownotes as plain text and the chapters. Over about 120,000 tokens (more than about nine hours) the device stops with a clear message for now. A five-hour episode is about 70,000 tokens.

DioraAI.ask({system, prompt, schema, effort: "medium", maxTokens: 32000, onProgress, signal}). onProgress(words) reports the words written so far while streaming ("Schreibe Artikel … 2.300 Wörter"); signal aborts. Both are optional and new in ai.js; the existing callers don't change.

Schema:

{
  "type": "object",
  "additionalProperties": false,
  "required": ["title", "summary", "kind", "speakers", "sections"],
  "properties": {
    "title": {"type": "string"},
    "summary": {"type": "string"},
    "kind": {"type": "string", "enum": ["Interview", "Gespräch", "Monolog", "Diskussion"]},
    "speakers": {"type": "array", "items": {"type": "string"}},
    "sections": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["heading", "start", "body_markdown"],
        "properties": {
          "heading": {"type": "string"},
          "start": {"type": "integer"},
          "body_markdown": {"type": "string"}
        }
      }
    }
  }
}

The HTML: the summary as a lead paragraph, then per section an <h2>, below it a link "▶ ab 12:34" (<a href="#t=754">; clean_html keeps fragment links, and feeds.js plays the item's episode_id from there when such a link is tapped) and the body. Markdown is limited to paragraphs, emphasis, lists and block quotes, converted on the device; the server cleans the result anyway.

System prompt (the user's words decide the perspective; keep its substance when changing it):

You turn the transcript of a podcast episode into an article that reads as if the people
speaking had written it themselves instead of recording the episode.

The reader wants to decide whether this episode still has something to teach them. They do not
want a summary; they want to understand what is argued here well enough to judge it for
themselves. Write for that reader.

Perspective. Write from the speakers' standpoint and state their positions directly, as their
own claims. Never distance yourself from them: no "im Podcast wird die These aufgestellt", "der
Sprecher meint", "in dieser Folge geht es um", "die Gäste diskutieren", "laut X". If the episode
is one person speaking, write as that person argues, in the third person or impersonal voice,
but without reporting verbs. Do not add your own opinion, criticism or fact checks, and do not
soften what is said.

Conversations. Interviews, conversations and discussions are condensed into their core theses;
the form of question and answer disappears. Where people hold different positions, keep them
apart and name who holds which ("Zuckermann hält dagegen, dass …"), but do not turn it into
minutes. Questions from a host matter only where they shape the argument. When the transcript
has no speaker names, infer them from the context (introductions, the shownotes, how people
address each other); if you can't, do not invent names.

Substance over summary. Give the arguments with their reasons, examples and evidence, not just
the conclusions. Keep figures, dates, quotations that carry an argument (short, in quotation
marks, attributed), and every work mentioned (books, articles, studies, films) with author and
title. Make clear what is new, distinctive or surprising in the episode and where it restates
what is commonly said, by how you weight and order it, without grading it.

Leave out: advertising and sponsors, appeals for donations, subscriptions or ratings, notes on
the podcast itself (schedules, technology, community), greetings, small talk and digressions that
lead nowhere.

Form. Write in the language of the episode. `title`: a title for the article, not the episode
title repeated. `summary`: three to six sentences up front that state the central thesis and how
the episode gets there. `sections`: as many as the content needs (usually four to ten), each
with a heading that says something (not "Einleitung"), `start` = the second in the episode where
its content begins (from the [mm:ss] marks), and `body_markdown` in plain paragraphs; lists only
where the speakers list things themselves. Follow the order of the episode unless a different
order makes the argument clearer. Length: as long as needed for the reader to judge the episode,
roughly a tenth of the transcript, at most about 5,000 words.

The transcript was made by speech recognition: fix obvious mishearings, especially of names,
using the metadata, and don't quote garbled passages.

Prompt (user turn): the metadata block, then Transkript: and the text.

The built-in instance (since 2026-11-09)

Whisper does not fit on the Strato server: with about 2 GB of memory free, large-v3-turbo needed more than 2.2 GB and even small broke a 1.2 GB limit, and earlyoom then killed other services. So the built-in instance runs on the user's PC (multi11, GTX 1660 Ti) and diora-prod reaches it over the tailnet: DIORA_WHISPER_URL=http://100.64.0.3:8766/v1 in ~/docker-compose/diora-prod/.env. It is deploy/whisper/server.py (a small OpenAI-compatible server around faster-whisper, large-v3-turbo, int8_float16 on CUDA; one request at a time; the model is loaded on demand and freed after 10 idle minutes), started at logon from F:\diora-whisper (start.cmd via start-hidden.vbs, shortcut in the user's Startup folder; log F:\diora-whisper\server.log), listening only on the tailnet address. One hour of speech takes about five minutes.

When the PC is off, a job does not fail: the server checks the instance before downloading, and a connection error (there or while pieces are out) puts the job back into the queue with error = "Whisper-Instanz nicht erreichbar, neuer Versuch um HH:MM", tried again every ten minutes (WAIT_RETRY) for up to two days after the button was pressed (WAIT_MAX). Only the built-in instance waits like this; a custom instance's connection errors fail the job as before.