diora-web/docs/transcription.md

# Smarte Transkription: Transkript und Artikel zu einer Folge

Decided with the user on 2026-11-09. Plan and order of work:
diora-android `plans/smarte-transkription.md`.

## What it is for

Sometimes the shownotes don't tell whether an episode still has something to teach. Then the
user presses a button and, a few hours later at most, gets a readable article written from the
speakers' own standpoint. It is meant to let them judge the episode for themselves, so it keeps
arguments, reasons and evidence and does not just summarise them. The article sits on the
episode and in Blogs.

Nothing happens on its own. The server never transcribes ahead, by cron or on subscribing; the
only way to start a transcription is the `POST` below, sent when the user presses a button.

## Who does what

The **server** transcribes. It already downloads and decodes episodes for skipping silence
(`podcasts/silence.py`), and browsers and the WebView can't fetch podcast audio (CORS). It cuts
the audio at pauses and sends the pieces in parallel to a Whisper instance with an
OpenAI-compatible API (`POST /v1/audio/transcriptions`): the built-in one by default (`DIORA_WHISPER_URL`),
or any instance the user enters. YouTube episodes are
not transcribed: the server never fetches their sound (docs/youtube.md).

The **device** writes the article with the user's own key (docs/ai.md). The key never reaches the
server. It is one LLM call of a few minutes. If diora is open when the transcript is done, it
happens right away; otherwise the next time diora is opened on any of the user's devices. The
server only remembers that an article was wanted.

Transcript and article are stored in plain text on the server, like all podcast and feed data.

## Flow

1. In the episode the user presses "Transkript erstellen" (the Transkript tab) or
   "✦ Artikel erstellen" / "✦ Transkribieren und Artikel erstellen". The second sends
   `article: true`. With a transcript already there (from the feed or an earlier run) only the
   article is written.
2. The server works through the episode (minutes to hours, no GPU on Strato). The UI shows
   "Transkribiere 7/38 …" while it is open.
3. Done: a notice for `GET /feeds/notifications/`. Android's BackgroundSync and the desktop show
   "Transkript fertig: <Folge>", with "Artikel wird beim Öffnen geschrieben" when one is wanted.
   Tapping it opens diora at the episode.
4. A device with a key (`static/js/podcast-article.js`) picks up the open wishes, reserves one,
   writes the article and saves it. It then says "Artikel fertig" (on Android as a system
   notification through the bridge, `DioraNative.notify(title, text, target)` with target
   `episode:<id>` or `item:<id>`; elsewhere a toast) and opens it on a tap.
5. The article is the first tab of the episode ("Artikel") and an item of the favourite blog
   "Podcast-Artikel", linking back to the episode.

## API

**Settings.** `GET /podcasts/transcription/settings/` returns
`{engine: "builtin"|"custom", url, model, language, parallel, has_token, builtin_available}`.
`POST` takes the same fields and `token` (write-only; `""` removes it). The token is kept
encrypted with Fernet like `feeds.SiteLogin.secret`. `language` is `""` for automatic or an ISO
code (`de`). `parallel` is 1 to 8, default 3. `builtin_available` is whether `DIORA_WHISPER_URL`
is set. `url` is the instance's base, e.g. `https://whisper.example.org/v1`; the server appends
`/audio/transcriptions`.

An empty `model` is sent as `whisper-1`. The built-in URL is read into `settings.WHISPER_URL` from
`DIORA_WHISPER_URL`. A custom engine needs a `url`; the token is only sent to custom instances.

`POST /podcasts/transcription/test/` sends one second of silence to the configured instance and
returns `{ok, model, error}`. A custom URL goes through the same SSRF check as chapters
(`podcasts/chapters.py` `_open`): no private, loopback or link-local addresses. Only the built-in
URL from `DIORA_WHISPER_URL` is exempt.

**Job.** `POST /podcasts/episodes/<id>/transcribe/` with `{article: true|false}` starts a job, or
updates `want_article` on a running one. With a transcript already there it returns
`{state: "done"}` straight away (and keeps `want_article`). `GET` on the same address returns
`{state, chunks_done, chunks_total, error, want_article, started_at, finished_at}`, with `state`
one of `none`, `queued`, `downloading`, `transcribing`, `done`, `error`. `DELETE` cancels a
queued or running job (its row is deleted, so `GET` then says `none`; a finished one stays). The server runs one job at a time; the others wait as `queued`. A YouTube
episode returns 400 `{error: "youtube"}`, an episode over 800 MB 400 `{error: "too large"}`. The size is asked with a `HEAD`
request (`Content-Length`; if that fails the job starts anyway) and checked again while
downloading, where it ends the job with the error `too large`. `article: true` is only ever
raised, never lowered, by a repeated `POST`; a `POST` after an `error` starts a new job (at the
back of the queue). Jobs run in id order, which is why the notice below can use ids as its cursor.
A feed transcript already there gives a job that is `done` with no `data`: it only holds the wish
for an article and causes no `transcripts` notice.

**Transcript.** The result is kept on the job (`TranscriptionJob.data`) in the format of
`EpisodeTranscript.data`: `{segments: [{start, end, speaker, text}], source: {type: "whisper",
model, language}}`. `speaker` is `""`. It is kept apart from `EpisodeTranscript`, which is a cache
of the feed's transcripts and is rewritten when the feed changes them.
`GET /podcasts/episodes/<id>/transcript/` returns the feed's transcript when the feed has a
usable one, else the Whisper one, else 404 as before. `episode_detail` adds
`transcript_origin` (`"feed"`, `"whisper"` or `null`), `has_transcript` is true for both, and it
adds `article_item_id` (or `null`) and `transcription` (the job's state as above, or `null`).

**Article.** `POST /podcasts/episodes/<id>/article/` takes `{title, summary, body_html, model,
kind}` and returns `{item_id}`. The server cleans `body_html` with `feeds/fetch.py clean_html`.
It creates the user's source "Podcast-Artikel" when missing (`kind` blog, `fav` true,
`synthetic` true, `url` `diora://podcast-artikel`) and adds a `feeds.Item` there, or replaces the
episode's earlier one. Item fields: the episode's link (else its audio URL) as `url`, the feed's
author as `author`, the episode's artwork as `image_url`, now as `published` (so it is on top of
the blog stream under "heute"), `words` counted. Saving clears the wish. `DELETE` on the same
address removes the article and its item. `GET /feeds/items/<id>/` and `_item_data` carry
`episode_id` for such items.
`body_html` may contain `href="#t=<seconds>"` links; `clean_html` keeps exactly this fragment form
(`podcasts/shownotes.py` `TIME_LINK`) and no other `#` links. Validation: `title` 1 to 300
characters, `body_html` non-empty after cleaning and at most 1,000,000 characters, `summary` at most
2000, `model` at most 100, `kind` at most 20 (400 otherwise). The item has `content_full` true (no
page is ever fetched) and, as the reader's furniture filter would strip repeated "▶ ab" blocks,
`GET /feeds/items/<id>/` skips that filter for synthetic sources. The endpoint is
`podcasts/article.py`.

Synthetic sources are skipped by the feed refresh (`refresh_newsfeeds`, `refresh_now`) and by the
pruning of old blog items (`feeds/refresh.py`). They can be hidden or have their bell turned off
like any blog; they can't be refreshed, renamed to another URL or turned into news. `refresh.refresh()` itself drops them, the OPML export leaves them out, and `_source_data` carries `synthetic`.

**Open wishes.** `GET /podcasts/transcription/pending/` returns
`[{episode_id, title, podcast, duration_seconds}]`: the user's episodes whose transcript is done,
that want an article, have none and aren't reserved. `POST
/podcasts/episodes/<id>/article/claim/` reserves one for 15 minutes for the calling device
(200), or answers 409 when another device holds it, so phone and desktop don't both write it.
`DELETE` on the claim releases it (the device gave up, e.g. no key or an error the user saw).
The device names itself with `device` in the JSON body (or, if that is missing, the
`X-Diora-Device` header): a short stable name such as `android`, `desktop` or `browser`, at most 60
characters; a claim without one is 400. The same device claiming again renews its 15 minutes. A
claim on an episode with no open wish (no job done, no article wanted, or an article already
there) is 404. `DELETE` by a device that does not hold the claim is 409.

**Notice.** `GET /feeds/notifications/` additionally takes `t_after` and returns
`transcripts: [{job_id, episode_id, title, podcast, want_article}]` (jobs done after that id) and
`t_last`. Without `t_after` (a device's first time) the list is empty and only `t_last` comes,
like `after`/`last`. Devices that don't know the fields ignore them.

## Transcribing (`podcasts/transcribe.py`)

Download with the same limits as `silence.py`, decode with imageio-ffmpeg to 16 kHz mono. Cut
into pieces of about eight minutes; each cut moves to the longest pause within ±30 seconds
(`silence.py`'s detection), so no word is cut. Pieces go out as FLAC to the built-in instance
and as Opus 32 kbit/s to custom ones, to keep uploads small.

Each piece is sent with `model`, `language` (when set), `response_format=verbose_json` and a
`prompt` of podcast title, episode title and author names, so Whisper spells names right. Pieces
run in a thread pool of `parallel` workers; each is retried up to three times with a growing
pause; progress is written after each. Segment times are shifted by the piece's start; the
segments are joined and smoothed with `transcripts.merge`. A job runs in a daemon thread, like
`silence.ensure()`; one per server. A job found `downloading` or `transcribing` after a restart
is started again. This happens when the first
request (`GET`/`POST` on `/transcribe/` or `/transcription/pending/`) after a start reaches the server.
Cancelling deletes the job's row; the worker notices at the latest after five seconds, stops
sending pieces and removes its temporary files (they live in a temporary directory that is always
removed). Only the worker's main thread writes to the database. The settings used are those of the
episode's owner, read when the job starts. A refusal by the instance with a 4xx status other than
408 and 429 is not retried and ends the job with `Whisper: HTTP <code>`.

## The article (`static/js/podcast-article.js`)

Input: the transcript as plain text, one line per segment group of about 30 seconds,
`[mm:ss] Sprecher: text` (or `[mm:ss] text` without speakers), plus the episode's metadata:
podcast title, author, episode title, date, duration, the shownotes as plain text and the
chapters. Over about 120,000 tokens (more than about nine hours) the device stops with a clear
message for now. A five-hour episode is about 70,000 tokens.

`DioraAI.ask({system, prompt, schema, effort: "medium", maxTokens: 32000, onProgress, signal})`.
`onProgress(words)` reports the words written so far while streaming ("Schreibe Artikel … 2.300
Wörter"); `signal` aborts. Both are optional and new in ai.js; the existing callers don't change.

Schema:

```json
{
  "type": "object",
  "additionalProperties": false,
  "required": ["title", "summary", "kind", "speakers", "sections"],
  "properties": {
    "title": {"type": "string"},
    "summary": {"type": "string"},
    "kind": {"type": "string", "enum": ["Interview", "Gespräch", "Monolog", "Diskussion"]},
    "speakers": {"type": "array", "items": {"type": "string"}},
    "sections": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["heading", "start", "body_markdown"],
        "properties": {
          "heading": {"type": "string"},
          "start": {"type": "integer"},
          "body_markdown": {"type": "string"}
        }
      }
    }
  }
}
```

The HTML: the summary as a lead paragraph, then per section an `<h2>`, below it a link
"▶ ab 12:34" (`<a href="#t=754">`; `clean_html` keeps fragment links, and feeds.js plays the
item's `episode_id` from there when such a link is tapped) and the body. Markdown is limited to paragraphs, emphasis, lists and block
quotes, converted on the device; the server cleans the result anyway.

System prompt (the user's words decide the perspective; keep its substance when changing it):

```text
You turn the transcript of a podcast episode into an article that reads as if the people
speaking had written it themselves instead of recording the episode.

The reader wants to decide whether this episode still has something to teach them. They do not
want a summary; they want to understand what is argued here well enough to judge it for
themselves. Write for that reader.

Perspective. Write from the speakers' standpoint and state their positions directly, as their
own claims. Never distance yourself from them: no "im Podcast wird die These aufgestellt", "der
Sprecher meint", "in dieser Folge geht es um", "die Gäste diskutieren", "laut X". If the episode
is one person speaking, write as that person argues, in the third person or impersonal voice,
but without reporting verbs. Do not add your own opinion, criticism or fact checks, and do not
soften what is said.

Conversations. Interviews, conversations and discussions are condensed into their core theses;
the form of question and answer disappears. Where people hold different positions, keep them
apart and name who holds which ("Zuckermann hält dagegen, dass …"), but do not turn it into
minutes. Questions from a host matter only where they shape the argument. When the transcript
has no speaker names, infer them from the context (introductions, the shownotes, how people
address each other); if you can't, do not invent names.

Substance over summary. Give the arguments with their reasons, examples and evidence, not just
the conclusions. Keep figures, dates, quotations that carry an argument (short, in quotation
marks, attributed), and every work mentioned (books, articles, studies, films) with author and
title. Make clear what is new, distinctive or surprising in the episode and where it restates
what is commonly said, by how you weight and order it, without grading it.

Leave out: advertising and sponsors, appeals for donations, subscriptions or ratings, notes on
the podcast itself (schedules, technology, community), greetings, small talk and digressions that
lead nowhere.

Form. Write in the language of the episode. `title`: a title for the article, not the episode
title repeated. `summary`: three to six sentences up front that state the central thesis and how
the episode gets there. `sections`: as many as the content needs (usually four to ten), each
with a heading that says something (not "Einleitung"), `start` = the second in the episode where
its content begins (from the [mm:ss] marks), and `body_markdown` in plain paragraphs; lists only
where the speakers list things themselves. Follow the order of the episode unless a different
order makes the argument clearer. Length: as long as needed for the reader to judge the episode,
roughly a tenth of the transcript, at most about 5,000 words.

The transcript was made by speech recognition: fix obvious mishearings, especially of names,
using the metadata, and don't quote garbled passages.
```

Prompt (user turn): the metadata block, then `Transkript:` and the text.

## The built-in instance (since 2026-11-09)

Whisper does not fit on the Strato server: with about 2 GB of memory free, large-v3-turbo needed
more than 2.2 GB and even `small` broke a 1.2 GB limit, and earlyoom then killed other services.
So the built-in instance runs on the user's PC (multi11, GTX 1660 Ti) and diora-prod reaches it over
the tailnet: `DIORA_WHISPER_URL=http://100.64.0.3:8766/v1` in `~/docker-compose/diora-prod/.env`.
It is `deploy/whisper/server.py` (a small OpenAI-compatible server around faster-whisper,
large-v3-turbo, int8_float16 on CUDA; one request at a time; the model is loaded on demand and freed
after 10 idle minutes), started at logon from `F:\diora-whisper` (start.cmd via start-hidden.vbs,
shortcut in the user's Startup folder; log `F:\diora-whisper\server.log`), listening only on the
tailnet address. One hour of speech takes about five minutes.

When the PC is off, a job does not fail: the server checks the instance before downloading, and a
connection error (there or while pieces are out) puts the job back into the queue with
`error = "Whisper-Instanz nicht erreichbar, neuer Versuch um HH:MM"`, tried again every ten
minutes (`WAIT_RETRY`) for up to two days after the button was pressed (`WAIT_MAX`). Only the
built-in instance waits like this; a custom instance's connection errors fail the job as before.