mrwnslz/diora-web
1
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

Smarte Transkription: Vertrag und Prompt (docs/transcription.md), Abschnitt Transkripte in docs/podcasts.md, Podcast-Artikel in docs/ai.md

Browse source 
Transkript und Artikel nur auf Knopfdruck; der Server transkribiert mit Whisper (eingebaut oder eigene Instanz), das Gerät schreibt den Artikel mit dem eigenen Schlüssel, sofort oder beim nächsten Öffnen; Benachrichtigung über /feeds/notifications/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mrwnslz 2026-11-09 09:16:16 +01:00
parent 1e36d8600b
commit f242605a97
3 changed files with 228 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/ai.md
View file

@ -29,6 +29,8 @@ kept by the server as a blob it can't read:
the key can't be read: the settings ask for it again. Not on diora Reader (e-readers).
- Server-side jobs (in the background, without an app open) were left out on purpose: they
would need the key on the server. The project Radar has its own server key (docs/projects.md).
Podcast articles wait on the server as a wish until a device with the key is open
(docs/transcription.md).
## Building blocks (`static/js/ai.js`, an ES module)
@ -53,5 +55,6 @@ kept by the server as a blob it can't read:
| Bücher, Zeitschriften | ✦ in the library's head (`static/js/app/55-library-tools.js`) | per book: id, title, author, folders, series, magazine mark; the existing folders | folders to add (existing first; new ones only for ≥ 3 books), series with volume, cleaned title/author where broken, magazine mark |
| Podcasts | ✦ in Abos (diora-android `web-native/static/native-podcast.js`, copied to `static/podcasts/`) | per subscription: title, author, folders, level, heard in 12 months, publishing interval, three newest episode titles | folders to add (≤ 2), level Stamm / Strom / Auf Abruf (only with a clear reason) |
| Nachrichten, Blogs | ✦ in the feeds' bar (`static/js/feeds.js`; the apps' section bar presses it) | per source: title, address, kind, Ressort or folders, recent titles shown | Ressort for news, folders for blogs, news ↔ blog when clearly wrong |
| Podcast-Artikel | ✦ Artikel in an episode (`static/js/podcast-article.js`, docs/transcription.md) | the episode's transcript (feed or Whisper), title, author, shownotes, chapters | an article from the speakers' standpoint with summary and sections, saved to the episode and to Blogs (no review sheet: it is new text, removable) |
Applied through `_updateBookMeta` (the encrypted meta, as when editing by hand).

9
docs/podcasts.md
View file

@ -182,3 +182,12 @@ Listening without committing: one button plays the Strom like a station.
- Web host (`static/js/podcasts-web.js`, `DioraPodcastHost`): `podcastLaterAdd(json episodes)`,
`podcastLaterRemove(json ids)`, `podcastListsSwitch(on)`, `podcastStreamSeen()` (keeps the
previous time as `state.stream_seen_before` for the marks).
## Transcripts
An episode's Transkript tab shows one of two kinds. **From the feed**: `<podcast:transcript>`
(SRT, VTT, JSON, Podlove), fetched and parsed on first use and cached in `EpisodeTranscript`
(`podcasts/transcripts.py`, `GET /podcasts/episodes/<id>/transcript/`). **Made by Whisper**
(since 2026-11): only when the user asks, kept on `TranscriptionJob`; the feed's wins when both
exist. From a transcript the device can write an article that is pinned to the episode and shows
in Blogs. All of that is in docs/transcription.md.

216
docs/transcription.md Normal file
View file

@ -0,0 +1,216 @@
# Smarte Transkription: Transkript und Artikel zu einer Folge
Decided with the user on 2026-11-09. Plan and order of work:
diora-android `plans/smarte-transkription.md`.
## What it is for
Sometimes the shownotes don't tell whether an episode still has something to teach. Then the
user presses a button and, a few hours later at most, gets a readable article written from the
speakers' own standpoint. It is meant to let them judge the episode for themselves, so it keeps
arguments, reasons and evidence and does not just summarise them. The article sits on the
episode and in Blogs.
Nothing happens on its own. The server never transcribes ahead, by cron or on subscribing; the
only way to start a transcription is the `POST` below, sent when the user presses a button.
## Who does what
The **server** transcribes. It already downloads and decodes episodes for skipping silence
(`podcasts/silence.py`), and browsers and the WebView can't fetch podcast audio (CORS). It cuts
the audio at pauses and sends the pieces in parallel to a Whisper instance with an
OpenAI-compatible API (`POST /v1/audio/transcriptions`): the built-in one by default (a container
beside diora-web, `DIORA_WHISPER_URL`), or any instance the user enters. YouTube episodes are
not transcribed: the server never fetches their sound (docs/youtube.md).
The **device** writes the article with the user's own key (docs/ai.md). The key never reaches the
server. It is one LLM call of a few minutes. If diora is open when the transcript is done, it
happens right away; otherwise the next time diora is opened on any of the user's devices. The
server only remembers that an article was wanted.
Transcript and article are stored in plain text on the server, like all podcast and feed data.
## Flow
1. In the episode the user presses "Transkript erstellen" (the Transkript tab) or
"✦ Artikel erstellen" / "✦ Transkribieren und Artikel erstellen". The second sends
`article: true`. With a transcript already there (from the feed or an earlier run) only the
article is written.
2. The server works through the episode (minutes to hours, no GPU on Strato). The UI shows
"Transkribiere 7/38 …" while it is open.
3. Done: a notice for `GET /feeds/notifications/`. Android's BackgroundSync and the desktop show
"Transkript fertig: <Folge>", with "Artikel wird beim Öffnen geschrieben" when one is wanted.
Tapping it opens diora at the episode.
4. A device with a key (`static/js/podcast-article.js`) picks up the open wishes, reserves one,
writes the article and saves it. It then says "Artikel fertig" (on Android as a system
notification through the bridge) and opens it on a tap.
5. The article is the first tab of the episode ("Artikel") and an item of the favourite blog
"Podcast-Artikel", linking back to the episode.
## API
**Settings.** `GET /podcasts/transcription/settings/` returns
`{engine: "builtin"|"custom", url, model, language, parallel, has_token, builtin_available}`.
`POST` takes the same fields and `token` (write-only; `""` removes it). The token is kept
encrypted with Fernet like `feeds.SiteLogin.secret`. `language` is `""` for automatic or an ISO
code (`de`). `parallel` is 1 to 8, default 3. `builtin_available` is whether `DIORA_WHISPER_URL`
is set. `url` is the instance's base, e.g. `https://whisper.example.org/v1`; the server appends
`/audio/transcriptions`.
`POST /podcasts/transcription/test/` sends one second of silence to the configured instance and
returns `{ok, model, error}`. A custom URL goes through the same SSRF check as chapters
(`podcasts/chapters.py` `_open`): no private, loopback or link-local addresses. Only the built-in
URL from `DIORA_WHISPER_URL` is exempt.
**Job.** `POST /podcasts/episodes/<id>/transcribe/` with `{article: true|false}` starts a job, or
updates `want_article` on a running one. With a transcript already there it returns
`{state: "done"}` straight away (and keeps `want_article`). `GET` on the same address returns
`{state, chunks_done, chunks_total, error, want_article, started_at, finished_at}`, with `state`
one of `none`, `queued`, `downloading`, `transcribing`, `done`, `error`. `DELETE` cancels a
queued or running job. The server runs one job at a time; the others wait as `queued`. A YouTube
episode returns 400 `{error: "youtube"}`, an episode over 800 MB 400 `{error: "too large"}`.
**Transcript.** The result is kept on the job (`TranscriptionJob.data`) in the format of
`EpisodeTranscript.data`: `{segments: [{start, end, speaker, text}], source: {type: "whisper",
model, language}}`. `speaker` is `""`. It is kept apart from `EpisodeTranscript`, which is a cache
of the feed's transcripts and is rewritten when the feed changes them.
`GET /podcasts/episodes/<id>/transcript/` returns the feed's transcript when the feed has a
usable one, else the Whisper one, else 404 as before. `episode_detail` adds
`transcript_origin` (`"feed"`, `"whisper"` or `null`), `has_transcript` is true for both, and it
adds `article_item_id` (or `null`) and `transcription` (the job's state as above, or `null`).
**Article.** `POST /podcasts/episodes/<id>/article/` takes `{title, summary, body_html, model,
kind}` and returns `{item_id}`. The server cleans `body_html` with `feeds/fetch.py clean_html`.
It creates the user's source "Podcast-Artikel" when missing (`kind` blog, `fav` true,
`synthetic` true, `url` `diora://podcast-artikel`) and adds a `feeds.Item` there, or replaces the
episode's earlier one. Item fields: the episode's link (else its audio URL) as `url`, the feed's
author as `author`, the episode's artwork as `image_url`, now as `published` (so it is on top of
the blog stream under "heute"), `words` counted. Saving clears the wish. `DELETE` on the same
address removes the article and its item. `GET /feeds/items/<id>/` and `_item_data` carry
`episode_id` for such items.
Synthetic sources are skipped by the feed refresh (`refresh_newsfeeds`, `refresh_now`) and by the
pruning of old blog items (`feeds/refresh.py`). They can be hidden or have their bell turned off
like any blog; they can't be refreshed, renamed to another URL or turned into news.
**Open wishes.** `GET /podcasts/transcription/pending/` returns
`[{episode_id, title, podcast, duration_seconds}]`: the user's episodes whose transcript is done,
that want an article, have none and aren't reserved. `POST
/podcasts/episodes/<id>/article/claim/` reserves one for 15 minutes for the calling device
(200), or answers 409 when another device holds it, so phone and desktop don't both write it.
`DELETE` on the claim releases it (the device gave up, e.g. no key or an error the user saw).
**Notice.** `GET /feeds/notifications/` additionally takes `t_after` and returns
`transcripts: [{job_id, episode_id, title, podcast, want_article}]` (jobs done after that id) and
`t_last`. Without `t_after` (a device's first time) the list is empty and only `t_last` comes,
like `after`/`last`. Devices that don't know the fields ignore them.
## Transcribing (`podcasts/transcribe.py`)
Download with the same limits as `silence.py`, decode with imageio-ffmpeg to 16 kHz mono. Cut
into pieces of about eight minutes; each cut moves to the longest pause within ±30 seconds
(`silence.py`'s detection), so no word is cut. Pieces go out as FLAC to the built-in instance
and as Opus 32 kbit/s to custom ones, to keep uploads small.
Each piece is sent with `model`, `language` (when set), `response_format=verbose_json` and a
`prompt` of podcast title, episode title and author names, so Whisper spells names right. Pieces
run in a thread pool of `parallel` workers; each is retried up to three times with a growing
pause; progress is written after each. Segment times are shifted by the piece's start; the
segments are joined and smoothed with `transcripts.merge`. A job runs in a daemon thread, like
`silence.ensure()`; one per server. A job found `downloading` or `transcribing` after a restart
is started again.
## The article (`static/js/podcast-article.js`)
Input: the transcript as plain text, one line per segment group of about 30 seconds,
`[mm:ss] Sprecher: text` (or `[mm:ss] text` without speakers), plus the episode's metadata:
podcast title, author, episode title, date, duration, the shownotes as plain text and the
chapters. Over about 120,000 tokens (more than about nine hours) the device stops with a clear
message for now. A five-hour episode is about 70,000 tokens.
`DioraAI.ask({system, prompt, schema, effort: "medium", maxTokens: 32000, onProgress, signal})`.
`onProgress(words)` reports the words written so far while streaming ("Schreibe Artikel … 2.300
Wörter"); `signal` aborts. Both are optional and new in ai.js; the existing callers don't change.
Schema:
```json
{
"type": "object",
"additionalProperties": false,
"required": ["title", "summary", "kind", "speakers", "sections"],
"properties": {
"title": {"type": "string"},
"summary": {"type": "string"},
"kind": {"type": "string", "enum": ["Interview", "Gespräch", "Monolog", "Diskussion"]},
"speakers": {"type": "array", "items": {"type": "string"}},
"sections": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["heading", "start", "body_markdown"],
"properties": {
"heading": {"type": "string"},
"start": {"type": "integer"},
"body_markdown": {"type": "string"}
}
}
}
}
}
```
The HTML: the summary as a lead paragraph, then per section an `<h2>`, below it a link
"▶ ab 12:34" (`<a href="#t=754">`; `clean_html` keeps fragment links, and feeds.js plays the
item's `episode_id` from there when such a link is tapped) and the body. Markdown is limited to paragraphs, emphasis, lists and block
quotes, converted on the device; the server cleans the result anyway.
System prompt (the user's words decide the perspective; keep its substance when changing it):
```text
You turn the transcript of a podcast episode into an article that reads as if the people
speaking had written it themselves instead of recording the episode.
The reader wants to decide whether this episode still has something to teach them. They do not
want a summary; they want to understand what is argued here well enough to judge it for
themselves. Write for that reader.
Perspective. Write from the speakers' standpoint and state their positions directly, as their
own claims. Never distance yourself from them: no "im Podcast wird die These aufgestellt", "der
Sprecher meint", "in dieser Folge geht es um", "die Gäste diskutieren", "laut X". If the episode
is one person speaking, write as that person argues, in the third person or impersonal voice,
but without reporting verbs. Do not add your own opinion, criticism or fact checks, and do not
soften what is said.
Conversations. Interviews, conversations and discussions are condensed into their core theses;
the form of question and answer disappears. Where people hold different positions, keep them
apart and name who holds which ("Zuckermann hält dagegen, dass …"), but do not turn it into
minutes. Questions from a host matter only where they shape the argument. When the transcript
has no speaker names, infer them from the context (introductions, the shownotes, how people
address each other); if you can't, do not invent names.
Substance over summary. Give the arguments with their reasons, examples and evidence, not just
the conclusions. Keep figures, dates, quotations that carry an argument (short, in quotation
marks, attributed), and every work mentioned (books, articles, studies, films) with author and
title. Make clear what is new, distinctive or surprising in the episode and where it restates
what is commonly said, by how you weight and order it, without grading it.
Leave out: advertising and sponsors, appeals for donations, subscriptions or ratings, notes on
the podcast itself (schedules, technology, community), greetings, small talk and digressions that
lead nowhere.
Form. Write in the language of the episode. `title`: a title for the article, not the episode
title repeated. `summary`: three to six sentences up front that state the central thesis and how
the episode gets there. `sections`: as many as the content needs (usually four to ten), each
with a heading that says something (not "Einleitung"), `start` = the second in the episode where
its content begins (from the [mm:ss] marks), and `body_markdown` in plain paragraphs; lists only
where the speakers list things themselves. Follow the order of the episode unless a different
order makes the argument clearer. Length: as long as needed for the reader to judge the episode,
roughly a tenth of the transcript, at most about 5,000 words.
The transcript was made by speech recognition: fix obvious mishearings, especially of names,
using the metadata, and don't quote garbled passages.
```
Prompt (user turn): the metadata block, then `Transkript:` and the text.