mrwnslz/diora-web
1
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

TUI: Markierungen, Notizen und Lesezeichen (anzeigen, anlegen, löschen, abgleichen)

Browse source 
Dieselben verschlüsselten Listen wie der Web-Reader (annotations.py). Die Anker des Webs
(Block-Index plus Zeichenposition im Rohtext) werden auf den angezeigten Text umgerechnet
(marks.py, blocks.collapse_map), alte XPath-Anker über das Zitat gefunden. Im Reader: v markiert
(Cursor je Wort, Anfang/Ende, Farbe oder Notiz), n legt eine Notiz an, b ein Lesezeichen,
a öffnet die Liste (springen, Notiz ändern, löschen).

Der Server ersetzt die Liste als Ganzes; die TUI legt ihre Änderungen vor dem Schreiben
dreiwege je Eintrag-ID auf die aktuelle Liste des Servers, offline bleiben sie vorgemerkt.
Geprüft: die echten resolveAnchorRange des Webs lösen 81 TUI-Anker exakt auf
(tools/parity/web_anchors.cjs), 34 von 34 echten Web-Markierungen werden angezeigt, der
Server-Smoke schreibt und mischt zwei Geräte. 75 Tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
mrwnslz 2026-10-26 16:42:32 +01:00
parent 695037fb9c
commit e246a32212
16 changed files with 1549 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

33
tui/README.md
View file

@ -33,6 +33,8 @@ Tastenkürzel:
- `↑`/`k`, `↓`/`j` — zeilenweise scrollen
- `n` — nächstes Kapitel, `p` — vorheriges Kapitel
- `w` — zur weitesten Stelle des Buchs springen (auf Wunsch; beim Öffnen weist ein Hinweis darauf hin)
- `v` — Text markieren (siehe unten), `b` — Lesezeichen setzen, `a` — Liste der Lesezeichen,
Markierungen und Notizen
- `f` — Fußnote in der Nähe der aktuellen Position anzeigen (Peek-Overlay, `Escape`/`f`/`q`
zum Schließen); Erkennung folgt derselben Heuristik wie `app.js`s
`_looksLikeFootnoteLink` (Link in/um `<sup>`, Klassenname mit note/footnote/fn, oder
@ -74,6 +76,34 @@ Wie die Bibliothek im Web (`docs/library.md`), nur im Terminal:
Nicht in der TUI: Serien-Stapel, Ausgaben (mehrere Fassungen eines Werks stehen einzeln),
Cover, Ordner anlegen/zuweisen, Eingang aus der Cloud.
## Markierungen, Notizen, Lesezeichen
Dieselben Daten wie im Web-Reader (verschlüsselt mit dem Buchschlüssel, je Buch und Art eine Liste
`/books/<id>/highlights/` und `/bookmarks/`; Format in `annotations.py`). Was im Web, in den Apps
oder hier gemacht wird, erscheint überall.
- **Anzeige:** Markierungen stehen im Text mit der Farbe (Gelb, Grün, Blau, Rot); eine freie Notiz
ist unterstrichen. Die Anker des Web-Readers (Block-Nummer plus Zeichenposition im Rohtext des
Absatzes) werden auf den angezeigten Text umgerechnet; alte Anker ohne Blocknummer findet die TUI
wie das Web über das Zitat. `a` öffnet die Liste (Enter springt dorthin, `e` ändert die Notiz,
`d` löscht, Escape schließt).
- **Markieren (`v`):** Der Cursor steht auf einem Wort; `h`/`l` wechseln das Wort, `j`/`k` den
Absatz. Enter setzt den Anfang, danach `h`/`l`/`j`/`k` zum Ende und Enter: Farbe wählen, oder
„Gelb mit Notiz“. `n` legt statt dessen eine Notiz auf das Wort bzw. die Auswahl. Escape bricht
ab. Stellen, die sich nicht genau zuordnen lassen (ein Fußnotenmarker mit Leerraum im Absatz),
lehnt die TUI ab, statt eine falsche Position zu schreiben.
- **Lesezeichen (`b`):** wie im Web das aktuelle Kapitel (Name der Kapiteldatei, neueste zuerst).
- **Abgleich:** Der Server ersetzt die Liste als Ganzes und führt nichts zusammen; die Apps
schreiben deshalb ihre ganze Liste. Die TUI merkt sich, was sie zuletzt vom Server gesehen hat, und
legt vor dem Schreiben ihre Änderungen (neu, gelöscht, geändert, je Eintrag-ID) auf die aktuelle
Liste des Servers: was ein anderes Gerät inzwischen markiert hat, geht nicht verloren. Offline
bleiben Änderungen vorgemerkt und gehen beim nächsten Abgleich, beim Verlassen des Buchs oder
beim Beenden hinaus. Beim Öffnen eines Buchs werden seine Listen geholt.
- **Prüfung gegen das Web:** `tools/parity/web_anchors.cjs` lässt den echten `resolveAnchorRange`
des Web-Readers die in der TUI erzeugten Anker auflösen (an zwei echten Büchern: alle 81 Anker
an der gemeinten Stelle); umgekehrt lassen sich 34 von 34 echten Web-Markierungen anzeigen (eine alte
XPath-Notiz über ihr Zitat). Nicht in der TUI: PDF-Markierungen.
## Anmelden und synchronisieren
```bash
@ -152,7 +182,8 @@ fügt Leerraum ein).
## Grenzen der aktuellen Version
- Nur EPUB, kein PDF (der Abgleich überspringt PDFs und meldet sie).
- Nur EPUB, kein PDF (der Abgleich überspringt PDFs und meldet sie); keine PDF-Markierungen.
- Notizen sind einzeilig; mehrzeilige Notizen aus dem Web werden angezeigt und lassen sich ersetzen.
- Text wird als Fließtext ohne Bild-/Layout-Rendering dargestellt.
- `innerFraction` (die Position *innerhalb* eines Blocks) ist eine Terminal-Näherung
(zeilenbasiert statt pixelbasiert wie im Browser) — für die Fortschritts-Sortierung

179
tui/diora_tui/annotation_screens.py Normal file
View file

@ -0,0 +1,179 @@
"""The reader's note editor and the list of a book's bookmarks, highlights and notes."""
from __future__ import annotations
from dataclasses import dataclass
from rich.text import Text
from textual.binding import Binding
from textual.containers import Container
from textual.screen import ModalScreen
from textual.widgets import Input, Label, ListItem, ListView, Static
from . import annotations, blocks
class NoteScreen(ModalScreen[str]):
"""Type a note; Enter keeps it, Escape leaves it as it was (dismisses with None)."""
BINDINGS = [Binding("escape", "cancel", "Abbrechen")]
def __init__(self, title: str, text: str = "") -> None:
super().__init__()
self._title = title
self._text = text
def compose(self):
with Container(id="note-body"):
yield Label(self._title, id="note-title")
yield Input(value=self._text, placeholder="Notiz …", id="note-input")
yield Label("Enter: speichern Escape: abbrechen", id="note-hint")
def on_mount(self) -> None:
self.query_one(Input).focus()
def on_input_submitted(self, event: Input.Submitted) -> None:
self.dismiss(event.value.strip())
def action_cancel(self) -> None:
self.dismiss(None)
@dataclass
class Entry:
kind: str # 'bookmarks' | 'highlights'
item: dict
block: int # where it belongs, for the jump
percent: int
def entries(book: blocks.FlatBook, ann: annotations.BookAnnotations, mark_index=None) -> list[Entry]:
out: list[Entry] = []
for bm in ann.bookmarks:
block = bookmark_block(book, bm)
out.append(Entry("bookmarks", bm, block, round(blocks.fraction_for_anchor(book, block, 0.0) * 100)))
for h in sorted(ann.highlights, key=annotations.sort_key):
a = h.get("anchor") or {}
block = a.get("startBlockIndex") if isinstance(a.get("startBlockIndex"), int) else None
if block is None and mark_index is not None: # an old anchor: where its quote is
found = mark_index._find_quote(a)
block = found[0] if found else None
block = min(max(block or 0, 0), len(book.blocks) - 1)
out.append(Entry("highlights", h, block, round(blocks.fraction_for_anchor(book, block, 0.0) * 100)))
return out
def bookmark_block(book: blocks.FlatBook, bm: dict) -> int:
"""The block a bookmark points at: its chapter's first block; else its fraction of the text."""
src = bm.get("anchor") or ""
if src in book.chapter_srcs:
return book.chapter_start_block[book.chapter_srcs.index(src)]
target = float(bm.get("scrollFraction") or 0) * book.total_chars
return next((i for i, b in enumerate(book.blocks) if b.char_start + b.char_len >= target), 0)
class AnnotationsScreen(ModalScreen[int]):
"""Lesezeichen, then Markierungen and Notizen in the order of the book. Enter jumps (dismisses
with the block), d deletes, e edits a note; Escape closes."""
BINDINGS = [
Binding("escape,q,a", "close", "Schließen"),
Binding("d", "delete", "Löschen"),
Binding("e", "edit", "Notiz"),
]
def __init__(self, book: blocks.FlatBook, store: annotations.AnnotationStore, key: str, on_change, mark_index=None) -> None:
super().__init__()
self.book = book
self.store = store
self.key = key
self.mark_index = mark_index
self.on_change = on_change # called after a delete or an edit: marks and the queue follow
self.rows: list[Entry] = []
def compose(self):
with Container(id="ann-body"):
yield Label("Lesezeichen, Markierungen und Notizen", id="ann-title")
yield ListView(id="ann-list")
yield Static("", id="ann-detail")
yield Label("Enter: hinspringen d: löschen e: Notiz bearbeiten Escape: schließen", id="ann-hint")
def on_mount(self) -> None:
self._fill()
def _fill(self, keep: int = 0) -> None:
self.rows = entries(self.book, self.store.get(self.key), self.mark_index)
lv = self.query_one(ListView)
lv.clear()
if not self.rows:
lv.append(ListItem(Label("Noch nichts. b: Lesezeichen, v: Text markieren, im Lesen.")))
for r in self.rows:
lv.append(ListItem(Label(Text(self._label(r)))))
if self.rows:
lv.index = min(keep, len(self.rows) - 1)
lv.focus()
self._detail()
@staticmethod
def _label(r: Entry) -> str:
if r.kind == "bookmarks":
return f"★ {r.item.get('label') or 'Lesezeichen'} {r.percent} %"
h = r.item
quote = " ".join(((h.get("anchor") or {}).get("quote") or "").split())
quote = quote if len(quote) <= 60 else quote[:59] + "…"
if h.get("type") == "note":
return f"✎ {h.get('note') or ''} {r.percent} %"
mark = {"yellow": "▌", "green": "▌", "blue": "▌", "red": "▌"}.get(h.get("color") or "yellow", "▌")
note = f" ✎ {h['note']}" if h.get("note") else ""
return f"{mark} „{quote}“{note} {r.percent} %"
def _detail(self) -> None:
lv = self.query_one(ListView)
detail = self.query_one("#ann-detail", Static)
if not self.rows or lv.index is None or lv.index >= len(self.rows):
detail.update("")
return
h = self.rows[lv.index].item
text = ""
if self.rows[lv.index].kind == "highlights":
text = " ".join(((h.get("anchor") or {}).get("quote") or "").split())
if h.get("note"):
text += "\n✎ " + h["note"]
detail.update(Text(text))
def on_list_view_highlighted(self, event: ListView.Highlighted) -> None:
self._detail()
def on_list_view_selected(self, event: ListView.Selected) -> None:
lv = self.query_one(ListView)
if self.rows and lv.index is not None and lv.index < len(self.rows):
self.dismiss(self.rows[lv.index].block)
def _current(self) -> Entry | None:
lv = self.query_one(ListView)
return self.rows[lv.index] if self.rows and lv.index is not None and lv.index < len(self.rows) else None
def action_close(self) -> None:
self.dismiss(None)
def action_delete(self) -> None:
r = self._current()
if r is None:
return
idx = self.query_one(ListView).index or 0
self.store.delete(self.key, r.kind, r.item["id"])
self.on_change()
self._fill(idx)
def action_edit(self) -> None:
r = self._current()
if r is None or r.kind != "highlights":
return
idx = self.query_one(ListView).index or 0
def edited(text: str | None) -> None:
if text is not None and self.store.set_note(self.key, r.item["id"], text):
self.on_change()
self._fill(idx)
self.app.push_screen(NoteScreen("Notiz", r.item.get("note") or ""), edited)

258
tui/diora_tui/annotations.py Normal file
View file

@ -0,0 +1,258 @@
"""Highlights, notes and bookmarks of a book, as the web reader keeps them.
Format (static/js/app/72-highlights.js, 68-bookmarks.js): per book and kind one JSON list,
encrypted with the book key (AES-GCM, {ct, iv}), replaced as a whole on the server:
highlight {id, type: 'highlight'|'note', anchor, color, note, createdAt}
anchor {type: 'epub', chapterSrc, startBlockIndex, startChar, endBlockIndex, endChar, quote}
bookmark {id, label, anchor (the chapter's file), scrollFraction, createdAt} newest first
The server does no merging, so the apps write their whole list. The TUI keeps what it last
saw (`base`), and before it writes it merges its list with the server's current one by id
(three-way: what it added, deleted or changed since `base`, on top of what the server holds), so
a highlight another device made meanwhile is not wiped out. Its edits are queued (`dirty`) while
offline.
"""
from __future__ import annotations
import functools
import json
import os
import threading
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from . import crypto, paths
COLORS = {"yellow": "#f1c40f", "green": "#2ecc71", "blue": "#3498db", "red": "#e63946"}
COLOR_NAMES = {"yellow": "Gelb", "green": "Grün", "blue": "Blau", "red": "Rot"}
KINDS = ("highlights", "bookmarks")
MAX_BYTES = {"highlights": 700 * 1024, "bookmarks": 100 * 1024} # the server's size guards (raw bytes)
QUOTE_MAX = 200
_DIR = paths.data_dir() / "annotations"
def _now_iso() -> str:
"""Like the browser's toISOString(): 2026-10-26T12:00:00.123Z."""
return datetime.now(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z")
# ---- the items
def new_highlight(anchor: dict, color: str = "yellow", note: str = "") -> dict:
return {"id": str(uuid.uuid4()), "type": "highlight", "anchor": anchor, "color": color, "note": note, "createdAt": _now_iso()}
def new_note(anchor: dict, text: str) -> dict:
"""A freeform note: no mark in the text, its anchor only says where it belongs."""
return {"id": str(uuid.uuid4()), "type": "note", "anchor": anchor, "color": None, "note": text, "createdAt": _now_iso()}
def new_bookmark(label: str, anchor: str, scroll_fraction: float) -> dict:
return {"id": str(uuid.uuid4()), "label": label, "anchor": anchor, "scrollFraction": scroll_fraction, "createdAt": _now_iso()}
def make_anchor(chapter_src: str, start_block: int, start_char: int, end_block: int, end_char: int, quote: str) -> dict:
return {"type": "epub", "chapterSrc": chapter_src, "startBlockIndex": start_block, "startChar": start_char,
"endBlockIndex": end_block, "endChar": end_char, "quote": quote[:QUOTE_MAX]}
def sort_key(h: dict) -> tuple:
a = h.get("anchor") or {}
return (a.get("startBlockIndex", 10**9), a.get("startChar", 0))
# ---- merge
def merge(base: list[dict] | None, ours: list[dict], theirs: list[dict], *, prepend_added: bool = False) -> list[dict]:
"""The server's list (`theirs`) with our changes since `base` laid on it, by id: what we
deleted is gone, what we added is in, what we changed is ours (also against their change).
`base` None: we never saw the server's list, so nothing counts as deleted."""
base_by = {x["id"]: x for x in (base or []) if isinstance(x, dict) and "id" in x}
ours_by = {x["id"]: x for x in ours if isinstance(x, dict) and "id" in x}
deleted = set(base_by) - set(ours_by) if base is not None else set()
added = [x for x in ours if x["id"] not in base_by]
changed = {i for i, x in ours_by.items() if i in base_by and x != base_by[i]}
out, seen = [], set()
for x in theirs:
i = x.get("id")
if i in deleted:
continue
out.append(ours_by[i] if i in changed else x)
seen.add(i)
new = [x for x in added if x["id"] not in seen]
# a change of ours to something they deleted stays: losing an edit is worse than a revived item
revived = [ours_by[i] for i in changed if i not in seen and ours_by[i] not in new]
return (new + revived + out) if prepend_added else (out + revived + new)
# ---- the blobs
def decode(key_b64: str, blob: dict) -> list[dict]:
"""The server's {ct, iv} (ct may be null: none yet) as a list."""
if not blob or not blob.get("ct"):
return []
data = json.loads(crypto.decrypt(key_b64, blob["iv"], blob["ct"]).decode("utf-8"))
return [x for x in data if isinstance(x, dict) and x.get("id")] if isinstance(data, list) else []
def encode(key_b64: str, items: list[dict]) -> dict:
iv, ct = crypto.encrypt(key_b64, json.dumps(items, ensure_ascii=False, separators=(",", ":")).encode("utf-8"))
return {"ct": ct, "iv": iv}
# ---- local state
@dataclass
class BookAnnotations:
highlights: list[dict] = field(default_factory=list)
bookmarks: list[dict] = field(default_factory=list)
base: dict[str, list[dict] | None] = field(default_factory=lambda: {"highlights": None, "bookmarks": None})
dirty: dict[str, bool] = field(default_factory=lambda: {"highlights": False, "bookmarks": False})
def items(self, kind: str) -> list[dict]:
return self.highlights if kind == "highlights" else self.bookmarks
def _locked(fn):
@functools.wraps(fn)
def wrapper(self, *a, **kw):
with self._lock:
return fn(self, *a, **kw)
return wrapper
class AnnotationStore:
"""One JSON file per book key; shared by the reader (edits) and the sync workers."""
def __init__(self, directory: Path | None = None) -> None:
self.dir = directory or _DIR
self._lock = threading.RLock()
self._books: dict[str, BookAnnotations] = {}
def _file(self, key: str) -> Path:
return self.dir / f"{key}.json"
def _load(self, key: str) -> BookAnnotations:
if key in self._books:
return self._books[key]
ann = BookAnnotations()
try:
raw = json.loads(self._file(key).read_text(encoding="utf-8"))
ann.highlights = list(raw.get("highlights") or [])
ann.bookmarks = list(raw.get("bookmarks") or [])
ann.base = {k: raw.get("base", {}).get(k) for k in KINDS}
ann.dirty = {k: bool(raw.get("dirty", {}).get(k)) for k in KINDS}
except (OSError, ValueError, AttributeError):
pass
self._books[key] = ann
return ann
def _save(self, key: str) -> None:
ann = self._books[key]
self.dir.mkdir(parents=True, exist_ok=True)
tmp = self._file(key).with_suffix(".tmp")
tmp.write_text(json.dumps({"highlights": ann.highlights, "bookmarks": ann.bookmarks, "base": ann.base,
"dirty": ann.dirty}, ensure_ascii=False), encoding="utf-8")
os.replace(tmp, self._file(key))
@_locked
def get(self, key: str) -> BookAnnotations:
"""A copy (the lists are copied, not the items: don't change those)."""
a = self._load(key)
return BookAnnotations(list(a.highlights), list(a.bookmarks),
{k: (list(v) if v is not None else None) for k, v in a.base.items()}, dict(a.dirty))
@_locked
def dirty_keys(self) -> list[str]:
keys = set(self._books)
if self.dir.exists():
keys |= {p.stem for p in self.dir.glob("*.json")}
return sorted(k for k in keys if any(self._load(k).dirty.values()))
# ---- edits (they make the kind dirty: queued for the server)
@_locked
def add(self, key: str, kind: str, item: dict) -> None:
ann = self._load(key)
if kind == "bookmarks":
ann.bookmarks.insert(0, item) # newest first, like the web
else:
ann.highlights.append(item)
ann.dirty[kind] = True
self._save(key)
@_locked
def delete(self, key: str, kind: str, item_id: str) -> bool:
ann = self._load(key)
items = ann.items(kind)
kept = [x for x in items if x.get("id") != item_id]
if len(kept) == len(items):
return False
items[:] = kept
ann.dirty[kind] = True
self._save(key)
return True
@_locked
def set_note(self, key: str, item_id: str, text: str) -> bool:
ann = self._load(key)
for i, h in enumerate(ann.highlights):
if h.get("id") == item_id:
if h.get("note") == text:
return False
ann.highlights[i] = {**h, "note": text}
ann.dirty["highlights"] = True
self._save(key)
return True
return False
# ---- the server
@_locked
def take_remote(self, key: str, kind: str, theirs: list[dict]) -> None:
"""The server's current list: ours becomes it, with our queued edits laid on top."""
ann = self._load(key)
if ann.dirty[kind]:
merged = merge(ann.base[kind], ann.items(kind), theirs, prepend_added=kind == "bookmarks")
ann.items(kind)[:] = merged
else:
ann.items(kind)[:] = theirs
ann.base[kind] = list(theirs)
self._save(key)
@_locked
def for_push(self, key: str, kind: str, theirs: list[dict]) -> tuple[list[dict], list[dict]]:
"""(what to write over the server's current list `theirs`, our list as it is now)."""
ann = self._load(key)
ours = list(ann.items(kind))
return merge(ann.base[kind], ours, theirs, prepend_added=kind == "bookmarks"), ours
@_locked
def pushed(self, key: str, kind: str, written: list[dict], snapshot: list[dict]) -> None:
"""The server holds `written`, made from our list `snapshot`. Ours becomes `written`;
what was edited while it was on its way stays queued on top."""
ann = self._load(key)
current = ann.items(kind)
ann.base[kind] = list(written)
if current == snapshot:
current[:] = written
ann.dirty[kind] = False
else:
current[:] = merge(snapshot, current, written, prepend_added=kind == "bookmarks")
ann.dirty[kind] = True
self._save(key)
_default: AnnotationStore | None = None
def default() -> AnnotationStore:
global _default
if _default is None:
_default = AnnotationStore()
return _default

13
tui/diora_tui/api.py
View file

@ -58,6 +58,19 @@ def fetch_book_data(server_url: str, token: str, book_id: int) -> dict:
return resp.json()
def fetch_annotations(server_url: str, token: str, book_id: int, kind: str) -> dict:
"""A book's encrypted highlights or bookmarks: {ct, iv} (both null while there are none)."""
resp = requests.get(f"{server_url}/books/{book_id}/{kind}/", headers=_headers(token), timeout=60)
_check(resp, f"Buch {book_id}")
return resp.json()
def post_annotations(server_url: str, token: str, book_id: int, kind: str, blob: dict) -> None:
"""Replace a book's highlights or bookmarks as a whole (the server doesn't merge)."""
resp = requests.post(f"{server_url}/books/{book_id}/{kind}/", headers=_headers(token), json=blob, timeout=60)
_check(resp, f"Buch {book_id}")
def post_progress(server_url: str, token: str, book_id: int, *, anchor: str, fraction: float, read_at: int,
device: str, force: bool = False) -> dict:
"""Send a queued position. `sent_at` is the moment it leaves: the server corrects this

55
tui/diora_tui/app.tcss
View file

@ -104,3 +104,58 @@ JumpOfferScreen {
margin-top: 1;
color: $text-muted;
}
NoteScreen {
align: center middle;
}
#note-body {
width: 70;
height: auto;
background: $panel;
border: round $primary;
padding: 1 2;
}
#note-title {
text-style: bold;
margin-bottom: 1;
}
#note-hint {
color: $text-muted;
margin-top: 1;
}
AnnotationsScreen {
align: center middle;
}
#ann-body {
width: 90%;
max-width: 110;
height: 80%;
background: $panel;
border: round $primary;
padding: 1 2;
}
#ann-title {
text-style: bold;
}
#ann-list {
height: 1fr;
}
#ann-detail {
height: auto;
max-height: 6;
color: $text-muted;
margin-top: 1;
}
#ann-hint {
color: $text-muted;
margin-top: 1;
}

73
tui/diora_tui/blocks.py
View file

@ -94,6 +94,9 @@ class FlatBook:
chapter_start_block: list[int] # blocks[chapter_start_block[i]] is chapter i's first block
footnote_targets: dict[str, int] # fragment id -> block index of its content
total_chars: int = 1 # all text nodes of the book (never 0: it divides)
# chapter i's file as the web reader names it (data-epub-src: the OPF's folder + the manifest
# href): what highlights and bookmarks store as chapterSrc / anchor
chapter_srcs: list[str] = field(default_factory=list)
def _is_leaf_div(tag: Tag) -> bool:
@ -176,6 +179,19 @@ def _make_block(el: Tag, chapter_index: int, char_start: int) -> Block:
)
def _opf_dir(path: Path) -> str:
"""The folder of the book's OPF inside the zip, with a trailing slash ('' at the root)."""
import re
import zipfile
try:
with zipfile.ZipFile(path) as z:
m = re.search(r'full-path="([^"]+)"', z.read("META-INF/container.xml").decode("utf-8", "replace"))
except (OSError, KeyError, zipfile.BadZipFile):
return ""
return m.group(1).rsplit("/", 1)[0] + "/" if m and "/" in m.group(1) else ""
def load_flat_book(path: Path) -> FlatBook:
from .epub import _book_id # reuse the same path-hash id scheme
@ -189,6 +205,8 @@ def load_flat_book(path: Path) -> FlatBook:
blocks: list[Block] = []
chapter_titles: list[str] = []
chapter_start_block: list[int] = []
chapter_srcs: list[str] = []
opf_dir = _opf_dir(path)
total_chars = 0
# app.js's parseEpub() includes every spine itemref unconditionally — no
@ -208,6 +226,7 @@ def load_flat_book(path: Path) -> FlatBook:
chapter_start_block.append(len(blocks))
first_text = next((b.text for b in chapter_blocks if b.text), None)
chapter_titles.append((first_text or item.get_name())[:60])
chapter_srcs.append(opf_dir + item.file_name)
blocks.extend(chapter_blocks)
footnote_targets: dict[str, int] = {}
@ -227,6 +246,7 @@ def load_flat_book(path: Path) -> FlatBook:
chapter_start_block=chapter_start_block,
footnote_targets=footnote_targets,
total_chars=total_chars or 1,
chapter_srcs=chapter_srcs,
)
@ -279,3 +299,56 @@ def book_pages(path: Path) -> int:
el.decompose()
chars += sum(len(n) for n in _text_nodes(soup.body or soup))
return pages_of_chars(chars)
# ---- raw text of blocks (for highlights: the web stores character offsets into a block's raw text)
def raw_block_texts(path: Path, chapter_index: int) -> list[str]:
"""The raw text (all text nodes, whitespace as in the file) of every block of one spine
document, in block order: what the web's _charOffsetInBlock counts in. `chapter_index` is the
index load_flat_book gave the document."""
raw = epub.read_epub(str(path), options={"ignore_ncx": True})
seen = -1
for idref, _linear in raw.spine:
item = raw.get_item_with_id(idref)
if item is None or item.get_type() != ebooklib.ITEM_DOCUMENT:
continue
seen += 1
if seen != chapter_index:
continue
soup = BeautifulSoup(item.content.decode("utf-8", "replace"), _PARSER)
for name in _STRIP_TAGS:
for el in soup.find_all(name):
el.decompose()
root = soup.body or soup
return ["".join(str(n) for n in _text_nodes(el)) for el in root.descendants
if isinstance(el, Tag) and (el.name in _BLOCK_TAGS or _is_leaf_div(el))]
return []
def collapse_map(raw: str) -> tuple[list[int], list[int], int]:
"""How a block's raw text becomes its displayed text (whitespace runs to one space, the ends
cut). Returns (norm_of_raw, raw_of_norm, length):
norm_of_raw[i] = offset in the displayed text of raw offset i (0..len(raw));
raw_of_norm[k] = raw offset of the character that is displayed character k."""
norm_of_raw: list[int] = []
raw_of_norm: list[int] = []
n = 0
started = False
prev_space = False
for i, ch in enumerate(raw):
norm_of_raw.append(n)
if _JS_SPACE_RE.fullmatch(ch):
if started and not prev_space:
raw_of_norm.append(i)
n += 1
prev_space = True
else:
started, prev_space = True, False
raw_of_norm.append(i)
n += 1
norm_of_raw.append(n)
length = n - 1 if prev_space and started else (n if started else 0)
if prev_space and started:
raw_of_norm.pop()
return [min(x, length) for x in norm_of_raw], raw_of_norm[:length], length

2
tui/diora_tui/cache.py
View file

@ -18,7 +18,7 @@ from . import paths
from .blocks import FlatBook
from .layout import BookLayout
_CACHE_VERSION = 2 # 2: blocks carry char_start/char_len
_CACHE_VERSION = 3 # 2: blocks carry char_start/char_len; 3: chapter_srcs
_CACHE_DIR = paths.cache_dir() / "layout_cache"

34
tui/diora_tui/layout.py
View file

@ -31,6 +31,28 @@ class BookLayout:
width: int
def render_block(text: str, width: int, console: Console, spans=()) -> list[Strip]:
"""One block wrapped to `width`, a Strip per row. `spans`: (start, end, style) over the text
(highlights, a selection); styling never changes how the text wraps."""
t = Text(text)
for start, end, style in spans:
t.stylize(style, start, end)
lines = list(t.wrap(console, width)) if text else []
if not lines:
lines = [Text("")]
return [Strip(line.render(console), None).adjust_cell_length(width) for line in lines]
def restyle_block(book_layout: "BookLayout", index: int, text: str, console: Console, spans=()) -> bool:
"""Draw block `index` again with `spans`, in place (its rows stay where they are)."""
strips = render_block(text, book_layout.width, console, spans)
if len(strips) != book_layout.heights[index]:
return False
start = book_layout.starts[index]
book_layout.row_strips[start:start + len(strips)] = strips
return True
def build_layout(block_texts: list[str], width: int, console: Console) -> BookLayout:
width = max(1, width)
starts: list[int] = []
@ -40,15 +62,11 @@ def build_layout(block_texts: list[str], width: int, console: Console) -> BookLa
row = 0
for text in block_texts:
starts.append(row)
wrapped_lines = list(Text(text).wrap(console, width)) if text else []
if not wrapped_lines:
wrapped_lines = [Text("")]
heights.append(len(wrapped_lines))
for line in wrapped_lines:
strip = Strip(line.render(console), None).adjust_cell_length(width)
row_strips.append(strip)
strips = render_block(text, width, console)
heights.append(len(strips))
row_strips.extend(strips)
row_strips.append(blank)
row += len(wrapped_lines) + _SEPARATOR_ROWS
row += len(strips) + _SEPARATOR_ROWS
return BookLayout(starts=starts, heights=heights, row_strips=row_strips, total_rows=row, width=width)

164
tui/diora_tui/marks.py Normal file
View file

@ -0,0 +1,164 @@
"""Highlights in the terminal text: from the web's anchors to spans of the displayed text, and
back from a selection to an anchor.
An anchor addresses characters of a block's *raw* text (every text node, whitespace as in the
file); the TUI shows the block with whitespace collapsed (blocks.collapse_map relates the two).
The raw text is read from the file for the blocks involved only, a chapter at a time.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from rich.style import Style
from . import annotations, blocks
SELECTION_STYLE = Style(reverse=True)
NOTE_STYLE = Style(underline=True, dim=False)
_WORD_RE = re.compile(r"\S+")
def highlight_style(color: str | None) -> Style:
return Style(color="black", bgcolor=annotations.COLORS.get(color or "yellow", annotations.COLORS["yellow"]))
# ---- words (the cursor moves by word)
def word_starts(text: str) -> list[int]:
return [m.start() for m in _WORD_RE.finditer(text)]
def word_at(text: str, pos: int) -> tuple[int, int] | None:
"""(start, end) of the word at or after `pos`, or None if there is none."""
for m in _WORD_RE.finditer(text):
if m.end() > pos:
return m.start(), m.end()
return None
def next_word(text: str, pos: int) -> int | None:
return next((s for s in word_starts(text) if s > pos), None)
def prev_word(text: str, pos: int) -> int | None:
return next((s for s in reversed(word_starts(text)) if s < pos), None)
@dataclass
class Maps:
raw: str
norm_of_raw: list[int]
raw_of_norm: list[int]
length: int
class MarkIndex:
"""Relates a book's displayed blocks to the raw text anchors count in."""
def __init__(self, book: blocks.FlatBook) -> None:
self.book = book
self._raws: dict[int, list[str]] = {} # chapter -> raw text of its blocks
self._maps: dict[int, Maps | None] = {}
def maps(self, block_index: int) -> Maps | None:
"""None when the displayed text isn't the plain collapse of the raw text (a footnote
marker with spaces inside): such a block can't be addressed exactly."""
if block_index in self._maps:
return self._maps[block_index]
b = self.book.blocks[block_index]
chapter = b.chapter_index
if chapter not in self._raws:
self._raws[chapter] = blocks.raw_block_texts(self.book.path, chapter)
first = self.book.chapter_start_block[chapter]
raws = self._raws[chapter]
result = None
if 0 <= block_index - first < len(raws):
raw = raws[block_index - first]
norm_of_raw, raw_of_norm, length = blocks.collapse_map(raw)
if length == len(b.text) and "".join(raw[i] if not blocks._JS_SPACE_RE.fullmatch(raw[i]) else " " for i in raw_of_norm) == b.text:
result = Maps(raw, norm_of_raw, raw_of_norm, length)
self._maps[block_index] = result
return result
# ---- annotation -> spans
def spans(self, items: list[dict]) -> dict[int, list[tuple[int, int, Style]]]:
"""Per block the (start, end, style) spans of the highlights and notes."""
out: dict[int, list[tuple[int, int, Style]]] = {}
n = len(self.book.blocks)
for h in items:
a = h.get("anchor") or {}
if "startBlockIndex" not in a: # an old anchor (XPath): the web falls back to the quote, so do we
found = self._find_quote(a)
if found is not None:
bi, s0, e0 = found
out.setdefault(bi, []).append((s0, e0, NOTE_STYLE if h.get("type") == "note" else highlight_style(h.get("color"))))
continue
try:
sb = int(a["startBlockIndex"])
eb = int(a.get("endBlockIndex", sb))
sc, ec = int(a.get("startChar", 0)), int(a.get("endChar", 0))
except (KeyError, TypeError, ValueError):
continue
if not (0 <= sb < n and sb <= eb < n):
continue
style = NOTE_STYLE if h.get("type") == "note" else highlight_style(h.get("color"))
for bi, s, e in self._ranges(sb, sc, eb, ec, a.get("quote") or ""):
if e > s:
out.setdefault(bi, []).append((s, e, style))
return out
def _find_quote(self, anchor: dict) -> tuple[int, int, int] | None:
"""(block, start, end) of the first place in the anchor's chapter that holds its quote."""
needle = " ".join((anchor.get("quote") or "").split())
src = anchor.get("chapterSrc")
if not needle or src not in self.book.chapter_srcs:
return None
chapter = self.book.chapter_srcs.index(src)
first = self.book.chapter_start_block[chapter]
last = self.book.chapter_start_block[chapter + 1] if chapter + 1 < len(self.book.chapter_start_block) else len(self.book.blocks)
for bi in range(first, last):
i = self.book.blocks[bi].text.find(needle)
if i >= 0:
return bi, i, i + len(needle)
return None
def _ranges(self, sb: int, sc: int, eb: int, ec: int, quote: str):
text = self.book.blocks[sb].text
m = self.maps(sb)
if m is None or (eb != sb and self.maps(eb) is None):
# not addressable by offsets: find the quote
i = text.find(" ".join(quote.split())) if quote else -1
if i >= 0:
yield sb, i, i + len(" ".join(quote.split()))
return
s = m.norm_of_raw[min(sc, len(m.raw))]
if sb == eb:
yield sb, s, m.norm_of_raw[min(ec, len(m.raw))]
return
yield sb, s, m.length
for bi in range(sb + 1, eb):
yield bi, 0, len(self.book.blocks[bi].text)
me = self.maps(eb)
yield eb, 0, me.norm_of_raw[min(ec, len(me.raw))]
# ---- selection -> anchor
def anchor(self, start: tuple[int, int], end: tuple[int, int]) -> dict | None:
"""The web anchor for the displayed range start..end ((block, char) each, end exclusive),
or None when a block can't be addressed exactly."""
(sb, sc), (eb, ec) = start, end
ms, me = self.maps(sb), self.maps(eb)
if ms is None or me is None:
return None
s_raw = ms.raw_of_norm[sc] if sc < ms.length else len(ms.raw)
e_raw = me.raw_of_norm[ec - 1] + 1 if ec > 0 else 0
if sb == eb:
quote = ms.raw[s_raw:e_raw]
else:
quote = ms.raw[s_raw:] + me.raw[:e_raw]
chapter_src = self.book.chapter_srcs[self.book.blocks[sb].chapter_index] if self.book.chapter_srcs else ""
return annotations.make_anchor(chapter_src, sb, s_raw, eb, e_raw, quote)

278
tui/diora_tui/reader_screen.py
View file

@ -18,6 +18,7 @@ is offered ("Auf X weitergelesen — springen?"), never applied by itself.
from __future__ import annotations
import bisect
import re
import threading
from pathlib import Path
@ -29,7 +30,8 @@ from textual.geometry import Size
from textual.screen import ModalScreen, Screen
from textual.widgets import Footer, Header, Label, LoadingIndicator, Static
from . import blocks, cache, config as config_mod, epub, layout, library, library_meta, progress, remote
from . import annotations, blocks, cache, config as config_mod, epub, layout, library, library_meta, marks, progress, remote
from .annotation_screens import AnnotationsScreen, NoteScreen
from .book_view import ContinuousBookView
from .statusbar import StatusBar
@ -45,6 +47,8 @@ FURTHEST_OFFER_SHARE = 0.01
# this fixed amount avoids a measure-after-constrain race against book_view's
# own (possibly already-constrained-from-a-previous-layout) width.
SCROLLBAR_GUTTER = 2
# A selection may span this many paragraphs (each is drawn again at every cursor move).
MAX_MARK_BLOCKS = 200
_EMPTY_LAYOUT = layout.BookLayout(starts=[], heights=[], row_strips=[], total_rows=0, width=1)
@ -95,6 +99,12 @@ class ReaderScreen(Screen):
Binding("p", "prev_chapter", "Vorheriges Kapitel"),
Binding("f", "peek_footnote", "Fußnote"),
Binding("w", "goto_furthest", "Weiteste Stelle"),
Binding("v", "select", "Markieren"),
Binding("b", "bookmark", "Lesezeichen"),
Binding("a", "annotations", "Markierungen"),
Binding("h,left", "word_back", "Wort zurück", show=False),
Binding("l,right", "word_forward", "Wort vor", show=False),
Binding("enter,space", "confirm", "Weiter", show=False),
Binding("q,escape", "back", "Zurück zur Bibliothek"),
]
@ -116,6 +126,11 @@ class ReaderScreen(Screen):
# server is what gets offered.
self._held_read_at: int | None = None
self._offer_open = False
self._marks: marks.MarkIndex | None = None
self._ann = annotations.default()
self._styled: set[int] = set() # blocks drawn with marks at the moment
self._sel: dict | None = None # the selection being made: phase, cursor, start
self._ann_timer = None
def compose(self):
yield Header()
@ -166,6 +181,10 @@ class ReaderScreen(Screen):
library_meta.set_local_pages(self._key, library.pages_of_chars(book.total_chars))
st = self._store.get(self._key)
self._held_read_at = st.read_at if st else None
self._marks = marks.MarkIndex(book)
self._apply_marks()
if self._cfg is not None:
self._pull_annotations()
self.call_after_refresh(self._restore_position)
self.set_interval(AUTOSAVE_INTERVAL, self._autosave)
if self._cfg is not None:
@ -199,6 +218,8 @@ class ReaderScreen(Screen):
new_layout = layout.build_layout(block_texts, width, console)
cache.save(self.path, width, self.book, new_layout)
self._apply_layout(new_layout, width)
self._styled = set() # a fresh layout is drawn without marks
self._apply_marks()
def on_resize(self) -> None:
if self.book is None:
@ -300,16 +321,17 @@ class ReaderScreen(Screen):
def on_unmount(self) -> None:
self._save_progress()
if self._cfg is not None and self._store.pending_keys():
if self._cfg is not None and (self._store.pending_keys() or self._ann.dirty_keys()):
# The screen's workers die with it; the queue still has to go out.
threading.Thread(target=self._flush_quietly, args=(self._cfg,), daemon=False).start()
@staticmethod
def _flush_quietly(cfg) -> None:
try:
remote.flush_pending(cfg)
except Exception:
pass
for send in (remote.flush_pending, remote.push_annotations):
try:
send(cfg)
except Exception:
pass
# ---- the server's newer positions
@ -378,9 +400,15 @@ class ReaderScreen(Screen):
# ---- navigation
def action_scroll_down_line(self) -> None:
if self._sel is not None:
self._move_block(1)
return
self._book_view.scroll_relative(y=1, animate=False)
def action_scroll_up_line(self) -> None:
if self._sel is not None:
self._move_block(-1)
return
self._book_view.scroll_relative(y=-1, animate=False)
def _current_chapter_index(self) -> int:
@ -394,6 +422,9 @@ class ReaderScreen(Screen):
self._book_view.scroll_to(y=y, animate=False, immediate=True)
def action_next_chapter(self) -> None:
if self._sel is not None:
self._selection_note() # n: a note, while marking
return
if self.book is None:
return
chapter = self._current_chapter_index()
@ -434,4 +465,239 @@ class ReaderScreen(Screen):
self.app.push_screen(FootnoteScreen(notes))
def action_back(self) -> None:
if self._sel is not None:
self._end_selection()
return
self.app.pop_screen()
# ---- marks on screen
def _marks_by_block(self) -> dict[int, list]:
spans: dict[int, list] = {}
if self._marks is not None:
for bi, lst in self._marks.spans(self._ann.get(self._key).highlights).items():
spans.setdefault(bi, []).extend(lst)
for bi, lst in self._selection_spans().items():
spans.setdefault(bi, []).extend(lst)
return spans
def _apply_marks(self) -> None:
"""Draw the highlights (and the selection) in place; blocks that lost theirs are drawn plain."""
if self.book is None or not self.book_layout.heights:
return
spans = self._marks_by_block()
console = Console(width=self.book_layout.width)
for bi in sorted(self._styled | set(spans)):
if 0 <= bi < len(self.book.blocks):
layout.restyle_block(self.book_layout, bi, self.book.blocks[bi].text, console, spans.get(bi, ()))
self._styled = set(spans)
self._book_view.refresh()
@work(thread=True, exclusive=True, group="ann-pull")
def _pull_annotations(self) -> None:
if remote.pull_annotations(self._cfg, self._server_id, self._key):
self.app.call_from_thread(self._apply_marks)
def _annotations_changed(self) -> None:
"""A highlight, note or bookmark was made, changed or deleted: draw it, and send it soon."""
self._apply_marks()
if self._cfg is None:
return
if self._ann_timer is not None:
self._ann_timer.stop()
self._ann_timer = self.set_timer(2.0, self._push_annotations) # like the web's debounce
@work(thread=True, exclusive=True, group="ann-push")
def _push_annotations(self) -> None:
try:
remote.push_annotations(self._cfg)
except Exception:
pass # stays queued
# ---- bookmarks and the list
def action_bookmark(self) -> None:
if self.book is None or not self.book_layout.heights or self._sel is not None:
return
block, inner = layout.get_position_anchor(self.book_layout, self._book_view.scroll_y)
chapter = max(0, self._current_chapter_index())
src = self.book.chapter_srcs[chapter] if chapter < len(self.book.chapter_srcs) else ""
label = re.sub(r"\.x?html?$", "", src.rsplit("/", 1)[-1], flags=re.I) or "Lesezeichen" # as the web names it
bm = annotations.new_bookmark(label, src, blocks.fraction_for_anchor(self.book, block, inner))
self._ann.add(self._key, "bookmarks", bm)
self.notify(f"★ Lesezeichen: {label}", timeout=2.5)
self._annotations_changed()
def action_annotations(self) -> None:
if self.book is None or self._sel is not None:
return
def picked(block: int | None) -> None:
if block is not None:
self._jump_quiet(block)
self.app.push_screen(AnnotationsScreen(self.book, self._ann, self._key, self._annotations_changed, self._marks), picked)
def _jump_quiet(self, block: int) -> None:
"""Go to a block as a jump, not as reading: nothing is saved until the reader moves on."""
y = layout.scroll_y_for_anchor(self.book_layout, block, 0.0)
with self._book_view.program_scroll():
self._book_view.scroll_to(y=y, animate=False, immediate=True)
self._last_anchor = self._current_anchor_str()
self._last_move_ms = None
# ---- selecting text (v): a cursor by word, Enter sets the start, Enter again the end
def _first_block_in_view(self) -> int:
y = self._book_view.scroll_y
i = bisect.bisect_left(self.book_layout.starts, y)
return min(i, len(self.book.blocks) - 1)
def _word_block(self, block: int, step: int) -> int | None:
"""The nearest block from `block` (inclusive) in direction `step` that has a word."""
while 0 <= block < len(self.book.blocks):
if marks.word_starts(self.book.blocks[block].text):
return block
block += step
return None
def action_select(self) -> None:
if self.book is None or not self.book_layout.heights:
return
if self._sel is not None:
self.action_confirm()
return
block = self._word_block(self._first_block_in_view(), 1)
if block is None:
return
self._sel = {"phase": "start", "cursor": (block, marks.word_starts(self.book.blocks[block].text)[0]), "start": None}
self.notify("Markieren: h/l Wort, j/k Absatz, Enter = Anfang, n = Notiz, Esc = Abbrechen", timeout=6)
self._selection_moved()
def _end_selection(self) -> None:
self._sel = None
self._apply_marks()
def _selection_moved(self) -> None:
self._apply_marks()
block, char = self._sel["cursor"]
text = self.book.blocks[block].text
row = self.book_layout.starts[block] + int(char / max(1, len(text)) * self.book_layout.heights[block])
top, height = self._book_view.scroll_y, self._book_view.size.height
if not (top <= row < top + height - 1):
with self._book_view.program_scroll():
self._book_view.scroll_to(y=max(0, row - height // 3), animate=False, immediate=True)
self._last_anchor = self._current_anchor_str()
def action_word_forward(self) -> None:
self._move_word(1)
def action_word_back(self) -> None:
self._move_word(-1)
def _move_word(self, step: int) -> None:
if self._sel is None:
return
block, char = self._sel["cursor"]
text = self.book.blocks[block].text
target = marks.next_word(text, char) if step > 0 else marks.prev_word(text, char)
if target is not None:
self._sel["cursor"] = (block, target)
else:
nb = self._word_block(block + step, step)
if nb is None:
return
ws = marks.word_starts(self.book.blocks[nb].text)
self._sel["cursor"] = (nb, ws[0] if step > 0 else ws[-1])
self._selection_moved()
def _move_block(self, step: int) -> None:
nb = self._word_block(self._sel["cursor"][0] + step, step)
if nb is not None:
self._sel["cursor"] = (nb, marks.word_starts(self.book.blocks[nb].text)[0])
self._selection_moved()
def _cursor_word(self) -> tuple[int, int, int]:
block, char = self._sel["cursor"]
w = marks.word_at(self.book.blocks[block].text, char) or (char, char)
return block, w[0], w[1]
def _selection_range(self) -> tuple[tuple[int, int], tuple[int, int]]:
"""(start, end) of what is selected, end exclusive, whichever way the cursor went."""
sb, sc = self._sel["start"]
sw = marks.word_at(self.book.blocks[sb].text, sc) or (sc, sc)
cb, cs, ce = self._cursor_word()
if (cb, cs) >= (sb, sw[0]):
return (sb, sw[0]), (cb, ce)
return (cb, cs), (sb, sw[1])
def _selection_spans(self) -> dict[int, list]:
if self._sel is None or self.book is None:
return {}
if self._sel["phase"] == "start":
block, s, e = self._cursor_word()
return {block: [(s, e, marks.SELECTION_STYLE)]}
(sb, sc), (eb, ec) = self._selection_range()
out: dict[int, list] = {}
for bi in range(sb, eb + 1):
text_len = len(self.book.blocks[bi].text)
out[bi] = [(sc if bi == sb else 0, ec if bi == eb else text_len, marks.SELECTION_STYLE)]
return out
def action_confirm(self) -> None:
if self._sel is None:
return
if self._sel["phase"] == "start":
block, s, _e = self._cursor_word()
self._sel["start"] = (block, s)
self._sel["phase"] = "end"
self.notify("Ende wählen (h/l/j/k), Enter = fertig.", timeout=4)
self._selection_moved()
return
(sb, sc), (eb, ec) = self._selection_range()
if eb - sb > MAX_MARK_BLOCKS:
self.notify(f"Zu lang zum Markieren (mehr als {MAX_MARK_BLOCKS} Absätze).", severity="warning")
return
anchor = self._marks.anchor((sb, sc), (eb, ec))
if anchor is None:
self.notify("Diese Stelle lässt sich nicht genau zuordnen (Fußnotenmarker mit Leerraum).", severity="warning")
return
names = [annotations.COLOR_NAMES[c] for c in annotations.COLORS] + ["Gelb mit Notiz …"]
colors = list(annotations.COLORS)
def picked(i: int | None) -> None:
if i is None:
return # the selection stays: pick again or Esc
if i < len(colors):
self._add_highlight(anchor, colors[i], "")
else:
self.app.push_screen(NoteScreen("Notiz zur Markierung"), lambda text: self._add_highlight(anchor, "yellow", text) if text is not None else None)
from .library_screen import ChoiceScreen
self.app.push_screen(ChoiceScreen("Markieren als", names, 0), picked)
def _add_highlight(self, anchor: dict, color: str, note: str) -> None:
self._ann.add(self._key, "highlights", annotations.new_highlight(anchor, color, note))
self._sel = None
self._annotations_changed()
def _selection_note(self) -> None:
"""n while marking: a freeform note at the cursor's word (phase 1) or on the selection."""
if self._sel["phase"] == "start":
block, s, e = self._cursor_word()
start, end = (block, s), (block, e)
else:
start, end = self._selection_range()
anchor = self._marks.anchor(start, end)
if anchor is None:
self.notify("Diese Stelle lässt sich nicht genau zuordnen.", severity="warning")
return
def noted(text: str | None) -> None:
if text:
self._ann.add(self._key, "highlights", annotations.new_note(anchor, text))
self._sel = None
self._annotations_changed()
self.app.push_screen(NoteScreen("Notiz"), noted)

51
tui/diora_tui/remote.py
View file

@ -16,7 +16,7 @@ from pathlib import Path
import requests
from . import api, crypto, library, library_meta, progress
from . import annotations, api, crypto, library, library_meta, progress
from .config import RemoteConfig
from .epub import server_key
@ -70,11 +70,60 @@ def flush_pending(cfg: RemoteConfig) -> int:
return pushed
def pull_annotations(cfg: RemoteConfig, server_id: int, key: str) -> bool:
"""The server's highlights and bookmarks of the book, laid under our queued edits. False when
offline or the key doesn't open them (what is local stays)."""
store = annotations.default()
try:
for kind in annotations.KINDS:
theirs = annotations.decode(cfg.enc_key_b64, api.fetch_annotations(cfg.server_url, cfg.api_token, server_id, kind))
store.take_remote(key, kind, theirs)
except (api.ApiError, crypto.DecryptError, ValueError, requests.RequestException):
return False
return True
def push_annotations(cfg: RemoteConfig) -> int:
"""Write every queued list: merged with the server's current one first (annotations.py).
Stops quietly at network trouble; the rest stays queued."""
store = annotations.default()
written = 0
for key in store.dirty_keys():
if not (key.startswith("s") and key[1:].isdigit()):
continue
book_id = int(key[1:])
for kind in annotations.KINDS:
if not store.get(key).dirty[kind]:
continue
try:
theirs = annotations.decode(cfg.enc_key_b64, api.fetch_annotations(cfg.server_url, cfg.api_token, book_id, kind))
merged, snapshot = store.for_push(key, kind, theirs)
blob = annotations.encode(cfg.enc_key_b64, merged)
if len(blob["ct"]) * 3 // 4 > annotations.MAX_BYTES[kind]:
raise SyncError("Zu viele Markierungen für dieses Buch (Grenze des Servers erreicht).")
api.post_annotations(cfg.server_url, cfg.api_token, book_id, kind, blob)
except api.ApiError as e:
if "nicht gefunden" in str(e):
continue # the book is gone
raise SyncError(str(e)) from e
except (crypto.DecryptError, ValueError) as e:
raise SyncError("Die Markierungen auf dem Server lassen sich nicht entschlüsseln.") from e
except requests.RequestException:
return written
store.pushed(key, kind, merged, snapshot)
written += 1
return written
def sync_library(library_dir: Path, cfg: RemoteConfig) -> SyncResult:
"""Push what is queued, then pull the book list and every book's reading state."""
result = SyncResult()
try:
result.pushed = flush_pending(cfg)
try:
push_annotations(cfg)
except SyncError:
pass # the notes stay queued; the book list is what this call is for
books, server_now, device_now = api.fetch_books(cfg.server_url, cfg.api_token)
except api.ApiError as e:
raise SyncError(str(e)) from e

142
tui/tests/test_annotations.py Normal file
View file

@ -0,0 +1,142 @@
"""Highlights, notes and bookmarks: the formats of the web reader and the merge the TUI adds
(the server replaces the whole list; nothing another device wrote may be lost)."""
import base64
import os
import pytest
import requests
from diora_tui import annotations as an, api, crypto, remote
from diora_tui.config import RemoteConfig
def hl(i, note="", block=1):
return {"id": i, "type": "highlight", "anchor": an.make_anchor("c.xhtml", block, 0, block, 5, "quote"), "color": "yellow", "note": note, "createdAt": "x"}
def ids(items):
return [x["id"] for x in items]
def test_merge_keeps_their_new_items_and_applies_our_changes():
base = [hl("a"), hl("b"), hl("c")]
ours = [hl("a", note="edited"), hl("c"), hl("ours")] # edited a, deleted b, added ours
theirs = [hl("a"), hl("b"), hl("c"), hl("theirs")] # added theirs
out = an.merge(base, ours, theirs)
assert ids(out) == ["a", "c", "theirs", "ours"] # b: we deleted it
assert [x for x in out if x["id"] == "a"][0]["note"] == "edited"
def test_merge_their_deletion_does_not_come_back_unless_we_changed_it():
base = [hl("a"), hl("b")]
assert ids(an.merge(base, [hl("a"), hl("b")], [hl("a")])) == ["a"] # they deleted b, we left it
assert ids(an.merge(base, [hl("a"), hl("b", note="mine")], [hl("a")])) == ["a", "b"] # we edited it: kept
def test_merge_without_a_base_deletes_nothing():
assert ids(an.merge(None, [hl("a")], [hl("a"), hl("b")])) == ["a", "b"]
def test_bookmarks_are_newest_first():
bm = lambda i: {"id": i, "label": i, "anchor": "c", "scrollFraction": 0, "createdAt": "x"} # noqa: E731
out = an.merge([bm("old")], [bm("new"), bm("old")], [bm("old"), bm("theirs")], prepend_added=True)
assert ids(out) == ["new", "old", "theirs"]
@pytest.fixture
def store(tmp_path):
return an.AnnotationStore(tmp_path / "ann")
def test_edits_are_queued_and_survive_a_restart(store, tmp_path):
h = an.new_highlight(an.make_anchor("c", 3, 0, 3, 4, "abcd"), "green", "n")
store.add("s1", "highlights", h)
store.add("s1", "bookmarks", an.new_bookmark("Kap. 1", "c", 0.1))
store.add("s1", "bookmarks", an.new_bookmark("Kap. 2", "c2", 0.2))
assert store.dirty_keys() == ["s1"]
again = an.AnnotationStore(tmp_path / "ann").get("s1")
assert ids(again.highlights) == [h["id"]] and [b["label"] for b in again.bookmarks] == ["Kap. 2", "Kap. 1"]
assert again.dirty == {"highlights": True, "bookmarks": True}
def test_note_edit_and_delete(store):
h = an.new_highlight(an.make_anchor("c", 3, 0, 3, 4, "abcd"))
store.add("s1", "highlights", h)
assert store.set_note("s1", h["id"], "text") and not store.set_note("s1", h["id"], "text")
assert store.get("s1").highlights[0]["note"] == "text"
assert store.delete("s1", "highlights", h["id"]) and not store.delete("s1", "highlights", h["id"])
def test_pull_replaces_when_clean_and_merges_when_dirty(store):
store.take_remote("s1", "highlights", [hl("a")])
assert ids(store.get("s1").highlights) == ["a"] and not store.get("s1").dirty["highlights"]
store.add("s1", "highlights", hl("mine"))
store.take_remote("s1", "highlights", [hl("a"), hl("b")])
assert ids(store.get("s1").highlights) == ["a", "b", "mine"] and store.get("s1").dirty["highlights"]
def test_push_flow_with_an_edit_during_the_request(store):
store.take_remote("s1", "highlights", [hl("a")])
store.add("s1", "highlights", hl("one"))
merged, snap = store.for_push("s1", "highlights", [hl("a"), hl("theirs")])
assert ids(merged) == ["a", "theirs", "one"]
store.add("s1", "highlights", hl("two")) # made while the request was out
store.pushed("s1", "highlights", merged, snap)
got = store.get("s1")
assert ids(got.highlights) == ["a", "theirs", "one", "two"] and got.dirty["highlights"]
assert ids(got.base["highlights"]) == ["a", "theirs", "one"]
def test_blob_roundtrip():
key = base64.b64encode(os.urandom(32)).decode()
items = [hl("ä")]
assert an.decode(key, an.encode(key, items)) == items
assert an.decode(key, {"ct": None, "iv": None}) == []
CFG = RemoteConfig("http://x", "tok", base64.b64encode(os.urandom(32)).decode())
class FakeServer:
def __init__(self):
self.blobs = {}
self.fail = None
def fetch(self, server, token, book_id, kind):
if self.fail:
raise self.fail
return self.blobs.get((book_id, kind), {"ct": None, "iv": None})
def post(self, server, token, book_id, kind, blob):
if self.fail:
raise self.fail
self.blobs[(book_id, kind)] = blob
@pytest.fixture
def server(store, monkeypatch):
s = FakeServer()
monkeypatch.setattr(an, "_default", store)
monkeypatch.setattr(api, "fetch_annotations", s.fetch)
monkeypatch.setattr(api, "post_annotations", s.post)
return s
def test_push_merges_with_what_another_device_wrote(server, store):
server.blobs[(7, "highlights")] = an.encode(CFG.enc_key_b64, [hl("theirs")])
store.add("s7", "highlights", hl("mine"))
assert remote.push_annotations(CFG) == 1
assert ids(an.decode(CFG.enc_key_b64, server.blobs[(7, "highlights")])) == ["theirs", "mine"]
assert store.dirty_keys() == [] and ids(store.get("s7").highlights) == ["theirs", "mine"]
def test_push_keeps_the_queue_offline_and_pull_survives_it(server, store):
store.add("s7", "highlights", hl("mine"))
server.fail = requests.ConnectionError("down")
assert remote.push_annotations(CFG) == 0 and store.dirty_keys() == ["s7"]
assert remote.pull_annotations(CFG, 7, "s7") is False and ids(store.get("s7").highlights) == ["mine"]
def test_local_books_are_never_pushed(server, store):
store.add("0123456789abcdef", "highlights", hl("mine"))
assert remote.push_annotations(CFG) == 0 and not server.blobs

79
tui/tests/test_marks.py Normal file
View file

@ -0,0 +1,79 @@
"""Anchors the way the web reader writes them: block index plus character offsets into the
block's raw text (whitespace as in the file)."""
from diora_tui import blocks, marks
from diora_tui.blocks import collapse_map, load_flat_book
BODY = (
"<h1>Titel</h1>\n"
"<p>\n Erster Satz mit\n Umbruch. <i>Kursiv</i> Ende.\n</p>\n"
"<p>Zweiter Absatz.</p>"
)
def test_collapse_map():
raw = "\n ab cd \n"
norm_of_raw, raw_of_norm, length = collapse_map(raw)
assert length == 5 and "".join(raw[i] if raw[i] != " " else " " for i in raw_of_norm) == "ab cd"
assert raw_of_norm == [3, 4, 5, 8, 9] # a, b, the first space of the run, c, d
assert norm_of_raw[3] == 0 and norm_of_raw[5] == 2 and norm_of_raw[len(raw)] == 5
def test_anchor_counts_in_raw_text_and_resolves_back(epub):
book = load_flat_book(epub([BODY]))
idx = marks.MarkIndex(book)
para = next(i for i, b in enumerate(book.blocks) if b.text.startswith("Erster"))
text = book.blocks[para].text
assert text == "Erster Satz mit Umbruch. Kursiv Ende."
s = text.index("Satz")
e = text.index("Umbruch") + len("Umbruch")
anchor = idx.anchor((para, s), (para, e))
raw = idx.maps(para).raw
assert raw[anchor["startChar"]:anchor["endChar"]] == "Satz mit\n Umbruch" # raw offsets: whitespace as in the file
assert anchor["startBlockIndex"] == anchor["endBlockIndex"] == para and anchor["type"] == "epub"
assert anchor["quote"] == "Satz mit\n Umbruch" and anchor["chapterSrc"].endswith("c0.xhtml")
spans = idx.spans([{"type": "highlight", "color": "green", "anchor": anchor}])
assert list(spans) == [para] and spans[para][0][:2] == (s, e)
def test_range_over_blocks_and_notes_and_bad_anchors(epub):
book = load_flat_book(epub([BODY]))
idx = marks.MarkIndex(book)
first = next(i for i, b in enumerate(book.blocks) if b.text.startswith("Erster"))
last = first + 1
anchor = idx.anchor((first, 7), (last, 6))
spans = idx.spans([{"type": "highlight", "anchor": anchor}])
assert spans[first][0][:2] == (7, len(book.blocks[first].text)) and spans[last][0][:2] == (0, 6)
note = {"type": "note", "anchor": idx.anchor((first, 0), (first, 6))}
assert idx.spans([note])[first][0][:2] == (0, 6)
junk = [{"anchor": {"startBlockIndex": 9999}}, {"anchor": None}, {"anchor": {"startBlockIndex": "x"}}, {}]
assert idx.spans(junk) == {}
def test_a_footnote_marker_with_spaces_cannot_be_addressed_so_the_quote_is_used(epub):
body = '<p>Vor <a href="#n"><sup> 1 </sup></a>nach dem Marker.</p><p id="n">Fuß</p>'
book = load_flat_book(epub([body]))
idx = marks.MarkIndex(book)
assert idx.maps(0) is None
assert idx.anchor((0, 0), (0, 3)) is None # refused: not exact
spans = idx.spans([{"type": "highlight", "anchor": {"startBlockIndex": 0, "endBlockIndex": 0, "startChar": 3, "endChar": 9, "quote": "nach dem"}}])
t = book.blocks[0].text
assert spans[0][0][:2] == (t.index("nach dem"), t.index("nach dem") + 8)
def test_words():
t = "ab cd e"
assert marks.word_starts(t) == [0, 4, 7]
assert marks.next_word(t, 0) == 4 and marks.next_word(t, 7) is None
assert marks.prev_word(t, 4) == 0 and marks.prev_word(t, 0) is None
assert marks.word_at(t, 2) == (4, 6) # in the gap: the next word
def test_an_old_xpath_anchor_is_found_by_its_quote(epub):
book = load_flat_book(epub([BODY]))
idx = marks.MarkIndex(book)
old = {"type": "note", "anchor": {"type": "epub", "chapterSrc": book.chapter_srcs[0], "startXpath": "/p[1]", "startOffset": 3,
"endXpath": "/p[1]", "endOffset": 8, "quote": "Zweiter Absatz"}}
spans = idx.spans([old])
(bi, [(s, e, _)]), = spans.items()
assert book.blocks[bi].text[s:e] == "Zweiter Absatz"
assert idx.spans([{"anchor": {"type": "epub", "chapterSrc": "elsewhere.xhtml", "quote": "x"}}]) == {}

137
tui/tests/test_reader_annotations.py Normal file
View file

@ -0,0 +1,137 @@
"""Marking text, bookmarks and the list, in the reader."""
import asyncio
import pytest
from textual.widgets import Input, ListView
from diora_tui import annotations, epub as epub_mod, progress
from diora_tui.annotation_screens import AnnotationsScreen, NoteScreen
from diora_tui.app import DioraTuiApp
from diora_tui.library_screen import ChoiceScreen
from diora_tui.reader_screen import ReaderScreen
@pytest.fixture
def world(tmp_path, monkeypatch):
monkeypatch.setattr(progress, "_default", progress.ProgressStore(tmp_path / "p.json"))
store = annotations.AnnotationStore(tmp_path / "ann")
monkeypatch.setattr(annotations, "_default", store)
lib = tmp_path / "lib"
lib.mkdir()
return lib, store
def book(epub):
body = "<h1>Anfang</h1>\n" + "\n".join(f"<p>\n Absatz {i} eins zwei drei vier fünf sechs sieben.\n</p>" for i in range(30))
return epub([body], title="Marken")
async def open_reader(app, pilot, path):
app.push_screen(ReaderScreen(path))
for _ in range(80):
await pilot.pause(0.1)
s = app.screen
if isinstance(s, ReaderScreen) and s.book is not None and s._last_anchor is not None:
await pilot.pause(0.2)
return s
raise AssertionError("reader did not load")
def run(lib, epub, scenario):
path = book(epub)
async def go():
app = DioraTuiApp(lib)
async with app.run_test(size=(100, 30)) as pilot:
reader = await open_reader(app, pilot, path)
await scenario(app, reader, pilot, epub_mod._book_id(path))
asyncio.run(go())
def painted_with(reader, color_hex):
wanted = color_hex.lstrip("#").lower()
return any(seg.style and seg.style.bgcolor and seg.style.bgcolor.get_truecolor().hex.lstrip("#").lower() == wanted
for strip in reader.book_layout.row_strips for seg in strip)
def test_mark_text_then_list_edit_and_delete(world, epub):
lib, store = world
async def scenario(app, reader, pilot, key):
await pilot.press("v")
assert reader._sel is not None and reader._sel["phase"] == "start"
await pilot.press("l", "l") # two words on
await pilot.press("enter") # start
assert reader._sel["phase"] == "end"
await pilot.press("l", "l", "l")
await pilot.press("enter") # end: pick a colour
assert isinstance(app.screen, ChoiceScreen)
await pilot.press("down", "enter") # Grün
await pilot.pause(0.2)
assert reader._sel is None
hs = store.get(key).highlights
assert len(hs) == 1 and hs[0]["color"] == "green" and hs[0]["type"] == "highlight"
a = hs[0]["anchor"]
assert a["startBlockIndex"] == a["endBlockIndex"] and a["quote"] and a["startChar"] < a["endChar"]
assert a["quote"] == reader._marks.maps(a["startBlockIndex"]).raw[a["startChar"]:a["endChar"]]
assert painted_with(reader, "#2ecc71")
assert store.get(key).dirty["highlights"]
await pilot.press("b")
bm = store.get(key).bookmarks
assert len(bm) == 1 and bm[0]["anchor"].endswith("c0.xhtml") and bm[0]["label"] == "c0"
await pilot.press("a")
assert isinstance(app.screen, AnnotationsScreen)
lv = app.screen.query_one(ListView)
assert len(app.screen.rows) == 2 and app.screen.rows[0].kind == "bookmarks"
await pilot.press("down", "e") # edit the note of the highlight
assert isinstance(app.screen, NoteScreen)
app.screen.query_one(Input).value = "klug"
await pilot.press("enter")
await pilot.pause(0.2)
assert store.get(key).highlights[0]["note"] == "klug"
await pilot.press("d") # delete the highlight
await pilot.pause(0.2)
assert store.get(key).highlights == [] and not painted_with(reader, "#2ecc71")
await pilot.press("escape")
assert isinstance(app.screen, ReaderScreen)
run(lib, epub, scenario)
def test_note_on_a_word_and_escape_leaves_the_selection(world, epub):
lib, store = world
async def scenario(app, reader, pilot, key):
await pilot.press("v", "l")
await pilot.press("escape")
assert reader._sel is None and isinstance(app.screen, ReaderScreen)
assert store.get(key).highlights == []
await pilot.press("v", "l", "n") # n while marking: a freeform note on this word
assert isinstance(app.screen, NoteScreen)
app.screen.query_one(Input).value = "merken"
await pilot.press("enter")
await pilot.pause(0.2)
h = store.get(key).highlights
assert len(h) == 1 and h[0]["type"] == "note" and h[0]["note"] == "merken" and h[0]["color"] is None
a = h[0]["anchor"]
assert reader._marks.maps(a["startBlockIndex"]).raw[a["startChar"]:a["endChar"]] == a["quote"]
run(lib, epub, scenario)
def test_jump_from_the_list_is_not_reading(world, epub):
lib, store = world
async def scenario(app, reader, pilot, key):
store.add(key, "bookmarks", annotations.new_bookmark("c0", "OEBPS/c0.xhtml", 0.5))
await pilot.press("a", "enter")
await pilot.pause(0.3)
assert isinstance(app.screen, ReaderScreen)
reader._save_progress()
assert progress.default().get(key) is None # nothing written by the jump
run(lib, epub, scenario)

42
tui/tools/parity/web_anchors.cjs Normal file
View file

@ -0,0 +1,42 @@
// Does the web reader find a highlight made in the TUI where the TUI put it? Runs the web's own
// resolveAnchorRange (72-highlights.js) on the book rendered like the reader renders it, and
// returns the text of the range it finds for each anchor.
//
// node web_anchors.cjs <diora-web checkout> <book.epub> <anchors.json> <out.json>
//
// anchors.json: a list of highlight objects ({anchor: {...}, ...}). Same setup as web_blocks.cjs.
const fs = require('fs');
const path = require('path');
const { chromium } = require('playwright-core');
(async () => {
const [web, epub, anchorsFile, out] = process.argv.slice(2);
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('about:blank');
await page.evaluate(() => {
window.$ = (id) => document.getElementById(id);
window.currentPdfDoc = null;
});
await page.addScriptTag({ path: path.join(web, 'static/js/jszip.min.js') });
for (const f of ['40-epub.js', '44-books.js', '72-highlights.js']) {
try { await page.addScriptTag({ path: path.join(web, 'static/js/app', f) }); }
catch (e) { console.error('loading', f, e.message); }
}
const items = JSON.parse(fs.readFileSync(anchorsFile, 'utf-8'));
const b64 = fs.readFileSync(epub).toString('base64');
const result = await page.evaluate(async ({ b64, items }) => {
const bin = Uint8Array.from(atob(b64), (c) => c.charCodeAt(0));
const book = await parseEpub(bin.buffer);
const el = document.createElement('div');
el.id = 'reader-content';
el.innerHTML = book.chapters.join('');
document.body.appendChild(el);
return items.map((h) => {
const r = resolveAnchorRange(h);
return r ? r.toString() : null;
});
}, { b64, items });
fs.writeFileSync(out, JSON.stringify(result));
await browser.close();
})().catch((e) => { console.error(e); process.exit(1); });

26
tui/tools/smoke_server.py
View file

@ -108,4 +108,30 @@ check("an older write loses on the server", row["position_anchor"] == other and
check("...and locally", store.get(skey).anchor == other and store.get(skey).pending is None)
check("...but raises the furthest place",
progress.cmp_progress(row["furthest_anchor"], 0, blocks.format_anchor(30, 0.0), 0) >= 0, row["furthest_anchor"])
# highlights and bookmarks: ours go up merged with another device's, in the web's format
from diora_tui import annotations as an # noqa: E402
astore = an.default()
sid = info["server_id"]
theirs = [{"id": "web-1", "type": "highlight", "anchor": an.make_anchor("c", 3, 0, 3, 4, "abcd"), "color": "red", "note": "", "createdAt": "2026-10-26T10:00:00.000Z"}]
iv, ct = crypto.encrypt(key_b64, json.dumps(theirs).encode())
check("another device wrote a highlight", requests.post(f"{server}/books/{sid}/highlights/", json={"ct": ct, "iv": iv}, headers=H).ok)
remote.pull_annotations(cfg, sid, skey)
check("pulled", [h["id"] for h in astore.get(skey).highlights] == ["web-1"])
mine = an.new_highlight(an.make_anchor("c", 5, 2, 5, 9, "efghijk"), "green", "meine Notiz")
astore.add(skey, "highlights", mine)
astore.add(skey, "bookmarks", an.new_bookmark("c", "c", 0.5))
# meanwhile the other device adds one more
theirs.append({**theirs[0], "id": "web-2", "color": "blue"})
iv, ct = crypto.encrypt(key_b64, json.dumps(theirs).encode())
requests.post(f"{server}/books/{sid}/highlights/", json={"ct": ct, "iv": iv}, headers=H)
check("pushed", remote.push_annotations(cfg) == 2 and astore.dirty_keys() == [])
got = an.decode(key_b64, requests.get(f"{server}/books/{sid}/highlights/", headers=H).json())
check("the server holds both devices' highlights", [h["id"] for h in got] == ["web-1", "web-2", mine["id"]] and got[2]["note"] == "meine Notiz")
bms = an.decode(key_b64, requests.get(f"{server}/books/{sid}/bookmarks/", headers=H).json())
check("and the bookmark", len(bms) == 1 and bms[0]["label"] == "c")
astore.delete(skey, "highlights", "web-1")
remote.push_annotations(cfg)
got = an.decode(key_b64, requests.get(f"{server}/books/{sid}/highlights/", headers=H).json())
check("a deletion goes up too", [h["id"] for h in got] == ["web-2", mine["id"]])
print("all good")