diff --git a/tui/README.md b/tui/README.md index bf2efa6..d997368 100644 --- a/tui/README.md +++ b/tui/README.md @@ -1,16 +1,20 @@ # diora-tui -Lokaler EPUB-Reader im Terminal — die erste Stufe einer TUI-Version von diora. +EPUB-Reader im Terminal — die erste Stufe einer TUI-Version von diora. -Läuft komplett offline gegen eine lokale Bibliothek aus `.epub`-Dateien und merkt sich -den Lesefortschritt pro Buch (`progress.json` im plattformüblichen Datenverzeichnis, -z.B. `~/.local/share/diora-tui/` unter Linux, via `platformdirs`). Der Fortschritt wird -— genau wie im Web-Reader von diora (siehe `books/models.py`, `save_progress`) — nur -vorwärts überschrieben ("furthest wins"), damit ein älterer/gestaffelter Lauf nie eine -bereits weiter gelesene Position zurücksetzt. +Zeigt ein Buch als **eine durchgehende Ansicht** über alle Kapitel hinweg (wie diora's +Web-Reader), nicht Kapitel für Kapitel. Lesefortschritt wird als dieselbe +`"blockIndex:innerFraction"`-Positionsangabe geführt wie der Web-Reader (`books/models.py`, +`EBookProgress`) — `blockIndex` zählt dabei exakt wie `static/js/app.js`s +`EPUB_BLOCK_SELECTOR` (`p, h1-h6, li, blockquote, dt, dd, figcaption` + textbasierte +`div`s ohne Element-Kinder), fortlaufend über das ganze Buch. Dadurch ist die Position +zwischen TUI und Web-Reader direkt vergleichbar, und `diora-tui sync` kann Fortschritt in +beide Richtungen synchronisieren (siehe unten). -`diora-tui sync` kann Bücher jetzt vom diora-Server holen und lokal entschlüsseln (siehe -unten) — Lesefortschritt bleibt aber weiterhin rein lokal, siehe **Grenzen** unten. +Fortschritt wird — genau wie im Web-Reader (`save_progress`) — nur vorwärts überschrieben +("furthest wins"), lokal in `progress.json` (`~/.local/share/diora-tui/`, via +`platformdirs`), damit ein älterer/gestaffelter Lauf nie eine bereits weiter gelesene +Position zurücksetzt. ## Setup @@ -28,11 +32,18 @@ diora-tui --library ~/Books # Standard: ~/Books Tastenkürzel: - `↑`/`k`, `↓`/`j` — zeilenweise scrollen -- `n` / `]` — nächstes Kapitel, `p` / `[` — vorheriges Kapitel +- `n` — nächstes Kapitel, `p` — vorheriges Kapitel +- `f` — Fußnote in der Nähe der aktuellen Position anzeigen (Peek-Overlay, `Escape`/`f`/`q` + zum Schließen); Erkennung folgt derselben Heuristik wie `app.js`s + `_looksLikeFootnoteLink` (Link in/um ``, Klassenname mit note/footnote/fn, oder + `epub:type="noteref"`) - `Enter` — markiertes Buch aus der Bibliothek öffnen - `Escape` / `q` — zurück zur Bibliothek (im Reader) bzw. beenden (in der Bibliothek) -## Bücher vom Server holen (`diora-tui sync`) +Der Fließtext ist auf 120 Zeichen Breite begrenzt und horizontal zentriert (lesbarer als +volle Terminalbreite bei breiten Fenstern). + +## Bücher + Fortschritt vom Server holen (`diora-tui sync`) ```bash diora-tui sync # nutzt gespeicherte Zugangsdaten, sonst interaktive Abfrage @@ -43,7 +54,14 @@ Lädt alle EPUBs des Accounts über `GET /api/sync/` + `GET /books//data/` h entschlüsselt sie lokal (AES-256-GCM, kompatibel zu `static/js/app.js`) und legt sie als normale `.epub`-Dateien in `--library` ab (Dateiname ` - .epub`) — von da an funktionieren sie wie jedes andere lokale Buch. Bereits heruntergeladene Bücher werden -beim nächsten Lauf übersprungen (kein erneuter Download). +beim nächsten Lauf übersprungen (kein erneuter Download). Der Fortschritt aus dem Snapshot +wird dabei ebenfalls übernommen (nur vorwärts, wie lokal auch). + +Während ein so heruntergeladenes Buch geöffnet ist (erkennbar am ` - `-Dateinamens- +Präfix), schickt der Reader Fortschritts-Updates zusätzlich zurück an den Server +(`POST /books//progress/`, `force: false` — überschreibt also nie eine weiter +gelesene Position, egal ob die vom Web-Reader oder einem anderen Gerät stammt). Rein +lokale Bücher (ohne dieses Präfix) bleiben unangetastet, kein Netzwerkzugriff. Dafür nötig, beim ersten Lauf abgefragt (danach optional lokal gespeichert unter `~/.config/diora-tui/config.json`, `chmod 600`): @@ -64,18 +82,29 @@ Dafür nötig, beim ersten Lauf abgefragt (danach optional lokal gespeichert unt `app.js` existiert, hat aber keinen sichtbaren Button); bis das nachgezogen ist, in der Browser-Devtools-Konsole auf der diora-Seite (nicht `/accounts/settings/` — die lädt `app.js` nicht) ausführen: `await exportEncKey()` — kopiert den Key ins - Clipboard, von dort ins `sync`-Prompt einfügen. + Clipboard, von dort ins `sync`-Prompt einfügen. Schlägt das mit einem + `NotAllowedError`/`InvalidAccessError` fehl, stattdessen direkt aus `localStorage` + lesen: `localStorage.getItem('diora_enc_key_' + window.USER_ID)`. Der Key/Token wird genauso vertrauensvoll behandelt wie im Web-Client (dort liegt der Schlüssel unverschlüsselt in `localStorage`): lokal als Klartext in einer 0600-Datei. +## Performance bei großen Büchern + +Sehr große Bücher (mehrstellige Tausend Absätze — z.B. Sammelbände) können beim +*ersten* Öffnen mehrere Sekunden bis niedrige zweistellige Sekunden brauchen (Parsing + +Zeilenumbruch-Berechnung für die durchgehende Ansicht). Ein zweites Öffnen desselben +Buchs bei gleicher Terminalbreite ist dank Cache (`~/.cache/diora-tui/layout_cache/`) +deutlich schneller. Das Rendering selbst skaliert nicht mit der Buchgröße — nur die +tatsächlich sichtbaren Zeilen werden gezeichnet (Textual Line API), nicht das ganze Buch +auf einmal. + ## Grenzen der aktuellen Version - Nur EPUB, kein PDF — `sync` lädt PDFs im Account gar nicht erst herunter (übersprungen, wird gemeldet), da der Reader sie ohnehin nicht darstellen kann. -- Text wird pro Kapitel als Fließtext ohne Bild-/Layout-Rendering dargestellt. -- Lesefortschritt bleibt rein lokal — `sync` holt nur Bücher, keinen Fortschritt. Der - Web-Reader verankert Position als `"blockIndex:innerFraction"` in seiner eigenen, über - das ganze Buch laufenden Absatz-Nummerierung; diese TUI zählt Position dagegen pro - Kapitel. Ohne eine echte Übersetzung zwischen beiden Schemata würde ein naiver Abgleich - falsche Positionen liefern — deshalb bewusst (noch) nicht gebaut. +- Text wird als Fließtext ohne Bild-/Layout-Rendering dargestellt. +- `innerFraction` (die Position *innerhalb* eines Blocks) ist eine Terminal-Näherung + (zeilenbasiert statt pixelbasiert wie im Browser) — für die Fortschritts-Sortierung + zählt primär `blockIndex`, der exakt mit dem Web-Reader übereinstimmt; `innerFraction` + ist nur ein Tiebreaker innerhalb desselben Blocks. diff --git a/tui/diora_tui/api.py b/tui/diora_tui/api.py index daa03d0..09890c7 100644 --- a/tui/diora_tui/api.py +++ b/tui/diora_tui/api.py @@ -32,3 +32,17 @@ def fetch_book_data(server_url: str, token: str, book_id: int) -> dict: raise ApiError(f"Buch {book_id} nicht gefunden (falscher Owner oder gelöscht?)") resp.raise_for_status() return resp.json() + + +def post_progress( + server_url: str, token: str, book_id: int, *, scroll_fraction: float, position_anchor: str, force: bool = False +) -> None: + body = {"scroll_fraction": scroll_fraction, "position_anchor": position_anchor, "force": force} + resp = requests.post( + f"{server_url}/books/{book_id}/progress/", headers=_headers(token), json=body, timeout=30 + ) + if resp.status_code == 401: + raise ApiError("Authentifizierung fehlgeschlagen — Token falsch oder abgelaufen?") + if resp.status_code == 404: + raise ApiError(f"Buch {book_id} nicht gefunden (falscher Owner oder gelöscht?)") + resp.raise_for_status() diff --git a/tui/diora_tui/app.py b/tui/diora_tui/app.py index 5277446..7b260ad 100644 --- a/tui/diora_tui/app.py +++ b/tui/diora_tui/app.py @@ -9,12 +9,12 @@ from pathlib import Path from textual.app import App, ComposeResult from textual.binding import Binding -from textual.containers import VerticalScroll from textual.screen import Screen -from textual.widgets import Footer, Header, Label, ListItem, ListView, Static +from textual.widgets import Footer, Header, Label, ListItem, ListView from . import config as config_mod -from . import crypto, epub, progress, remote +from . import crypto, epub, remote +from .reader_screen import ReaderScreen DEFAULT_LIBRARY = Path.home() / "Books" @@ -57,89 +57,6 @@ class LibraryScreen(Screen): self.action_open_selected() -class ReaderScreen(Screen): - BINDINGS = [ - Binding("j,down", "scroll_down_line", "Runter", show=False), - Binding("k,up", "scroll_up_line", "Hoch", show=False), - Binding("n,]", "next_chapter", "Nächstes Kapitel"), - Binding("p,[", "prev_chapter", "Vorheriges Kapitel"), - Binding("q,escape", "back", "Zurück zur Bibliothek"), - ] - - AUTOSAVE_INTERVAL = 5.0 - - def __init__(self, path: Path) -> None: - super().__init__() - self.book = epub.load_epub(path) - self.chapter_index = 0 - saved = progress.load(self.book.id) - if saved is not None and 0 <= saved.chapter_index < len(self.book.chapters): - self.chapter_index = saved.chapter_index - self._pending_scroll_fraction = saved.scroll_fraction if saved else 0.0 - - def compose(self) -> ComposeResult: - self._text = Static(id="chapter-text") - yield Header() - with VerticalScroll(id="reader-scroll") as scroll: - self._scroll = scroll - yield self._text - yield Footer() - - def on_mount(self) -> None: - self._render_chapter(restore_scroll=True) - self.set_interval(self.AUTOSAVE_INTERVAL, self._save_progress) - - def on_unmount(self) -> None: - self._save_progress() - - def _render_chapter(self, *, restore_scroll: bool = False) -> None: - chapter = self.book.chapters[self.chapter_index] - text = "\n\n".join(chapter.paragraphs) - self._text.update(text) - self.sub_title = f"{self.book.title} — Kapitel {self.chapter_index + 1}/{len(self.book.chapters)}" - - self._scroll.scroll_home(animate=False) - if restore_scroll and self._pending_scroll_fraction: - fraction = self._pending_scroll_fraction - self.call_after_refresh(lambda: self._scroll_to_fraction(fraction)) - - def _scroll_to_fraction(self, fraction: float) -> None: - max_y = max(0, self._scroll.virtual_size.height - self._scroll.size.height) - self._scroll.scroll_to(y=round(max_y * fraction), animate=False) - - def _current_fraction(self) -> float: - max_y = max(1, self._scroll.virtual_size.height - self._scroll.size.height) - return max(0.0, min(1.0, self._scroll.scroll_y / max_y)) - - def _save_progress(self) -> None: - progress.save( - self.book.id, - progress.Position(chapter_index=self.chapter_index, scroll_fraction=self._current_fraction()), - ) - - def action_scroll_down_line(self) -> None: - self._scroll.scroll_relative(y=1, animate=False) - - def action_scroll_up_line(self) -> None: - self._scroll.scroll_relative(y=-1, animate=False) - - def action_next_chapter(self) -> None: - if self.chapter_index + 1 < len(self.book.chapters): - self.chapter_index += 1 - self._pending_scroll_fraction = 0.0 - self._render_chapter() - self._save_progress() - - def action_prev_chapter(self) -> None: - if self.chapter_index > 0: - self.chapter_index -= 1 - self._pending_scroll_fraction = 0.0 - self._render_chapter() - - def action_back(self) -> None: - self.app.pop_screen() - - class DioraTuiApp(App): CSS_PATH = "app.tcss" TITLE = "diora-tui" @@ -219,6 +136,8 @@ def run_sync(library_dir: Path, server_override: str | None, *, save: bool) -> N if result.failed: titles = ", ".join(result.failed) print(f"{len(result.failed)} Buch/Bücher konnten nicht entschlüsselt werden (falscher Key?): {titles}") + if result.progress_pulled: + print(f"Lesefortschritt für {result.progress_pulled} Buch/Bücher vom Server übernommen.") def main() -> None: diff --git a/tui/diora_tui/app.tcss b/tui/diora_tui/app.tcss index 9b0b4b4..d707750 100644 --- a/tui/diora_tui/app.tcss +++ b/tui/diora_tui/app.tcss @@ -5,8 +5,36 @@ #reader-scroll { height: 1fr; padding: 1 2; + align-horizontal: center; } -#chapter-text { - width: 100%; +#reader-loading { + height: 1fr; +} + +FootnoteScreen { + align: center middle; +} + +#footnote-body { + width: 80%; + max-width: 100; + height: auto; + max-height: 80%; + background: $panel; + border: round $primary; + padding: 1 2; +} + +.footnote-note { + margin-bottom: 1; +} + +#footnote-hint { + dock: bottom; + width: 80%; + max-width: 100; + background: $panel; + color: $text-muted; + padding: 0 2; } diff --git a/tui/diora_tui/blocks.py b/tui/diora_tui/blocks.py new file mode 100644 index 0000000..d8a3d62 --- /dev/null +++ b/tui/diora_tui/blocks.py @@ -0,0 +1,214 @@ +"""Whole-book flat block extraction, matching static/js/app.js's EPUB_BLOCK_SELECTOR +and getPositionAnchor()/restoreFromAnchor() exactly in *structure* (blockIndex is +purely DOM order, no rendering needed) so position anchors are comparable across +the web reader and this TUI. See CLAUDE.md's `tui/` section for the full picture. + +innerFraction on the web is defined via getBoundingClientRect() pixel geometry, +which has no TUI equivalent — a terminal has no font metrics/reflow the way a +browser does. We approximate it with row-based geometry from Rich's own text +wrapping (see reader.py), which is close enough because the server's "furthest +wins" comparison (_progress_is_further) only falls back to comparing fractions +when two anchors share the exact same block index — block index is the primary, +exactly-reproducible signal. + +Footnote references are detected with the same heuristic as app.js's +_looksLikeFootnoteLink: wrapped in/wrapping a , a class name containing +note/footnote/fn, or an epub:type="noteref" attribute. +""" + +from __future__ import annotations + +import re +import warnings +from dataclasses import dataclass, field +from pathlib import Path + +import ebooklib +from bs4 import BeautifulSoup, NavigableString, Tag, XMLParsedAsHTMLWarning +from ebooklib import epub + +# EPUB content documents are XHTML; treating them as HTML (matching app.js's +# `new DOMParser().parseFromString(html, 'text/html')`, static/js/app.js:2638) +# is intentional, not a mistake — silence bs4's XML-vs-HTML nudge for it. +# lxml's HTML parser is ~3x faster than bs4's built-in html.parser, which +# matters here: some real-world EPUBs run to tens of thousands of blocks. +warnings.filterwarnings("ignore", category=XMLParsedAsHTMLWarning) +_PARSER = "lxml" + +# Matches app.js's EPUB_BLOCK_SELECTOR = 'p, h1, h2, h3, h4, h5, h6, li, +# blockquote, dt, dd, figcaption, div:not(:has(*))' +_BLOCK_TAGS = {"p", "h1", "h2", "h3", "h4", "h5", "h6", "li", "blockquote", "dt", "dd", "figcaption"} +# Matches app.js's sanitizeEpubHtml() strip list (script/style are also stripped +# via regex before DOMParser even runs there; decomposing here is equivalent). +_STRIP_TAGS = ["script", "style", "iframe", "object", "embed", "head", "meta", "link"] +# Matches app.js's _looksLikeFootnoteLink's class-name check. +_FOOTNOTE_CLASS_RE = re.compile(r"\bnote|\bfootnote|\bfn\b") + +# Anchor format the server accepts (books/views.py:save_progress); anything else +# is silently discarded back to ''. +_ANCHOR_RE = re.compile(r"\d{1,7}:\d(\.\d{1,6})?") + + +@dataclass +class FootnoteRef: + marker: str # visible text of the reference link, e.g. "1" + offset: int # character offset into the block's text where the marker sits + target_id: str # fragment id to resolve against block ids + + +@dataclass +class Block: + text: str + tag: str + chapter_index: int + id: str | None = None + footnotes: list[FootnoteRef] = field(default_factory=list) + # Every id found anywhere in this block's subtree, not just on the block + # element itself — footnote *targets* are frequently an or + # similar nested a level or two inside the actual containing paragraph + # (see showFootnotePopover's `.closest('.footnote') || .parentElement` + # walk-up in app.js), so a target id resolves to "the block containing + # it" rather than requiring the id to sit on the block tag itself. + ids: list[str] = field(default_factory=list) + + +@dataclass +class FlatBook: + id: str + title: str + author: str + path: Path + blocks: list[Block] + chapter_titles: list[str] + chapter_start_block: list[int] # blocks[chapter_start_block[i]] is chapter i's first block + footnote_targets: dict[str, int] # fragment id -> block index of its content + + +def _is_leaf_div(tag: Tag) -> bool: + return tag.name == "div" and tag.find(True) is None + + +def _looks_like_footnote_link(el: Tag, href: str) -> bool: + if "#" not in href: + return False + cls = " ".join(el.get("class") or []).lower() + if _FOOTNOTE_CLASS_RE.search(cls): + return True + epub_type = (el.get("epub:type") or "").lower() + if "noteref" in epub_type: + return True + return el.find_parent("sup") is not None or el.find("sup") is not None + + +def _walk_text(el: Tag, footnotes: list[FootnoteRef], out: list[str]) -> None: + for child in el.children: + if isinstance(child, NavigableString): + out.append(str(child)) + elif isinstance(child, Tag): + if child.name == "a": + href = child.get("href") or "" + if _looks_like_footnote_link(child, href): + marker = child.get_text(" ", strip=True) + offset = len("".join(out)) + target_id = href.split("#", 1)[1] if "#" in href else "" + footnotes.append(FootnoteRef(marker=marker, offset=offset, target_id=target_id)) + out.append(marker) + continue + _walk_text(child, footnotes, out) + + +def _extract_blocks(html: bytes, chapter_index: int) -> list[Block]: + soup = BeautifulSoup(html, _PARSER) + for name in _STRIP_TAGS: + for el in soup.find_all(name): + el.decompose() + + blocks: list[Block] = [] + for el in soup.find_all(True): + if el.name in _BLOCK_TAGS or _is_leaf_div(el): + footnotes: list[FootnoteRef] = [] + parts: list[str] = [] + _walk_text(el, footnotes, parts) + text = re.sub(r"\s+", " ", "".join(parts)).strip() + ids = [tag_id for tag_id in (el.get("id"), *(d.get("id") for d in el.find_all(True))) if tag_id] + blocks.append( + Block( + text=text, + tag=el.name, + chapter_index=chapter_index, + id=el.get("id"), + footnotes=footnotes, + ids=ids, + ) + ) + return blocks + + +def load_flat_book(path: Path) -> FlatBook: + from .epub import _book_id # reuse the same path-hash id scheme + + raw = epub.read_epub(str(path), options={"ignore_ncx": True}) + + title_meta = raw.get_metadata("DC", "title") + title = title_meta[0][0] if title_meta else path.stem + author_meta = raw.get_metadata("DC", "creator") + author = author_meta[0][0] if author_meta else "Unbekannt" + + blocks: list[Block] = [] + chapter_titles: list[str] = [] + chapter_start_block: list[int] = [] + + # app.js's parseEpub() includes every spine itemref unconditionally — no + # `linear` filtering (static/js/app.js:2623-2625) — since footnote/endnote + # targets are commonly parked in a linear="no" document. Skipping it here + # would both break footnote-target resolution and shift blockIndex + # numbering out of sync with the web reader for every block after it. + for idref, _linear in raw.spine: + item = raw.get_item_with_id(idref) + if item is None or item.get_type() != ebooklib.ITEM_DOCUMENT: + continue + chapter_index = len(chapter_titles) + chapter_blocks = _extract_blocks(item.get_content(), chapter_index) + chapter_start_block.append(len(blocks)) + first_text = next((b.text for b in chapter_blocks if b.text), None) + chapter_titles.append((first_text or item.get_name())[:60]) + blocks.extend(chapter_blocks) + + footnote_targets: dict[str, int] = {} + for idx, b in enumerate(blocks): + for tag_id in b.ids: + footnote_targets.setdefault(tag_id, idx) + + return FlatBook( + id=_book_id(path), + title=title, + author=author, + path=path, + blocks=blocks, + chapter_titles=chapter_titles, + chapter_start_block=chapter_start_block, + footnote_targets=footnote_targets, + ) + + +def format_anchor(block_index: int, inner_fraction: float) -> str: + inner_fraction = max(0.0, min(1.0, inner_fraction)) + return f"{block_index}:{inner_fraction:.6f}" + + +def parse_anchor(anchor: str) -> tuple[int, float] | None: + if not anchor or not _ANCHOR_RE.fullmatch(anchor): + return None + block_str, _, frac_str = anchor.partition(":") + return int(block_str), float(frac_str) + + +def anchor_is_further(new_anchor: str, old_anchor: str) -> bool: + """Mirrors _progress_is_further / _cmpProgress (books/views.py, app.js).""" + new_parts = parse_anchor(new_anchor) + old_parts = parse_anchor(old_anchor) + if new_parts is None or old_parts is None: + return bool(new_anchor) and not old_anchor + nb, ni = new_parts + ob, oi = old_parts + return ni >= oi if nb == ob else nb >= ob diff --git a/tui/diora_tui/book_view.py b/tui/diora_tui/book_view.py new file mode 100644 index 0000000..f92a5b6 --- /dev/null +++ b/tui/diora_tui/book_view.py @@ -0,0 +1,29 @@ +"""Line-API scroll view for the continuous reader — renders only the rows +actually visible on screen (via Widget.render_line), reading from the flat +Strip list diora_tui.layout.build_layout() precomputed once. This is what +keeps very large books responsive: nothing here scales with book size at +paint time, only with viewport height. +""" + +from __future__ import annotations + +from textual.geometry import Size +from textual.scroll_view import ScrollView +from textual.strip import Strip + +from .layout import BookLayout + + +class ContinuousBookView(ScrollView): + def __init__(self, book_layout: BookLayout) -> None: + super().__init__() + self.book_layout = book_layout + self.virtual_size = Size(book_layout.width, book_layout.total_rows) + + def render_line(self, y: int) -> Strip: + _scroll_x, scroll_y = self.scroll_offset + row = scroll_y + y + strips = self.book_layout.row_strips + if row < 0 or row >= len(strips): + return Strip.blank(self.size.width, self.rich_style) + return strips[row].crop_extend(0, self.size.width, self.rich_style) diff --git a/tui/diora_tui/cache.py b/tui/diora_tui/cache.py new file mode 100644 index 0000000..21340b8 --- /dev/null +++ b/tui/diora_tui/cache.py @@ -0,0 +1,52 @@ +"""On-disk cache for the (FlatBook, BookLayout) pair a book open computes — +extraction + row-layout for very large books (tens of thousands of blocks) +can take several seconds each; reopening the same book at the same terminal +width should be near-instant instead of paying that cost again every time. + +Not a correctness-critical cache: any miss (new book, different width, edited +file) just falls back to recomputing from scratch, so a stale/corrupt cache +entry is handled by overwriting it, never by crashing the reader. +""" + +from __future__ import annotations + +import hashlib +import pickle +from pathlib import Path + +from platformdirs import user_cache_dir + +from .blocks import FlatBook +from .layout import BookLayout + +_CACHE_DIR = Path(user_cache_dir("diora-tui", "diora")) / "layout_cache" + + +def _cache_key(path: Path, width: int) -> str: + stat = path.stat() + raw = f"{path.resolve()}|{stat.st_size}|{stat.st_mtime_ns}|{width}" + return hashlib.sha256(raw.encode()).hexdigest()[:32] + + +def load(path: Path, width: int) -> tuple[FlatBook, BookLayout] | None: + cache_file = _CACHE_DIR / f"{_cache_key(path, width)}.pickle" + if not cache_file.exists(): + return None + try: + with cache_file.open("rb") as f: + book, book_layout = pickle.load(f) + if not isinstance(book, FlatBook) or not isinstance(book_layout, BookLayout): + return None + return book, book_layout + except Exception: + return None + + +def save(path: Path, width: int, book: FlatBook, book_layout: BookLayout) -> None: + _CACHE_DIR.mkdir(parents=True, exist_ok=True) + cache_file = _CACHE_DIR / f"{_cache_key(path, width)}.pickle" + try: + with cache_file.open("wb") as f: + pickle.dump((book, book_layout), f, protocol=pickle.HIGHEST_PROTOCOL) + except Exception: + pass # best-effort — a failed cache write shouldn't break reading diff --git a/tui/diora_tui/epub.py b/tui/diora_tui/epub.py index 2ca16f9..e5eaaa2 100644 --- a/tui/diora_tui/epub.py +++ b/tui/diora_tui/epub.py @@ -1,30 +1,11 @@ -"""EPUB parsing: turn a .epub file into a spine-ordered list of chapters.""" +"""Library scanning and the stable per-book id shared across progress.py, +blocks.py, and remote.py.""" from __future__ import annotations import hashlib -from dataclasses import dataclass from pathlib import Path -import ebooklib -from bs4 import BeautifulSoup -from ebooklib import epub - - -@dataclass -class Chapter: - title: str - paragraphs: list[str] - - -@dataclass -class Book: - id: str - title: str - author: str - path: Path - chapters: list[Chapter] - def _book_id(path: Path) -> str: # Identifies a book by its resolved path for now. Once the sync API exists, @@ -32,49 +13,6 @@ def _book_id(path: Path) -> str: return hashlib.sha256(str(path.resolve()).encode()).hexdigest()[:16] -def _extract_paragraphs(html: bytes) -> list[str]: - soup = BeautifulSoup(html, "html.parser") - for tag in soup(["script", "style"]): - tag.decompose() - - paragraphs = [] - for el in soup.find_all(["p", "h1", "h2", "h3", "h4", "li", "blockquote"]): - text = el.get_text(" ", strip=True) - if text: - paragraphs.append(text) - - if not paragraphs: - text = soup.get_text(" ", strip=True) - if text: - paragraphs.append(text) - - return paragraphs - - -def load_epub(path: Path) -> Book: - raw = epub.read_epub(str(path), options={"ignore_ncx": True}) - - title_meta = raw.get_metadata("DC", "title") - title = title_meta[0][0] if title_meta else path.stem - - author_meta = raw.get_metadata("DC", "creator") - author = author_meta[0][0] if author_meta else "Unbekannt" - - chapters: list[Chapter] = [] - for idref, linear in raw.spine: - if linear == "no": - continue - item = raw.get_item_with_id(idref) - if item is None or item.get_type() != ebooklib.ITEM_DOCUMENT: - continue - paragraphs = _extract_paragraphs(item.get_content()) - if not paragraphs: - continue - chapters.append(Chapter(title=paragraphs[0][:60], paragraphs=paragraphs)) - - return Book(id=_book_id(path), title=title, author=author, path=path, chapters=chapters) - - def scan_library(library_dir: Path) -> list[Path]: if not library_dir.exists(): return [] diff --git a/tui/diora_tui/layout.py b/tui/diora_tui/layout.py new file mode 100644 index 0000000..e7700f4 --- /dev/null +++ b/tui/diora_tui/layout.py @@ -0,0 +1,90 @@ +"""Row-based layout for the continuous reader view — the TUI's terminal-grid +analogue of the browser's pixel-based getBoundingClientRect() geometry (see +blocks.py's module docstring for why blockIndex is exact but innerFraction is +only an approximation here). + +Each block is wrapped independently at a known width, once, into a flat list +of pre-rendered Strip objects (one per terminal row) that book_view.py's +Line-API widget indexes directly in render_line() — this is what makes very +large books (tens of thousands of blocks) open in seconds rather than +minutes: Textual only ever renders the rows actually on screen, instead of +laying out the whole book up front the way a single giant Static would. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from rich.console import Console +from rich.text import Text +from textual.strip import Strip + +_SEPARATOR_ROWS = 1 # one blank row between blocks, matching the old "\n\n".join style + + +@dataclass +class BookLayout: + starts: list[int] # row where block i begins + heights: list[int] # rendered row count for block i + row_strips: list[Strip] # one Strip per absolute row, len == total_rows + total_rows: int + width: int + + +def build_layout(block_texts: list[str], width: int, console: Console) -> BookLayout: + width = max(1, width) + starts: list[int] = [] + heights: list[int] = [] + row_strips: list[Strip] = [] + blank = Strip.blank(width) + row = 0 + for text in block_texts: + starts.append(row) + wrapped_lines = list(Text(text).wrap(console, width)) if text else [] + if not wrapped_lines: + wrapped_lines = [Text("")] + heights.append(len(wrapped_lines)) + for line in wrapped_lines: + strip = Strip(line.render(console), None).adjust_cell_length(width) + row_strips.append(strip) + row_strips.append(blank) + row += len(wrapped_lines) + _SEPARATOR_ROWS + + return BookLayout(starts=starts, heights=heights, row_strips=row_strips, total_rows=row, width=width) + + +def get_position_anchor(layout: BookLayout, scroll_y: int) -> tuple[int, float]: + """Mirrors app.js's getPositionAnchor(): the last block whose top row is at + or above scroll_y, or the first block if none has scrolled that far yet.""" + n = len(layout.heights) + if n == 0: + return 0, 0.0 + + best_index = 0 + found = False + for i in range(n): + if layout.heights[i] < 1: + continue + if layout.starts[i] > scroll_y: + break + best_index = i + found = True + + if not found: + best_index = next((i for i in range(n) if layout.heights[i] >= 1), 0) + + top = layout.starts[best_index] + height = max(1, layout.heights[best_index]) + inner_fraction = max(0.0, min(1.0, (scroll_y - top) / height)) + return best_index, inner_fraction + + +def scroll_y_for_anchor(layout: BookLayout, block_index: int, inner_fraction: float) -> int: + """Mirrors app.js's restoreFromAnchor().""" + n = len(layout.heights) + if n == 0: + return 0 + idx = max(0, min(block_index, n - 1)) + top = layout.starts[idx] + height = layout.heights[idx] + return top + round(inner_fraction * height) diff --git a/tui/diora_tui/progress.py b/tui/diora_tui/progress.py index ba10fcf..588e24b 100644 --- a/tui/diora_tui/progress.py +++ b/tui/diora_tui/progress.py @@ -1,28 +1,33 @@ """Local reading-progress storage. -Mirrors diora's web-reader "furthest wins" semantics (see books/models.py, -EBookProgress.save_progress): a saved position only ever advances, so an -older or offline run can't regress a further-along read position. The -position is kept anchor-shaped (chapter index + fraction within it) so a -future sync layer against diora's API can adopt it without a rewrite. +Positions are stored as the same "blockIndex:innerFraction" anchor string the +server uses (books/models.py, EBookProgress.save_progress) — see +diora_tui/blocks.py for how blockIndex is computed to match static/js/app.js +exactly, and anchor_is_further() for the identical "furthest wins" comparison +used server-side (_progress_is_further in books/views.py). A saved position +only ever advances (unless forced), so an older/offline run can't regress a +further-along read position — matching that same rule locally. """ from __future__ import annotations import json -from dataclasses import asdict, dataclass +import time +from dataclasses import dataclass from pathlib import Path from platformdirs import user_data_dir +from .blocks import anchor_is_further + _DATA_DIR = Path(user_data_dir("diora-tui", "diora")) _PROGRESS_FILE = _DATA_DIR / "progress.json" @dataclass class Position: - chapter_index: int - scroll_fraction: float # 0..1 within the chapter's paragraph list + anchor: str # "blockIndex:innerFraction", e.g. "42:0.500000" + updated_at: float = 0.0 def _load_all() -> dict[str, dict]: @@ -41,23 +46,23 @@ def _save_all(data: dict[str, dict]) -> None: def load(book_id: str) -> Position | None: entry = _load_all().get(book_id) - if entry is None: + if not isinstance(entry, dict) or not entry.get("anchor"): return None - return Position(chapter_index=entry["chapter_index"], scroll_fraction=entry["scroll_fraction"]) + return Position(anchor=entry["anchor"], updated_at=entry.get("updated_at", 0.0)) -def _is_further(new: Position, old: Position) -> bool: - if new.chapter_index != old.chapter_index: - return new.chapter_index > old.chapter_index - return new.scroll_fraction > old.scroll_fraction - - -def save(book_id: str, position: Position, *, force: bool = False) -> None: +def save(book_id: str, position: Position, *, force: bool = False) -> bool: + """Returns True if the position was actually written (i.e. it was further + along, or forced) — callers that push to the server only need to do so + when this returns True.""" data = _load_all() existing = data.get(book_id) if existing is not None and not force: - old = Position(chapter_index=existing["chapter_index"], scroll_fraction=existing["scroll_fraction"]) - if not _is_further(position, old): - return - data[book_id] = asdict(position) + old_anchor = existing.get("anchor", "") + if not anchor_is_further(position.anchor, old_anchor): + return False + if not position.updated_at: + position.updated_at = time.time() + data[book_id] = {"anchor": position.anchor, "updated_at": position.updated_at} _save_all(data) + return True diff --git a/tui/diora_tui/reader_screen.py b/tui/diora_tui/reader_screen.py new file mode 100644 index 0000000..9dbd17f --- /dev/null +++ b/tui/diora_tui/reader_screen.py @@ -0,0 +1,257 @@ +"""Continuous whole-book reader screen — mirrors the web reader's single +scrollable view (see blocks.py's module docstring) instead of the old +per-chapter pagination, so reading position is expressed in the same +"blockIndex:innerFraction" anchor space as the server and the web client. + +Uses book_view.ContinuousBookView (a Line-API widget) rather than dumping the +whole book into one Static: for large books (tens of thousands of blocks) a +single giant renderable made Textual's layout/paint pass take upwards of a +minute, whereas the Line API only ever renders the rows on screen. +""" + +from __future__ import annotations + +import bisect +import re +from pathlib import Path + +from rich.console import Console +from textual import work +from textual.binding import Binding +from textual.containers import Container, VerticalScroll +from textual.geometry import Size +from textual.screen import ModalScreen, Screen +from textual.widgets import Footer, Header, Label, LoadingIndicator, Static + +from . import api, blocks, cache, config as config_mod, layout, progress +from .book_view import ContinuousBookView + +MAX_LINE_WIDTH = 120 +AUTOSAVE_INTERVAL = 5.0 + +_EMPTY_LAYOUT = layout.BookLayout(starts=[], heights=[], row_strips=[], total_rows=0, width=1) + +# Downloaded-via-sync files are named " - .epub" (see +# remote.py) — reused here to find the server book id for progress push. +_SERVER_ID_RE = re.compile(r"^(\d{4,7}) - ") + + +def _server_book_id(path: Path) -> int | None: + m = _SERVER_ID_RE.match(path.name) + return int(m.group(1)) if m else None + + +class FootnoteScreen(ModalScreen[None]): + BINDINGS = [Binding("escape,f,q", "dismiss_self", "Schließen")] + + def __init__(self, notes: list[str]) -> None: + super().__init__() + self._notes = notes + + def compose(self): + with VerticalScroll(id="footnote-body"): + for note in self._notes: + yield Static(note, classes="footnote-note") + yield Label("Escape/f zum Schließen", id="footnote-hint") + + def action_dismiss_self(self) -> None: + self.dismiss(None) + + +class ReaderScreen(Screen): + BINDINGS = [ + Binding("j,down", "scroll_down_line", "Runter", show=False), + Binding("k,up", "scroll_up_line", "Hoch", show=False), + Binding("n", "next_chapter", "Nächstes Kapitel"), + Binding("p", "prev_chapter", "Vorheriges Kapitel"), + Binding("f", "peek_footnote", "Fußnote"), + Binding("q,escape", "back", "Zurück zur Bibliothek"), + ] + + def __init__(self, path: Path) -> None: + super().__init__() + self.path = path + self.book: blocks.FlatBook | None = None + self.book_layout: layout.BookLayout = _EMPTY_LAYOUT + self._server_id = _server_book_id(path) + self._remote_cfg = config_mod.load() if self._server_id is not None else None + self._last_pushed_anchor = "" + + def compose(self): + yield Header() + yield LoadingIndicator(id="reader-loading") + with Container(id="reader-scroll"): + # Stays visible (empty) from the start rather than toggling display + # on once loaded — a widget that's just been switched from hidden + # to visible hasn't been through a layout pass yet, so its `.size` + # is still (0, 0) and an immediate scroll_to() right after has + # nothing to clamp against and silently resets to 0. + self._book_view = ContinuousBookView(_EMPTY_LAYOUT) + yield self._book_view + yield Footer() + + def on_mount(self) -> None: + self._load_book(self._content_width()) + + @work(thread=True) + def _load_book(self, width: int) -> None: + cached = cache.load(self.path, width) + if cached is not None: + book, book_layout = cached + else: + book = blocks.load_flat_book(self.path) + console = Console(width=width) + book_layout = layout.build_layout([b.text for b in book.blocks], width, console) + cache.save(self.path, width, book, book_layout) + self.app.call_from_thread(self._on_book_loaded, book, book_layout, width) + + def _on_book_loaded(self, book: blocks.FlatBook, book_layout: layout.BookLayout, width: int) -> None: + self.book = book + self.sub_title = book.title + self._apply_layout(book_layout, width) + self.query_one("#reader-loading", LoadingIndicator).display = False + self.call_after_refresh(self._restore_position) + self.set_interval(AUTOSAVE_INTERVAL, self._autosave) + + def _content_width(self) -> int: + return max(20, min(MAX_LINE_WIDTH, self.size.width - 4)) + + def _apply_layout(self, book_layout: layout.BookLayout, width: int) -> None: + self.book_layout = book_layout + self._book_view.styles.width = width + self._book_view.book_layout = book_layout + self._book_view.virtual_size = Size(width, book_layout.total_rows) + self._book_view.refresh() + + def _build_layout(self) -> None: + assert self.book is not None + width = self._content_width() + console = Console(width=width) + block_texts = [b.text for b in self.book.blocks] + new_layout = layout.build_layout(block_texts, width, console) + cache.save(self.path, width, self.book, new_layout) + self._apply_layout(new_layout, width) + + def on_resize(self) -> None: + if self.book is None: + return + old_y = self._book_view.scroll_y + anchor_block, anchor_frac = layout.get_position_anchor(self.book_layout, old_y) + self._build_layout() + new_y = layout.scroll_y_for_anchor(self.book_layout, anchor_block, anchor_frac) + self._book_view.scroll_to(y=new_y, animate=False, immediate=True) + + def _restore_position(self, attempt: int = 0) -> None: + if self.book is None: + return + saved = progress.load(self.book.id) + if saved is None: + return + parsed = blocks.parse_anchor(saved.anchor) + if parsed is None: + return + block_index, inner_fraction = parsed + y = layout.scroll_y_for_anchor(self.book_layout, block_index, inner_fraction) + if y <= 0: + return + self._book_view.scroll_to(y=y, animate=False, immediate=True) + # A widget that's only just become part of the layout doesn't always + # honor an immediate scroll on the first attempt (its own size/scroll + # bounds can still be mid-update) — verify it actually landed and + # retry a bounded number of times rather than guessing a fixed delay. + if attempt < 20 and abs(self._book_view.scroll_y - y) > 1: + self.call_after_refresh(lambda: self._restore_position(attempt + 1)) + + def _current_anchor_str(self) -> str: + block_index, inner_fraction = layout.get_position_anchor(self.book_layout, self._book_view.scroll_y) + return blocks.format_anchor(block_index, inner_fraction) + + def _autosave(self) -> None: + self._save_progress() + + def _save_progress(self) -> None: + if self.book is None or not self.book_layout.heights: + return + anchor = self._current_anchor_str() + advanced = progress.save(self.book.id, progress.Position(anchor=anchor)) + if advanced and self._remote_cfg is not None and self._server_id is not None: + self._push_remote_progress(anchor) + + @work(thread=True, exclusive=True, group="progress-push") + def _push_remote_progress(self, anchor: str) -> None: + if anchor == self._last_pushed_anchor or self._remote_cfg is None or self._server_id is None: + return + try: + api.post_progress( + self._remote_cfg.server_url, + self._remote_cfg.api_token, + self._server_id, + scroll_fraction=0.0, + position_anchor=anchor, + force=False, + ) + self._last_pushed_anchor = anchor + except Exception: + pass # best-effort — local progress is already saved regardless + + def on_unmount(self) -> None: + self._save_progress() + + def action_scroll_down_line(self) -> None: + self._book_view.scroll_relative(y=1, animate=False) + + def action_scroll_up_line(self) -> None: + self._book_view.scroll_relative(y=-1, animate=False) + + def _current_chapter_index(self) -> int: + if self.book is None: + return 0 + block_index, _ = layout.get_position_anchor(self.book_layout, self._book_view.scroll_y) + return bisect.bisect_right(self.book.chapter_start_block, block_index) - 1 + + def _jump_to_block(self, block_index: int) -> None: + y = layout.scroll_y_for_anchor(self.book_layout, block_index, 0.0) + self._book_view.scroll_to(y=y, animate=False, immediate=True) + + def action_next_chapter(self) -> None: + if self.book is None: + return + chapter = self._current_chapter_index() + if chapter + 1 < len(self.book.chapter_start_block): + self._jump_to_block(self.book.chapter_start_block[chapter + 1]) + self._save_progress() + + def action_prev_chapter(self) -> None: + if self.book is None: + return + chapter = self._current_chapter_index() + if chapter > 0: + self._jump_to_block(self.book.chapter_start_block[chapter - 1]) + self._save_progress() + + def action_peek_footnote(self) -> None: + if self.book is None or not self.book_layout.heights: + return + current_block, _ = layout.get_position_anchor(self.book_layout, self._book_view.scroll_y) + + forward = [(i, b) for i, b in enumerate(self.book.blocks) if i >= current_block and b.footnotes] + backward = [(i, b) for i, b in enumerate(self.book.blocks) if i < current_block and b.footnotes] + candidate = forward[0] if forward else (backward[-1] if backward else None) + if candidate is None: + self.notify("Keine Fußnote in diesem Buch gefunden.", timeout=3) + return + + _, block = candidate + notes: list[str] = [] + for ref in block.footnotes: + target_idx = self.book.footnote_targets.get(ref.target_id) + if target_idx is None: + continue + notes.append(self.book.blocks[target_idx].text) + if not notes: + self.notify("Fußnote konnte nicht aufgelöst werden.", timeout=3) + return + self.app.push_screen(FootnoteScreen(notes)) + + def action_back(self) -> None: + self.app.pop_screen() diff --git a/tui/diora_tui/remote.py b/tui/diora_tui/remote.py index c6ba299..c4705e8 100644 --- a/tui/diora_tui/remote.py +++ b/tui/diora_tui/remote.py @@ -1,12 +1,12 @@ -"""Fetch + decrypt books from a diora server into the local TUI library. +"""Fetch + decrypt books from a diora server into the local TUI library, and +pull reading progress for them into the local progress store. -Reading progress is intentionally NOT synced yet: the server tracks EPUB -position as a "blockIndex:innerFraction" anchor into the *web* reader's flat -paragraph numbering (see books/models.py, EBookProgress), which doesn't line -up with this TUI's per-chapter scroll_fraction (diora_tui/progress.py) -without an actual translation layer between the two block-numbering -schemes. Downloaded books simply start fresh in the local progress store; -wiring up that translation is future work. +Progress is pull-only here (server -> local); the reader pushes local -> +server itself while a book is open (see reader_screen.py). Both directions +use the same "blockIndex:innerFraction" anchor format as the web reader +(diora_tui/blocks.py) and the same furthest-wins merge rule +(diora_tui/progress.py, books/views.py's _progress_is_further) — a pull can +only ever advance local progress, never regress it, unless forced. """ from __future__ import annotations @@ -20,6 +20,8 @@ import requests from . import api, crypto from .config import RemoteConfig +from .epub import _book_id +from .progress import Position, save as save_progress class SyncError(Exception): @@ -32,6 +34,7 @@ class SyncResult: unchanged: int = 0 skipped_pdf: list[str] = field(default_factory=list) failed: list[str] = field(default_factory=list) + progress_pulled: int = 0 def _sanitize_filename(name: str) -> str: @@ -53,6 +56,7 @@ def sync_library(library_dir: Path, cfg: RemoteConfig) -> SyncResult: library_dir.mkdir(parents=True, exist_ok=True) result = SyncResult() + local_path_by_server_id: dict[int, Path] = {} for book in snapshot.get("books", []): book_id = book["id"] @@ -69,6 +73,7 @@ def sync_library(library_dir: Path, cfg: RemoteConfig) -> SyncResult: existing = next(library_dir.glob(f"{book_id:04d} - *.epub"), None) if existing is not None: result.unchanged += 1 + local_path_by_server_id[book_id] = existing continue try: @@ -82,5 +87,17 @@ def sync_library(library_dir: Path, cfg: RemoteConfig) -> SyncResult: dest = library_dir / f"{book_id:04d} - {_sanitize_filename(title)}.epub" dest.write_bytes(raw) result.downloaded.append(dest.name) + local_path_by_server_id[book_id] = dest + + for entry in snapshot.get("book_progress", []): + anchor = entry.get("position_anchor") or "" + if not anchor: + continue # PDF-only scroll_fraction progress — this TUI is EPUB-only + dest = local_path_by_server_id.get(entry.get("book_id")) + if dest is None: + continue + local_id = _book_id(dest) + if save_progress(local_id, Position(anchor=anchor)): + result.progress_pulled += 1 return result diff --git a/tui/pyproject.toml b/tui/pyproject.toml index 3fa7cf1..80d5a24 100644 --- a/tui/pyproject.toml +++ b/tui/pyproject.toml @@ -1,12 +1,13 @@ [project] name = "diora-tui" version = "0.1.0" -description = "Lokaler EPUB-Reader als Vorstufe eines diora-TUI-Clients; synct später Fortschritt über die diora-API." +description = "Terminal-EPUB-Reader für diora mit Server-Sync für Bücher und Lesefortschritt." requires-python = ">=3.9" dependencies = [ "textual>=0.60", "ebooklib>=0.18", "beautifulsoup4>=4.12", + "lxml>=5.0", "platformdirs>=4.0", "requests>=2.31", "cryptography>=42.0",