Projekte mit Preset Forschung: Quellen (Paper per DOI, Bücher, Folgen mit Zeitmarke, Artikel), Zitierschlüssel, BibTeX/CSL-Export, Markdown-Notizen mit [@key]; Ende-zu-Ende-verschlüsselt. Navigation: Projekte als Tab am Handy, Suche als Lupe; sw v141

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

diora/settings.py

@@ -32,6 +32,7 @@ INSTALLED_APPS = [
     'gpodder',
     'tts',
     'feeds',
+    'projects',
 ]
 
 EBOOK_MAX_BYTES = 100 * 1024 * 1024  # 100 MB (big scanned PDFs, complete editions)

diora/urls.py

@@ -13,6 +13,7 @@ urlpatterns = [
     path('podcasts/', include('podcasts.urls')),
     path('books/', include('books.urls')),
     path('feeds/', include('feeds.urls')),
+    path('projects/', include('projects.urls')),
     path('tts/', include('tts.urls')),
     path('api/2/', include('gpodder.urls')),
     path('api/sync/', sync_snapshot, name='api_sync'),

docs/projects.md

@@ -0,0 +1,70 @@
+# Projekte
+
+Projects of a preset: the first is **Forschung** (a research project: sources, citations,
+notes; Radar and Schreiben follow). In the navigation (`nav.js`, section and group `projekte`,
+flag `side`): a tab beside Home · Lesen · Hören on a phone, at the foot of the sidebar and of
+the wide bar above Suche · Abos & Quellen. The search moved out of the phone's tab bar: a
+magnifier in the bar on top (`suche` is a footer section; the apps' section bars show it).
+
+## End-to-end encrypted
+
+Like the books (docs/library.md): a project's data (`{name, description}`) and every item's
+data are AES-GCM blobs with the user's key (`32-crypto.js`, iv hex, ciphertext base64). The
+server stores them and knows only: a project exists, its preset, archived, and what an item
+points to — `kind` + `ref` (a book's, episode's or article's id, never its title).
+
+## Items
+
+| kind | ref | data |
+|---|---|---|
+| `paper` | – | `{key, csl, note, doi}` (CSL-JSON from doi.org) |
+| `book` | book id | `{key, csl, note, type}` (CSL from the book's meta) |
+| `episode` | episode id | `{key, csl, note, podcast, feed, position}` (position: seconds, "ab 12:34") |
+| `article` | feeds item id | `{key, csl, note}` |
+| `note` | – | `{title, body (Markdown), folder}` |
+
+A book, episode or article is in a project once (adding again answers `existing: true`).
+`key` is the citation key for `[@key]` / `[@key, S. 12]` in notes: first author's family name
+(or the podcast's), year, first word of the title — unique in the project, editable.
+
+Notes: Markdown (headings, lists, quotes, bold/italic, links); the view shows citations as
+"(Amodio & Jost 2007, S. 3)" and a list of the works cited. Saved as one types (0.9 s).
+Export: BibTeX and CSL-JSON of the sources (all or the chosen kind).
+
+## Linking from elsewhere
+
+`DioraProjects.link(kind, ref, info)` opens the sheet "Mit Projekt verknüpfen" (the projects,
+the ones it is in ticked, ＋ Neues Projekt). Called from the book sheet (library, apps), the
+article reader (◇ in its bar), the episode sheet and the player (apps, native-podcast.js).
+`DioraProjects.of(kind, ref)` → `[{id, name}]` the projects it's in; `onChange(fn)` after changes.
+
+## Desktop: a window of its own
+
+The project's menu (⋯) has "In neuem Fenster öffnen" when `dioraDesktop.openWindow` is there:
+the desktop opens the page with `?project=<id>`; `projects.js` then shows only that project
+(`html.prj-solo` hides the navigation and the dock).
+
+## Writing and conflicts
+
+Every write sends `base_iv` (the IV it changed). Changed elsewhere since: 409 with the current
+state. A note's text then goes beside it as a copy "(Konflikt)"; other data reloads.
+
+## API (signed in)
+
+- `GET /projects/` → `{projects: [{id, preset, data_ct, data_iv, archived, position, items, …}], links: [{project, kind, ref}], presets}`
+- `POST /projects/create/` `{preset, data_ct, data_iv}` → `{project}`
+- `GET /projects/<id>/` → `{project, items: [{id, kind, ref, data_ct, data_iv, created_at, updated_at}]}`
+- `POST /projects/<id>/` `{data_ct, data_iv, base_iv}` and/or `{archived}` / `{position}`
+- `POST /projects/<id>/delete/`
+- `POST /projects/<id>/items/add/` `{kind, ref?, data_ct, data_iv}` → `{item, existing}`
+- `POST /projects/items/<id>/` `{data_ct, data_iv, base_iv}` → `{item}` (409 `{item}` when stale)
+- `POST /projects/items/<id>/delete/`
+- `GET /projects/doi/?q=<DOI | doi.org link | arXiv link or id>` → `{doi, csl}` — fetched from
+  doi.org (Crossref, DataCite …) as CSL-JSON, cached a week, nothing else kept.
+
+## Next
+
+Radar (the weekly digests and new papers with lesen / später / verwerfen / → Quellen; who
+fetches the papers is open), papers as books (a paper's PDF into the library, its highlights
+into the project's notes), Schreiben (drafts with the bibliography in a citation style),
+the projects in the search, more presets.

docs/search.md

@@ -6,7 +6,7 @@ section and group `suche`, flag `side`):
 - phone (the apps' tab bar, the web page's narrow bar): beside Home · Lesen · Hören;
 - the apps' sidebar and the web page's wide bar: at the foot, above Abos & Quellen.
 
-It's never the section the app opens in next time (`diora_nav_section` isn't written for it).
+It's never the section the app opens in next time (`transient`: `diora_nav_section` isn't written for it).
 
 ## What is searched where
 

e2e/navigation.spec.js

@@ -14,7 +14,7 @@ test('wide: every section in the bar under its group, each shows its panel and v
   const wide = page.locator('#tabs .nav-wide');
   await expect(wide.locator('.nav-group-label')).toHaveText(['Lesen', 'Hören']);
   await expect(wide.locator('[data-nav]')).toHaveText(
-    ['Home', 'Bücher', 'Zeitschriften', 'Nachrichten', 'Blogs', 'Hörbücher', 'Podcasts', 'Radio', 'Ambient', 'Fokus', 'Suche', 'Abos & Quellen']);
+    ['Home', 'Bücher', 'Zeitschriften', 'Nachrichten', 'Blogs', 'Hörbücher', 'Podcasts', 'Radio', 'Ambient', 'Fokus', 'Projekte', 'Suche', 'Abos & Quellen']);
 
   await wide.locator('[data-nav="blogs"]').click();
   await expect(page).toHaveURL(/#blogs$/);
@@ -33,7 +33,8 @@ test('narrow: three groups, the sections as a row on top, each group remembers i
   await page.setViewportSize({ width: 390, height: 800 });
   await page.goto('/#news');
   const narrow = page.locator('#tabs .nav-narrow');
-  await expect(narrow.locator('[data-nav]')).toHaveText(['Home', 'Lesen', 'Hören', 'Suche', '']);
+  await expect(narrow.locator('[data-nav]')).toHaveText(['Home', 'Lesen', 'Hören', 'Projekte', '', '']);
+  await expect(narrow.locator('[data-nav="suche"] svg.dnav-ic')).toBeVisible();   // the magnifier
   await expect(narrow.locator('[data-nav="sources"] svg.dnav-ic')).toBeVisible();
   await expect(narrow.locator('[data-nav="lesen"]')).toHaveClass(/active/);
   const segs = page.locator('#nav-segments');
@@ -89,6 +90,52 @@ test('Suche: books from this device, the rest from the server, a hit opens where
   expect(await page.evaluate(() => localStorage.getItem('diora_nav_section'))).not.toBe('suche');
 });
 
+test('Projekte: a Forschung project, a paper by DOI, a linked book, a note citing them', async ({ page }) => {
+  await page.setViewportSize({ width: 1200, height: 900 });
+  await page.route('**/projects/doi/?q=*', route => route.fulfill({json: {doi: '10.1038/nn.1234', csl: {
+    type: 'article-journal', title: 'Neural correlates of political ideology', 'container-title': 'Nature Neuroscience',
+    author: [{family: 'Amodio', given: 'David'}, {family: 'Jost', given: 'John'}], issued: {'date-parts': [[2007, 9]]},
+    DOI: '10.1038/nn1979', volume: '10', page: '1246-1247'}}}));
+  await page.goto('/#projekte');
+  const tab = page.locator('#tab-projekte');
+  await tab.locator('[data-act="new"]').click();
+  await expect(tab.locator('.prj-preset.on')).toContainText('Forschung');
+  await tab.locator('input[name="name"]').fill('Neuropolitik');
+  await tab.locator('button[type="submit"]').click();
+  await expect(tab.locator('.prj-h1')).toHaveText('Neuropolitik');
+  await expect(page).toHaveURL(/#projekte\/\d+$/);
+  // The server keeps only ciphertext.
+  const raw = await page.evaluate(() => fetch('/projects/').then(r => r.json()));
+  expect(JSON.stringify(raw)).not.toContain('Neuropolitik');
+
+  await tab.locator('.prj-add input').fill('https://doi.org/10.1038/nn1979');
+  await tab.locator('.prj-add button').click();
+  await tab.locator('[data-add-doi]').click();
+  await expect(tab.locator('.prj-row .prj-title')).toHaveText(['Neural correlates of political ideology']);
+  await expect(tab.locator('.prj-key')).toHaveText(['amodio2007neural']);
+
+  // From a book's sheet elsewhere: DioraProjects.link.
+  await page.evaluate(() => DioraProjects.link('book', 4242, {title: 'Das politische Gehirn', author: 'Hans Muster'}));
+  await page.locator('.prj-picker [data-pick]').filter({hasText: 'Neuropolitik'}).click();
+  await expect(tab.locator('.prj-row')).toHaveCount(2);
+  expect(await page.evaluate(() => DioraProjects.of('book', 4242).map(p => p.name))).toEqual(['Neuropolitik']);
+
+  await tab.locator('[data-view="notes"]').click();
+  await tab.locator('[data-act="new-note"]').click();
+  await tab.locator('[data-note-title]').fill('These');
+  await tab.locator('[data-note-body]').fill('# Idee\nIdeologie zeigt sich im Gehirn [@amodio2007neural, S. 3] und bei [@muster0jdas].');
+  await expect(tab.locator('.prj-saved')).toHaveText('gespeichert', {timeout: 5000});
+  await tab.locator('[data-act="preview"]').click();
+  await expect(tab.locator('.prj-md .prj-cite').first()).toHaveText('(Amodio & Jost 2007, S. 3)');
+  await expect(tab.locator('.prj-refs li').first()).toContainText('Nature Neuroscience');
+
+  await tab.locator('[data-view="sources"]').click();
+  const [dl] = await Promise.all([page.waitForEvent('download'), tab.locator('[data-export="bib"]').click()]);
+  const bib = require('fs').readFileSync(await dl.path(), 'utf8');
+  expect(bib).toContain('@article{amodio2007neural,');
+  expect(bib).toContain('doi = {10.1038/nn1979}');
+});
+
 test('Abos & Quellen: one page for every subscription, reached from the feeds too', async ({ page }) => {
   await page.setViewportSize({ width: 1200, height: 800 });
   await page.goto('/#news');

projects/apps.py

@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class ProjectsConfig(AppConfig):
+    default_auto_field = 'django.db.models.BigAutoField'
+    name = 'projects'

projects/migrations/0001_initial.py

@@ -0,0 +1,52 @@
+# Generated by Django 6.1 on 2026-10-19 18:00
+
+import django.db.models.deletion
+from django.conf import settings
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Project',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('preset', models.CharField(choices=[('research', 'Forschung')], default='research', max_length=20)),
+                ('data_ct', models.TextField()),
+                ('data_iv', models.CharField(max_length=32)),
+                ('archived', models.BooleanField(default=False)),
+                ('position', models.IntegerField(default=0)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+                ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='projects', to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'ordering': ['position', '-updated_at'],
+            },
+        ),
+        migrations.CreateModel(
+            name='ProjectItem',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('kind', models.CharField(choices=[('book', 'Buch'), ('episode', 'Folge'), ('article', 'Artikel'), ('paper', 'Paper'), ('note', 'Notiz')], max_length=10)),
+                ('ref', models.BigIntegerField(blank=True, null=True)),
+                ('data_ct', models.TextField()),
+                ('data_iv', models.CharField(max_length=32)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+                ('project', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='items', to='projects.project')),
+            ],
+            options={
+                'ordering': ['created_at'],
+                'indexes': [models.Index(fields=['kind', 'ref'], name='projects_pr_kind_3ad4df_idx')],
+                'constraints': [models.UniqueConstraint(condition=models.Q(('ref__isnull', False)), fields=('project', 'kind', 'ref'), name='project_item_once')],
+            },
+        ),
+    ]

projects/models.py

@@ -0,0 +1,52 @@
+"""Projekte (docs/projects.md): a project of a preset (Forschung first), its sources and notes.
+
+End-to-end encrypted like the books: a project's name and settings, and every item's content
+(a note's text, a source's citation data, a remark) are AES-GCM blobs the server can't read.
+What it knows: that a project exists, its preset, and which book, episode or article an item
+points to (kind + ref) — the ids, never their titles."""
+from django.contrib.auth.models import User
+from django.db import models
+
+
+class Project(models.Model):
+    RESEARCH = 'research'
+    PRESETS = [(RESEARCH, 'Forschung')]
+
+    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name='projects')
+    preset = models.CharField(max_length=20, choices=PRESETS, default=RESEARCH)
+    data_ct = models.TextField()   # {name, description, …} encrypted
+    data_iv = models.CharField(max_length=32)
+    archived = models.BooleanField(default=False)
+    position = models.IntegerField(default=0)
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ['position', '-updated_at']
+
+
+class ProjectItem(models.Model):
+    """Something in a project. Sources point at what's elsewhere in diora (a book, an episode,
+    an article: ref is its id) or stand alone (a paper known by its citation data: no ref);
+    notes are text of their own (no ref)."""
+    BOOK = 'book'
+    EPISODE = 'episode'
+    ARTICLE = 'article'
+    PAPER = 'paper'
+    NOTE = 'note'
+    KINDS = [(BOOK, 'Buch'), (EPISODE, 'Folge'), (ARTICLE, 'Artikel'), (PAPER, 'Paper'), (NOTE, 'Notiz')]
+    LINKED = (BOOK, EPISODE, ARTICLE)
+
+    project = models.ForeignKey(Project, on_delete=models.CASCADE, related_name='items')
+    kind = models.CharField(max_length=10, choices=KINDS)
+    ref = models.BigIntegerField(null=True, blank=True)
+    data_ct = models.TextField()
+    data_iv = models.CharField(max_length=32)
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ['created_at']
+        constraints = [models.UniqueConstraint(fields=['project', 'kind', 'ref'], name='project_item_once',
+                                               condition=models.Q(ref__isnull=False))]
+        indexes = [models.Index(fields=['kind', 'ref'])]

projects/tests.py

@@ -0,0 +1,109 @@
+"""Projekte: the API (projects/views.py)."""
+import json
+from unittest.mock import patch
+
+from django.contrib.auth.models import User
+from django.core.cache import cache
+from django.test import TestCase
+
+from .models import Project, ProjectItem
+from .views import doi_of
+
+IV = 'a1' * 12
+
+
+class Base(TestCase):
+    def setUp(self):
+        cache.clear()
+        self.user = User.objects.create_user(username='forscher', password='pw12345678')
+        self.client.force_login(self.user)
+
+    def post(self, url, body=None):
+        return self.client.post(url, json.dumps(body or {}), content_type='application/json')
+
+    def make(self, ct='name'):
+        return self.post('/projects/create/', {'preset': 'research', 'data_ct': ct, 'data_iv': IV}).json()['project']
+
+
+class ProjectTests(Base):
+    def test_create_list_update(self):
+        p = self.make()
+        self.assertEqual(p['preset'], 'research')
+        r = self.client.get('/projects/').json()
+        self.assertEqual([x['id'] for x in r['projects']], [p['id']])
+        self.assertIn('research', r['presets'])
+        r = self.post(f"/projects/{p['id']}/", {'data_ct': 'neu', 'data_iv': 'b2' * 12, 'base_iv': IV})
+        self.assertEqual(r.json()['project']['data_ct'], 'neu')
+        # Written from an old state: refused with the current one.
+        r = self.post(f"/projects/{p['id']}/", {'data_ct': 'alt', 'data_iv': 'c3' * 12, 'base_iv': IV})
+        self.assertEqual((r.status_code, r.json()['project']['data_ct']), (409, 'neu'))
+
+    def test_bad_input(self):
+        self.assertEqual(self.post('/projects/create/', {'data_ct': 'x', 'data_iv': 'zz'}).status_code, 400)
+        self.assertEqual(self.post('/projects/create/', {'preset': 'nope', 'data_ct': 'x', 'data_iv': IV}).status_code, 400)
+
+    def test_only_ones_own(self):
+        p = self.make()
+        other = User.objects.create_user(username='andere', password='pw12345678')
+        self.client.force_login(other)
+        self.assertEqual(self.client.get(f"/projects/{p['id']}/").status_code, 404)
+        self.assertEqual(self.post(f"/projects/{p['id']}/items/add/", {'kind': 'note', 'data_ct': 'x', 'data_iv': IV}).status_code, 404)
+        self.assertEqual(self.client.get('/projects/').json()['projects'], [])
+        self.client.logout()
+        self.assertEqual(self.client.get('/projects/').status_code, 401)
+
+    def test_delete_takes_the_items(self):
+        p = self.make()
+        self.post(f"/projects/{p['id']}/items/add/", {'kind': 'note', 'data_ct': 'x', 'data_iv': IV})
+        self.post(f"/projects/{p['id']}/delete/")
+        self.assertEqual((Project.objects.count(), ProjectItem.objects.count()), (0, 0))
+
+
+class ItemTests(Base):
+    def test_links_once_notes_many(self):
+        p = self.make()
+        url = f"/projects/{p['id']}/items/add/"
+        a = self.post(url, {'kind': 'book', 'ref': 12, 'data_ct': 'b', 'data_iv': IV}).json()
+        b = self.post(url, {'kind': 'book', 'ref': 12, 'data_ct': 'b2', 'data_iv': IV}).json()
+        self.assertEqual((a['existing'], b['existing'], a['item']['id']), (False, True, b['item']['id']))
+        self.post(url, {'kind': 'note', 'data_ct': 'n', 'data_iv': IV})
+        self.post(url, {'kind': 'note', 'ref': 5, 'data_ct': 'n', 'data_iv': IV})   # a note has no ref
+        self.post(url, {'kind': 'episode', 'ref': 7, 'data_ct': 'e', 'data_iv': IV})
+        self.assertEqual(self.post(url, {'kind': 'episode', 'data_ct': 'e', 'data_iv': IV}).status_code, 400)
+        got = self.client.get(f"/projects/{p['id']}/").json()
+        self.assertEqual([(i['kind'], i['ref']) for i in got['items']],
+                         [('book', 12), ('note', None), ('note', None), ('episode', 7)])
+        links = self.client.get('/projects/').json()['links']
+        self.assertEqual(sorted((l['kind'], l['ref']) for l in links), [('book', 12), ('episode', 7)])
+
+    def test_update_with_base_and_delete(self):
+        p = self.make()
+        i = self.post(f"/projects/{p['id']}/items/add/", {'kind': 'note', 'data_ct': 'v1', 'data_iv': IV}).json()['item']
+        r = self.post(f"/projects/items/{i['id']}/", {'data_ct': 'v2', 'data_iv': 'b2' * 12, 'base_iv': IV})
+        self.assertEqual(r.status_code, 200)
+        r = self.post(f"/projects/items/{i['id']}/", {'data_ct': 'v3', 'data_iv': 'c3' * 12, 'base_iv': IV})
+        self.assertEqual((r.status_code, r.json()['item']['data_ct']), (409, 'v2'))
+        self.post(f"/projects/items/{i['id']}/delete/")
+        self.assertFalse(ProjectItem.objects.exists())
+
+
+class DoiTests(Base):
+    def test_recognised(self):
+        self.assertEqual(doi_of('https://doi.org/10.1038/s41586-020-2649-2.'), '10.1038/s41586-020-2649-2')
+        self.assertEqual(doi_of('https://arxiv.org/abs/2401.01234v2'), '10.48550/arXiv.2401.01234')
+        self.assertEqual(doi_of('2401.01234'), '10.48550/arXiv.2401.01234')
+        self.assertIsNone(doi_of('kein doi'))
+
+    def test_lookup_trimmed_and_cached(self):
+        class R:
+            status_code = 200
+
+            def json(self):
+                return {'title': 'Ein Paper', 'DOI': '10.1234/x', 'reference': [1, 2, 3], 'author': [{'family': 'A'}]}
+        with patch('projects.views.requests.get', return_value=R()) as get:
+            a = self.client.get('/projects/doi/', {'q': 'doi:10.1234/x'}).json()
+            self.client.get('/projects/doi/', {'q': '10.1234/X'})
+        self.assertEqual(get.call_count, 1)
+        self.assertEqual(a['csl']['title'], 'Ein Paper')
+        self.assertNotIn('reference', a['csl'])
+        self.assertEqual(self.client.get('/projects/doi/', {'q': 'nichts'}).status_code, 400)

projects/urls.py

@@ -0,0 +1,14 @@
+from django.urls import path
+
+from . import views
+
+urlpatterns = [
+    path('', views.projects, name='projects'),
+    path('create/', views.create, name='projects_create'),
+    path('doi/', views.doi, name='projects_doi'),
+    path('<int:pk>/', views.project, name='projects_project'),
+    path('<int:pk>/delete/', views.delete, name='projects_delete'),
+    path('<int:pk>/items/add/', views.item_add, name='projects_item_add'),
+    path('items/<int:pk>/', views.item_update, name='projects_item_update'),
+    path('items/<int:pk>/delete/', views.item_delete, name='projects_item_delete'),
+]

projects/views.py

@@ -0,0 +1,285 @@
+"""Projekte (docs/projects.md): the API. Everything a project says is encrypted on the device;
+here it's stored and handed back. A write names the IV it changed (base_iv): changed
+elsewhere since, it's refused with the current state (409) for the device to merge."""
+import json
+import re
+
+import requests
+from django.core.cache import cache
+from django.db import IntegrityError, transaction
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+from django.views.decorators.http import require_http_methods
+
+from .models import Project, ProjectItem
+
+MAX_DATA = 2_000_000   # characters of base64: a long draft, not a PDF
+MAX_PROJECTS = 200
+MAX_ITEMS = 5000
+
+
+def _auth(request):
+    if not request.user.is_authenticated:
+        return JsonResponse({'error': 'authentication required'}, status=401)
+    return None
+
+
+def _body(request):
+    try:
+        body = json.loads(request.body or b'{}')
+    except (json.JSONDecodeError, ValueError):
+        return None
+    return body if isinstance(body, dict) else None
+
+
+def _hex_iv(value):
+    return isinstance(value, str) and 8 <= len(value) <= 32 and all(c in '0123456789abcdefABCDEF' for c in value)
+
+
+def _blob(body):
+    """(data_ct, data_iv) from a request, or None."""
+    ct, iv = body.get('data_ct'), body.get('data_iv')
+    if not isinstance(ct, str) or not ct or len(ct) > MAX_DATA or not _hex_iv(iv):
+        return None
+    return ct, iv
+
+
+def _project_json(p, counts=None):
+    return {'id': p.id, 'preset': p.preset, 'data_ct': p.data_ct, 'data_iv': p.data_iv, 'archived': p.archived,
+            'position': p.position, 'created_at': p.created_at.isoformat(), 'updated_at': p.updated_at.isoformat(),
+            'items': (counts or {}).get(p.id, 0)}
+
+
+def _item_json(i):
+    return {'id': i.id, 'project': i.project_id, 'kind': i.kind, 'ref': i.ref, 'data_ct': i.data_ct,
+            'data_iv': i.data_iv, 'created_at': i.created_at.isoformat(), 'updated_at': i.updated_at.isoformat()}
+
+
+def _project(request, pk):
+    return Project.objects.filter(pk=pk, user=request.user).first()
+
+
+# ---- projects ------------------------------------------------------------------------------
+
+@require_http_methods(['GET'])
+def projects(request):
+    """All projects, and every link (kind, ref, project) — what a book's or an episode's sheet
+    shows as "in Projekt …"."""
+    err = _auth(request)
+    if err:
+        return err
+    rows = list(Project.objects.filter(user=request.user))
+    counts = {}
+    links = []
+    for pid, kind, ref in ProjectItem.objects.filter(project__user=request.user).values_list('project_id', 'kind', 'ref'):
+        counts[pid] = counts.get(pid, 0) + 1
+        if ref is not None:
+            links.append({'project': pid, 'kind': kind, 'ref': ref})
+    return JsonResponse({'projects': [_project_json(p, counts) for p in rows], 'links': links,
+                         'presets': [k for k, _ in Project.PRESETS]})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def create(request):
+    err = _auth(request)
+    if err:
+        return err
+    body = _body(request)
+    blob = body and _blob(body)
+    if not blob:
+        return JsonResponse({'error': 'data_ct, data_iv required'}, status=400)
+    preset = body.get('preset', Project.RESEARCH)
+    if preset not in dict(Project.PRESETS):
+        return JsonResponse({'error': 'unknown preset'}, status=400)
+    if Project.objects.filter(user=request.user).count() >= MAX_PROJECTS:
+        return JsonResponse({'error': 'too many projects'}, status=400)
+    p = Project.objects.create(user=request.user, preset=preset, data_ct=blob[0], data_iv=blob[1])
+    return JsonResponse({'project': _project_json(p)})
+
+
+@csrf_exempt
+@require_http_methods(['GET', 'POST'])
+def project(request, pk):
+    """GET: the project with its items. POST: new data (with base_iv), archived, position."""
+    err = _auth(request)
+    if err:
+        return err
+    p = _project(request, pk)
+    if not p:
+        return JsonResponse({'error': 'not found'}, status=404)
+    if request.method == 'GET':
+        items = list(p.items.all())
+        return JsonResponse({'project': _project_json(p, {p.id: len(items)}), 'items': [_item_json(i) for i in items]})
+    body = _body(request)
+    if body is None:
+        return JsonResponse({'error': 'invalid JSON'}, status=400)
+    fields = []
+    if 'data_ct' in body:
+        blob = _blob(body)
+        if not blob:
+            return JsonResponse({'error': 'data_ct, data_iv required'}, status=400)
+        base = body.get('base_iv')
+        if base and base != p.data_iv:
+            return JsonResponse({'error': 'stale', 'project': _project_json(p)}, status=409)
+        p.data_ct, p.data_iv = blob
+        fields += ['data_ct', 'data_iv']
+    if 'archived' in body:
+        p.archived = bool(body['archived'])
+        fields.append('archived')
+    if 'position' in body:
+        try:
+            p.position = int(body['position'])
+        except (TypeError, ValueError):
+            return JsonResponse({'error': 'position'}, status=400)
+        fields.append('position')
+    if fields:
+        p.save(update_fields=fields + ['updated_at'])
+    return JsonResponse({'project': _project_json(p)})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def delete(request, pk):
+    err = _auth(request)
+    if err:
+        return err
+    p = _project(request, pk)
+    if not p:
+        return JsonResponse({'error': 'not found'}, status=404)
+    p.delete()
+    return JsonResponse({'ok': True})
+
+
+# ---- items ---------------------------------------------------------------------------------
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def item_add(request, pk):
+    """{kind, ref?, data_ct, data_iv}. A book, episode or article is in a project once: adding
+    it again answers with the item there (existing: true)."""
+    err = _auth(request)
+    if err:
+        return err
+    p = _project(request, pk)
+    if not p:
+        return JsonResponse({'error': 'not found'}, status=404)
+    body = _body(request)
+    blob = body and _blob(body)
+    if not blob:
+        return JsonResponse({'error': 'data_ct, data_iv required'}, status=400)
+    kind = body.get('kind')
+    if kind not in dict(ProjectItem.KINDS):
+        return JsonResponse({'error': 'kind'}, status=400)
+    ref = body.get('ref')
+    if kind in ProjectItem.LINKED:
+        if not isinstance(ref, int) or isinstance(ref, bool) or ref <= 0:
+            return JsonResponse({'error': 'ref required'}, status=400)
+    else:
+        ref = None
+    if p.items.count() >= MAX_ITEMS:
+        return JsonResponse({'error': 'too many items'}, status=400)
+    if ref is not None:
+        have = p.items.filter(kind=kind, ref=ref).first()
+        if have:
+            return JsonResponse({'item': _item_json(have), 'existing': True})
+    try:
+        with transaction.atomic():
+            i = ProjectItem.objects.create(project=p, kind=kind, ref=ref, data_ct=blob[0], data_iv=blob[1])
+    except IntegrityError:
+        return JsonResponse({'item': _item_json(p.items.get(kind=kind, ref=ref)), 'existing': True})
+    p.save(update_fields=['updated_at'])
+    return JsonResponse({'item': _item_json(i), 'existing': False})
+
+
+def _item(request, pk):
+    return ProjectItem.objects.filter(pk=pk, project__user=request.user).select_related('project').first()
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def item_update(request, pk):
+    """{data_ct, data_iv, base_iv}: a changed note or source; 409 with the current one when it
+    was changed elsewhere since."""
+    err = _auth(request)
+    if err:
+        return err
+    i = _item(request, pk)
+    if not i:
+        return JsonResponse({'error': 'not found'}, status=404)
+    body = _body(request)
+    blob = body and _blob(body)
+    if not blob:
+        return JsonResponse({'error': 'data_ct, data_iv required'}, status=400)
+    base = body.get('base_iv')
+    if base and base != i.data_iv:
+        return JsonResponse({'error': 'stale', 'item': _item_json(i)}, status=409)
+    i.data_ct, i.data_iv = blob
+    i.save(update_fields=['data_ct', 'data_iv', 'updated_at'])
+    i.project.save(update_fields=['updated_at'])
+    return JsonResponse({'item': _item_json(i)})
+
+
+@csrf_exempt
+@require_http_methods(['POST'])
+def item_delete(request, pk):
+    err = _auth(request)
+    if err:
+        return err
+    i = _item(request, pk)
+    if not i:
+        return JsonResponse({'error': 'not found'}, status=404)
+    i.delete()
+    return JsonResponse({'ok': True})
+
+
+# ---- a paper's data by its DOI ----------------------------------------------------------------
+
+DOI_RE = re.compile(r'(10\.\d{4,9}/[^\s"<>]+)', re.I)
+
+
+def doi_of(text):
+    """The DOI in a DOI, a doi.org link, a publisher's link that carries it, or an arXiv id."""
+    text = (text or '').strip()
+    m = DOI_RE.search(text)
+    if m:
+        return m.group(1).rstrip('.,;)')
+    m = re.search(r'arxiv\.org/(?:abs|pdf)/(\d{4}\.\d{4,5})|^(?:arxiv:)?(\d{4}\.\d{4,5})(?:v\d+)?$', text, re.I)
+    if m:
+        return '10.48550/arXiv.' + (m.group(1) or m.group(2))
+    return None
+
+
+@require_http_methods(['GET'])
+def doi(request):
+    """?q=<DOI, link or arXiv id>: the work's citation data (CSL-JSON) from doi.org — Crossref,
+    DataCite (arXiv) and the other registries answer there. Not kept beyond a cache."""
+    err = _auth(request)
+    if err:
+        return err
+    d = doi_of(request.GET.get('q', ''))
+    if not d:
+        return JsonResponse({'error': 'Keine DOI erkannt.'}, status=400)
+    key = 'doi:' + re.sub(r'\W+', '_', d.lower())[:200]
+    csl = cache.get(key)
+    if csl is None:
+        try:
+            r = requests.get('https://doi.org/' + d, timeout=15,
+                             headers={'Accept': 'application/vnd.citationstyles.csl+json',
+                                      'User-Agent': 'diora (https://diora.creamfresh.xyz)'})
+        except requests.RequestException:
+            return JsonResponse({'error': 'doi.org nicht erreichbar.'}, status=502)
+        if r.status_code == 404:
+            return JsonResponse({'error': 'DOI unbekannt.'}, status=404)
+        try:
+            csl = r.json()
+        except ValueError:
+            return JsonResponse({'error': 'Keine Daten zu dieser DOI.'}, status=502)
+        if not isinstance(csl, dict):
+            return JsonResponse({'error': 'Keine Daten zu dieser DOI.'}, status=502)
+        # What a citation needs; the reference lists and licences stay out.
+        for k in ('reference', 'license', 'link', 'relation', 'assertion', 'funder', 'content-domain',
+                  'indexed', 'deposited', 'update-policy', 'resource'):
+            csl.pop(k, None)
+        cache.set(key, csl, 7 * 24 * 3600)
+    return JsonResponse({'doi': d, 'csl': csl})

static/js/app/20-tabs.js

@@ -4,7 +4,7 @@
 // Tabs
 // ---------------------------------------------------------------------------
 
-const TOP_TABS = ['home', 'radio', 'focus', 'podcasts', 'books', 'feeds', 'ambient', 'audiobooks', 'sources', 'suche'];
+const TOP_TABS = ['home', 'radio', 'focus', 'podcasts', 'books', 'feeds', 'ambient', 'audiobooks', 'sources', 'suche', 'projekte'];
 const RADIO_SUB_TABS = ['search', 'saved', 'history'];
 
 // ---------------------------------------------------------------------------

static/js/app/53-library.js

@@ -684,6 +684,11 @@ function _libSeriesManage() {
 
 // ---- a book's actions (the apps may bring their own sheet: __dioraBookSheet) -------------------------
 
+function _libProjectsLabel(b) {
+  const inIt = window.DioraProjects ? DioraProjects.of('book', b.id) : [];
+  return 'Mit Projekt verknüpfen …' + (inIt.length ? ` <span class="muted">(${inIt.map(p => _libEsc(p.name)).join(', ')})</span>` : '');
+}
+
 function _libSheet(b) {
   if (window.__dioraBookSheet && window.__dioraBookSheet(b) !== false) return;
   const broken = typeof _brokenBooks !== 'undefined' && _brokenBooks.has(b.id);
@@ -695,6 +700,8 @@ function _libSheet(b) {
     ['Serie …', () => _libSeriesSheet(b)],
     [_libEditionsOf(b).length > 1 ? `Ausgaben (${_libEditionsOf(b).length}) …` : 'Als Ausgabe verbinden …', () => _libEditionsSheet(b)],
     [b.is_read ? 'Als ungelesen markieren' : 'Als gelesen markieren', () => toggleBookRead(b.id, !!b.is_read)],
+    // Projekte (projects.js): "Forschung › Mit Projekt verknüpfen", and where it is already.
+    window.DioraProjects && [_libProjectsLabel(b), () => DioraProjects.link('book', b.id, {title: b.title, author: b.author})],
     b.has_highlights && ['Markierungen exportieren', () => downloadBookAnnotationsFromList(b.id, null)],
     ['Regal-Kategorie abrufen', () => lookupBookMetadata(b.id)],
     ['Mehrere auswählen', () => { _lib.selecting = new Set([b.id]); renderBookList(_lastBookListData); }],

static/js/feeds.js

@@ -639,6 +639,8 @@
       '<span class="fd-art-src">' + esc(i ? i.source_title : '') + '</span>' +
       '<button type="button" class="fd-icon" data-r="font" aria-label="Schriftgröße" title="Schriftgröße">Aa</button>' +
       (i ? '<button type="button" class="fd-icon' + (i.starred ? ' on' : '') + '" data-r="star" aria-label="Merken" title="Merken">' + (i.starred ? '★' : '☆') + '</button>' : '') +
+      // Projekte (projects.js): this article as a source of a project.
+      (i && window.DioraProjects ? '<button type="button" class="fd-icon" data-r="project" aria-label="Mit Projekt verknüpfen" title="Mit Projekt verknüpfen">◇</button>' : '') +
       (i && i.url ? '<a class="fd-icon" href="' + esc(i.url) + '" target="_blank" rel="noopener" aria-label="Im Browser öffnen" title="Im Browser öffnen">↗</a>' : '') +
       '</div><div class="fd-art" style="font-size:' + fontSize() + 'px">' + inner + '</div>';
   }
@@ -757,6 +759,9 @@
       try { localStorage.setItem('diora_feeds_font', String(v)); } catch (err) {}
       const art = reader.querySelector('.fd-art');
       if (art) art.style.fontSize = v + 'px';
+    } else if (r === 'project' && readerItem && window.DioraProjects) {
+      window.DioraProjects.link('article', readerItem.id, {title: readerItem.title, source: readerItem.source_title,
+        author: readerItem.author, date: readerItem.published, url: readerItem.url, news: readerItem.kind === 'news'});
     } else if (r === 'star' && readerItem) {
       readerItem.starred = !readerItem.starred;
       api('/feeds/items/' + readerItem.id + '/star/', {starred: readerItem.starred}).catch(() => {});

static/js/nav.js

@@ -1,7 +1,7 @@
 /*
  * The navigation (docs/navigation.md): diora's sections, sorted by what one does —
  * Home; Lesen: Bücher, Zeitschriften, Nachrichten, Blogs; Hören: Hörbücher, Podcasts,
- * Radio, Ambient; Suche across all of them. One definition for every platform: the web page's #tabs are drawn
+ * Radio, Ambient; Projekte; Suche across all of them. One definition for every platform: the web page's #tabs are drawn
  * here, the apps' tab bar and sidebar (native-radio.js) from DioraNav.groups() and
  * DioraNav.sections(). A section is a panel (showTab, as before) plus a view in it.
  * On a phone, Lesen and Hören show their sections as a row on top (#nav-segments).
@@ -25,9 +25,12 @@
     {id: 'ambient', group: 'hoeren', label: 'Ambient', icon: '∿', panel: 'ambient'},
     // The focus timer: the web page's own (the apps never had it).
     {id: 'focus', group: 'hoeren', label: 'Fokus', icon: '◷', panel: 'focus', webOnly: true},
-    // The search across diora (docs/search.md, search.js): beside Home · Lesen · Hören on a
+    // Projects (docs/projects.md, projects.js): a tab beside Home · Lesen · Hören on a phone,
-    // phone, at the foot of the sidebar (above Abos & Quellen) and of the wide bar ("side").
+    // at the foot of the sidebar and of the wide bar ("side"), above Suche.
-    {id: 'suche', group: 'suche', label: 'Suche', icon: '⌕', panel: 'suche', account: true, side: true},
+    {id: 'projekte', group: 'projekte', label: 'Projekte', icon: '◇', panel: 'projekte', account: true, side: true},
+    // The search across diora (docs/search.md, search.js): at the foot, above Abos & Quellen;
+    // on a phone a magnifier in the bar on top (the apps' section bars, the page's narrow bar).
+    {id: 'suche', group: 'more', label: 'Suche', icon: '⌕', panel: 'suche', account: true, footer: true, transient: true},
     // Managing every subscription (docs/sources.md): at the foot of the sidebar, in the app
     // menu on a phone — not a tab of its own.
     {id: 'sources', group: 'more', label: 'Abos & Quellen', icon: '⚙', panel: 'sources', account: true, footer: true},
@@ -36,7 +39,7 @@
     {id: 'home', label: 'Home', icon: '⌂'},
     {id: 'lesen', label: 'Lesen', icon: '▤'},
     {id: 'hoeren', label: 'Hören', icon: '◉'},
-    {id: 'suche', label: 'Suche', icon: '⌕'},
+    {id: 'projekte', label: 'Projekte', icon: '◇'},
   ];
   const byId = {};
   SECTIONS.forEach(s => { byId[s.id] = s; });
@@ -63,6 +66,7 @@
       '<path d="M3 9.5c2-2.6 4-2.6 6 0s4 2.6 6 0 4-2.6 6 0"/><path d="M3 14.5c2-2.6 4-2.6 6 0s4 2.6 6 0 4-2.6 6 0"/>',
     focus: '<circle class="f" cx="12" cy="13" r="8"/><circle cx="12" cy="13" r="8"/><path d="M12 9v4l3 2M9 3h6"/>',
     sources: '<circle class="f" cx="6" cy="18" r="2"/><circle cx="6" cy="18" r="1.6"/><path d="M5 11a8 8 0 0 1 8 8"/><path d="M5 5a14 14 0 0 1 14 14"/>',
+    projekte: '<path class="f" d="M4 7h16v13H4z"/><path d="M4 7h16v13H4z"/><path d="M9 7V4h6v3"/><path d="M4 12h16"/>',
     suche: '<circle class="f" cx="10.5" cy="10.5" r="6.5"/><circle cx="10.5" cy="10.5" r="6.5"/><path d="M15.5 15.5L21 21"/>',
     menu: '<rect class="f" x="10.5" y="4" width="3" height="3"/><rect x="10.5" y="4" width="3" height="3"/>' +
       '<rect x="10.5" y="10.5" width="3" height="3"/><rect x="10.5" y="17" width="3" height="3"/>',
@@ -142,7 +146,7 @@
   function setCurrent(id) {
     current = id;
     const s = byId[id];
-    if (!s.side) save('diora_nav_section', id);   // the app opens where one was, not in the search
+    if (!s.transient) save('diora_nav_section', id);   // the app opens where one was, not in the search
     if (!s.footer) save('diora_nav_last_' + s.group, id);
     syncHash();
     render();

static/js/sw.js

@@ -2,7 +2,7 @@
  * diora service worker — caches the app shell for offline use.
  */
 
-const CACHE = 'diora-v140';
+const CACHE = 'diora-v141';
 const SHELL = [
   '/static/css/app.css',
   '/static/js/app/01-core.js',
@@ -49,6 +49,7 @@ const SHELL = [
   '/static/js/gencover.js',
   '/static/js/sources.js',
   '/static/js/search.js',
+  '/static/js/projects.js',
   '/static/podcasts/native-podcast.js',
   '/static/podcasts/podcast-ui.css',
   '/static/podcasts/podcasts-web.css',

templates/radio/player.html

@@ -267,6 +267,8 @@
 {% if user.is_authenticated %}<section class="tab-panel" id="tab-audiobooks" style="display:none;"></section>{% endif %}
 {% comment %}Abos & Quellen, every subscription in one place (docs/sources.md): filled by static/js/sources.js.{% endcomment %}
 {% if user.is_authenticated %}<section class="tab-panel" id="tab-sources" style="display:none;"></section>{% endif %}
+{% comment %}Projekte, e.g. Forschung (docs/projects.md): filled by static/js/projects.js.{% endcomment %}
+{% if user.is_authenticated %}<section class="tab-panel" id="tab-projekte" style="display:none;"></section>{% endif %}
 {% comment %}The search across diora (docs/search.md): filled by static/js/search.js.{% endcomment %}
 {% if user.is_authenticated %}<section class="tab-panel" id="tab-suche" style="display:none;"></section>{% endif %}
 {% comment %}The feed reader, news and blogs: filled by static/js/feeds.js.{% endcomment %}
@@ -398,4 +400,5 @@
 <script src="/static/js/audiobooks.js"></script>
 <script src="/static/js/sources.js"></script>
 <script src="/static/js/search.js"></script>
+<script src="/static/js/projects.js"></script>
 {% endblock %}