Projekte: Radar – wöchentlich neue Papers (PubMed, arXiv, bioRxiv, medRxiv), von Claude im Batch sortiert, Entscheidungen verschlüsselt, → Quellen; Preset Neuropolitik aus F:\neuropolitik\tools; sw v145
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
a329eab091
commit
c8fd139b44
16 changed files with 1220 additions and 14 deletions
|
|
@ -62,9 +62,35 @@ state. A note's text then goes beside it as a copy "(Konflikt)"; other data relo
|
|||
- `GET /projects/doi/?q=<DOI | doi.org link | arXiv link or id>` → `{doi, csl}` — fetched from
|
||||
doi.org (Crossref, DataCite …) as CSL-JSON, cached a week, nothing else kept.
|
||||
|
||||
## Radar
|
||||
|
||||
A project can have a Radar (`projects/radar.py`, `projects/papers.py`): every week the new papers
|
||||
from PubMed, arXiv, bioRxiv and medRxiv, sorted by Claude into the config's groups (the first
|
||||
one — "lesenswert" — with what it's about, how solid it is, the political hook), the week's
|
||||
news (web search), a basic term explained. One Claude batch (Opus, half price): about 0.40 $
|
||||
a week. Ported from the "Neuropolitik Grundlagen" session's `F:
|
||||
europolitik ools`
|
||||
(fetch_papers.py, digest_batch.py); the digest is their bundle format (version 1).
|
||||
|
||||
- Config per radar (`Radar.config`, presets in `projects/radar_presets.py`): PubMed queries per
|
||||
topic, arXiv query, keyword pattern for the preprint servers, groups and their labels, the
|
||||
prompts, weekday and hour (Europe/Berlin), days. **Plain on the server**: it searches with them.
|
||||
The papers and digests are public material, plain too (`RadarDigest.bundle`).
|
||||
- The decisions (gelesen / später / verworfen / in Quellen) are the user's: an encrypted item
|
||||
of kind `radar` (`{decisions: {paper id: state}}`). "→ Quellen" makes a `paper` source
|
||||
(citation data from doi.org when there's a DOI, else from the record).
|
||||
- Runs from the host's crontab every half hour: `docker exec diora-prod python manage.py radar`
|
||||
(submits what is due, collects finished batches). Needs `ANTHROPIC_API_KEY` in the container.
|
||||
`radar --setup <project> neuropolitik` gives a project its Radar; `radar --import <project>
|
||||
<file>` takes a digest made elsewhere; `radar --seen <project> <file>` ids reported before.
|
||||
- API: `GET /projects/<id>/radar/` → `{radar: {enabled, title, groups, group_labels, weekday,
|
||||
hour, days, sources, running, submitted, last_run_at, last_error, next_run, key}, digests:
|
||||
[{id, generated, bundle}]}` (the latest 12); `POST …/radar/settings/` `{enabled}`;
|
||||
`POST …/radar/run/` (fetch at the next cron run); `POST …/radar/import/` (a bundle, e.g.
|
||||
`digest_batch.py --diora-url`).
|
||||
|
||||
## Next
|
||||
|
||||
Radar (the weekly digests and new papers with lesen / später / verwerfen / → Quellen; who
|
||||
fetches the papers is open), papers as books (a paper's PDF into the library, its highlights
|
||||
Papers as books (a paper's PDF into the library, its highlights
|
||||
into the project's notes), Schreiben (drafts with the bibliography in a citation style),
|
||||
the projects in the search, more presets.
|
||||
|
|
|
|||
|
|
@ -137,6 +137,48 @@ test('Projekte: a Forschung project, a paper by DOI, a linked book, a note citin
|
|||
expect(bib).toContain('doi = {10.1038/nn1979}');
|
||||
});
|
||||
|
||||
test('Radar: the week\'s digest, decisions kept encrypted, a paper into the sources', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1200, height: 900 });
|
||||
const digest = {id: 1, generated: '2026-10-19', bundle: {version: 1, generated: '2026-10-19',
|
||||
auf_einen_blick: 'Ein Decoder kommt der Absicht zuvor.', grundlagenbegriff: {begriff: 'Decoder', erklaerung: 'Ein Modell …'},
|
||||
news_markdown: '- **Paradromics:** Kooperation. [Quelle](https://example.org/n)',
|
||||
stats: {records: 3, per_source: {PubMed: 2, arXiv: 1}, errors: [], cost_usd: 0.38},
|
||||
items: [
|
||||
{id: 'pmid:1', doi: '', source: 'PubMed', title: 'Intention decoded', journal: 'Nature', first_author: 'Verbaarschot',
|
||||
n_authors: 5, date: '2026', url: 'https://example.org/1', gruppe: 'lesenswert', worum_es_geht: 'Ein Spiel.',
|
||||
einordnung: 'Preprint.', politischer_haken: 'Libet.', kurz: 'Kurz.'},
|
||||
{id: 'arxiv:2', source: 'arXiv', title: 'EEG artefacts', first_author: 'Adeika', n_authors: 2, url: 'https://example.org/2',
|
||||
gruppe: 'technik', worum_es_geht: '', einordnung: '', politischer_haken: '', kurz: 'Muskeln statt Gedanken.'},
|
||||
{id: 'pmid:3', source: 'PubMed', title: 'Off topic', gruppe: 'rauschen', kurz: ''}]}};
|
||||
await page.route(/\/projects\/\d+\/radar\/$/, route => route.fulfill({json: {radar: {enabled: true, title: 'Neuropolitik-Digest',
|
||||
groups: ['lesenswert', 'propriozeption', 'gesellschaft', 'technik', 'rauschen'], group_labels: {lesenswert: 'Lesenswert', technik: 'Technik im Überblick', rauschen: 'Rauschen'},
|
||||
weekday: 6, hour: 18, days: 7, sources: ['PubMed', 'arXiv'], running: false, next_run: '2026-10-26T18:00:00+01:00',
|
||||
last_error: '', key: true}, digests: [digest]}}));
|
||||
await page.goto('/#projekte');
|
||||
const tab = page.locator('#tab-projekte');
|
||||
await tab.locator('[data-act="new"]').click();
|
||||
await tab.locator('input[name="name"]').fill('Radarprojekt');
|
||||
await tab.locator('button[type="submit"]').click();
|
||||
await expect(tab.locator('.prj-tab.on')).toContainText('Radar');
|
||||
await expect(tab.locator('.prj-radar-h')).toContainText('KW 43');
|
||||
await expect(tab.locator('.prj-radar-group[open] > summary')).toHaveText([/Lesenswert/, /Technik/, /Branche/]);
|
||||
await expect(tab.locator('.prj-paper').first()).toContainText('Politischer Haken: Libet.');
|
||||
await tab.locator('[data-radar-decide="arxiv:2"][data-state="later"]').click();
|
||||
await tab.locator('[data-radar-decide="pmid:1"][data-state="read"]').click();
|
||||
await expect(tab.locator('[data-radar-later]')).toHaveText('Später 1');
|
||||
// Kept, encrypted: after a reload still there.
|
||||
await page.waitForTimeout(800);
|
||||
await page.reload();
|
||||
await expect(tab.locator('[data-radar-later]')).toHaveText('Später 1');
|
||||
await expect(tab.locator('.prj-paper').first()).toContainText('gelesen');
|
||||
const raw = await page.evaluate(() => fetch(location.pathname.replace(/\/$/, '') + '/projects/').then(r => r.json()));
|
||||
expect(JSON.stringify(raw)).not.toContain('arxiv:2');
|
||||
await tab.locator('[data-radar-source="arxiv:2"]').click();
|
||||
await expect(tab.locator('.prj-paper').nth(1)).toContainText('✓ in Quellen');
|
||||
await tab.locator('[data-view="sources"]').click();
|
||||
await expect(tab.locator('.prj-row .prj-title')).toHaveText(['EEG artefacts']);
|
||||
});
|
||||
|
||||
test('Abos & Quellen: one page for every subscription, reached from the feeds too', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1200, height: 800 });
|
||||
await page.goto('/#news');
|
||||
|
|
|
|||
0
projects/management/__init__.py
Normal file
0
projects/management/__init__.py
Normal file
0
projects/management/commands/__init__.py
Normal file
0
projects/management/commands/__init__.py
Normal file
47
projects/management/commands/radar.py
Normal file
47
projects/management/commands/radar.py
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
"""The Radars (projects/radar.py): collect finished batches, submit what is due.
|
||||
Run from the host's crontab every half hour:
|
||||
docker exec diora-prod python manage.py radar
|
||||
... radar --setup <project id> neuropolitik a project gets a Radar from a preset
|
||||
... radar --import <project id> <file.json> a digest made elsewhere (also marks its papers seen)
|
||||
"""
|
||||
import json
|
||||
|
||||
from django.core.management.base import BaseCommand, CommandError
|
||||
|
||||
from projects import radar
|
||||
from projects.models import Project, Radar
|
||||
from projects.radar_presets import PRESETS
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = 'Radar: collect finished batches, submit what is due'
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument('--setup', nargs=2, metavar=('PROJECT', 'PRESET'))
|
||||
parser.add_argument('--import', dest='import_', nargs=2, metavar=('PROJECT', 'FILE'))
|
||||
parser.add_argument('--seen', nargs=2, metavar=('PROJECT', 'FILE'), help='ids reported before (JSON list)')
|
||||
|
||||
def handle(self, *args, setup=None, import_=None, seen=None, **opts):
|
||||
if setup:
|
||||
p = Project.objects.filter(pk=setup[0]).first()
|
||||
if not p or setup[1] not in PRESETS:
|
||||
raise CommandError('project or preset unknown')
|
||||
r, made = Radar.objects.update_or_create(project=p, defaults={'config': PRESETS[setup[1]]})
|
||||
self.stdout.write(f"radar {p.id}: {'set up' if made else 'config updated'}")
|
||||
return
|
||||
if import_:
|
||||
r = Radar.objects.get(project_id=import_[0])
|
||||
with open(import_[1], encoding='utf-8') as f:
|
||||
d = radar.store(r, json.load(f))
|
||||
self.stdout.write(f'radar {r.project_id}: digest {d.id} ({d.generated})')
|
||||
return
|
||||
if seen:
|
||||
r = Radar.objects.get(project_id=seen[0])
|
||||
with open(seen[1], encoding='utf-8') as f:
|
||||
ids = json.load(f)
|
||||
r.seen = list(dict.fromkeys(list(r.seen) + list(ids)))[-radar.SEEN_MAX:]
|
||||
r.save(update_fields=['seen'])
|
||||
self.stdout.write(f'radar {r.project_id}: {len(r.seen)} seen')
|
||||
return
|
||||
for line in radar.tick():
|
||||
self.stdout.write(line)
|
||||
46
projects/migrations/0002_radar.py
Normal file
46
projects/migrations/0002_radar.py
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Generated by Django 6.1 on 2026-10-19 18:32
|
||||
|
||||
import django.db.models.deletion
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('projects', '0001_initial'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AlterField(
|
||||
model_name='projectitem',
|
||||
name='kind',
|
||||
field=models.CharField(choices=[('book', 'Buch'), ('episode', 'Folge'), ('article', 'Artikel'), ('paper', 'Paper'), ('note', 'Notiz'), ('radar', 'Radar')], max_length=10),
|
||||
),
|
||||
migrations.CreateModel(
|
||||
name='Radar',
|
||||
fields=[
|
||||
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
|
||||
('enabled', models.BooleanField(default=True)),
|
||||
('config', models.JSONField(default=dict)),
|
||||
('seen', models.JSONField(default=list)),
|
||||
('pending', models.JSONField(blank=True, null=True)),
|
||||
('last_run_at', models.DateTimeField(blank=True, null=True)),
|
||||
('last_error', models.TextField(blank=True)),
|
||||
('project', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='radar', to='projects.project')),
|
||||
],
|
||||
),
|
||||
migrations.CreateModel(
|
||||
name='RadarDigest',
|
||||
fields=[
|
||||
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
|
||||
('generated', models.DateField()),
|
||||
('batch_id', models.CharField(blank=True, max_length=100)),
|
||||
('bundle', models.JSONField()),
|
||||
('created_at', models.DateTimeField(auto_now_add=True)),
|
||||
('radar', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='digests', to='projects.radar')),
|
||||
],
|
||||
options={
|
||||
'ordering': ['-generated', '-created_at'],
|
||||
},
|
||||
),
|
||||
]
|
||||
|
|
@ -34,7 +34,10 @@ class ProjectItem(models.Model):
|
|||
ARTICLE = 'article'
|
||||
PAPER = 'paper'
|
||||
NOTE = 'note'
|
||||
KINDS = [(BOOK, 'Buch'), (EPISODE, 'Folge'), (ARTICLE, 'Artikel'), (PAPER, 'Paper'), (NOTE, 'Notiz')]
|
||||
# The Radar's decisions ({paper id: read | later | drop}), one per project.
|
||||
RADAR = 'radar'
|
||||
KINDS = [(BOOK, 'Buch'), (EPISODE, 'Folge'), (ARTICLE, 'Artikel'), (PAPER, 'Paper'), (NOTE, 'Notiz'),
|
||||
(RADAR, 'Radar')]
|
||||
LINKED = (BOOK, EPISODE, ARTICLE)
|
||||
|
||||
project = models.ForeignKey(Project, on_delete=models.CASCADE, related_name='items')
|
||||
|
|
@ -50,3 +53,29 @@ class ProjectItem(models.Model):
|
|||
constraints = [models.UniqueConstraint(fields=['project', 'kind', 'ref'], name='project_item_once',
|
||||
condition=models.Q(ref__isnull=False))]
|
||||
indexes = [models.Index(fields=['kind', 'ref'])]
|
||||
|
||||
|
||||
class Radar(models.Model):
|
||||
"""A project's Radar (docs/projects.md): every week new papers from PubMed, arXiv, bioRxiv,
|
||||
medRxiv (projects/papers.py), sorted and summed up by Claude in a batch (projects/radar.py).
|
||||
Plain on the server, unlike the rest of a project: the server searches with the config and
|
||||
the papers are public."""
|
||||
project = models.OneToOneField(Project, on_delete=models.CASCADE, related_name='radar')
|
||||
enabled = models.BooleanField(default=True)
|
||||
config = models.JSONField(default=dict) # projects/radar_presets.py
|
||||
seen = models.JSONField(default=list) # paper ids reported before
|
||||
pending = models.JSONField(null=True, blank=True) # {batch_id, generated, days, records, errors, submitted}
|
||||
last_run_at = models.DateTimeField(null=True, blank=True)
|
||||
last_error = models.TextField(blank=True)
|
||||
|
||||
|
||||
class RadarDigest(models.Model):
|
||||
"""One week's digest: the bundle as digest_batch.py makes it (version 1)."""
|
||||
radar = models.ForeignKey(Radar, on_delete=models.CASCADE, related_name='digests')
|
||||
generated = models.DateField()
|
||||
batch_id = models.CharField(max_length=100, blank=True)
|
||||
bundle = models.JSONField()
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
|
||||
class Meta:
|
||||
ordering = ['-generated', '-created_at']
|
||||
|
|
|
|||
143
projects/papers.py
Normal file
143
projects/papers.py
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
"""New papers for a project's Radar (docs/projects.md): PubMed, arXiv, bioRxiv, medRxiv.
|
||||
|
||||
What is searched comes from the radar's config (projects/radar_presets.py): PubMed queries per
|
||||
topic, one arXiv query, the preprint servers filtered by a keyword pattern. collect() returns
|
||||
(records, errors) and keeps nothing — the caller remembers what was reported, so a failed run
|
||||
loses nothing. Ported from F:\\neuropolitik\\tools\\fetch_papers.py (Neuropolitik session)."""
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import xml.etree.ElementTree as ET
|
||||
from datetime import date, timedelta
|
||||
|
||||
import requests
|
||||
|
||||
UA = 'diora-radar/1.0 (personal research digest; https://diora.creamfresh.xyz)'
|
||||
ABSTRACT_CHARS = 1200
|
||||
RXIV_SERVERS = ('biorxiv', 'medrxiv')
|
||||
|
||||
|
||||
def get(url, params=None, retries=3, timeout=60):
|
||||
for attempt in range(retries):
|
||||
try:
|
||||
r = requests.get(url, params=params, headers={'User-Agent': UA}, timeout=timeout)
|
||||
r.raise_for_status()
|
||||
return r.text
|
||||
except requests.RequestException:
|
||||
if attempt == retries - 1:
|
||||
raise
|
||||
time.sleep(3 * (attempt + 1))
|
||||
|
||||
|
||||
def clip(text):
|
||||
text = re.sub(r'\s+', ' ', text or '').strip()
|
||||
return text if len(text) <= ABSTRACT_CHARS else text[:ABSTRACT_CHARS].rsplit(' ', 1)[0] + ' …'
|
||||
|
||||
|
||||
def fetch_pubmed(queries, days):
|
||||
base = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/'
|
||||
records = []
|
||||
for topic, query in queries.items():
|
||||
ids = json.loads(get(base + 'esearch.fcgi', {'db': 'pubmed', 'term': query, 'reldate': days,
|
||||
'datetype': 'edat', 'retmax': 300, 'retmode': 'json'}))
|
||||
ids = ids['esearchresult']['idlist']
|
||||
for i in range(0, len(ids), 100):
|
||||
root = ET.fromstring(get(base + 'efetch.fcgi', {'db': 'pubmed', 'id': ','.join(ids[i:i + 100]),
|
||||
'retmode': 'xml'}))
|
||||
for art in root.iter('PubmedArticle'):
|
||||
pmid = art.findtext('.//PMID')
|
||||
t = art.find('.//ArticleTitle')
|
||||
abstract = ' '.join(''.join(a.itertext()) for a in art.findall('.//Abstract/AbstractText'))
|
||||
doi = next((e.text for e in art.findall('.//ArticleId') if e.get('IdType') == 'doi'), None)
|
||||
authors = art.findall('.//Author')
|
||||
records.append({
|
||||
'id': f'pmid:{pmid}', 'doi': doi, 'source': 'PubMed', 'topic': topic,
|
||||
'title': clip(''.join(t.itertext()) if t is not None else ''),
|
||||
'journal': art.findtext('.//Journal/Title'),
|
||||
'first_author': authors[0].findtext('LastName') if authors else None,
|
||||
'n_authors': len(authors), 'pub_types': [p.text for p in art.findall('.//PublicationType')],
|
||||
'date': art.findtext('.//PubDate/Year'), 'url': f'https://pubmed.ncbi.nlm.nih.gov/{pmid}/',
|
||||
'abstract': clip(abstract),
|
||||
})
|
||||
time.sleep(0.4) # NCBI: 3 requests a second without a key
|
||||
return records
|
||||
|
||||
|
||||
def fetch_arxiv(query, days):
|
||||
cutoff = (date.today() - timedelta(days=days)).isoformat()
|
||||
xml = get('https://export.arxiv.org/api/query', {'search_query': query, 'sortBy': 'submittedDate',
|
||||
'sortOrder': 'descending', 'max_results': 150})
|
||||
ns = {'a': 'http://www.w3.org/2005/Atom', 'arxiv': 'http://arxiv.org/schemas/atom'}
|
||||
records = []
|
||||
for e in ET.fromstring(xml).findall('a:entry', ns):
|
||||
published = e.findtext('a:published', '', ns)[:10]
|
||||
if published < cutoff:
|
||||
continue
|
||||
arxiv_id = e.findtext('a:id', '', ns).rsplit('/abs/', 1)[-1]
|
||||
cat = e.find('arxiv:primary_category', ns)
|
||||
authors = e.findall('a:author', ns)
|
||||
records.append({
|
||||
'id': 'arxiv:' + re.sub(r'v\d+$', '', arxiv_id), 'doi': None, 'source': 'arXiv', 'topic': None,
|
||||
'title': clip(e.findtext('a:title', '', ns)),
|
||||
'journal': f"arXiv {cat.get('term') if cat is not None else ''}".strip(),
|
||||
'first_author': authors[0].findtext('a:name', '', ns) if authors else None,
|
||||
'n_authors': len(authors), 'pub_types': ['Preprint'], 'date': published,
|
||||
'url': f'https://arxiv.org/abs/{arxiv_id}', 'abstract': clip(e.findtext('a:summary', '', ns)),
|
||||
})
|
||||
return records
|
||||
|
||||
|
||||
def fetch_rxiv(server, days, keywords):
|
||||
start = (date.today() - timedelta(days=days)).isoformat()
|
||||
end = date.today().isoformat()
|
||||
records, cursor, total = [], 0, None
|
||||
while total is None or cursor < total:
|
||||
data = json.loads(get(f'https://api.biorxiv.org/details/{server}/{start}/{end}/{cursor}'))
|
||||
msg = data['messages'][0]
|
||||
if msg.get('status') != 'ok':
|
||||
break
|
||||
total = int(msg.get('total', 0))
|
||||
batch = data.get('collection', [])
|
||||
if not batch:
|
||||
break
|
||||
for p in batch:
|
||||
if not keywords.search(f"{p.get('title', '')} {p.get('abstract', '')}"):
|
||||
continue
|
||||
authors = (p.get('authors') or '').split(';')
|
||||
records.append({
|
||||
'id': 'doi:' + p['doi'], 'doi': p['doi'], 'source': server, 'topic': None,
|
||||
'title': clip(p.get('title')), 'journal': f"{server} ({p.get('category')})",
|
||||
'first_author': authors[0], 'n_authors': len(authors), 'pub_types': ['Preprint'],
|
||||
'date': p.get('date'), 'url': f"https://www.{server}.org/content/{p['doi']}v{p.get('version', 1)}",
|
||||
'abstract': clip(p.get('abstract')), 'institution': p.get('author_corresponding_institution'),
|
||||
})
|
||||
cursor += len(batch)
|
||||
time.sleep(0.3)
|
||||
return records
|
||||
|
||||
|
||||
def collect(config, days, seen=frozenset()):
|
||||
"""Every source of the config; doubles (same id, DOI or title) and what was reported
|
||||
before (seen) left out. (records, errors)."""
|
||||
keywords = re.compile(config.get('keywords') or r'(?!)', re.I)
|
||||
jobs = []
|
||||
if config.get('pubmed'):
|
||||
jobs.append(('PubMed', lambda: fetch_pubmed(config['pubmed'], days)))
|
||||
if config.get('arxiv'):
|
||||
jobs.append(('arXiv', lambda: fetch_arxiv(config['arxiv'], days)))
|
||||
for server in config.get('rxiv', RXIV_SERVERS) if config.get('keywords') else ():
|
||||
jobs.append((server, lambda s=server: fetch_rxiv(s, days, keywords)))
|
||||
records, errors = [], []
|
||||
for name, fn in jobs:
|
||||
try:
|
||||
records += fn()
|
||||
except Exception as e: # noqa: BLE001 - a source down mustn't stop the others
|
||||
errors.append(f'{name}: {e}')
|
||||
unique, keys = [], set()
|
||||
for r in records:
|
||||
k = {r['id'], (r['doi'] or '').lower(), re.sub(r'\W', '', r['title'].lower())} - {''}
|
||||
if keys & k or r['id'] in seen or (r['doi'] and f"doi:{r['doi']}" in seen):
|
||||
continue
|
||||
keys |= k
|
||||
unique.append(r)
|
||||
return unique, errors
|
||||
379
projects/radar.py
Normal file
379
projects/radar.py
Normal file
|
|
@ -0,0 +1,379 @@
|
|||
"""The Radar of a project (docs/projects.md): once a week the new papers (projects/papers.py)
|
||||
go to Claude in a batch (half price, done within the hour as a rule): one request sorts every
|
||||
paper into the config's groups and writes about the worth-reading ones (JSON schema), one looks
|
||||
for the week's news (web search). The answer becomes a RadarDigest — the bundle format of
|
||||
F:\\neuropolitik\\tools\\digest_batch.py (version 1), so a digest made elsewhere can come in too.
|
||||
|
||||
Runs from the host's crontab: `manage.py radar` every half hour submits what is due and
|
||||
collects what is done. Needs ANTHROPIC_API_KEY in the container's environment.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from datetime import date, timedelta
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from django.db import transaction
|
||||
from django.http import JsonResponse
|
||||
from django.utils import timezone
|
||||
from django.views.decorators.csrf import csrf_exempt
|
||||
from django.views.decorators.http import require_http_methods
|
||||
|
||||
from . import papers
|
||||
from .models import Project, Radar, RadarDigest
|
||||
|
||||
DEFAULT_MODEL = 'claude-opus-5-5'
|
||||
# Batch prices: half of Opus 5.5's list prices ($4 / $20 per million tokens); web search $10 / 1000.
|
||||
PRICE_IN, PRICE_OUT, PRICE_SEARCH = 2.00 / 1e6, 10.00 / 1e6, 10.00 / 1000
|
||||
SEEN_MAX = 20000
|
||||
# The weekday and hour of a config are the user's (the server runs on UTC).
|
||||
ZONE = ZoneInfo('Europe/Berlin')
|
||||
DIGESTS_SHOWN = 12
|
||||
|
||||
|
||||
def papers_schema(groups):
|
||||
return {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'auf_einen_blick': {'type': 'string'},
|
||||
'items': {'type': 'array', 'items': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'id': {'type': 'string'},
|
||||
'gruppe': {'type': 'string', 'enum': list(groups)},
|
||||
'worum_es_geht': {'type': 'string'},
|
||||
'einordnung': {'type': 'string'},
|
||||
'politischer_haken': {'type': 'string'},
|
||||
'kurz': {'type': 'string'},
|
||||
},
|
||||
'required': ['id', 'gruppe', 'worum_es_geht', 'einordnung', 'politischer_haken', 'kurz'],
|
||||
'additionalProperties': False,
|
||||
}},
|
||||
'grundlagenbegriff': {
|
||||
'type': 'object',
|
||||
'properties': {'begriff': {'type': 'string'}, 'erklaerung': {'type': 'string'}},
|
||||
'required': ['begriff', 'erklaerung'],
|
||||
'additionalProperties': False,
|
||||
},
|
||||
},
|
||||
'required': ['auf_einen_blick', 'items', 'grundlagenbegriff'],
|
||||
'additionalProperties': False,
|
||||
}
|
||||
|
||||
|
||||
def _model_record(r):
|
||||
"""What the model needs of a paper to sort it."""
|
||||
return {k: r.get(k) for k in ('id', 'source', 'topic', 'journal', 'first_author', 'n_authors', 'pub_types',
|
||||
'date', 'title', 'abstract', 'institution') if r.get(k) not in (None, '', [])}
|
||||
|
||||
|
||||
def _terms(radar):
|
||||
"""The basic terms explained before: not again."""
|
||||
out = []
|
||||
for b in radar.digests.values_list('bundle', flat=True)[:52]:
|
||||
t = ((b or {}).get('grundlagenbegriff') or {}).get('begriff')
|
||||
if t:
|
||||
out.append(t)
|
||||
return out
|
||||
|
||||
|
||||
def build_requests(config, records, terms, today):
|
||||
model = config.get('model') or DEFAULT_MODEL
|
||||
groups = config.get('groups') or ['lesenswert', 'rauschen']
|
||||
prompt = config['papers_task'].format(
|
||||
n=len(records), glossar=', '.join(terms) if terms else '(noch keine)',
|
||||
records=json.dumps([_model_record(r) for r in records], ensure_ascii=False))
|
||||
reqs = [{'custom_id': 'papers', 'params': {
|
||||
'model': model, 'max_tokens': 48000, 'thinking': {'type': 'adaptive'},
|
||||
'output_config': {'effort': 'high', 'format': {'type': 'json_schema', 'schema': papers_schema(groups)}},
|
||||
'system': config['system'], 'messages': [{'role': 'user', 'content': prompt}],
|
||||
}}]
|
||||
if config.get('news_task'):
|
||||
reqs.append({'custom_id': 'news', 'params': {
|
||||
'model': model, 'max_tokens': 16000, 'thinking': {'type': 'adaptive'},
|
||||
'output_config': {'effort': 'medium'}, 'system': config['system'],
|
||||
'tools': [{'type': 'web_search_20260209', 'name': 'web_search', 'max_uses': 4}],
|
||||
'messages': [{'role': 'user', 'content': config['news_task'].format(today=today)}],
|
||||
}})
|
||||
return reqs
|
||||
|
||||
|
||||
def build_bundle(config, pending, papers_json, news_md, usage, batch_id, project_name=''):
|
||||
groups = config.get('groups') or ['lesenswert', 'rauschen']
|
||||
records = {r['id']: r for r in pending['records']}
|
||||
items, seen = [], set()
|
||||
for it in papers_json.get('items') or []:
|
||||
r = records.get(it.get('id'))
|
||||
if not r or it['id'] in seen or it.get('gruppe') not in groups:
|
||||
continue # an id or group the model made up, or twice: left out
|
||||
seen.add(it['id'])
|
||||
items.append({**r, **it})
|
||||
# What the model forgot shows, unsorted, among the rest.
|
||||
fallback = 'technik' if 'technik' in groups else groups[-1]
|
||||
for rid, r in records.items():
|
||||
if rid not in seen:
|
||||
items.append({**r, 'gruppe': fallback, 'worum_es_geht': '', 'einordnung': '', 'politischer_haken': '',
|
||||
'kurz': '(nicht eingeordnet)'})
|
||||
per_source = {}
|
||||
for r in pending['records']:
|
||||
per_source[r['source']] = per_source.get(r['source'], 0) + 1
|
||||
cost = (usage['input_tokens'] * PRICE_IN + usage['output_tokens'] * PRICE_OUT
|
||||
+ usage['web_search_requests'] * PRICE_SEARCH)
|
||||
return {
|
||||
'version': 1, 'project': project_name, 'generated': pending['generated'], 'days': pending['days'],
|
||||
'model': config.get('model') or DEFAULT_MODEL, 'batch_id': batch_id,
|
||||
'auf_einen_blick': papers_json.get('auf_einen_blick', ''),
|
||||
'grundlagenbegriff': papers_json.get('grundlagenbegriff') or {'begriff': '', 'erklaerung': ''},
|
||||
'items': items, 'news_markdown': news_md,
|
||||
'stats': {'records': len(records), 'per_source': per_source, 'errors': pending.get('errors', []),
|
||||
'usage': usage, 'cost_usd': round(cost, 4)},
|
||||
}
|
||||
|
||||
|
||||
def _client():
|
||||
import anthropic # only where the Radar runs
|
||||
return anthropic.Anthropic()
|
||||
|
||||
|
||||
# ---- running ---------------------------------------------------------------------------------
|
||||
|
||||
def due(radar, now=None):
|
||||
"""Its weekday and hour have come (local time), and this week has no digest yet."""
|
||||
if not radar.enabled or radar.pending or not radar.config.get('papers_task'):
|
||||
return False
|
||||
now = (now or timezone.now()).astimezone(ZONE)
|
||||
c = radar.config
|
||||
weekday, hour = int(c.get('weekday', 6)), int(c.get('hour', 18))
|
||||
start = (now - timedelta(days=(now.weekday() - weekday) % 7)).replace(hour=hour, minute=0, second=0,
|
||||
microsecond=0)
|
||||
if start > now:
|
||||
start -= timedelta(days=7)
|
||||
last = radar.last_run_at.astimezone(ZONE) if radar.last_run_at else None
|
||||
return last is None or last < start
|
||||
|
||||
|
||||
def submit(radar, force=False):
|
||||
"""Fetch the new papers and send the batch. Returns what happened, in words."""
|
||||
if radar.pending and not force:
|
||||
return 'Es läuft schon ein Abruf.'
|
||||
c = radar.config
|
||||
days = int(c.get('days', 7))
|
||||
records, errors = papers.collect(c, days, set(radar.seen))
|
||||
radar.last_run_at = timezone.now()
|
||||
today = date.today().isoformat()
|
||||
if not records:
|
||||
radar.last_error = '; '.join(errors)
|
||||
radar.pending = None
|
||||
radar.save(update_fields=['last_run_at', 'last_error', 'pending'])
|
||||
return 'Keine neuen Papers.' + (' Fehler: ' + radar.last_error if errors else '')
|
||||
batch = _client().messages.batches.create(requests=build_requests(c, records, _terms(radar), today))
|
||||
radar.pending = {'batch_id': batch.id, 'generated': today, 'days': days, 'records': records,
|
||||
'errors': errors, 'submitted': timezone.now().isoformat(timespec='seconds')}
|
||||
radar.last_error = ''
|
||||
radar.save(update_fields=['last_run_at', 'last_error', 'pending'])
|
||||
return f'{len(records)} Papers an Claude geschickt ({batch.id}).'
|
||||
|
||||
|
||||
def _text(message):
|
||||
return ''.join(b.text for b in message.content if b.type == 'text')
|
||||
|
||||
|
||||
def _add_usage(total, message):
|
||||
u = message.usage
|
||||
total['input_tokens'] += (u.input_tokens or 0) + (getattr(u, 'cache_read_input_tokens', 0) or 0) \
|
||||
+ (getattr(u, 'cache_creation_input_tokens', 0) or 0)
|
||||
total['output_tokens'] += u.output_tokens or 0
|
||||
stu = getattr(u, 'server_tool_use', None)
|
||||
if stu:
|
||||
total['web_search_requests'] += getattr(stu, 'web_search_requests', 0) or 0
|
||||
|
||||
|
||||
def collect(radar):
|
||||
"""The batch's answer, if it's there: a digest. Returns 'done', 'waiting' or an error text."""
|
||||
p = radar.pending
|
||||
if not p:
|
||||
return 'done'
|
||||
client = _client()
|
||||
batch = client.messages.batches.retrieve(p['batch_id'])
|
||||
if batch.processing_status != 'ended':
|
||||
return 'waiting'
|
||||
usage = {'input_tokens': 0, 'output_tokens': 0, 'web_search_requests': 0}
|
||||
papers_json, news_md, problems = None, '', []
|
||||
for res in client.messages.batches.results(batch.id):
|
||||
if res.result.type != 'succeeded':
|
||||
detail = getattr(getattr(res.result, 'error', None), 'type', '')
|
||||
problems.append(f'{res.custom_id}: {res.result.type} {detail}'.strip())
|
||||
continue
|
||||
msg = res.result.message
|
||||
_add_usage(usage, msg)
|
||||
if msg.stop_reason == 'refusal':
|
||||
problems.append(f'{res.custom_id}: abgelehnt')
|
||||
elif res.custom_id == 'papers':
|
||||
if msg.stop_reason == 'max_tokens':
|
||||
problems.append('papers: max_tokens erreicht, Ausgabe unvollständig')
|
||||
else:
|
||||
try:
|
||||
papers_json = json.loads(_text(msg))
|
||||
except ValueError:
|
||||
problems.append('papers: keine gültige JSON-Antwort')
|
||||
elif res.custom_id == 'news':
|
||||
news_md = _text(msg)
|
||||
if papers_json is None:
|
||||
# Nothing remembered as seen: the next run gets the papers again.
|
||||
radar.pending = None
|
||||
radar.last_error = 'Einordnung fehlgeschlagen: ' + '; '.join(problems)
|
||||
radar.save(update_fields=['pending', 'last_error'])
|
||||
return radar.last_error
|
||||
p['errors'] = list(p.get('errors') or []) + problems
|
||||
bundle = build_bundle(radar.config, p, papers_json, news_md, usage, batch.id)
|
||||
store(radar, bundle, mark_seen=[r['id'] for r in p['records']])
|
||||
return 'done'
|
||||
|
||||
|
||||
def store(radar, bundle, mark_seen=None):
|
||||
"""A digest in; what it reports counts as seen. Replaces one with the same batch."""
|
||||
with transaction.atomic():
|
||||
if bundle.get('batch_id'):
|
||||
radar.digests.filter(batch_id=bundle['batch_id']).delete()
|
||||
d = RadarDigest.objects.create(radar=radar, generated=date.fromisoformat(str(bundle['generated'])[:10]),
|
||||
batch_id=bundle.get('batch_id') or '', bundle=bundle)
|
||||
seen = list(dict.fromkeys(list(radar.seen) + list(mark_seen or [i.get('id') for i in bundle.get('items', [])])))
|
||||
radar.seen = [s for s in seen if s][-SEEN_MAX:]
|
||||
radar.pending = None
|
||||
radar.last_error = ''
|
||||
radar.save(update_fields=['seen', 'pending', 'last_error'])
|
||||
return d
|
||||
|
||||
|
||||
def tick(now=None):
|
||||
"""What the cron job does: collect what's done, submit what's due. Returns lines to log."""
|
||||
out = []
|
||||
for radar in Radar.objects.filter(enabled=True).select_related('project'):
|
||||
try:
|
||||
if radar.pending:
|
||||
r = collect(radar)
|
||||
if r != 'waiting':
|
||||
out.append(f'radar {radar.project_id}: {r}')
|
||||
elif due(radar, now):
|
||||
out.append(f'radar {radar.project_id}: {submit(radar)}')
|
||||
except Exception as e: # noqa: BLE001 - one radar's trouble mustn't stop the others
|
||||
radar.last_error = str(e)[:2000]
|
||||
radar.save(update_fields=['last_error'])
|
||||
out.append(f'radar {radar.project_id}: Fehler {e}')
|
||||
return out
|
||||
|
||||
|
||||
# ---- API ---------------------------------------------------------------------------------------
|
||||
|
||||
def _auth(request):
|
||||
if not request.user.is_authenticated:
|
||||
return JsonResponse({'error': 'authentication required'}, status=401)
|
||||
return None
|
||||
|
||||
|
||||
def next_run(radar, now=None):
|
||||
"""When the cron job will fetch next: now (it's due), else the coming weekday and hour."""
|
||||
if not radar.enabled:
|
||||
return None
|
||||
now = (now or timezone.now()).astimezone(ZONE)
|
||||
if radar.pending:
|
||||
return None
|
||||
if due(radar, now):
|
||||
return now.isoformat()
|
||||
c = radar.config
|
||||
weekday, hour = int(c.get('weekday', 6)), int(c.get('hour', 18))
|
||||
t = (now + timedelta(days=(weekday - now.weekday()) % 7)).replace(hour=hour, minute=0, second=0, microsecond=0)
|
||||
if t <= now:
|
||||
t += timedelta(days=7)
|
||||
return t.isoformat()
|
||||
|
||||
|
||||
def _radar_json(radar):
|
||||
c = radar.config
|
||||
return {'enabled': radar.enabled, 'title': c.get('title', 'Radar'), 'groups': c.get('groups', []),
|
||||
'group_labels': c.get('group_labels', {}), 'weekday': c.get('weekday', 6), 'hour': c.get('hour', 18),
|
||||
'days': c.get('days', 7), 'sources': [s for s, on in (('PubMed', c.get('pubmed')), ('arXiv', c.get('arxiv')),
|
||||
('bioRxiv/medRxiv', c.get('keywords'))) if on],
|
||||
'running': bool(radar.pending), 'submitted': (radar.pending or {}).get('submitted'),
|
||||
'last_run_at': radar.last_run_at.isoformat() if radar.last_run_at else None,
|
||||
'last_error': radar.last_error, 'next_run': next_run(radar),
|
||||
'key': bool(os.environ.get('ANTHROPIC_API_KEY'))}
|
||||
|
||||
|
||||
def _digest_json(d):
|
||||
return {'id': d.id, 'generated': d.generated.isoformat(), 'bundle': d.bundle}
|
||||
|
||||
|
||||
@require_http_methods(['GET'])
|
||||
def radar_view(request, pk):
|
||||
"""The project's Radar and its latest digests (or radar: null)."""
|
||||
err = _auth(request)
|
||||
if err:
|
||||
return err
|
||||
p = Project.objects.filter(pk=pk, user=request.user).first()
|
||||
if not p:
|
||||
return JsonResponse({'error': 'not found'}, status=404)
|
||||
radar = Radar.objects.filter(project=p).first()
|
||||
if not radar:
|
||||
return JsonResponse({'radar': None, 'digests': []})
|
||||
return JsonResponse({'radar': _radar_json(radar),
|
||||
'digests': [_digest_json(d) for d in radar.digests.all()[:DIGESTS_SHOWN]]})
|
||||
|
||||
|
||||
@csrf_exempt
|
||||
@require_http_methods(['POST'])
|
||||
def radar_settings(request, pk):
|
||||
"""{enabled}."""
|
||||
err = _auth(request)
|
||||
if err:
|
||||
return err
|
||||
radar = Radar.objects.filter(project__pk=pk, project__user=request.user).first()
|
||||
if not radar:
|
||||
return JsonResponse({'error': 'not found'}, status=404)
|
||||
try:
|
||||
body = json.loads(request.body or b'{}')
|
||||
except ValueError:
|
||||
return JsonResponse({'error': 'invalid JSON'}, status=400)
|
||||
if 'enabled' in body:
|
||||
radar.enabled = bool(body['enabled'])
|
||||
radar.save(update_fields=['enabled'])
|
||||
return JsonResponse({'radar': _radar_json(radar)})
|
||||
|
||||
|
||||
@csrf_exempt
|
||||
@require_http_methods(['POST'])
|
||||
def radar_run(request, pk):
|
||||
"""Fetch now rather than on the weekday: the cron job takes it within half an hour."""
|
||||
err = _auth(request)
|
||||
if err:
|
||||
return err
|
||||
radar = Radar.objects.filter(project__pk=pk, project__user=request.user).first()
|
||||
if not radar:
|
||||
return JsonResponse({'error': 'not found'}, status=404)
|
||||
if not radar.pending:
|
||||
radar.last_run_at = None
|
||||
radar.enabled = True
|
||||
radar.save(update_fields=['last_run_at', 'enabled'])
|
||||
return JsonResponse({'radar': _radar_json(radar)})
|
||||
|
||||
|
||||
@csrf_exempt
|
||||
@require_http_methods(['POST'])
|
||||
def radar_import(request, pk):
|
||||
"""A digest made elsewhere (digest_batch.py --diora-url, Bearer token): the bundle."""
|
||||
err = _auth(request)
|
||||
if err:
|
||||
return err
|
||||
radar = Radar.objects.filter(project__pk=pk, project__user=request.user).first()
|
||||
if not radar:
|
||||
return JsonResponse({'error': 'not found'}, status=404)
|
||||
try:
|
||||
bundle = json.loads(request.body or b'{}')
|
||||
date.fromisoformat(str(bundle['generated'])[:10])
|
||||
if bundle.get('version') != 1 or not isinstance(bundle.get('items'), list):
|
||||
raise ValueError
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return JsonResponse({'error': 'kein Digest (version 1)'}, status=400)
|
||||
if len(request.body) > 5_000_000:
|
||||
return JsonResponse({'error': 'zu groß'}, status=400)
|
||||
d = store(radar, bundle)
|
||||
return JsonResponse({'digest': d.id})
|
||||
120
projects/radar_presets.py
Normal file
120
projects/radar_presets.py
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
"""Radar configurations (docs/projects.md): what a project's Radar searches and how Claude
|
||||
sorts it. NEUROPOLITIK is taken as it was from F:\neuropolitik\tools (fetch_papers.py,
|
||||
digest_batch.py, the "Neuropolitik Grundlagen" session); generated, edit with care: papers_task
|
||||
keeps {n}, {glossar} and {records}, news_task {today}."""
|
||||
|
||||
# flake8: noqa: E501
|
||||
NEUROPOLITIK = {'title': 'Neuropolitik-Digest',
|
||||
'model': 'claude-opus-5-5',
|
||||
'days': 7,
|
||||
'weekday': 6,
|
||||
'hour': 18,
|
||||
'pubmed': {'technik': '"brain-computer interface*"[tiab] OR "brain computer interface*"[tiab] OR '
|
||||
'"brain-machine interface*"[tiab] OR "neural interface*"[tiab] OR neuroprosthe*[tiab] '
|
||||
'OR "speech decoding"[tiab] OR "Brain-Computer Interfaces"[Mesh]',
|
||||
'gesellschaft': 'neuroethic*[tiab] OR neuroright*[tiab] OR "neural data"[tiab] OR "mental '
|
||||
'privacy"[tiab] OR "cognitive liberty"[tiab] OR "neurotechnolog*"[tiab] OR '
|
||||
'"neural privacy"[tiab]',
|
||||
'propriozeption': '(propriocept*[tiab] OR "sensory feedback"[tiab] OR "somatosensory '
|
||||
'feedback"[tiab] OR "artificial touch"[tiab] OR "intracortical '
|
||||
'microstimulation"[tiab] OR embodiment[tiab] OR "body schema"[tiab]) AND '
|
||||
'(prosthe*[tiab] OR neuroprosthe*[tiab] OR bionic[tiab] OR "neural '
|
||||
'interface*"[tiab] OR "brain-computer"[tiab] OR "brain-machine"[tiab] OR '
|
||||
'"nerve stimulation"[tiab] OR "spinal cord stimulation"[tiab] OR '
|
||||
'exoskeleton*[tiab])'},
|
||||
'arxiv': 'abs:"brain-computer interface" OR abs:"brain-machine interface" OR abs:"brain computer interface" '
|
||||
'OR abs:"neural interface" OR abs:neurotechnology OR abs:"neural decoding" OR abs:neuroprosthesis '
|
||||
'OR (abs:proprioception AND (abs:prosthesis OR abs:prosthetic OR abs:"neural interface")) OR '
|
||||
'abs:"intracortical microstimulation"',
|
||||
'keywords': 'brain[- ]?(computer|machine) interface|\\bBCIs?\\b|neuroprosthe|neural interface|intracortical '
|
||||
'(decod|record|microstim|array|implant)|neural decod|speech decod|brain '
|
||||
'decod|neurotechnolog|neuroethic|neuroright|neural data|mental privacy|cognitive '
|
||||
'liberty|closed[- ]loop (neuro)?(stimulation|modulation)|adaptive deep brain '
|
||||
'stimulation|brain[- ]to[- ](text|speech|image)|intracortical microstimulation|artificial '
|
||||
'(touch|sensation|proprioception)|propriocept\\w* '
|
||||
'(feedback|restoration|neuroprosthe)|bidirectional (BCI|brain|neural)|bionic '
|
||||
'(limb|leg|arm|hand)|myoneural|somatosensory (feedback|neuroprosthe)|(sensory|somatosensory) '
|
||||
'feedback[^.]{0,60}prosthe',
|
||||
'rxiv': ['biorxiv', 'medrxiv'],
|
||||
'groups': ['lesenswert', 'propriozeption', 'gesellschaft', 'technik', 'rauschen'],
|
||||
'group_labels': {'lesenswert': 'Lesenswert',
|
||||
'propriozeption': 'Propriozeption und Körper',
|
||||
'gesellschaft': 'Gesellschaft, Ethik, Recht',
|
||||
'technik': 'Technik im Überblick',
|
||||
'rauschen': 'Rauschen'},
|
||||
'system': 'Du schreibst den wöchentlichen Digest für „Neuropolitik“, ein geplantes Publikationsprojekt '
|
||||
'(Blog/Podcast) über Brain-Computer-Interfaces (BCIs), Neurotechnologie, Propriozeption und '
|
||||
'Neuroprothetik und ihre Auswirkungen auf die Gesellschaft – aus einer linken Perspektive '
|
||||
'(Marxismus, Ljubljana-Schule: Žižek, Zupančič, Dolar; dazu Malabou, Johnston, Pasquinelli, '
|
||||
'Rose/Abi-Rached).\n'
|
||||
'\n'
|
||||
'Die Person hinter dem Projekt ist theoretisch geschult, hat aber keine neurowissenschaftliche '
|
||||
'Ausbildung und arbeitet sich gerade in die Grundlagen ein. Theorie musst du nicht erklären, '
|
||||
'Technik schon – knapp und präzise.\n'
|
||||
'\n'
|
||||
'Ein eigener Schwerpunkt ist Propriozeption: künstliche Rückmeldung von Körperlage und Berührung '
|
||||
'(intrakortikale Mikrostimulation, bidirektionale BCIs, Nervenschnittstellen in Prothesen, '
|
||||
'myoneurale Schnittstellen), Embodiment und Körperschema bei Prothesen und Exoskeletten, Verlust '
|
||||
'und Wiederherstellung der Eigenwahrnehmung.\n'
|
||||
'\n'
|
||||
'Regeln:\n'
|
||||
'- Schreib auf Deutsch, sachlich und präzise. Die linke Perspektive gehört nur in den '
|
||||
'„politischen Haken“.\n'
|
||||
'- Behaupte nichts, was nicht in der Quelle steht. Du kennst von den Papers nur Titel und '
|
||||
'(gekürztes) Abstract – schreib „laut Abstract“, wo das wichtig ist. Erfinde keine Zahlen, '
|
||||
'Autor:innen oder Ergebnisse.\n'
|
||||
'- Gib Abstracts nicht wörtlich wieder, sondern fasse in eigenen Worten zusammen.',
|
||||
'papers_task': 'Ordne jedes der folgenden {n} Papers genau einer Gruppe zu und schreib die passenden Texte. '
|
||||
'Jede ID muss genau einmal in `items` vorkommen.\n'
|
||||
'\n'
|
||||
'Gruppen:\n'
|
||||
'- lesenswert (höchstens 6, möglichst 3–6): Studien an Menschen und klinische BCI-Studien; '
|
||||
'alles zu Propriozeption und sensorischer Rückmeldung; Decoding von Sprache, innerer '
|
||||
'Sprache, Bildern oder Bedeutung (besonders nicht-invasiv: EEG, fMRT, MEG, fNIRS); '
|
||||
'„schreibende“ Neurotechnik (Stimulation, Closed-Loop, Tiefe Hirnstimulation mit BCI-Bezug); '
|
||||
'Durchbrüche bei Implantaten; Papers kommerzieller Akteure (Neuralink, Synchron, '
|
||||
'Paradromics, Precision Neuroscience, Blackrock Neurotech, Merge Labs, Meta, Apple, '
|
||||
'chinesische BCI-Firmen); Einsätze in Militär, Arbeitswelt oder Konsumprodukten; neuronale '
|
||||
'Daten; große Reviews. Was politisch oder gesellschaftlich Gewicht hat, geht vor rein '
|
||||
'inkrementeller Technik.\n'
|
||||
'- propriozeption: Papers mit Propriozeptions-Bezug, die nicht lesenswert sind.\n'
|
||||
'- gesellschaft: Ethik, Recht, Regulierung, Neurorechte, Datenschutz, Disability Studies, '
|
||||
'Wissenschaftsforschung (STS), Philosophie.\n'
|
||||
'- technik: übrige relevante BCI- und Neuroprothetik-Technik.\n'
|
||||
'- rauschen: kein echter BCI- oder Neurotech-Bezug (Schlagwort-Fehltreffer) oder sehr '
|
||||
'inkrementelle ML-Arbeiten zur EEG-Klassifikation.\n'
|
||||
'\n'
|
||||
'Felder je Paper:\n'
|
||||
'- lesenswert: `worum_es_geht` (2–3 Sätze), `einordnung` (Studientyp: Peer Review oder '
|
||||
'Preprint, Mensch/Tier/Simulation, Zahl der Teilnehmenden falls genannt; invasiv oder '
|
||||
'nicht-invasiv; lesend oder schreibend; Hype-Check: was zeigt das Paper wirklich, was '
|
||||
'nicht), `politischer_haken` (1–2 Sätze als Anregung, nicht als fertige These: Eigentum und '
|
||||
'Plattformabhängigkeit, Arbeit und Überwachung, neuronale Daten als Ware, Militär, '
|
||||
'Behinderung und Legitimationsnarrative, Ideologie des „Gehirnsubjekts“, Gedankenlesen und '
|
||||
'Subjektivität, Körperschema und Eigenleib …) und `kurz` (ein Satz).\n'
|
||||
'- propriozeption, gesellschaft: `kurz` (1–2 Sätze), die übrigen Felder leer.\n'
|
||||
'- technik: `kurz` (eine Zeile: was daran neu ist), die übrigen Felder leer.\n'
|
||||
'- rauschen: alle Textfelder leer.\n'
|
||||
'\n'
|
||||
'Außerdem:\n'
|
||||
'- `auf_einen_blick`: 3–5 Sätze – was war diese Woche wichtig, gibt es einen Trend?\n'
|
||||
'- `grundlagenbegriff`: Erkläre in 3–5 Sätzen einen technischen Begriff, der in den Papers '
|
||||
'dieser Woche vorkommt und für Leser:innen mit Theorie-, aber ohne Neuro-Hintergrund wichtig '
|
||||
'ist. Nicht wiederholen: {glossar}.\n'
|
||||
'\n'
|
||||
'Bei einer ruhigen Woche fällt alles kürzer aus. Nichts künstlich aufblähen.\n'
|
||||
'\n'
|
||||
'Papers (JSON):\n'
|
||||
'{records}',
|
||||
'news_task': 'Recherchiere mit höchstens 4 Websuchen Nachrichten der letzten 7 Tage (Stichtag {today}) zu '
|
||||
'Neurotechnologie und BCIs: Neuralink, Synchron und andere BCI-Firmen samt Finanzierung, '
|
||||
'Gesetze zu neuronalen Daten und Neurorechten (USA, EU, Chile, UNESCO), Chinas '
|
||||
'BCI-Industriepolitik, DARPA und Militär.\n'
|
||||
'\n'
|
||||
'Antworte nur mit einer Markdown-Liste, ohne Einleitung, ein Eintrag pro Meldung:\n'
|
||||
'- **Kurztitel:** 1–2 Sätze. [Quelle](URL)\n'
|
||||
'\n'
|
||||
'Nimm nur Meldungen auf, deren Quelle du in den Suchergebnissen gesehen hast, und nur aus den '
|
||||
'letzten 7 Tagen. Gibt es nichts Relevantes, antworte genau: Keine relevanten Meldungen.'}
|
||||
|
||||
PRESETS = {'neuropolitik': NEUROPOLITIK}
|
||||
144
projects/test_radar.py
Normal file
144
projects/test_radar.py
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
"""The Radar (projects/radar.py, projects/papers.py) with a pretend Claude."""
|
||||
import json
|
||||
from datetime import datetime
|
||||
from types import SimpleNamespace as NS
|
||||
from unittest.mock import patch
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from django.contrib.auth.models import User
|
||||
from django.test import TestCase
|
||||
|
||||
from . import papers, radar
|
||||
from .models import Project, Radar
|
||||
from .radar_presets import NEUROPOLITIK
|
||||
|
||||
BERLIN = ZoneInfo('Europe/Berlin')
|
||||
REC = [
|
||||
{'id': 'pmid:1', 'doi': '10.1/a', 'source': 'PubMed', 'topic': 'technik', 'title': 'Speech BCI',
|
||||
'journal': 'Nature', 'first_author': 'Willett', 'n_authors': 9, 'pub_types': [], 'date': '2026',
|
||||
'url': 'https://pubmed.ncbi.nlm.nih.gov/1/', 'abstract': 'A speech neuroprosthesis.'},
|
||||
{'id': 'arxiv:2610.1', 'doi': None, 'source': 'arXiv', 'topic': None, 'title': 'EEG noise',
|
||||
'journal': 'arXiv eess.SP', 'first_author': 'A', 'n_authors': 2, 'pub_types': ['Preprint'],
|
||||
'date': '2026-10-18', 'url': 'https://arxiv.org/abs/2610.1', 'abstract': 'EEG.'},
|
||||
]
|
||||
|
||||
|
||||
def message(text, stop='end_turn', searches=0):
|
||||
return NS(content=[NS(type='text', text=text)], stop_reason=stop,
|
||||
usage=NS(input_tokens=1000, output_tokens=200, cache_read_input_tokens=0, cache_creation_input_tokens=0,
|
||||
server_tool_use=NS(web_search_requests=searches) if searches else None))
|
||||
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, status='ended', papers_text=None):
|
||||
self.created = []
|
||||
self.status = status
|
||||
self.papers_text = papers_text or json.dumps({
|
||||
'auf_einen_blick': 'Eine ruhige Woche.',
|
||||
'grundlagenbegriff': {'begriff': 'Decoder', 'erklaerung': 'Ein Modell, das …'},
|
||||
'items': [{'id': 'pmid:1', 'gruppe': 'lesenswert', 'worum_es_geht': 'Sprache.', 'einordnung': 'Peer Review.',
|
||||
'politischer_haken': 'Stimme.', 'kurz': 'Sprache aus dem Kortex.'},
|
||||
{'id': 'erfunden', 'gruppe': 'technik', 'worum_es_geht': '', 'einordnung': '',
|
||||
'politischer_haken': '', 'kurz': 'x'}]})
|
||||
outer = self
|
||||
|
||||
class Batches:
|
||||
def create(self, requests):
|
||||
outer.created.append(requests)
|
||||
return NS(id='msgbatch_1', processing_status='in_progress')
|
||||
|
||||
def retrieve(self, bid):
|
||||
return NS(id=bid, processing_status=outer.status)
|
||||
|
||||
def results(self, bid):
|
||||
return [NS(custom_id='papers', result=NS(type='succeeded', message=message(outer.papers_text))),
|
||||
NS(custom_id='news', result=NS(type='succeeded', message=message('- **X:** y. [Q](https://q)', searches=2)))]
|
||||
self.messages = NS(batches=Batches())
|
||||
|
||||
|
||||
class RadarTests(TestCase):
|
||||
def setUp(self):
|
||||
self.user = User.objects.create_user(username='radar', password='pw12345678')
|
||||
self.client.force_login(self.user)
|
||||
self.project = Project.objects.create(user=self.user, data_ct='x', data_iv='a1' * 12)
|
||||
self.radar = Radar.objects.create(project=self.project, config=NEUROPOLITIK)
|
||||
|
||||
def test_due_on_its_weekday_once_a_week(self):
|
||||
sat = datetime(2026, 10, 24, 20, 0, tzinfo=BERLIN)
|
||||
sun_early = datetime(2026, 10, 25, 17, 0, tzinfo=BERLIN)
|
||||
sun = datetime(2026, 10, 25, 18, 5, tzinfo=BERLIN)
|
||||
self.radar.last_run_at = datetime(2026, 10, 18, 18, 1, tzinfo=BERLIN)
|
||||
self.assertFalse(radar.due(self.radar, sat))
|
||||
self.assertFalse(radar.due(self.radar, sun_early))
|
||||
self.assertTrue(radar.due(self.radar, sun))
|
||||
self.radar.last_run_at = sun
|
||||
self.assertFalse(radar.due(self.radar, datetime(2026, 10, 25, 23, 0, tzinfo=BERLIN)))
|
||||
self.radar.last_run_at = None
|
||||
self.assertTrue(radar.due(self.radar, sat), 'never run: at once')
|
||||
|
||||
def test_submit_and_collect(self):
|
||||
fake = FakeClient(status='in_progress')
|
||||
with patch.object(radar, '_client', return_value=fake), \
|
||||
patch.object(papers, 'collect', return_value=(list(REC), ['medrxiv: down'])):
|
||||
self.assertIn('2 Papers', radar.submit(self.radar))
|
||||
reqs = fake.created[0]
|
||||
self.assertEqual([r['custom_id'] for r in reqs], ['papers', 'news'])
|
||||
self.assertIn('Speech BCI', reqs[0]['params']['messages'][0]['content'])
|
||||
self.assertEqual(radar.collect(self.radar), 'waiting')
|
||||
fake.status = 'ended'
|
||||
self.assertEqual(radar.collect(self.radar), 'done')
|
||||
self.radar.refresh_from_db()
|
||||
self.assertIsNone(self.radar.pending)
|
||||
self.assertEqual(set(self.radar.seen), {'pmid:1', 'arxiv:2610.1'})
|
||||
b = self.radar.digests.get().bundle
|
||||
self.assertEqual({i['id']: i['gruppe'] for i in b['items']}, {'pmid:1': 'lesenswert', 'arxiv:2610.1': 'technik'})
|
||||
self.assertEqual(b['items'][0]['title'], 'Speech BCI', 'titles from the sources, not the model')
|
||||
self.assertEqual(b['stats']['usage'], {'input_tokens': 2000, 'output_tokens': 400, 'web_search_requests': 2})
|
||||
self.assertEqual(b['stats']['errors'], ['medrxiv: down'])
|
||||
# The next week the basic term isn't asked for again.
|
||||
self.assertIn('Decoder', radar.build_requests(NEUROPOLITIK, REC, radar._terms(self.radar), '2026-10-26')[0]
|
||||
['params']['messages'][0]['content'])
|
||||
|
||||
def test_a_failed_sorting_forgets_nothing(self):
|
||||
fake = FakeClient(papers_text='kein json')
|
||||
self.radar.pending = {'batch_id': 'b', 'generated': '2026-10-19', 'days': 7, 'records': REC, 'errors': []}
|
||||
self.radar.save()
|
||||
with patch.object(radar, '_client', return_value=fake):
|
||||
self.assertIn('fehlgeschlagen', radar.collect(self.radar))
|
||||
self.radar.refresh_from_db()
|
||||
self.assertEqual((self.radar.seen, self.radar.pending, self.radar.digests.count()), ([], None, 0))
|
||||
|
||||
def test_nothing_new_no_batch(self):
|
||||
fake = FakeClient()
|
||||
with patch.object(radar, '_client', return_value=fake), patch.object(papers, 'collect', return_value=([], [])):
|
||||
self.assertEqual(radar.submit(self.radar), 'Keine neuen Papers.')
|
||||
self.assertEqual(fake.created, [])
|
||||
|
||||
def test_api_view_run_import(self):
|
||||
r = self.client.get(f'/projects/{self.project.id}/radar/').json()
|
||||
self.assertEqual((r['radar']['title'], r['digests']), ('Neuropolitik-Digest', []))
|
||||
bundle = {'version': 1, 'generated': '2026-10-19', 'items': [{'id': 'pmid:9', 'gruppe': 'technik'}],
|
||||
'batch_id': 'msgbatch_x'}
|
||||
self.client.post(f'/projects/{self.project.id}/radar/import/', json.dumps(bundle), content_type='application/json')
|
||||
self.client.post(f'/projects/{self.project.id}/radar/import/', json.dumps(bundle), content_type='application/json')
|
||||
r = self.client.get(f'/projects/{self.project.id}/radar/').json()
|
||||
self.assertEqual(len(r['digests']), 1, 'the same batch once')
|
||||
self.assertIn('pmid:9', Radar.objects.get().seen)
|
||||
self.assertEqual(self.client.post(f'/projects/{self.project.id}/radar/import/', '{}',
|
||||
content_type='application/json').status_code, 400)
|
||||
r = self.client.post(f'/projects/{self.project.id}/radar/run/').json()
|
||||
self.assertTrue(radar.due(Radar.objects.get()))
|
||||
other = User.objects.create_user(username='x', password='pw12345678')
|
||||
self.client.force_login(other)
|
||||
self.assertEqual(self.client.get(f'/projects/{self.project.id}/radar/').status_code, 404)
|
||||
|
||||
|
||||
class PapersTests(TestCase):
|
||||
def test_collect_leaves_out_doubles_and_seen(self):
|
||||
dup = dict(REC[0], id='doi:10.1/a', source='biorxiv')
|
||||
with patch.object(papers, 'fetch_pubmed', return_value=[REC[0]]), \
|
||||
patch.object(papers, 'fetch_arxiv', return_value=[REC[1]]), \
|
||||
patch.object(papers, 'fetch_rxiv', side_effect=[[dup], OSError('down')]):
|
||||
got, errors = papers.collect(NEUROPOLITIK, 7, seen={'arxiv:2610.1'})
|
||||
self.assertEqual([r['id'] for r in got], ['pmid:1'])
|
||||
self.assertEqual(errors, ['medrxiv: down'])
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
from django.urls import path
|
||||
|
||||
from . import views
|
||||
from . import radar, views
|
||||
|
||||
urlpatterns = [
|
||||
path('', views.projects, name='projects'),
|
||||
|
|
@ -8,6 +8,10 @@ urlpatterns = [
|
|||
path('doi/', views.doi, name='projects_doi'),
|
||||
path('<int:pk>/', views.project, name='projects_project'),
|
||||
path('<int:pk>/delete/', views.delete, name='projects_delete'),
|
||||
path('<int:pk>/radar/', radar.radar_view, name='projects_radar'),
|
||||
path('<int:pk>/radar/settings/', radar.radar_settings, name='projects_radar_settings'),
|
||||
path('<int:pk>/radar/run/', radar.radar_run, name='projects_radar_run'),
|
||||
path('<int:pk>/radar/import/', radar.radar_import, name='projects_radar_import'),
|
||||
path('<int:pk>/items/add/', views.item_add, name='projects_item_add'),
|
||||
path('items/<int:pk>/', views.item_update, name='projects_item_update'),
|
||||
path('items/<int:pk>/delete/', views.item_delete, name='projects_item_delete'),
|
||||
|
|
|
|||
|
|
@ -23,3 +23,5 @@ imageio-ffmpeg==0.6.0
|
|||
trafilatura==2.3.0
|
||||
# Subscriber logins of the feed reader are kept encrypted (feeds/logins.py).
|
||||
cryptography==50.0.1
|
||||
# The Radar of a project sorts the week's papers with Claude (Batch API, projects/radar.py).
|
||||
anthropic==1.15.0
|
||||
|
|
|
|||
|
|
@ -201,7 +201,9 @@
|
|||
if (routing) return;
|
||||
let hash = current;
|
||||
if (current === 'radio') hash += '/' + (load('diora_active_radio_tab') || 'saved');
|
||||
if (location.hash.slice(1) !== hash) history.pushState(null, '', '#' + hash);
|
||||
// A place inside the section (#projekte/12) stays: the section's script reads it.
|
||||
const now = location.hash.slice(1);
|
||||
if (now !== hash && now.indexOf(current + '/') !== 0) history.pushState(null, '', '#' + hash);
|
||||
}
|
||||
|
||||
function route() {
|
||||
|
|
|
|||
|
|
@ -19,7 +19,8 @@
|
|||
research: {
|
||||
label: 'Forschung',
|
||||
hint: 'Papers, Bücher, Folgen und Artikel als Quellen, Zitationen, Notizen.',
|
||||
tabs: [['sources', 'Quellen'], ['notes', 'Notizen']],
|
||||
// Radar only where the project has one (a server-side config, docs/projects.md).
|
||||
tabs: [['radar', 'Radar'], ['sources', 'Quellen'], ['notes', 'Notizen']],
|
||||
},
|
||||
};
|
||||
const KIND = {
|
||||
|
|
@ -340,10 +341,14 @@
|
|||
const d = await api('/projects/' + id + '/');
|
||||
const items = [];
|
||||
for (const i of d.items) items.push(Object.assign({}, i, {iv: i.data_iv, data: await open_(i.data_ct, i.data_iv)}));
|
||||
let radar = {radar: null, digests: []};
|
||||
try { radar = await api('/projects/' + id + '/radar/'); } catch (e) {}
|
||||
st.open = {id: d.project.id, preset: d.project.preset, archived: d.project.archived, iv: d.project.data_iv,
|
||||
data: await open_(d.project.data_ct, d.project.data_iv), items: items};
|
||||
data: await open_(d.project.data_ct, d.project.data_iv), items: items, radar: radar.radar, digests: radar.digests || []};
|
||||
if (!keepView) {
|
||||
st.view = 'sources';
|
||||
st.view = st.open.radar ? 'radar' : 'sources';
|
||||
st.digest = 0;
|
||||
st.radarShow = 'all';
|
||||
st.filter = 'all';
|
||||
st.expanded = null;
|
||||
st.note = null;
|
||||
|
|
@ -628,7 +633,7 @@
|
|||
}
|
||||
|
||||
function sources() {
|
||||
return st.open.items.filter(i => i.kind !== 'note');
|
||||
return st.open.items.filter(i => SOURCE_KINDS.indexOf(i.kind) >= 0);
|
||||
}
|
||||
|
||||
function notes() {
|
||||
|
|
@ -651,10 +656,11 @@
|
|||
'<button type="button" class="prj-btn" data-act="archive">' + (p.archived ? 'Aus dem Archiv holen' : 'Archivieren') + '</button>' +
|
||||
'<button type="button" class="prj-btn prj-danger" data-act="delete">Löschen …</button></div>';
|
||||
}
|
||||
h += '<div class="prj-tabs">' + preset.tabs.map(t => '<button type="button" class="prj-tab' + (st.view === t[0] ? ' on' : '') +
|
||||
'" data-view="' + t[0] + '">' + esc(t[1]) + '<span class="prj-count">' +
|
||||
(t[0] === 'sources' ? sources().length : notes().length) + '</span></button>').join('') + '</div>';
|
||||
h += st.view === 'notes' ? notesHtml() : sourcesHtml();
|
||||
const count = id => id === 'sources' ? sources().length : id === 'notes' ? notes().length : radarNew();
|
||||
h += '<div class="prj-tabs">' + preset.tabs.filter(t => t[0] !== 'radar' || p.radar).map(t =>
|
||||
'<button type="button" class="prj-tab' + (st.view === t[0] ? ' on' : '') + '" data-view="' + t[0] + '">' + esc(t[1]) +
|
||||
'<span class="prj-count">' + (count(t[0]) || '') + '</span></button>').join('') + '</div>';
|
||||
h += st.view === 'notes' ? notesHtml() : st.view === 'radar' && p.radar ? radarHtml() : sourcesHtml();
|
||||
return h;
|
||||
}
|
||||
|
||||
|
|
@ -727,6 +733,174 @@
|
|||
'<button type="button" class="prj-btn prj-danger" data-remove="' + i.id + '">Entfernen</button></div></div>';
|
||||
}
|
||||
|
||||
// ---- the Radar: the week's papers, sorted by Claude on the server; the decisions are ours ----------
|
||||
|
||||
const DECISIONS = {read: 'gelesen', later: 'später', drop: 'verworfen', source: 'in Quellen'};
|
||||
|
||||
// The project's item holding the decisions ({id: state}), encrypted like everything else.
|
||||
function radarItem() {
|
||||
return st.open.items.filter(i => i.kind === 'radar' && i.data)[0] || null;
|
||||
}
|
||||
|
||||
function decisions() {
|
||||
if (st.open.radarDec) return st.open.radarDec; // changed here, maybe not saved yet
|
||||
const i = radarItem();
|
||||
return (i && i.data.decisions) || {};
|
||||
}
|
||||
|
||||
// Undecided papers of the latest digest's first group (the worth-reading): the tab's number.
|
||||
function radarNew() {
|
||||
const d = st.open.digests[0];
|
||||
if (!d || !st.open.radar) return 0;
|
||||
const dec = decisions();
|
||||
return (d.bundle.items || []).filter(it => it.gruppe === (st.open.radar.groups || [])[0] && !dec[it.id]).length;
|
||||
}
|
||||
|
||||
let deciding = Promise.resolve();
|
||||
function decide(id, state) {
|
||||
// Shown at once; the saves follow one after the other, each with everything decided here.
|
||||
const open = st.open;
|
||||
const dec = Object.assign({}, decisions());
|
||||
if (state) dec[id] = state; else delete dec[id];
|
||||
open.radarDec = dec;
|
||||
const changed = {};
|
||||
changed[id] = state || null;
|
||||
deciding = deciding.then(async () => {
|
||||
const it = open.items.filter(i => i.kind === 'radar' && i.data)[0];
|
||||
if (!it) { await addItem('radar', null, {decisions: Object.assign({}, open.radarDec)}); return; }
|
||||
if (await saveItem(it, {decisions: Object.assign({}, open.radarDec)})) return;
|
||||
// Decided elsewhere meanwhile: theirs with this one on top.
|
||||
const merged = Object.assign({}, (it.data && it.data.decisions) || {});
|
||||
if (changed[id]) merged[id] = changed[id]; else delete merged[id];
|
||||
open.radarDec = merged;
|
||||
await saveItem(it, {decisions: Object.assign({}, merged)});
|
||||
if (st.open === open) render();
|
||||
}).catch(e => toast('Speichern ging nicht: ' + e.message));
|
||||
return deciding;
|
||||
}
|
||||
|
||||
function paperById(id) {
|
||||
for (const d of st.open.digests) {
|
||||
const it = (d.bundle.items || []).filter(x => x.id === id)[0];
|
||||
if (it) return it;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// A paper of the Radar as a source: its citation data from doi.org, else from what the Radar has.
|
||||
async function paperToSource(it) {
|
||||
let csl = null;
|
||||
if (it.doi) {
|
||||
try { csl = (await api('/projects/doi/?q=' + encodeURIComponent(it.doi))).csl; } catch (e) {}
|
||||
}
|
||||
if (!csl) {
|
||||
const pre = /preprint/i.test((it.pub_types || []).join(' ')) || /arxiv|rxiv/i.test(it.source || '');
|
||||
csl = {type: pre ? 'article' : 'article-journal', title: it.title, 'container-title': it.journal || '',
|
||||
author: it.first_author ? names(it.first_author).concat(it.n_authors > 1 ? [{literal: 'u. a.'}] : []) : [],
|
||||
issued: it.date ? {'date-parts': [String(it.date).split('-').map(Number)]} : undefined, URL: it.url,
|
||||
DOI: it.doi || undefined};
|
||||
}
|
||||
const data = {csl: csl, note: it.kurz || '', doi: it.doi || '', radar: it.id};
|
||||
await keyed(data, st.open.id);
|
||||
await addItem('paper', null, data);
|
||||
}
|
||||
|
||||
function weekOf(iso) {
|
||||
const d = new Date(iso + 'T12:00:00');
|
||||
const t = new Date(Date.UTC(d.getFullYear(), d.getMonth(), d.getDate()));
|
||||
t.setUTCDate(t.getUTCDate() + 4 - (t.getUTCDay() || 7));
|
||||
const y = new Date(Date.UTC(t.getUTCFullYear(), 0, 1));
|
||||
return Math.ceil(((t - y) / 864e5 + 1) / 7);
|
||||
}
|
||||
|
||||
function dayText(iso, withTime) {
|
||||
const d = new Date(iso.length <= 10 ? iso + 'T12:00:00' : iso);
|
||||
if (isNaN(d)) return '';
|
||||
const o = {weekday: 'short', day: 'numeric', month: 'long'};
|
||||
if (withTime) { o.hour = '2-digit'; o.minute = '2-digit'; }
|
||||
return d.toLocaleString('de-DE', o);
|
||||
}
|
||||
|
||||
function radarStatusHtml(r) {
|
||||
let s = '';
|
||||
if (r.running) s = 'Claude sortiert gerade die Papers' + (r.submitted ? ' (seit ' + dayText(r.submitted, true) + ')' : '') + '.';
|
||||
else if (!r.enabled) s = 'Pausiert.';
|
||||
else if (r.next_run && new Date(r.next_run) - Date.now() < 60 * 60 * 1000) s = 'Der nächste Abruf kommt in der nächsten halben Stunde.';
|
||||
else if (r.next_run) s = 'Nächster Abruf: ' + dayText(r.next_run, true) + '.';
|
||||
let h = '<div class="prj-radar-status"><span class="prj-muted">' + esc(s) +
|
||||
(r.sources.length ? ' ' + esc(r.sources.join(', ')) + ', die letzten ' + r.days + ' Tage.' : '') + '</span>';
|
||||
if (!r.running) h += '<button type="button" class="prj-chip" data-radar-run="1">Jetzt abrufen</button>';
|
||||
h += '<button type="button" class="prj-chip" data-radar-pause="1">' + (r.enabled ? 'Pausieren' : 'Wieder an') + '</button></div>';
|
||||
if (!r.key) h += '<p class="prj-warn">Auf dem Server fehlt noch der API-Schlüssel (ANTHROPIC_API_KEY): bis dahin kommt kein neuer Digest.</p>';
|
||||
if (r.last_error) h += '<p class="prj-warn">' + esc(r.last_error) + '</p>';
|
||||
return h;
|
||||
}
|
||||
|
||||
function paperHtml(it, full, dec) {
|
||||
const state = dec[it.id];
|
||||
const who = (it.first_author || '?') + ((it.n_authors || 0) > 1 ? ' u. a.' : '');
|
||||
const meta = [who, it.journal || it.source, it.date].filter(Boolean).join(' · ');
|
||||
let h = '<div class="prj-paper' + (state === 'drop' ? ' prj-dropped' : '') + '">' +
|
||||
'<div class="prj-paper-title">' + (it.url ? '<a href="' + esc(it.url) + '" target="_blank" rel="noopener">' + esc(it.title) + '</a>' : esc(it.title)) + '</div>' +
|
||||
'<div class="prj-muted">' + esc(meta) + (state ? ' · <b>' + esc(DECISIONS[state]) + '</b>' : '') + '</div>';
|
||||
if (full) {
|
||||
if (it.worum_es_geht) h += '<p><b>Worum es geht:</b> ' + inline(it.worum_es_geht) + '</p>';
|
||||
if (it.einordnung) h += '<p><b>Einordnung:</b> ' + inline(it.einordnung) + '</p>';
|
||||
if (it.politischer_haken) h += '<p><b>Politischer Haken:</b> ' + inline(it.politischer_haken) + '</p>';
|
||||
} else if (it.kurz) h += '<div class="prj-paper-short">' + inline(it.kurz) + '</div>';
|
||||
const b = (s, label) => '<button type="button" class="prj-chip' + (state === s ? ' on' : '') + '" data-radar-decide="' +
|
||||
esc(it.id) + '" data-state="' + s + '">' + label + '</button>';
|
||||
h += '<div class="prj-paper-acts">' + b('later', 'Später') + b('read', 'Gelesen') + b('drop', 'Verwerfen') +
|
||||
(state === 'source' ? '<span class="prj-muted">✓ in Quellen</span>'
|
||||
: '<button type="button" class="prj-chip" data-radar-source="' + esc(it.id) + '">→ Quellen</button>') + '</div></div>';
|
||||
return h;
|
||||
}
|
||||
|
||||
function radarHtml() {
|
||||
const r = st.open.radar;
|
||||
const digests = st.open.digests;
|
||||
const dec = decisions();
|
||||
let h = radarStatusHtml(r);
|
||||
if (!digests.length) return h + '<p class="prj-muted prj-empty">Noch kein Digest.</p>';
|
||||
const later = [];
|
||||
digests.forEach(d => (d.bundle.items || []).forEach(it => { if (dec[it.id] === 'later') later.push(it); }));
|
||||
h += '<div class="prj-chips">' + digests.map((d, n) => '<button type="button" class="prj-chip' +
|
||||
(st.radarShow !== 'later' && (st.digest || 0) === n ? ' on' : '') + '" data-radar-digest="' + n + '">KW ' + weekOf(d.generated) + '</button>').join('') +
|
||||
(later.length ? '<button type="button" class="prj-chip' + (st.radarShow === 'later' ? ' on' : '') + '" data-radar-later="1">Später ' + later.length + '</button>' : '') + '</div>';
|
||||
if (st.radarShow === 'later') {
|
||||
return h + '<h3 class="prj-h3">Für später</h3>' + later.map(it => paperHtml(it, !!it.worum_es_geht, dec)).join('');
|
||||
}
|
||||
const d = digests[Math.min(st.digest || 0, digests.length - 1)];
|
||||
const b = d.bundle;
|
||||
const groups = r.groups && r.groups.length ? r.groups : ['lesenswert'];
|
||||
const label = g => (r.group_labels && r.group_labels[g]) || g;
|
||||
h += '<h3 class="prj-radar-h">' + esc(r.title || 'Digest') + ' · KW ' + weekOf(d.generated) + ', ' + esc(dayText(d.generated)) + '</h3>';
|
||||
if (b.auf_einen_blick) h += '<p class="prj-radar-glance">' + inline(b.auf_einen_blick) + '</p>';
|
||||
groups.forEach((g, n) => {
|
||||
const items = (b.items || []).filter(it => it.gruppe === g);
|
||||
if (!items.length) return;
|
||||
const noise = n === groups.length - 1 && groups.length > 2; // the last group (Rauschen): closed
|
||||
h += '<details class="prj-radar-group"' + (noise ? '' : ' open') + '><summary class="prj-h3">' + esc(label(g)) +
|
||||
'<span class="prj-count">' + items.length + '</span></summary>' +
|
||||
items.map(it => paperHtml(it, n === 0 && !!it.worum_es_geht, dec)).join('') + '</details>';
|
||||
});
|
||||
if (b.news_markdown && !/^Keine relevanten Meldungen/.test(b.news_markdown.trim())) {
|
||||
h += '<details class="prj-radar-group" open><summary class="prj-h3">Aus Branche und Politik</summary><div class="prj-md">' +
|
||||
markdown(b.news_markdown) + '</div></details>';
|
||||
}
|
||||
const g = b.grundlagenbegriff;
|
||||
if (g && g.begriff) {
|
||||
h += '<div class="prj-radar-term"><div class="prj-h3">Grundlagenbegriff der Woche</div><b>' + esc(g.begriff) + '.</b> ' +
|
||||
inline(g.erklaerung || '') + '</div>';
|
||||
}
|
||||
const s = b.stats || {};
|
||||
h += '<p class="prj-muted prj-radar-stats">' + (s.records || 0) + ' Papers gesichtet' +
|
||||
(s.per_source ? ' (' + Object.keys(s.per_source).map(k => k + ' ' + s.per_source[k]).join(', ') + ')' : '') +
|
||||
(typeof s.cost_usd === 'number' ? ' · ' + s.cost_usd.toFixed(2).replace('.', ',') + ' $' : '') +
|
||||
((s.errors || []).length ? ' · Fehler: ' + esc(s.errors.join('; ')) : '') + '</p>';
|
||||
return h;
|
||||
}
|
||||
|
||||
function notesHtml() {
|
||||
if (st.note != null) return noteEditorHtml();
|
||||
const list = notes().filter(n => n.data).sort((a, b) => String(b.updated_at).localeCompare(String(a.updated_at)));
|
||||
|
|
@ -928,6 +1102,35 @@
|
|||
render();
|
||||
return;
|
||||
}
|
||||
if (ds.radarDigest != null) { st.digest = Number(ds.radarDigest); st.radarShow = 'all'; render(); return; }
|
||||
if (ds.radarLater) { st.radarShow = 'later'; render(); return; }
|
||||
if (ds.radarDecide) {
|
||||
const now = decisions()[ds.radarDecide];
|
||||
decide(ds.radarDecide, now === ds.state ? null : ds.state);
|
||||
render();
|
||||
return;
|
||||
}
|
||||
if (ds.radarSource) {
|
||||
const it = paperById(ds.radarSource);
|
||||
if (!it) return;
|
||||
t.disabled = true;
|
||||
await paperToSource(it);
|
||||
await decide(it.id, 'source');
|
||||
toast('In den Quellen');
|
||||
render();
|
||||
return;
|
||||
}
|
||||
if (ds.radarRun) {
|
||||
if (!confirm('Jetzt die Papers der letzten Tage abrufen und von Claude sortieren lassen? Das kostet etwa 0,40 $.')) return;
|
||||
st.open.radar = (await api('/projects/' + st.open.id + '/radar/run/', {})).radar;
|
||||
render();
|
||||
return;
|
||||
}
|
||||
if (ds.radarPause) {
|
||||
st.open.radar = (await api('/projects/' + st.open.id + '/radar/settings/', {enabled: !st.open.radar.enabled})).radar;
|
||||
render();
|
||||
return;
|
||||
}
|
||||
if (ds.openNote) { st.note = Number(ds.openNote); st.preview = true; saveState = ''; render(); return; }
|
||||
if (ds.cite) {
|
||||
const i = citeIndex()[ds.cite];
|
||||
|
|
@ -1167,6 +1370,25 @@
|
|||
'#tab-projekte .prj-md h3 { font-size: 19px; } #tab-projekte .prj-md h4 { font-size: 16px; } #tab-projekte .prj-md h5, #tab-projekte .prj-md h6 { font-size: 15px; }',
|
||||
'#tab-projekte .prj-md blockquote { margin: 8px 0; padding-left: 12px; border-left: 3px solid var(--border-strong, #444); color: var(--muted, #9a9a9a); }',
|
||||
'#tab-projekte .prj-md a { color: var(--accent, #e63946); }',
|
||||
'#tab-projekte .prj-radar-status { display: flex; flex-wrap: wrap; gap: 6px; align-items: center; margin: 4px 0 6px; }',
|
||||
'#tab-projekte .prj-radar-status .prj-muted { flex: 1 1 260px; }',
|
||||
'#tab-projekte .prj-warn { color: var(--accent, #e63946); font-size: 13px; margin: 6px 0; }',
|
||||
'#tab-projekte .prj-radar-h { font-size: 17px; margin: 16px 0 6px; }',
|
||||
'#tab-projekte .prj-radar-glance { line-height: 1.6; font-size: 15px; }',
|
||||
'#tab-projekte .prj-radar-group summary { cursor: pointer; list-style: none; }',
|
||||
'#tab-projekte .prj-radar-group summary::-webkit-details-marker { display: none; }',
|
||||
'#tab-projekte .prj-radar-group summary::before { content: "▸ "; } #tab-projekte .prj-radar-group[open] > summary::before { content: "▾ "; }',
|
||||
'#tab-projekte .prj-paper { padding: 10px 0 12px; border-bottom: 1px solid var(--border, #2a2a2a); }',
|
||||
'#tab-projekte .prj-paper p { margin: 6px 0; line-height: 1.55; font-size: 14px; }',
|
||||
'#tab-projekte .prj-paper-title { font-size: 15px; font-weight: 700; margin-bottom: 2px; }',
|
||||
'#tab-projekte .prj-paper-title a { color: inherit; text-decoration: none; } #tab-projekte .prj-paper-title a:hover { text-decoration: underline; }',
|
||||
'#tab-projekte .prj-paper-short { font-size: 14px; margin-top: 4px; line-height: 1.5; }',
|
||||
'#tab-projekte .prj-paper-acts { display: flex; flex-wrap: wrap; gap: 6px; align-items: center; margin-top: 8px; }',
|
||||
'#tab-projekte .prj-paper-acts .prj-chip { min-height: 30px; font-size: 12px; }',
|
||||
'#tab-projekte .prj-dropped { opacity: .45; }',
|
||||
'#tab-projekte .prj-radar-term { margin: 16px 0; padding: 12px 14px; border-radius: 8px; background: var(--card, #0b0b0b); border: 1px solid var(--border, #2a2a2a); line-height: 1.55; font-size: 14px; }',
|
||||
'#tab-projekte .prj-radar-term .prj-h3 { margin-top: 0; }',
|
||||
'#tab-projekte .prj-radar-stats { margin-top: 14px; }',
|
||||
'#tab-projekte .prj-table { overflow-x: auto; margin: 10px 0; }',
|
||||
'#tab-projekte .prj-table table { border-collapse: collapse; font-size: 14px; }',
|
||||
'#tab-projekte .prj-table th, #tab-projekte .prj-table td { border: 1px solid var(--border, #2a2a2a); padding: 6px 8px; text-align: left; vertical-align: top; }',
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
* diora service worker — caches the app shell for offline use.
|
||||
*/
|
||||
|
||||
const CACHE = 'diora-v144';
|
||||
const CACHE = 'diora-v145';
|
||||
const SHELL = [
|
||||
'/static/css/app.css',
|
||||
'/static/js/app/01-core.js',
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue