NSCT Agent
a60cf21a2c
feat(stage6): source independence & citation graph — detect syndication, shared origins, text similarity
...
- SourceIndependenceModel: per-source independence_score (0.0-1.0), syndication_group_id, primary_source_id, content_hash, shared_urls
- CitationGraphEdgeModel: directed edges (SYNDICATED, QUOTES, LINKS_TO, REPOST, SIMILAR_CONTENT) with confidence + evidence
- Content-Hash (SHA-256): instant syndication detection for identical content
- difflib Vorfilterung: >60% → LLM, >80% → high confidence, 100% → immediate syndication
- LLM-Pairwise-Analysis: two-text-comparison for suspicious pairs only (bounded concurrency)
- independence_score: 1.0 base, -0.4 for syndicated, -0.1 per high-similarity pair
- Pydantic schemas: SourceIndependenceScore, CitationGraphEdge, SyntacticSimilarityResult, LlmSyndicationAnalysis, SourceIndependenceAnalysisResult
- LLM response parser: handles JSON, markdown code blocks, partial/invalid JSON
- 43 tests: content hash, similarity thresholds, LLM parsing, analyzer integration, edge cases, prompt templates
2026-08-23 18:47:24 +00:00
NSCT Agent
e8b6515f67
feat(stage5): implement claim extraction — atomic verifiable claims from sources
...
- Claim model with provenance, evidence_span, attribution, claim_type
- Stage5Extractor: LLM-based atomic claim extraction from source content
- Never summarizes — always extracts atomic, verifiable claims
- Claims require evidence span (exact quote from source)
- Attribution per claim (who says what)
- Claim types: fact, opinion, prediction, recommendation, claim
- Confidence score 0.0–1.0 per claim
- Bounded concurrency, SSRF-safe, max content truncation
- REST API: GET/POST /research/{run_id}/claims
- 36 tests: parsing, edge cases, integration, validation
2026-08-23 17:56:01 +00:00
NSCT Agent
b8181deb05
Stage 4: Research Planner — LLM-basierte Recherchestrategie-Generierung
...
- src/nsct/agents/planner.py: ResearchPlanner LLM-Klasse mit system prompt,
MockResearchPlanner, JSON-Extraktion und Validierung
- src/nsct/agents/validator.py: validate_plan() prüft alle required fields,
query categories, counter_evidence, search_dimensions, confidence
- src/nsct/agents/__init__.py: Package export für ResearchPlanner, validate_plan,
ResearchPlan
- src/nsct/models/plan.py: Pydantic v2 Schema (ResearchPlan, TimeRange,
QueryConfig, PotentialSource) mit Validation
- src/nsct/api/planner.py: POST /research/planner Endpoint mit Debug-Support
- src/nsct/api/main.py: Mount des planner routers
- src/nsct/crawler/pdf.py: exportiere extract_pdf_content als Alias
- src/nsct/api/crawler.py: Pydantic BaseModel für Request-Models
- tests/test_planner.py: 25 Tests für Planner, Validator, Schema, API
- Search-Bias-Reduktion: 6+ Query-Typen, counter_evidence, beide Seiten
- Keine TODOs, keine unvollständigen Funktionen
- Alle Dateien syntaktisch korrekt und getestet
2026-08-23 13:28:23 +00:00
NSCT Agent
e9410be941
Stage 0: Repository und Architekturgrundlage
...
- Pyproject.toml mit FastAPI, Pydantic v2, SQLAlchemy, httpx, asyncio,
BeautifulSoup4, selectolax, trafilatura, uvicorn, pytest-asyncio
- Multi-stage Dockerfile (Python 3.12-slim, Non-Root-User nsct)
- docker-compose.yml (nsct-api + postgres + optional searxng)
- .env.example mit allen Config-Parametern
- Config-System: AppSettings mit LLMConfig, VisionConfig, AudioConfig,
DatabaseConfig — komplett aus Environment, keine Hardcodes
- Strukturiertes Logging mit research_id/llm_request_id Tracking
- Pydantic v2 Schemas: SearchQuery, Source, Claim, EvidenceRelation,
CitationEdge, ResearchReport
- SQLAlchemy 2.0 Declarative Models + async Engine Factory
- SSRF-Schutz: URL-Validation, IP-Blocklist (RFC1918, Cloud Metadata,
file://, ftp://)
- Provider-Interfaces: LLMProvider, VisionProvider, AudioProvider,
SearchProvider, ContentFetcher als ABCs
- Health-Endpoints: /health, /ready (LLM-Connect-Test), /providers
- FastAPI App mit CORS, lifespan (LLM Pre-Flight)
- CLI-Stub mit Entry-Points: nsct, nsct-core, nsct-api
- 6 Test-Cases: /health, /ready, /providers + No-Secrets-Test
- Vollständige Dokumentation: README, ARCHITECTURE, SECURITY,
METHODOLOGY, API, DEPLOYMENT
- .gitignore (Python, Docker, IDE, .env)
2026-08-23 11:33:45 +00:00
