Fix research pipeline provider failures

HANDOFF.md

@@ -85,39 +85,34 @@ Mit einem gültigen, bereits erzeugten Key im Browser testen:
    Antwortcodes gemeinsam auswerten. Klartext-Keys niemals in Chat oder Logs
    kopieren.
 
-### Aktueller Blocker im Browser-Flow — 2026-09-06
+### Browser-Flow: Backend-Blocker lokal behoben — 2026-09-06
 
-Der Browser-Flow erreicht inzwischen die Detailseite und fragt Status,
+Die drei Backend-Ursachen für leere, scheinbar erfolgreiche Researches sind
-Quellen, Claims, Evidenz und Bericht ab. Eine tatsächlich gestartete Recherche
+lokal implementiert, getestet und mit `docker compose up -d --build` deployed:
-liefert jedoch derzeit keine Ergebnisse. Dies ist ein Backend-Problem, nicht
-mehr der Frontend-Proxy oder die API-Key-Loginstrecke.
 
-Reproduzierbare Logbefunde aus `docker compose logs nsct-api`:
+1. `LLMProvider._create_client()` normalisiert die Basis-URL und hängt `/v1`
+   nur an, wenn sie nicht bereits enthalten ist. Ein authentifizierter,
+   sanitisiert ausgegebener Chat-Completions-Probe war erfolgreich.
+2. `SearXNGProvider` ist in `src/nsct/providers/searxng.py` implementiert und
+   wird vom Orchestrator registriert. Compose mountet eine minimale SearXNG-
+   Konfiguration, die `format=json` erlaubt, und nutzt den vom aktuellen
+   SearXNG-Image erwarteten Variablennamen `SEARXNG_SECRET`. Ein echter,
+   sanitisiert ausgegebener Search-Probe lieferte ein Ergebnis.
+3. Die Pipeline führt wieder `fetching → extracting → analyzing` aus.
+   Fehler beim Planen, Suchen, Abrufen oder Extrahieren werden nicht mehr als
+   leere Fallbacks als Erfolg gemeldet: Der Run wird `FAILED`, mit Fehlergrund
+   im REST-Status (`error`). Fehlende Provider führen ebenfalls zu `FAILED`.
 
-- `Planning failed: Error code: 401 - {'error': 'authentication required'}`
+Validierung: Syntaxprüfung, Compose-Konfigurationsprüfung und 29 fokussierte
-- `No search providers configured — search will return empty results`
+Tests (inklusive neuer Regressionen für URL-Normalisierung, SearXNG und
-- abgelehnte State-Transitions, etwa `fetching -> analyzing`,
+State-Flow) sind erfolgreich. `tests/test_rest_research.py` wurde nicht als
-  `fetching -> synthesizing` und `fetching -> completed`.
+Gesamtsuite abgewartet, da seine Background-Tasks echte externe Retries
+auslösen; die neuen deterministischen Tests decken den geänderten Fehlerpfad.
 
-Ursachen und nächste Reparaturen:
+Nächster Schritt: Den Browser-End-to-End-Test mit einem gültigen, **nicht im
-
+Chat offengelegten** Frontend-API-Key durchführen. Dabei insbesondere prüfen,
-1. `src/nsct/providers/llm.py` ergänzt in `_create_client()` immer `/v1`.
+dass Quellen, Claims und Bericht nach einem echten Research-Lauf erscheinen
-   `NSCT_LLM_BASE_URL` ist laut `.env.example` und Deployment-Dokumentation
+und ein Provider-/LLM-Ausfall in der UI den REST-Fehlergrund zeigt.
-   bereits eine OpenAI-kompatible URL mit `/v1`. Den Basis-URL-Join
-   normalisieren, damit nie `/v1/v1` entsteht. Danach einen authentifizierten
-   Chat-Completions-Probe ausführen, ohne URL oder Key auszugeben.
-2. `src/nsct/orchestration/orchestrator.py::_get_multi_search()` importiert
-   `nsct.providers.searxng.SearXNGProvider`, aber
-   `src/nsct/providers/searxng.py` existiert nicht. Der `ImportError` wird
-   still geschluckt; deshalb hat die Pipeline keinen Search-Provider. Entweder
-   den SearXNG-Provider implementieren oder den vorhandenen
-   `DuckDuckGoProvider` als Fallback registrieren. Compose setzt bereits
-   `NSCT_SEARXNG_BASE_URL=http://searxng:8080/`.
-3. Das Orchestrator-/REST-Fehlerhandling muss bei nicht ausführbarer Pipeline
-   einen nachvollziehbaren `FAILED`-Status mitsamt Fehlergrund liefern, statt
-   mit leeren Fallbacks scheinbar abgeschlossene Schritte zu zeigen. Die
-   Transition-Matrix und die Fallback-Logik in `orchestrator.py` prüfen und
-   gezielt testen.
 
 Frontend-Fixes des laufenden Browser-Tests (alle im Frontend-Repository
 `/home/faligam/apps/NSCT-FrontEnd`, `main`, gepusht):

config/searxng/settings.yml

@@ -0,0 +1,8 @@
+# NSCT needs SearXNG's structured result endpoint.  Keep HTML enabled for
+# manual administration while allowing the backend's `format=json` requests.
+use_default_settings: true
+
+search:
+  formats:
+    - html
+    - json

docker-compose.yml

@@ -52,9 +52,10 @@ services:
       - "8888:8080"
     volumes:
       - searxng_cache:/var/cache/searxng
+      - ./config/searxng/settings.yml:/etc/searxng/settings.yml:ro
     environment:
       - SEARXNG_BASE_URL=http://localhost:8888/
-      - SEARXNG_SECRET_KEY=nsct_searxng_secret_key_change_me
+      - SEARXNG_SECRET=nsct_searxng_secret_key_change_me
     restart: unless-stopped
     deploy:
       resources:
@@ -64,7 +65,6 @@ services:
     read_only: true
     tmpfs:
       - /tmp
-      - /etc/searxng:mode=1777
     cap_drop:
       - ALL
 

src/nsct/api/rest_research.py

@@ -147,6 +147,7 @@ class StatusResponse(BaseModel):
     search_count: int = 0
     source_count: int = 0
     claim_count: int = 0
+    error: str | None = None
     is_completed: bool = False
     is_running: bool = False
 
@@ -468,6 +469,8 @@ async def _run_pipeline(run: _ResearchRunState, request: ResearchRequest, budget
             )
         else:
             run.state = ResearchRunState.FAILED.value
+            run.metadata["error"] = result.get("error", "Pipeline failed")
+            run.metadata["failed_step"] = result.get("failed_step", "unknown")
             metrics.increment(C_RESEARCH_FAILED_TOTAL)
             logger.warning(
                 "[pipeline] run_id=%s failed: %s",
@@ -519,6 +522,7 @@ async def get_research(research_id: str) -> StatusResponse:
         search_count=len(run.search_results),
         source_count=len(run.sources),
         claim_count=len(run.claims),
+        error=run.metadata.get("error"),
         is_completed=(run.state == ResearchRunState.COMPLETED.value),
         is_running=run.state not in (
             ResearchRunState.COMPLETED.value,
@@ -546,6 +550,7 @@ async def get_status(research_id: str) -> StatusResponse:
         search_count=len(run.search_results),
         source_count=len(run.sources),
         claim_count=len(run.claims),
+        error=run.metadata.get("error"),
         is_completed=(run.state == ResearchRunState.COMPLETED.value),
         is_running=run.state not in (
             ResearchRunState.COMPLETED.value,

src/nsct/orchestration/orchestrator.py

@@ -208,8 +208,6 @@ class ResearchOrchestrator:
         """
         logger.error("Research failed: %s (state=%s)", reason, self._state_machine.current_state.value)
 
-        # Der State Machine _apply erzwingt FAILED direkt,
-        # weil _mark_failed als "Sondertransition" gedacht ist.
         if self._state_machine.current_state in (
             ResearchRunState.COMPLETED,
             ResearchRunState.FAILED,
@@ -217,8 +215,16 @@ class ResearchOrchestrator:
         ):
             return False
 
-        self._state_machine.current_state = ResearchRunState.FAILED  # type: ignore[assignment]
+        transitioned = self._transition_to(ResearchRunState.FAILED.value)
-        return True
+        if transitioned and self._run is not None:
+            self._run = self._run.model_copy(
+                update={
+                    "state": ResearchRunState.FAILED.value,
+                    "updated_at": datetime.now(timezone.utc),
+                    "metadata": {**self._run.metadata, "error": reason},
+                }
+            )
+        return transitioned
 
     # ---------------------------------------------------------------
     # Private: Budget
@@ -297,14 +303,14 @@ class ResearchOrchestrator:
             if self._config.searxng_base_url:
                 provider = SearXNGProvider(base_url=self._config.searxng_base_url)
                 providers.append(provider)
-        except ImportError:
+        except ImportError as exc:
-            pass
+            logger.error("SearXNG provider is unavailable: %s", exc)
         except Exception as exc:
-            logger.warning("SearXNG provider init failed (%s) — search will return empty", exc)
+            logger.error("SearXNG provider initialization failed: %s", exc)
 
-        self._multi_search = MultiProviderSearch(providers=providers if providers else None)
+        self._multi_search = MultiProviderSearch(providers=providers)
         if not providers:
-            logger.warning("No search providers configured — search will return empty results")
+            logger.error("No search providers configured")
         return self._multi_search
 
     def _get_llm_provider(self):
@@ -406,6 +412,7 @@ class ResearchOrchestrator:
             "planning",
             "searching",
             "fetching",
+            "extracting",
             "analyzing",
             "comparing",
             "synthesizing",
@@ -440,6 +447,7 @@ class ResearchOrchestrator:
                         "success": False,
                         "error": error_msg,
                         "failed_step": step_name,
+                        "state": self._state_machine.current_state.value,
                         "report": None,
                     }
 
@@ -458,6 +466,7 @@ class ResearchOrchestrator:
                     "success": False,
                     "error": str(exc),
                     "failed_step": step_name,
+                    "state": self._state_machine.current_state.value,
                     "report": None,
                 }
 
@@ -552,18 +561,7 @@ class ResearchOrchestrator:
 
         except Exception as exc:
             logger.error("Planning failed: %s", exc)
-            # Fallback: Create a minimal plan
+            return {"success": False, "error": f"Planning failed: {exc}", "data": {}}
-            fallback = self._get_fallback_plan()
-            self._plan = fallback
-            # Store error in _run metadata via new instance
-            if self._run is not None:
-                self._run = self._run.model_copy(
-                    update={
-                        "plan_error": str(exc),
-                        "updated_at": datetime.utcnow(),
-                    }
-                )
-            return {"success": True, "data": {"plan": fallback}, "plan": fallback, "warning": "Used fallback plan"}
 
     async def _step_searching(self) -> dict[str, Any]:
         """SEARCHING: Führe Suchanfragen durch und sammle URLs."""
@@ -571,6 +569,12 @@ class ResearchOrchestrator:
 
         try:
             multi_search = self._get_multi_search()
+            if not multi_search.enabled_providers():
+                return {
+                    "success": False,
+                    "error": "No search provider is configured or available",
+                    "data": {},
+                }
             plan_queries = self._plan.get("queries", []) if self._plan else []
 
             if not plan_queries:
@@ -615,9 +619,8 @@ class ResearchOrchestrator:
             return {"success": True, "data": {"urls": self._search_results}, "url_count": len(self._search_results)}
 
         except Exception as exc:
-            logger.warning("Searching failed (graceful fallback): %s — returning empty results", exc)
+            logger.error("Searching failed: %s", exc)
-            self._search_results = []
+            return {"success": False, "error": f"Searching failed: {exc}", "data": {}}
-            return {"success": True, "data": {"urls": []}, "url_count": 0, "warning": str(exc)}
 
     async def _step_fetching(self) -> dict[str, Any]:
         """FETCHING: Crawle gesammelte URLs und extrahiere Inhalt."""
@@ -669,9 +672,8 @@ class ResearchOrchestrator:
             }
 
         except Exception as exc:
-            logger.warning("Fetching failed (graceful fallback): %s — returning empty sources", exc)
+            logger.error("Fetching failed: %s", exc)
-            self._sources = []
+            return {"success": False, "error": f"Fetching failed: {exc}", "data": {}}
-            return {"success": True, "data": {"sources": []}, "source_count": 0, "warning": str(exc)}
 
     async def _step_extracting(self) -> dict[str, Any]:
         """EXTRACTING: Extrahiere Claims aus den extrahierten Quellen."""
@@ -717,9 +719,8 @@ class ResearchOrchestrator:
             return {"success": True, "data": {"claims": [c.model_dump() for c in self._claims]}, "claim_count": len(self._claims)}
 
         except Exception as exc:
-            logger.warning("Claim extraction failed (graceful fallback): %s", exc)
+            logger.error("Claim extraction failed: %s", exc)
-            self._claims = []
+            return {"success": False, "error": f"Claim extraction failed: {exc}", "data": {}}
-            return {"success": True, "data": {"claims": []}, "claim_count": 0, "warning": str(exc)}
 
     async def _step_analyzing(self) -> dict[str, Any]:
         """ANALYZING: Platzhalter — Stage 6/7/8 werden später eingebunden.

src/nsct/orchestration/state.py

@@ -39,7 +39,7 @@ class ResearchRunState(str, Enum):
 # Übergangsmatrix: Quelle -> zulässige Ziele (frozenset für Immutable)
 _VALID_TRANSITIONS: dict[ResearchRunState, FrozenSet[ResearchRunState]] = {
     ResearchRunState.CREATED: frozenset(
-        (ResearchRunState.PLANNING,)
+        (ResearchRunState.PLANNING, ResearchRunState.FAILED)
     ),
     ResearchRunState.PLANNING: frozenset(
         (ResearchRunState.SEARCHING, ResearchRunState.FAILED)

src/nsct/providers/llm.py

@@ -151,8 +151,14 @@ class _LLMProviderImpl(LLMProvider):
                 pool=5,
             ),
         )
+        # OpenAI-compatible endpoints are commonly configured either as the
+        # service root or with the API version already included.  Appending
+        # blindly turns an explicit ``.../v1`` into ``.../v1/v1``.
+        base_url = self._base_url.rstrip("/")
+        if not base_url.endswith("/v1"):
+            base_url += "/v1"
         return AsyncOpenAI(
-            base_url=self._base_url.rstrip("/") + "/v1",
+            base_url=base_url,
             api_key=self._api_key,
             http_client=http_client,
         )

src/nsct/providers/searxng.py

@@ -0,0 +1,82 @@
+"""SearXNG JSON search provider."""
+
+from __future__ import annotations
+
+from typing import Any
+
+import httpx
+
+from nsct.providers.abstract import NormalizedResult, SearchProvider
+
+
+class SearXNGProvider(SearchProvider):
+    """Search through a self-hosted SearXNG instance's JSON endpoint."""
+
+    _provider_name = "searxng"
+
+    def __init__(self, base_url: str, timeout_seconds: float = 15.0) -> None:
+        if not base_url.strip():
+            raise ValueError("SearXNG base URL must not be empty")
+        self._base_url = base_url.rstrip("/")
+        self._timeout_seconds = timeout_seconds
+        self._client: httpx.AsyncClient | None = None
+
+    @property
+    def _http_client(self) -> httpx.AsyncClient:
+        if self._client is None:
+            self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout_seconds))
+        return self._client
+
+    async def search(
+        self, query: str, language: str = "de", max_results: int = 10
+    ) -> list[NormalizedResult]:
+        response = await self._http_client.get(
+            f"{self._base_url}/search",
+            params={"q": query, "format": "json", "language": language},
+            # The bundled SearXNG instance enables bot detection and rejects
+            # requests without a client IP header.  This is the loopback IP of
+            # the API-to-SearXNG hop, not an end-user supplied value.
+            headers={"X-Forwarded-For": "127.0.0.1"},
+        )
+        response.raise_for_status()
+        payload = response.json()
+        raw_results = payload.get("results", [])
+        if not isinstance(raw_results, list):
+            raise ValueError("Invalid SearXNG response: results is not a list")
+
+        results: list[NormalizedResult] = []
+        for rank, item in enumerate(raw_results[:max_results], start=1):
+            if not isinstance(item, dict) or not isinstance(item.get("url"), str):
+                continue
+            url = item["url"]
+            if not url.startswith(("http://", "https://")):
+                continue
+            results.append(
+                NormalizedResult.from_raw(
+                    provider_name=self._provider_name,
+                    title=str(item.get("title") or url),
+                    url=url,
+                    snippet=str(item.get("content") or item.get("snippet") or ""),
+                    rank=rank,
+                    extra={"engine": item.get("engine"), "category": item.get("category")},
+                )
+            )
+        return results
+
+    async def get_metadata(self) -> dict[str, Any]:
+        return {
+            "name": self._provider_name,
+            "version": "0.1.0",
+            "provider": self.__class__.__name__,
+            "capabilities": ["web_search", "json_api"],
+            "requires_api_key": False,
+            "base_url": self._base_url,
+            "note": "Search ranking is NOT a trust indicator.",
+        }
+
+    async def health_check(self) -> bool:
+        try:
+            response = await self._http_client.get(f"{self._base_url}/healthz")
+            return response.is_success
+        except httpx.HTTPError:
+            return False

tests/test_backend_pipeline_repairs.py

@@ -0,0 +1,107 @@
+"""Regression tests for the deployed backend research pipeline fixes."""
+
+from __future__ import annotations
+
+import asyncio
+from uuid import uuid4
+
+import httpx
+from unittest.mock import AsyncMock
+
+from nsct.config import AppSettings, AudioConfig, DatabaseConfig, LLMConfig, VisionConfig
+from nsct.orchestration.orchestrator import ResearchOrchestrator
+from nsct.orchestration.state import ResearchRunState
+from nsct.providers.llm import _LLMProviderImpl
+from nsct.providers.metrics import ProviderMetrics
+from nsct.providers.searxng import SearXNGProvider
+
+
+def _config(*, searxng_base_url: str | None = "http://searxng:8080") -> AppSettings:
+    return AppSettings(
+        llm=LLMConfig(base_url="http://llm.example/v1", model="test-model"),
+        vision=VisionConfig(base_url="http://vision.example/v1", model="vision"),
+        audio=AudioConfig(base_url="http://audio.example/v1"),
+        postgres=DatabaseConfig(url="sqlite+aiosqlite://"),
+        searxng_base_url=searxng_base_url,
+    )
+
+
+def test_llm_base_url_does_not_duplicate_existing_v1() -> None:
+    provider = _LLMProviderImpl(_config(), ProviderMetrics())
+    provider._api_key = "test-key"
+    client = provider._create_client()
+    try:
+        assert str(client.base_url) == "http://llm.example/v1/"
+    finally:
+        asyncio.run(client.close())
+
+
+def test_searxng_provider_normalizes_json_results() -> None:
+    async def run() -> None:
+        provider = SearXNGProvider("http://searxng:8080")
+        provider._client = httpx.AsyncClient(
+            transport=httpx.MockTransport(
+                lambda request: httpx.Response(
+                    200,
+                    json={
+                        "results": [
+                            {"url": "https://example.org/a", "title": "A", "content": "Snippet"},
+                            {"url": "mailto:invalid@example.org", "title": "Invalid"},
+                        ]
+                    },
+                )
+            )
+        )
+        try:
+            results = await provider.search("test")
+            assert len(results) == 1
+            assert results[0].provider == "searxng"
+            assert results[0].url == "https://example.org/a"
+            assert results[0].snippet == "Snippet"
+            assert provider._client is not None
+        finally:
+            await provider._client.aclose()
+
+    asyncio.run(run())
+
+
+def test_orchestrator_registers_searxng_provider() -> None:
+    orchestrator = ResearchOrchestrator(_config(), uuid4(), "test")
+    provider = orchestrator._get_multi_search().get_provider("searxng")
+    assert isinstance(provider, SearXNGProvider)
+
+
+def test_pipeline_executes_extracting_between_fetching_and_analyzing() -> None:
+    async def run() -> None:
+        orchestrator = ResearchOrchestrator(_config(), uuid4(), "test")
+        orchestrator._comparison_data = {}
+        for step in ("planning", "searching", "fetching", "extracting", "analyzing", "comparing", "synthesizing"):
+            async def handler(target: str = step) -> dict[str, object]:
+                orchestrator._transition_to(target)
+                return {"success": True, "data": {}}
+
+            setattr(orchestrator, f"_step_{step}", AsyncMock(side_effect=handler))
+
+        result = await orchestrator.run()
+
+        assert result["success"] is True
+        assert orchestrator._step_extracting.await_count == 1
+        assert orchestrator.state is ResearchRunState.COMPLETED
+
+    asyncio.run(run())
+
+
+def test_no_search_provider_marks_run_failed_with_reason() -> None:
+    async def run() -> None:
+        orchestrator = ResearchOrchestrator(_config(searxng_base_url=None), uuid4(), "test")
+        await orchestrator.start()
+        orchestrator._step_planning = AsyncMock(return_value={"success": True, "data": {}})
+
+        result = await orchestrator.run()
+
+        assert result["success"] is False
+        assert result["failed_step"] == "searching"
+        assert "No search provider" in result["error"]
+        assert orchestrator.state is ResearchRunState.FAILED
+
+    asyncio.run(run())