Troubleshooting

This page stays on the supported GitLab artifact-first PatchPatrol path. Use it when setup, diff planning, provider calls, policy gates, or GitLab delivery break and you need one concrete next move.

Use this order for every failure:

1. Confirm the exit code and first failing stderr line.
2. Open artifacts at AI_REVIEW_OUTPUT_DIR (default: .ai-review).
3. Check .ai-review/ai-review.json for the matching meta.* section.

Operational Diagnostics

Every completed review ends with a bounded Operational Diagnostics section in the job console. It describes the current execution; it is not part of the persistent GitLab Review Summary.

Normal output

Normal output is the routine handoff for developers and operators. A sanitized example looks like this:

Operational Diagnostics:
  Review Outcome: Attention needed
  Review Coverage: Full (4/4 validated chunks)
  Finding Counts: blocker=0 high=1 medium=2 suggestions=1 total=4
  Provider / Model: openai / review-model
  Total Duration: 42.80s
  Total Token Usage: calls=4 usage=4/4 prompt=18400 completion=2200 total=20600
  GitLab Delivery: updated (current, summary+inline)
  Artifact Paths: .ai-review/ai-review.json, .ai-review/ai-review.md, .ai-review/ai-review.html
  Exit Reason: SUCCESS (status 0)

The values come from the run that just finished. Use the artifact paths to open its Run Reports, and use the typed exit reason before changing configuration.

Verbose output

Set VERBOSE=1 when normal output identifies the failing area but you need deeper run evidence. The console groups these rows under Verbose Diagnostics:. The verbose tier adds resolved configuration, phase outputs and next actions, per-phase and per-call usage, chunk-plan and execution detail, incremental-review and GitLab delivery reason codes, state-recovery signals, and rerun side-effect counts.

Debug output

Set LOG_LEVEL=DEBUG for allowlisted structural troubleshooting fields such as report-section presence, metadata-key presence, finding counts, and bounded reason-code context. The additional rows appear under Debug Diagnostics:. Debug includes the verbose tier; it does not unlock raw runtime content.

No console tier prints credentials, API tokens, raw prompts, source excerpts, diffs, or raw provider request/response payloads. Use the typed reason codes and Run Report metadata instead of adding ad hoc secret-bearing logging.

Symptom Map

Exit semantics you can use to triage

Recovery lanes

Access and readiness failures

Use this lane when the run never really starts.

Common signals:

• CONFIG_ERROR: ...
• OPENAI_CONFIG_ERROR[...]
• TRUST_ALLOWLIST_ENDPOINT_MISSING
• TRUST_ALLOWLIST_BLOCKED_ENDPOINT

What to check:

• Run ai-review test --chat --readiness-json in the same environment the job uses for artifact-only provider/config readiness.
• Provide PATCHPATROL_LICENSE or PATCHPATROL_LICENSE_FILE for real-provider chat probes; license failures exit 12 before provider calls.
• Add --semantic-readiness only when the failing rollout enables diff+semantic.
• Run ai-review test --gitlab-readiness --readiness-json only when troubleshooting MR feedback delivery with the required GitLab token/context.
• Confirm the admin-owned handoff is complete: provider endpoint, model, GitLab project access, and merge request pipeline wiring.
• When AI_REVIEW_PROVIDER=openai, verify OPENAI_BASE_URL, OPENAI_API_KEY when required, and remove unsupported OpenAI-only knobs such as AI_REVIEW_NUM_CTX.
• When allowlist trust-gate signals appear, normalize the actual provider base URL and set AI_REVIEW_PROVIDER_ALLOWLIST_BASE_URLS to that exact value, or leave the variable unset to skip endpoint allowlist enforcement.

Next action: Fix the missing prerequisite first, then return to Access and roles or Configuration Reference before rerunning the review.

Self-Hosted License failures

Use this lane when stderr contains LICENSE_ERROR[...], readiness metadata uses the license failure category, or the command exits 12.

Inspect the configured runtime source without running review work or calling a provider:

ai-review license inspect --json

The JSON status is the License Status:

Use the first reason_codes entry for the concrete recovery action:

License evaluation fails before provider-backed work. When a blocked run can write artifacts, inspect meta.license; the raw Signed License and signature are never emitted. Empty findings on a license-blocked artifact mean no provider review ran, so empty findings do not mean the diff or codebase is clean.

Next action: Get ai-review license inspect --json to return valid before rerunning a real-provider review or test --chat probe.

Repository setup and diff resolution failures

Use this lane when PatchPatrol starts but cannot decide what to review.

Common signals:

• DIFF_CONTEXT_ERROR[INVALID_MODE_COMBINATION]
• DIFF_CONTEXT_ERROR[MISSING_MR_BASE_REF]
• DIFF_CONTEXT_ERROR[MERGE_BASE_NOT_FOUND]
• DIFF_CONTEXT_ERROR[GITLAB_DIFF_BASE_REF_NOT_FOUND]
• DIFF_EXTRACTION_ERROR[NO_WORKING_TREE_CHANGES]
• DIFF_EXTRACTION_ERROR[NO_STAGED_CHANGES]
• DIFF_EXTRACTION_ERROR[MAX_DIFF_BYTES_EXCEEDED]

What to check:

• Run ai-review run --mode auto --dry-run to print the resolved review context before a full rerun.
• Confirm the GitLab runner can fetch the target branch used as the merge base.
• For branch/MR runs, inspect meta.diff_base.strategy in .ai-review/ai-review.json: merge-base means PatchPatrol reviewed from the resolved merge base, while gitlab-diff-base-sha means it used GitLab's MR diff base SHA.
• For GitLab MR reruns, inspect meta.incremental_review: status=incremental means the provider reviewed the delta since the prior PatchPatrol-reviewed head SHA, while status=full records the full-review fallback reason.
• If meta.diff_base.status is error, read meta.diff_base.diagnostic, then fetch enough source/target history for git merge-base <base> <head> or GitLab's diff-base SHA to resolve.
• Inspect meta.limits.chunk_omitted_reasons, meta.limits.chunk_omitted_count, and the configured limits when large files or many files were skipped.
• Increase only the limit that matches the omission signal: AI_REVIEW_MAX_FILES, AI_REVIEW_MAX_DIFF_BYTES, AI_REVIEW_MAX_CHUNKS, or AI_REVIEW_MAX_FILE_DIFF_BYTES.

Next action: Get the diff plan into a known-good state first, then continue with First review output once artifacts are present again.

Provider, trust gate, and review-start failures

Use this lane when diff planning worked but the remote review step failed or never cleared the trust boundary.

Common signals:

• PROVIDER_ERROR[...]
• OPENAI_API_ERROR[...]
• OLLAMA_API_ERROR[...]
• EXIT_REASON[LLM_OUTPUT_INVALID]
• TRUST_ALLOWLIST_*

What to check:

• Run ai-review test --chat --readiness-json and read the final readiness.metadata line before changing provider settings.
• Use readiness failure_categories as the first branch: configuration means required settings are missing or invalid, license means the Self-Hosted License is missing or not trusted, trust_gate_blocked means the allowlist/redaction boundary stopped the call, authentication means credentials or scopes failed, model_unavailable means the endpoint did not list the configured model, network_connectivity means DNS/routing/proxy/endpoint reachability failed, structured_output_unsupported means schema-style response formatting is not accepted by the endpoint, and transport_retry_exhausted means retryable transport failures persisted through the configured attempt budget.
• In .ai-review/ai-review.json, inspect meta.provider_runtime.status, meta.provider_runtime.category, and meta.provider_runtime.reason_codes.
• Confirm the configured model exists at the endpoint and the runner can reach OLLAMA_HOST or OPENAI_BASE_URL.
• If TRANSPORT_RETRY_EXHAUSTED appears, increase AI_REVIEW_TRANSPORT_RETRY_MAX_ATTEMPTS only when transient network failures are normal for that CI path.
• If meta.limits.structured_output_reason_codes includes STRUCTURED_OUTPUT_JSON_SCHEMA_UNSUPPORTED_FALLBACK, the OpenAI-compatible endpoint rejected schema-mode output. Keep AI_REVIEW_STRUCTURED_OUTPUT_MODE=json for that endpoint, or enable json_schema / auto only after endpoint compatibility is verified.
• If the run stopped at EXIT_REASON[LLM_OUTPUT_INVALID], inspect meta.limits.error_flags and reduce diff pressure before retrying.
• If allowlist trust-gate signals appear before artifacts exist, return to Admin Quickstart and confirm AI_REVIEW_PROVIDER_ALLOWLIST_BASE_URLS exactly matches OPENAI_BASE_URL or OLLAMA_HOST, or leave the allowlist unset.
• If a trust-gate failure writes artifacts, read it as a blocked review: meta.trust_gate.status should explain the fail-closed reason, no provider call was made, and meta.usage_ledger.calls.call_count may be 0 with zero prompt/completion/total tokens.
• Do not treat findings: [] on an exit 11 trust-gate-blocked run as a clean review signal. It means PatchPatrol stopped before provider execution, so no merge request findings were produced.

Next action: Repair the provider or trust-boundary failure first, then re-run readiness checks and only then start another full review.

Policy and partial-coverage failures

Use this lane when PatchPatrol completed enough work to produce findings or omission data, but your configured gate still failed the run.

Common signals:

• EXIT_REASON[FAIL_ON_BLOCKER]
• EXIT_REASON[FAIL_ON_HIGH]
• EXIT_REASON[FAIL_ON_MEDIUM]
• EXIT_REASON[FAIL_ON_PARTIAL_COVERAGE]

What to check:

• Read ai-review.md first to confirm whether the gate was triggered by validated findings or by omitted chunks.
• In .ai-review/ai-review.json, inspect meta.limits.chunk_omitted_count, meta.limits.chunk_omitted_reasons, and any related error_flags.
• Confirm whether the current policy is intentional: AI_REVIEW_FAIL_ON sets the severity gate and AI_REVIEW_FAIL_ON_PARTIAL_COVERAGE=true turns omissions into a hard failure.
• Treat policy changes as a team decision, not an emergency workaround.

Next action: Decide whether to fix the findings, increase review coverage, or intentionally change policy in Configuration Reference.

GitLab delivery and artifact-location failures

Use this lane when the review artifacts exist or should exist, but developers cannot find them or GitLab feedback delivery failed after artifact writing.

Common signals:

• GITLAB_NOTE_ERROR[GITLAB_TIMEOUT]
• GITLAB_NOTE_ERROR[NETWORK_ERROR]
• GITLAB_NOTE_ERROR[GITLAB_HTTP_401], GITLAB_HTTP_403, GITLAB_HTTP_404
• GITLAB_NOTE_ERROR[MISSING_*]
• FEEDBACK_DRAFT_SUPPRESSED

What to check:

• Open the job artifacts first and confirm .ai-review/ai-review.html, .ai-review/ai-review.md, and .ai-review/ai-review.json exist.
• In .ai-review/ai-review.json, inspect meta.feedback.delivery.status and meta.feedback.delivery.reason_codes.
• Confirm the job publishes .ai-review/** as GitLab artifacts and that AI_REVIEW_OUTPUT_DIR matches the published path.
• If the run is still artifact-only, remember that missing MR notes are expected; the artifacts remain the source of truth.

Next action: Recover the artifact path first, then fix MR feedback delivery only if your team has already moved beyond the artifact-only baseline.

Suspicious throughput numbers

Use this lane when a report shows unexpectedly high or low token-per-second values, especially on local Ollama runs.

What to check:

• Start with Generation tok/s. For Ollama, this is the model decode-speed metric because it uses generated tokens over provider generation time.
• Treat Wall total tok/s as user-visible provider-call throughput, not decode speed. It divides total prompt plus completion tokens by PatchPatrol wall time, so a large prompt and short completion can look much faster than the model is actually generating tokens.
• Use Prompt tok/s to diagnose prompt ingestion pressure, Load time (s) for cold model loads, and Overhead (s) for Docker, network, wrapper, retry, or orchestration overhead outside provider runtime.
• If provider-native prompt/generation/provider-total rates are n/a, the selected provider did not expose comparable timing data. OpenAI-compatible endpoints may still provide token counts and PatchPatrol wall throughput.
• Do not enable raw prompt, completion, diff, API-key, or provider-payload logging for performance debugging. Use meta.usage_ledger and meta.performance_diagnostics metadata instead.

Next action: Compare Generation tok/s, Load time (s), and Overhead (s) before changing model size, review bounds, timeouts, or runner placement.

Read artifact metadata like a runbook

Open .ai-review/ai-review.json whenever the stderr line tells you what failed but not what to do next.

Runbook habit:

1. Start with the HTML or Markdown Run Report for the human-readable result.
2. Move to .ai-review/ai-review.json for the matching meta.* section.
3. Take one next action from the lane above instead of changing multiple variables at once.

Provider configuration failures

Use this shortcut when the access/readiness lane points to provider config specifically.

• OPENAI_CONFIG_ERROR[...] means PatchPatrol rejected provider settings before review execution.
• The most common public fixes are a valid OPENAI_BASE_URL, the right secret handoff for OPENAI_API_KEY, and removing unsupported AI_REVIEW_NUM_CTX when AI_REVIEW_PROVIDER=openai.
• After the fix, rerun ai-review test --chat --readiness-json with a valid Self-Hosted License before a full artifact-only review.

Provider runtime failures

Use this shortcut when review execution reached the provider but did not finish cleanly.

• OPENAI_API_ERROR[...], OLLAMA_API_ERROR[...], and PROVIDER_ERROR[...] belong here.
• Read meta.provider_runtime before changing timeouts or models so you know whether the failure came from transport, endpoint contract, or provider response.
• If the same failure repeats with TRANSPORT_RETRY_EXHAUSTED, fix endpoint availability or CI network path before widening the retry budget.

Diff extraction and scope misses

Use this shortcut when the repository/diff lane points to selection or omission problems.

• DIFF_CONTEXT_ERROR[...] means PatchPatrol could not resolve the intended review target.
• DIFF_EXTRACTION_ERROR[...] means PatchPatrol resolved the target but could not extract a usable review scope.
• meta.limits.chunk_omitted_reasons tells you whether the miss came from large files, chunk caps, or another bounded-review control.

GitLab feedback failures

Use this shortcut when artifacts exist but merge request delivery still failed.

• GITLAB_NOTE_ERROR[...] happens after artifact writing, so the first recovery move is always to open the artifacts.
• meta.feedback.delivery.status and meta.feedback.delivery.reason_codes tell you whether the failure was a real GitLab transport problem or a suppression state such as draft/manual gating.
• If your team is still in AI_REVIEW_FEEDBACK_MODE=artifact-only, keep using artifacts and do not treat missing MR notes as a broken baseline.

Artifact location confusion checklist

If the developer cannot find results after a run:

1. Verify the job publishes .ai-review/ai-review.html, .ai-review/ai-review.md, and .ai-review/ai-review.json as artifacts.
2. Confirm AI_REVIEW_OUTPUT_DIR still resolves to .ai-review.
3. If artifacts are missing entirely, inspect the runner log around ARTIFACT_ERROR[...] or exit code 6.
4. If needed, reproduce locally with ai-review run --output-dir ./tmp-review-debug --mode auto --dry-run.

Return to recovery flow

Use these pages once the immediate failure is understood:

• Configuration reference
• Access and roles
• First review output
• Feedback modes
• Reference