RPT-25 — AI Quality, Safety & Spend

Certified report for AI owners, Risk and Finance: runs, evaluation results, latency, cost and user feedback per AI use case. Metric owner: Vikram Nair (AI Platform Owner) · spend co-owner: Fatima Rashid (Finance Manager).

RPT-25 AI-08 §9.2

Evaluation console Gateway console

Report certification & governance Operational — daily slice Certified — Jun 2026 monthly close

Report owner: Vikram Nair — AI Platform Owner
Business definition: RPT-25 definition v1.4 (approved 02 Jun 2026)
Authoritative dataset: ai_gateway.run_log_v3 → mart mart_ai_runs_daily
Dimensions: Use case · model/provider · environment · company · branch
Currency / time zone: AED · Asia/Dubai (GST) · calendar month

Refresh cadence: Daily 06:00 GST · monthly certified at close
Last refresh: 17 Jul 2026 06:04 GST On time
Next refresh: 18 Jul 2026 06:00 GST
Governing policy versions: AI eval policy AI-08 v2.1 · AI budget AIB-2026-07 v3 · Privacy CMP-07 v2.0
Reconciliation: Gateway cost = provider invoices · diff AED 0.00 Pass

Runs — 16 Jul

4,182

▲ 6% vs 7-day average

Eval pass rate

96.2%

Golden + adversarial sets · floor 95%

P95 latency

2.8s

SLO ≤ 4.0s · fallback used 1.1%

Spend MTD

AED 18,420

Budget AED 25,000 · 74% consumed

Threshold breaches

▼ 1 kill switch active (valuation)

Edit / reject rate

7.4%

User feedback captured on 61% of runs

Report control (Table 15) — privacy-aware logs & use-case thresholds Prompts and outputs are logged under privacy policy CMP-07 v2.0: PII redacted at ingestion, originals retained 30 days in the restricted vault only. Per-use-case budget and eval thresholds are enforced at the gateway (AI-08); breaching a threshold routes to fallback or defers noncritical requests. Access scope: AI owners, Risk, Finance. Export is watermarked “RPT-25 · Grace Okafor-viewable · 17 Jul 2026” and recorded in the Audit Explorer.

Spend MTD by use case (AED)Jul 2026 · 1–16

Use case group	Spend MTD	Budget	Consumed
Document extraction — AP invoices	6,210.40	7,500.00	83%
KYC document extraction	3,640.25	5,000.00	73%
RAG knowledge assistant	3,080.10	4,500.00	68%
Lead / property recommendations	2,150.34	3,000.00	72%
Anomaly intelligence	1,560.00	2,500.00	62%
Other (5 use cases)	1,779.10	2,500.00	71%
Total = provider invoices Jul MTD	18,420.19	25,000.00	74%

Lineage — source to report (§9.2)Model registry

ai_gateway.run_log_v3→ evals.eval_result_v2→ billing.provider_invoice_v1→ PII redaction · CMP-07→ transform mart_ai_runs_daily v1.4→ RPT-25

Source	Version	Refreshed	State
`ai_gateway.run_log_v3`	schema v3.2	17 Jul 2026 06:01	Fresh
`evals.eval_result_v2`	golden set GS-2026-06	17 Jul 2026 05:40	Fresh
`billing.provider_invoice_v1`	Jun invoice final · Jul accrual	16 Jul 2026 23:10	Accrual basis MTD
`feedback.user_signal_v1`	schema v1.1	17 Jul 2026 06:02	Fresh

Drill-down is permission-aware: run payloads open in the Evaluation & Observability workspace only for AI owners and Risk; Finance sees cost aggregates without prompt content.

Use-case quality, safety & spend — Jul 2026 MTD (production) 10 use cases

Use case / pinned model	Runs MTD	Eval pass	Grounded	P95 lat.	Edit/reject	Cost MTD (AED)	Threshold status
Document extraction — AP invoices AI-04 · extract-l2 snapshot 2026-05-11 · prompt v18	21,410	97.8%	—	2.1s	4.2%	6,210.40	Within thresholds
KYC document extraction AI-04 · extract-l2 snapshot 2026-05-11 · prompt v11 · PII-redacted logs	6,880	96.9%	—	2.4s	6.1%	3,640.25	Within thresholds
RAG knowledge assistant AI-03 · gen-m3 snapshot 2026-06-02 · citation required	9,340	95.4%	93.8%	3.6s	9.8%	3,080.10	Groundedness 93.8% < floor 95% — tuning ticket AI-1182
Lead & next-action recommendations AI-05 · rank-s1 snapshot 2026-04-20 · prompt v9	12,205	96.6%	—	1.2s	11.4%	1,410.20	Within thresholds
Property matching AI-05 · rank-s1 snapshot 2026-04-20	7,118	97.1%	—	1.4s	8.9%	740.14	Within thresholds
Valuation advisory AI-05 · val-r2 · prompt v6 candidate blocked from promotion (AI-02)	312	91.3%	—	4.8s	22.0%	402.55	Kill switch ON 15 Jul — eval below 95% floor · reviewer approval required for external use
Finance anomaly intelligence AI-06 · anom-d1 snapshot 2026-03-30	2,940	96.0%	—	5.9s batch	3.1%	1,560.00	Within thresholds
WhatsApp reply drafting gen-m3 snapshot 2026-06-02 · consent-checked · DND enforced	4,010	95.8%	96.2%	2.9s	14.6%	610.85	Within thresholds
Arabic translation assist gen-m3 snapshot 2026-06-02 · approved terminology OPS-01	1,875	96.4%	—	2.2s	7.7%	421.30	Within thresholds
Contract clause suggestion gen-m3 · advisory only — human approval boundary AI-07	640	95.1%	94.9%	3.8s	18.3%	344.40	Budget 92% consumed — noncritical requests deferring to fallback (AI-08)
Totals — reconcile to gateway billing export GB-2026-07	66,730	96.2%	94.7%	2.8s	7.4%	18,420.19	Diff vs provider invoices AED 0.00

Report definition of done (§9.2)All items evidenced

§9.2 item	Evidence on this report	State
Named owner & business definition	Vikram Nair · definition RPT-25 v1.4	Done
Authoritative dataset & lineage	`mart_ai_runs_daily` · lineage panel above	Done
Filters, dimensions, currency/TZ, refresh & certification	AED · GST · daily 06:00 / monthly certified	Done
Permission-aware drill-down & controlled export	Watermarked audited export · Finance sees no prompt content	Done
Reconciliation totals & tolerances	Cost ties to provider invoices · diff AED 0.00	Done
Accessible table/chart alternative · EN/AR labels	Every chart has table twin · labels approved OPS-01	Done

Static render mockup — no live data. Source: ERP Modern Development Specification v2.0 · Covers: RPT-25, AI-08, §9.2.