Project Audit — 2026-06-24¶

Index · Product Roadmap · V1 Release Roadmap · Technical Debt Tracker

Full audit of the Testo (testo-core) project. Covers code, docs, git history, CI, release gates, and open PRs. Intended as a point-in-time snapshot to inform the V1 Release Roadmap.

Executive Summary¶

Version: 0.1.0 (pre-release) Package: testo-core — NOT on PyPI Repo: GitHub

All 4 planned delivery phases are implemented in code. The project is feature-complete but not release-ready. Key blockers: release checklists never executed (210 unchecked gates), not published to PyPI, persistence module missing, 16 draft PRs with critical fixes unmerged.

Code Implementation Status¶

Fully Implemented (Production-Quality Code)¶

Component	Files	Maturity
Config engine (`testo_core/config/`)	`loader.py`, `schema.py`, `resolver.py`, `database_section.py`, `errors.py`	Mature
Orchestration engine (`testo_core/engine/`)	`orchestrator.py`, `executor.py`, `events.py`, `exit_codes.py`, `result.py`, `log_buffer.py`	Mature
Framework adapters (`testo_core/frameworks/`)	`pytest_adapter.py`, `behave_adapter.py`, `behavex_adapter.py`, `base.py`	Mature
Reporting pipeline (`testo_core/reporting/`)	Allure (3 modules), collector, exporter, native reports, server	Mature
Test pyramid (`testo_core/reporting/pyramid_viz.py`, `pyramid_data.py`)	ASCII renderer + tier aggregation, wired into `testo report pyramid RUN_ID` (2026-07-23)	Mature — was orphaned/unwired before this date, see "CLI-UI Parity" ADR
Reporters (`testo_core/reporting/reporters/`)	`allure_reporter.py`, `extent_reporter.py`, `testbeats_reporter.py`, `reportportal_reporter.py`, `factory.py`	Mature
Repository layer (`testo_core/repository/`)	`base.py`, `adapters.py`, `models.py`, `factory.py`, `report_archive_repository.py`	Mature
Services (`testo_core/services/`)	`headless_engine.py`, `dashboard_service.py`, `delta_service.py`, `failure_analysis_service.py`, AI providers	Mature
Docker runner (`testo_core/runners.py`)	1000+ lines, streaming, S3 sync, audit workflow	Mature
S3 client (`testo_core/s3_client.py`)	Boto3, singleton, auto-endpoint	Mature
Run history (`testo_core/run_history.py`)	Postgres integration, file snapshots, Allure URL	Mature
CLI (`testo_core/cli/`)	12 commands: run, config, report, cycles, doctor, clean, watch, diff, summary, init, version, plans	Mature
API (`testo_api/`)	16+ endpoints, SSE streaming, CORS, error handlers, request tracking	Mature
Frontend (`frontend/`)	7 pages: Dashboard, Execution, Runner Console, History, Run Detail, Compare, AI Settings	Mature
Docker infra (`docker-compose.yml`)	Postgres, MinIO, Allure static, sync daemon, bridge network	Mature
CI workflows (`.github/workflows/`)	`pr-fast.yml`, `pr-heavy.yml`, `release-gate.yml`, `nightly-external.yml`	Mature

Code Quality Metrics¶

Metric	Result
TODO/FIXME/HACK markers in production code	0
`NotImplementedError` stubs	0
Orphaned stub methods	0
Test files	123+ (unit, integration, contract, e2e)
Test tiers	3 (`tier_fast`, `tier_heavy`, `tier_external`)

Documentation Status¶

Obsidian Vault (`docs/`)¶

Section	Files	Status
Architecture	`Architecture Overview.md`, `Deep Dive - Execution Logic.md`	Complete
CLI Commands	`Command Reference.md`, `Troubleshooting and Error Codes.md`	Complete
Roadmap & Strategy	`Product Roadmap.md`, `UQO Engineering Hub.md`	Complete
Release Management	8 phase-specific checklists + `README.md` + `Delta Comparison Policy.md`	Templates only (see below)
Specs & ADRs	Repository pattern, packaging, delta engine, dashboard, Phase 4 AI, Allure 3 migration	Complete
Processes & Guides	CI/CD setup, Streamlit migration, E2E harness, ReportPortal, Allure 3	Complete
Testing Workflows	`Technical Debt Tracker.md`, `QA Strategies.md`	Complete

Documentation Gaps¶

No CHANGELOG.md at repo root
Release checklists are templates — all - [ ] unchecked across all 8 files (210 items)
README CI quickstarts — Phase 2 checklist requires GitHub Action and GitLab template quickstarts; not verified
No "last updated" dates on individual docs
No PyPI publishing guide
No Docker image publishing guide

Git & Development Status¶

Recent Activity¶

49 commits in recent history
Current branch: cursor/report-infra-e976a
Main latest: c86d7838 — "Merge branch 'feat/testo-report-archive-db-hybrid-cli'" (2026-06-24)
Active feature branches: 23
Cursor-driven branches: 30+ (bug inspection, test coverage)

Open PRs¶

Type	Count
Open (ready for review)	1 (PR #31 — Claude AI code review)
Draft	16
Total	17

Update — 2026-07-01: The 4 non-draft PRs open as of this date (#31, #35, #38, #39) were triaged and resolved: - PR #35 (sensitive-key redaction & failure context wiring) — merged as-is, checks were clean. - PR #38 (Sprint 4 — persistence module, publish pipelines, CHANGELOG, full Obsidian vault restore) — rebased onto main, 401 fast-tier tests passing, merged. - PR #31 (CI-based Claude AI code review via claude-cr.yml) — closed, not merged. Superseded by commit 990be121 (merged via #37) which had already replaced CI-based review with a local PrePush git hook (.claude/settings.json) after the original claude-cr.yml "referenced a non-existent action, blocked all PRs." - PR #39 (unified format → test → deploy CI pipeline, replacing pr-fast.yml) — kept the pipeline consolidation, dropped its ai-review job (a second, independently-built attempt at CI-based review, same conflict as #31) to avoid duplicating the local pre-push hook. Rebased, tests green, merged.

Net effect: code review on this repo is local-only (pre-push hook), by design — not CI-enforced. The 7 draft PRs (#11, #14, #15, #17, #18, #19, #20) were left untouched; drafts are intentionally out of scope for this pass. A new PR (#41, from a separate concurrent session) landed changelog/commitlint automation shortly after — not covered by this audit update.

Draft PR Breakdown¶

Category	Count	PRs
Critical correctness bugs	3	#22, #28, #30
Missing test coverage	4	#21, #23, #24, #26
Regression test coverage	4	#27, #29, and 2 others
Bug fixes	3	Execution race, sandbox, dockerless
CI automation	1	#31 (Claude code review)
Other	1	UI audit removal

CI/CD Pipelines¶

Workflow	Trigger	Timeout	Status
`pr-fast.yml`	All PRs	12 min	Superseded 2026-07-01 by `ci.yml` (see update note above)
`pr-heavy.yml`	`e2e-heavy` label / manual	30 min	Active
`release-gate.yml`	Manual only	45 min	Active
`nightly-external.yml`	Scheduled	—	Active
`ci.yml`	PRs + push to `main` + release	format 8min / test 12min / deploy 10min	Active — replaced `pr-fast.yml`; no `ai-review` job (see update note above)

Missing: No publish.yml (PyPI) or docker-publish.yml (container registry).

Release Gate Status¶

All 8 checklists: 0/210 items checked.

Checklist	Unchecked Items
Phase 1 Foundation	27
Phase 2 CI Integrations	36
Phase 2 Ghost Mode	18
Phase 2 Runner Image	17
Phase 3 Frontend Migration	24
Phase 3 Delta Engine	23
Phase 3 Unified Dashboard	34
Phase 4 AI and Failure Analysis	31

The Product Roadmap marks all phases [x] done (code delivery), but the operational go/no-go gates have never been formally executed.

Technical Debt Summary¶

From Technical Debt Tracker — 16 items total:

Priority	Open	Fixed
High	2	2 (exit-code drift, async-report-db)
Medium	6	0
Low	6	0

Key open items: - Dual execution stacks — modern vs legacy engine with duplicate logic (High) - Persistence module — referenced but never created (High) - Broad except Exception — silent degradation in Docker/S3/reporters (Medium ×5) - Sequential-only orchestrator — parallel stages = future work (Low)

Architecture Gaps¶

Gap	Severity	Notes
`testo_core/persistence/` module	High	Referenced in `orchestrator.py:228`, never created
Dual `EngineExitCode` definitions	Medium	`engine/exit_codes.py` and `headless_engine.py`
Streamlit UI coexistence	Low	`testo_ui/` still exists alongside React
No production deployment manifests	Low	Docker Compose only (local dev)
No Locust formal adapter	Low	Code in `runners.py` but no `LocustAdapter` class

Infrastructure Gaps¶

Gap	Impact
Not on PyPI	Users cannot `pip install testo-core`
No PyPI publish CI workflow	Cannot automate releases
No Docker image registry	Runner image not published
No Docker image publish CI workflow	Cannot automate image releases

Recommended Next Steps¶

See V1 Release Roadmap for the full task breakdown with 110 tasks across 11 workstreams, prioritized and ordered into 6 sprints (~10–15 working days to v1.0).

Audit Methodology¶

Full file tree exploration of testo_core/, testo_api/, frontend/, tests/, docs/
rg / grep scans for TODO, FIXME, HACK, NotImplementedError, orphan stubs
All 8 release checklists audited: grep -c '\- \[ \]' and grep -c '\- \[x\]'
gh pr list --state open for PR inventory
pip3 index versions testo-core for PyPI status
git log --oneline -50 for activity
Technical Debt Tracker cross-referenced with code