Testing Operations Analytics Platform

The data and AI backbone of a global remote testing operation, designed, built, and maintained by one person. When I joined, operational reporting lived in fragile, ad-hoc notebooks run by hand. I rebuilt it into a production platform of roughly 195 Python scripts and 87,000 lines of code that powered the operation's financial reporting, hours accounting, quality metrics, and automated communications.

Platform and pipelines

I migrated the entire reporting layer from ad-hoc notebooks to a production-grade system: multiple pipeline orchestrators coordinating multi-stage data flows on scheduled runs, with structured logging, error handling, and retries. It became the single maintained source of operational reporting instead of scripts executed manually.

Financial and billing system

Contractor payments spanned multiple vendors, each with different rates and markups, reconciled by hand. I built an object-oriented billing pipeline with dedicated processor classes, vendor-specific markups, and activity-based cost allocation, plus a suite of cost-analysis and forecasting scripts. It processed millions of dollars a year in payments with a full audit trail and gave leadership cost visibility they could rely on.

Multi-source reconciliation

Hours worked lived across four separate scheduling and time-tracking systems that never agreed. I built an engine that unified all four into a single source of truth, with premium-hour tracking, validation, and overtime detection, eliminating weekly manual reconciliation.

LLM-powered reporting

Performance reporting was manual, inconsistent, and did not scale. I built an automated reporting system on Google Gemini and OpenAI models using context injection, accuracy-based tone modulation, structured machine-parseable output, and chain-of-thought aggregation.

The system's centerpiece was individualized performance reporting at scale: for every team member, it generated a personalized report comparing their metrics against team benchmarks and surfacing specific, tailored recommendations, work that had previously required manual review by team leads. It ran in parallel with checkpoint and resume, used versioned prompt templates, and flowed from the data warehouse through PDF generation to automated distribution, standardizing more than two dozen report types.

Infrastructure and security

I ran the platform on ten scheduled GitHub Actions workflows (hourly, daily, and weekly cron), including a self-hosted runner for memory-intensive jobs. Workflows assumed a scoped, OIDC-federated IAM role for short-lived AWS credentials, keeping long-lived keys out of the pipeline entirely, and managed the pipeline's own S3 and its encrypted, locked remote state within that role's permissions. On the GCP side, all secrets were centralized in Google Cloud Secret Manager with no credentials hardcoded anywhere; a single credential layer detected its environment automatically, so the same code sourced secrets correctly whether running locally or in CI. Infrastructure was managed as code with Terraform, within which I configured least-privilege access and cost-attribution tagging.

Also built

Machine-learning anomaly detection for operational metrics, rolling KPIs feeding real-time dashboards, a browser-automation suite, and integrations across eight-plus external APIs unified into a single data layer.

Stack: Python, SQL, BigQuery, pandas, scikit-learn, Gemini, OpenAI, GCP, AWS, GitHub Actions, Docker