Panagiotis (Panos) Gkilis

AI & Backend Engineer • ML Reliability & Evaluation • Applied AI • Speech Systems • Automation

I build production AI systems — and the instruments that catch them failing silently.

A fault that returns an error is one you fix that afternoon. A fault that returns a plausible success is one you ship. Everything below carries an instrument built for the second kind — an evaluation gate that can refuse a deployment, a linter that fails a training run before the GPU hours are spent, a coverage layer that reports what it could not check instead of staying silent.

AI engineer and solo builder of BedVibe Studios and related products, spanning multilingual TTS datasets, model workflows, inference APIs, audiobook tools, avatars, backend services, subscriptions, authentication, and production deployment. Creator of ttsproof, trainproof, notchecked and spkproof — four open-source Python libraries on PyPI — and author of thirteen citable research records with DOIs — eleven publications and two software releases.

The Evidence Stack — one architecture, one rule

Five systems, not five projects. At every layer, an absent or invalid value must never be able to read as a good one.

  • Retrieval — hybrid cosine + BM25, behind a deploy gate that refuses to ship a corpus no evaluator has scored
  • Coverage — an answer grounded in six chunks scores a perfect faithfulness and can still be false about the 997 nobody read
  • Contracts — eight distinct MCP failures, each with its own remedy, instead of one generic error
  • Traffic — bounded admission and deadline-aware failover; load shedding took usable answers from 1 to 6 out of 32
  • Observation — one address in, what was actually observed out. Read-only by contract and by test
This page is intentionally written as proof of work. Every project below is live or publicly documented, linked for direct verification, and included to show what I built, which stack was involved, and how the system can be checked.

Selected Projects

Each project card shows a live product or documented system, the technical scope behind it, and the business or product function it serves. The goal is clarity: what was built, which technologies were used, and how the work can be checked when a public verification link is available.

From-Scratch Speech Model & Training-Data Engine

Built the lower-level ML and training infrastructure behind the speech system: a 730M-parameter AR + NAR neural-codec speech model trained from scratch on consumer GPUs, spanning 13 languages, 15 human speakers plus 2 synthetic voices, and 6 emotional states with metadata conditioning — three independent spans, not a crossed grid. To make a 108,000-sample corpus trainable on limited hardware, I designed a custom memory-mapped binary dataset format for zero-copy loading and a Rust metadata pipeline for fast, validated data preparation — backed by an automated verification suite with ablation probes to catch silent training failures before they reach production.

Key results
  • 730M-parameter AR + NAR neural-codec model trained from scratch on consumer GPUs
  • 108,000-sample multilingual corpus spanning 13 languages, 15 human speakers plus 2 synthetic voices, and 6 emotional states — three independent spans, not a crossed grid
  • Published engineering report with a citable DOI (Zenodo)
What I built
730M AR + NAR model Trained from scratch ML training infrastructure Custom binary dataset format Memory-mapped data loader Rust metadata pipeline Verification & ablation suite
Stack
Python PyTorch Rust EnCodec / DAC codec SentencePiece ECAPA-TDNN embeddings

Grounded RAG Portfolio Agent — a Chatbot That Validates Itself

The chat button on this page is a live, grounded assistant that answers questions about my papers and projects from an indexed, provenance-tracked corpus only — every answer cites its sources, and a Retrieval X-ray exposes the raw retrieved chunks with their real relevance scores. It ships behind an adversarial eval gate (answerability, refusal, prompt injection, credential traps), and a self-maintaining pipeline re-validates and redeploys it only when it still passes. Retrieval runs on one of three interchangeable vector backends, and the whole answer path is traced span by span.

Key results
  • 32/32 on the adversarial eval suite — answerability, refusal, prompt injection and credential traps; retrieval hit@6 20/20 (hybrid retrieval: static embeddings + BM25). Re-run in full as a gate on every corpus change
  • Three vector backends benchmarked, the simplest one shipped — brute-force NumPy 0.86 ms, FAISS 0.035 ms, Qdrant over gRPC 1.3 ms and over REST 12.4 ms, so transport cost 9× the database itself. The crossover where FAISS wins by 302× is measured and written down, not guessed
  • Request p50 12.41 → 2.91 ms after tracing found the agent was constructing a new model client on every single answer — 8.27 ms at p50, more than twice the cost of all retrieval combined
  • A deliberately broken approximate index that finds only 14% of correct neighbours still produced right answers 97% of the time, because the keyword half of the retriever silently carried it — which is why the index is monitored directly against an exact baseline rather than by watching answer quality
  • Self-maintaining — an eval-gated pipeline detects drift, re-tests, and redeploys only on a green suite; a red eval blocks the deploy
  • The eval suite caught a real prompt-injection breach and a factual error in a published paper — both fixed pre-launch (paper republished as Zenodo v2)
What I built
Grounded RAG pipeline Hybrid retrieval (embeddings + BM25) Adversarial eval harness Self-maintaining sync (eval-gated) Injection hardening Retrieval X-ray UI Provenance-tracked corpus Pluggable vector backends Recall@k regression harness Per-page scope boosting Cross-page session memory End-to-end distributed tracing Derived deploy manifest
Stack
Python FastAPI Static embeddings (model2vec) BM25 FAISS (IVF / FlatIP) Qdrant (HNSW, gRPC + REST) OpenTelemetry Langfuse Jaeger SQLite Gemini API Vanilla JS nginx + systemd

The Evidence Stack — One Agent Architecture Where Every Layer Separates “No” From “I Don’t Know”

The five systems below are not five projects. They are one stack, built to a single rule: at every layer, an absent value must never be able to read as a good one. A retrieval agent answers from six chunks of a thousand and reports it exactly as it would report reading all of them. A gateway returns HTTP 200 to a caller who gave up two seconds ago. A tool server changes a field name and the client keeps parsing. Each of those is the same failure wearing different clothes, and each layer here is instrumented to make it visible instead of silent.

The layers talk over MCP, which is the part that makes it an architecture rather than a collection: one client, several servers, one contract, and a boundary where what crosses it is checked rather than trusted. The client discovers a server’s tools at runtime and validates every later call against what that server actually advertised — not against what the client assumed when it was written.

The five layers, and what each one refuses to let pass
  • Retrieval — the grounded agent. Hybrid cosine + BM25 over a 1,000-chunk corpus with per-page scope, and a deploy gate that refuses to ship a corpus no evaluator has scored. The proof is fingerprinted on corpus, questions, prompt and model, so a cached pass cannot certify changed behaviour
  • Coverage — the accounting layer. An answer grounded in six retrieved chunks scores a perfect faithfulness 1.0 and can still be false about the 997 nobody read. Coverage is recorded per claim, and the required coverage is set by the shape of the sentence: “the policy says X” needs only the chunk it cites; “there is no policy about X” needs all of them
  • Contracts — the MCP client. Eight distinct failures, each with its own remedy, because an agent’s next move differs for every one: the server never started, the tool is gone, the arguments changed, the deadline passed, the reply parsed but broke the contract. A surface digest pins tool names, schemas and the declared meaning of each field, so a silent rename fails a test instead of a user
  • Traffic — the model gateway. Per-key quota and token metering, bounded admission, deadline-aware retry and failover. Load shedding took usable answers from 1 to 6 out of 32 — without it, 32 callers all got HTTP 200 and 31 had already gone
  • Observation — the sensor lookup. One address in, what was actually observed out. Lookup, never listing; observation, never accusation. Read-only to the production host by contract and by test
Why it is built this way

Every evaluation framework in this space measures whether an answer is faithful to what was retrieved. None measures whether what was retrieved was enough to support the shape of the claim being made. That gap is the reason the coverage layer exists, it is measured on a live system rather than argued from theory, and it is published as an open-source library so the accounting can be checked rather than taken on trust.

What I built
MCP client with runtime tool discovery Contract + surface-digest validation Retrieval coverage accounting Claim-shape gating Evidence-anchored provenance Structural claim comparison Deploy gate on scored proof Per-key quota + metering Deadline-aware failover
Stack
Python MCP (stdio) FastAPI asyncio SQLite OpenTelemetry pytest ADR-driven design

Model Gateway — Admission Control, Deadlines and Failover in Front of Several LLM Backends

A single OpenAI-compatible endpoint sitting in front of multiple inference backends, with per-key authentication and spend quota, token metering to SQLite, bounded admission, deadline-aware retries, failover and streaming. Every backend is one file and they speak entirely different protocols, so the layer above the engine does not know or care which engine is underneath. I was asked to put vLLM on the box and did not: the GPU already carries a live production text-to-speech service holding 7.4 of 16 GB, vLLM has no Windows build, and its default configuration reserves 90% of the card. That decision is recorded as an ADR rather than hidden, along with what it costs — continuous batching is what vLLM is for, and I did not measure it.

Key results
  • Load shedding took usable answers from 1 to 6 out of 32. With no admission control, 32 callers on a 700 ms deadline all received HTTP 200 — and 31 of those arrived after the caller had already given up. A 100% success rate with one usable answer. Bounded, with a fast 429: eight answered, twenty-four refused in milliseconds, and the success rate fell to 25% while goodput rose six-fold
  • Failover 0 → 8 in time, p50 3001 → 1046 ms. A backend made slow rather than dead had the gateway logging eight successes without ever calling the healthy backend once, because a retry budget counted attempts instead of watching the clock. The deadline now belongs to the request, and time is reserved for the backend being failed over to
  • Gateway overhead 3.47 → 0.60 ms. I predicted the quota check was the hotspot and was wrong — it was 0.146 ms, about 4%. Three quarters of the overhead was one metering row flushing to disk before answering
  • Admission limit set at 8 because that is where the knee is — measured saturation of 189 tok/s at one concurrent request, 433 at eight, then flat: 474 at thirty-two for 3.3× the latency
What I built
OpenAI-compatible API surface Per-key auth and spend quota Token metering and pricing Bounded admission / load shedding Deadline-aware retry budget Backend failover Streaming responses Fault-injection benchmarks
Stack
Python FastAPI asyncio Ollama Gemini SQLite (WAL) OpenTelemetry + Jaeger pytest (94 tests) ADR-driven design

Agent Framework Evaluation — CrewAI Measured Against a LangGraph Pipeline in Production

I built the same real task four ways and ran all four against eight fault scenarios, to decide on evidence whether to move my live content-automation pipeline to CrewAI. The LangGraph arm is not a rewrite — it calls the production graph builder itself, with only the image picker and the model call swapped, and both arms are graded by the live validator and rendered by the live publishing renderers. The answer was no, and it is supported by numbers rather than preference.

Key results
  • Plain CrewAI returned success while publishing content that violates my own rules, in two of eight scenarios, with nothing a machine could act on. The graph stopped and said why
  • Made strict, CrewAI costs 4 model calls and 9,466 characters of prompt against the graph's 3 and 2,319; it installs 683 MB across 143 packages against 32 MB and 36, and imports in 2 s against 0.6 s. Its own cost meter reports three times the truth when several agents share one model object
  • The finding that mattered most: I reproduced a real production failure and all four implementations passed it, because the output had the right shape. A framework decides what happens when a step misbehaves; it never decides whether that step should have been a model call at all. The real fix was removing the model call from the send path entirely
  • 77 tests plus one deliberate xfail in ten seconds with zero API calls — and the zero is proven, by a scripted local stub and a test that makes every outbound connection fail
What I built
Four-way implementation harness Eight fault scenarios Production-graph reuse (not a rewrite) Cost and prompt-size instrumentation Network-isolation proof Framework decision record
Stack
Python LangGraph LangChain CrewAI FastAPI n8n pytest (77 tests + 1 xfail)

GEO Observatory — Live AI-Crawler Observability Platform

A deployed observability platform that records how AI crawlers and search engines discover my sites. A daily collector pulls crawler events from Cloudflare’s GraphQL Analytics API into an append-only SQLite event ledger (raw payloads preserved, deterministic dedup hashing, hourly buckets), and a public dashboard exposes live per-crawler timelines — ClaudeBot, GPTBot, PerplexityBot, Googlebot — under a strict honesty policy: sampled aggregates labeled as such, temporal association only, no fabricated causation or confidence scores.

Key results
  • Live in production — public dashboard with real data, collected automatically every day (systemd timer, hardened non-root service)
  • Probe-gated integration — the Cloudflare GraphQL provider was built only after a verification probe confirmed the exact datasets and fields available on the plan
  • Day-one catch — logged a spoofed “Googlebot” (a Google-Cloud scanner hunting for leaked .env files) within hours of the subdomain’s TLS certificate appearing in transparency logs
What I built
Provider architecture Append-only event ledger Cloudflare GraphQL ingestion Bot classification (AI / search / SEO) Public dashboard + llms.txt Daily automated collection
Stack
Python Flask SQLite Cloudflare GraphQL API nginx + systemd Hetzner

Web Control — Gated Browser Automation for AI Agents

A browser-control platform exposed to AI assistants over MCP (Model Context Protocol), built around the question most agent tooling skips: what stops an agent from doing something irreversible, and what stops a web page from telling it to? Interactive tools handle exploration and page-local work, but anything that changes the world outside the machine runs only inside a workflow, behind an approval gate that cannot be satisfied non-interactively — there is no auto-approve path in the codebase. Every piece of web-derived text enters agent context explicitly wrapped as untrusted data rather than instructions, with extraction caps and paging so a hostile page cannot flood the context window. Layered so the browser engine is usable as a plain Python library with no MCP involved.

Key results
  • Prompt-injection boundary by construction — page text is framed as untrusted at the tool-result level, not left to the model to infer, with explicit truncation markers reporting the full size
  • Credential-exfiltration primitive closed — arbitrary JavaScript execution is disabled whenever a persistent logged-in profile is active; ephemeral sessions keep it
  • Auditable by default — domain allowlist, append-only audit log with redaction of sensitive typed text, stdio-only transport, and local identity stores that are never copied, synced or logged
  • Real workflows shipped — sitemap resubmission, indexing requests and article publishing, each emitting a documented result-file contract that downstream tooling reads
What I built
MCP server (stdio) Workflow engine + approval gates Untrusted-content framing Domain allowlist Append-only audit log Persistent auth profiles Layered browser/tool separation
Stack
Python Playwright FastMCP Pydantic pytest
BedVibe TTS platform screenshot

BedVibe TTS Platform

Built and deployed the user-facing BedVibe TTS product layer: authenticated accounts, token accounting, Stripe checkout and subscriptions, proprietary multi-voice speech generation endpoints, voice/language/emotion selection, custom voice support, token-aware generation, and production backend services for real user traffic. The product exposes creative speech controls such as Flow/Chaos generation parameters, voice warmth, equalizer and reverb, HD enhancement/denoise, an inline library of ~60 per-speaker expressive vocal effects (laughs — sarcastic, sinister, hysterical, escalating — gasps, breaths, coughs, throat-clears, wheezes, hiccups and shivers, and more) inserted by right-click, background ambience mixing, optional viseme timing output, and long-form audiobook/project workflows. Voices are proprietary models trained on curated datasets, and on a target-language change the product offers automatic text translation (for example English → Japanese) or keeps the original text to be spoken in the chosen language's accent. This card represents the deployed application and business system around the speech models — not the from-scratch model-training work itself.

Key results
  • 13 languages with per-language accent synthesis and optional automatic translation
  • ~60 inline expressive vocal effects per speaker, inserted directly into text
  • In production: Stripe checkout + subscriptions with token-aware generation
Generation, Audio & Multilingual Controls

The interface exposes direct generation and audio controls rather than a single opaque “generate” button. Flow controls speaking pace, allowing users to move delivery faster or slower. Chaos controls sampling temperature, allowing more stable or more varied output. Brightness is an EQ-driven tone control: lowering it emphasizes lower-frequency body, while increasing it reduces low-frequency weight and increases high-frequency presence; this can also help tame sharp sibilance. Space controls reverberation amount for a drier or more spacious voice presentation.

HD mode combines spectral and psychoacoustic enhancement with DeepFilterNet cleanup to improve perceived clarity and reduce unwanted noise. A separate diffusion enhancement path exists for further output-quality improvement, but it is disabled in the current interactive path because its latency is not suitable for live generation.

Each available voice can be auditioned through sample playback by selecting the performer portrait. The product supports 13 languages and, when a user changes the target language, asks whether to translate the text into that language or keep the original text and synthesize it with the selected language accent. Translation uses a local Argos Translate path, with a Google-based translation option available for paid subscription workflows where higher-quality translation is requested. Custom voice cloning is not enabled as open self-service; it is handled only for rights-cleared, permissioned requests.

What I built
Deployed TTS SaaS layer Proprietary generation API Voice / language / emotion selection Custom voice support Flow / Chaos controls Warmth / EQ / reverb HD denoise / enhancement 60+ inline vocal effects Background ambience mixing Viseme timing output Auth / token wallet system Stripe checkout + subscriptions Production backend APIs Auto text translation Curated-dataset voices Author intake + rights attestation
Stack
Python FastAPI Supabase PostgreSQL / SQL Stripe FFmpeg DeepFilter / df.exe NumPy / SoundFile JavaScript Linux / Nginx

Production Backend Infrastructure — Auth, Billing & Device Licensing

Architected and operate the full backend behind BedVibe's products: a fleet of independent services handling user authentication and registration, Stripe billing and webhook processing, device-bound (MAC-ID) software licensing, subscription and entitlement management, a TTS inference job queue, and security telemetry integrations. Runs on a self-managed Linux server behind nginx, with each service isolated as its own systemd unit and PostgreSQL as the shared data layer — provisioned and maintained at the operating-system level.

Key results
  • 10 independent services, each isolated as its own systemd unit behind nginx
  • Self-managed Linux server: PostgreSQL data layer, TLS, OS-level provisioning
  • Replay-safe Stripe webhooks and MAC-ID device licensing running in production
What I built
Authentication & registration Stripe billing + webhooks MAC-ID device licensing Subscription management PostgreSQL data layer TTS inference job queue Security telemetry integration Self-managed Linux ops Compliance-aware intake workflow
Stack
Rust (Axum / Actix-web) PostgreSQL 16 nginx systemd ufw Certbot / TLS Stripe Linux (self-managed VPS)

BookProof — AI Book Summary Auditor

Built a reproducible workflow for auditing AI-generated book summaries and answers against long manuscripts. BookProof checks evidence grounding, missing details, unsupported claims, answer-term coverage, and evidence-term coverage. The underlying LongBook Verifier research package is published on Zenodo and includes retrieval baselines, model-output scoring, plots, reports, and a 240,767-word long-form stress-test benchmark, while excluding private manuscript text from the public release. A follow-up ablation study (Experiment C) diagnoses why hierarchical book RAG underperformed chapter-summary retrieval on this corpus — isolating first-stage chapter selection and neighbor-expansion dilution as the two failure modes — and is published as a separate Zenodo record.

Key results
  • 240,767-word long-form stress-test benchmark
  • Two published Zenodo records — research package + Experiment C retrieval ablation
  • Evidence-grounding and retrieval-baseline scoring, fully reproducible
What I built
Long-book benchmark Evidence-grounding evaluator Retrieval baselines Model-output scoring Research package Zenodo DOI release Hierarchical ablation (Exp. C)
Stack
Python JSONL CSV metrics Matplotlib Retrieval evaluation Long-document QA
How I build a RAG system, end to end

Built the workflow as a complete RAG/evaluation pipeline: ingestion, structure-aware chunking, retrieval baselines, citation-grounded generation checks, gold-question scoring, and controlled ablations across retrieval strategies.

Automated Failure-Mode QA Framework for Neural TTS — Published Technical Report

A backend-only, fully automated quality-assurance framework for a reference-based neural text-to-speech system. For each edge-case input it normalizes the text, runs ASR-independent structural audio checks (empty / short / long audio, long silence, clipping, loop/repeat, end-of-clip artifact), scores pronunciation with equivalence-aware WER/CER, and — where ASR is itself unreliable — isolates the case for review instead of guessing. Evaluated on 130 edge cases across 3 neutral voices (390 samples): zero structural audio-integrity defects. A one-time blinded human validation of the 42 ASR-uncertain cases (with 15 ASR-passed controls; controls 15/15) showed the quarantined zone is a genuine 45/55 mix — 19 real TTS mispronunciations and 23 ASR false-negatives — confirming quarantine-for-review is the correct design, and localizing the synthesizer's weak spot to short letters/acronyms. Published as a citable technical report with an explicit claims-and-limitations statement.

Key results
  • 390 samples (130 edge cases × 3 voices) — zero structural audio-integrity defects
  • Blinded human validation — 15/15 controls correct; quarantine design confirmed
  • Published as a citable technical report (Zenodo DOI, CC-BY-4.0)
What I built
Automated TTS QA harness Edge-case text normalization ASR-independent audio checks Silence / clipping / loop / tail-artifact detection Equivalence-aware WER / CER ASR-uncertainty quarantine Blinded human validation (controls 15/15) Failure-mode taxonomy CI threshold gate Reproducibility scripts Zenodo DOI release (CC-BY-4.0)
Stack
Python faster-whisper (ASR) numpy / soundfile Reference-based neural TTS

ttsproof — Open-Source TTS QA & Benchmark Library (PyPI)

Published the QA-framework method above as an installable open-source library: pip install ttsproof. It stress-tests any TTS engine through a one-command benchmark — structural audio checks that need no model (loops, silences, clipping, duration explosions), equivalence-aware WER/CER on canonical spoken form, and an ASR-uncertainty quarantine — then renders a self-contained HTML report with per-category score bars, waveforms, and audio players for every failure. Engines plug in via a command template or a Python callable; a regress command gates CI on category-level quality drops.

Key results
  • Live on PyPIpip install ttsproof (MIT, sole author)
  • Benchmark Corpus 1.0 — 817 curated edge cases across 39 categories, versioned independently of the tool so published scores stay comparable
  • Closed-source engines supported — documented SpeechSDK integration benchmarks commercial models (OpenAI, ElevenLabs, 17+ providers); shipped within a day of the request from the SpeechSDK team at Jellypod
  • CI regression gate + self-contained HTML failure reports; method backed by a published technical report (DOI)
What I built
Built-in benchmark corpus (817 cases) Policy-based scoring (strict / keywords / structural) Structural audio checks Equivalence-aware WER / CER ASR-uncertainty quarantine HTML reports (waveforms + audio) Engine compare + CI regression gate PyPI packaging + CLI
Stack
Python numpy / soundfile faster-whisper (optional) pytest setuptools / twine

trainproof — Deterministic Reliability Layer for ML Training (PyPI)

pip install trainproof. A linter for ML training runs that catches broken fine-tunes across the whole lifecycle — before training (dataset + tokenizer pre-flight), during (a live HuggingFace callback with step-time telemetry that can abort a doomed run), after (divergence / dead-run / overfitting / NaN / gradient-spike detection from log files), and against a baseline (N-way ratio rules that catch failures a single loss curve can't see). The flagship trainproof doctor . auto-discovers every training log in a directory and prints a triage-sorted autopsy. Every rule is a deterministic threshold with a stable ID (TP-DIVERGE, …) and cited evidence — no inferred causes, no invented confidence scores. Sibling of ttsproof (which it reuses for speech-dataset checks).

Key results
  • Live on PyPIpip install trainproof (MIT, sole author); published releases behind a 470-test suite, exposing 98 stable rule IDs, and a core that declares no runtime dependencies at all, with its rules validated against real training runs from three frameworks (HuggingFace, Coqui XTTS, PyTorch Lightning) rather than synthetic fixtures alone. It judges a run before it starts as well as after: an environment preflight imports the training stack in a subprocess and inspects checkpoints as ZIP archives without ever unpickling them, because torch.load executes arbitrary code by design. CONTRACTS.md pins exit codes, JSON schema policy, rule-ID stability and a verdict-stability guarantee, so downstream CI can depend on this tool without reading its source.
  • A PASS that states what it did not check (v0.12) — an audit found a run whose loss was exactly zero on every step passed silently, because every loss-shape check was guarded against dividing by zero and skipped, while the report went on to name those same three checks as having run. A PASS now lists each check that executed and each that did not, with its reason, exposed as structured data. A skipped check is not a passed check.
  • SARIF 2.1.0 output--sarif on every judging command turns findings into GitHub PR annotations, so a doomed fine-tune is flagged inline on the diff that caused it. Exit 2 now means “trainproof could not judge” and is never conflated with 1, a real FAIL verdict about your run — CI can tell a broken run from an unreadable log.
  • Live-abort proof — the guardian stopped a real diverging QLoRA fine-tune at step 20 of 300, so 93% of the scheduled GPU steps never ran
  • Overfitting detection (TP-OVERFIT) — flags a run whose eval-loss climbs past 1.2× its minimum while train-loss keeps falling, naming the run whose best checkpoint has already gone by
  • Regression-locked by construction — every gallery verdict and its complete rule-ID set is frozen in tests/golden/; a rule that stops firing and one that starts firing spuriously both fail the build. All 38 snapshots are byte-identical across v0.11 and v0.12, so “no existing verdict changed” is a checkable claim rather than a promise.
  • Fault-injection study shipped as evidence — six controlled QLoRA configurations (Qwen2.5-3B, RTX 5080) at three seeds each: all 18 real logs are committed, not just a representative one, and every verdict is locked in tests/golden/. A nineteenth log ships alongside them — a 9.8-hour Coqui XTTS fine-tune that diverged on its own, the first shipped failure nobody injected
  • Documented honest limitation — proved loss curves can't detect corrupted data (a shuffled-labels run cut its loss 62%); the compare command exists to close exactly that gap
What I built
Zero-config doctor (directory autopsy) Pre-flight dataset + tokenizer linter Live guardian (HF TrainerCallback, opt-in auto-abort, step-time telemetry) Log-based run verdicts (HF / Coqui / JSONL / CSV) Reference comparison engine Overfitting detection (TP-OVERFIT) Deterministic rules + 98 stable rule IDs SARIF 2.1.0 (GitHub PR annotations) Published contract (CONTRACTS.md) Golden-locked regression suite CI exit codes + HTML reports Multi-seed evidence matrix PyPI packaging + CLI
Stack
Python numpy transformers (optional) pytest setuptools / twine

notchecked — Coverage Accounting for Validators (GitHub, MIT)

A schema and accounting layer for tools that check things. It answers the question a validator usually cannot: how much of what you claim to have judged did you actually judge? Three states instead of two — what was checked, what could not be checked, and what was never in scope — and eight once you account for who owns each gap and whether it can ever change. Zero dependencies, standard library only. It grew out of the same defect found in four unrelated systems: an ML training linter, a directory walk, an infrastructure-compliance report and a retrieval evaluation harness, every one of them rendering absence of evidence as a positive result.

Key results
  • Eight terminal states, each with a distinct owner — a test enforces that no two states share both an owner and a remediation, because two gaps in one bucket hand the reader a to-do they cannot action. WAIVED is in scope and deliberately unevaluated by a named person; PREREQUISITE_FAILED points at the upstream target that blocked it; OUT_OF_SCOPE/DATA_PERMANENT is nobody's to fix, ever.
  • Coverage and verdict are orthogonal, and the constructor enforces itCHECKED is not a result; it says a determination was made, not what it was. A gap cannot carry a verdict and a checked record cannot carry a skip reason. Collapsing those two axes is how a training run that learned nothing once returned PASS alongside a list of the checks that had “cleared” it.
  • Two denominators, and only one is quotable — a percentage over everything that exists is a claim about the framework you named; a percentage over what was mechanically checkable is a claim about your own evidence. The library computes only the second, and returns no value rather than zero when nothing was evaluable, because an absence of coverage is not a coverage of zero.
  • Hardened by two attacks on its own design, before release — an API attack found five holes, two of which were the library committing its own thesis error one level up: declaring every target out of scope produced a clean exit, and a target that never became a record was invisible. A taxonomy attack then put 24 realistic cases from ML training, compliance, RAG evaluation, CI/CD and production monitoring against the model: twelve fit exactly one state, five fit none, four fit two. Two new states came out of it, plus three tie-break rules — including unreachable is not unhealthy, which was one commit from shipping as a feature.
  • Every finding is a regression test named for the attack that found it — 112 tests, no dependencies, MIT.
  • Worked out in public, against a second domain — the abstraction was settled in a technical exchange with an infrastructure engineer who had hit the identical shape in compliance reporting, and who is credited by name in the README. Two unrelated domains agreeing is what makes it a primitive rather than one person's preference.
What I built
Eight-state coverage taxonomy Owner + permanence per state Declared reason vocabulary (no free text) Counts derived from records, never stored Coverage / verdict axis separation CI exit codes (incomplete never exits 0) JSON schema for CI Adversarial test design
Stack
Python Standard library only pytest hatchling

External Review & Adoption — Who Else Has Used or Corrected This Work

The hardest thing for a solo engineer to evidence is not that the code works — it is that anyone outside his own projects has looked at it, argued with it, and changed his mind. This is that record. Every item happened in public and every name is used only where the person stated their own affiliation.

A schema review that changed the design three times

Boris Teplitsky — by his own public description an IBM Certified Expert IT Architect with 21 years at IBM, creator of Merlin Studio, and lead architect of a real-time public-transport ticketing system on GCP for the Israel Ministry of Transport — replied to my trainproof write-up saying he had hit the identical failure in infrastructure compliance, a domain I have never worked in. Two independent domains hitting one shape is what turned it from a preference into a design.

He then reviewed the schema twice before it was tagged and gave three corrections, all accepted, all schema-level rather than wording: permanence must name the target it is relative to, or two reports on the same framework disagree and both are correct; the excluded corpus must enter the denominator, so excluding 412 of 415 reports 99% instead of a flattering silence; and the taxonomy must state that it begins after the unit exists, because turning prose into checkable requirements is a judgment call it does not govern. He asked to be credited by his real name rather than his handle, and is credited individually per correction.

His review also found a defect in my code within a minute of landing: the renderer printed “413 out of scope (100% of all targets)” while one target had in fact been checked — 99.5% rounded to 100. A reader takes that as “nothing was measured”, which is exactly the failure the library exists to remove, committed by its own reporter one layer above the schema it protects.

An outside product maintainer requested an integration

After ttsproof was released, the maintainer of SpeechSDK (speechsdk.dev, by Jellypod) — a commercial unified API covering more than 17 TTS providers — asked whether ttsproof could benchmark engines through it. The integration shipped. It means ttsproof’s 817-case edge-case corpus can be run against closed-source commercial engines — OpenAI, ElevenLabs, Google and the rest of that roster — from one command, which was not previously possible. ttsproof has also taken a community fix for a real number-formatting bug.

Key results
  • Three schema corrections accepted from a 21-year IBM Certified Expert IT Architect — including two that broke my own tests on the first run
  • An integration requested by an external commercial maintainer, not pitched to them — shipped and documented
  • trainproof is the schema’s first production adopter, shipped additively so nothing reading the existing report changed behaviour
  • Adopting it caught a distinction I had collapsed in my own code: “no learning-rate column in the log” and “no finite gradient norms in the log” are one word apart and opposite instructions to a human
What this is not

An IBM architect reviewed this as an individual. IBM endorsed nothing, and no claim is made that it did. Jellypod is a company with a commercial product; its size was never verified and is not claimed. Nothing here is described as proven — it is a record of who engaged and what changed as a result.

Left: the hand-generated Delaunay mesh around a quarter of a circular hole, graded so elements grow with distance from the hole. Right: the computed axial stress field, brightest at the hole crown where the concentration peaks.

2D Finite Element Solver Written From Scratch — Validated Against a 128-Year-Old Exact Result

A complete 2D linear-elasticity FEM solver with no FEA library and no scipy: the Delaunay mesher (Bowyer–Watson), the mesh generator, the quadrature rules, the linear and quadratic isoparametric elements, the element-by-element assembly and the preconditioned conjugate-gradient solver are all hand-written. It is pointed at a problem with a known exact answer — a stretched plate with a circular hole, where the stress at the hole edge is exactly three times the far-field stress, a result published by Kirsch in 1898. The code is given the geometry, the material and the load, and never the number three. The validation criteria were written into a literature audit, with their thresholds, before any code existed.

Key results
  • Recovers the analytic answer it was never given — the stress concentration factor comes out at 3.00002 against an exact 3 (0.0005%) on a domain where the analytic field is the exact solution, and 2.99970 (0.0101%) by a second, independent route on a finite plate extrapolated to zero width. That the target is not an input is made checkable rather than asserted: grepping the source for the value returns only hits that never feed the computation.
  • Convergence orders match the theory, which is the real proof — linear elements converge at 2.026 and 1.019 against a predicted 2 and 1; quadratic isoparametric elements at 3.249 and 2.013 against a predicted 3 and 2. Landing on one number can be luck; reproducing the predicted rates across two element families cannot.
  • Ten gates pre-registered, nine pass, and the failure is published — a variational crime committed on purpose (approximating the curved hole by straight edges) was predicted to cost energy-norm order. It cost a full order in L2 and none in the energy norm, so that gate FAILED. Two candidate explanations were then tested and both refuted, and all of it is in the record rather than removed.
  • A passing gate criticised for passing — the quadratic L2 rate cleared its band by 0.001. The record states that a two-sided band was the wrong shape for a one-sided theorem and that the gate would have failed on a coarser mesh family: a flaw in the gate, not the code, written down with nobody forcing it.
  • Degeneracy handled where it actually bites — every node on the hole boundary lies exactly on one circle, so exact co-circularity is the normal case for the in-circle predicate, not a rare accident. The mesher keeps only the connected component of the bad-triangle set, and the empty-circumcircle property is then measured on every finished mesh after four rounds of smoothing: worst violation zero at all five refinement levels.
  • Reproducible in one commandpython fem_003.py, about fifty seconds, shipping its own evidence: the full gate log, convergence tables, hoop-stress data and figures. Largest system solved is 21,220 degrees of freedom.
What I built
Bowyer–Watson Delaunay triangulation Graded mesh generation + Lloyd smoothing Mesh topology & quality gates Duffy-mapped Gauss quadrature (constructed, not recalled) P1 + P2 isoparametric elements (curved edges) Element-by-element assembly Jacobi-preconditioned conjugate gradients Patch tests + scale-invariance checks Convergence-order measurement Pre-registered validation gates
Stack
Python numpy (arrays only) matplotlib no scipy no FEA or meshing library
n8n workflow: a Schedule Trigger feeding an HTTP Request node that drives the AI content pipeline

AI Content Automation Pipeline — n8n · LangChain · LangGraph

Built an end-to-end content-automation system on the same open-source AI stack used in production teams. A self-hosted n8n workflow fires on a daily schedule and calls a decoupled FastAPI service over HTTP (via host.docker.internal, bridging Docker to the host). That service runs a LangGraph state machine: pick an unused image → caption it with a multimodal LLM (Gemini, structured output enforced by a Pydantic schema) → a validate → retry loop that regenerates any caption failing length, keyword, or tag rules. Per-folder context files keep names and lore accurate (no hallucination), and per-folder link overrides deep-link each item to the right page. The output is platform-neutral, and a thin publisher adapter per target ships it live every day to Bluesky, Mastodon and Tumblr — each with its own character budget, media rules and link semantics, each keeping a separate ledger so one dead platform never blocks or double-posts another. A second set of connectors reads the engagement back and reports who replied.

The publishing stage is deliberately model-free. Generation uses an LLM; the daily write path does not. It draws from a pre-written ordered queue in which every entry carries the source citation for each fact it states, so on a channel nobody reviews there is no surface on which a model can invent anything. The safety property comes from the architecture rather than from a reviewer. The engagement side is tiered the same way: counts are verified, but “this person is waiting for you” never is — an incomplete view raises incomplete and suppresses the signal entirely, because a stale snapshot that asserts a negative sends a human to redo work that was already done.

Key results
  • Runs unattended on a daily schedule, fully decoupled over HTTP (Docker-to-host bridge)
  • Publishes live to three platforms from one package contract — adding a fourth touches nothing existing
  • Model-free write path: citation-backed queue, so the unreviewed channel cannot hallucinate
  • Evidence-tiered engagement reporting across four surfaces, with an explicit incomplete-view rule
  • Schema-enforced structured output (Pydantic) with a validate → retry loop
  • Public repository on GitHub
What I built
n8n scheduled orchestration Self-hosted in Docker Decoupled FastAPI service host.docker.internal bridge LangGraph state machine Validate → retry loop LangChain structured output Multimodal image captioning Pydantic schema enforcement Hallucination guardrails Per-folder deep-linking Rotating de-duplicated picker Pluggable publisher adapters Model-free write path Citation-backed content queue OAuth 1.0a & AT Protocol Richtext facet computation Engagement read-back Evidence tiering Per-platform idempotent ledgers
Stack
n8n LangChain LangGraph FastAPI Gemini (multimodal) Docker Python Pydantic REST / HTTP AT Protocol (Bluesky) Mastodon API Tumblr API OAuth 1.0a Pillow
Bed Vibe AI Companion running in the interactive room environment

Bed Vibe AI Companion — Hybrid Local AI Desktop Runtime

Built and shipped a hybrid AI-companion app in Unity HDRP: a downloadable, installable desktop product run locally on the user's machine behind paid subscription access. Unity launches and supervises a fleet of local subprocesses — a Rust voice-activity gateway, a Whisper ASR server, a Rust local server, and a llama.cpp wrapper serving a custom GGUF model — with hardware-aware LLaMA build selection, CPU/GPU Whisper selection, TCP port health checks, and clean shutdown handling. At startup, the runtime profiles CPU capability, GPU availability, RAM, VRAM, model size, quantization, and context-window fit so it can select a reliable local Whisper and LLaMA/GGUF configuration for the machine. Authentication, MAC-address device licensing, and Stripe-webhook subscriptions run on backend services I operate. Speech-provider credentials remain server-side: the Unity client sends only the bounded synthesis request, while the backend performs the authenticated provider call and returns generated audio plus lip-sync timing data. The real-time conversation loop is microphone → Rust VAD → Whisper → local LLaMA/GGUF → AWS Polly TTS → FFmpeg conversion → OVRLipSync lip-synced avatar playback, bridged over local HTTP/TCP IPC with mute/unmute coordination so the character does not transcribe its own generated voice. Each LLM request is assembled from the character backstory, personality rules, selected model settings, the user profile, relevant prior conversation history, and the newly transcribed utterance. The character is autonomous and conversation-aware through a hand-built locomotion/idle state machine, spoken-cue behavior, talking-gated eye contact, Animation-Rigging head look-at, and a cinematic physical-camera rig. Users configure the AI's name, backstory, personality, temperature, and token limit per profile, inside a production shell with SHA-256/HTTPS login, file-integrity anti-tamper checks, server-synced gold economy, in-app store, and chat-history export.

Behavior, World & Product Layer

Beyond the speech pipeline, I built the companion as a persistent interactive character inside a real-time environment. A behavior state machine controls first-contact greeting, engagement, idle activity, movement, sitting, sleeping, spoken-cue reactions, and talking-gated attention. On first greeting or when the user calls her, the character turns toward the user, establishes eye contact, performs a wave animation, and presents a positive engagement response before entering the normal conversation loop. When the user is inactive, the character can transition through autonomous behaviors rather than remaining frozen: after idle time she may move through the room, sit on the couch, and later sleep according to the environment state.

The environment includes time-based presentation such as changing daylight outside the room, while the avatar uses camera-aware eye contact, animation-rigging head tracking, locomotion, idle animations, and cinematic camera behavior to create a continuous social presence. The product layer also includes a server-synced in-app gold economy, shop and unlock flow, room and furniture customization, character/personality configuration, selectable voice and language options, and user-facing personalization such as placing a user photo inside the virtual space. The goal was not only to make an LLM answer, but to combine AI conversation, avatar behavior, environment state, customization, and product systems into one interactive desktop experience.

What I built
Hybrid local-AI desktop product Shipped subscription app Unity HDRP runtime Unity-orchestrated subprocesses Hardware-aware LLaMA build selection CPU/GPU Whisper server selection Rust voice-activity gateway Mic → Whisper → LLaMA → TTS loop HTTP/TCP IPC bridge Mute/unmute feedback control TCP port health checks Clean subprocess shutdown OVRLipSync lip sync FFmpeg audio conversion Autonomous behavior state machine Talking-gated eye contact Cinematic camera rig AI personality/backstory config Stripe-webhook billing MAC-ID device licensing File-integrity anti-tamper Gold economy + store Chat-history export
Stack
Unity HDRP C# Rust llama.cpp / GGUF Whisper ASR AWS Polly FFmpeg OVRLipSync Animation Rigging Stripe HTTP/TCP IPC SHA-256

Aether — Real-Time Aerospace Telemetry Engine (Rust)

Built a live aerospace surveillance and state-estimation engine in Rust: asynchronous ADS-B ingestion over Tokio, WGS84 → ENU coordinate transformation, a constant-velocity Kalman filter per axis, innovation gating, track lifecycle management, closed-form closest-approach screening against ICAO separation minima, and a C ABI so the estimator can be called from existing C/C++ systems. Running it against live traffic exposed two real defects. The engine was stamping every measurement with the time it arrived rather than the time it was observed, and a controlled A/B against the same feed measured the cost: 16.1% of good observations rejected before the fix, 1.9% after. The same work revealed that the earlier build had been silently failing two-thirds of its polls and never ageing a single track out. A second defect — the ENU “Up” axis is not altitude, and reads 19,233 ft below the tangent plane for an aircraft truly at 36,089 ft at 463 km — was corrected in the display without disturbing the validated screening geometry, because the curvature error is common-mode across a pair. Surveillance and estimation only: no targeting, engagement or weapon functionality.

What I built
Async ADS-B ingestion Per-axis Kalman tracking WGS84 → ENU transformation Innovation gating Closed-form conjunction screening Observation-time temporal model Bounded ingestion boundary Sensor-health observability C ABI for C/C++ interop
Stack
Rust Tokio reqwest / rustls serde Kalman filtering Geodesy (WGS84 / ECEF / ENU) cdylib / extern "C" 55 tests, clippy -D warnings

BedVibe Security Telemetry Lab

Built a defensive security telemetry lab around BedVibe’s production infrastructure: an isolated HTTP honeypot on a self-managed Linux VPS, event capture into JSONL and SQLite, hourly/daily human-readable reports, honeytoken tracking, scanner fingerprinting, and safe pull-only local report mirroring so the home machine remains unexposed.

What I built
Isolated HTTP honeypot JSONL event logging SQLite collector database Hourly security reports Honeytoken tracking Scanner fingerprinting Safe pull-only report mirroring Linux service monitoring
Stack
Linux VPS Rust Axum / Tokio SQLite systemd timers JSONL SSH read-only inspection Defensive security telemetry

BookProof — Production API & MCP Server

Shipped the deterministic LongBook retrieval-evaluation engine for real use, two ways. (1) A deployed HTTP API on a self-managed Linux server — Cloudflare → nginx → FastAPI/uvicorn, Let's Encrypt TLS, systemd service — exposing a token-gated developer API (/api/bookproof/v1/verify) and a free public demo (/api/bookproof/public/verify) with per-IP daily rate limiting keyed on the real Cloudflare visitor IP, strict text/claim caps, a one-run concurrency lock, and ephemeral processing (uploaded text and results deleted immediately after each run). Deterministic local retrieval only — no external model calls. (2) An MCP server (Model Context Protocol) over stdio that exposes the same engine as allowlisted tools with sandboxed read/write paths, including a diagnosed transport bug where the child process inherited the MCP stdio channel, fixed with stdin isolation.

What I built
Deployed FastAPI backend Cloudflare → nginx → uvicorn Let's Encrypt TLS systemd service Token-gated developer API Free public demo endpoint Per-IP daily rate limiting (real Cloudflare IP) Strict text / claim caps One-run concurrency lock Ephemeral processing (inputs deleted) Deterministic retrieval, no model calls MCP server (Model Context Protocol) MCP stdio transport Allowlisted MCP tools Sandboxed read/write paths Subprocess-under-stdio debugging fix
Stack
Python FastAPI uvicorn MCP (Model Context Protocol) nginx systemd Cloudflare Let's Encrypt Linux
Audio Drama Studio — browser multitrack timeline with voice channels, SFX and music lanes

Audio Drama Studio — Browser Multitrack Production Editor

Built a full multitrack production editor that runs entirely in the browser: a script parser auto-casts dialogue lines onto 8 voice channels plus SFX and music lanes; clips drag between channels to recast; per-clip volume, pan, and reverb run through a Web Audio API node graph with canvas-rendered waveforms; final mixdown renders offline via OfflineAudioContext through a hand-written PCM WAV encoder with peak normalization; projects save and reload as JSON with full audio rehydration. A working demo project loads automatically — no account needed.

What I built
Script parser auto-casting Drag-to-recast timeline Web Audio node graph Offline mixdown rendering Custom PCM WAV encoder Canvas waveforms
Stack
Web Audio API OfflineAudioContext Canvas 2D TTS REST API Token billing

All voices are contracted, rights-cleared voice performers, generated through BedVibe's proprietary TTS pipeline — no scraped or third-party audio.

BedVibe Story Parser interface converting tagged story text into structured narrator and speaker blocks

BedVibe Story Parser — Tagged Text → Audiobook & Audio Drama Projects

Built a deterministic browser-based text-to-project parser that converts pasted novels, scripts, dialogue, or TXT files into structured production JSON. It detects named speakers, narrator passages, optional emotion tags, and tag-only speaker declarations; untagged text is assigned to NARRATOR. The parser supports formats such as [Name] text, [Name|angry] text, (Name|happy) text, Name: text, Name|angry: text, S1: text, and Speaker 1: text.

The output is a structured JSON project containing a title, speaker list, and ordered blocks with block ID, speaker, emotion, and text. Users can download the project JSON or hand it directly to either Audiobook Studio Editor or Audio Drama Studio through an in-browser project-transfer flow. This creates one shared ingestion layer for converting raw writing into long-form narration or multitrack dramatized production.

What I built
Tagged-text parser Narrator fallback Speaker detection Emotion-tag preservation Tag-only speaker state Structured JSON project output JSON download Audiobook Studio handoff Audio Drama Studio handoff Long-form production ingestion
Stack
JavaScript Regex parsing JSON sessionStorage transfer Browser download API HTML / CSS
Audiobook Studio Editor screenshot

Audiobook Studio Editor

Built a multi-block editor for long-form speech generation with voice selection, emotion controls, project save and load, UI logic, cost estimation, and API-connected generation workflows for spoken content production.

Block-Level Generation & Project Control

The editor turns a parsed or manually written audiobook into an editable sequence of generation blocks. Each block can receive its own voice, language, emotion, text, and production settings. Users can include or exclude individual blocks before generation, generate only the selected material, regenerate a specific block after editing, and review estimated token cost before committing a larger project run.

The production workflow supports project save and load, imported Story Parser projects, speaker-to-voice mapping, inline expressive effects embedded in text, and post-generation controls for warmth and space. Global production settings and per-block choices allow the same project to move from clean narration to more stylized audiobook delivery without rebuilding the structure from scratch.

What I built
Frontend editor logic Project workflows Voice / emotion controls Token-aware generation API integration Story Parser project import Per-block voice / language / emotion Include / skip generation control Generate selected blocks Per-block regeneration Pre-generation cost estimation Speaker-to-voice mapping Inline expressive effects Global production controls Warmth / space post-effects Project save / load
Stack
JavaScript HTML / CSS Python APIs Supabase auth Browser data flow
The Mirelands battle screen The Mirelands character selection screen The Mirelands inventory and profile screen The Mirelands tower ballista upgrade screen

The Mirelands — Fantasy Card Game / Interactive Web Game

Built a custom browser game engine in JavaScript from scratch — no framework, no external game library — with deterministic turn logic, combat resolution, AI opponent behavior, persistent state, deck validation, tower/projectile systems, and backend sync. Includes row-level security on the PostgreSQL backend, cloud-save sync, and replay-safe Stripe purchase handling through an idempotent webhook flow.

What I built
Hero selection flow Profile / inventory UI Deckbuilding systems Battlefield combat logic Tower / projectile systems Persistent progression
Stack
Custom JavaScript game engine HTML / CSS AI opponents Supabase PostgreSQL (RLS) Stripe webhooks Offline-first sync
Fantasy talking avatars screenshot

Fantasy Talking Avatars

Built a fantasy multi-character avatar stage where transparent PNG characters can speak alone or together using BedVibe TTS with aligned lip sync. Supports one to four visible avatars on the same screen, dialogue sequencing, debate-style scenes, and creator-focused workflows for comics, animated storytelling, fantasy scenes, and game-style character presentation.

What I built
Multi-avatar scene system 1–4 character layouts TTS-aligned lip sync Fantasy character pipeline Dialogue / debate scenes Transparent PNG avatar workflow
Use cases
Talking comics Fantasy animations Story scenes Game dialogue mockups Character debates Creator content
Norwegian puzzle and exercise book factory

Book-Publishing Automation Factories

Built two content-generation factories that turn structured data into print-ready books: a published puzzle-book series (crosswords, word-hunts, matching) on Amazon KDP, and a Norwegian exam-prep exercise engine (trinn 1–3: verb conjugation, noun declension, adjective agreement, possessives and word order). Each generates exercises from CSV data pools, rendered through HTML/CSS into 6×9 KDP-print PDFs — with correctness machine-verified by construction (every crossword reconstructs, every hidden word is findable, every answer key is right) and automated KDP preflight for page count, bleed and margins.

What I built
Deterministic content generators Machine-verified correctness CSV data pools → generators HTML / CSS → 6×9 print PDF Automated answer keys KDP preflight (bleed / margins / page count) Norwegian grammar-rule engines Multi-agent workflow + QA gates
Stack
Python CSV data pipelines HTML / CSS Headless Chrome → PDF pypdf preflight Amazon KDP publishing
Datasets page screenshot

Speech Datasets & Curation

Designed and curated multilingual speech datasets with structured metadata, controlled emotion labels, speaker coverage, studio-grade recordings, and training-oriented organization for TTS and speech model workflows.

What I built
Dataset design Metadata schema Tokenization workflow Speaker embeddings Training preparation
Scope
100+ hours curated recordings Multilingual speech Emotion labels Trait recordings Studio-quality capture
Speech Dataset Toolkit and Recorder screenshot

Speech Dataset Recorder & Audio Toolkit

Built a Unity-based speech dataset recorder for profile-driven collection workflows, with language and emotion session setup, guided text recording, countdown flow, and organized output for large-scale dataset creation. In parallel, built an 11-tool batch-processing and QA toolkit to complement RX11 for mass curation workflows, including trimming, duration scanning, RMS and peak checks, onset / pre-post checks, clipping detection, noise screening, and dataset-wide analysis.

What I built
Unity dataset recorder 11-tool bundle Rust utilities Batch dataset QA Mass curation workflow
Toolkit coverage
Language / emotion sessions Recorder UI workflow Batch audio QA / cleanup tools Folder-level analysis Large dataset processing

The Mirelands Audiobooks & Lore Pipeline

Built a connected dark fantasy audiobook line that introduces the core world, characters, creatures, and lore behind The Mirelands. The audiobooks function as both standalone content and a worldbuilding pipeline tied directly to the game’s cards, factions, character identity, voice continuity, and long-form story systems.

What I built
Fantasy worldbuilding Character continuity Lore-to-game pipeline Long-form audio production Voice identity system Narrative content design
Connected game elements
Legendary characters Creature factions Card inspiration Story-driven game world The Mirelands setting

BedVibe Author Intake & Rights Authorization Workflow

Built a serverless production-intake workflow for managed audiobook and digital voice projects. The system collects author metadata, book details, narration requirements, voice preferences, manuscript/material links, rights declarations, content-policy attestations, privacy consent, and electronic signature before production begins. It validates required fields server-side, rejects incomplete submissions before email processing, stores the submission record, generates a signed authorization PDF, and routes one internal BedVibe notification with the PDF attached and the submitted author email set as Reply-To. The workflow also includes SEO metadata, sitemap integration, and a noindex confirmation page.

What I built
Public author intake page Rights/content attestation Content-policy gating Electronic signature capture Server-side validation Signed PDF generation Internal notification routing Reply-To author routing Netlify Forms storage Netlify Functions workflow Email deliverability hardening SEO metadata Sitemap integration Noindex confirmation page
Stack
HTML / CSS JavaScript Netlify Forms Netlify Functions Serverless workflow PDF generation Email routing Server-side validation JSON-LD Sitemap management

Real-Time Voice-Activity Detection — custom-built

Built a custom Rust voice-activity gateway for a real-time Unity avatar pipeline. The tool selects a named microphone device, captures live audio through cpal, detects speech using dB thresholding, finalizes an utterance after 1.2 seconds of silence, writes the captured segment as a temporary WAV, sends it as multipart audio to a local Whisper transcription server, forwards the transcription back to Unity over localhost TCP, and deletes the temporary file after processing. Unity can also control the gateway with local start/mute commands.

What I built
Named microphone selection Real-time audio capture dB-threshold speech detection Silence-based utterance finalization Temporary WAV segmentation Multipart Whisper upload Unity TCP transcription bridge Start / mute control port Temp-file cleanup
Stack
Rust cpal hound WAV writer reqwest multipart Whisper ASR TCP localhost bridge Unity integration
Reception AI screenshot

Reception AI / Lead Intake System

Built a hosted reception and lead-capture flow with live deployment, form handling, email routing, persistence, and business-facing web intake connected to backend services.

What I built
Hosted web flow Form / webhook handling Email routing Data persistence Server deployment
Stack
Python Linux server Nginx SMTP / email flow Static + API deployment
Business talking avatar screenshot

Business Talking Avatars

Built a business-facing talking avatar system for reception, presentations, ads, e-learning, and instruction flows. Users can generate speech with BedVibe TTS or upload their own audio, then drive branded character lip sync with natural breathing, blinking, positioning, mirroring, and on-character company logo placement.

What I built
Talking avatar workflow TTS-driven lip sync Own-audio lip sync Logo placement on avatar Screen positioning / mirroring Business presentation use cases
Use cases
Receptionists Secretaries E-learning Instruction videos Ad creatives Branded spokesperson avatars
Norwegian grammar book English edition cover Norwegian grammar book Polish edition cover Norwegian grammar book Ukrainian edition cover

Norwegian Grammar Books — Multi-language Editions

Developed Norwegian grammar material from 14 years of teaching experience and published full-length editions for English, Polish, and Ukrainian readers. The books span the complete CEFR range — A1 beginner through C2 proficiency — from basic structures all the way to advanced and specialised grammar, not just introductory material. They combine structured explanations, exercises, and practical grammar presentation for foreign learners of Norwegian.

What I built
Norwegian grammar Language teaching Multilingual editions Educational content Structured explanations Published books
Languages
English edition Polish edition Ukrainian edition Norwegian grammar CEFR A1–C2 (full proficiency)
Polly Rust Server repository screenshot

Polly Rust Server

Built a Rust-based HTTP server for Amazon Polly TTS generation with async request handling, AWS SDK integration, configurable voice selection, and deployable API structure for speech-serving workflows.

What I built
Rust backend service Amazon Polly integration Async request handling HTTP API endpoints Configurable voice flow
Stack
Rust Axum Tokio AWS SDK dotenv HTTP / JSON

Technical Scope

These are the technical areas demonstrated across the projects above. The portfolio is organized by shipped systems, not by isolated code snippets, because the strongest proof is working product ownership.

Backend & APIs

Built backend services with Python and FastAPI for TTS generation, authenticated account flows, token accounting, checkout and subscription handling, webhooks, admin logic, email routing, and production debugging. Worked across Supabase and PostgreSQL persistence, JWT/auth flows, payment-linked state changes, and API-connected product systems. Also built a Node.js serverless function (Netlify Functions, CommonJS) for the author-intake workflow — server-side validation, signed-PDF generation with pdf-lib, and transactional email routing via Brevo on form submission.

Speech & ML Systems

Worked across dataset curation, metadata schema design, tokenization, speaker conditioning, emotion labeling, evaluation splits, training workflows, vocoder paths, inference integration, and deployment of multilingual speech systems. Experience spans proprietary neural speech production systems, custom voice-conditioning research, multilingual neural-codec and audio-token pipelines, speaker-embedding conditioning, and custom encoder-decoder architectures.

Product & Infrastructure

Built interactive browser products with hand-written JavaScript and HTML/CSS — including a framework-free game engine, Web Audio production tooling, and canvas-rendered interfaces — with Supabase-backed persistence, PostgreSQL data models, Linux hosting, Nginx reverse proxy, and production deployment. Comfortable owning the full path from UI and game/application logic to backend services and public release.

AI Systems & LLM Orchestration

Built LLM-powered systems beyond single prompts: retrieval-augmented generation (RAG) with hierarchical retrieval and claim grounding, multimodal image-to-structured-output captioning, and schema-enforced structured output (Pydantic). Orchestrated multi-step AI pipelines with LangChain, LangGraph (stateful validate/retry graphs), and self-hosted n8n on Docker — wiring Gemini and local LLMs (llama.cpp / GGUF) behind decoupled HTTP services. Also build and run custom MCP (Model Context Protocol) servers that give agents typed, auditable access to real systems rather than free-form shell access: a mail bridge exposing search, threading and draft composition across providers; a narrative-state server answering structured queries over a long-form corpus (timeline, world state, entity knowledge, passage provenance); and a cross-model critic server that routes a plan or diff to a second model for independent review. Outbound actions are gated draft-then-approve by design, so an agent proposes and a human commits.

QA & Evaluation

Systematic evaluation of ML and retrieval systems — precision, recall, F1, and accuracy alongside task-specific metrics, evaluation splits, baselines, and ablations. Built an automated failure-mode QA framework for neural TTS (ASR-based, reproducible scoring across voices and sample sets) and a retrieval-evaluation and claim-grounding toolkit for long-document RAG (multiple retrieval methods, coverage and grounding scoring, deterministic and model-free). That work is packaged as four open-source Python libraries, all on PyPI: ttsproof, automated failure-mode qa for text-to-speech; trainproof, a deterministic linter for ml training runs; notchecked, coverage accounting for validators: what was checked, what could not be checked, and what was never in scope; and spkproof, deterministic checks for speaker-verification studies. Detectors are validated by controlled fault injection — known faults are deliberately introduced into otherwise healthy runs to prove the tool catches them, rather than trusting that a check works because it never fires. Findings published as technical reports with DOIs.

Inference & Latency Optimization

Optimized a deployed neural speech inference stack layer by layer, profiling before changing anything. Replaced repeated reference-audio encoding with a persistent VQ-token cache on disk, using modification-time invalidation so a replaced reference regenerates itself automatically. Applied PyTorch Inductor compilation with CUDA graphs to the autoregressive decoder for roughly 9× faster token generation — 144 tokens/sec against 16 — taking the system from slower than realtime to several times faster than realtime. Then profiled the remaining request path and lifted per-request GPU allocator and garbage-collection work off the hot path for a further ~1.3× on production-length lines. Every result validated with isolated Docker A/B benchmarks, cross-process reproducibility tests, a 40-request VRAM soak, and an ASR-based regression suite scoring every generated clip for reference-audio leakage, gibberish, truncation, repetition and silence. A known-good production image was frozen, verified and preserved as a rollback target before any change was deployed.

Technical Q&A / Evaluation Notes

This section is included for technical reviewers who want to see how I think about model evaluation, dataset curation, training quality, and production tradeoffs. It summarizes applied practices from the systems and datasets behind the projects above.

How do I evaluate an ML or speech model?

I evaluate models across data quality, held-out performance, robustness, and production behavior. For speech systems, I combine offline checks with perceptual evaluation because many real failures are audible before they are visible in a single metric.

  • Data quality: clean transcripts, correct speaker/emotion labels, trimmed audio, consistent metadata, and separation of train / validation / test sets.
  • Held-out evaluation: I keep dedicated validation and test data so the model is not judged on material it has already seen during training.
  • Task behavior: intelligibility, speaker consistency, emotional fidelity, pronunciation, multilingual robustness, and failure cases on long or difficult inputs.
  • Production behavior: latency, memory, stability, repetition, drift, clipping, and degradation under real user prompts.

Example: dataset splitting and validation discipline

In large speech datasets, I work with explicit held-out sets rather than trusting training loss alone. In one large multilingual workflow, the total corpus scale was on the order of one hundred thousand samples, with dedicated validation and test subsets kept separate from training material.

  • Typical split logic: the majority of samples remain in training, with smaller held-out subsets reserved for validation and final test.
  • No leakage: I avoid overlap of duplicated clips, near-duplicate text/audio pairs, or reused evaluation material across splits.
  • Why it matters: a model can look strong if evaluated on data that is too similar to training material, but then fail in real deployment.

How do I avoid overfitting?

I do not assume lower training loss means better real performance. I look for generalization.

  • Track validation behavior instead of training loss alone.
  • Stop or revise runs when validation stalls while training continues improving.
  • Use held-out speakers, languages, texts, or styles where appropriate to test real generalization.
  • Inspect qualitative failures: metallic artifacts, unstable prosody, repetition, emotional drift, or speaker collapse.

How do I think about data quality, noise, and distribution shift?

Data problems often matter more than architecture changes. I check whether the training data actually represents the way the system will be used.

  • Noise: clipping, bad denoising, wrong transcripts, inconsistent loudness, wrong language tags, poor trims, or mislabeled emotion/speaker metadata.
  • Balance: avoiding a dataset that is dominated by one language, one emotion, or one speaker profile while claiming broader performance.
  • Distribution shift: for example, training mostly on short clean studio phrases but deploying on long-form narration, dialogue, or multilingual edge cases.

Training and model scope I have worked across

My work is not limited to one model family. The portfolio reflects hands-on experimentation, training, integration, and evaluation across multiple speech and AI systems.

  • Proprietary neural speech production systems, neural-codec language-model and audio-token pipelines, and multilingual narration tooling.
  • Large neural vocoder paths, mel-spectrogram and audio-token workflows, speaker-embedding conditioning, and custom encoder-decoder architecture.
  • LLM-side work including local model orchestration and AI companion / product integration.

What I optimize for in real systems

I care about the full path from data to deployed behavior, not just isolated benchmark numbers.

  • Model quality relative to real user tasks
  • Inference cost, memory footprint, and GPU practicality
  • Reliability of the API or product flow around the model
  • Whether the improvement is meaningful enough to justify deployment complexity

Credentials / Education

Officially confirmed university coursework and laboratory training in the natural sciences.

Officially confirmed university-level coursework in Natural Sciences — Hellenic Open University

Current academic status. Completed 11 of the 12 required modules and all laboratory requirements for the Natural Sciences programme at the Hellenic Open University (222 ECTS completed). One module remains before the Bachelor's degree can be awarded. Plans to complete the remaining module beginning in September 2026.

222 ECTS of university coursework completed (registry-confirmed): Mathematics I–II, Physics I–II, General & Inorganic Chemistry, Physical Chemistry, Organic Chemistry, Cell Structure & Function, Genetics, Introduction to Natural Sciences, and Science Education — plus six completed laboratory courses.

Completed laboratory courses
  • Physics Laboratory Course I & II
  • Chemistry Laboratory Course I & II
  • Biology Laboratory Course I & II

Profiles & Public Links

Public profiles and repositories that support the live products and technical work shown above.