01 / Case Study
DataYetu
Building infrastructure for human data used by AI.
Problem
Enterprise and local-language AI work often stalls before modelling starts. Records are messy, context is stripped away, and teams rebuild the same cleanup pipeline for every product. Generic benchmarks also miss meaning failures in languages and registers that models were not trained to handle well.
Context
DataYetu is a data infrastructure platform for collecting, annotating, validating and versioning human-generated datasets for AI systems. The current proof point is rights-cleared Kenyan language data — including Swahili, Sheng and code-switching — with the broader product aimed at structured, context-preserving datasets that AI teams can actually use.
I founded the company and own the product and engineering direction. The work is early. I do not claim production scale or customer numbers here.
What I Owned
I designed and built the product and technical foundations:
- annotation workflows for meaning, intent, tone and translation quality
- data processing from collection through validation and delivery
- AI interfaces for evaluation and dataset use
- versioned datasets with provenance attached to each record
- the public product surface at datayetu.com
Architecture
The system is organised as a repeatable pipeline rather than a one-off cleanup job: collect, record provenance, de-identify, validate, test models, deliver and refresh. Provenance — source, consent, context and permitted uses — travels with the record so a dataset stays auditable after it leaves the pipeline.
Technical Decisions
I treated local-language data as the origin and proof point, not a generic “African data” abstraction. Independent native review sits in the path before delivery. Evaluation is defined in countable outputs — reviewed failures, reproducible errors, reviewer agreement — instead of vague quality claims.
Hard Problems
The hard part is not storing text. It is preserving meaning across code-switching, slang, tone and intent, then making that meaning inspectable later. Schema design, reviewer workflow and provenance all have to survive a later licensing or R&D handoff.
Trade-Offs
Human review is slower than automated labelling. I accepted that cost because the product is specifically about failures that generic translation benchmarks miss. I also kept the first pilot narrow — language and linguistics — rather than promising healthcare, finance and agriculture as current proof points.
Result / Current Status
The product foundations, pipeline design and public site are in place. Related open work under the DataYetu organisation includes Postgres-oriented tools such as pgContext and pgGraph. The language pilot is the active validation surface.
Stack
TypeScript · React · Node.js · AWS · AI / LLMs · Postgres