Input Pipeline

Challenge

Voice memos, meeting recordings, and dictated notes pile up fast. Manual transcription is too slow to be useful at scale. The structured data embedded in those recordings — todos, CRM facts, project notes — is valuable but inaccessible without an automated extraction layer.

What we built

We built an end-to-end daily pipeline: voice memos land in a watched bucket, a 60-second sample drives automatic language detection, full Whisper transcription runs in the detected language, an LLM extractor emits typed items per topic (fleeting note, todo, CRM fact) using a shared prompt, a PII-redaction pass cleans each item, and everything routes to the relevant knowledge bases by topic. Every raw artefact is archived to GCS so re-extraction is always possible.

Result

The pipeline has run daily since deployment. Every voice memo becomes structured, searchable data within minutes. The extraction layer serves as the data backbone for the knowledge base and everything downstream — the pipeline's output is what feeds the site's own use-case showcase at build time.

Tech used

• Whisper speech-to-text with language detection
• LLM structured extraction (multi-topic)
• PII redaction layer
• GCS archiving
• Postgres storage
• Daily automated run