Taming the "Verbatim" Chaos

Session Overview & Thesis

Standard sentiment APIs fail on real text — collapsing a review like “great pizza, but it arrived cold and late” into a useless “mixed” score. Instead, this production pipeline decomposes each verbatim into aspect-level sentiment tuples, maps them to owned business objectives, and enriches them with real-world context like weather and major events.

It walks through an end-to-end architecture on Azure (Microsoft Fabric for ingestion and cleaning, PostgreSQL with pgvector for storage, Azure AI Language for aspect-based sentiment, and Microsoft Foundry for the coordinated agent layer), then covers production concerns like evaluation, governance, cost, and disaster recovery. The through-line is a reframe: the goal isn’t “how positive is this text?” but “which objective is affected, in which direction, and what should happen next?”

Pipeline Architecture (Azure Stack)

1. 01 Ingest & Clean (Microsoft Fabric): Medallion architecture (Bronze → Silver → Gold), OneLake ingestion, PII redaction, and deduplication.
2. 02 Relational & Vector Store (PostgreSQL + pgvector): Structured aspect tuples, embeddings, master objective taxonomy, and immutable audit logs.
3. 03 Semantic Intelligence (Azure AI Language): Aspect-based sentiment analysis (ABSA), target/opinion extraction, and few-shot objective mapping.
4. 04 Agent Decision Swarm (Microsoft Foundry): Hierarchical orchestrator routing to specialized Extraction, Mapping, Enrichment, and Action agents.