Mapping Clarity

The problem

What Mapping Clarity does

End-to-end pipeline:

1. Step 1 — Pipeline setup (one time)A Tenant Admin creates a pipeline: which CSV columns to classify, which targets (account codes, cost centers, custom categories), and hierarchy. Reused on every future upload.
2. Step 2 — File uploadCSV via web UI or REST API. File stored in Cloud Storage; validation and atomic credit reservation before processing users are not charged for work that did not run.
3. Step 3 — Pattern matching (NGram engine)High-speed pattern index from prior confirmed rows. Strong matches (~80%+) classify instantly without AI fast, no AI cost, improves as the library grows.
4. Step 4 — Vector similarity & RAG contextEmbeddings via Vertex AI; BigQuery search for similar historical rows. A RAG context package (hierarchy definitions, examples, patterns) is sent with the row to the model.
5. Step 5 — AI classification (Gemini)Batches go to Gemini 2.5 Flash Lite on Vertex AI. Structured output: category, confidence (low/medium/high), plain-language reasoning. Firestore updates in real time; the UI refreshes row by row without reload.
6. Step 6 — Human reviewReview UI: color-coded confidence, inline edits, hierarchical dropdowns, filters, search, bulk edit. Every change logged with user and timestamp for audit.
7. Step 7 — Self-improvement loopConfirmations feed the NGram index and BigQuery training examples with embeddings. Similar future rows resolve faster and may skip AI entirely.
8. Step 8 — Results & downloadEmail on completion; downloadable CSV with classification columns and time-limited secure links.

What makes it different

The classification engine in more detail

• Pattern targets — regex rules for predictable formats (e.g. references starting with "INV-").
• List targets — hierarchical taxonomies from CSV (e.g. Account Type → Sub-Account → Code) that the model navigates.
• List definition targets — same as list targets plus text definitions per level, passed to the AI for ambiguous categories.

Multiple targets can run in one pipeline one upload can classify against account hierarchy, cost centers, and custom patterns in a single pass.

• Financial Services and Accounting
• Accounts Payable and Accounts Receivable Processing
• Enterprise Expense Management
• Bookkeeping and Tax Preparation
• Operational Data Management
• Business Intelligence and Financial Reporting

Multi-tenant SaaS with full isolation per organization; intuitive for business users and integrators.

High-level processing flow

User Uploads CSV ──→ Google Cloud Storage
                              │
                        Eventarc Trigger
                              │
                     Cloud Workflow (Orchestrator)
                              │
            ┌─────────────────┼────────────────────┐
            ▼                 ▼                     ▼
     NGram Engine      Vector Search         AI Classification
     (Pattern Match)   (BigQuery RAG)        (Gemini via Vertex AI)
            │                 │                     │
            └─────────────────┴────────────────────┘
                              │
                    Firestore (Real-Time Write)
                              │
                    React Frontend (Live Update)
                              │
                    Human Review & Correction
                              │
                    Learning Loop (NGram + BigQuery)

• Nine Cloud Run services for distinct pipeline stages; Cloud Tasks with OIDC between services
• Streaming CSV from Storage rolling batches, not full file in memory
• Firestore listeners push each row to the UI as it completes
• Large jobs (5,000+ rows): server-side pagination to avoid browser memory issues
• Atomic credit reserve, accrue, and reconcile across stages
• Multi-tenant isolation via Firestore rules and custom claims cross tenant access structurally blocked

• End-to-end HTTPS for data in transit
• Firebase Authentication with custom role and tenant claims
• Firestore Security Rules for database-level multi-tenant isolation
• Workload Identity Federation no service account key files in code or storage
• OIDC verification for service-to-service calls via Cloud Tasks
• Two-factor authentication (TOTP) for user accounts
• Secret Manager for all sensitive credentials — no hardcoded secrets
• Signed, time-limited URLs for downloads
• API key scoping, restrictions, and rotation support
• Input validation and path traversal prevention on file operations
• Rate limiting: 100 requests / 15 minutes on public API endpoints
• Break-glass super admin access with full session audit logging
• Complete audit trail for data modifications and access events

• Hosting: Firebase Hosting, global CDN main app and super admin portal
• CI/CD: GitHub Actions for build, test, and deploy
• Infrastructure as Code: Terraform for all GCP resources
• Container registry: Google Artifact Registry for Cloud Run images
• Testing: Playwright E2E suite plus 176+ unit tests (auth, processing, billing, RBAC)

• 22 frontend pages across the full user journey
• 9 Cloud Run microservices for independent processing stages
• 30+ Cloud Functions for distinct API operations
• 4 user roles with fine-grained permissions
• 3 custom target types for any taxonomy
• 4-stage AI pipeline with progressive fallback from patterns to AI
• Full REST API with OpenAPI and API key auth
• Multi-language SendGrid emails with templated layouts

Live: app.mappingclarity.com · Marketing: mappingclarity.com