Medubaris Research

LISI Platform Technical Specification

Source: 9_LISI_Platform_Technical_Specification_v1_1.md Updated:

LISI Platform Technical Specification

Version 1.1 (Engineering Detail)

Status

Approved

Institution

Medubaris Research

Product

Luxury Lot Inventory Index (LISI)

1. Purpose

This document extends the Technical Platform Architecture into a concrete implementation specification for the LISI platform.

It defines exact BigQuery table names, Astro routing, Supabase configurations, Streamlit application structures, JSON schema versioning, and deployment pipelines to be executed during the MVP build phase.

2. System Architecture & Component Mapping

Network Flow

User ─────────────────► Cloudflare DNS
                             │
       ┌─────────────────────┴─────────────────────┐
       ▼                                           ▼
[Unauthenticated]                           [Authenticated]
 Cloudflare Pages                          Cloudflare Access (Zero Trust)
       │                                           │
  Astro Website                           Supabase Identity Verification
       │                                           │
       │                                           ▼
       │                                       Cloud Run
       │                                           │
       │                                  Streamlit Dashboard
       │                                           │
       └───────────────┐   ┌───────────────────────┘
                       ▼   ▼
               Google Cloud Platform
                     BigQuery
           (storied-precept-464615-g7)

3. BigQuery Data Architecture

Dataset Definition

All LISI analytical resources reside within the following GCP project and dataset: Project: storied-precept-464615-g7 Dataset: macro_indicators

Note: Project IDs and dataset names represent implementation details and may change over time. The institutional requirement is that a single authoritative analytical datastore exists. BigQuery is the current implementation of that requirement.

Dataset Versioning

Analytical datasets may include version identifiers when methodology changes require schema or logic revisions.

Examples:

  • lisi_statistical_layer_v3_15
  • lisi_dashboard_kpis_v3_15

Deprecated datasets should remain available during transition periods and must not be silently overwritten.

Data Preservation

BigQuery remains the institutional system of record. Backup and recovery procedures should support reconstruction of historical analytical outputs consistent with institutional record-preservation policies.

Layered Table Mapping

The theoretical data layers are mapped to existing concrete tables and views:

1. Raw / Standardized Layer

  • observed_inventory_counts (Daily volume observations)
  • sheet_inventory_weights (Config: Metro/Make/Model/Segment/Tier)
  • trim_buckets (Config: High/Mid/Low trim designations)
  • lisi_confounder_log_v3_15 (Macro event and strike overlays)

2. Research Layer

  • lisi_statistical_layer_v3_15
    • Action: Transforms raw counts into 8-week (momentum) and 52-week (anchor) Z-scores, handling Track A (Model) and Track B (Trim) aggregations.

3. Product Layer (“Gold Layer”)

  • lisi_dashboard_kpis_v3_15
    • Action: Pre-computes market regimes, behavioral trajectories, UI flags, extreme values, percentiles, and the lisi_headline_index.

4. Presentation Layer

  • LISI_Layer_3p7_Signal_Stack_V2 (Looker/Streamlit normalized time-series)
  • lisi_commentary_assembly (AI-generated Looker text and 5-Tier master scenario output)
  • lisi_distribution_metrics (Rolling historical percentiles)

4. Product Versioning Convention

Triggers for Increment

To provide readers and API consumers with clear traceability, institutional products (like LISI) follow semantic versioning based on analytical impact:

  • Major Version (e.g., v3.0 → v4.0): Methodology change. Modifies the core mathematical logic, structural thresholds, or introduces new data sources that break backward compatibility.
  • Minor Version (e.g., v3.15 → v3.16): Signal logic adjustment. Tuning parameters, adding new tracked segments, or adjusting config weights without changing the foundational math.
  • Patch Version (e.g., v3.15.0 → v3.15.1): Bug fix or correction. Pipeline repairs, data backfills, or fixing calculation errors that do not reflect a change in institutional intent.

Version increments are permanently recorded in the Master Patch Log (/products/lisi/patch-log).

5. Public Signal Interface (Heartbeat)

Purpose

Decouple the static Astro website from BigQuery to eliminate live querying costs for unauthenticated visitors. Separate the machine-readable data layer from the editorial commentary layer to allow future flexibility (e.g., swapping AI commentary for human commentary without touching the signal infrastructure).

Mechanic

A weekly GitHub Action runs post-pipeline, queries the Product and Presentation layers, and generates two static JSON files. Signal artifacts MUST be committed to /apps/website/public/ via GitHub Actions. Cloudflare Pages handles deployment. R2 is explicitly excluded in v1.0.

Crucially: signal.json is derived exclusively from the Product Layer (e.g., lisi_dashboard_kpis_v3_15) and not from dashboard presentation code. The dashboard consumes the signal; it never produces the signal. The Product Layer remains the authoritative source.

signal.json (Pure Data)

{
  "schema_version": "1.0",
  "generated_at": "2026-06-11T12:00:00Z",
  "product": "LISI",
  "geography": "National",
  "headline_index": 63.4,
  "regime_label": "BROAD HESITATION",
  "accumulation_z_score": 0.85,
  "momentum_diffusion": 72.0,
  "trend_direction": "INTENSIFYING",
  "data_quality_score": 97,
  "extreme_metros": {
    "scarcity": "Dallas -0.45σ",
    "glut": "SF_Bay +0.85σ"
  }
}

commentary.json (Optional Editorial Layer)

{
  "schema_version": "1.0",
  "generated_at": "2026-06-11T12:00:00Z",
  "product": "LISI",
  "geography": "National",
  "narrative_assessment": "Inventory accumulation remains elevated (+0.85σ) and broadly distributed across 72% of tracked segments..."
}

6. Website Architecture (Astro)

Directory & Route Structure

medubaris/website/
├── src/
│   ├── components/
│   │   ├── SignalHeartbeat.astro    # Consumes signal.json
│   │   └── StaticChart.astro        # Renders pre-generated pipeline PNGs
│   ├── layouts/
│   │   └── InstitutionalLayout.astro
│   ├── pages/
│   │   ├── index.astro              # / (Homepage)
│   │   ├── research/
│   │   │   ├── index.astro
│   │   │   ├── white-papers.astro   # /research/white-papers
│   │   │   └── weekly/              # /research/weekly/[slug] (Substack)
│   │   ├── products/
│   │   │   ├── index.astro
│   │   │   └── lisi/                # /products/lisi
│   │   │       ├── index.astro
│   │   │       └── patch-log.astro  # /products/lisi/patch-log
│   │   ├── methodology/
│   │   │   └── index.astro          # /methodology
│   │   ├── archive/
│   │   │   └── index.astro          # /archive
│   │   └── institution/
│   │       ├── index.astro          # /institution
│   │       └── contact.astro        # /institution/contact
│   └── content/                     # Astro Content Collections
│       ├── research/                # White papers, publications
│       ├── products/                # LISI methodology, validation, patch logs
│       └── organizational/          # Governance documents
├── public/
│   ├── signal.json                  # Fallback or pipeline-injected
│   └── commentary.json              
└── astro.config.mjs                 # Cloudflare Pages integration

Spam Protection

The /institution/contact form must implement Cloudflare Turnstile to prevent bot submissions. Turnstile provides invisible spam protection without requiring user-facing CAPTCHAs and integrates natively within the Cloudflare ecosystem.

7. Content Distribution & Integration

Canonical Content Ownership

To maintain a single source of truth and avoid duplication overhead, content ownership is strictly delineated:

  • Website Canonical (Git Repository): White Papers, Methodologies, Patch Logs, Product Documentation, and Organizational Governance.
  • Substack Canonical: Weekly Observations and narrative commentary. (The website mirrors this content via RSS).

Substack Ingestion Mechanic (Write Once)

  1. The Astro project utilizes a build-time script (src/scripts/fetch-substack.js).
  2. During the GitHub Actions deployment workflow, the script requests the Substack RSS feed.
  3. The XML feed is parsed into Markdown and populated into the src/content/weekly/ directory with appropriate frontmatter (date, title).
  4. Astro statically generates the /research/weekly/[slug] routes.

Fallback Provision: If Substack RSS ingestion becomes unreliable (due to feed format changes, truncation, or payload stripping), the institution may instead archive Weekly Observations directly within the content repository while continuing to distribute through Substack.

8. Authentication Architecture (Supabase)

Environment & Schema

  • Platform: Supabase (Free Tier / Pro Tier)
  • Auth Providers Enabled:
    • Email (Magic Link)
    • Google OAuth
    • Microsoft OAuth (Azure AD)
  • Roles: No custom roles table required for MVP. All authenticated users are granted Researcher-level access and permitted dashboard exploration. Unauthenticated users are Visitors.

9. Dashboard Architecture (Streamlit)

MVP Delivery Platform

Streamlit is designated as the MVP dashboard platform. Future versions of the platform may migrate portions of the dashboard experience into custom web applications without changing the underlying BigQuery architecture.

Dashboard Application Structure

medubaris/dashboard/
├── app.py                      # Streamlit entry point (Auth validation)
├── pages/
│   ├── 1_Macro_Regime.py       # National KPIs & State Space
│   ├── 2_Regional_Drilldown.py # Metro-level cross-sections
│   ├── 3_Prestige_Dynamics.py  # Tier 1 vs Tier 2 / Trim Divergence
│   └── 4_System_Health.py      # Confounders & Data Quality Scores
├── queries/
│   └── lisi_kpi_queries.py     # Encapsulated BigQuery SQL strings
├── .streamlit/
│   └── config.toml             # Theming and telemetry suppression
├── requirements.txt
└── Dockerfile

Security & Access Control

  • Perimeter: Cloudflare Access acts as a Zero Trust reverse proxy in front of Cloud Run.
  • Validation: Cloudflare verifies the Supabase identity/session token before routing the request to the Streamlit container.
  • Public Access: Explicitly denied. Streamlit is 100% authenticated.

BigQuery Cost Control (TTL Caching)

To prevent interaction-driven billing spikes, all queries/lisi_kpi_queries.py functions must be wrapped with @st.cache_data.

@st.cache_data(ttl=43200) # 12 Hours
def fetch_macro_regime_data():
    # BigQuery fetch against lisi_dashboard_kpis_v3_15
    pass

10. Deployment Architecture (CI/CD)

The monorepo uses GitHub Actions to enforce separation of concerns during deployment.

Workflow 1: deploy-website.yml

  • Trigger: Push to main affecting website/ or content/.
  • Action: Runs Astro build (triggering Substack RSS ingestion).
  • Destination: Deploys to Cloudflare Pages.

Workflow 2: deploy-dashboard.yml

  • Trigger: Push to main affecting dashboard/.
  • Action: Builds Docker container. Pushes to Google Artifact Registry.
  • Destination: Deploys to Google Cloud Run.

Workflow 3: update-signal.yml (Cron / Pipeline Trigger)

  • Primary Trigger: Post-ETL success webhook.
  • Fallback Trigger: Weekly Cron.
  • Action: Runs Python script to query BigQuery, serialize signal.json, and commit directly to the Website repo (/apps/website/public/).

11. Next Steps

Execute npm create astro@latest utilizing the finalized “Medubaris Research” institutional name to ensure correct hardcoded paths, titles, and branding throughout the source code initialization.