OMOP on the cloud
Last reviewed: 2026-06-28 — see the freshness policy.
Learning objectives
After this chapter you will be able to:
- Implement the OMOP CDM as a gold layer on a cloud lakehouse or warehouse.
- Sequence the ETL and data-quality steps that produce a trustworthy OMOP instance.
- Choose between Databricks, Snowflake, and BigQuery for an OMOP build.
From the CDM spec to a running platform
OMOP CDM defines what the model is. This chapter is about building and operating it on the cloud as the gold layer of a lakehouse. The work is mostly ETL + vocabulary mapping + data quality, not schema design — the schema is given.
flowchart LR
Bronze["Bronze<br/>FHIR / claims / labs"] --> Stage["Silver<br/>conformed"]
Vocab["OMOP vocabularies<br/>(Athena download)"] --> Map
Stage --> Map["Source→standard concept mapping"]
Map --> CDM["Gold: OMOP CDM tables"]
CDM --> DQ["Data Quality Dashboard + Achilles"]
DQ -->|pass| Analytics["ATLAS / HADES / SQL"]
DQ -->|fail| Stage
The build, step by step
- Load vocabularies. Download the OMOP vocabulary tables from Athena and load
concept,concept_relationship,concept_ancestor, etc. These are the backbone of every mapping. - Stage and conform (silver). Parse sources, dedupe patients into a single
person, and deriveobservation_periodwindows. - Map source codes to standard concepts.ICD-10-CM→ SNOMED, local labs → LOINC, drugs → RxNorm. Track the unmapped rate — unmapped clinical codes are lost evidence.
- Populate the CDM tables (
condition_occurrence,drug_exposure,measurement,visit_occurrence, …) referencing standard concepts. - Run data quality. Achilles (characterization) and the Data Quality Dashboard catch ETL errors systematically. Treat DQ failures as blocking.
- Validate with the tools. If ATLAS can build a cohort on your CDM, the structure is sound.
Platform options
| Platform | OMOP fit | Notes |
|---|---|---|
| Databricks | Delta tables as CDM; Spark ETL; Glow to join genomics | Strong for large-scale ETL + ML; Unity Catalog governs PHI. See Databricks. |
| Snowflake | Native tables as CDM; SQL/Snowpark ETL | Strong for SQL-centric teams; dynamic masking + clean rooms for sharing. See Snowflake. |
| BigQuery | CDM as columnar tables | Google publishes OMOP-on-BigQuery patterns; serverless scale. |
All three run the same OMOP schema; pick based on your team and the rest of the estate (see the capability map).
Operational considerations
- Incremental ETL. New data arrives continuously; design silver→gold to upsert, not full-reload, at RWD scale.
- Vocabulary versioning. OMOP vocabularies update; pin the version used for any reproducible study.
- PHI governance. The
persontable and dates are PHI — apply column masking / row filters (Unity Catalog, Snowflake masking) and audit access (see Governance & data contracts). - Provenance. Keep bronze immutable and record ETL/version lineage so any analysis is reproducible.
Lab
hls-lakehouse-rwd — OMOP CDM gold layer on Databricks and Snowflake with vocabulary mapping and DQ checks.
Check yourself
- Why are the OMOP vocabulary tables loaded first, and what breaks without them?
- What should an ETL do with source diagnosis codes that fail to map to a standard concept?
- Why pin the vocabulary version for a published RWE study?