Most healthcare organizations have already decided generative AI can reduce administrative burden. The review meeting is the hard part. Security asks where protected health information goes, privacy asks who can see it and how you would know, and the clinical lead asks what happens when the output is wrong. A diagram does not answer those questions. An evidence chain does.
This post describes a defense-in-depth pattern for retrieval-augmented generation on Oracle Cloud Infrastructure (OCI), built as six layers designed to cover one another’s failures. It will not make a workload HIPAA compliant because compliance depends on the covered entity’s or business associate’s risk analysis, policies, workforce practices, contracts, and operations. What it produces is the material your reviewers need to decide.
Prerequisites
- A Business Associate Agreement (BAA) executed with Oracle covering the services in use. Under 45 CFR §164.502(e) and §164.308(b)(1), a covered entity must obtain written satisfactory assurances before a vendor creates, receives, maintains, or transmits PHI on its behalf. A BAA is not implied by a subscription.
- Confirmation that every service in your design appears on Oracle’s HIPAA-assessed services and regions list.
- A completed risk analysis under §164.308(a)(1)(ii)(A), which several decisions below defer to.
Where this applies
The worked example is a clinical summary: an application retrieves a patient’s recent history for an authorized care manager, sends limited context to a model, and returns a draft with source identifiers. The pattern also supports prior authorization, utilization management, documentation review, member service, and population health reporting.
Two constraints govern everything downstream. The model is not a new source of clinical truth — a qualified user reviews output against the record before it informs care or coverage, enforced by preserved source identifiers (Layer 4) and correlated telemetry (Layer 6), not by policy alone. And access is authorized before retrieval, not after. Scope should reflect the minimum necessary standard at §164.502(b), with specifications at §164.514(d). Minimum necessary does not reach uses or disclosures for treatment (§164.502(b)(2)(i)) — confirm with counsel which of your use cases it covers.
Figure 1: ePHI data flow across the six layers
Layer 1: A governed tenancy boundary
Put ePHI-processing resources in a dedicated compartment hierarchy, separate from development and non-ePHI workloads. Compartments are logical boundaries for policy and resource management, not network boundaries.
OCI Security Zones associate a compartment with a recipe and deny operations that violate it. Review the recipe against your bill of materials first, because several catalog policies collide with Layer 2: deny SGW_gateway blocks the service gateway and deny DRG_gateway blocks the dynamic routing gateway that FastConnect and Site-to-Site VPN terminate on. Others make optional controls mandatory, including deny public_load_balancer and deny buckets_without_vault_key. Cloud Guard must be enabled before you can create a zone.
Use the OCI Core Landing Zone as an infrastructure-as-code starting point for the compartment hierarchy, IAM, networking, logging, and security services, then tailor it to this design. Its CIS-aligned defaults provide a versioned, redeployable baseline for auditors to compare with the running configuration.
Layer 2: Private network paths
Target state: no public ingress to ePHI-processing components, no unrestricted internet egress. Place the application behind a private OCI Load Balancer, connect the clinical network through FastConnect or a Site-to-Site VPN, and allow only documented flows through network security groups and security lists.
Use an OCI Generative AI private endpoint for model access, which are supported for on-demand models and models on dedicated AI clusters, in regions where those models are available. Two details are easy to miss: on-demand access requires explicitly enabling Allow Usage In On-Demand Mode, and a tenancy is limited to five private endpoints by default.
Use a private endpoint for Oracle Autonomous AI Database, and a regional service gateway for Object Storage, so neither connection needs an internet gateway or a NAT gateway. And note that a compartment does not create default-deny egress, it should be enforced through route tables, security rules, OCI Network Firewall, or an approved proxy.
Layer 3: Protecting data at rest, in transit, and in use
HIPAA’s encryption specifications are addressable, which does not mean optional. Under Although HIPAA classifies encryption as addressable, treat it as a baseline requirement for ePHI in this architecture. Confirm encryption across every storage layer, including Object Storage, Block Volume, File Storage, and Autonomous AI Database. Use OCI Vault where customer-managed keys, controlled rotation, or separation of duties are required, and test key recovery.
For data in transit, require TLS and certificate validation for application, database, and service connections. Use OCI Certificates to issue or import, renew, and revoke certificates for supported resources such as the private OCI Load Balancer.
For data in use, OCI Confidential Computing provides hardware-backed isolation on supported Compute shapes. Enable it at instance launch; it cannot be enabled or disabled later. Scope it carefully: protection applies to the workload on that instance, not to managed OCI Generative AI or Autonomous AI Database processing. Remote attestation uses a customer-operated Bring Your Own Attestation Service (BYAS) and is supported only on E5 and E6 SEV-SNP shapes.
Layer 4: Governing models, retrieval, and output
Where the model actually runs
Oracle states that OCI Generative AI does not store inference inputs or outputs inside the service or share prompts and responses with third-party model providers. Data handling and execution location are nevertheless separate questions.
- Gemini: Models execute at Google locations identified in each model and region’s External Calls notice.
- Grok: Models execute in an OCI data center, but in a tenancy provisioned for and managed by xAI.
- Private endpoint: A private endpoint protects the path from your VCN; it does not change where the model executes.
- Serving mode: At publication, the listed Gemini and Grok models are on-demand only and cannot run on a dedicated AI cluster.
Before sending ePHI, check the selected model and region’s External Calls notice, serving mode, and BAA coverage. Treat retention separately: conversations, responses, files, memory, fine-tuning, and application logs each have their own storage behavior. OCI Generative AI projects can persist long-term memory across conversations.
An agent is not HIPAA compliant by itself. Oracle’s HIPAA-assessed services list names Generative AI but does not separately enumerate individual agent features. Before processing ePHI, confirm with Oracle that each model, external call, tool, and stateful feature—including projects, conversations, files, vector stores, and memory—is within the BAA and assessed scope.
Patient-level authorization
This is the hardest problem in the design, and the one most often left unsolved.
Oracle AI Database 26ai includes Oracle AI Vector Search, so embeddings and associated data sit under the database’s access controls, encryption, and audit policies. Be precise about which system authorizes what: OCI Identity and Access Management controls who can manage the database resource, while database users, roles, privileges, and audit policies control access to the data inside it. Compartment permissions provide neither patient-level nor row-level authorization.
Make the authorization decision before retrieval, against your source-of-truth care-relationship model. Enforce the resulting patient set in the database, not after retrieval or only in the orchestration layer. In Oracle AI Database 26ai, Deep Data Security can use propagated user and agent identity plus runtime context to apply declarative policies to rows, columns, cells, and vector embeddings during SQL execution. Log the decision with the retrieval’s correlation identifier.
If your vector store cannot enforce a per-request patient predicate, you do not have minimum necessary, and no other layer compensates.
Tools, injection, and wrong output
Tools are privileges. Every tool or function expands the system’s authority. Allowlist tools, authorize every action, validate inputs, enforce strict parameter schemas, and block arbitrary outbound access.
Network controls contain; they do not prevent injection. A malicious instruction in retrieved content can still persuade a model to misuse a permitted tool. Layer 2 limits reachable destinations, helping turn exfiltration attempts to unapproved destinations into logged, blocked connections.
Govern changes. Inventory and version models, prompts, and tools, and require evaluation before promotion. Test for hallucination, prompt injection, unsafe tool use, and data leakage against thresholds set by clinical informatics, security, and compliance teams.
Escalate wrong-patient output. Treat cross-patient leakage or unauthorized ePHI disclosure as a potential breach under §164.402 requiring immediate incident response and breach assessment—not only placement in the model-quality backlog.
Layer 5: Workforce identity and workload identity
Federate workforce access through OCI IAM identity domains or your approved provider, and require multifactor authentication, unique identities, role-based access, session timeouts, and periodic review. Care managers should use the application instead of the database, buckets, keys, or model endpoints.
For service-to-service calls, use dynamic groups with instance or resource principals. These remove the need for long-lived API keys but shift the trust boundary to the host: anyone who can open a shell on the instance inherits its privileges, so control interactive access as tightly as the model endpoint. Scope each policy to what the component needs. For example, the orchestrator invokes the approved model, retrieval queries an approved schema, neither does both.
Healthcare availability demands a tested emergency-access, or “break-glass,” path, distinct from key recovery and able to survive a primary identity-provider outage. Restrict initiation, require strong authentication and expedited approval, grant short-lived privileges, and correlate OCI Audit with identity and change-management logs for post-use review. Test it regularly; least privilege without workable emergency access creates its own patient-safety risk.
Layer 6: An evidence chain that does not duplicate ePHI in logs
OCI Audit records calls to supported public API endpoints, but it is not a complete record of every data-plane event. Object Storage is the case that matters: Audit captures bucket-related events, not object-level operations, so it will not tell you who read a given object. Enable service logs, key management logs, VCN flow logs, application events, and database auditing to close that gap.
Retention needs a number. OCI Audit retains events for 365 days, fixed at the tenancy level and unchangeable, while §164.316(b)(2)(i) requires six years. Export to Object Storage through Service Connector Hub and apply retention and immutability there — otherwise half your evidence chain disappears in month thirteen.
Application telemetry has one job: make a single request reconstructable months later. Each record should carry who asked, for which patient, the authorization decision, which sources returned, which model and prompt-template versions answered, when, the outcome, and the correlation identifier tying it together. What it should not carry is content. Keep prompts, clinical notes, and generated summaries out of general logs unless your risk analysis specifically calls for capturing them and the platform protects those records appropriately — both conditions, not either one.
How the layers cover one another’s failures
Defense in depth is a claim, not a label: when one control fails, the next must contain the damage. Test the architecture by walking a failure through the stack.
Threat walk: stolen care-manager session
- Initial access: Phishing captures an authenticated session despite MFA.
- Layer 5—Identity: The session reaches the application, but not the database, keys, or model endpoint directly.
- Layer 4—Authorization: Pre-retrieval controls limit requests to the care manager’s assigned patients rather than the full corpus.
- Layer 2—Network: Default-deny egress blocks summaries from being sent to unapproved destinations.
- Layer 6—Evidence: The correlation identifier reveals which records were touched and which actions occurred, supporting a scoped breach assessment.
Repeat the walk for a compromised service principal, prompt injection, and an over-broad IAM policy. If any scenario ends with “and then the attacker has the corpus,” the architecture is not layered. It is just long.
These controls are intended to support applicable HIPAA Security Rule requirements; they do not establish compliance. Each organization must validate the design through its own risk analysis, implementation evidence, and legal review.
A staged path to approval
Start with scope and paper: BAA, assessed services and regions, data residency, shared responsibility. Document data flows, backups, recovery, support access, incident response, and third-party dependencies. Then deploy the baseline as code.
Build first with synthetic data, or with data de-identified under §164.514(b) by expert determination or Safe Harbor. Masking, tokenization, and pseudonymization do not de-identify PHI, and a retained re-identification key keeps data in scope under §164.514(c) — which collides directly with this design’s requirement to preserve source identifiers. Reconcile those before you build.
Then try to break it. Can a care manager pull a record for a patient they have no relationship to? Does a hostile string in a progress note change what the model does? Work the list – minimum-necessary retrieval, patient-level access authorization, model quality, injection defenses, tool permissions, key recovery, logging, emergency access, failover, and be honest about which you have exercised end to end and which you have only configured.
Introduce ePHI only after security, privacy, legal, clinical safety, incident response, and change management have each reviewed the design. Some answers will be “not yet,” and a few “not this workflow.” An architecture that never produces those answers is not being tested hard enough.
Next steps
No cloud service delivers compliance, and an architecture that suggests otherwise should not survive its first review. What a credible one does is show how the pieces hold each other up, and leave your organization holding what a reviewer actually asks for.
Before your review board, produce a versioned, time-stamped evidence package: the data-flow diagram; a working query answering, “who viewed this patient’s summary, and under what authorization?”; and a release manifest linking model, prompt, embedding, retrieval-index, and tool versions to validation results and approved changes. Designs fail review more often on missing evidence than on a missing control.
Further reading: OCI Generative AI · Dedicated AI Clusters · Confidential computing for healthcare workloads · Oracle Autonomous AI Database · OCI Security · Oracle HIPAA-assessed services