When Is Your Application Ready to Handle PHI?
Your application is ready to handle protected health information (PHI) only when your team can explain and demonstrate how the whole lifecycle is controlled: where PHI enters, who and what can access it, which vendors touch it, how the AI behaves, and what happens when something goes wrong. A cloud account, a business associate agreement (BAA), or encryption by itself does not establish that readiness.
That answer is deliberately broader than a checklist. For a healthcare AI product, readiness is an evidence-backed operating condition that spans people, process, vendors, infrastructure, model behavior, and incident response. The goal is to identify the work that must be complete before real PHI is introduced, while keeping legal interpretations with qualified counsel.
Photography: Tima Miroshnichenko via Pexels (cover) and Cup of Couple via Pexels (go-live scene). The PHI lifecycle map is an original Software Sushi illustration. These photos are illustrative workplace scenes; they do not depict a healthcare organization, client data, or compliance status.
Start by mapping the PHI lifecycle
Security work is difficult to validate when the system boundary is vague. Begin with an inventory of every place PHI can be created, received, maintained, or transmitted. For a healthcare AI application, that may include an intake form, an EHR integration, file uploads, support tickets, prompts, retrieved documents, model outputs, tool calls, vector indexes, traces, analytics events, backups, and local development environments.
Do not stop at the happy path. A data-flow map should show what happens when an LLM call times out, when a user exports a conversation, when a support engineer investigates an error, when a deleted record reaches a backup, and when a third-party API returns an exception. The HHS Security Rule overview frames the safeguards around the confidentiality, integrity, and availability of electronic PHI (ePHI). Your map gives the engineering team something concrete to protect and the organization something concrete to review.
Assess risk as an ongoing lifecycle
A risk assessment is more than naming a threat once. HHS describes risk analysis as a foundational and ongoing process for identifying threats and vulnerabilities to ePHI, then using that understanding to shape safeguards. Its risk analysis guidance does not prescribe one methodology, but it does point teams toward a documented, repeatable evaluation.
For an AI product, ask what could expose sensitive content, corrupt a clinical workflow, or make the service unavailable. Consider an overly broad service account, a prompt that includes more context than a task needs, a retrieval index with weak tenant isolation, a trace provider that stores full inputs, a tool that can write to a system without confirmation, or an incident process that cannot reconstruct what happened. NIST SP 800-66 Rev. 2 provides practical implementation resources for the HIPAA Security Rule and is a useful engineering companion to the regulation.
| Lifecycle stage | Question to answer | Evidence a buyer should expect |
|---|---|---|
| Collect | What PHI is collected, and is each field necessary for the task? | Data inventory, purpose definition, and input validation. |
| Process | Which identities, services, models, and tools can access it? | Access matrix, service boundaries, and vendor review. |
| Store | Where do records, embeddings, prompts, outputs, logs, and backups persist? | Retention policy, encryption design, and deletion procedure. |
| Use | Can the AI produce an unsafe result or disclose context to the wrong user? | Representative evaluations, isolation tests, and human escalation paths. |
| Respond | Can the team detect, contain, investigate, and report an incident? | Alerting, runbooks, ownership, and an exercised response plan. |
Establish vendor boundaries before sending data
Every external service in the path deserves a clear answer. That includes the cloud platform, foundation-model provider, embedding service, vector database, observability tool, email provider, customer-support system, and any subcontractor that can receive or maintain the data. For each one, record what data is sent, why it is sent, how long it is retained, where it is processed, who can access it, and what happens when the relationship ends.
HHS explains that a cloud service provider that creates, receives, maintains, or transmits ePHI on behalf of a covered entity or business associate is generally a business associate, even if the provider cannot view the information because it is encrypted. The HHS cloud computing guidance also explains why a BAA and risk analysis matter. The HHS business associate guidance uses examples that are directly relevant to modern healthcare applications, including cloud providers and AI chatbots.
A BAA defines important obligations, but it does not make an architecture sound by itself. If a model provider's terms allow retention or secondary use that your workflow cannot accept, the design needs to change before PHI is sent. Where a product can work with properly de-identified information, document the method and residual re-identification risk. HHS recognizes Expert Determination and Safe Harbor as the two HIPAA de-identification methods; de-identification should be reviewed for the actual context rather than treated as a casual label.
Build controls around the minimum necessary path
Once the boundary is known, design the narrowest path that can perform the job. Use strong identity proofing and authentication, role- and attribute-based authorization, least-privilege service accounts, and separation between tenants and environments. Protect keys and secrets with managed controls, encrypt data in transit and at rest, and make the key and access model understandable to the people who operate it.
Logging needs the same discipline as application data. Audit events should help answer who accessed what, when, through which service, and what action occurred without copying full prompts or responses into every log line. Define retention and deletion behavior for primary records, indexes, caches, traces, queues, and backups. Test restoration, not just backup creation. A recovery plan that has never been exercised is an assumption, not evidence.
AWS describes this as a shared responsibility model. AWS provides services and controls that can support a compliant workload, but customers remain responsible for how they configure and use them. The AWS HIPAA compliance guidance explains the BAA and shared responsibilities, while the AWS HIPAA-eligible services reference identifies services that can be used within the stated conditions. "HIPAA-eligible" is not a certification of the application you build on top.
Test AI behavior, not only infrastructure
Traditional security checks will not tell you whether a model follows a malicious instruction hidden in retrieved content, reveals another tenant's context, calls a tool with the wrong authorization, or produces a confident answer when the evidence is missing. AI behavior needs its own test plan using synthetic or properly de-identified data before production PHI is introduced.
Test the ordinary path and the uncomfortable paths: prompt injection, indirect instructions in documents, sensitive-data leakage, cross-tenant retrieval, overbroad tool access, unsafe output, malformed files, conflicting sources, partial outages, retries, and human override. For each test, define the expected behavior, the threshold for release, and the evidence that will be retained. Include reviewers who understand the clinical and operational consequences of an incorrect result; a language model's fluency is not proof of accuracy.
The NIST AI Risk Management Framework is a voluntary resource for organizing trustworthy AI risk work. It does not replace HIPAA analysis or legal review, but it can help teams make evaluation, monitoring, and human oversight explicit instead of leaving them implicit in a prompt or demo.
When the right design depends on whether information or model behavior needs to change, our guide to RAG versus fine-tuning gives a separate decision framework. In either case, validate the behavior with representative data before introducing real PHI.
When evaluation exposes product gaps, a production LLM development workflow should address retrieval, tool boundaries, and testable behavior as one system rather than treating them as separate launch tasks.
Operate with evidence after launch
Readiness is not a one-time approval. Assign an owner for the PHI lifecycle and define how changes are reviewed. A new model, prompt template, retrieval source, vendor, region, logging destination, or support workflow can change the risk profile even if the user interface looks the same.
At minimum, the operating model should include workforce training, access reviews, dependency and vendor reviews, model and data evaluations, incident triage, escalation, and a documented change record. Runbooks should make the first response obvious: contain access, preserve evidence, identify affected systems and data, and involve the right privacy, security, clinical, and legal stakeholders. Exercise the plan with synthetic scenarios so the organization learns before a real event.
HHS has also proposed changes to the Security Rule. The HHS NPRM page makes clear that the current Security Rule remains in effect while that rulemaking proceeds. Proposed safeguards should be tracked as possible future work, not described to customers as current requirements or treated as final law.
Questions to ask before calling an application ready
Can you trace one PHI field end to end?
Pick a realistic field and follow it from collection through processing, retrieval, output, logs, support, deletion, and backup. If the answer stops at the primary database, the system boundary is incomplete.
What can the model provider retain or reuse?
Ask about prompts, outputs, abuse monitoring, support access, regional processing, deletion, and subcontractors. The answer should match the actual contract and configuration, not a generic statement about enterprise AI.
What happens when the model is wrong?
Define where a human reviews, which actions require confirmation, how uncertainty or missing evidence is surfaced, and how a customer can correct the record. High-stakes workflows need a safe path when the model cannot answer.
Who owns the decision after launch?
There should be a named owner for monitoring, access review, vendor changes, incident escalation, and release approval. A team can share the work, but accountability cannot be left to everyone and therefore no one.
Frequently Asked Questions
Does using a HIPAA-eligible cloud service make my application HIPAA compliant?
No. A cloud service can be part of a supportable architecture when configured correctly and covered by the appropriate agreement, but the application owner remains responsible for its data flows, controls, vendors, workforce, and operations. Legal and compliance conclusions should be reviewed with qualified counsel.
Can we send PHI to an LLM if the provider signs a BAA?
A BAA is an important boundary, not a complete technical approval. Confirm that the provider's actual terms, retention behavior, region, access model, and configuration fit the use case, then test the application around leakage, authorization, retrieval, tool use, logging, and failure recovery.
What should we use instead of PHI while building?
Start with synthetic data or data that has been de-identified using a documented method appropriate to the context. PHI is never risk-free, and a development team should not introduce it simply because a prototype feels more realistic.
Is a risk analysis a one-time document?
No. It should be updated when the data, model, vendor, architecture, threat environment, or operating process changes. The initial assessment creates a baseline; ongoing review keeps that baseline meaningful.
What does Software Sushi actually provide?
Software Sushi can help map the PHI lifecycle, review vendor and infrastructure boundaries, evaluate AI behavior, and turn open risks into an implementation plan. That engineering support is designed to help clients meet their HIPAA compliance obligations; it is not a legal opinion or a certification.
Ready to Turn PHI Readiness Into Evidence?
Healthcare AI becomes easier to govern when the team can point to the data flows, controls, tests, owners, and operating decisions behind it. The question is not whether a product has a compliant-looking cloud diagram. It is whether the organization can control and explain the whole lifecycle before and after launch.
If you are moving from a prototype to a production healthcare AI workflow, we can help assess the architecture, vendors, AI behavior, and launch evidence.
This article is educational and not legal advice. HIPAA obligations depend on the facts, role, contracts, and applicable law. Have qualified privacy and legal counsel review close interpretations before production use.