PHI Data-Flow Architecture
An end-to-end map of where PHI is collected, processed, stored, and deleted, including retries, exceptions, support access, and backups.
Is your application ready to handle PHI?Healthcare LLM Development
HIPAA-Ready LLM and AI Development for Healthcare
We build healthcare LLM applications around the path protected health information takes: what the model sees, where it is stored, who can access it, and how the system behaves when the evidence is missing.
- PHI path mapped first
- Controls built into the app
- Tested before launch
Beyond the BAA
A signed agreement does not make an LLM workflow safe for PHI
A business associate agreement with a model provider is an important contractual boundary, but it does not secure the application around it. PHI can still leak through prompts, retrieved documents, logs, traces, caches, support tooling, and tool calls.
We engineer healthcare LLM systems so each of those paths is deliberate: minimized data, server-side permissions, controlled retention, and evaluation that shows how the system behaves before real patients and clinicians depend on it.
Start with the data path
We map every place PHI can travel before choosing a model
Intake forms, EHR or partner integrations, prompt assembly, retrieval, model inputs and outputs, vector indexes, logs, exports, and backups all belong on the map. That inventory drives the architecture, the vendor review, and the controls that come next.
Healthcare AI with the controls designed in
We choose the simplest architecture that meets the clinical or operational need, then engineer the privacy, security, and evaluation work around it.
Minimization and De-identification
Workflows that send the model only the information a task needs, using de-identified or synthetic data wherever it is sufficient.
Private and BAA-Covered Model Hosting
Deployment on managed model services covered by a provider BAA, or self-hosted models in a cloud account you control, chosen per workflow.
RAG Over Clinical and Policy Content
Retrieval over policies, guidelines, and records with server-side permission checks, so answers are traceable and users only see what they are allowed to.
How we evaluate RAG retrieval qualityAccess Control and Audit Logging
Authentication, least-privilege access, encryption, and audit trails across the application, model calls, and supporting services.
Evaluation and Safety Testing
Tests for accuracy, missing-evidence behavior, prompt injection, cross-patient leakage, and unsafe tool use, run before launch and after changes.
Production experience
Hardening a safety monitoring platform for sensitive health data
We helped a correctional technology company bring an AI-powered safety monitoring platform to production, including HIPAA-focused security, architecture, and remediation work, alongside AI and computer vision workflows and deployment support.
See the work we have delivered- HIPAA
- focused security, architecture, and remediation
- AI
- and computer vision workflows for safety monitoring
- Prod
- engineering, testing, and deployment support
Engineering, not legal advice
We help you meet your HIPAA obligations; we do not replace your counsel
Software Sushi follows HIPAA-compliant development practices for PHI across our healthcare AI and data work. That engineering support is designed to help clients meet their own HIPAA compliance obligations.
It is not a legal opinion or a certification. We work alongside your privacy, security, and legal stakeholders, and we document the decisions and evidence they need to review.
Can we send PHI to an LLM? Read the frameworkFrom PHI readiness to a monitored healthcare AI system
- 01
PHI Readiness Assessment
Map the workflow and the PHI path, review vendors and infrastructure boundaries, and turn open risks into a prioritized plan.
Learn how our consulting works - 02
Proof of Concept
Validate the approach with synthetic or de-identified data first, so feasibility is established before real PHI enters the system.
How our AI proof of concept service works - 03
Build and Test
Implement the application, retrieval, permissions, logging boundaries, and evaluation as one tested system.
- 04
Launch and Operate
Go live with documented controls, then monitor quality, access, and changes, with clear ownership for reviews and incidents.
What teams ask before we start
Clear answers on scope, architecture, data, and delivery.
Is a healthcare LLM safe to use with PHI?
It can be, but only when the complete workflow is controlled. A signed BAA with the model provider is one part of the decision. You also need to understand where prompts, outputs, retrieved documents, logs, and backups persist, who can access them, and how the application behaves under missing evidence or hostile inputs.
Is Software Sushi HIPAA compliant?
Yes. We follow HIPAA-compliant development practices to protect PHI across our healthcare AI and data processing work. HIPAA compliance is a distinct practice for us, not a team certification or a platform partnership.
Which models can be used with PHI?
That depends on the provider agreement, the specific service and configuration, and your own requirements. Options include managed model services offered under a BAA and self-hosted open models in an environment you control. We confirm what the actual contract and configuration cover rather than assuming.
Do we need a private or self-hosted model?
Not always. A self-hosted model can simplify some data boundaries but adds operational responsibility. We compare managed and self-hosted options against your data, risk, latency, and cost requirements before recommending one.
Can you work with de-identified data?
Yes. Where de-identified or synthetic data is sufficient, we design the workflow around it, and we use it for early proofs of concept before real PHI is involved.
Is your work a substitute for legal review?
No. Our engineering work supports your HIPAA compliance obligations, but it is not legal advice or a certification. Qualified privacy and legal counsel should review the decisions before production use.
Related expertise
All services Keep exploring
Your workflow
Book a Discovery CallPlan a healthcare AI system you can defend
Tell us the workflow, the data involved, and the users who depend on it. We will help you map the PHI path and the responsible next step.