Building AI Systems Where Security Comes First
This post was written by mpathic CTO, Brian Williams. If you’d like to learn more about how we’re building AI safety infrastructure with security at the core, we’d love to talk! [email protected]
Earlier this week, reports surfaced about a security incident impacting Mercor, an AI company working with human data and model evaluation workflows.
The breach appears to have been linked to a vulnerability in LiteLLM, an open-source package deeply embedded in many AI infrastructure stacks and downloaded millions of times per day. Through that pathway, unauthorized access may have exposed sensitive information, including data tied to enterprise AI projects. In response, at least one major partner paused work while assessing the situation.
This incident was a painful reminder of a structural risk across modern AI systems.
Many AI systems today rely on layers of external tools, APIs, and open-source packages that often operate with significant access to data and infrastructure. That creates a new kind of risk surface.
A few key takeaways that stand out to me:
Supply chain risk is a core AI security risk
If an external tool sits in your runtime path or has access to your data, it should be treated as part of your attack surface.
Every dependency should be evaluated by what it can access
The question isn’t whether a tool is widely used or trusted. It’s what happens if that tool is compromised. What data can it see? What systems can it reach?
Security posture is defined by blast radius, not policy language
Policies and certifications matter, but they don’t prevent breaches. What matters is how much damage can be done when something fails.
Systems have to be designed for failure
AI Systems should be built with the assumption that something, somewhere, will break.
At mpathic, because we work with leading AI builders to make their models safer, our work involves highly sensitive data—ranging from human-generated inputs to regulated information across healthcare, education, and enterprise environments.
Because of that, and thanks to our origins in highly-regulated clinical environments, we’ve built our systems with the assumption that dependencies are not fully trusted, and failures are inevitable.
This shows up in several ways for us:
We minimize trust in third-party runtime components
External tools are carefully evaluated and scoped. We avoid giving any single dependency broad or unnecessary access to data or systems.
We isolate environments to limit blast radius
Work is segmented across tightly controlled environments so that a compromised tool or workflow cannot access unrelated data.
We aggressively constrain credentials and access
Access is role-based, limited, and continuously monitored. Credentials are scoped as narrowly as possible to reduce the impact of misuse.
We design systems assuming dependencies can fail
Rather than relying on the safety of upstream tools, we build controls that contain failures, so that one weak point doesn’t cascade into a larger breach.
We operate in tightly controlled execution environments
This includes options for VDI-based, locked-down environments or secure physical workspaces, ensuring sensitive data is never broadly exposed or locally stored.
We align with established security and compliance frameworks
Our processes comply with SOC 2, HIPAA, FDA 21 CFR Part 11, FERPA, CCPA/GDPR, and ISO standards. These frameworks act as validation of practices we already follow.
Because this foundation has been in place from the beginning, it doesn’t slow us down. We’re able to deploy teams of mpathic Experts within 24 hours and handle large volumes of sensitive data while maintaining strong controls.
As AI becomes more embedded in critical workflows, the systems behind it are handling more sensitive data, more frequently, and with broader impact. The companies that will be trusted in this next phase of AI are the ones that design for that reality upfront.
We’re committed to holding that high bar here at mpathic, and we hope others in the space are doing the same.
