Navigating the labyrinth of AI vendor security

Whether it’s predicting stock market trends or crafting personalized shopping experiences, AI is the new golden child of the tech world. But as businesses race to embrace this new frontier, there's a consideration that often goes unnoticed - the looming specter of security concerns.

Imagine, if you will, a grand expo hall echoing with the buzz of innovation. AI vendors from across the globe have set up shop, each booth more dazzling than the last. Here, a vendor showcases a virtual assistant so intuitive it seems to read minds. There, another presents a machine-learning model that claims to outpace human intelligence. The air is thick with promise and potential.

But as you meander through this maze of marvels, a persistent thought tugs at you: How secure is all this wizardry? It's a question that's easy to overlook amidst the glitz and glamour but arguably the most crucial query of all.

Beneath the polished presentations and slick demos lies a vast, intricate network. Data flows like lifeblood, algorithms churn, and servers hum in distant data centers. And just as a body is vulnerable to viruses, this digital organism is susceptible to breaches, hacks, and cyber-attacks.

You might be thinking, "These are reputable vendors; surely, they've fortified their fortresses!" And indeed, many have. But in the ever-evolving game of cyber warfare, new threats emerge daily. Today's impenetrable defense could be tomorrow's Achilles' heel.

So, as you evaluate potential AI partners, it's essential to don your detective hat.

Start with the elementary questions you’d ask any software solution. Where does the vendor store data? Is it encrypted both in transit and at rest? Who can access it, and under what circumstances? How frequently do they conduct security audits? What's their protocol in the event of a breach? And perhaps most tellingly, how transparent are they willing to be about all of this?

AI adds a bit extra spice to this line of questioning. It’s important to deeply understand what and how AI models are working with your information.

For example, a common practice when using large language models (LLMs) like GPT4 is fine-tuning the model on a subset of your data. In this case, you’ll want to dig deeper into what data is the LLM being trained on, whether any PII is being leaked to the LLM, who is hosting the LLM, what are their associated security protocols, and what are the steps involved in a situation where you need to delete customer data.

You can think about the unique challenges of AI security in a few ways:

1. Data Dependency: AI models are trained on vast amounts of data. The more diverse and comprehensive the data, the better the model. But with vastness comes vulnerability.

2. Model Transparency: Understanding how AI models make decisions, especially deep learning models and LLMs, can be like trying to decipher a black box.

3. Data Sensitivity: If your company is working with sensitive data - like personal health information or financial records - how is that extra sensitive data being protected?

Here’s a handy checklist of some specific questions you can ask your AI vendor:

1. Data Handling: How is data ingested, processed, and stored? Are there multiple layers of encryption in place, both at rest and in transit?

   • Ideally, the vendor is ingesting data through secure channels, processing it in isolated environments, and storing it in encrypted databases. Throughout this process, end-to-end encryption should be used, ensuring data is protected both in transit and at rest.

2. AI Data Access: How much access are AI agents or algorithms given to your data? What AI agents are being used and what are the security policies of those companies? Is data anonymized if being used in an algorithm?

   • Depending on what AI models the vendor is using, you want to ensure they aren’t passing sensitive data to other vendors - that means all model training and storage happens on-premises in a safe and secure environment. If the data is being used to train a model, also ensure it’s being anonymized before being ingested.

3. Data Sensitivity: If you’re working with sensitive data, how does the AI have access to it?

   • Ideally, PII should be anonymized and constrained so only necessary sensitive data is being transferred to the vendor. If the AI is being trained on PII, keep in mind that some models, like LLMs, have the risk of leaking data downstream.

4. Access Control: Who can access the data and what is the trail? Is there a robust role-based access control system in place? How frequently are access logs reviewed?

   • Only authorized personnel with specific roles should have access to data. The vendor should have a stringent role-based access control system in place, and access logs should be reviewed daily to ensure no unauthorized access occurs. If the vendor is using other vendors to provide AI features (i.e. Open AI or Anthropic), what are the related privacy and security policies?

5. Transparency: How open is the vendor about their security practices? Are they willing to share insights, audits, or even potential vulnerabilities they're working on and the data practices, especially in regard to AI?

   • The vendor should believe in complete transparency, be open about their security practices, share audit results, and actively engage with the community about potential vulnerabilities and their mitigation strategies. For services that utilize AI, there should be transparency on how data flows through the system.

In your quest to harness the power of AI, remember to tread with caution. Prioritize security as fervently as you would innovation. In this brave new world, a breach isn't just a technical glitch; it's a dent in trust, a blemish on reputation, and a potential unraveling of the very fabric of your business.

In the end, as you stand at the crossroads of potential and precaution, always choose the path that safeguards your enterprise's most precious asset for its customers: trust.