A Chief AI Officer on Shadow AI in Today’s Enterprises

Rob Williams, a Fractional Chief AI Officer and founder of SHS Advisory, Works with some of the most heavily regulated organizations in the world, helping them streamline and advance digital processes, which often means confronting the fact that the majority of their employees are using artificial intelligence tools that nobody in IT or compliance knows about — also known as shadow AI.

And from his experience, the standard response to it, banning and blocking, is often making the problem worse. As part of or recent FILED Talks series, I spoke with him about what shadow AI actually looks like inside organizations today, where the real risks lie, and why he always starts with an amnesty.

The following is an excerpt, of that conversation edited and condensed for brevity and clarity. Watch the full conversation: Shining a Light on Shadow AI.

Joe Pearce: With me today, we have Rob Williams. He is a fractional Chief AI Officer and a thought leader in the industry, and someone that I've looked up to for a while. So welcome, Rob. It's great to have you.

Last year, there was a lot of chitter chatter around AI policies, but we seem to have moved on a bit as an industry. Today, we're going to talk about the big hot topic in AI governance, especially as we talk to privacy professionals, security folks: Shadow AI — all the AI systems that you don't know are being used as they're rolled out across your entire enterprise ecosystem.  

What's your definition of shadow AI?

Rob Williams: It’s pretty simple: It's unauthorized use of AI within the workplace, or more specifically AI that interfaces in any way with work content, which includes AI in remote workplace settings.  

Because there's some really, really novel shadow AI uses that I've come across recently, which are kind of stretching the definition of in the workplace or how you would consider people to use AI as part of their roles.

Joe Pearce: What are some of the most common shadow AI patterns that you're seeing? I'd love for you to just give me a few examples.  

Rob Williams: I've come across situations where people have had a pendant that's constantly recording, and they just have the microphone and the audio just coming out of the computer — recording everything that everyone says in every meeting, no interface with work, no way to track it, because it's just on their desk at home.  

I've also come across people who have had — when they are in certain meetings — their iPhone or their phone pointed at their screen in meetings, recording then pushing that through AI so they can ask questions about it at any point. They’re trying to avoid detection by keeping these tools isolated, completely off the network. Which speaks to how valuable these tools must be, that people are prepared to go to that length.

The most common pattern is still people using a simple one-shot tool like ChatGPT for pretty straightforward things — responding to an email, drafting a proposal. But I'm seeing more of using it as a personal tool to keep tabs on what's happening at work.

Sometimes that's about productivity. Sometimes it's actually an employee who feels in a difficult situation using it to protect themselves.

What are some of the biggest gaps and misconceptions around what shadow AI exactly is within an organization?

One of the biggest misconceptions is that ChatGPT is this tool people use to draft an email. While that’s still true, a lot more people actually use AI to build applications, and they don't even realize that's what they're doing. With something like Cowork, they ask it to create some kind of process — and before they know it they've got a Python script running scheduled on their work PC.

I don't think management quite realize just how far that has gone. I'm seeing people build shared web services, host them on Vercel, have them run across departments. It's not just shadow AI. The shadow AI is now building shadow IT infrastructure. In some organizations, there's as much code being written by the finance department as there is by the development team.

Joe Pearce: What do you think is the biggest gap between what executives think is happening with AI and what employees are actually doing?

Rob Williams: I think one of the biggest gaps is that leaders don't realize just how talented and how far some people have gotten with this.  

What comes up to leadership are the challenges, risks, the problems — and don’t get me wrong there are plenty of risks. But we have Sally in the finance department who has spent weekends of her own time building solutions that mean she can now do the work of five people, and that's not being appreciated. Sally isn't being brought into a room and asked to share with the rest of the company.

A lot of the time it becomes about governance, risk, and shutting this down, rather than encouraging and enabling these people to understand the risks and guiding them down the right path.

I get asked, ‘how do I solve this? How do I stop this?’ It's not about how you stop it. It's about how you make it safe. All these people are trying to drive value for the company? So how do we create a safe space where they can actually play and do this stuff? I think that's the misconception.

Joe Pearce: Let's dig into the real risks. A lot of people hyperfocus on whether their prompts are training the model. What should they actually be worried about?

Rob Williams: ‘Am I training the model?’ That's still the thing every single person I speak to in a position of authority thinks is the risk. It’s actually incredibly low risk that a particular piece of data is going to end up within the model. A lot of organizations in heavily regulated industries have employees purchasing full accounts — SOC 2, GDPR compliant AI that's not going to train, not going to store data.

The real challenge is someone else finding out. If their clients have contracts that preclude the use of AI, or they're in a regulated industry that precludes it, the fact that those tools aren't on the approved list means the organization can face massive reputational damage, lose clients, lose respect. The employee is unknowingly breaching serious conditions that could mean the company ends up in court.

Another risk is people using free models, unregulated models, or models from other countries, and they're putting PII data in there. That’s a really big risk.  

The final risk — and sometimes the biggest — is that people building with AI don't realize how much it can hallucinate, how sycophantic it can be, or the fact that, that unless you tell it to include security controls, it won't. They're building a lot of code that may be insecure, vulnerable to prompt injection, leaking confidential information — all because they haven't been embraced or trained by the company.

How do organizations get a handle on shadow AI?

Rob Williams: I always start with an amnesty. Let's talk about what we're actually doing. Bring all of that up and say: ‘What are all the things we're doing with AI?’

Then create a community of practice and tease it all out. You can start to identify the seven, eight, nine use cases being heavily driven within shadow AI. Rather than thinking about tools or policy, build a policy around the use case, and bring in the people who've been doing that and ask: how do we make this safe? How do we take this shadow AI use case and make it into a formal policy? Which then frees you up from having to think in an abstract way because you're actually looking across a set number of use cases.

If you can drive those use cases to actual corporate adoption in a safe way, you've probably captured 60 to 80% of shadow AI use.  

Joe Pearce: I love the term amnesty — it changes the tone from punishment to enablement. But the stats say 70% shadow AI use in an organization. How do you actually stop it?

Rob Williams: You can't. Between 60 and 80% of people in the organization are using AI in this way. To try and stop them is impossible.

And these people are driving value. Anything that saves them time will ultimately save you time. This is a software product that people are going out and trying — with a real sense of discovery and achievement. If you can capture that safely and harness that enthusiasm, it can make the difference.

Get the full interview, including Rob’s real-world stories about discovering and enabling safe shadow AI.

Watch on demand: FILED talks: Shining a Light on Shadow AI >

‍