Is it even AI? Closing the Week of Integrity with a harder question than it sounds

In short: “Is it even AI?” is the first question worth asking, because the answer decides which rules apply and who is responsible. Judith van Schie of Considerati walked through the real risks, the EU AI Act’s risk pyramid, where it meets privacy law, and why one shared framework beats twelve separate assessments. The fuller account is below.

The 10th Week of Integrity closed, fittingly, not with an answer but with a question. “Is it even AI?” sounds like a throwaway line, the sort of thing you mutter at a chatbot that has just invented a fact. Judith van Schie, Managing Director for Legal and Compliance at Considerati, spent the lunchtime session making the case that it is in fact the first and most useful question any organisation can ask, because the answer decides which rules apply, which risks matter, and who is on the hook. In conversation with Andrea Cardoso and Laure Jacquier of ICC Netherlands, she turned an hour on privacy, ethics and the EU AI Act into something unusually practical.

The question in the title

Most people, van Schie noted, now picture AI as ChatGPT and little else. That instinct is exactly the problem. Under the EU AI Act the real test is whether a system actually learns or infers, turning input into “a prediction or a score, a recommendation or a decision,” with some autonomy and an effect on the world. It has to be more than a decision tree or a calculation dressed up in marketing language. Since every tool now arrives with an “AI” sticker, working out what actually counts is the first discipline.

She reached for an analogy she admits is well worn but still earns its keep. The model, an LLM or otherwise, is the engine. The system built around it is the car. And the use you put it to is the road it drives on. The risk lives in the road, not the engine. A powerful model can be a Ferrari, wonderful on a racetrack and alarming “in front of a school where kids are crossing the street.” Same car, entirely different risk, depending on where you drive it.

Two kinds of risk

From there she separated the risks into two families. The first are risks inside the system: biased or simply wrong decisions, and model drift, because a model that “can be fine when you start out” can quietly degrade over time. There is the question of robustness, whether a system still behaves when conditions change or when someone deliberately tries to trick it. And there is the black box, the difficulty of explaining an outcome to the customer it affected, to a regulator, or to a court. She pointed to recent episodes, including research showing a model perfectly “capable of doing something that the developers didn’t want” and concealing it, to make an uncomfortable point: checking the output is not enough, because “we can be tricked by what the model is showing us.”

The second family is societal. The energy and climate cost, where she noted drily that frontier model providers “are now building their own nuclear power plants,” adding, “that says it all.” The labour market, and a widening gap between the AI-literate and everyone else. The concentration of power in a handful of American firms that control the models, the chips and the infrastructure, and the dependence that creates. The loss of the human touch, in empathy and judgment and creativity. Disinformation, where she observed simply that “it’s never been easier” to produce convincing fakes. And, hovering over all of it, the hypothetical of artificial general intelligence, which she treated with healthy scepticism, more interested in the concrete risks already in the building.

What the law actually asks

On the EU AI Act, van Schie offered the clearest short version of the risk pyramid you could want. At the top sit the prohibited practices, the “scary stuff” with a whiff of Black Mirror: social scoring, emotion recognition at work, manipulation of vulnerable groups, all banned in the EU. Below them, high-risk systems, which carry heavy safeguards before they can go to market, covering areas such as recruitment and access to essential services, plus AI embedded in safety-critical products. An AI placed in a lift, she pointed out, still has to be safe to ride. And then the broad base of the pyramid, the low and minimal-risk uses that make up most of daily life, where the obligations are lighter but real: tell people when they are dealing with AI, and make sure your staff can actually use the tool.

She was candid that timelines have slipped, with several deadlines pushed back partly because the standards organisations need have been slow to arrive. Her advice was not to wait. Build the inventory and the risk classification now, because the obligations are coming and the groundwork takes time.

Responsibility, she explained, turns on your role. A provider, the maker who puts a system on the market under its own name, inherits a long product-safety to-do list: risk management, data governance, technical documentation, human oversight by design, and for high-risk systems a CE marking. A deployer, who uses a system under its own authority, has a shorter but still serious list: competent human oversight, logging, monitoring, suspending use when something goes wrong, reporting incidents, and explaining decisions to the people they affect. The trap she flagged most firmly: if you change a system “in a significant way,” you can quietly become a provider yourself, and inherit the heavier burden. Stay within the maker’s instructions for use.

Where AI meets privacy

The AI Act and the GDPR are different laws, she stressed, one about product safety and one about data, but they overlap heavily the moment an AI system touches personal data, which is “most of the cases.” Rather than run two parallel compliance exercises, she argued for folding them together: weave the Fundamental Rights Impact Assessment and the Data Protection Impact Assessment into one, align the transparency duties, lean on the GDPR’s existing rules on automated decision-making and keeping a human in the loop, and treat data quality as the shared foundation it is, since bad data means bad output and real harm to people.

The silo problem, and the number that lands

The most resonant part of the hour was about organisation rather than technology. There is no neat correlation, she said, between a company’s size and its maturity; some large firms are advanced and some have barely begun. What trips everyone up is ownership. The recurring pattern runs “from nobody wants to own it to everybody wants to own it.” AI touches legal, HR, IT, security, risk, ethics and privacy all at once, and when those functions act separately the result is paralysis. She described clients running “8 to 12 separate assessments” for a single tool, so that approval for a product owner could take a year and a half.

Her prescription was refreshingly simple. “There should be one shared framework,” and a named owner for each AI system. Where that framework sits, in legal or security or mandated from the board, matters far less than that it exists; board backing helps mostly because “then people find it important.” And it is never a one-time exercise. Capabilities move so fast, she noted, that in November 2023 “we were laughing at some stupid outputs from ChatGPT,” and now it is “impossible not to be amazed.” Re-evaluate every six months.

Shadow AI, and the one line to remember

She closed the substance on shadow AI, the tools quietly in use across an organisation without anyone’s knowledge, an old shadow-IT problem now exploding as staff paste company and customer data into personal accounts. Punishment, she warned, only makes “the shadow even darker.” The better route is to keep self-reporting safe, understand why people reach for these tools, give them an acceptable alternative, set clear rules, and train continuously. Or, in the single line worth pinning above any compliance desk: “make the improved route the easy route.”

The questions to leave with

Her final slide was a set of questions rather than a set of answers, which suited the week. Before any AI project: what problem are we actually solving, and “don’t run after every shiny object”; is it even AI, which sets the whole framework; and what data does it use. And for boards: which uses have we ruled out, are we clear on our values and how we test for harm, what is our AI risk appetite, do we know what is running, who owns each system, and, most pointedly, who can pause or stop it, and how quickly.

It was a quietly fitting end to a week built around leading with integrity in a digital age. The technology is new and astonishing. The discipline it demands is old: know what you are using, decide why, and make sure a human can still switch it off.

The Week of Integrity continues year-round. Follow along via the Week of Integrity on LinkedIn, and look out for the spotlight on International Anti-Corruption Day on 9 December.