Strategies, Approaches and Critical Lenses to Use and Manage AI with Integrity

Week of Integrity interview series

Making Integrity Tangible in a Digital Age: Concrete Practices, Insights, and Reflections for Fostering Integrity in Professional Settings 

Marianna: How is artificial intelligence challenging existing structures of integrity within organizations and institutions?

Sofia Romansky:

There are many dimensions to consider. First, it is important to note that artificial intelligence in institutions across all sectors is not new. Various forms of artificial intelligence, including intelligent algorithms, have been used already for many years. But the attention that we are currently paying to artificial intelligence comes from a specific subcategory of the technology, which is generative artificial intelligence. For background and clarity, generative artificial intelligence at the highest level can be defined as a model that can, with various degrees of autonomy, process a particular input and based on the input cues, create some kind of output. This technology, as it is right now, is a probabilistic system, meaning that the modification from input to output happens based on a certain subset of training data. And these elements, the training data, the input, and the output, are all related to how integrity can be challenged.

As for the first element, the training data, we must acknowledge that humans are incredibly biased creatures. But that is not necessarily a problem. Bias and heuristics are things that have helped us survive throughout our evolutionary history. It is a feature of our biology and of our intellect which helps us navigate this very complex world where a lot of things are always changing and where we need to rely on pattern recognition to a certain extent to make quick decisions, especially when it comes down to questions of life and death and survival as such. However, as our societies changed, becoming progressively globalized communities, countries and cities, we became more aware of the entire population of humanity existing. When we think about bias between people, in-group and out-group, this bias is slightly different than our heuristic reflexes. It is where experiences that are directly in front of us make us potentially have a negative predisposition towards other people, or other experiences, because they are different from ours, or because we have been conditioned by our communities, our cultures and our societies to think about things in a certain way. Omitting a lot more complexity, these modes of thinking, as well as our reliance on heuristics, are transferred into the training data that generative models rely on. And this means that as an organization, when you are trying to ensure equal treatment, when you are trying to ensure equity, transparency, clarity of communication, not only is it then more important to be aware of these potential structures and biases with the use of generative artificial intelligence, but also to know how they could potentially transfer over into the data that is subsequently used for models. And that is something that is the responsibility of both developers, but also of the users.

As for the second element, our input and prompts, as individuals, we also need to be aware of our own biases when we are engaging with generative models. Knowing not only where we are positioned as individuals, but also what the strengths and weaknesses of the models are. What can they tell us and what can they cannot. I am using the words ‘tell’, but they provide outputs. We are using excessively humanizing language in a lot of cases when interacting with these models. I worked with artificial intelligence in the military domain and daily our bias comes into our interactions with these models.

Let me better illustrate this point with an example. Imagine this is a question to you, Marianna. If you were to prompt a generative model to give you advice on a decision about whether you should go to war, how would you ask the question?

Marianna: I would list all the motivations why I want to go to war and ask the machine to confirm. I would expect the machine to say that I am right, based on the reasons I listed in my input. And I would expect it to say: ‘You are right. So go to war!’.

Sofia Romansky:

Exactly. The way that you ask the question predisposes the model to take on your worldview as correct. If the model was designed to question everything that was said and had its own reference frame, it would not work the way we know it to right now. So it has to take on whatever the user is inputting as true or correct. It will probably say to you that considering these factors, yes, you should go to war because you are assuming that the right thing to do is to go to war because we, as people, also have a war bias. You know, think about theories of international relations, first mover advantage, those kind of things. So that is on the individual level, that is on the input side. When we prompt, it is a question of integrity to be aware of the biases that both our prompt and the model might have and might transfer to the output.

And then finally, on the third element, the output, it might challenge integrity both at the individual and institutional level. Questions like ‘What happens with the outputs? How do we perceive these outputs? Do we accept them automatically? What are our processes to verify and to check?’ It is the responsibility of individuals and of institutions to make sure that we have the systems in place to both benefit from what the models can do and to interrogate ourselves on what kind of questions are we actually asking, but also to still develop knowledge outside of these systems and to not become overly reliant on them.

Because then, and this is the final point that I will bring up for this question, there is another type of bias that comes into the question. Put simply, automation bias is the assumption that because these models have done the difficult processing of potentially massive amounts of data, they will come to an answer that is going to be better than whatever a human can come up with. However, the assumptions about the model are just assumptions.

But because of this bias, this belief that whatever is produced by a machine is more ‘neutral’ and ‘better’ than what a human can produce, we often end up seeing a lot of cognitive offloading, where analysis and thinking that actually allows us to reach good answers as people are handed over to the models.

Therefore, all three elements, the training side, the input side, and the output side are key points where integrity is challenged. And a lot of it has to do with awareness and taking time to think about how these technologies are used. But these technologies, paradoxically, are designed to encourage faster interaction and less thinking. That is the dilemma that we are stuck with.

Marianna: Why have the promised benefits to efficiency from artificial intelligence not manifested as expected?

Sofia Romansky:

With generative artificial intelligence specifically, the promised benefit was that we would become much more efficient by having access to a lot of pre-processed information. People would be able to tap into that information and be able to use more time to do other things requiring uniquely human attention. Things that are repetitive and monotonous would be left to the models, and the things that really require attention would be done by humans.

There were also statements about how much time these models would save. For example, now a lot of people use ChatGPT instead of Googling something, because everything you need to know now is supposedly contained within one answer. Users do not have to spend time going through links, identifying which ones are relevant to their query, and which ones are not. On the technical side, these models use something called natural language processing. Natural language processing is a fancy way of saying that they speak the same way that we do. They are speaking in prose, they use expressions like ‘this is what you should consider when you are looking at this and this and this...’. Supposedly, that also makes it easier for us to absorb information because we are most used to speaking with each other in the same way to exchange information. However, these promised benefits have not come to fruition for both technical and institutional reasons.

On the technology side, what we see is that there are limitations to generative models that people are not aware of. For one, interactions with these models often end up costing more time. These are probabilistic systems, and there is a chance that they are going to say absolute nonsense. That is just part of the design, not a flaw per se. It is just a side effect of the way that these models work. It is not something that you necessarily need to fix if you want them to work in a certain way. Because of that, we know that we have to be skeptical of the outputs. That means that, on the individual side, we still have to verify the outputs if we are responsible people and work with integrity, we cannot just passively accept the outputs. Ultimately, we still have to do the research ourselves. If we do not do the research and we do not verify the outputs, then we could end up with a lot of trouble down the line when quality assurance is carried out, for example, or when somebody else discovers that something is false, and then we have to start over.

On the institutional side, generative artificial intelligence is a relatively new category of tools that have now become accessible to people. And individuals do not necessarily have an intuitive or an appropriate way of interacting with these systems. There was a lot of hype around these models and given that the promises were so appealing within our current capitalist system, which often incentivizes quantity-driven practices, in many cases artificial intelligence came to be adopted just for the sake of AI itself, under an assumption that using it would create gains no matter how it was used. In many ways, the challenge of integrating generative models positively into organizations is a change management challenge. It is not about just getting the model and giving people access to it. It is about convincing people that it has benefits for your specific organization and showing them how they could benefit from it. I think that generative models can be very useful, but in very specific contexts with specific guidelines and also training that helps people understand where they should use them and where they probably should not use them.

This didn't happen initially. And so, effectively, a lot of organizations became split into immediate adopters and later adopters. Immediate adopters were rushing ahead but not always doing so responsibly because they did not receive the resources to do it responsibly. Also, people were resistant because they did not have the opportunity to learn how to use it responsibly. It is never good when AI is treated as a solution in and of itself rather than a tool to achieve a solution.

Marianna: What strategies can organizations use to encourage and incentivize integrity in the use of artificial intelligence?

Sofia Romansky:

The key word in that question is ‘incentivize’. If, for example, the main emphasis within an organization is on deadlines, that is going to create an incentive to only focus on meeting deadlines. And that, among other things, encourages the negative interactions with generative artificial intelligence that we discussed earlier. But if as an organization or as a company, you encourage people to think critically and managers set, let’s say, an interim deadline where updates can be shared and where they can re-evaluate whether the deadline can still be met without compromising quality standards, that is a very different approach that prioritizes quality over just meeting the deadline.

As an organization and especially as leaders of an organization, you need to lead by example. Whatever management is going to be doing is also what is going to set the standard for what employees are going to be doing. If management is using generative artificial intelligence as ‘shortcut’, then it is also going to be difficult for employees to find a justification and argue against these practices and ask for more time to understand how to use these models to develop good outputs and good results. Something that I encourage is to set up organizational guidelines which are aligned with your organization's goals, with its identity; a strategy of how to interact with generative models institutionally. This also means accepting that, in some cases, it might be unacceptable to use generative artificial intelligence because of the principles that are at the core of a company’s strategy or promises to particular stakeholders.

To summarize, the use of artificial intelligence with integrity within organizations is a matter of change management and organizational culture as well as an awareness of both of those things impacting all employees’ everyday work. Without the space to do things responsibly, no one is going to do things responsibly. Responsibility does not just happen, it has to be designed for. But designed for not just in a technical sense, but also in an institutional sense: in the structures, in the interactions, and in the culture.

Marianna: What methods can individuals adopt to use artificial intelligence with integrity?

Sofia Romansky:

Let's assume that everything I discussed in the previous sections is done: the institution gives you space, guidelines, training, information, and has a very clear way of approaching the use of generative artificial intelligence while the models themselves are designed in responsible ways. Best possible scenario. That still does not mean that artificial intelligence is going to be used effectively, because ultimately, it is still up to the individual to put in effort to use it responsibly and thus benefit from efficiency. And this is more of a cultural, social, psychological question.

When people use generative models for dealing with personal matters and seek advice, like questions about your landlord or questions about where to study, these interactions are going to impact the way they interact with the models professionally as well, because it becomes a habit to use the models. Finally, people may fall into the mindset that they just have to focus on the outputs, without the difficult processes which usually lead to a satisfying result. Going away from that is difficult because you must slow down. And in this world, in our current context, we are really encouraged to speed up everything all the time.

Marianna: Also the fact that people are thinking that everybody is better than them, or that they need to be perfect all the time. In this society, we are not allowed to fail.

Sofia Romansky:

Exactly. Such conditioning, if you experience it, is very difficult to unlearn. That is where the psychological part comes in. And to use generative models positively, you need to slow down. What is the best defense against automation bias? Let's say you prompt the model to tell you how to connect two sentences. You were writing something and you could not come up with a good connector. The flow was just all wrong, and you prompt the model to give you an answer. You read the output and are satisfied with it. Do you just automatically copy it into your document? Or do you take just a couple of extra minutes? This extra time you spend is not going to make or break the project, unless you are working at midnight right before a deadline, which you should not be doing anyways. But, if you take the extra time to think critically, you may learn from the output so that the next time you encounter a similar problem, you will not need to rely on the model to solve it.

In this way, these generative AI models can become tools for learning. But learning takes time, and you have to give yourself the time to learn as well. And this is a responsibility not just for individuals, but also for institutions and developers. The interface of most generative models that people interact with is actually really bad at encouraging people to be reflexive about what they do. The chat scrolling up and kind of disappearing means that you are more likely to forget what you already prompted. There is not a kind of overview that appears, even though it is fully within the capacity of these models to do that, to help you track your previous activity. The models want you to prompt more. The companies they belong to still have a business strategy that they have to account for. But if you as an individual are aware of these things, aware of the strengths and the weaknesses of these models, aware of your own biases, you can decide to use this technology to learn and improve your skills, for it to be a net positive. But that takes effort. And that takes an active kind of inoculation against cognitive offloading, which is very easy to fall into, given the context that we are in. It takes deliberate effort on all sides to start improving.

Marianna: How can individuals improve their critical sense?

Sofia Romansky:

I think by making it a habit. Every time you prompt something, you have to ask yourself: is this actually better than what I could come up with? Or even before you prompt, you can ask yourself: why do I want to prompt? Is it because I genuinely need help figuring something out? Or is it because I want to offload it? Because I do not want to think about it? And that is a difficult question to be honest about, but it does require you to be willing to be honest, and in the long-term it is going to help you a lot more.

We also have to remember that sometimes there are questions for which there is actually no answer, for example when you ask the model to tell you whether you will find a job, a partner, or if you will be admitted to a specific study program. And that is okay. Sometimes we just have to sit with discomfort and reflect. If you may need a second opinion, you have to ask yourself: do I want a second opinion from a generative model, or should I just ask my friend? Asking yourself this kind of critical questions, is a habit that people should develop. And ultimately, it will help people in all aspects of their life, to move with more integrity.

About author:

Sofia Romansky’s primary areas of engagement concern the impact of artificial intelligence (AI) on the social and military domains as well as issues around narratives and disinformation in online spheres as hybrid threats. Sofia also explores the importance of epistemic responsibility as a pedagogical element of the responsible development and use of AI, focusing on critical and reflexive interaction with generative AI.

She currently works as a Research Manager for ERA Cambridge, Subject Matter Expert at The Hague Centre for Strategic Studies, and Subject Matter Expert at the NATO Defense Education Enhancement Program.

From 2023-2026, Sofia was a Strategic Analyst at The Hague Centre for Strategic Studies, where she served as Project Coordinator for the Global Commission on Responsible AI in the Military Domain (GC REAIM).