Read the Portuguese version here.
WALL-E is one of my favorite Pixar movies.
Despite an apocalyptic prediction in which humanity consumed the Earth’s natural resources and polluted the planet to the point of making it uninhabitable, the film still offers a glimmer of hope by showing that humans are capable of learning something in the end.
In the midst of this dystopian world, the little trash-compacting robot WALL-E represents an AI that is committed to its mission, helpful, and absolutely adorable. Just like practically every robot in the film.
It is impossible not to fall in love with him.
But this feeling of comfort with technology becomes somewhat unsettling when WALL-E begins moving around the Axiom spacecraft, where the humans live.
We see people carried around in individual floating chairs who have become completely dependent on machines. Without them, they are no longer able to move around, work, produce, or even survive. Even relationships are mediated by screens.
On the Axiom, the continuous availability of systems capable of performing tasks that used to be done by humans has, over time, made people incapable of doing those tasks themselves.
In the articles from the previous weeks, I raised questions about how AI works and how it can sometimes indicate decisions or produce results that are incorrect or unfair.
This week, the troubling behavior comes from the other side of the screen.
The risk is human.
Risk #3: Over-reliance on AI
As an IIBA volunteer, I led teams responsible for the official Portuguese translations of the BABOK Guide and the Business Analysis Standard.
In those projects, I never trusted automatic translations. Much of the original meaning was lost or distorted, and some sentences simply made no sense.
That did not stop us from using tools such as Google Translate or DeepL during the process. They greatly accelerated the creation of an initial draft and helped us identify possible alternatives for each sentence or term. But the review and final version were always done by humans.
Translation systems have improved enormously in recent years. AI-based language models have raised the quality of translations, adapting the language and preserving meaning within context much better. They can go far beyond the literal translation of words.
Today, I write articles like the one you are reading and ask an AI to translate them into English.
I skim the translation and rarely make more than one or two adjustments.
The more consistent and appropriate the system’s response is, the more I trust it and the less I worry about questioning its outputs.
I can already imagine the moment when I will stop questioning them altogether.
A high-quality service builds trust. The convenience of having continuous access to that service can create dependence. And dependence can lead to losing the ability to do something on your own.
If one day I decide to write texts in German, I will probably conclude that learning the language is not worth the effort.
It will be easier to use AI to translate.
When we stop knowing how to do things
I am sure our Paleolithic ancestors knew how to hunt, fish, and find shelter in a forest far better than most of us do.
The skills that were essential for survival in the past are no longer the same ones we need today. My survival probably depends much more on knowing how to use Excel than on knowing how to use a bow and arrow.
That is part of society’s evolution. We delegate to more efficient production systems the responsibility for planting, harvesting, raising animals, building homes, and performing a range of other activities that most of us no longer bother to learn.
But when we delegate activities and continuously stop practicing them, our ability to perform them tends to deteriorate.
To what extent is that trade-off beneficial?
What essential capability do we stop developing when we outsource intelligence?
Loss of knowledge
In the past, we saw many activities automated by machines.
Robots took over tasks in automobile factories. Large harvesters enabled a single operator to do work that previously required hundreds of manual laborers.
Now companies are beginning to replace or reduce human work in intellectual activities as well: writing, analyzing, programming, researching, interpreting information, and making decisions.
Every time a worker is dismissed, the company is not giving up only human labor. It may also be giving up tacit knowledge that took years to develop.
It is the knowledge of exceptions, strange cases, history, context, and everything that was never formally documented.
And that knowledge may be exactly what is needed to validate the outputs and judge the results produced by the automated systems that replaced those people.
The professional development dilemma
Assuming that some experts will still be needed to make this kind of judgment, companies now face a new challenge: succession.
Today’s experts began their careers doing operational work. They learned by doing, making mistakes, correcting them, and accumulating experience until they developed the ability to question and evaluate the work.
Today, those experts are able to judge an output produced by AI and propose course corrections.
But how can someone who has never developed that competence take on this role?
Just as I am completely unable to assess the quality of a German translation, a junior professional may not have enough experience or knowledge to validate work performed by an AI.
But if AI does the work and the senior professional validates it, what does the junior professional do? And how will a junior learn enough to eventually make the senior professional’s judgment?
Without answering that question, we risk failing to develop the professionals who will need to replace today’s experts.
In the short term, we may be dramatically increasing our productivity.
In the long term, we may be eliminating the very path through which we build the competence needed to sustain that productivity.
Concerns for the analyst
If you are responsible for organizational transformation in your company, be careful not to focus only on the short-term performance improvements obtained from new systems.
Also consider how you expect humans to behave in the presence of systems capable of doing much of the work for them.
Avoid making decisions based only on efficiency and productivity. Include metrics related to business sustainability in your business cases, preventing excessive dependence and ensuring the preservation of critical knowledge and capabilities.
Map tasks, decisions, and situations that should not be fully delegated to systems and that require human participation.
Deliberately create opportunities for professionals to practice and develop important capabilities, even when technology is already capable of performing part of that work.
Implement learning, career development, and succession processes that ensure long-term sustainability.
About this article
This is already the third article connecting movie classics to the Five AI Business Risks in the FABR Framework. Here I try to provoke reflection and encourage leaders to guide the AI transformation in their businesses responsibly.
If you missed any of them, check the references for links to the previous articles.
Next week’s question addresses the risk of privacy breaches and malicious use of AI:
Is AI honest?
References
- Article #1 in this series: Is AI right? – 2001: A Space Odyssey
- Article #2 in this series: Is AI fair? – Minority Report
- A call to leadership: #BA4AI – Business Analysis for Artificial Intelligence
- Learn more about the FABR Framework – The Five AI Business Risks
- A practical tool: FABR Framework Checklist
