Imagine deciding to build your own alarm clock powered by an Artificial Intelligence system.
Not because you actually need yet another alarm clock in your life. But because you can, you want to experiment a little more with AI programming and... well, we all know that’s already a perfectly valid reason to spend a few hours making a system that worked perfectly well much more complicated. 😄
The initial idea is simple. The alarm clock checks your calendar, determines what time you need to get up and randomly selects songs from your music collection.
But since we’re using Artificial Intelligence, we can make it a little more interesting, right? Of course we CAN.
We add geolocation so it can take our current time zone into account, make the volume gradually increase, and add a few voice commands:
- Weather — gives us the weather forecast for our current location.
- Schedule — tells us about the meetings, activities and tasks we have for the day.
- News — prepares a personalized summary using only the sources and industries we actually care about. Nothing about the latest blunder by whichever government happens to be in power, the fashionable footballer’s latest holiday or the romantic adventures of influencers, singers and rappers. Basically, the traditional celebrity gossip about singers and bullfighters, but in its 2.0 version.
- Stop — stops the alarm.
Weeks of small tweaks, new features and testing. Real tests and also tests just for the sake of testing.
Everything works perfectly.
And One Day, Pixies Starts Playing
And then, on the very morning when you have a super-important meeting first thing, the alarm clock randomly decides to start with Where Is My Mind?, by Pixies.
Even if you know NOTHING about Pixies —which, by the way, you should probably have looked at— you’ll probably recognize the song from Fight Club. It’s one of my favorite songs, although it’s far too well known to make my Top 5 Pixies songs.
Well, that song starts with one word:
STOP!
And the alarm clock obeys.
The music stops, we carry on sleeping and we’re late —or simply never make it— to our super-important meeting.
But there’s one small detail. The alarm clock hasn’t failed. We told it to listen for instructions. We told it to stop the music when it heard the word Stop.
It heard Stop. It stopped the music. From its point of view: FLAW-LESS.
The problem is that what we actually meant was slightly different:
When I say Stop with the intention of stopping the alarm, stop the music.
To us, the difference is obvious. To a machine, not necessarily.
Instructions and Intent Are Not the Same Thing
Humans constantly interpret context, irony, jokes, mistakes and double meanings without even realizing how much we’re doing it.
AI systems can also make overly literal interpretations when they don’t have enough context. The image above is already somewhat old and AI models have improved enormously since then, but it remains a pretty funny example of what can happen when a joke is interpreted literally.
Back in My Day, We Fixed This with a Simple if
For decades, we have automated processes using relatively deterministic rules:
IF condition THEN action.
If A happens, do B.
With AI, we are introducing something extraordinarily powerful into our automations: the ability to interpret.
We no longer have to explicitly program every possible situation. We can provide information, context and instructions and allow a model to decide what it should do.
But that also changes the problem.
Now we need to design everything surrounding the model: what information it receives, what context it knows, which tools it can use, what permissions it has, when it can act and when it should ask.
The entire system we build around the model is commonly referred to as an AI harness, and designing it is increasingly known as harness engineering.
The Model Can Be Intelligent. But the Process Still Has to Be Well Designed.
We also need to be able to reconstruct what happened afterwards.
For our alarm clock, a log might show something like this:
- 08:00:01 — Playing Where Is My Mind?
- 08:00:03 — Audio detected: Stop.
- 08:00:03 — Classified as instruction.
- 08:00:04 — Alarm stopped.
Now we know where the problem occurred.
It isn’t enough to know that something failed. We need to be able to answer:
Where? When? Why?
When Interpreting Means Acting
In our example, the consequence is being late for an important meeting.
But now imagine an agent capable of sending emails, modifying our CRM or ERP, updating documents or running applications.
The sequence becomes:
Perceive → interpret → decide → act.
The greater the consequences of an action, the more important it becomes to decide what the system can execute autonomously and what requires our approval.
This is where Human in the Loop comes in.
It doesn’t mean that a person has to manually approve absolutely everything. It means placing human intervention where it genuinely adds value because of risk, ambiguity or the difficulty of reversing an action.
Automation isn’t just about designing how a process should work.
It’s also about imagining how it can fail.
Where Is My Mind?
And this is where we humans still have an interesting advantage.
I joke around with my kids a lot. Too much. I can pretend to be angry and they know perfectly well that I’m joking. But they also know very well when I’m actually angry.
They interpret my words, but also my tone of voice, my gestures, the situation and our shared experience. If we’re waiting somewhere or walking down the street, I might tell them, “Don’t come too close to me or people will think you’re my kids,” and their usual response is to laugh, move even closer or grab my hand.
And if I tell them, “Put the dirty laundry in the fridge,” neither of them obediently opens the fridge and puts their socks inside. (Yes, sometimes I really do say things like that. And deliberately.)
They know something doesn’t add up.
They understand that I probably meant the washing machine, or that I said fridge as a joke. Or they simply ask me.
And I find that particularly interesting:
Sometimes being intelligent means precisely not doing literally what we’ve been told to do.
It means interpreting intent, using context, noticing when something doesn’t make sense and, when we’re not sure, asking.
An Overly Obedient AI
Perhaps one of the biggest challenges when automating processes with Artificial Intelligence will not be getting AI to do what we ask.
It will be getting it to understand what we meant, when we are actually asking it to do something and when it should ask us before doing it.
Because the problem with AI won’t necessarily be that it disobeys our instructions.
Sometimes the problem will be that it follows them too well.
So, if you ever decide to use AI to program your alarm clock, run one final test before putting it into production.
Play Where Is My Mind?
And see what happens after...
STOP!