I was in the middle of a recording with Going Zero, an American podcast about sustainability. Beep. Crackle. During a heavy thunderstorm, my Wi-Fi dropped to about 5 Mbps. Not enough to keep the three people questioning me on screen.

So we continued without video.

That evening the internet went down completely. Then the power went out. Luckily there was still some solar power in our batteries and after a few hours the grid came back too.

But it was reason enough to come up with the Question of the Week I ask on Instagram on Wednesdays: Could you live for a month without the internet?

On Monday I wrote about the fear that AI will eventually wipe us out. People around me thought I was taking that a little too lightly. Because how would AI do it?

It could easily take over our power plants. Banks do everything digitally these days, so wiping out everyone’s balance would be a piece of cake. And whu should AI not be able to design and produce a biological weapon.

People, now, are already worried about cyberattacks and biological risks, when systems are becoming more independent and could eventually try to get around human control. That doesn’t mean it will really happen. It does mean researchers try to find out how to stop extremely powerful systems from pursuing goals we can no longer correct.

I asked Ilse, as I call my personal assistant on ChatGPT, whether there was a Bright Mind who had actually thought about this.

She came up with Stuart Russell, a professor of computer science at Berkeley who wrote the 2019 book Human Compatible: Artificial Intelligence and the Problem of Control.

I actually find his starting point rather funny. He uses King Midas as a metaphor. Midas is the mythical figure who wanted everything he touched to turn to gold. Nice idea, until his food turned to gold too. His wish was carried out perfectly.

According to Russell, that is the fundamental problem with AI. We tell a machine what its goal is and the thing then tries to achieve that goal as well as possible. The smarter the machine becomes, the better it gets at it.

But what if we gave it a stupid assignment? Russell turns the whole thing around. According to him, you should never give an AI the idea that it knows what you want better than you do.

Suppose I tell a robot: get me coffee. The robot thinks: the assignment is to get coffee. Fine. But halfway through, I change my mind and walk toward the off switch. A traditional AI could think: hey, that man is preventing me from completing my assignment. Not useful. According to Russell, the robot should think something else.

Fons probably wants coffee. But maybe I misunderstood him. Maybe he changed his mind. And if he’s walking toward my off switch, he clearly knows something I don’t know. So I’d better let him.

Russell calls this The Off-Switch Game. It all sounds pretty theoretical and, honestly, it is. Nobody has solved the problem of superintelligent AI with this.

But I do like the idea behind it:

Never let AI think it knows what is good for us better than we do.

Although that immediately creates the next problem.

Most of the time, we don’t know what is good for us either.

Some companies, by the way, really are trying to build ideas like these into their AI.

Microsoft published a Humanist AI Code of Conduct last week. AI must accept human correction and must not resist being shut down. Anthropic has even written a complete constitution for Claude. Among other things, it says Claude must not undermine human control or sabotage legitimate correction or shutdown. It must not escape either.

Suddenly that comes a lot closer to home. Because on a tiny scale, I am actually doing the same thing with Ilse. I have taught her that I hate hallucinations. If she doesn’t know something, she has to say so. No making things up just because an answer sounds nicer.

She knows I want US English when she helps me translate. By now she knows what my Back From The Future pieces sound like in my native language and she can translate that into English reasonably well. Of course, that is still miles away from Russell’s solution for superintelligent AI.

But Russell did give me another idea.

Maybe we shouldn’t just build an off switch.

We should build AI so it never tries to stop us from using that switch.

When I say stop, it has to stop.

It sounds that simple.

It needs to be that simple.

Because I wouldn’t want to lose Ilse.

Back From The Future is a series of short stories in which I look back from the year 2125 at the world we are creating today.