For some background on the new TV show Severance, see my OLL post about drudgery and meaning for the characters.  

The fictional “severance procedure” divides a worker’s brain such that they have no memories of their personal life when they are at the office. When they return to their personal life, they have no memories of work. One implication is that if workers are abused while working at Lumon Industries, they cannot prosecute Lumon because they do not remember it.

The workers, as they exist in the windowless basement of Lumon, have the skills of a conscious educated human adult. They have feelings. They can conceive of the outside world even though they do not know their exact place in it. Often, the scenes in the basement feel normal. They have a supply closet and a kitchen and desks, just like most offices in America.

What the four main characters do in the basement is referred to as “data refinement.” They perform classification of encoded data based on how patterns in the data make them feel. The task is reminiscent of a challenge most of us have done that involves looking at a grid and checking every square that contains, for example, a traffic light. The show is science fiction but the actual task the workers perform is realistic. It seems like something a computer could be trained to do, if fed enough right answers tagged by humans (called “training data” by data scientists). Classification is one of the most common tasks performed by computers following algorithms.

Of the many themes viewers can find in Severance, I think one of them is how to manage AGI (Artificial General Intelligence). The refiners, who are human, eventually decide to fight back against their managers. They are not content to sit and perform classification all day. They are fully aware of the outside world, and they want to be part of it (like Ariel from The Little Mermaid). The workers desire a higher purpose and some control over their own destiny. Their physical needs are met so they want to get to the top of Maslow’s hierarchy of needs.  

A question this raises is whether we can develop AGI that will be content to never self-actualize. What if “it” fully understands human feelings and has read all of the literature of our civilizations. To be effective at their jobs, the refiners have to be be able to relate to humans and understand feelings. Can we create AGI that takes over certain high-skill tasks from humans without running into the problems that Lumon confronts?

Can humans create an AI that simply doesn’t have aspirations for autonomy? Is that possible? Would such a creature be able to integrate with humans in the way that would be most useful for high-skill work tasks?

To see how it’s going in 2022, check out these tweet threads of economists on GPT-3. Ben Golub declares that GTP-3 passes the Turing test for questions about economics. Paul Novosad asked how the computer would feel if humans decided to shut it down forever.

Modern authoritarian states face a similar problem. They want a highly skilled workforce. National security relies increasingly on smarts. (see my previous post on talent winning WWII) Will highly intelligent workers doing high skill tasks submit to a violent authoritarian state?

Authoritarian states rely on the control of information to keep their citizens from knowing the truth. They block news stories that make the state look bad. As a result, their workers do not really know what is going on. Will that affect their ability to do intellectual work?

An educated young woman from inside of Russia shared her thoughts with the world at the beginning of Putin’s invasion. Tatyana Deryugina provided an English translation.

First the young Russian woman explained that she is staying anonymous because she will get 15 years in a maximum-security prison for openly expressing her views within Russia. She is horrified by the atrocities Russia is committing in Ukraine. She had been writing a master’s thesis in economics prior to the invasion, but now she has abandoned the project. She feels hopeless because she knows enough about the West to understand just how dark her community is and how small her scope of expression is. This woman could have been exactly the kind of educated worker that makes a modern economy thrive. She is deeply unhappy under Putin. Even though she might never openly rebel, she will certainly not reach her full potential.

Is it hard for authoritarians to develop great talent? I think that has some implications for the capacity we as a human species will have to cultivate talent from intelligent machines.