There’s a better name for it

Introduction 
As promised here is the second blog post about AI. It is divided into three parts. In the first part, I try to summarise some recent developments. The second part contains my first interview, done with an artificial customer (created with OpenAI’s GPT-3). Lastly, I want to share a fascinating proposal from a friend on how to redefine the meaning of the abbreviation “AI”.

Part 1: Progress Update

Disclaimer: The speed of innovation and jaw-dropping progress is at the moment just crazy. It is very likely that even more breakthroughs will be published in the next few weeks. Therefore, this collection can be only of temporary nature because, in my perception, we are riding right now an exponential curve. Furthermore, I want to emphasize that I am writing from the perspective of interested laypeople – I am not an expert.

As promised here the second part about the current developments in the AI field. Foremost, it is worth mentioning the latest buzz about the generalist AI GATO from Deepmind. What is GATO? In their own words: “The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm, and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens.” 

Overview of different tasks that GATO can handle on a (sometimes) super-human level, from computer games to controlling a robotic arm, it is a “weak” generalist (which still needs supervision during the learning phase )

There is one paragraph in the paper that really struck me:

This sounds as if three ingredients are enough to pave the way towards a “real” AGI. To express it as a simple equation: 

AGI = Mountcastle + Deep Learning + Massive Scaling

More about this aspect will be discussed later.

Secondly, Flamingo is another example of the fast progress, which is going on at the moment. This algorithm is dito from Deepmind and is a Visual Language Model for Few-Shot Learning (FSL).

Example of Flamingo detecting the context of an image 

Additionally, there are tremendous advances in the context of language models, as Google Research has shown with the PaLM model, which utilizes pathways and massive scaling to train such a model (540bn variables/features, 6144 TPU v4 chips, running for 64 days!):

Overview of different models and their number of features 

Indeed, the results are impressive, because this model is able to extract the context of a given statement and is able to answer questions. Eventually, the following examples have been cherry-picked, nevertheless, they are quite astonishing.

Explaining a joke

Input: I was supposed to start writing the paper at 5:00 PM. But then I started playing with this cool new language model for 10 minutes. 10 minutes later, it’s suddenly 9:30 PM!

Model Output: This joke is about how time flies when you’re having fun. The person was supposed to start writing a paper at 5:00 PM, but they started playing with a language model instead. They thought they were only playing with it for 10 minutes, but it was actually 4.5 hours.

Logical Inference

Input: Michael is at that really famous museum in France looking at its most famous painting. However, the artist who made this painting just makes Michael think of his favorite cartoon character from his childhood. What was the country of origin of the thing that the cartoon character usually holds in his hand?

Model Output: The most famous painting in the Louvre is the Mona Lisa. The artist who made the Mona Lisa is Leonardo da Vinci. Leonardo da Vinci is also the name of the main character in the cartoon Teenage Mutant Ninja Turtles. Leonardo da Vinci is from Italy. The thing that Leonardo da Vinci usually holds in his hand is a katana. The country of origin of the katana is Japan. The answer is “Japan”.

Lastly, DALL-E from OpenAI attracted a lot of attention in the last weeks. According to the website “DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.” The next example illustrates the power of this approach. The following text input created the shown image: “A teddybear on a skateboard in Times Square“

You can browse more images on the official Instagram account or do your own research – there are already hundreds of examples on the web. For instance, there is someone who created a set of pictures of three-star dishes that might be quite surprising. 

This kind of development attracts of course artists, therefore I want to share this piece of work from Merzmensch, even though the “Oxford monkeys” are more on the humorous side.

What else is going on? 

Next to the developments in the field of models, there is another trend observable. Because building a model is one aspect, training it is another topic. Therefore, the costs are of course critical to creating viable businesses. According to Governance AI the costs dropped: “The final training run of AlphaGo Zero in 2017 is estimated to have cost $35M. GPT-3, a SOTA NLP model developed in 2020 that is accessible via an API, has been estimated to have cost around $4.6M to train.2 Gopher – a recent frontier NLP model developed in 2021 by DeepMind – already doubled the compute requirements, costing around $9.2M. PaLM, a new SOTA NLP model from Google Research, is estimated to have cost between $9 and $17M to train. The training compute cost of developing the next SOTA NLP model will likely be even greater.”

And they show a very interesting plot:

“Since 2012, the computational resources for training SOTA models have been doubling every 5.7 months“

Now let’s combine this training compute trend with another signal, coming from the open-source hardware world. It is about the Kickstarter Project of the Turing Pi Cluster Board for Cloud Native Data Science Development. 

The campaign has collected already more than 1.3m$, even though there are still 24 days to go (as of May, 22nd). Its relevancy is obvious: This board will trigger the creativity of thousands of developers and we can only imagine how hardware hackers will use it to enhance products of daily life (make sure that you check out the last Twitter link in this post!).

The update is almost done, but one last piece is missing. It is about a legend in the machine intelligence field (and a person who is famous for the so-called Future Shock Levels from 1999). It is about Elizier Yudkowski who belongs probably to the avantgarde of futurists who have thought a lot about the challenges when dealing with an AGI. One central topic which needs to be tackled is called the “Alignment Problem“, which explains the difficulties of getting humankind committed to certain rules and principles that make it possible to deal with a super-human AI = to shut it off or contain it if it gets out of control. 

Therefore, at first, it seemed to be April’s fool that he stated that the Alignment Problem won’t be solved in time and that humankind may die in dignity… It is quite some strong tobacco in times of an ongoing pandemic, a climate catastrophe, and a war in Europe. However, it is meant to be an honest evaluation of the current developments… At least he believes that an AGI is not that far away anymore. Warning: It is a long read.

A few predictions 

While I stated in the last post that Deep Learning might have hit a wall (because I quoted Geoffrey Hinton as a relevant source), I feel like I have to revise the statement. GATO indicates to me that the scaling hypothesis might lead to some sort of AGI. This fits at least into my overall prediction, that this “thing” called intelligence is eventually, at its core, much more simple than we want it to be. Even Yann LeCun is tweeting and posting about his perception that Deep Learning has not hit the wall. It seems likely that at least a “weakly general AI” is achievable.

But… LeCun is also pointing out that the term AGI is misleading and that many more problems are waiting till something like a Human-Level AI (HLAI) can be achieved.

I would like to close this part with two crowd-sourced predictions. At first, let’s see what the Metaculus community of super-forecasters estimates when a weakly general AI will be devised (as of May, 22nd):

According to this prediction a weakly AI will be devised by end of July, 2028 

Another interesting prediction deals with the question, by which date a machine learning model will score above 50.0% on the MATH dataset before 2025? FYI: The MATH data set contains 12.500 challenging competition mathematics problems.

BTW: Further exploration of potential evolutionary paths of an AGI with the help of “Power Laws” like Wright’s Law, Koomey’s Law, Metcalfe’s Law, or the Bass Distribution could lead to exciting insights, but this shall be part of a completely new piece.

Part 2: Praxis: An interview with an artificial customer persona

First a shout out to Katharina for inspiring me to run such an experiment. In this experiment, I wanted to find out, to which extent it is possible to generate an artificial customer persona, that can be interviewed via GPT-3 from OpenAI. 

How does it work?

It is straightforward: I just described the task via a piece of written text. In the following screenshot, you see what I wrote. Somehow it feels a bit strange that there is no need to code anything in the Playground – you enter some text, hit submit, and the predicted text is added to the text body. It is just a text input field (also known as ‘prompt).

Screenshot of the OpenAI Playground Prompt (DaVinci)

Step by step the interview evolved: From a simple “hello” to an almost philosophical discussion, the sequence was always the same. I wrote the text of the interviewer and asked a question and hit the submit button – and then Luigi “answered” my questions. 

The interview

(For better readability, the answers are set to italic. Enjoy reading and discover how Luigi provides business insights!)

Create a persona that works in the hospitality industry and let it answer questions in an interview:

The persona’s name is Luigi. He is the owner of an Italian restaurant in a fancy neighborhood in the middle of a big city in Germany. He is 48 years old and is self-employed for 20 years. He knows the restaurant business very well. He started his career as a cook and took over the business from his father, so his restaurant has a very high reputation due to its long tradition. The motto of the restaurant: Customer first! He is married to his wife Diana and they have one daughter. Luigi’s music taste varies from modern jazz to opera, so the ambiance of the restaurant changes often. He loves to be together with his family – of course in Italy to enjoy good food and good times. He is open-minded and prefers liberal political views and thinks that everyone shall be happy, as long nobody gets hurt or limited in each’s freedom. He loves to experiment and changes regularly the menu. This implies he enjoys surprising his guests with Italian classic and modern cuisine. He uses his mobile phone a lot to promote the restaurant on social media, for example on Instagram he shows his newest creations, or on Facebook, he announces changes to the menu or opening hours. One of his biggest problems is that he has not enough time to create new dishes, because he has to do so many tasks: from marketing to shopping, up to planning the staff – and replacing waiters if they spontaneously cancel a shift. He wants to have more time for the value-creating tasks and NOT do administrative paperwork.

Interviewer: Hey Luigi, nice meeting you!

Luigi: Nice to meet you too!

Interviewer: Why do you love cooking?

Luigi: I really enjoy cooking because it’s a way for me to express myself. I also love to see the look of satisfaction on my guests’ faces when they enjoy my food.

Interviewer: Would you use a food delivery service that saves you time?

Luigi: I would definitely consider using a food delivery service to save me time in the kitchen. I would need to make sure that the quality of the food is up to my standards, but if it is, then I would definitely consider using it.

Interviewer: What else would you expect from your favorite food supplier?

Luigi: I would expect my favorite food supplier to provide me with high quality ingredients that I can use to create delicious dishes for my guests. I would also appreciate if they could provide me with helpful tips and advice on how to best use their ingredients.

Interviewer: What is more important to you? The price of an ingredient, the availability or quality?

Luigi: The quality of an ingredient is definitely more important to me than the price or the availability. I want my guests to enjoy the best possible food, so I will always choose quality over anything else.

Interviewer: How much would you pay for an integrating office solution that saves you 10% of your administrative working time?

Luigi: I would be willing to pay a reasonable price for an integrating office solution that saves me 10% of my administrative working time. I would need to make sure that the quality of the product is good, but if it is, then I would be happy to pay for it.

Interviewer: Please be clear – tell me a number! How much would you pay per month as a subscription fee?

Luigi: I would be willing to pay a monthly subscription fee of €50 for an integrating office solution that saves me 10% of my administrative working time.

Interviewer: Thank you for your answer! How much would you pay for a solution that saves you 20% of your administrative working time?

Luigi: I would be willing to pay a monthly subscription fee of €100 for a solution that saves me 20% of my administrative working time.

Interviewer: And if you could save 100% of our administrative working time?

Luigi: I would be willing to pay a monthly subscription fee of €500 for a solution that saves me 100% of my administrative working time.

Interviewer: Why EUR 500? What is the reason for this amount of money?

Luigi: €500 is a reasonable price for a solution that saves me 100% of my administrative working time. I would be happy to pay this amount of money in order to save time and be more productive in my restaurant.

Interviewer: Let’s switch to a different topic! Why do you use Amazon to buy equipment for your restaurant?

Luigi: I use Amazon to buy equipment for my restaurant because it is a reliable and trustworthy source for high quality equipment. I know that I can trust Amazon to provide me with quality products that will help me run my restaurant more efficiently.

Interviewer: What does it need to switch from Amazon to an alternative supplier?

Luigi: It would need to be a supplier that is reliable and trustworthy, and that can provide me with high quality equipment at a competitive price. I would also need to be sure that the supplier can meet my needs in terms of the type of equipment that I need.

Interviewer: What do you mean with trustworthy?

Luigi: I mean that the supplier is reliable and will always provide me with quality products that meet my needs. I would also need to be sure that the supplier will be there for me when I need them, and that they will provide me with good customer service.

Interviewer: Aha! Customer Service – what does that mean to you?

Luigi: Customer service means that the supplier is always available to help me when I need them, and that they are willing to go the extra mile to help me meet my needs. I would also need to be sure that the supplier has a good reputation.

Interviewer: What trends do you foresee for restaurateurs?

Luigi: I think that the trend for restaurateurs will be to move towards more online ordering and delivery. I believe that this is the future of the restaurant industry, and that more and more restaurateurs will move in this direction.

Interviewer: Thank you Luigi! Finally, do you have any words of wisdom for our readers?

Luigi: I would advise restaurateurs to move towards more online ordering and delivery. I believe that this is the future of the restaurant industry, and that more and more restaurateurs will move in this direction.

– End of Interview –

Does such an interview provides insights?

I would say that the presented example does not contain new insights, but it rather confirms the described persona. So the insights are not surprising but reflect the quality of the input – and it was pretty short. Imagine which complexities could be created if the persona’s description were a full-fledged biography? Again a teaser: check out the last link in this post!

Part 3: So what? What else could be the meaning of AI?

Yann LeCun proposed HLAI (Human-Level AI) as an alternative to ‘Artificial Intelligence’ and it sounds very feasible to use this term. But personally, I prefer another idea, coined by the initially mentioned friend. 

AI = Alien Intelligence

Does it sound strange to you? I hope it does because that’s what this name is about – it is not human intelligence but about a “non-human” flavor. When looking especially at the more serious art pieces of Dall-E and GPT-3, it feels really alien… 

The last example in this post might illustrate the general idea of ‘Alien Intelligence’.