AI

We’re All Magical

Zachary BartschLeave a comment

The widespread availability and easy user interface of artificial intelligence (AI) has put great power at everyone’s fingertips. We can do magical things.

Before the internet existed we would use books to help us better interpret the world.  Communication among humans is hard. Expressing logic and even phenomena is complex. This is why social skills matter. Among other things, they help us to communicate. The most obvious example of a communication barrier is language. I remember having a pocket-sized English-Spanish dictionary that I used to help me memorize or query Spanish words. The book helped me communicate with others and to translate ideas from one language to another.

Math books do something similar but the translation is English-Math. We can get broader and say that all textbooks are translation devices. They define field-specific terms and ideas to help a person translate among topic domains, usually with a base-language that reaches a targeted generalizability. We can get extreme and say that all books are translators, communicating the content of one person’s head to another.

But sometimes the field-to-general language translation doesn’t work because readers don’t have an adequate grasp of either language. It isn’t necessarily that readers are generally illiterate. It may be that the level of generality and degree of focus of the translation isn’t right for the reader. Anyone who has ever tried to teach anything with math has encountered this.  Students say that the book doesn’t translate clearly, and the communication fails. The book gets the reader’s numeracy or understood definitions wrong. Therefore, there is diversity among readers about how ‘good’ a textbook is.

Search engines are so useful because you can enter some keywords and find your destination, even if you don’t know the proper nouns or domain-specific terms. People used to memorize URLs and that’s becoming less common. Wikipedia is so great because if you want to learn about an idea, they usually explain it in 5 different ways. They tell the story of who created something and who they interacted with. They describe the motivation, the math, the logic, the developments, and usually include examples. Wikipedia translates domain-specific ideas to multiple general languages of different cognitive aptitudes or interests. It scatters links along the way to help users level-up their domain-specific understanding so that they can contextualize and translate the part that they care about.

Historical translation technology was largely for the audience. More recently, translation technology has empowered the transmitters.

Continue reading →

Why Low Returns Are Predicted for Stocks Over the Next Decade

Scott Buchanan2 Comments

I saw this scary-looking graphic of S&P 500 returns versus price/earnings (P/E) ratios a couple of days ago:

JPMorgan

The left-hand side shows that there is very little correlation between the current forward P/E ratio and the returns in the next year; as we have seen in the past few years, and canonically in say 1995-1999, market euphoria can commonly carry over from one year to the next. (See here for discussion of momentum effect in stock prices). So, on this basis, the current sky-high P/E should give us no concern about returns in the next year.

However, the right-hand side is sobering. It shows a very strong tendency for poor ten-year returns if the current P/E is high. In fact, this chart suggests a ten-year return of near zero, starting with the current market pricing. Various financial institutions are likewise forecasting a decade of muted returns [1].

The classic optimistic-but-naïve response to unwelcome facts like these is to argue, “But this time it’s different.” I am old enough to remember those claims circa 1999-2000 as P/E’s soared to ridiculous heights. Back then, it was “The internet will change EVERYTHING!”.  By that, the optimists meant that within a very few years, tech companies would find ways to make huge and ever-growing profits from the internet. Although the internet steadily became a more important part of life, the rapid, huge monetization did not happen, and so the stock market crashed in 2000 and took around ten years to recover.

A big reason for the lack of early monetization was the lack of exclusive “moats” around the early internet businesses. Pets.com was doomed from the start, because anyone could also slap together a competing site to sell dog food over the internet. The companies that are now reaping huge profits from the internet are those like Google and Meta (Facebook) and Amazon that have established quasi-monopolies in their niches.

The current mantra is, “Artificial intelligence will change EVERYTHING!” It is interesting to note that the same challenge to monetization is evident. ChatGPT cannot make a profit because customers are not willing to pay big for its chatbot, when there are multiple competing chatbots giving away their services for practically free. Again, no moat, at least at this level of AI. (If Zuck succeeds in developing agentic AI that can displace expensive software engineers, companies may pay Meta bigly for the glorious ability to lay off their employees).

My reaction to this dire ten-year prognostication is two-fold. First, I have a relatively high fraction of my portfolio in securities which simply pump out cash. I have written about these here and here. With these investments, I don’t much care what stock prices do, since I am not relying on some greater fool to pay me a higher price for my shares than I paid. All I care is that those dividends keep rolling in.

My other reaction is…this time it may be different (!), for the following reason: a huge fraction of the S&P 500 valuation is now occupied by the big tech companies. Unlike in 2000, these companies are actually making money, gobs of money, and more money every year. It is common, and indeed rational, to value (on a P/E basis) firms with growing profits more highly than firms with stagnant earnings. Yes, Nvidia has a really high P/E of 43, but its price to earnings-growth (PEG) ratio is about 1.2, which is actually pretty low for a growth company.

So, with a reasonable chunk of my portfolio, I will continue to party like it’s 1999.

[1] Here is a blurb from the Llama 3.1 chatbot offered for free in my Brave browser, summarizing the muted market outlook:

Financial institutions are forecasting lower stock market returns over the next decade compared to recent historical performance. According to Schwab’s 2025 Long-Term Capital Market Expectations, U.S. large cap equities are expected to deliver annualized returns of 6% over the next decade, while international developed market equities are projected to slightly outperform at 7.1%.1 However, Goldman Sachs predicts a more modest outlook, with the S&P 500 expected to return around 3% annually over the next decade, within a range of –1% and 7%.42 Vanguard’s forecasts also indicate a decline in expected returns, with U.S. equities falling to a range of 2.8% to 4.8% annually. These forecasts suggest that investors may face a period of lower returns compared to the past decade’s 13% annualized total return.

After the Fall: What Next for Nvidia and AI, In the Light of DeepSeek

Scott BuchananLeave a comment

Anyone not living under a rock the last two weeks has heard of DeepSeek, the cheap Chinese knock-off of ChatGPT that was supposedly trained using much lower resources that most American Artificial Intelligence efforts have been using. The bearish narrative flowing from this is that AI users will be able to get along with far fewer of Nvidia’s expensive, powerful chips, and so Nvidia sales and profit margins will sag.

The stock market seems to be agreeing with this story. The Nvidia share price crashed with a mighty crash last Monday, and it has continued to trend downward since then, with plenty of zig-zags.

I am not an expert in this area, but have done a bit of reading. There seems to be an emerging consensus that DeepSeek got to where it got to largely by using what was already developed by ChatGPT and similar prior models. For this and other reasons, the claim for fantastic savings in model training has been largely discounted. DeepSeek did do a nice job making use of limited chip resources, but those advances will be incorporated into everyone else’s models now.

Concerns remain regarding built-in bias and censorship to support the Chinese communist government’s point of view, and regarding the safety of user data kept on servers in China. Even apart from nefarious purposes for collecting user data, ChatGPT has apparently been very sloppy in protecting user information:

Wiz Research has identified a publicly accessible ClickHouse database belonging to DeepSeek, which allows full control over database operations, including the ability to access internal data. The exposure includes over a million lines of log streams containing chat history, secret keys, backend details, and other highly sensitive information.

Shifting focus to Nvidia – – my take is that DeepSeek will have little impact on its sales. The bullish narrative is that the more efficient algos developed by DeepSeek will enable more players to enter the AI arena.

The big power users like Meta and Amazon and Google have moved beyond limited chatbots like ChatGPT or DeepSeek. They are aiming beyond “AI” to “AGI” (Artificial General Intelligence), that matches or surpasses human cognitive capabilities across a wide range of cognitive tasks. Zuck plans to replace mid-level software engineers at Meta with code-bots before the year is out.

For AGI they will still need gobs of high-end chips, and these companies show no signs of throttling back their efforts. Nvidia remains sold out through the end of 2025. I suspect that when the company reports earnings on Feb 26, it will continue to demonstrate high profits and project high earnings growth.

Its price to earnings is higher than its peers, but that appears to be justified by its earnings growth. For a growth stock, a key metric is price/earnings-growth (PEG), and by that standard, Nvidia looks downright cheap:

Source: Marc Gerstein on Seeking Alpha

How the fickle market will react to these realities, I have no idea.

The high volatility in the stock makes for high options premiums. I have been selling puts and covered calls to capture roughly 20% yields, at the expense of missing out on any rise in share price from here.

Disclaimer: Nothing here should be considered as advice to buy or sell any security.

DeepSeek vs. ChatGPT: Has China Suddenly Caught or Surpassed the U.S. in AI?

Scott Buchanan1 Comment

The biggest single-day decline in stock market history occurred yesterday, as Nvidia plunged 17% to shave $589 billion off the AI chipmaker’s market cap. The cause of the panic was the surprisingly good performance of DeepSeek, a new Chinese AI application similar to ChatGPT.

Those who have tested DeepSeek find it to perform about as well as the best American AI models, with lower consumption of computer resources. It is also available much cheaper. What really stunned the tech world is that the developers claimed to have trained the model for only about six million dollars, which is way, way less than the billions that a large U.S. firm like OpenAI, Google, or Meta would spend on a leading AI model. All this despite the attempts by the U.S. to deny China the most advanced Nvidia chips. The developers of DeepSeek claim they worked with a modest number of chips, models with deliberately curtailed capacities which met U.S. export allowances.

One conclusion, drawn by the Nvidia bears, is that this shows you *don’t* need ever more of the most powerful and expensive chips to get good development done. The U.S. AI development model has been to build more, huge, power-hungry data centers and fill them up with the latest Nvidia chips. That has allowed Nvidia to charge huge profit premiums, as Google and other big tech companies slurp up all the chips that Nvidia can produce. If that supply/demand paradigm breaks, Nvidia’s profits could easily drop in half, e.g., from 60+% gross margins to a more normal (but still great) 30% margin.

The Nvidia bulls, on the other hand, claim that more efficient models will lead to even more usage of AI, and thus increase the demand for computing hardware – – a cyber instance of Jevons’ Paradox (where the increase in the efficiency of steam engines in burning coal led to more, not less, coal consumption, because it made steam engines more ubiquitous).

I read a bunch of articles to try to sort out hype from fact here. Folks who have tested DeepSeek find it to be as good as ChatGPT, and occasionally better. It can explain its reasoning explicitly, which can be helpful. It is open source, which I think means the code or at least the “weights” have been published. It does seem to be unusually efficient. Westerners have downloaded it onto (powerful) PCs and have run it there successfully, if a bit slowly. This means you can embed it in your own specialized code, or do your AI apart from the prying eyes of ChatGPT or other U.S. AI providers. In contrast, ChatGPT I think can only be run on a powerful remote server.

Unsurprisingly, in the past two weeks DeepSeek has been the most-uploaded free app, surpassing ChatGPT.

It turns out that being starved of computing power led the Chinese team to think their way to several important innovations that make much better use of computing. See here and here for gentle technical discussions of how they did that. Some of it involved hardware-ish things like improved memory management. Another key factor is they figured out a way to only do training on data which is relevant to the training query, instead of training each time on the entire universe of text.

A number of experts scoff at the claimed six million dollar figure for training, noting that if you include all the costs that were surely involved in the development cycle, it can’t be less than hundreds of millions of dollars. That said, it was still appreciably cheaper than the usual American way. Furthermore, it seems quite likely that making use of answers generated by ChatGPT helped DeepSeek to rapidly emulate ChatGPT’s performance. It is one thing to catch up to ChatGPT; it may be tougher to surpass it. Also, presumably the compute-efficient tricks devised by the DeepSeek team will now be applied in the West, as well. And there is speculation that DeepSeek actually has use of thousands of the advanced Nvidia chips, but they hide that fact since it involved end-running U.S. export restrictions. If so, then their accomplishment would be less amazing.

What happens now? I wish I knew. (I sold some Nvidia stock today, only to buy it back when it started to recover in after-hours trading). DeepSeek has Chinese censorship built into it. If you use DeepSeek, your information gets stored on servers in China, the better to serve the purposes of the government there.

Ironically, before this DeepSeek story broke, I was planning to write a post here this week pondering the business case for AI. For all the breathless hype about how AI will transform everything, it seems little money has been made except for Nvidia. Nvidia has been selling picks and shovels to the gold miners, but the gold miners themselves seem to have little to show for the billions and billions of dollars they are pouring into AI. A problem may be that there is not much of a moat here – – if lots of different tech groups can readily cobble together decent AI models, who will pay money to use them? Already, it is being given away for free in many cases. We shall see…

Free Webinar, Jan. 25: Practical and Ethical Aspects of Future Artificial Intelligence

Scott BuchananLeave a comment

As most of us know, artificial intelligence (AI) has taken big steps forward in the past few years, with the advent of Large Language Models (LLM) like ChatGPT. With these programs, you can enter a query in plain language, and get a lengthy response in human-like prose. You can have ChatGPT write a computer program or a whole essay for you (which of course makes it challenging for professors to evaluate essays handed in by their students).

However, the lords of Big Tech are not content. Their goal is to create AI with powers that far surpass human intelligence, and that even mimics human empathy. This raises a number of questions:

Is this technically possible? What will be the consequences if some corporations or nations succeed in owning such powerful systems? Will the computers push us bumbling humans out of the way? Will this be a tool for liberation or for oppression? This new technology coming at us may affect us all in unexpected ways. 

For those who are interested, there will be a 75-minute webinar on Saturday, January 25 which addresses these issues, and offers a perspective by two women who are leaders in the AI field (see bios below). They will explore the ethical and practical aspects of AI of the future, from within a Christian tradition. The webinar is free, but requires pre-registration:

https://network.asa3.org/events/EventDetails.aspx?id=1889760&group=

Here are bios of the two speakers:

Joanna Ng is a former IBM-er, pivoted to a start-up founder, focusing on Artificial Intelligence, specialized in Augmented Cognition, by integrating with IoT and Blockchain, in the context of web3, by applying design-thinking methodology. With forty-nine patents granted to her name, Joanna was accredited as an IBM Master Inventor. She held a seven-year tenure as the Head of Research, Director of the Center for Advanced Studies, IBM Canada. She has published over twenty peer-reviewed academic publications and co-authored two computer science books with Springer, The Smart Internet, and The Personal Web. She published a Christianity Today article called “How Artificial Intelligence Is Today’s Tower of Babel” and published her first book on faith and discipleship in October 2022, titled Being Christian 2.0.

Rosalind Picard is founder and director of the Affective Computing Research Group at the MIT Media Laboratory; co-founder of Affectiva, which provides Emotion AI; and co-founder and chief scientist of Empatica, which provides the first FDA-cleared smartwatch to detect seizures. Picard is author of over three hundred peer-reviewed articles spanning AI, affective computing, and medicine. She is known internationally for writing the book, Affective Computing, which helped launch the field by that name, and she is a popular speaker, with a TED talk receiving ~1.9 million views. Picard is a fellow of the IEEE and the AAAC, and a member of the National Academy of Engineering. She holds a Bachelors in Electrical Engineering from Georgia Tech and a Masters and Doctorate, each in Electrical Engineering and Computer Science, from MIT. Picard leads a team of researchers developing AI/machine learning and analytics to advance basic science as well as to improve human health and well-being, and has served as MIT’s faculty chair of their MindHandHeart well-being initiative.

Study Shows AI Can Enable Information-Stealing (Phishing) Campaigns

Scott BuchananLeave a comment

As a computer user, I make a modest effort to stay informed regarding the latest maneuvers by the bad guys to steal information and money. I am on a mailing list for the Malwarebytes blog, which publishes maybe three or four stories a week in this arena.

Here are three stories from the latest Malwarebytes email:

 ( 1 )   AI-supported spear phishing fools more than 50% of targets A controlled study reveals that 54% of users were tricked by AI-supported spear phishing emails, compared to just 12% who were targeted by traditional, human-crafted ones. ( 2 )  Dental group lied through teeth about data breach, fined $350,000 Westend Dental denied a 2020 ransomware attack and associated data breach, telling its customers that their data was lost due to an “accidentally formatted hard drive”. The company agreed to pay $350,000 to settle HIPAA violations ( 3 ) “Can you try a game I made?” Fake game sites lead to information stealers Victims lured to a fake game website where they were met with an information stealer instead of the promised game.

The first item here fits with our interest in the promise and perils of AI, so I will paste a couple of self-explanatory excerpts in italics:

One of the first things everyone predicted when artificial intelligence (AI) became more commonplace was that it would assist cybercriminals in making their phishing campaigns more effective.

Now, researchers have conducted a scientific study into the effectiveness of AI supported spear phishing, and the results line up with everyone’s expectations: AI is making it easier to do crimes.

The study, titled Evaluating Large Language Models’ Capability to Launch Fully Automated Spear Phishing Campaigns: Validated on Human Subjects, evaluates the capability of large language models (LLMs) to conduct personalized phishing attacks and compares their performance with human experts and AI models from last year.

To this end the researchers developed and tested an AI-powered tool to automate spear phishing campaigns. They used AI agents based on GPT-4o and Claude 3.5 Sonnet to search the web for available information on a target and use this for highly personalized phishing messages.

With these tools, the researchers achieved a click-through rate (CTR) that marketing departments can only dream of, at 54%. The control group received arbitrary phishing emails and achieved a CTR of 12% (roughly 1 in 8 people clicked the link).

Another group was tested against an email generated by human experts which proved to be just as effective as the fully AI automated emails and got a 54% CTR. But the human experts did this at 30 times the cost of the AI automated tools.

…

…The key to the success of a phishing email is the level of personalization that can be achieved by the AI assisted method and the base for that personalization can be provided by an AI web-browsing agent that crawls publicly available information.

Based on information found online about the target, they are invited to participate in a project that aligns with their interest and presented with a link to a site where they can find more details.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~

But there is good news as well. We can use AI to fight AI: … LLMs are also getting better at recognizing phishing emails. Claude 3.5 Sonnet scored well above 90% with only a few false alarms and detected several emails that passed human detection. Although it struggles with some phishing emails that are clearly suspicious to most humans.

In addition, the blog article cited some hard evidence for year-over-year progress in AI capabilities: a year ago, unassisted AI was unable to match the phishing performance of human-generated phishing messages. But now, AI can match and even slightly exceed the effectiveness of human phishing. This is….progress, I guess.

P.S. I’d feel remiss if I did not remind us all yet again, it’s safest to never click on a link embedded in an email message, if you can avoid it. If the email purports to be from a company, it’s safest to go directly to the company’s website and do your business there.

WSJ: Nothing Important Happened in China, India, or AI This Year

James Bailey3 Comments

I normally like the Wall Street Journal; it is the only news page I check directly on a regular basis, rather than just following links from social media. But their “Biggest News Stories of 2024” roundup makes me wonder if they are overly parochial. When I try to zoom out and think of the very biggest stories of the past five to ten years, three of the absolute top would be the rapid rise of China and India, together with the astonishing growth in artificial intelligence capabilities.

All three of those major stories continued to play out this year, along with all sorts of other things happening in the two most populous countries in the world, and all the ways existing AI capabilities are beginning to be integrated into our businesses, research, and lives. But the Wall Street Journal thinks that none of this is important enough to be mentioned in their 100+ “Biggest Stories”.

To be fair, China and AI do show up indirectly. AI is driving the 4 (!) stories on NVIDIA’s soaring stock price, and China shows up in stories about spying on the US, hacking the US, and the US potentially forcing a sale of TikTok. But there are zero stories regarding anything that happened within the borders of China, and zero that let you know that AI is good for anything besides NVIDIA’s stock price.

Plus of course, zero stories that let you know that India- now the world’s most populous country, where over one out of every six people alive resides- even exists.

AI’s take on India’s Prime Minister using AI

This isn’t just an America-centric bias on WSJ’s part, since there is lots of foreign coverage in their roundup; indeed the Middle East probably gets more than its fair share thanks to “if it bleeds, it leads”. For some reason they just missed the biggest countries. They also seem to have a blind spot for science and technology; they don’t mention a single scientific discovery, and only had two technology stories, on SpaceX catching a rocket and doing the first private spacewalk.

The SpaceX stories at least are genuinely important- the sort of thing that might show up in a history book in 50+ years, along with some of the stories on U.S. politics and the Russia-Ukraine war, but unlike most of the trivialities reported.

I welcome your pointers to better takes on what was important in 2024, or on what you consider to be the best news source today.

Bonkers BBC Weather App: 18,995 mile/hour Winds in U.K.

Scott BuchananLeave a comment

Last week as the U.S. National Weather Service was warning Florida residents of incoming 120 mile/hour winds, folks in the U.K were informed by the BBC weather app that the wind speed outside was – – wait for it – – 18,995 mph.  Hurricane force winds were displayed for many cities worldwide. Not to mention temperatures of around 400 F for Nottingham, New York, and Sydney.

The BBC of course issued retractions and apologies, blaming a third-party data supplier. Said supplier, meteorological forecasting company DTN has apologized for what it called a “technical error”.

I had my Brave browser do an AI search/summary on the topic, “What measures does the BBC take to prevent or mitigate the impact of third-party supplier issues on its weather forecasting app?”. Here is what it told me:

Based on the provided search results, the BBC takes the following measures to prevent or mitigate the impact of third-party supplier issues on its weather forecasting app:

  • Diversified data sources: The BBC uses multiple data suppliers, including MeteoGroup, to ensure that its forecasts are accurate and reliable. This diversification helps to reduce dependence on a single supplier and minimizes the impact of issues with a particular supplier.
  • Real-time updates: The BBC’s weather app provides real-time updates, which enable it to quickly respond to and correct any errors or inaccuracies caused by third-party supplier issues.
  • Error detection and correction: The BBC’s weather experts closely monitor forecasts and are able to detect and correct errors or inaccuracies caused by third-party supplier issues.
  • Technical redundancy: The BBC has implemented technical redundancy measures, such as backup systems and fail-safes, to minimize the impact of third-party supplier issues on its weather forecasting app.
  • Collaboration with other organizations: The BBC collaborates with other organizations, such as the Met Office and Scottish airline Loganair, to improve forecast accuracy and mitigate the impact of third-party supplier issues.
  • Investment in technology and AI: The BBC invests in advanced technology and artificial intelligence to improve the accuracy and reliability of its weather forecasts, reducing the reliance on third-party suppliers and minimizing the impact of issues.
  • Monitoring and maintenance: The BBC regularly monitors and maintains its weather forecasting app and underlying systems to identify and address potential issues before they become critical.

These measures demonstrate the BBC’s proactive approach to mitigating the impact of third-party supplier issues on its weather forecasting app, ensuring that users receive accurate and reliable forecasts.

That all sounds great in theory, but in practice something failed last week. I guess there is so far no fool-proof substitute for having a pair of human eyeballs look at material before it is flung out into cyberspace. But that might cost money.

Nobody is saying, but it would not surprise me if the BBC adds yet another layer of auto-checking to its robo-forecasts, to at least flag physically unrealistic numbers.

Many Impressive AI Demos Were Fakes

Scott Buchanan1 Comment

I recently ran across an article on the Seeking Alpha investing site with the provocative title “ AI: Fakes, False Promises And Frauds “, published by LRT Capital Management. Obviously, they think the new generative AI is being oversold. They cite a number of examples where demos of artificial general intelligence were apparently staged or faked.  I followed up on a few of these examples, and it does seem like this article is accurate. I will quote some excerpts here to give the flavor of their remarks.

In 2023, Google found itself facing significant pressure to develop an impressive innovation in the AI race. In response, they released Google Gemini, their answer to OpenAI’s ChatGPT. The unveiling of Gemini in December 2023 was met with a video showcasing its capabilities, particularly impressive in its ability to handle interactions across multiple modalities. This included listening to people talk, responding to queries, and analyzing and describing images, demonstrating what is known as multimodal AI. This breakthrough was widely celebrated. However, it has since been revealed that the video was, in fact, staged and that it does not represent the real capabilities of Google’s Gemini.

… OpenAI, the company behind the groundbreaking ChatGPT, has a history marked by dubious demos and overhyped promises. Its latest release, Chat GPT-4-o, boasted claims that it could score in the 90th percentile on the Unified Bar Exam. However, when researchers delved into this assertion, they discovered that ChatGPT did not perform as well as advertised.[10] In fact, OpenAI had manipulated the study, and when the results were independently replicated, ChatGPT scored on the 15th percentile of the Unified Bar Exam.

… Amazon has also joined the fray. Some of you might recall Amazon Go, its AI-powered shopping initiative that promised to let you grab items from a store and simply walk out, with cameras, machine learning algorithms, and AI capable of detecting what items you placed in your bag and then charging your Amazon account. Unfortunately, we recently learned that Amazon Go was also a fraud. The so-called AI turned out to be nothing more than thousands of workers in India working remotely, observing what users were doing because the computer AI models were failing.

… Facebook introduced an assistant, M, which was touted as AI-powered. It was later discovered that 70% of the requests were actually fulfilled by remote human workers. The cost of maintaining this program was so high that the company had to discontinue its assistant.

… If the question asked doesn’t conform to a previously known example ChatGPT will still produce and confidently explain its answer – even a wrong one.

For instance, the answer to “how many rocks should I eat” was:

…Proponents of AI and large language models contend that while some of these demos may be fake, the overall quality of AI systems is continually improving. Unfortunately, I must share some disheartening news: the performance of large language models seems to be reaching a plateau. This is in stark contrast to the significant advancements made by OpenAI’s ChatGPT, between its second iteration (GPT-2), and the newer GPT-3 – that was a meaningful improvement. Today, larger, more complex, and more expensive models are being developed, yet the improvements they offer are minimal. Moreover, we are facing a significant challenge: the amount of data available for training these models is diminishing. The most advanced models are already being trained on all available internet data, necessitating an insatiable demand for even more data. There has been a proposal to generate synthetic data with AI models and use this data for training more robust models indefinitely. However, a recent study in Nature has revealed that such models trained on synthetic data often produce inaccurate and nonsensical responses, a phenomenon known as “Model Collapse.”

OK, enough of that. These authors have an interesting point of view, and the truth probably lies somewhere between their extreme skepticism and the breathless hype we have been hearing for the last two years. I would guess that the most practical near-term uses of AI may involve some more specific, behind the scenes data-mining for a business application, rather than exactly imitating the way a human would think.

Will the Huge Corporate Spending on AI Pay Off?

Scott Buchanan2 Comments

Last Tuesday I posted on the topic, “Tech Stocks Sag as Analysists Question How Much Money Firms Will Actually Make from AI”. Here I try to dig a little deeper into the question of whether there will be a reasonable return on the billions of dollars that tech firms are investing into this area.

Cloud providers like Microsoft, Amazon, and Google are building buying expensive GPU chips (mainly from Nvidia) and installing them in power-hungry data centers. This hardware is being cranked to train large language models on a world’s-worth of existing information. Will it pay off?

Obviously, we can dream up all sorts of applications for these large language models (LLMs), but the question is much potential downstream customers are willing to pay for these capabilities. I don’t have the capability for an expert appraisal, so I will just post some excerpts here.

Up until two months ago, it seemed there was little concern about the returns on this investment.  The only worry seemed to be not investing enough. This attitude was exemplified by Sundar Pichai of Alphabet (Google). During the Q2 earnings call, he was asked what the return on Gen AI investment capex would be. Instead of answering the question directly, he said:

I think the one way I think about it is when we go through a curve like this, the risk of under-investing is dramatically greater than the risk of over-investing for us here, even in scenarios where if it turns out that we are over investing. [my emphasis]

Part of the dynamic here is FOMO among the tech titans, as they compete for the internet search business:

The entire Gen AI capex boom started when Microsoft invested in OpenAI in late 2022 to directly challenge Google Search.

Naturally, Alphabet was forced to develop its own Gen AI LLM product to defend its core business – Search. Meta joined in the Gen AI capex race, together with Amazon, in fear of not being left out – which led to a massive Gen AI capex boom.

Nvidia has reportedly estimated that for every dollar spent on their GPU chips, “the big cloud service providers could generate $5 in GPU instant hosting over a span of four years. And API providers could generate seven bucks over that same timeframe.” Sounds like a great cornucopia for the big tech companies who are pouring tens of billions of dollars into this. What could possibly go wrong?

In late June, Goldman Sachs published a report titled, GEN AI: TOO MUCH SPEND,TOO LITTLE BENEFIT?.  This report included contributions from bulls and from bears. The leading Goldman skeptic is Jim Covello. He argues,

To earn an adequate return on the ~$1tn estimated cost of developing and running AI technology, it must be able to solve complex problems, which, he says, it isn’t built to do. He points out that truly life-changing inventions like the internet enabled low-cost solutions to disrupt high-cost solutions even in its infancy, unlike costly AI tech today. And he’s skeptical that AI’s costs will ever decline enough to make automating a large share of tasks affordable given the high starting point as well as the complexity of building critical inputs—like GPU chips—which may prevent competition. He’s also doubtful that AI will boost the valuation of companies that use the tech, as any efficiency gains would likely be competed away, and the path to actually boosting revenues is unclear.

MIT’s Daron Acemoglu is likewise skeptical:  He estimates that only a quarter of AI-exposed tasks will be cost-effective to automate within the next 10 years, implying that AI will impact less than 5% of all tasks. And he doesn’t take much comfort from history that shows technologies improving and becoming less costly over time, arguing that AI model advances likely won’t occur nearly as quickly—or be nearly as impressive—as many believe. He also questions whether AI adoption will create new tasks and products, saying these impacts are “not a law of nature.” So, he forecasts AI will increase US productivity by only 0.5% and GDP growth by only 0.9% cumulatively over the next decade.

Goldman economist Joseph Briggs is more optimistic:  He estimates that gen AI will ultimately automate 25% of all work tasks and raise US productivity by 9% and GDP growth by 6.1% cumulatively over the next decade. While Briggs acknowledges that automating many AI-exposed tasks isn’t cost-effective today, he argues that the large potential for cost savings and likelihood that costs will decline over the long run—as is often, if not always, the case with new technologies—should eventually lead to more AI automation. And, unlike Acemoglu, Briggs incorporates both the potential for labor reallocation and new task creation into his productivity estimates, consistent with the strong and long historical record of technological innovation driving new opportunities.

The Goldman report also cautioned that the U.S. and European power grids may not be prepared for the major extra power needed to run the new data centers.

Perhaps the earliest major cautionary voice was that of Sequoia’s David Cahn. Sequoia is a major venture capital firm. In September, 2023 Cahn offered a simple calculation estimating that for each dollar spent on (Nvidia) GPUs, and another dollar (mainly electricity) would need be spent by the cloud vendor in running the data center. To make this economical, the cloud vendor would need to pull in a total of about $4.00 in revenue. If vendors are installing roughly $50 billion in GPUs this year, then they need to pull in some $200 billion in revenues. But the projected AI revenues from Microsoft, Amazon, Google, etc., etc. were less than half that amount, leaving (as of Sept 2023) a $125 billion dollar shortfall.

As he put it, “During historical technology cycles, overbuilding of infrastructure has often incinerated capital, while at the same time unleashing future innovation by bringing down the marginal cost of new product development. We expect this pattern will repeat itself in AI.” This can be good for some of the end users, but not so good for the big tech firms rushing to spend here.

In his June, 2024 update, Cahn notes that now Nvidia yearly sales look to be more like $150 billion, which in turn requires the cloud vendors to pull in some  $600 billion in added revenues to make this spending worthwhile. Thus, the $125 billion shortfall is now more like a $500 billion (half a trillion!) shortfall. He notes further that the rapid improvement in chip power means that the value of those expensive chips being installed in 2024 will be a lot lower in 2025.

And here is a random cynical comment on a Seeking Alpha article: It was the perfect combination of years of Hollywood science fiction setting the table with regard to artificial intelligence and investors looking for something to replace the bitcoin and metaverse hype. So when ChatGPT put out answers that sounded human, people let their imaginations run wild. The fact that it consumes an incredible amount of processing power, that there is no actual artificial intelligence there, it cannot distinguish between truth and misinformation, and also no ROI other than the initial insane burst of chip sales – well, here we are and R2-D2 and C3PO are not reporting to work as promised.

All this makes a case that the huge spends by Microsoft, Amazon, Google, and the like may not pay off as hoped. Their share prices have steadily levitated since January 2023 due to the AI hype, and indeed have been almost entirely responsible for the rise in the overall S&P 500 index, but their prices have all cratered in the past month. Whether or not these tech titans make money here, it seems likely that Nvidia (selling picks and shovels to the gold miners) will continue to mint money. Also, some of the final end users of Gen AI will surely find lucrative applications. I wish I knew how to pick the winners from the losers here.

For instance, the software service company ServiceNow is finding value in Gen AI. According to Morgan Stanley analyst Keith Weiss, “Gen AI momentum is real and continues to build. Management noted that net-new ACV for the Pro Plus edition (the SKU that incorporates ServiceNow’s Gen AI capabilities) doubled [quarter-over-quarter] with Pro Plus delivering 11 deals over $1M including two deals over $5M. Furthermore, Pro Plus realized a 30% price uplift and average deal sizes are up over 3x versus comparable deals during the Pro adoption cycle.”