This blog post continues a list of articles, sources and papers on artificial intelligence that I started in 2023 (see here). The post became a bit too long and unwieldy to maintain in a single page, so have broken it out for the start of 2024. Enjoy!
Some AI Forecasts for 2024 (1)
Alberto Romano, who maintains a thoughtful blog called Algorithmic Bridge, observes that the benchmark LLM at the end of 2023 was GPT-4, which was built in 2022, and other models, including Gemini, Llama or Claud have not come close yet. He therefore speculates that we may be reaching the limit of what is possible with current technology. On the other hand, he muses at what is next from Meta, who have invested $20 billion on AI compute(!) while Sam Altman hints that GPT-5 will be a lot better than GPT-4.
Some AI Forecasts for 2024 (2)
In a paper for VentureBeat, Gary Grossman notes that AI is at the peak of the hype cycle. For example, you’d be hard-pressed to find a CES announcement that didn’t include some form of AI spin. Nevertheless, despite headwinds in the form of concerns about compute cost, environmental implications, training and data bias, security, copyright and hallucination, AI advancements are likely to continue to build upon one another. Grossman believes that Amara’s Law applies: “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”
How to Rank LLMs like Chess Grandmasters
With advances in large language models improving so quickly, it is quite a challenge to rank them against standard benchmark tests. The most useful ranking system is maintained by Hugging Face, who use an approach called Elo ranking, more commonly used to rank players in zero-sum competitive games such as chess to rank LLM-powered chatbots. The current ranking, dominated by GPT-4, is put together by crowdsourcing 200,000 preferences.
AI as a National Strategy
A couple of weeks ago, an interesting article in The Economist explored how different governments are pursuing AI industrial policy – a blend of scientific, investment and geopolitical strategy. Unsurprisingly, the USA dominates VC investment, whilst China and the United States are neck-and-neck when it comes to government investment, primarily for chip fabs. Cash-rich petrostates are ploughing money into GPUs and building their own models (such as Abu Dhabi’s Falcon LLM), though analysts are sceptical of the potential for government-sponsored AI models. (subscription required, though article may be downloaded for free)