The Weasel is fond of saying that steam engines come at steam engine time. So, we need a combination of technology (metallurgy having advanced to the point that you can make a boiler that won't burst under pressure), a killer app (pumping water out of mines) and a set of technoeconomic innovations that make it easier to bring together clients, entrepreneurs, inventors, investors and workers. A fourth requirement would be the set of inventions related directly to how the steam engine works.
The thing is that people have been using cannons in Europe since the C14th, so was it really advances in metallurgy that made steam engines practical at that epoch? People have been mining tin in Cornwall for thousands of years, so was there really a sudden need for pumping water out with steam engines? There were certainly a lot of social and economic innovations in the late C17th/early !8th (central banks, stock markets, insurance, limited liability companies, etc, etc), but it is not as though people weren't doing large, medium and small scale commercial and state enterprises for hundreds or thousands of years before that.A technology has intricate as printing became widespread from the mid-C15th, shipyards were churning out ships, people were toiling away in mines and building water and windmills as well as elaborate clocks, so it is not obvious that steam engine time, sat the least couldn't have been at least a century or two earlier.
As I said on Bluesky yesterday, AI-1956, the Dartmouth summer workshop programme, symbolic AI, GOFAI, the Royal Route to Human-Level Intelligence, turned out to be a bust. But that wasn't obvious at the time. The term AI was coined to avoid having to use the C-word, cybernetics for the workshop, because then they would have to have invited Norbert Wiener. But it's hardly the case that there were lots of other approaches at the time - neural nets, Ashby's Design for a Brain, Rosenblatt's Perceptron - that were potentially promising. 1950s computers were incredibly feeble compared to modern ones, but it didn't seem like to people at the time and you only need ~8 kB to start doing really interesting and useful things. It wasn't obvious in 1956 that intelligence couldn't be achieved through the combination of a few algorithms and heuristics. After all, no-one had in a position to try to build a brain up until then.
So, Lighthill was right! There was no point doing AI research in the 1970s because you just didn't have the scale - compute and data - to do anything useful. Of course, Lighthill wouldn't have thought of it like that, and it might be that a lot of what was going on was academic knife grinding, but, of course, to first approximation, it was true. For decades, AI was 20 or 50 years off because for decades, no-one had the scale to do anything useful It's still possible that AGI is 20 or 50 years off. That seems unlikely in an age where we have robust quasi-AGI already (by robust, I mean that you can ask Gemini 2.5 Pro anything in natural language and it will usually have a pretty good go at answering your question or doing the requested task, even if it doesn't always get it 100% right, that's better than 95% of co-workers in my bitter professional experienc; this capability is probably the most extraordinary thing about FMs - we complain about hallucinations or incomplete tasks, rightly so, but the shift from the brittleness of traditional software engineering is still astonishing), but there could be many gotchas yet and, of course, we probably want hybrid systems that incorporate reinforcement learning and symbolic AI too as well as improved capacity to generalise from small samples and possibly not train with backprop and gradient descent because the brain doesn't. Let a thousand philosophies bloom.
Noam Brown has achieved many extraordinary things - solved poker, systems that play chess like humans, a system for playing Diplomacy - and said at the recent NVIDIA conference that we could have got here a couple of decades earlier. AGI time could have been in the late 2000s rather than the late 2020s! I want to dig deeper his thinking, but it seems to be that more thinking time would help with reasoning models. FMs respond typically very quickly, but what if they thought about a problem for hours instead? But obviously you could go back to the 1970s, as the Weasel suggested to me, to ICL in 1973, and design a programme that would get us to AGI sooner. We will talk about this more later, but you would build neural nets trained with backprop and gradient descent. This could have been done in 1973. You could use an ICL Distributed Array Processor,do lots of OCR applications, create a big training corpus, do self-supervised learning, get to advanced architectures like transformers, mixture of experts, thinking time. But you probably also need a couple of breakthroughs, such as more efficient learning. Get the internet earlier and maybe you can have enough data two decades earlier, even if you raw compute isn't much or any faster. Obviously, specialised processors, like GPUs and neuromorphic chips. It doesn't seem to unreasonable that we could have had systems a bit like modern FMs in 2005. But then hindsight is 20/20.Steam engines in 1580? AGI in 2007? Combine the two. We could have the Singularity in about 1809. There's a novel in that.
Add comment
Comments