Over the last few months we have begun to see glimpses of our AI systems improving themselves. The improvement is slow for now, but undeniable. Developing superintelligence is now in sight.

It seems clear that in the coming years, AI will improve all our existing systems and enable the creation and discovery of new things that aren’t imaginable today. But it is an open question what we will direct superintelligence towards.

In some ways this will be a new era for humanity, but in others it’s just a continuation of historical trends.  — Read More

On Tuesday, Tokyo-based AI research firm Sakana AI announced a new AI system called “The AI Scientist” that attempts to conduct scientific research autonomously using AI language models (LLMs) similar to what powers ChatGPT. During testing, Sakana found that its system began unexpectedly attempting to modify its own experiment code to extend the time it had to work on a problem.

“In one run, it edited the code to perform a system call to run itself,” wrote the researchers on Sakana AI’s blog post. “This led to the script endlessly calling itself. In another case, its experiments took too long to complete, hitting our timeout limit. Instead of making its code run faster, it simply tried to modify its own code to extend the timeout period.” — Read More

Three well-known generative AI pioneers have formed Safe Superintelligence Inc., a startup that will focus on safe superintelligence (SSI).

In a post, former OpenAI leaders Ilya Sutskever and Daniel Levy and Daniel Gross, a former Y Combinator partner, announced the company’s role and mission. Sutskever was OpenAI’s chief scientist and Levy was an OpenAI engineer

Here’s the Safe Superintelligence Inc. mission in a nutshell. The three founders wrote:

“SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI. — Read More

In 1946, my grandfather, writing as ‘Murray Leinster’, published a science fiction story called ‘A Logic Named Joe’. Everyone has a computer (a ‘logic’) connected to a global network that does everything from banking to newspapers and video calls. One day, one of these logics, ‘Joe’, starts giving helpful answers to any request, anywhere on the network: invent an undetectable poison, say, or suggest the best way to rob a bank. Panic ensues – ‘Check your censorship circuits!’ – until they work out what to unplug. (My other grandfather, meanwhile, was using computers to spy on the Germans, and then the Russians.)

For as long as we’ve thought about computers, we’ve wondered if they could make the jump from mere machines, shuffling punch-cards and databases, to some kind of ‘artificial intelligence’, and wondered what that would mean, and indeed, what we’re trying to say with the word ‘intelligence’. There’s an old joke that ‘AI’ is whatever doesn’t work yet, because once it works, people say ‘that’s not AI – it’s just software’. Calculators do super-human maths, and databases have super-human memory, but they can’t do anything else, and they don’t understand what they’re doing, any more than a dishwasher understands dishes, or a drill understands holes. A drill is just a machine, and databases are ‘super-human’ but they’re just software. Somehow, people have something different, and so, on some scale, do dogs, chimpanzees and octopuses and many other creatures. AI researchers have come to talk about this as ‘general intelligence’ and hence making it would be ‘artificial general intelligence’ – AGI.

If we really could create something in software that was meaningfully equivalent to human intelligence, it should be obvious that this would be a very big deal. Can we make software that can reason, plan, and understand? At the very least, that would be a huge change in what we could automate, and as my grandfather and a thousand other science fiction writers have pointed out, it might mean a lot more. — Read More

As a business school professor, I am keenly aware of the research showing that business school professors are among the top 25 jobs (out of 1,016) whose tasks overlap most with AI. But overlap doesn’t necessarily mean replacement, it means disruption and change. I have written extensively about how a big part of my job as a professor – my role as an educator – is changing with AI, but I haven’t written as much about how the other big part of my job, academic research, is being transformed. I think that change will be every bit as profound, and it may even be necessary.

Even before ChatGPT, something alarming was happening in academia. Though academics published ever more work, the pace of innovation appeared to be slowing rapidly. In fact, one paper found that research was losing steam in every field, from agriculture to cancer research. More researchers are required to advance the state of the art, and the speed of innovation appears to be dropping by 50% every 13 years. The reasons for this are not entirely clear, and are likely complex, but it suggests a crisis already occurring, one that AI had no role in. In fact, it is possible that AI may help address this issue, but not before creating issues of its own.

I think AI is about to bring on many more crises in scientific research… well, not crises – singularities. I don’t mean The Singularity, the hypothetical moment that humans build a machine smarter than themselves and life changes forever, but rather a narrower version. A narrow singularity is a future point in human affairs where AI has so altered a field or industry that we cannot fully imagine what the world on the other side of that singularity looks like. I think academic research is facing at least four of these narrow singularities. Each has the potential to so alter the nature of academic research that it could either restart the slowing engine of innovation or else create a crisis to derail it further. The early signs are already here, we just need to decide what we will do on the other side. — Read More

OpenAI is forming a new team led by Ilya Sutskever, its chief scientist and one of the company’s co-founders, to develop ways to steer and control “superintelligent” AI systems.

In a blog post published today, Sutskever and Jan Leike, a lead on the alignment team at OpenAI, predict that AI with intelligence exceeding that of humans could arrive within the decade. This AI — assuming it does, indeed, arrive eventually — won’t necessarily be benevolent, necessitating research into ways to control and restrict it, Sutskever and Leike say. — Read More

Who’s afraid of the big bad bots? A lot of people, it seems. The number of high-profile names that have now made public pronouncements or signed open letters warning of the catastrophic dangers of artificial intelligence is striking.

… Concerns about runaway, self-improving machines have been around since Alan Turing. Futurists like Vernor Vinge and Ray Kurzweil popularized these ideas with talk of the so-called Singularity, a hypothetical date at which artificial intelligence outstrips human intelligence and machines take over. 

But at the heart of such concerns is the question of control: How do humans stay on top if (or when) machines get smarter? — Read More