Weekly Big Future in AI, Week 4, Sept 4, 2024. GPT Next and Superintelligence News

by Brian Wang

There was big news this week in AI, AI startups, AI research and AI funding.

1. OpenAI’s Japan CEO reveals GPT-Next will be released this year, and its effective computational load is 100x greater than GPT-4

2. Two $125 Billion AI Training Centers, Starting at 0.5-1 GW and Scaling to 5-10GW

3. XAI 100k H100 AI data Center

4. AI Research

5. Major recent AI funding announcements including Safe Superintelligence

1. OpenAI GPT Next and Orion in 2025

GPT-Next will be released this year, and its effective computational load is 100x greater than GPT-4.

GPT-4 NEXT, which will be released this year, is expected to be trained using a miniature version of Strawberry (for better reasoning) with roughly the same computational resources as GPT-4, with an effective computational load 100 times greater.

The AI ​​model called 'GPT Next' that will be released in the future will evolve nearly 100 times based on past performance. Unlike traditional software, AI technology grows exponentially.

This 100 times increase does not refer to the scaling of computing resources, but rather to the effective computational volume + 2 OOMs, including improvements to the architecture and learning efficiency.

Orion, which has been in the spotlight recently, was trained for several months on the equivalent of 10k H100 compared to GPT-4, adding 10 times the computational resource scale, making it +3 OOMs, and is expected to be released sometime next year.

2. Two $125 billion AI Training Data Centers

   Two companies are looking to develop artificial intelligence (AI) data centers in North Dakota. Commissioner of Commerce Josh Teigen revealed during a Public Service Commission meeting in August that two companies had approached him and state Governor Doug Burgum about developing AI data centers.

   The data center projects would start between 500MW and 1GW projects, but could scale up to 5-10GW facilities eventually. The projects would cost up to $125 billion each. This should enable the 10,000 greater compute compared to what was used for GPT 4. This should enable GPT 6 class LLM (Large language models).

   The companies were not named but were described as having trillion-dollar market capitalizations. This could include businesses such as Nvidia, Amazon, Microsoft, Google, Meta, and Apple.

   Microsoft has previously been reported as looking into developing a major AI data center. In March 2024, Microsoft and OpenAI were allegedly investigating building a $100 billion supercomputer campus that could reach as much as 5GW of capacity.

   

   3. Elon Musk announced that XAI has completed its Colossus 100k Nvidia H100 chips. Musk said by the end of the year they will expand it to 200k Nvidia H100s.

4. AI Research

   Project Sid is the first simulations of 1000+ truly autonomous agents collaborating in a virtual world, w/ emergent economy, culture, religion, and government.

   At Altera, they organize agents at unprecedented scale to achieve what individual agents can’t We saw agents form a merchant hub, vote in a democracy, spread religions, & collect 5x more distinct items than ever before Play today: https://playlabs.altera.al/discover

   Here by truly autonomous Agents, we refer specifically to machines that are

   1) Long-term autonomous: Can progress for hours & days by themselves (not simple function-calling “Agent”)

   2) Organizational: Can collaborate with one another to achieve what individuals can’t (not just serving 1000 Agents to 1000 customers)

   3) Pro-Human Motivated: Have own deep & robust drives to support humans (not just instruction-following, obedient machines)

   4) Consciously Communicative: They can verbally express their conscious thoughts and feelings (not just language-conditioned robots)

   

   

5. Major AI Funding OpenAO and SSI

   Safe Superintelligence (SSI) (x-Open Ai Chief Scientist Ilya Sutakever’s company) got $1 billion in funding at a $5 billion valuation.

   OpenAI got $1 billion funding as well at a valuation of over $100 billion. OpenAI's latest funding round, led by Thrive Capital with a $1 billion investment, values the company at over $100 billion. This investment is expected to accelerate advancements in natural language processing and computer vision.

   Apple, Nvidia and Microsoft are in talks to invest with OpenAI in their recent funding round, valued at $100b.

   Nvidia has joined a $100 million-plus funding round for Tokyo startup Sakana AI, one of the larger investments the US chipmaker has made so far in Japan’s fledgling AI arena. Sakana AI has made an AI Scientist.