We have just used the
@Nvidia H100 onboard Starcloud-1 to train the first LLM in space! We trained the nano-GPT model from Andrej
@Karpathy on the complete works of Shakespeare and successfully ran inference on it. — Read More
Tag Archives: Nvidia
Touching the Elephant – TPUs
There is mythological reverence for Google’s Tensor Processing Unit. While the world presently watches NVIDIA’s gravity drag more companies into its orbit, there sits Google, imperial and singular. Lots of companies participate in the “Cambrian-style explosion of new-interesting accelerators”[14] – Groq, Amazon, and Tenstorrent come to mind – but the TPU is the original existence proof. NVIDIA should take credit for the reemergence of deep learning, but the GPU wasn’t designed with deep learning in mind. What’s strange is that the TPU isn’t a secret. This research is indebted to Google’s public chest-thumping, but the devices themselves have long been exclusive to Google’s datacenters. That is over a decade of work on a hardware system sequestered behind their walls. That the TPU is so well documented yet without a true counterpart creates a strange asymmetry. Google is well positioned in the AI race because of their decision over a decade ago to build a hardware accelerator. It is because of the TPU. — Read More
Google’s Ironwood TPUs represent a bigger threat than Nvidia would have you believe
Look out, Jensen! With its TPUs, Google has shown time and time again that it’s not the size of your accelerators that matters but how efficiently you can scale them in production.
Now with its latest generation of Ironwood accelerators slated for general availability in the coming weeks, the Chocolate Factory not only has scale on its side but a tensor processing unit (TPU) with the grunt to give Nvidia’s Blackwell behemoths a run for their money. — Read More
Meet Project Suncatcher, Google’s plan to put AI data centers in space
The tech industry is on a tear, building data centers for AI as quickly as they can buy up the land. The sky-high energy costs and logistical headaches of managing all those data centers have prompted interest in space-based infrastructure. Moguls like Jeff Bezos and Elon Musk have mused about putting GPUs in space, and now Google confirms it’s working on its own version of the technology. The company’s latest “moonshot” is known as Project Suncatcher, and if all goes as planned, Google hopes it will lead to scalable networks of orbiting TPUs.
The space around Earth has changed a lot in the last few years. A new generation of satellite constellations like Starlink has shown it’s feasible to relay Internet communication via orbital systems. Deploying high-performance AI accelerators in space along similar lines would be a boon to the industry’s never-ending build-out. Google notes that space may be “the best place to scale AI compute.”
Google’s vision for scalable orbiting data centers relies on solar-powered satellites with free-space optical links connecting the nodes into a distributed network. Naturally, there are numerous engineering challenges to solve before Project Suncatcher is real. As a reference, Google points to the long road from its first moonshot self-driving cars 15 years ago to the Waymo vehicles that are almost fully autonomous today. — Read More
THERMODYNAMIC COMPUTING: FROM ZERO TO ONE
Three years ago, Extropic made the bet that energy would become the limiting factor for AI scaling.
We were right.[1]
Scaling AI will require a major breakthrough in either energy production, or the energy efficiency of AI hardware and algorithms.
We are proud to unveil our breakthrough AI algorithms and hardware, which can run generative AI workloads using radically less energy than deep learning algorithms running on GPUs. — Read More
High Stakes in the U.S.-China AI Chip Race
China’s decision to use domestic AI chips instead of buying from Nvidia signals progress — and newfound confidence — in its own semiconductor industry.
st week, China barred its major tech companies from buying Nvidia chips. This move received only modest attention in the media, but has implications far beyond what’s widely appreciated. Specifically, it signals that China has progressed sufficiently in semiconductors to break away from dependence on advanced chips designed in the U.S., the vast majority of which are manufactured in Taiwan. It also highlights the U.S. vulnerability to possible disruptions in Taiwan at a moment when China is becoming less vulnerable.
After the U.S. started restricting AI chip sales to China, China dramatically ramped up its semiconductor research and investment to move toward self-sufficiency. These efforts are starting to bear fruit, and China’s willingness to cut off Nvidia is a strong sign of its faith in its domestic capabilities. — Read More
AMD reveals next-generation AI chips with OpenAI CEO Sam Altman
AMD on Thursday unveiled new details about its next-generation AI chips, the Instinct MI400 series, that will ship next year. CEO Lisa Su unveiled the chips at a launch event in San Jose, California.
The chips will be able to be used as part of a “rack-scale” system, AMD said. That’s important for customers that want “hyperscale” clusters of AI computers that can span entire data centers.
OpenAI CEO Sam Altman appeared on stage on with Su and said his company would use the AMD chips. “It’s gonna be an amazing thing,” Altman said. — Read More
NVIDIA to Manufacture American-Made AI Supercomputers in US for First Time
NVIDIA is working with its manufacturing partners to design and build factories that, for the first time, will produce NVIDIA AI supercomputers entirely in the U.S.
Together with leading manufacturing partners, the company has commissioned more than a million square feet of manufacturing space to build and test NVIDIA Blackwell chips in Arizona and AI supercomputers in Texas. — Read More
Accelerate Generalist Humanoid Robot Development with NVIDIA Isaac GR00T N1
Humanoid robots are designed to adapt to human workspaces, tackling repetitive or demanding tasks. However, creating general-purpose humanoid robots for real-world tasks and unpredictable environments is challenging. Each of these tasks often requires a dedicated AI model. Training these models from scratch for every new task and environment is a laborious process due to the need for vast task-specific data, high computational cost, and limited generalization.
NVIDIA Isaac GR00T helps tackle these challenges and accelerates general-purpose humanoid robot development by providing you with open-source SimReady data, simulation frameworks such as NVIDIA Isaac Sim and Isaac Lab, synthetic data blueprints, and pretrained foundation models. — Read More
AI Promise and Chip Precariousness
Yesterday Anthropic released Claude Sonnet 3.7; Dylan Patel had the joke of the day about Anthropic’s seeming aversion to the number “4”, which means “die” in Chinese.
Jokes aside, the correction on this post by Ethan Mollick suggests that Anthropic did not increment the main version number because Sonnet 3.7 is still in the GPT-4 class of models as far as compute is concerned. I love Mollick’s work, but reject his neutral naming scheme: whoever gets to a generation first deserves the honor of the name. In other words, if Gen2 models are GPT-4 class, then Gen3 models are Grok 3 class.
And, whereas Sonnet 3.7 is an evolution of Sonnet 3.5’s fascinating mixture of personality and coding prowess, likely a result of some Anthropic special sauce in post-training, Grok 3 feels like a model that is the result of a step-order increase in compute capacity, with a much lighter layer of reinforcement learning with human feedback (RLHF). Its answers are far more in-depth and detailed (model good!), but frequently becomes too verbose (RLHF lacking); it gets math problems right (model good!), but its explanations are harder to follow (RLHF lacking). It is also much more willing to generate forbidden content, from erotica to bomb recipes, while having on the surface the political sensibilities of Tumblr, with something more akin to 4chan under the surface if you prod. Grok 3, more than any model yet, feels like the distilled Internet; it’s my favorite so far. — Read More
Rick's Cafe AI 