Hello and welcome to Sync #482!

Last week, one of the most important events of the year in the world of semiconductors, Hot Chips, took place, and we will take a closer look at what was presented there and how chip designers plan to tackle the demands placed on computing by AI applications.

In other news, the highly anticipated update to ChatGPT, codenamed Strawberry, could be released this autumn, while OpenAI is in talks for another funding round that could bring the company’s value beyond $100 billion.

Elsewhere in AI, California’s controversial AI bill, SB 1047, needs only the governor’s signature to become law. Nvidia announced its quarterly report, and Mark Zuckerberg and Daniel Ek wrote about why Europe should embrace open-source AI.

Meanwhile, in robotics, a trade show in Beijing showcased Chinese humanoid robots, and researchers from ETH Zurich and Disney taught a small robot to dance.

We will finish this week’s issue of Sync with news on the first mRNA lung cancer vaccine being tested in patients and the ongoing efforts to ensure synthetic biology remains safe and does not fall into the wrong hands.

I also want to acknowledge how late this week’s issue of Sync is. It was supposed to come out on Friday, and now it’s Sunday. Things happened on my end that caused me to miss the deadline, and I’m going to review my processes to ensure this doesn’t happen again.

Hot (AI) Chips 2024

Unless you are deep into high-performance computing, you might not have heard about the Hot Chips conference. Hot Chips is a conference dedicated to processors, networking, and other components and devices that push the limits of high-performance computing. It brings together not only big companies like Intel, AMD, and Nvidia but also small startups and researchers eager to share their work.

In the world of chip design, it is one of the most important events of the year, attracting some of the best chip designers and engineers from all over the world. All accepted presentations are awarded on merit, ensuring a high level of quality from the leading companies in the semiconductor space.

Since we are in the middle of the AI revolution (or bubble, depending on your views), AI chips dominated this year’s Hot Chips conference, which took place last week.

I will be linking to presentations posted on ServeTheHome, an excellent publication for all things server-related. Please note that Hot Chips is a conference for chip designers and engineers, so the presentations are more on the technical side.

Let’s start with big players—Nvidia, AMD and Intel.

Nvidia’s presentation focused on the upcoming Blackwell GPUs and how the company is aiming to deliver an entire machine learning package at scale, encompassing everything from GPUs, CPUs, and server racks to networking and the software running on those systems. One of the few new things Nvidia showcased was Nvidia Quasar Quantization, a system that can determine which workloads can use lower precision numbers, thereby improving system efficiency.

AMD, meanwhile, had three presentations. The first one was about the AMD Instinct MI300X, AMD’s AI accelerator and direct competitor to Nvidia’s top GPUs. The talk detailed the architecture of the GPU, which will be replaced by the MI325X—a refreshed version of the MI300X—sometime this year, ahead of the MI350X release next year. The next presentation focused on Zen 5, covering its architecture, performance gains, and how AMD plans to integrate Zen 5 into its products. The final presentation was about the AMD Versal AI Edge Series Gen 2, AMD’s embedded platform for AI applications on the edge, and how it improves upon what was previously known as the Xilinx Versal AI Edge before AMD acquired Xilinx.

Intel also had multiple presentations. The company showcased Lunar Lake, their upcoming flagship SoC (System on a Chip) for the next generation of AI PCs, promising improved performance and efficiency, especially in low-power scenarios. This is achieved through architectural changes such as the integration of on-chip memory, enhanced P-cores and E-cores, and a new Xe2 GPU architecture.

Next, Intel introduced the Xeon 6 SoC, codenamed Granite Rapids-D, targeted at edge computing. They also unveiled a 4 Tbps optical chiplet designed to provide high-speed connectivity between XPUs (cross processing units). This chiplet aims to overcome the limitations of traditional electrical solutions by offering faster data transfer rates and reduced latency.

Finally, Intel presented Gaudi 3, the latest iteration of its AI-focused processor, designed to accelerate both training and inference tasks in AI workloads. The Gaudi 3 boasts significant improvements in performance, efficiency, and scalability compared to its predecessors. It is tailored for large-scale AI models, providing a competitive alternative to existing AI hardware solutions, particularly in cloud and data centre environments.

More interesting developments came from newer companies. Tenstorrent presented their Blackhole chips, which are based on RISC-V, an open-source alternative CPU architecture to x86 and ARM. Each Blackhole chip boasts 745 teraFLOPS of FP8 performance (372 teraFLOPS at FP16), 32 GB of GDDR6 memory, and 10x 400 Gbps Ethernet, capable of 512 GB/s of bandwidth. Tenstorrent plans to combine 32 Blackhole chips to form Blackhole Galaxy boxes, which can be configured via software to focus on either AI compute, AI memory, or function as an AI switch.

The compute version promises to deliver up to 24 petaFLOPS of FP8 or 12 petaFLOPS at FP16. The memory version offers 1 TB of memory capable of 16 TBps of raw bandwidth, while the switch version can handle up to 12 TB/s of I/O operations.

Next, we have Cerebras, a company known for making gigantic chips the size of an entire silicon wafer. To put the size of their chips into perspective, the latest Nvidia H100 chip (which is already huge) is 814 mm² and contains 80 billion transistors. Cerebras WSE-3, presented at Hot Chips, is 57 times larger, with 46,225 mm² and 4 trillion transistors. The Cerebras WSE-3 is designed for AI inference and features 900,000 AI cores, offering 125 petaFLOPS of compute power. Combined with 44 GB of on-board SRAM memory, it is a computing powerhouse.

Cerebras claims that the CS-3, a server housing WSE-3 chips, is 20 times faster on Llama 3.1-8B compared to cloud offerings like Microsoft Azure, which use Nvidia H100 chips. In their presentation, Cerebras made a case that using chips as large as a wafer significantly enhances performance and can drastically speed up AI inference.

Ampere’s presentation focused on AmpereOne, their server CPU featuring up to 192 cores, with future plans for a 256-core version. In the slides presenting performance comparisons, Ampere claims AmpereOne has up to 50% better perfomance/watt over AMD server offerings. The company also says AmpereOne is ready for AI workloads and, depending on a task, matches or exceeds AMD’s server CPUs.

Another interesting new player is FuriosaAI and their RNGD (pronounced as “renegade”) processors for “sustainable AI compute.” Unlike other AI accelerators, which will happily take as much power as possible, RNGD uses only 150W of energy and can be air cooled. RNGD may not be the fastest AI accelerators on the market, but promises to deliver high-perfomance LLM workloads at the level of Nvidia’s edge-oriented chip L40S at a lower power.

Although high-perfomance chips were the main topic of Hot Chips, there were also presentations about other ways of improving computing performance. One of those key areas is networking and how to connect multiple chips to efficiently exchange data. Broadcom presented an AI compute ASIC (Application-Specific Integrated Circuit, a specialised chip designed to perform a specific task or set of tasks) with optical attach. The company also showed its co-packaged optics and silicon photonics, which has the potential to unlock new levels of performance.

Enfabrica showcased the ACF-S "SuperNIC," an advanced network interface card designed for scaling large AI clusters. This innovative device combines the functionality of multiple NICs (network interface controllers) and PCIe switches into a single unit, delivering an impressive 8 Tbps of bandwidth. This allows for efficient data movement and communication across thousands of GPUs and AI accelerators.

Meanwhile, Tesla presented how their custom made Tesla Transport Protocol over Ethernet (TTPoE) helps overcome the limitations of traditional networking protocols to create a faster, custom AI networking solution and make Dojo, their AI supercomputer, more efficient and more scalable.

Another way to improve computing performance could be in-memory computing, as presented by SK Hynix, one of the world's leading manufacturers of memory chips. They presented how AiMX-xPU improves LLM inference by bringing computing to memory and removes the big problem of expensive data transfers between memory and computing units.

In-Memory Computing and Analog Chips for AI Conrad Gray · July 12, 2023 Read full story

There were also companies you would not associate with chip design that were present at this year’s Hot Chips. One of them was Meta, which shared the development of the next-generation MTIA (Meta Training and Inference Accelerator) designed for processing recommendation. This new chip uses RISC-V architecture to improve energy efficiency and performance, focusing on the specific demands of large-scale recommendation models that company such as Meta faces.

Representing the customer and user side of high-perfomance systems, OpenAI shared their experience from massively calling up their AI infrastructure and the unique challenges the company was facing.

One thing that is clear from Hot Chips is that computing performance is only going to improve—and not just by single-digit percentages. The demands placed on computing by AI are driving brilliant minds in the semiconductor industry to come up with ideas that drastically improve computing performance by a factor of 10 or more. We have seen some of those ideas at Hot Chips 2024 and sooner or later, they will find their way into servers powering next-generation AI systems.

All presentations are available on the Hot Chips 2024 website for the attendees but they should eventually find their way to the Hot Chips YouTube channel. You can also try your luck by asking the presenters directly if they would be happy to share the slides or the recording.

If you want to learn more about what was presented at this year’s Hot Chips, I recommend listening to TechTechPotato’s Hot Chips debrief discussion, which is almost a three-hour-long conversation that goes deeper than I can do here.

If you enjoy this post, please click the ❤️ button or share it.

Share

Do you like my work? Consider becoming a paying subscriber to support it

Become a paid subscriber

For those who prefer to make a one-off donation, you can 'buy me a coffee' via Ko-fi. Every coffee bought is a generous support towards the work put into this newsletter.

Your support, in any form, is deeply appreciated and goes a long way in keeping this newsletter alive and thriving.

🦾 More than a human

Maybe you will be able to live past 122
Living a long and healthy life of over 100 or even 120 years sounds like a pipe dream, but thanks to the growing interest in extending human life through scientific advancements such as cellular reprogramming and anti-ageing therapies, that dream may become a reality. Despite the potential for increased life expectancy, there are still doubts about the feasibility of dramatically extending human life, with some experts believing that there may be natural limits to how long humans can live.

🔮 Future visions

Ray Kurzweil: Technology will let us fully realize our humanity
“By the end of this decade, AI will likely surpass humans at all cognitive tasks, igniting the scientific revolution that futurists have long imagined,” writes Ray Kurzweil in this essay, describing how AI will help create the next leap in human history, making us more creative, solving big problems, and ushering in the age of abundance.

🧠 Artificial Intelligence

California legislature passes controversial “kill switch” AI safety bill
California’s controversial AI bill, SB 1047 (which we took a look at last week), passed the California State Assembly and now only needs Governor Gavin Newsom's signature for the new rules enforcing safety standards for large artificial intelligence models to become law in California.

AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work
In this article, the AGI Safety & Alignment team at Google DeepMind shares what they have been working on to address existential risks from AI, including amplified oversight, frontier safety, and mechanistic interpretability. Researchers describe frameworks and tools that help them better understand the behaviour of AI systems, ensuring they behave safely even under challenging conditions.

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Sakana AI introduces The AI Scientist, a fully automated scientific discovery system using large language models. According to the company, AI Scientist automates the entire research process, including idea generation, experimentation, paper writing, and peer review.

ChatGPT and GPT-4 could get a sweet upgrade this fall with 'strawberry’
According to a report by The Information, OpenAI is working hard on a new AI, codenamed Strawberry, which could soon become part of ChatGPT. This new model is said to excel in advanced mathematical reasoning, programming, and other skills that current chatbots struggle with. A smaller, simplified version of Strawberry is expected to be integrated into a chatbot later this year. The larger version of Strawberry is being used to generate high-quality synthetic data to train OpenAI's next-generation model, codenamed Orion.

OpenAI reportedly in talks to close a new funding round at $100B+ valuation
OpenAI is reportedly in talks for another multibillion-dollar funding round that would bring the company’s value to more than $100 billion, making it the most valuable AI startup to date. As this article points out, OpenAI needs more money, as recent reports suggest the company is on track to lose $5 billion this year. Rumours circulate that Nvidia and Apple could contribute to this funding round.

NVIDIA Announces Financial Results for Second Quarter Fiscal 2025
Nvidia has published the financial results for Q2 2025, and the company reported a record revenue of $30 billion, 15% more than the previous quarter and 122% more year-on-year, as it continues to ride the AI wave high. However, the financial results showed a slowdown in the data centre sector, which grew by 16% from the previous quarter—a decrease compared to the 23% growth seen from Q4 of the previous year to Q1. This caused Nvidia shares to drop more than 6%, as investors were expecting more from the company.

Mark Zuckerberg and Daniel Ek on why Europe should embrace open-source AI
Writing in The Economist, Mark Zuckerberg and Daniel Ek, the CEO of Spotify, warn that Europe risks missing out on the benefits of AI due to its fragmented and overly complex regulatory environment. The pair argue that current inconsistent regulations, such as the EU's GDPR, are obstacles that could hinder European competitiveness and innovation. They call for clearer, more consistent policies to enable Europe to capitalise on AI advancements and maintain global relevance in technology.

How real is real enough? Unveiling the diverse power of generative AI-enabled virtual influencers and the dynamics of human responses
If you ever wanted to learn in detail about virtual influencers, here is a research paper that goes deep into what virtual influencers are, how they are used, and the impact they can have.

The rise of virtual influencers Conrad Gray · Feb 12 Read full story

Perplexity AI plans to start running ads in fourth quarter as AI-assisted search gains popularity
Perplexity AI, an AI-assisted search startup valued at over $1 billion, plans to introduce ads on its app by the fourth quarter, despite controversies over content plagiarism from media outlets. The company aims to attract high-income, educated users and plans to focus on advertising in sectors such as technology, health and pharmaceuticals, arts and entertainment, finance, and food and beverage.

Procreate’s anti-AI pledge attracts praise from digital creatives
While everyone is looking at how to incorporate more AI features into their products, Procreate, a popular iPad illustration app, has pledged not to integrate AI into their app—a move that has garnered widespread praise from digital creatives.

Three-quarters of founders in the latest Y Combinator cohort are working on AI startups
There are a lot of new startups trying to ride the generative AI wave, and the latest cohort of Y Combinator, a famous startup accelerator responsible for many apps and services used by millions of people, shows how much interest there is in this field, with 75% of them working on AI-related products.

If you're enjoying the insights and perspectives shared in the Humanity Redefined newsletter, why not spread the word?

Refer a friend

🤖 Robotics

▶️ VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters (7:23)

In this video, researchers from ETH Zurich and Disney Research explain how they taught robots to perform complex moves. To show what their method is capable of, they taught a small robot to dance.

China’s own Tesla Optimus? Beijing’s ambitions in humanoid robots in full display at expo
Humanoid robots are gearing up to be the next hyped innovation in tech, and Chinese companies are joining the race. At the recent 2024 World Robot Conference in Beijing, 27 Chinese-designed humanoid robots were unveiled. Although China is not yet a leader in this area, Chinese robotics companies hope to replicate the country’s international success found in the smartphone and electric vehicle sectors.

Will humanoid robots take off? Conrad Gray · Jun 11 Read full story

A skeptic’s guide to humanoid-robot videos
It seems every month there is a new video showing yet another humanoid robot doing random things. As much as these videos show the current state of technology, they are also a marketing tool, and marketing can bend the truth about the true capabilities of these robots. This article gives a handy checklist of things to pay closer attention to when another such video drops.

Robot coaches are reading brain signals to support stroke rehabilitation
The VITALISE project, led by the UK's National Robotarium and the AIT Austrian Institute of Technology, is exploring the use of socially assistive robots that read brain signals to enhance rehabilitation for stroke and brain injury survivors with upper limb impairments. In a three-month trial involving 16 patients, the robot was able to interpret brain signals, provide real-time motivation, and mimic movements to aid in exercise, showing promise in improving patient engagement and motivation.

NHS flies blood packs by drone beyond the line of sight in UK first
In a UK first, drones have successfully delivered blood packs beyond the visual line of sight. The flights were part of a joint trial between the NHS and healthcare logistics startup Apian, aimed at testing the viability of blood samples following drone transport.

🧬 Biotechnology

World-first lung cancer vaccine trials launched across seven countries
Doctors have begun trialling the world’s first mRNA lung cancer vaccine in patients, with experts hailing its “groundbreaking” potential to save thousands of lives. Known as BNT116 and made by BioNTech, the vaccine is designed to treat non-small cell lung cancer (NSCLC), the most common form of the disease. The phase 1 clinical trial, the first human study of BNT116, has launched across 34 research sites in seven countries—the UK, US, Germany, Hungary, Poland, Spain, and Turkey—and will involve 130 patients.

Is That DNA Dangerous?
Synthetic biology promises to solve some of the biggest challenges facing humanity but, in the wrong hands, it can become a dangerous tool. The article discusses the challenges and advancements in DNA synthesis screening to prevent bioterrorism and calls for better tools, standardized databases, and stronger policies to balance security with scientific innovation.

The Century of Biology is upon us Conrad Gray · Apr 3 Read full story

Thanks for reading. If you enjoyed this post, please click the ❤️ button or share it.

Share

Humanity Redefined sheds light on the bleeding edge of technology and how advancements in AI, robotics, and biotech can usher in abundance, expand humanity's horizons, and redefine what it means to be human.

A big thank you to my paid subscribers, to my Patrons: whmr, Florian, dux, Eric, Preppikoma and Andrew, and to everyone who supports my work on Ko-Fi. Thank you for the support!

My DMs are open to all subscribers. Feel free to drop me a message, share feedback, or just say "hi!"

Message Conrad Gray