📝 Editorial 

Large neural networks are the norm in modern deep learning, and they certainly require a lot of computation power. From that perspective, access to large computing resources has become a key requirement to advance AI research. Considering the progress in areas such as self-supervised learning of transformers, it’s hard not to be curious about what those AI models could accomplish with virtually limitless computation power. This week Meta (Facebook) AI announced a giant step towards that goal by unveiling a new AI supercomputer.   

Called the AI Research SuperCluster (RSC), Meta’s supercomputer was designed to train large language and computer vision models in trillions of parameters. RSC’s architecture is based on 760 NVIDIA DGX A100 systems as its compute nodes, for a total of 6,080 GPUs. The storage tier is equally impressive, having 175 petabytes of Pure Storage FlashArray, 46 petabytes of cache storage. These are just some of the details of a remarkable infrastructure design. RSC is up and running now, but there is a new phase of ongoing development. The new supercomputer is likely to play a pivotal role in augmenting Meta AI Research capabilities continuing the path towards massively large AI models.      

Share

🔺🔻 TheSequence Scope is our Sunday free digest. To receive high-quality educational content about the most relevant concepts, research papers, and developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#161: we start a new series about deep generative models; explore Optimus, a large generative model for language tasks; overview ART that uses generative models to protect neural networks.

Edge#162: we deep dive into CoreWeave, a modern cloud infrastructure.

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

InstructGPT 

OpenAI published a paper detailing InstructGPT, a transformer model that surpassed GPT-3 in following natural language instructions and is claimed to be less toxic →read more on OpenAI blog

LaMDA 

Google Research published a paper unveiling LaMDA, a model for high-quality, specialized dialogs →read more on Google Research blog

Robot Learning with Language and Video 

Stanford University published a paper detailing a method to use crowdsourced language descriptions of videos to train robots on different tasks →read more on Stanford University blog

MILAN

MIT researchers published a paper detailing MILAN, an interpretability technique that can attach language descriptions to components of a neural network →read more on MIT News

🤖 Cool AI Tech Releases

Meta AI SuperCluster 

Meta (Facebook) provided details about its AI Research SuperCluster, which is likely to become the fastest AI supercomputer in the world when it is fully built →read more on Meta AI blog

OpenAI Embeddings API 

OpenAI announced a new endpoint to its API that enables text and code embedding models →read more on OpenAI blog

🛠 Real World ML

Performance tests at Netflix

A very detailed blog about techniques that helps the Netflix team identify and fix regressions before they happen ->read more on Netflix tech blog 

DeepMind Podcast 

DeepMind announces the second season of their amazing podcast →read more on their blog

Big Data Cost Management at Uber 

Uber provided some details about the architecture and techniques used to reduce costs in their big data infrastructure →read more on Uber Eng blog

💸 Money in AI

• AI software solutions startup SparkCognition raised $123 Million in Series D Funding. Hiring remote.

• Enterprise AI startup InstaDeep raised $100 million in Series B financing led by Alpha Intelligence Capital and CDIB. They offer internships.

• Cloud data warehouse Firebolt raised $100 million in a Series C round led by Alkeon Capital. Hiring globally.

• Casual AI startup causaLens raised a $45 million Series A round led by Dorilton Ventures and Molten Ventures. Hiring in London/UK.

• Container optimization startup Slim.AI raised a $31 million Series A funding round co-led by Insight Partners and StepStone Group.

• AI solutions provider Vanti Analytics raised $16 million in Series A funding led by Insight Partners. Hiring in Tel Aviv/Israel.

• Computer vision startup Pimloc raised $7.5 million in a seed round led by Zetta Venture Partners. Hiring in London/UK.

• Metaphysic, the AI startup behind Tom Cruise deepfakes, raised $7.5 million in a seed round led by Section 32.

• Ethical AI governance platform Anch.ai raised a $2.1 million seed funding round led by Benhamou Global Ventures (BGV). They offer internships.

AI-powered

• Self-service automation provider NLX raised $5 million in seed funding led by Aquila Capital Partners. Hiring in Los Angeles, New York, and remote in the US.

• Managed service provider (MSP) platform SuperOps.ai raised $14 million in a Series A round of funding led by Addition. Hiring in Chennai/India.

• Medical records analytics platform DigitalOwl raised $20 million in a Series A funding round led by Insight Partners. Hiring in Israel and the US.

• Supply chain intelligence platform Verusen raised $25 Million in a Series B funding round led by Scale Venture Partners. Hiring in Atlanta/US.

• Supply chain intelligence platform o9 Solutions raised a $295 million round led by General Atlantic, its BeyondNetZero, and Generation Investment Management. Hiring globally.

• Symptom analysis provider Infermedica raised $30 million in a Series B funding round led by One Peak. Hiring mainly in Poland.

• Content moderation platform Spectrum Labs raised $32 million in a Series B round led by Intel Capital. They offer internships.

• Virtual agent technology Percept.AI was acquired by Atlassian for an undisclosed amount.