📝 Editorial 

The term ‘foundation models’ is becoming one of the hottest buzzwords in the machine learning (ML) lingo. Researchers from Stanford University originally coined the term to describe models that have been trained in large amounts of unlabeled data and can be fine-tuned to specific domains. Think about fine-tuning GPT-like models for domains such as law or science. Foundation models are shifting the ML development paradigm from creating brand-new models to fine-tuning large pretrained models.  

The efforts around foundation models are increasing remarkably fast. Stanford University created the Center for Research on Foundation Models (CRFM), a new initiative focused on studying best practices around foundation models. Just this week, Snorkel AI released Data-centric Foundation Model Development, a new series of addition to the Snorkel Flow platform to fine-tune and distill foundation models. Meta AI also unveiled details about MultiRay, their platform for running foundation models at scale. Finally, the CRFM team unveiled a new benchmark to facilitate the holistic evaluation of foundation models. Foundation models efforts are popping up everywhere, from large AI labs to innovative startups.  

Building by fine-tuning the new paradigm. The era of foundation models is definitely upon us! 

🔺🔻TheSequence Scope – our Sunday edition with the industry’s development overview – is free. To receive high-quality content about the most relevant developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#245: we start a new series about machine learning interpretability; discuss Manifold, an architecture for debugging ML models; explore Meta’s Captum, a framework for deep learning interpretability. 

Edge#246: we discuss OpenAI’s best practices that they used to mitigate risks while training DALL-E2

📌 Our LinkedIn account

In this uncertain times for Twitter, we’d like to introduce to you TheSequence’s LinkedIn account. We are building a unique resource and support system for all ML&AI aficionados. Let’s connect!

FOLLOW US on LinkedIn

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

HELM 

Stanford University published HELM, a benchmark for the holistic evaluation of foundation models →read more

MultiRay

Meta AI discusses MultiRay, the architecture used to power large foundation ML models  at scale across their different organizations →read more

Data Enrichment Practices

DeepMind published an insightful paper discussing human data collection best practices used in real-world ML scenarios →read more

MoE with Expert Routing  

Google Research published a research paper proposing a routing algorithm in mixture of experts (MoE) neural networks →read more

🤖 Cool AI Tech Releases

Data-centric Foundation Model Development 

Snorkel AI released Data-centric Foundation Model Development, a new set of capabilities in the Snorkel Flow platform to adapt large foundation models to domain-specific scenarios →read more

Data Cards Playbook 

Google Brain released Data Cards Playbook, a toolkit for transparency in ML datasets →read more

🛠 Real World ML 

Anomaly Detection in Prime Video 

Amazon Science discusses the ML techniques used for anomaly detection in their Prime Video application →read more

Einstein Search Answers 

Salesforce Research discusses the ML techniques powering Einstein Search Answers, a new search architecture for customer support →read more

Netflix Video Quality 

Netflix discusses the neural network techniques used for video encoding optimizations in the media giant →read more

💸 Money in AI

ML&AI&Data

• Chipmaker Astera Labs raised $150 million in a Series D funding round led by Fidelity Management & Research. Hiring globally.

• Data platform provider WekaIO raised $135 million in a Series D funding round led by Generation Investment Management. Hiring in San Francisco, US/Tel Aviv, Israel.

AI-powered

• Video and audio editor Descript raised a $50 million Series C fundraising round led by OpenAI Startup Fund. Hiring in San Francisco, US/Montreal and Quebec, Canada/remote.

• Video intelligence service Spot AI raised a $40 million Series B financing round led by Scale Venture Partners. Hiring remote and in Lehi, Utah, US.

• Video conference startup Owl Labs raised $25 million in a Series C investment round led by HP Tech Ventures. Hiring in Boston, US and remote.

• Language learning platform Speak raised $27 million in a Series B round led by OpenAI Startup Fund. Hiring in San Francisco, US/ Seoul, South Korea/Ljubljana, Slovenia.

• Contract intelligence platform Terzo raised $16 million in a Series A funding led by Align Ventures. Hiring in Los Angeles and Atlanta, US.