Next Week in The Sequence:

• Edge 329: The second installment of our series about fine-tuning covers the different types of fine-tuning methods, MIT’s research around multi-task prompt tuning and the Lamini platform.

• Edge 330: We discuss DSPy, a new language model programming framework that represents an alternative to LangChain and LlamaIndex.

You can subscribe below:

📝 Editorial: Do Amazon and Apple Have Any Moats in Generative AI?

In the world of technology investing, a "moat" refers to a competitive advantage that allows a given company to stay ahead of its competitors. When the term is applied to startups, "moats" are typically synonymous with defensibility. For incumbents, the term often refers to factors that prevent disruption from newcomers in the market. Many tech incumbents have established clear moats in generative AI:

• OpenAI's moat is undoubtedly the GPT-x family of models.

• Microsoft's moat lies in its alliance with OpenAI and extensive distribution channels such as Office, Windows, GitHub, and LinkedIn.

• NVIDIA's moat is its dominance in GPU computing.

• Google's moat encompasses the PaLM and DeepMind family of models, combined with its vast distribution through search.

• Meta's moat appears to be leaning towards open-source distribution.

The question of moats becomes trickier when discussing Amazon and Apple. Both tech giants haven't made significant inroads in creating proprietary foundation models, and their research in this domain pales in comparison to competitors. Amazon seems to be banking on partnerships with startups like Hugging Face to enable generative AI capabilities in AWS. However, this approach appears vulnerable and could potentially allow Microsoft and Google to gain more cloud market share. While Apple's distribution moat is robust due to the iPhone, their apparent lack of a clear vision and roadmap for generative AI is surprising.

Acquisitions are always a possibility for establishing a moat for tech incumbents. However, neither company has been particularly acquisitive lately, and the valuations for top generative AI companies are in the double-digit billions.

Generative AI evolves at a multi-exponential pace, making it incredibly challenging for companies of any size to regain a lost market. Both Apple and Amazon should be formidable competitors in the generative AI space, but for now, they appear to lack a defining moat.

🔎 ML Research

AlphaMissense

Google DeepMind published a paper detailing AlphaMissense, a new model able to catalog missense variant in proteins. A missense variant is a DNA letter substitution that creates a different amino acid within a protein —> Read more.

Agents

Researchers from AI Waves, Zhejiang University and ETH Zürich published a paper discussing Agents, a framework for building autonomous AI agents based on LLMs. Agents include capabilities such as planning, memory, tool usage, multi-agent communication and symbolic control →Read more.

Distilling Step-By-Step

Google Research published a paper detailing distilling step-by-step, a new distillation technique that seems to outperform both distillation and fine-tuning alternatives. The method is based on the idea of extracting reasoning steps from an LLM and use that to build a new dataset to train a more efficient model —> Read more.

Neural Graphical Models

Microsoft Research published a paper introducing neural graphical models. The core idea of this new technique is to learn to represent the probability function over a given domain traditional deep learning —> Read more.

Alexa LLM

Amazon Science published a detailed post about the generative AI techniques behind the recently announced Alexa LLM. The model combines language and audio with API calls to take actions —> Read more.

🤖 Cool AI Tech Releases

DALL-E 3 Research Preview

OpenAI unveiled some details about the research preview of DALL-E 3 —> Read more.

CoPilot for Windows

The latest Windows 11 update includes a version of Windows Copilot —> Read more.

🛠 Real World ML

Execution Traces at Meta

Meta unveils some details about Chakra execution traces, the system used to benchmark network and compute in their ML models —> Read more.

Label Noise at Walmart

Walmart Global Tech details some of the techniques used to prevent label noise in training datasets —> Read more.

📡AI Radar

• Salesforce acquired Airkit.ai to boost its AI customer service capabilities.

• Enterprise AI platform Writer raised $100 million in a Series B.

• AI-based enterprise search platform Pryon announced $100 million in new funding.

• YouTube unveiled new generative AI capabilities for video creation.

• AI-education startup Gizmo announced a $3.5 million seed round.

• AI data management startup Secoda raised $14 million in a new round.

• Generative AI education appp Sizle launched with an impressive set of capabilties.

• Amazon unveiled generative AI capabilities for FireTV.

• Roblox acquired AI voice chat startup Speechly.

• AI legal platform Darrow raised $35 million.

• Corti, an AI healthcare platform, raised $60 million in a new funding round.

• AI cybersecurity startup HiddenLayer raised $50 million for its MLSec platform.