He co-authored the 155-page research paper, Sparks of Artificial General Intelligence: Early Experiments with GPT-4, published on April 13, 2023.

He was instrumental in developing the Microsoft’s Phi series of SLMs, efficient AI models optimized for edge devices like smartphones and laptops. The first Phi model was introduced in the paper Textbooks Are All You Need, which made waves and sparked big ideas on efficient AI – quality data, less compute, same power!

Developing Efficient AI Models for Hardware Integration: Drawing on his SLM expertise, Bubeck may work on compact AI models optimized for OpenAI's new hardware, ensuring peak performance on devices with limited resources.

Enhancing On-Device AI Capabilities: He could contribute to advancing AI features that function directly on consumer devices, decreasing reliance on cloud computing and improving user privacy.

Collaborating on Custom AI Chip Development: With OpenAI’s partnerships with Broadcom and TSMC to develop custom AI chips, Bubeck's insight could help create models tailored for these chips, boosting both efficiency and performance.

How models are optimized (through architectural design, training efficiency, and compression).

Which constraints are prioritized (e.g., compute, memory, energy) based on the intended application environment and deployment needs.

Osmo Labs Digitized Smell
“A fresh summer plum was the first fruit and scent to be fully digitized and reprinted with no human intervention.” →Alex Wiltschko’s twitter

Decart & Etched introduced Oasis: A Fully AI-Generated Game World
It’s the first fully playable, real-time, open-world AI game, revolutionizing gaming with AI-generated experiences →Oasis’s GitHub

Waymo | EMMA: End-to-end Multimodal Driving Model, using Google’s Gemini, combines sensor and language data, excelling in trajectory prediction and object detection. Short-term memory and lack of LiDAR remain limitations →Waymo’s blog

Amazon | AWS Pits Q Developer Against Copilot. Why it might be good? It’s backed by Claude 3.5 – usual programmers’ choice →Amazon’s blog

Google

• Gemini’s “Grounding with Google Search” lets apps use live data, improving factual accuracy and trustworthiness →Google’s blog

• Big Sleep, Google’s AI tool, found critical flaws in SQLite, showcasing AI’s potential to detect complex vulnerabilities in software →Project Zero’s blog

• Google’s new “Learn About” tool turns any search query into a structured, interactive learning experience, powered by Gemini →Learn About playground

A16z & Microsoft: Teaming Up for AI’s Future. They rally for a balanced AI landscape where both startups and giants can thrive. Their pitch: open-source AI, shared data pools, and policies to empower U.S. innovation – garage dreamers and corporate titans alike →Microsoft’s blog

Anthropic’s Case for Targeted AI Regulation. Anthropic urges swift, focused AI regulation to curb potential risks. Their proposal: adaptive safety policies to keep up with model advancements, aiming for minimal red tape and maximum protection →Anthropic’s blog

The Present Future: AI's Impact Long Before Superintelligence by Ethan Mollick

Why I build open language models – an important read from Nathan Lambert

Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models

Researchers at Stellenbosch University examined strategies for reducing hallucinations in LLMs through prompt engineering and external tool integration. Testing approaches like self-consistency (SC) and Chain-of-Thought (CoT) on math and trivia tasks, they found SC best reduced hallucinations in reasoning tasks. Meanwhile, simpler prompts and avoiding tool complexity were more effective overall. Tool-using agents like ReAct increased hallucination rates, especially in less powerful LLMs, highlighting tool integration challenges →read the paper

Mind Your Step (By Step): CoT Can Reduce Performance on Tasks Where Thinking Makes Humans Worse

Princeton researchers identify tasks where chain-of-thought (CoT) reasoning degrades LLM performance. Testing across implicit statistical learning, visual recognition, and exception-based classification, CoT reduced accuracy by up to 36%. These reductions mirror human performance issues in similar tasks, linking specific cognitive constraints in humans to LLMs. However, CoT did not impair tasks like spatial reasoning or memory-intensive selection, highlighting cases where human-model constraints diverge →read the paper

Measuring Memorization Through Probabilistic Discoverable Extraction

Researchers from Google DeepMind and Boston University propose a probabilistic method to better measure memorization in LLMs. Current methods, focused on single-attempt extraction via greedy sampling, underestimate memorization. This study introduces the "(𝑛, 𝑝)-discoverable extraction" metric, capturing the probability of extracting memorized data across multiple attempts and sampling schemes →read the paper

Advancing Embodied AI Through Touch And Dexterity provides tools for enhanced touch perception and human-robot interaction.

A Large Recurrent Action Model: xLSTM Enables Fast Inference For Robotics Tasks introduces xLSTM for efficient, real-time robotics actions.

Counting Ability Of Large Language Models And Impact Of Tokenization explores tokenization’s effect on LLM counting abilities.

On Memorization Of Large Language Models In Logical Reasoning examines memorization vs. reasoning in LLM logical tasks.

What Happened In LLM Layers When Trained For Fast Vs. Slow Thinking studies gradient stability in LLMs trained for detailed thinking.

Language Models Can Self-Lengthen To Generate Long Texts introduces a method for extending LLM responses.

Hybrid Preferences: Learning To Route Instances For Human Vs. AI Feedback balances human and AI feedback for improved preference tasks.

LongReward: Improving Long-Context Large Language Models With AI Feedback uses AI feedback to enhance long-context LLMs.

Accelerating Direct Preference Optimization With Prefix Sharing reduces training redundancy in preference optimization.

BITSTACK: Fine-Grained Size Control For Compressed Large Language Models provides dynamic memory compression for LLMs.

NeuZip: Memory-Efficient Training And Inference With Dynamic Compression lowers memory use in training and inference for neural networks.

AgentStore: Scalable Integration Of Heterogeneous Agents integrates multiple agents for dynamic task automation.

OS-ATLAS: A Foundation Action Model For Generalist GUI Agents builds a GUI agent model for universal navigation.

Document Parsing Unveiled: Techniques And Prospects reviews document parsing for structured data extraction.

Neural Fields In Robotics: A Survey covers neural fields’ applications in 3D robotics.

Teaching Embodied RL Agents: Informativeness And Diversity Of Language studies language feedback’s impact on RL agent learning.

Personalization Of Large Language Models: A Survey details techniques for LLM personalization.

Survey Of UI Design And Interaction In Generative AI explores user interaction designs for generative AI.

COAT: Compressing Optimizer States And Activation For FP8 Training introduces FP8 training for memory-efficient optimization.

EoRA: Training-Free Compensation For Compressed LLM compensates for errors in compressed LLMs.

SocialGPT: Prompting LLMs For Social Relation Reasoning combines vision and language models for social recognition.

AutoKaggle: A Multi-Agent Framework For Autonomous Data Science
Automates data science competition tasks with multi-agents.

Stealing User Prompts From Mixture Of Experts identifies a security flaw in MoE models.

Zero-Shot Dense Retrieval With Embeddings From Relevance Feedback enhances zero-shot retrieval accuracy.

RARe: Retrieval Augmented Retrieval With In-Context Examples uses examples to boost retrieval performance.

Beyond Text: Optimizing RAG With Multimodal Inputs tests multimodal RAG in industrial settings.

FACT: Examining Iterative Context Rewriting For Multi-Fact Retrieval improves multi-fact retrieval through iterative context updates.

TOKENFORMER: Rethinking Transformer Scaling With Tokenized Parameters proposes token-parameter attention for efficient scaling.

Zipfian Whitening enhances word embeddings by adjusting for word frequency.