Tuesday, Guest Post: Your infrastructure shouldn’t live in a “black box”

Wednesday, AI 101: What is DoRA, QLoRA, QDoRA?

Friday, Agentic Workflows series: The History of Agents

Retrieval-Augmented Generation (RAG) to process content from multiple sources.

Text-to-Speech (TTS): Generates the voices for the AI podcast hosts, creating a convincing conversational experience.

SoundStorm to generate realistic audio conversations. It converts scripts into natural dialogue with high-quality, engaging audio output.

Disfluency Injection to add human-like pauses, filler words, and natural speech patterns, making the dialogue sound more realistic.

Prompt Engineering to structure AI interactions and ensure the hosts maintain a natural, conversational tone.

Chunking strategies

Embedding and reranking model selection

Vector database comparisons

RAG architecture best practices

Testing and evaluation methods

California’s AI Bill Hits a Wall 

• Governor Gavin Newsom vetoed California’s landmark AI safety bill SB 1047, citing concerns over stifling innovation and prompting AI firms to relocate. The bill, aimed at regulating powerful AI models with mandatory safety tests and "kill switch" mechanisms, faced strong opposition from tech giants like OpenAI and Google. Supporters argue it’s essential to prevent unchecked AI risks.

OpenAI’s Revolving Door: Who’s Next?

• OpenAI faces another leadership shake-up as Chief Research Officer Bob McGrew and VP Barret Zoph exit, following CTO Mira Murati’s abrupt departure. CEO Sam Altman downplays the resignations and says he will be the one now focusing more on technical matters (me! me! me!) But with whispers of a $150B valuation and a potential 7% stake for Altman himself, the real question is whether the company’s shift toward a for-profit model is driving the talent exodus.

• Also, according to NYTimes, OpenAI is projecting a $5 billion loss for 2024 despite 1,700% revenue growth since the beginning of 2023. The company is targeting $11.6 billion in revenue next year and is raising $7 billion in a funding round that could value it at $150 billion. Rising computational costs, and ops expenses are contributing to its financial challenges. Thrive Capital leads this round, Microsoft is involved. Apple just exited talks to unvest.

• Meanwhile, a new, mysterious image generation model named "Blueberry" has surfaced on the leaderboards, beating FLUX.1. We are not into speculation usually but sounds like OpenAI to us.

🥳 Hugging Face Hits 1 Million Models!

• Hugging Face now hosts over 1 million public models, from big names like LLaMA to countless specialized, custom AI models. With a new repository created every 10 seconds, the platform is proving that tailored AI is the future.

Meta’s Orion Glasses: The Future is Holographic

• At Meta Connect 2024, Project Orion stole the show last week. These futuristic AR glasses feature holographic displays and a neural interface that responds to wrist gestures, bringing sci-fi tech into reality. While Orion is still in development, the potential for blending the digital and physical worlds promises to push augmented reality to new heights. The best demo experience described in Stratechery.

Nvidia Gobbles Up OctoAI: Acquisition Fever Continues

• It is the fifth startup Nvidia has acquired in 2024. Before OctoAI, it brought under its roof Run:ai, Deci AI, Shoreline, and Brev.dev. As Nvidia tightens its grip on the AI infrastructure market, concerns about regulatory scrutiny and competition intensify.

Microsoft's AI Trust Plan: Locking It Down

• Microsoft unveils its latest push for "Trustworthy AI," emphasizing robust security, safety, and privacy. New capabilities like confidential inferencing and safety measures for content keep AI outputs clean and compliant. A key player in AI, Microsoft’s all-in on responsible AI, ensuring users are protected while unlocking the full potential of AI-driven innovations.

AI adoption: Insights

• The NBER Working Paper on generative AI adoption reveals rapid growth in the U.S., with 39.4% of adults aged 18-64 using the technology by August 2024. AI adoption is particularly high among younger, educated, and higher-income individuals, with men using it more than women. Usage is widespread across occupations, especially in management and tech roles, though notable adoption exists even among blue-collar workers. Generative AI primarily assists with writing, administrative tasks, and data interpretation. An estimated 0.5-3.5% of work hours are now supported by AI, suggesting its growing influence on productivity and economic impact.

Making Text Embedders Few-Shot Learners

This paper is important because it introduces a novel approach to text embeddings by leveraging in-context learning (ICL) capabilities of LLMs. By integrating few-shot examples, the method significantly improves the performance of embeddings across multiple tasks, enhancing generalization and task relevance. This approach achieves state-of-the-art results on widely-used benchmarks (MTEB and AIR-Bench) without requiring complex model changes, making it a practical and efficient solution for advancing natural language processing tasks →read the paper

Prithvi WxC: Foundation Model for Weather and Climate addresses weather forecasting and climate modeling, outperforming traditional methods in tasks like hurricane tracking and extreme event prediction. Read the paper

TIME-MOE: Billion-Scale Time Series Foundation Models with Mixture of Experts scales time series forecasting with a mixture of experts architecture, optimizing computational efficiency and improving forecasting accuracy. Read the paper

Hackphyr: A Local Fine-Tuned LLM Agent for Network Security Environments applies large language models to network security tasks, offering adaptable performance for red teaming in complex scenarios. Read the paper

An adapted large language model facilitates multiple medical tasks in diabetes care specializes in diabetes-related medical tasks, achieving superior performance in clinical evaluations and personalized healthcare. Read the paper

Boosting Healthcare LLMs Through Retrieved Context improves factual accuracy in healthcare-specific models by integrating context retrieval systems, bridging the gap between open and proprietary models. Read the paper

Zero-shot Cross-lingual Voice Transfer for TTS enables voice transfer across languages using only one speech sample, significantly improving voice similarity and application to dysarthric speech. Read the paper

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models enhances vision-language models using human-annotated datasets, achieving state-of-the-art performance on multimodal benchmarks. Read the paper

MONOFORMER: One Transformer for Both Diffusion and Autoregression simplifies architecture for text and image generation by using one transformer for both autoregressive and diffusion-based tasks, showing competitive performance across benchmarks. Read the paper

Phantom of Latent for Large Language and Vision Models enhances vision-language learning by expanding latent dimensions temporarily, boosting performance in resource-constrained environments. Read the paper

EMOVA: Empowering Language Models to See, Hear, and Speak with Vivid Emotions integrates speech, vision, and text capabilities to improve emotional understanding in spoken dialogues and multimodal tasks. Read the paper

Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections improves mirror reflection generation in diffusion models, ensuring accurate geometric reflections for image editing and AR. Read the paper

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models reduces inference costs by applying a learnable pruning method, improving both efficiency and performance in large language models. Read the paper

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction optimizes long-context language models by reducing input tokens, achieving faster processing and lower memory usage without sacrificing performance. Read the paper

Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models enhances LLMs' reasoning by integrating logical prompts, boosting performance across various reasoning tasks. Read the paper

REWARD-ROBUST RLHF IN LLMS improves reinforcement learning from human feedback by introducing robust reward models that account for uncertainty, increasing learning stability. Read the paper

RRM: Robust Reward Model Training Mitigates Reward Hacking enhances reward model training to prevent reward hacking, improving preference alignment in large language models. Read the paper

Minstrel: Structural Prompt Generation with Multi-Agents Coordination for Non-AI Experts automates prompt generation using multi-agent collaboration, improving ease of use for non-AI experts in generating high-quality prompts for LLMs. Read the paper

NoTeeline: Supporting Real-Time Notetaking from Keypoints with Large Language Models aids users in real-time notetaking by expanding micronotes into full-length notes, improving writing efficiency and quality. Read the paper