Wednesday, AI 101: In the second episode, we discuss Mamba, a deep learning architecture focused on sequence modeling, and how it rivals Transformers;

Friday: A new investigation into the world if AI Unicorns.

Practical knowledge and a foundational understanding of AI's current state

The ability to identify and leverage AI opportunities for organizational growth

A focus on the managerial rather than technical aspects of AI to prepare you for strategic decision making

Anthropic: Mapping the Mind of an LLM

OpenAI in turmoil

Meta: New AI and Technology Advisory Group

Google: Ads in AI-Generated Search Answers

Cohere For AI has launched Aya 23, a new family of multilingual generative language models supporting 23 languages. Available in 8-billion and 35-billion parameter models with open weights, Aya 23 builds on the Aya initiative with a focus on multilingual performance and global AI research support. The models are accessible on Hugging Face, and more information can be found on Cohere's research page.

Researchers from Stanford University, Princeton University, and MIT analyzed transparency among leading foundation model developers. The Foundation Model Transparency Index v1.1 evaluated 14 developers based on 100 indicators, revealing an average score increase from 37 to 58 over six months. The improvement was driven by direct reporting of previously undisclosed information. Persistent opacity issues include data access and labor transparency. These findings highlight the potential for policy interventions to further enhance transparency →read the paper

Researchers from Humane Intelligence, Seed AI, and AI Village organized the first public red teaming event for closed-source API models at DEF CON 31, evaluating eight LLMs. This event highlighted the potential for public red teaming to enhance AI oversight and policy development by identifying biases, misdirections, and cybersecurity vulnerabilities in AI models →read the report

Layer-Condensed KV Cache for Efficient Inference of Large Language Models - Introduces a method to reduce memory usage in LLMs by condensing the key-value cache to only a few layers, enhancing inference efficiency read the paper.

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization - Enhances transformer efficiency by simplifying linear attention and using progressively re-parameterized batch normalization read the paper.

Your Transformer is Secretly Linear - Discovers that transformer decoders exhibit a near-perfect linear relationship between sequential layers, providing insights into transformer architectures read the paper.

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning - Proposes a fine-tuning method for LLMs using high-rank updating to preserve parameter efficiency read the paper.

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention - Further reduces the key-value cache size in transformers, extending efficiency improvements read the paper.

Grounded 3D-LLM with Referent Tokens - Integrates 3D vision and language models using referent tokens, unifying 3D tasks under a generative framework read the paper.

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models - Leverages a Mamba architecture to enhance comprehension and response capabilities in vision-language tasks read the paper.

AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability - Develops a multimodal LLM that enhances alignment capabilities between text and image pairs read the paper.

Octo: An Open-Source Generalist Robot Policy - Trains a transformer-based policy for robotic manipulation, adapting to diverse sensors and action spaces read the paper.

Visual Echoes: A Simple Unified Transformer for Audio-Visual Generation - A lightweight transformer model for generating audio from images and vice versa read the paper.

Observational Scaling Laws and the Predictability of Language Model Performance - Proposes a generalized scaling law to predict LLM performance variations with scale read the paper.

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework - Introduces a framework for scalable reinforcement learning from human feedback in LLMs read the paper.

The Road Less Scheduled - Proposes a novel optimization approach that eliminates the need for learning rate schedules by unifying scheduling and iterate averaging read the paper.

Dynamic Data Sampler for Cross-Language Transfer Learning in Large Language Models - Introduces a cross-language transfer learning framework using a dynamic data sampler read the paper.

Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach - Proposes a clustering-based method for automatic data curation in self-supervised learning read the paper.

INDUS: Effective and Efficient Language Models for Scientific Applications - Develops a suite of LLMs tailored for scientific applications, introducing new benchmarks and outperforming general-purpose models read the paper.

Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization - Explores whether transformers can implicitly reason over parametric knowledge through extended training read the paper.

DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data - Enhances LLM theorem-proving capabilities with a model fine-tuned on synthetic data read the paper.

FIFO-Diffusion: Generating Infinite Videos from Text without Training - Presents an innovative method for text-based infinite video generation using pretrained diffusion models read the article.

DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis - Combines State Space Models with diffusion models for efficient high-resolution image synthesis read the paper.

Privacy-Preserving Diffusion Models Using Homomorphic Encryption - Trains diffusion models on encrypted data, ensuring privacy while maintaining performance read the paper.

RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance - Customizes diffusion models to generate identity-preserving images from user-provided references read the paper.

Semantica: An Adaptable Image-Conditioned Diffusion Model - Develops a diffusion model for generating images based on the semantics of a conditioning image read the paper.