Editor Picks

• Best Practices for Deploying Language Models
  Cohere, OpenAI & AI21Labs have announced a set of best practices for responsible deployment of large language models. The joint statement is a first step towards fostering an industry-wide conversation to bring alignment to the community...

• After millennia of agricultural expansion, the world has passed ‘peak agricultural land’
  The world produces more food than ever, but the amount of land we use is now falling. This means we can feed more people while restoring wild habitat...

• Compact word vectors with Bloom embeddings
  A high-coverage word embedding table will usually be quite large. One million 32-bit floats occupies 4MB of memory, so one million 300-dimension vectors will be 1.2GB in size. Such a large model size is at least annoying for many applications, while for others it’s completely prohibitive...Probabilistic data structures are a natural fit for machine learning models, so they’re quite widely used...We’ll start by introducing the full algorithm, without dwelling too long on why it works. We’ll then go back and fill in more of the intuition, and then describe how we use it in practice in Thinc, spaCy and floret...

A Message from this week's Sponsor:

Data Science Articles & Videos

• Folia: authoring tool for narrative games
  With Folia, you can write interactive, branching stories using a simple syntax based on Lua. Folia presents your game in a two-page 'storybook' format, with one leaf dedicated to a scrolling log of text, and the other displaying images. You can make your story as simple or complex as you like, with the full power of Lua available whenever you need it. When you're done, you can upload your finished game to itch.io or wherever you like...

• The death of the Netflix stars: A brief primer on online user ratings
  A few weeks back, Netflix announced that they were adding a two thumbs up option to give feedback on their content. At the same time I was being asked on Twitter why we had given up on 5 star ratings at Netflix back in the day. Given that I have personally spent a lot of time in my career looking into ratings and user feedback, I thought I’d share some of what I have learned not only during my time at Netflix, but also before and after...

• Advocating for the LGBTQ+ community in AI research
  Research scientist, Kevin McKee, tells how his early love of science fiction and social psychology inspired his career, and how he’s helping advance research in ‘queer fairness’, support human-AI collaboration, and study the effects of AI on the LGBTQ+ community...

• Multi-Game Decision Transformers
  A longstanding goal of the field of AI is a strategy for compiling diverse experience into a highly capable, generalist agent. In the subfields of vision and language, this was largely achieved by scaling up transformer-based models and training them on large, diverse datasets. Motivated by this progress, we investigate whether the same strategy can be used to produce generalist reinforcement learning agents. Specifically, we show that a single transformer-based model -- with a single set of weights -- trained purely offline can play a suite of up to 46 Atari games simultaneously at close-to-human performance...

• Friends don’t let friends train small diffusion models
  For my next project, I wanted to play around in the music generation space...The architecture I decided to use is fairly conventional: I used the UNet model like the one you can get from the OpenAI Guided Diffusion repo with the a structural guidance scheme I developed while working on Tortoise. The conversion from MEL <=> waveform does not need global attention, so all global attention layers were removed as well...The results were awful...

• The Logic of Strategic Assets: From Oil to AI
  What resources and technologies are strategic? Policy and theoretical debates often focus on this question, since the “strategic” designation yields valuable resources and elevated attention...We offer a theory of when decision makers should designate assets as strategic based on the presence of important rivalrous externalities for which firms or military organizations will not produce socially optimal behavior on their own...To illustrate the analytic value of our framework for thinking about strategic technologies, we examine the US-Japan technology rivalry in the late 1980s and current policy discussions about artificial intelligence....

• Qualitative humanities research is crucial to AI
  Following the thread of any seemingly quantitative issue around AI ethics quickly leads to a host of qualitative questions. Throughout AI, qualitative decisions are made about what metrics to optimise for, which categories to use, how to define their bounds, who applies the labels. Similarly, qualitative research is necessary to understand AI systems operating in society: evaluating system performance beyond what can be captured in short term metrics...

• On the Link Between Optimization and Polynomials, Part 5 - Cyclical Step-sizes.
  Momentum with cyclical step-sizes has been shown to sometimes accelerate convergence, but why? We'll take a closer look at this technique, and with the help of polynomials unravel some of its mysteries...

• Optimism Shapes Reality - How to go at warp speed
  Posting a memo I sent to the @scale_AI team back in 2019...The core idea is that most organizations fall prey to a slow death of optimism, causing a slow, excruciating halt....

• Narrative AI
  Hilary Mason on how AI can be used to create playable role playing games...This week’s guest is Hilary Mason, co-founder of Hidden Door, a startup that uses AI and machine learning to help create and power role-playing games (RPG). Hidden Door creates RPGs based on user inputs and their platform dynamically generates the texts, the art, the composition based on users choices and what they decide ought to happen next. Our conversation centered on Hidden Door and its underlying technology...

• A developer's guide to responsible AI review processes [Video]
  From startups to corporations across industries, organizations are creating AI principles and ethics review processes to complement technical approaches to developing ML and AI responsibly. Listen to emerging socio-technical practices, ML tools, and lessons learned from Google’s ethics review teams who support developers as they build products...

Course*

Jobs

• Data Scientist - Hungryroot - Remote
  
  Hungryroot is looking for a Data Scientist to join our growing Data Team. As a Data Scientist, you will work closely with other Data Scientists and Data Engineers to develop various Machine Learning models that power Hungryroot and it’s AI functions. These models include traditional forecasting models, as well as more industry-specific optimization challenges.
  
  As a Data Scientist at Hungryroot, you will work on answering questions like: how do you tell what food someone would like to eat this week, how do you determine whether they enjoyed it or not, maybe they liked their means last week, but are now looking for different options, maybe they like the same food on Tuesdays, but variety on Fridays, what about spicy food, is Green Chilly as spicy as Green Curry?
   

Training & Resources

• MLU-EXPLAIN
  Visual explanations of core machine learning concepts...Machine Learning University (MLU) is an education initiative from Amazon designed to teach machine learning theory and practical application...As part of that goal, MLU-Explain exists to teach important machine learning concepts through visual essays in a fun, informative, and accessible manner...

• Data Science in Context: Foundations, Challenges, Opportunities [website/PDF]
  This website provides the Authors’ Manuscript for Data Science in Context: Foundations, Challenges, Opportunities...Book by Alfred Spector, Massachusetts Institute of Technology, Peter Norvig, Stanford University, California, Chris Wiggins, Columbia University, New York, Jeannette M. Wing, Columbia University, New York...

• Machine Learning Compilation [free course, launces June 17]
  In this tutorial sequence, we offer the first comprehensive treatment of its kind to study key elements in this emerging field systematically. We will learn the key abstractions to represent machine learning programs, automatic optimization techniques, and approaches to optimize dependency, memory, and performance in end-to-end machine learning deployment....

What you’re up to – notes from DSW readers

• Olly is working on a web app that uses computer vision to provide exercise technique feedback...

• Hylke is working to Prevent overtreatment of lung cancer patients through AI...

* To share your projects and updates, share the details here.

** Want to chat with one of the above people? Hit reply and let us know :)

Last Week's Newsletter's 3 Most Clicked Links

• Discover Weekly: How the Music Platform Spotify Collects and Uses Your Data

• You're Relying on Data Too Much

• Stanford MLSys Seminar Episode 5: Chip Huyen [Video]

* Based on unique clicks.

** Find last week's newsletter here.