Import AI 298: Mimetic models; LLM search engine raises $25m; UK splits from Europe on AI regulation

How many parameters will be tunable in the 'default settings'  of a superintelligence? 
View this email in your browser

Welcome to Import AI, a newsletter about artificial intelligence. Forward this email to give your chums an AI upgrade. Subscribe here.

Digital artist: DALL-E is a scam:
…Gen models have brought a ton of people a ton of joy, but some are skeptical..
Here's a post from artist/game developer David OReilly arguing that generative models like Dall-E 2 are a scam. Specifically, because these models scrape a vast amount of image data and spit out new images on tap (in exchange for $, per OpenAI's recent commercialization of Dall-E), then that means "paying for it benefits a tech company on the back of a century of human effort - a bullshit deal", according to OReilly.

Why this matters: This kind of argument reminds me against early arguments against things like sampling (for music creation), or collage (for making art out of other people's art). I think what makes (some) people nervous about Dall-E is the scale of resources required to develop it means, at least under capitalism, the destiny of these models is mostly to be as products. It feels like the reaction to stuff like Dall-E 2 would be radically different if it was provided as a public good (including free inference services). Many criticisms about AI are really criticisms about 'technology under capitalism' and it's worth trying to disentangle the two. 

   Read OReilly's post here on his Instagram (Instagram).

####################################################

Is AI alignment getting too much money?

…AI alignment is important, but so is progress…

Is the field of AI alignment sucking up too much funding? Researcher Bharath Ramsundar thinks so, arguing that the rapid expansion in funding for alignment might be silly. "AI alignment dollars could probably be better directed to funding next generation American foundry companies to ensure that the entire AI industry isn’t cast into turmoil by a potential future CCP invasion of Taiwan," he writes. 

Jack's thoughts: As someone who works at the intersection of AI capabilities, policy, and alignment, I find this argument a bit confusing - it basically assumes funding sources for alignment are fungible with resources for things like chips and foundries, but I'd argue that funding here typically comes from different sources with different types of experience. It's not either/or, it's both. (Though I do agree we desperately need to increase funding for semiconductors, given how crucial they are to economic and national competitiveness, and the fact they're currently centralized in some unstable geographies).

   Read more: An Argument Against Funding AI Alignment (Deep into the forest, Substack).

####################################################

Now that models can imitate people, what do we do?

…All hail the era of the funhouse mirror model…
A language model can do an impression of Einstein, a lawyer from Texas in the 19th century, and - given enough information - you. Now, researchers with the University of Toronto, Cornell University and Microsoft Research have grappled with the issues these so-called 'Mimetic Models' may produce. 

What they are: A mimetic model is "an algorithm that is trained on data from a specific individual in a given domain, and which is designed to accurately predict and simulate the behavior of this individual in new situations from the domain", they write. "Interacting with a mimetic model can be used as preparation for interactions in real life - essentially, as a means to an end."


How they might be used: These models will be used for tasks as varied as being a stand-in for oneself (e.g, answering emails for you), or being a stand-in for an opponent (e.g, preparing for a competition with someone, or a debate). They could also be used as 'mimetic counterfactuals' - how might a person change if they did something different with their life? 

   Real world use: Mimetic models are already out there in the world - like AI21's marketing stunt to create a virtual 'Ruth Bader Ginsburg' model people can talk to (Import AI 296), or this experiment by an independent artist where they resurrect a childhood friend and the mimetic model tries to kill them using a microwave (Import AI 292).

How to think about them: We should think about these models with reference to four key roles - the target that the model is designed to imitate, the person or organization that created the model, the operator who uses the model, and the interactor who interacts with the model or views its outputs. 


Why this matters: Because language models can approximate specific data distributions, it makes sense they can eventually represent people to a high level of fidelity. But I'm not sure the world is ready for the economic, security, and cultural implications of (digital) clones on tap. 

   Read more: Mimetic Models: Ethical Implications of AI that Acts Like You (arXiv).

####################################################

London heatwave downs Oracle and Google clouds:
AI, meet climate change…
The recent heatwave across the UK caused outages in data centers used by Oracle and Google, according to Bloomberg. While only temporary, this illustrates the fragility of the infrastructure AI requires, and highlights how, as climate change gets more extreme, some of the 'input costs' for AI-supporting infrastructure may increase.
  Read more: Google, Oracle Data Centers Knocked Offline by London Heat (DataCenter Knowledge).

####################################################

LLM-powered search app You raises $25m:

…Language models might eat search engines…

You, a search engine co-founded by Richard Socher, an AI researcher, has raised a $25m funding round. Socher says You has hundreds of thousands of users and a decent retention rate - not Google numbers, but not totally inconsequential.

Why You matters: The most interesting part of You is how it incorporates a bunch of contemporary language models, providing inbuilt services for things like text analysis, summarization, code search, code completion, and so on. You.com also sits on LMs built by others, such as OpenAI's GPT-3 which powers the 'YouWrite' service. 

Why this matters: Contemporary AI models are very general and very powerful - startups like You.com help test out whether these AI systems could obviate or replace prior technology 'stacks'. This funding means You will be around for a while longer, so we can watch the experiment play out.
  Read more: You raises $25M to fuel its AI-powered search engine (TechCrunch)


####################################################

UK looks at European Commission AI regulations and says 'that's too much', and proposes lightweight regulatory approach:
…Which way, Western governments?...
The UK government's Office for Artificial Intelligence has published a policy paper about how the UK government is going to approach AI regulation. The approach is designed to strike a balance between control and laissez faire development. The government describes its approach as "a pro-innovation, light-touch and coherent regulatory framework, which creates clarity for businesses and drives new investment". 


Key principles: The UK says it's going to approach AI regulation as a context-specific area, so it will create specific regulations for specific use cases. It also wants regulators to "focus on high risk concerns rather than hypothetical or low risks associated with AI," as well as "look for ways to support and encourage regulatory coordination" given that the UK has a bunch of overlapping authorities with regard to AI. It's also generally steering away from hard regulation, noting that "we will ask that regulators consider lighter touch options, such as guidance or voluntary measures, in the first instance".

Things that make you go 'hmmm': "We will ask that regulators focus on high risk concerns rather than hypothetical or low risks associated with AI," it writes. 

Challenges for regulation: Regulating AI also comes with some challenges - for one thing, merely by introducing regulation you can make it harder for small businesses to operate (relative to large businesses, which will simply lawyer up). There are also standard things to work through, like overlaps across different authorities, and inconsistencies among regulators.

Defining AI: Any policy document needs to define AI, and this is no different. Here, they try and do a pretty light touch, where they define an AI system as having two big characteristics - how adaptive it is to different scenarios, and how autonomously it can function. These feel like somewhat useful definitions, though in practice they're a bit mangled (e.g, the report defines a transformer-based language model as being highly autonomous as it can generate a bunch of text itself, whereas I suspect most people would think of AI systems being autonomous if they took a bunch of actions in an environment, like an RL agent). 

AI principles: In regulating AI, the UK government says it will stick to the following principles: 

  • Ensure that AI is used safely.
  • Ensure that AI is technically secure and functions as designed.
  • Make sure that AI is appropriately transparent and explainable. 
  • Embed considerations of fairness into AI.
  • Define legal persons' responsibility for AI governance.
  • Clarify routes to redress or contestability 
  • "We propose that regulators will lead the process of identifying, assessing, prioritizing and contextualizing the specific risks addressed by the principles."

Feedback requested: Like most government policies, the UK government is taking feedback on these ideas. Specifically, it wants to hear from people about what the contemporary challenges of regulating AI are, whether the proposed context-driven approach is effective, if and how the UK could establish cross-sectoral principles, how best to implement this approach, and if any data sources exist which could help the government monitor the effectiveness of its approach. 

   Read more: Establishing a pro-innovation approach to regulating AI (GOV.UK).


####################################################

The Immemorial Now

"It used to cost millions of dollars and terabytes of data to reanimate a family member. But these days you just need a few photographs, about a hundred dollars, and some patience. Basically you describe the family member and then your glasses layer them into your world, and then they give the family member a voice and back it onto a customized language model. If you've got some old movies of them, you can clone the voice. They act a bit strange at first, but if you just keep describing them and recounting your memories of them, the underlying model is able to capture them eventually. Then you look around and you're there with them," he said. "Honestly, I think it could really help you."

I was uneasy about it. It didn't feel right to me. But on the other hand, there I was, sitting with my sadness and bumming out my friends and talking, as I tended to, about the dead and departed. 

   "Of course we're gonna support you," he said. "But maybe this is a way to support yourself."

   "And you've done it?"

   "Oh, absolutely! Why do you think I talk about my grandad so much? He passed years ago, but this way I can still see him sometimes. I like his jokes." 

   "But they're not his jokes, they're some AI coming up with jokes."

   "Doesn't make much of a difference - they're the same jokes he used to tell, and he looks like himself, and sounds like himself. What's it - if it walks like a granddad and talks like a grandad, then it's probably a granddad you know?"

My dream helped me make the decision. It was a warped memory. We were in the kitchen of the old house and she was there and we were making bread together. She turned to me and asked me to pass her something and though I knew what she meant, I couldn't hear her voice. I stared at her and started to panic and then I woke up in bed, sweating, grasping mentally at the memory of her. 

   I tried to calm myself down by imagining her talking to me. Then I realized I couldn't remember her voice. 

   I became very sad and also very angry. I cried into my pillow. I tried to remember. I couldn't remember. 

A few days later,  I was uploading some old videos of her into the resurrection machine. Then I spent a few days talking to the machine about her, telling it little anecdotes - even recounting some of my dreams. I gave it all the images I had of her. I obsessively searched over all my computers until I was sure I'd given it everything I had.    Then one day I asked it to generate her. I put the glasses on and closed my eyes. Then I heard the little sound engineered to sound both reassuring and insistent. She was ready.
  I opened my eyes and there she was, and she looked at me and smiled and said "I've missed you", and it felt so real I let myself forget her unreality.

Things that inspired this story: Resurrecting the dead with AI and how it can be both helpful and deeply personal; generative models; the intersection of augmented reality and AI; multimodal models, few-shot learning for vast multi-modal models; ideas about how, in the limit, AI lets us generate a stand-in for anything we have data for; mimetic models.



Thanks for reading. If you have suggestions, comments or other thoughts you can reach me at jack@jack-clark.net or tweet at me@jackclarksf

Twitter
Facebook
Website
Copyright © 2022 Import AI, All rights reserved.
You are receiving this email because you signed up for it. Welcome!

Our mailing address is:
Import AI
Many GPUs
Oakland, California 94609

Add us to your address book


Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

Email Marketing Powered by Mailchimp

Key phrases

Older messages

Import AI 297: Ukrainians add object detection to killer drones; YOLOv7; and a $71,000 AI audit competition

Monday, July 18, 2022

In the same way humans have a notion of the aesthetics of abstract sciences, like theoretical physics or advanced mathematics, might computers develop notions of 'beauty' at even higher levels

Import AI 296: $100k to find flaws in LLMs, NVIDIA uses RL to make better chip parts; + 256gb of law data, and a story about the cyber gerontocracy!

Monday, July 11, 2022

Will we ever have 'old growth' computers in the way we have 'old growth' forests today. In the same way there are mainframes that have been running (albeit with parts swapped out and

Import AI 295: DeepMind's baby general agent; NVIDIA simulates a robot factory; AI wars.

Friday, May 20, 2022

If it is possible to develop human-level AI, at one point will we make the first AI magician that perplexes even the most accomplished human magician? View this email in your browser Welcome to Import

Import AI 294: China makes a vast facial recognition dataset; Facebook releases a 30bn parameter model; real world RL

Tuesday, May 10, 2022

If an AI designed the world we lived within, would we know? View this email in your browser Welcome to Import AI, a newsletter about artificial intelligence. Forward this email to give your chums an AI

Import AI 293: Generative humans; few shot learning comes for vision-text models; and another new AI startup is born

Monday, May 2, 2022

How will fashion be influenced by the aesthetics of early AI generation technology? View this email in your browser Welcome to Import AI, a newsletter about artificial intelligence. Forward this email

Issue #11: Focused Work by Michael Tigas

Tuesday, August 16, 2022

Today, we're taking a look at Focused Work by Michael Tigas. Focused Work is a powerful timer that helps you have highly productive days. Inspired by the Pomodoro Technique, it's purpose-built

New Blogs on ThomasMaurer.ch for 08/16/2022

Tuesday, August 16, 2022

View this email in your browser Thomas Maurer Cloud & Datacenter Update This is the update for blog posts on ThomasMaurer.ch. Schedule and Run PowerShell Scripts for Azure VMs using Azure

[Python Mastery] What does it mean to "master" Python?

Tuesday, August 16, 2022

Hey there, A couple of years ago I'd become quite interested in martial arts. Hours upon hours of watching "The Karate Kid" growing up must've taken their toll on me... And so, I

Using HPC clusters to run simulations & solve complex equations

Tuesday, August 16, 2022

How to select the best processor & HPC system View this email in your browser engineering.com White Paper - How to Select the Best Processor and HPC System for Your Workloads How to Select the Best

Don't miss out: Ready, Set, Actionable search results with Elastic Enterprise Search

Tuesday, August 16, 2022

Tailor-made for participants in tech and non-tech roles elastic | Search. Observe. Protect Search & Stream with Elastic Enterprise Search You've looked for a good movie to watch, but ever

Declining quality of consumer-grade products – 2009 fridge compressor autopsy — and Breaking all macOS security layers with a single vulnerability

Monday, August 15, 2022

Issue #860 — Top 20 stories of August 16, 2022 Issue #860 — August 16, 2022 You receive this email because you are subscribed to Hacker News Digest. You can open it in the browser if you prefer. 1

Meta backs SMB e-commerce app launched by former Facebook engineering manager

Monday, August 15, 2022

TechCrunch Newsletter TechCrunch logo The Daily Crunch logo By Christine Hall and Haje Jan Kamps Monday, August 15, 2022 Hello, Crunchers! Wait, that's kind of a weird nickname, as if you're

JSK Daily for Aug 15, 2022

Monday, August 15, 2022

JSK Daily for Aug 15, 2022 View this email in your browser A community curated daily e-mail of JavaScript news Handling Optimistic Concurrency in Web Frontends Building scalable web applications

Live Long, Vote and Nominate #Noonies2022

Monday, August 15, 2022

for the brightest stars in tech ✨ The Noonies 2022 Greetings, Hackers 👋🏽 As you well know, HackerNoon is recognizing the incredible work of those who have boldly gone where no one has gone before, for

Max Q - Testing, testing

Monday, August 15, 2022

TechCrunch Newsletter TechCrunch logo Max Q logo By Aria Alamalhodaei Monday, August 15, 2022 Hello and welcome back to Max Q. In this issue: Northrop Grumman and Firefly Aerospace want to make the