|
Welcome to Import AI, a newsletter about artificial intelligence. Forward this email to give your chums an AI upgrade. Subscribe here.
GPT-3 can simulate people very, very well - social science might change:
…Turns out a synthesis engine trained on the exhaust of human culture can be pretty good at simulating people…
Researchers with Brigham Young University have written a paper which I think is among the most significant things I've ever covered in this newsletter. Specifically, they do three social science experiments on GPT-3 and discover that GPT-3 has biases that are "fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups."
Put another way: You can simulate people in GPT-3 and they might respond with uncanny similarity to real people in real life.
Sit with that for a minute and spool out the implications, while mentally turning the crank on model size advancements.
What their study showed: The authors did this research by "conditioning GPT3 on thousands of socio-demographic backstories from real human participants in multiple large surveys in the United States: the 2012, 2016, and 2020 waves of the American National Election Studies (ANES)[16], and Rothschild et al.’s “Pigeonholing Partisans” data ". They found that GPT3 "when properly conditioned, is able to produce outputs biased both toward and against specific groups and perspectives in ways that strongly correspond with human response patterns along fine-grained demographic axes. In other words, these language models do not contain just one bias, but many".
In other words: When they did some tests to try and see if GPT3 would make similar responses as people when given the priors of the same demographic background data, GPT3 responds in a remarkably similar-to-people way. :"We provide evidence that algorithmic fidelity is a crucial attribute of tools like GPT-3 because it demonstrates that these language models can be used prior to or in the absence of human data."
Silicon Sampling: The researchers call this approach 'silicon sampling'; simulate people in GPT3, then poll them as a substitute for real world data. The approach seems sufficiently useful that some people will do this as a way to try out a few variations of survey design ahead of polling a real population, for instance.
Social science simulation is cool, but do you know other people think is cool? Full-Spectrum AI-Facilitated Information Warfare! Because models like GPT3 can, at a high level, simulate how different human populations respond to certain things, we can imagine people using these models to simulate large-scale information war and influence operations, before carrying them out on the internet. "Models with such fidelity, coupled with other computational and methodological advances, could be used to target human groups for misinformation, manipulation, fraud, and so forth," the authors note.
Read more: Out of One, Many: Using Language Models to Simulate Human Samples (arXiv).
####################################################
We might have figured out some 'scaling laws' for reinforcement learning:
…RL agents could be better if they have bigger neural nets, study suggests…
Researchers with Goethe University have tried to figure out some 'scaling laws' for reinforcement learning agents. "Scaling laws" help researchers figure out the right mix of compute and data to allocate to a machine learning model to get a particular level of performance and have been widely studied in fields like natural language and image generation.
Here, the researchers try to do a 'scaling law' style analysis of AlphaZero RL agents playing two distinct games; Connect Four and Pentago. "These two games are non-trivial to learn and light enough to allow for training a larger number of agents with a reasonable amount of resources," the researchers write.
What they found: In tests, they found that "playing strength scales as a power law with neural network size when models are trained until convergence at the limit of abundant compute," and they extrapolate their results to indicate AlphaGo Zero and AlphaZero (two landmark DeepMind research systems for playing Go) likely used neural nets that were too small and they could therefore "achieve better performance with larger neural nets".
Why this matters: "We find it noteworthy that scaling laws that are common to language and other supervised learning models are also present in one of the most important MARL models. This scaling behavior could be common to other reinforcement learning algorithms, which would provide an opportunity to optimize their resource allocation," they write.
Read more: Scaling Laws for a Multi-Agent Reinforcement Learning Model (arXiv).
####################################################
Want to train an LM with RL? Now there's some free software to help you:
…Train up to 20B parameter models using RL…
Researchers with CarperAI, a language model collective which span off from the open source model people at Eleuther, has released Transformer Reinforcement Learning X (trlX), software for training language models with reinforcement learning.
"the trlX repo allows you to fine-tune Huggingface supported language models up to 20B parameters via either reinforcement learning using a provided scoring function or reward-labeled dataset. We aim to support a range of both online and offline RL algorithms including Proximal Policy Optimization (PPO), Natural Language Policy Optimization (NLPO), Actor Critic (A2C), and Implicit Q Learning (ILQL)," they write. "The library supports gpt2 and gptj with plans to include GPT-NeoX, T5 and more."
Why this matters: Reinforcement learning training is a super effective way to 'bake in' additional capabilities for a given language model. RL training is also pretty difficult and buggy. Software like trLX will make it easier for more people to train more capable language models.
Read more: Welcome to Transformer Reinforcement Learning X (trlX) (GitHub).
####################################################
Microsoft warns about smart deepfakes, and deepfake-realworld influence campaigns:
…Reality collapse via sub-sentient generative avatars…
Microsoft's Chief Scientific Officer, Eric Horvitz, is very worried about the future of deepfakes in two particular ways: first, deepfakes are going to soon become a lot more intelligent and will be able to carry out plausible conversations, and second, people are going to conduct well-resourced influence campaigns that pair deepfake disinformation with carefully scripted real world events.
Interactive deepfakes: "Automated interactive deepfakes could be endowed with basic understandings of the status of flow of a conversation to inform decisions about if and when to interject," Horvitz notes. These kinds of deepfakes will lever all the advances happening in generative imagery, video, audio, language, and so on, and create increasingly capable and persuasive fake avatars.
Compositional deepfakes: The other big worry is what happens when people use deepfakes as part of lengthy influence campaigns. "Compositional deepfakes can be designed to create fictional narratives that are persuasive in their ability to tie together and provide powerful explanations of sets of events in the world to citizens and government leaders," Horvitz writes. "It is not hard to
imagine how the explanatory power of custom-tailored synthetic histories could out-compete the explanatory power of the truthful narratives".
What can we do: Horvitz does list out a few interventions that we can make, which all net out to "invest a ton more money in X", where X is any of the following: Journalism and reporting; media literacy; authenticity protocols; content provenance; watermarks and fingerprints; detection; regulation and self-regulation, and red-teaming and continuous monitoring.
While these are all nice, viable technocrat solutions to the various problems deepfakes imply, I'm skeptical they'll work. The fact so many people around the world these days are retreating to choose-your-own adventure fantasies is because of some deep changes in culture in past few years, ranging from boom in production of media content to flattening of the world via things like the internet, and more. Put bluntly: Horvitz's solutions are all nice but assuming we had all of them, I still suspect deepfakes will become an increasingly significant driver of strange cultural phenomena, and people may even knowingly interact with known-fake entities and do it all the same.
Read more: On the Horizon: Interactive and Compositional Deepfakes (arXiv).
####################################################
DeepMind trains an RL agent which figures out a more efficient form of matrix multiplication:
…AI accelerating AI at a hugely basic level…
DeepMind has built AlphaTensor, an AlphaZero-style agent which discovered algorithms that improve upon human ones for basic tasks like matrix multiplication. "Our AI-designed algorithms outperform human-designed ones, which is a major step forward in the field of algorithmic discovery," DeepMind writes.
It's probably a big deal, folks! DeepMind CEO Demis Hassabis writes: "Since 1969 Strassen’s algorithm has famously stood as the fastest way to multiply 2 matrices - but with #AlphaTensor we’ve found a new algorithm that’s faster, with potential to improve efficiency by 10-20% across trillions of calculations per day!" DeepMind also designed specific ways to do matrix multiplication optimizations for Nvidia V100 GPus and Google TPU v2, illustrating how you can couple this system to target particular hardware.
Possibly overhyped: The practical implications of this result might be a bit overhyped - I myself thought 'cool, this seems like a drop-in speedup', but others who know more about this area than me are somewhat disagreeing with that. E.g, James Bradbury writes: "these algorithms are helpful for integer multiplication (but require some extra bits) and high precision floats, but not so much for the lower precision floats that drive most ML work. And at low precision multiplies are no longer as dominant (vs adds)."
Regardless, this matters: Even if the practical implications are small, the fact we were able to further refine a math thing that humans have been trying to further optimize for 50 years is a big deal. This is a case where an AI has had an insight that the combined efforts of many human brains have failed to have.
How they did it - everything's a game: To get this to work, DeepMind reframed the problem of algorithm discovery as a single player game, which they then trained an RL agent in.
" At each step of TensorGame, the player selects how to combine different entries of the matrices to multiply. A score is assigned based on the number of selected operations required to reach the correct multiplication result," DeepMind writes. "This is a challenging game with an enormous action space (more than 1012 actions for most interesting cases) that is much larger than that of traditional board games such as chess and Go (hundreds of actions)."
They design an RL agent, AlphaTensor, which comes with some inductive biases for tensor inputs.
Why this matters: "The discovery of matrix multiplication algorithms has far-reaching implications, as matrix multiplication sits at the core of many computational tasks, such as matrix inversion, computing the determinant and solving linear systems," DeepMind writes.
More broadly, this work sits within the subfield of AI research where we're using AI systems to improve the efficiency of the things we use to develop AI; for example, we've already used RL agents to improve the design of TPUs which will be used to train future AI systems (Import AI 254), and this work uses an RL agent to speed up one of the most basic and widely performed operations in deep learning.
Read more: Discovering novel algorithms with AlphaTensor (DeepMind blog).
Get the code (including the better matrix multiplication) here (DeepMind GitHub).
Read more: Discovering faster matrix multiplication algorithms with reinforcement learning (Nature).
####################################################
The US government comes up with an AI "Bill of Rights" (minus the broad enforcement):
…The rights are one way the government can alter how AI systems show up to the American public…
The White House's Office of Science and Technology Policy (OSTP) has published a 'Bill of Rights' for AI systems. The idea is that the federal government will try to build and deploy AI systems in line with these rights, and the announcement of the Bill of Rights was paired with actions by federal agencies in line with the rights.
"The rights": These rights are framed, at a high level, as five "common sense protections". These include the right to use safe and effective systems, protection from algorithmic discrimination protections, data privacy, notice and explanation about the use of AI, and the ability to use human alternatives and/or opt out of certain systems.
Those rights in full:
- You should be protected from unsafe or ineffective systems.
- You should not face discrimination by algorithms and systems should be used and designed in an equitable way.
- You should be protected from abusive data practices via built-in protections and you should have agency over how data about you is used.
- You should not face discrimination by algorithms and systems should be used and designed in an equitable way.
- You should be protected from abusive data practices via built-in protections and you should have agency over how data about you is used.
- You should know that an automated system is being used and understand how and why it contributes to outcomes that impact you.
- You should be able to opt out, where appropriate, and have access to a person who can quickly consider and remedy problems you encounter.
Why this matters: Ultimately, how much the AI Bill of RIghts matters seems to rest on two things: a) how much the White House is able to enforce alignment with the Bill of Rights across federal agencies, and b) whether third-parties like academic or corporate research groups build systems that themselves fall in line with the Bill of Rights. It'll take time, but these rights may serve as a good way to develop more of the norms around the use of AI.
Read more: Blueprint for an AI Bill of Rights: A Vision for Protecting Our Civil Rights in the Algorithmic Age (White House blog).
Read more: FACT SHEET: Biden-Harris Administration Announces Key Actions to Advance Tech Accountability and Protect the Rights of the American Public (White House blog).
Read the Bill of Rights: BLUEPRINT FOR AN AI BILL OF RIGHTS MAKING AUTOMATED
SYSTEMS WORK FOR THE AMERICAN PEOPLE (White House, PDF).
####################################################
Maybe it is Crazy, Maybe it is Magic
I didn't think the route to intelligence was through insanity, but at this point, I'm open to being wrong about any of my assumptions.
We'd been banging our heads against a model for a few months and though it was very capable in a bunch of ways, it couldn't really reflect on things or update its own priors or do any of the things that felt important for creating an actual no-shit superintelligence.
So one day we shipped something cwe called 'the personality system'. I coded it in partnership with the AI model. I forget which of us came up with the term, but we gave it something we called 'a Greek chorus prompt'; a whole bunch of distinct personalities which modeled over different problems and exchanged information with each other.
The way I visualized it in my head was when we talked to the model, the model now spent a while talking to itself before answering us.
The results surprised us; model capabilities went up across the board, and its answers attained a new level of specificity and detailed. So then we trained the model using reinforcement learning to try and bake the 'greek chorus prompt' into the model at a deeper level.
After that was done, the model started to freak us out. It was now significantly faster and generally more capable.
When we hooked it up to some interpretability tools, we realized our mistake. The different personalities had formed into what we called 'personality circuits'; different personalities interacted with eachother to apply different methods of reasoning to tasks, and try as we might, we could never work out what rules governed how these personalities were used or exactly what they did - they were too high-dimensional, or perhaps a better way to put it is we were staring at the shadows on the wall from something of incalculably large dimensionality, projected back down.
What would you do with a deeply capable person who was smarter than you, but who you knew to be, in terms of how we'd evaluate people, functionally insane? How much power would you give that thing?
Perhaps, based on how things are these days, you can guess what we decided to do.
Things that inspired this story: Magic and mysticism in deep learning; prompting; RLHF; finetuning; various pitfalls in AI development; interpretability; the fact people are generally uninterpretable; capabilities versus safety overhangs.
Thanks for reading. If you have suggestions, comments or other thoughts you can reach me at jack@jack-clark.net or tweet at me@jackclarksf
|
