|
|
HURRY: Wednesday is your last chance to save $100 on your INDUSTRY ticket! |
Brought to you in colloboration with CustomerIQ
the AI platform that seamlessly integrates with your CRM, help desk, and messaging apps to automate tedious administrative processes, extract valuable insights, and provide the whole GTM team with data they need to close deals faster.
Learn more here.
In 2012, MIT graduates Spenser Skates and Curtis Liu founded a startup called Sonalight, an app for voice-activated text messaging. While developing this app, they also created a piece of analytics software to understand user behavior.
Ironically, that analytics software started generating significant interest from other companies facing similar needs. Despite Sonalight's initial traction (about 200,000 downloads), Skates and Liu realized it would be a grind to achieve significant impact.
They decided to pivot to focus on their analytics software. And Amplitude was born.
Amplitude quickly gained traction with its digital analytics tools helping businesses optimize user experiences.
The company grew through strategic acquisitions, including ClearBrain in 2020 and Iteratively in 2021. In June 2021, Amplitude secured a $150 million investment, valuing the company at $4 billion. On September 28, 2021, Amplitude went public on the Nasdaq, achieving a valuation of $7.1 billion on its first trading day.
Today, Amplitude is a leading player in digital analytics, serving over 14,000 companies globally, including 26 of the Fortune 100.
I was so excited to learn that Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Francois Ajenstat, the Chief Product Officer (CPO) of Amplitude.
Here’s what we learned:
- Advice for implementing AI in your product
- What AI means for the industry
- Lessons learned from shipping AI features
- Francois’ “Don’t give a f****” philosophy
- How Francois thinks about experimentation
- What Francois thinks about frameworks (spoiler: he’s not a fan)
- What the next few years will look like
Please enjoy Mike’s conversation with Francois, CPO of Amplitude below.
To say AI is impactful on the life of a Product Manager is certainly an understatement, considering how transformative this era-defining marvel is. So, what are we going to do about it?
An excellent first step is to assess how we can leverage this technology today. We are doing just that at this year’s INDUSTRY: The Product Conference, which will be held September 23-25 in Cleveland, Ohio.
There, you'll be treated to several sessions delving deep into the topic:
- 4hr Workshop: Mastering the use of Generative AI
- Keynote: Product in the Age of AI
- Keynote: Selling AI Transformation
- Keynote: A.I. & the Future of Product Management
- 1 hr Working Session: AI for Product Managers
And Group Roundtable Discussions to discuss the findings with your peers.
You are running out of time to save on your INDUSTRY ticket! Earlybird pricing ends on August 1st -- so register before then and save $100.
What advice do you have for building with AI?
This space is moving so fast, it's moving at a clip I don't think we've ever seen before. If you're not continuously learning, experimenting, and trying things out, someone else will overtake you very quickly. You have to get the reps in; you have to get the learning in because that's how you get the better product.
A big part of this is overcoming fear. Just get it out there—experiment, make mistakes, learn from them. Have an open mindset to the possibilities. There are literally thousands of companies that no longer exist because they weren't open to new technologies. For instance, look at how Snowflake and Databricks have become hot companies in the data space. They were born in the cloud. Meanwhile, older companies that optimized on bare metal saw these new technologies and dismissed them, thinking they could never catch up. But the cloud got to a good enough state for the vast majority of customers, and those older companies are now irrelevant.
Whenever I hear someone say "it will never be" or "it is bad for these reasons," I see opportunity. That skepticism can create blind spots. So, don't have the blinders on. Open up the aperture, have a learning mindset, and lean into transformation.
What’s surprised you about AI and product building in the last few years?
The thing that surprised all of us was the speed of ChatGPT in terms of how it captured people's imaginations and how it changed literally every company's roadmaps and perspective of their priorities. I mean, it's incredible how fast that happened. I don't think we've ever, ever seen anything that fast. Even the mobile revolution happened really fast, but not that quickly. In the grand scheme of things, this is unlike anything else.
What's also been interesting to see is the number of models and the speed at which these models get developed while at the same time how you're kind of almost having a race to the bottom. It's now not only just about the largest models, but about which model is actually smaller and cheaper while still driving the right level of accuracy. So there's actually more choice than there's ever been, which can be a bit overwhelming at times.
Another surprising aspect is how last year seemed to be all about AI hype. Every company had an AI press release—it was AI this, AI that. But how much of it has actually been used in practice? How many companies are truly transformed as a result of AI? I think every company is better with AI, but the real potential hasn't fully become a reality yet—we're still in that learning phase.
It's also kind of fun to liken it to the launch of the iPhone and mobile technologies. Back then, you couldn't imagine not having a mobile app, and everyone rushed to build one. Today, you have web apps and other ways of interfacing with products, but it took a few years for new companies born in mobile to really make an impact, like Uber. Uber wouldn't exist without mobile. So I think it's going to be really fun to see in the next two years what new companies will emerge that are made possible because this AI technology is available.
The iPhone was big. How big is this AI movement?
I mean, think of the transitions we've gone through: the cloud transition, mobile, even social. Everything's changed. Has it completely transformed how we work every day? Not really, but it’s transformed maybe how we communicate, how we build relationships, how we're all connected, and maybe how we consume news. I think this is, again, transformational and you're starting to see it in your daily lives. Think of the emergence of Copilots. You write an email and you have auto-complete, you have suggestions. I use ChatGPT every day to write better. So, I think summarization is one of those core use cases that we see, but the launch of agents where you can have more automation, I think will be truly transformative. Some of the AI you won't actually see; it'll be happening behind the scenes, but it gets smarter and smarter over time. And the way you interact may evolve. There's still a human in the loop, but some tasks get, again, simplified and automated with the use of AI.
Tell us about how you’ve integrated AI into Amplitude’s product
I'll tell you the good and the bad of the story. Good. Actually, no, I'll start with the bad because that's always more fun. It gets more spicy. We launched about a year ago, we launched a capability that we call Ask Amplitude, and the whole idea was instead of interacting with Amplitude and picking and choosing the fields that you want to analyze, what if you could just speak it or type it in and say, here's the question that I have, and Amplitude should figure out how to convert that into the appropriate chart or the appropriate answer. I put it in the bad category because while we thought it would dramatically open up the possibilities and bring more people in, it didn't actually do that. What it did instead was two things.
First, it actually lowered the entry point into the product, so it was kind of like the bootstrapping process that accelerated, but then people went in and they did all the other things. So there was some of that low-value drudgery that I replaced because now you bootstrap faster. Second, it highlighted where there were taxonomy problems. For example, users couldn’t actually express their questions clearly due to data deficiencies in the product. Bad data equals bad answers, and if your data is not structured in the right way, it doesn't matter how you ask the question, you'll never get a good answer.
On the good side of things, we started investing in what we call the data assistant. Now, as data came in, we could start applying rules—some of it's machine learning, some of it's more predictive rather than generative—where we could tell you, here's the data that you have and here are some suggestions to improve it. It even provided a data quality score. For instance, if your data quality score is at 52, it’s obvious that's not good. It showed exactly what you needed to do to improve it. As the data quality scores went up, customer satisfaction went up, usage went up, and people became more successful with the Ask Amplitude functionality. By tackling the data assistant problem first, which was highlighted by users' interactions with Ask Amplitude, we were able to add value and improve the overall experience.
So the integration of AI into our product was very much a learning experience. It was about moving fast to learn and then iterating based on what we discovered.
What advice do you have for those thinking about integrating AI in their products?
The first thing is to focus on the customer problem you're trying to solve. I think we sometimes get so enamored by the technology that we forget the core problems people have that you could solve using this technology. For example, Slack recently introduced some new AI features to address the problem of being overwhelmed by too many messages in various channels. People find it exhausting to keep up. Slack AI can now summarize or get you up to speed on what happened in a channel. So, you start with the problem and then figure out how the technology can come in.
In a different space, like customer service or incident management, you have all these conversations happening, and you need to do a Root Cause Analysis (RCA). AI is great at summarizing all these things and writing the RCA document that you need. It can save 80-90% of the effort. In product analytics, it's similar—you've got tons of data and feedback coming in. AI can help summarize and guide the user to where they need to go, instead of having a human look under every rock to find the problem, which is slow and laborious. AI can automatically find those valuable insights and guide you there.
While AI may not have the right answer every time because you still need human augmentation, you always start with the problem. If analyzing data is too hard or laborious, especially in a complex organization, you need to find ways to get to insights faster. Can AI help with that? Yes. We started experimenting with AI in this area by asking how we can simplify processes, augment humans, and automate tasks. But you've got to start with the problem—fall in love with the problem first, and then figure out how technology can help accelerate solving it.
What’s unique about product management at Shopify?
I sometimes call it "don't give a care less." It's all about not worrying about other people and just doing the right thing. Really, the core of this mantra is that people often self-center themselves. They worry about how they'll be perceived or that they'll say something dumb in a meeting, and as a result, they limit their possibilities. I discovered early in my career that when you stop caring about how you are perceived and focus more on the outcomes, you get dramatically better results.
The "don't give a f***" philosophy really means stop caring about yourself, stop caring about how you'll be perceived, and instead care deeply about the outcomes and things that are outside of you. You'll find it liberating, and it creates a kind of shield because it's not about you; it's about the outcome. This approach leads to dramatically better results.
This whole mantra came to me very early in my career, around when I was 22 or 23. As a new product manager at Microsoft, I was asked to present on the state of the market to the most senior people in the company. They had assembled the top 80 people, and I was supposed to present on business intelligence. I arrived late and didn't know who was in the room. One of the leaders made a comment I disagreed with. I respectfully said, "No, that's actually not true. This is the reality. Here's why," and I presented the facts. People were stunned. They wondered how I could talk that way to a senior leader.
But I spoke the truth, and what's the worst that could happen? They'd fire me? Great. But speaking the truth started propelling me in my career. It opened new doors because I wasn't afraid.
When I started at Tableau, my first six months were uncertain. I kept asking for permission all the time. Then I realized I was self-censoring and decided to just go do the right thing and solve problems. Suddenly, people saw me differently. I was pulled into more meetings, got a founder's award in my first year, and kept getting promoted.
So, if you stop caring about yourself and think more about the outcomes you want to drive, you'll get a lot more success. You'll feel liberated, and you'll achieve better results.
How do you think about experimentation in product management?
Depending on the kind of company you are and the scale or size of the company, you always want to have room for innovation and experimentation. You might have a core part of the product that you're focusing on, and I don't want to call it the "slow lane" because that implies something negative. Instead, I like to think of it as the rigorous lane, where you're a lot more thoughtful about what you do.
In contrast, you need an innovation lane where you can explore, learn, and test out a bunch of hypotheses. That's where you can start integrating new models, especially with AI, as those models are constantly improving. If you don't have a mechanism to move fast, you'll never be able to adapt to changes in the space. Whether it’s infrastructure behind the scenes or user experiences and interaction modalities, there are numerous opportunities. Staying static is a risk in itself and can be a path to obsolescence. You have to continuously learn, evolve, and experiment to uncover real possibilities for your business.
Risk in experimentation is tied to the tolerance or fear of failure. How willing are you to fail? In modern companies, experimentation often involves AB testing and feature or product experimentation. The goal is not just to find what works but to have enough experiments that fail so you learn along the way. Risk-taking is very much tied to managing this fear of failure and adopting a beginner’s mind, a willingness to learn, and a growth mindset.
Every one of us who's built something has experienced these “oh moments”—those instances when something completely unexpected happens, like a traffic spike or a server crash. While they can feel uncomfortable in the moment, they are excellent learning opportunities. If you try to avoid risk and think through all possible scenarios, you’ll be paralyzed with inaction because it’s impossible to think of everything. You have to be willing to take some risks, even if it means stepping into discomfort, as that's where growth happens. Of course, there are areas like security where risks can't be tolerated and need to be handled with utmost priority, but in other areas, a bit of leaping in and dealing with discomfort can lead to significant growth and learning. I've had many "oh moments" in my career, and while they're uncomfortable at the time, they’ve always led to valuable lessons and improvements.
Give us an example of when things went wrong
One particularly challenging incident occurred when we launched Tableau Public, which we envisioned as the YouTube for data. Essentially, it allowed anyone to upload any data or visualization and share it publicly on the internet. Things were going smoothly; the platform was experiencing good traffic and steady growth. Then WikiLeaks happened. Someone took all the cables from WikiLeaks and published it as a data visualization on Tableau Public. Overnight, our traffic spiked by a hundred times, and it seemed like the whole world was pointing at Tableau Public.
Our hard drives were spinning relentlessly, CPU usage maxed out, and we simply couldn't keep up with the demand. This was before the days of cloud computing, so we were still operating on bare metal servers. On top of the technical strain, we started receiving calls from senators telling us we couldn't publish leaked data, along with furious emails from customers questioning our judgment. We found ourselves in a two-week storm of trying to keep the servers running while dealing with officials and irate customers.
Ultimately, this chaos forced us to scrutinize our system's behavior under extreme load and catalyzed several improvements. We developed fail-safes, revamped our terms of service to better define fair use, and refined our response plans for mitigating similar incidents in the future. Interestingly enough, in the grand scheme of our product's growth, what seemed like an overwhelming spike in traffic then is just a typical day's traffic now. That incident significantly raised awareness of our platform, proving to be both a marketing moment and a critical turning point for our resilience and scalability.
How do you feel about product management frameworks?
I have some strong feelings. People often fall in love with their frameworks rather than falling in love with the impact they want to create. Frameworks are meant to help you think through a problem and approach it in different ways. However, an over-reliance on the framework can become an issue when you start celebrating the framework itself instead of focusing on the desired impact. Steve Jobs talked about this as the disease of the process people, those who are so enamored with the process that they forget about creativity, the customer, and the reason behind what they're building.
I constantly battle to ensure there's more focus on the impact and then work backwards from that point. The key is to see how the framework can help you achieve that impact. One framework I love, which is more of a tool really, is the PR FAQ framework that Amazon uses as part of their Working Backwards process. It's a great mechanism for articulating and thinking through problems. It doesn't tell you exactly what to do, but it helps you think critically. I love those kinds of mechanisms—they're super helpful. But again, even if they’re not perfect, what matters is going through and thinking about the customer problem you want to solve.
What do you think the next few years will look like?
I wouldn't be surprised if three years from now, most of the world looks fairly similar to what we have today. However, we've transitioned from AI merely being a topic of conversations to AI becoming an ingredient in every single one of our products. I believe that differentiation in the future will come through the proper use of AI. If you have two tools, one with AI and one without, the AI tool will win every single time. So, as you're building, you have to think about how you're integrating that technology to solve customer problems.
It's going to be exciting to see new companies emerge that disrupt the dinosaurs of the past and come up with a whole new approach to this space, whatever space that may be. You have to have a willingness to disrupt yourself, to innovate, and to move things forward. There's a classic innovator's dilemma: do you want to be the one disrupting, or do you want to be the one being disrupted? I'd rather be the disruptor, the winner, the leader. That's where I want to be.
No matter what industry you're in, it's clear that this technology will have an impact. So, I really push people to adopt a learning mindset, a growth mindset, and lean into the opportunities in front of us instead of shying away and being scared of them. Lean in and focus on solving problems. Those are core aspects of every builder out there. No matter what you're building, you have to get uncomfortable to grow and to innovate. If you already knew where you were going and took the safe path, well, you're not the innovator—you're the follower.
I think what I would tell folks is that success is not guaranteed. The only thing that is guaranteed is your ability to learn, grow, and make an impact. That's in all of us. The more you're open to growing and learning, the more you'll be open to how technologies, whether it's AI or whatever comes next, can enable you to amplify your impact. Whether you're a founder or a product manager, always be learning, always be growing, and always be focusing on how you can have the most impact. Don't ever settle.
A huge thank you to Francois Ajenstat for sitting down with us and sharing his time and expertise. You can follow along with Francois on Linkedin here.
•
Again, we hope you enjoyed this article and got value from it. Please consider sharing this Product Collective email with your colleagues, or head directly to our blog to view this post on the web.