📝 Guest Post: Democratizing Vector Databases: Empowering Access & Equality*
Was this email forwarded to you? Sign up here In this guest post, Yujian Tang, Developer Advocate at Zilliz uncovers the true meaning behind democratizing a vector database and its profound implications to promote accessibility, equality, and inclusivity. The 21st century is all about the democratization of technology. The internet boom enabled large-scale collaboration leading to open source becoming a typical software adoption pattern. As the pace and scale of technological innovation grow, we must work to make it more accessible. As a software engineer, democratization of technology means making it as widely available as possible. It means using what I know to make creating, adopting, and understanding technological advances easier for others. Here at Zilliz, we have always been about accelerating the adoption of vector databases, not just about increasing adoption of the open source project Milvus. In this article, I’m going to cover:
What Does Democratizing Vector Databases Mean for Devs?Whenever I hear “democratize” in the context of “democratizing XYZ technology,” I think of expanding access to that technology. So when it comes to vector databases, I think of expanding access to vector databases. Traditionally, vector databases have only been available to software developers at enterprises. Milvus began the process of democratizing vector databases when it became an open-source Linux Foundation project. It was one of the first vector databases available to developers through being an open-source project. As the project has grown, more and more developers have been able to use, learn about, and contribute to vector databases. Pillars of Technology DemocratizationDemocratization of technology comes with specific challenges — especially the democratization of complex tools like vector databases. There are three pillars to look at when it comes to democratizing technology. They are education, increasing accessibility, and evangelism. Education on Vector Databases and Related ToolingEducation is the most crucial topic that many companies often get wrong. Education is about educating on your specific product, the technology at large, and related tools. That’s why here at Zilliz, we create content about many things, not just Milvus. The content we write reflects our desire to accelerate the adoption of vector databases through education. Therefore, we have content about essential concepts like Hierarchical Navigable Small Worlds (HNSW), scalar and product quantization, and inverted file indices. In addition to providing resources for understanding the concepts behind vector databases, we must provide resources for related tooling. For example, the popularity of large language models (LLMs) has ushered in a range of new tools. Some new tools that have come to the forefront include LlamaIndex, Auto-GPT, and LangChain. Additionally, in alignment with our goal to provide educational resources for the community, many of our content pieces go out to third parties, such as TheSequence, The New Stack, and some Medium publications. Increasing Accessibility to Vector DatabasesWhile providing education about technology is excellent, it’s not helpful unless you offer ways to access it. In our case, open-sourcing Milvus was the first step to increasing access to vector databases. Moving beyond simply being open source, the Milvus project has also pursued other avenues of increasing accessibility. Milvus is also available through Docker images with templates for Docker Compose and Helm. In addition, we recently made it available through In addition to Milvus, Zilliz has worked to increase accessibility as well. Initially, Zilliz provided $400 in free credits and now offers a free tier that allows up to half a million vectors! That’s enough for pretty much any developer. With Zilliz Cloud's free tier, almost any developer can get started with vector databases — for free. Technology EvangelismThe last pillar to address in democratizing technology is to evangelize it. What use is making technology available and providing education about it if you don’t tell people why it’s useful? In terms of accelerating adoption, education explains the how, and increasing the accessibility accounts for the what — evangelizing provides the why. We do evangelism mainly through content that shows the power of vector databases. You can see this through some of the educational material I provided above. We also give talks about vector databases. Some virtual and some in-person talks. I recently gave a talk in Seattle about the use of vector databases as a solution to solving data problems with LLMs. Summing Up: Democratizing Vector DatabasesDemocratizing vector databases is critical because vector databases solve many problems in unstructured data. They were previously only available to developers at large companies due to the sheer complexity and scale of such a project. However, the popularity of LLMs has thrust the idea of vector databases into the mainstream and given rise to countless use cases that they didn’t have before. This makes democratization even more critical. At Zilliz, we approached democratization with three pillars — education, accessibility, and evangelism. Education is the most crucial part of these pillars for us. For developers, educational material provides the “how” to use vector databases and complementary tools. Additionally, we’ve always worked to increase accessibility and continue to do so. Open-sourcing the software was the first step. Other steps to increase accessibility include providing templates and images for containerization. We recently released Milvus Lite, a vector database that can run directly in your Jupyter Notebook. Finally, we engage in technical evangelism to spread the word about the use and power of vector databases. We do this through providing webinars, speaking at community events, and being present on social media. Zilliz continues to make efforts to democratize vector databases and this is exciting because of how important they are. Vector databases are critical for solving data problems in LLMs and are the best existing solution for things like reverse image search, semantic text search, and product recommendations. I’m personally excited to be part of a team who’s helping to grow the vector database space and look forward to all the amazing things being built for and by the community! *This post was written by Yujian Tang, Developer Advocate at Zilliz, exclusively for TheSequence. We thank Zilliz for their ongoing support of TheSequence.You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Key phrases
Older messages
Yann LeCun's Vision Starts Materializing
Tuesday, June 20, 2023
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
📝 Guest Post: Achieving real enterprise outcomes with GPT-You, not GPT-X*
Tuesday, June 20, 2023
Introducing Snorkel's Foundation Model Data Platform
Edge 299: A Taxonomy to Understand Tool-Augmented Language Models
Tuesday, June 13, 2023
What are the different ways to augment LLMs with tools.
📝 Guest Post: Enhancing ChatGPT's Efficiency – The Power of LangChain and Milvus*
Monday, June 12, 2023
In this guest post, the Zilliz team lists the challenges of using ChatGPT and explores how to enhance the intelligence and efficiency of ChatGPT to overcome the obstacles of hallucinations. While
Edge 297: Tool-Augmented Language Models
Monday, June 12, 2023
Can LLMs master knowledge tools?
You Might Also Like
PD#572 Good Ideas in Computer Science
Sunday, May 5, 2024
Ideas every programmer likes and why Garbage Collection and Object Oriented Programming don't count
RD#454 API Layer & Fetch Functions
Sunday, May 5, 2024
ixing API and UI code quickly leads to messy and unmaintainable code
The Shiny Toy Syndrome & Tiny macOS utility apps I love
Sunday, May 5, 2024
Lex launching its redesign, Raycast shares another monthly update packed with AI updates, prompts should be designed not engineered, and a lot more in this week's issue of Creativerly. Creativerly
Hyundai antes up $1B for AV startup Motional and Elon unplugs the Tesla Supercharger team
Sunday, May 5, 2024
Plus, layoffs come for Luminar, Fisker and Ola View this email online in your browser By Kirsten Korosec Sunday, May 5, 2024 Image Credits: Motional Welcome back to TechCrunch Mobility — your central
C#504 Adventures serializing absolutely everything in C#
Sunday, May 5, 2024
A fantastic journey porting Newtonsoft.Json to System.Text.Json
Sunday Digest | Featuring 'Which City Has the Most Billionaires in 2024?' 📊
Sunday, May 5, 2024
Every visualization published this week, in one place. Visual Capitalist Sunday Digest logo May 5, 2024 | View Online | Subscribe | VC+ The Best of This Week's Visuals Presented by Voronoi: The
The dark side of startup accelerators
Sunday, May 5, 2024
Plus: No easy solution to AI hallucinations View this email online in your browser By Anthony Ha Sunday, May 5, 2024 Image Credits: Bryce Durbin This Week, TechCrunch dug into the struggles at two
Android Weekly #621
Sunday, May 5, 2024
View in web browser 621 May 5th, 2024 Articles & Tutorials Sponsored Genius Scan SDK: a document scanner in your app Embed a reliable document scanner with OCR in your app, enabling your customers
This Week's Daily Tip Roundup
Sunday, May 5, 2024
Missed some of this week's tips? No problem. We've compiled all of them here in one convenient place for you to enjoy. Happy learning! iPhoneLife Logo View In Browser Your Tip of the Day is
NativePHP now supports Windows! - №511
Sunday, May 5, 2024
Your Laravel week in review ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏