Vector Similarity Search | Future of Data & AI | Data Science Dojo

Let’s face it, the challenge of search today is indexing billions of entries while delivering relevant results quickly. Traditional keyword-based methods have limitations, leaving us searching for a better way to improve search. But what if we could use deep learning to revolutionize search? Imagine representing data as vectors, where the distance between vectors reflects similarity, and using Vector Similarity Search algorithms to search billions of vectors in milliseconds. It’s the future of search, and it can transform text, multimedia, images, recommendations, and more.

Thank you for reading this post, don't forget to subscribe!

💼 Learn to build LLM-powered apps in just 40 hours with our Large Language Models bootcamp:

Check out this panel discussion, where you will learn how you can incorporate vector search and vector database into your own applications to harness deep learning insights at scale.

—

About the Speakers:

Sam Partee – Principal Applied AI Engineer at Redis
Sam is a Principal Engineer who helps guide the direction of AI efforts within Redis. Sam assists customers and partners in deploying Redis in ML pipelines for feature storage, search, and inference workloads. His background is in high-performance computing and machine learning systems.

Harmke Alkemade – AI Cloud Solution Architect at Microsoft
Harmke Alkemade is a Specialized AI Cloud Solution Architect at Microsoft, working in a global team that supports customers in various industries to use Azure services for high-impact AI use cases. Previously, she worked as a Cloud Solution Architect for Microsoft Netherlands in the Data & AI domain.

Jacky Koh – Founder at Relevance AI
Jacky Koh is the founder of Relevance AI, a low-code platform to helps everyone analyze unstructured data 10x faster with embeddings and large language models. Previous to Relevance, Jacky was an app builder and award-winning data scientist, leading teams to deploy impactful machine learning algorithms.

Daniel Svonava – Co-Founder at Superlinked
Daniel is the CEO of Superlinked – an infrastructure product that helps evaluate, launch, and operate real-time ML personalization for consumer apps. Previously, Daniel was a tech lead at YouTube where he built user modeling and ad performance prediction systems that guide the purchase process for $10 Billion+.

Bob van Luijt – CEO & Co-Founder at Weaviate
Bob van Luijt, CEO and Co-Founder of Weaviate, an open-source vector search engine. At just 15 years of age, Bob started his own software company in the Netherlands. He went on to study music at ArtEZ University of the Arts and Berklee College of Music and completed the Harvard Business School Program.

—

Table of Contents:
0:00 – Introduction
1:38 – What are Embeddings.
3:10 – How to get embeddings?
5:10 – What are vector databases
7:50 – Types of indices, when you use them, and how to get access?
12:50 – How to use indices, and how to combine them with other services.
14:45 – Why is there an increased interest in this space?
23:19 – Day-to-day things used in workflows
25:21 – Contact Center Analytics using Speech API & Open AI
31:20 – Generative search methodology
34:30 – Recommendation systems and how vector search use case
37:40 – Off-the-shelves models for particular use cases
46:26 – One thing you’re excited about in this space

#VectorSimilarity #generativeai #openai #chatgpt #Redis #llm