11-07, 10:55–11:35 (US/Eastern), Winter Garden
Learn how we built a lightning-fast search engine using Python, balancing speed, relevance, and scalability. In this session, we’ll explore our hybrid approach, blending vector search with traditional keyword indexing to deliver high quality, accurate results. Discover how we harness a high-performance NoSQL database for efficient data management and fine-tune our results with a re-ranking algorithm for top-notch accuracy.
We’ll dive into the hurdles we overcame, like ensuring data consistency in a NoSQL setup, balancing search precision and performance, and designing a scalable architecture. By the end, you’ll understand how this Python-powered engine works, its real-world applications, and the innovative solutions that set it apart.
In this talk, we'll dive into some cool aspects of building a search engine, covering:
-The essential components of a hybrid search engine and how they work together
-The trials and tribulations of scraping data
-How we break down and vectorize documentation to make it searchable
-Building an inverted index
-The role of TF-IDF (Term Frequency-Inverse Document Frequency) in determining keyword importance within a document
-Proximity scoring, which helps us choose and rank keywords more effectively
-A look at reranking algorithms and combining hybrid results
And of course, we’ll bring it all together using Python.
No previous knowledge expected
Director of Developer Advocacy
www.linkedin.com/in/artdanderson
Art is a passionate tech enthusiast, builder, and lifelong learner with a knack for simplifying complex concepts through real-world applications. With a diverse background spanning tax and accounting software, convolutional neural networks in machine vision, and NoSQL databases, Art excels in teaching and demonstrating how systems connect. Whether tinkering with tech or creating innovative solutions, Art’s unique perspective bridges the gap between understanding and application.