Session Description

Over the last few years we have been pushing to the limits our full-text search solution for the French Audiovisual Institute. However, some areas of our immense corpus are still inaccessible, either because the multimedia content lacks textual annotations, or because the automatic transcriptions are not self-sufficient for an efficient full-text search.

Semantic search appears as a natural complement, but the scalability of the implementation reveals specific challenges in capacity planning and chunking strategies to accommodate different embedding models.

Nevertheless, when it comes to merging the benefits of both text and vector search methods, the success of the hybrid search approach relies essentially on the reranking algorithm. To address this, we developed an alternative to the reciprocal rank fusion based on our needs, specifically tailored for a multi-index architecture and integrating multiple embedding sets.

In this talk, we share our experience in building an extensible hybrid search solution, covering everything from complex functional modeling to cluster architecture design. Attendees will gain practical insights into handling billions of vectors in real-world scenarios, such as within large graph data structures. Additionally, we will explore the challenges of hybrid reranking, discussing the limitations of standard fusion techniques and the rationale behind our novel approach.

While relevance evaluation is still ongoing, our modular architecture enables continuous iteration, ensuring the adaptability to the rapid evolution of embedding models and vector optimizations. This flexibility positions our solution to remain at the forefront of large-scale semantic search, balancing precision, scalability, and efficiency.