Mixture of Encoders: A Vector-Native Approach to Search
Session Abstract
Mixture of Encoders is a vector-native alternative that models both structured and unstructured data in a unified embedding space. We will introduce the method, show how it powers natural language search and real-time recommendations, and share open-source tools and benchmarks for replacing complex hybrid stacks.
Session Description
Filters, hybrid search, rank fusion, re-ranking. Most retrieval systems today are stitched together from separate components, each tuned in isolation. There is no systematic way to integrate structured data into vector search. Ask anyone maintaining a mature Elasticsearch deployment with 100+ boosts and hand-written scoring logic whether they can still evaluate retrieval quality end to end and iterate quickly. The answer is almost always no.
To address this, you need models that understand both your unstructured and your structured data. That includes numeric, categorical, relational, spatial, and temporal metadata, all of which are critical for powering modern search, recommendations, and agentic retrieval systems. These signals drive both end-user precision and business impact. At M&S(Marks and Spencer), we solved this problem using a set of custom pipelines, but the process required significant development effort and lacked a unified framework. There is a better way.
We call our approach the Mixture of Encoders. It is a vector-native alternative to hybrid search that brings structure to retrieval by embedding each data type with a specialised encoder and composing them into a unified vector space. Text, images, categories, numerical features, and contextual signals all become searchable through a single query. This enables nuanced, real-time retrieval across modalities without relying on filters or post-processing stages.
In this talk, we will introduce the technique and show how it supports natural language query decomposition, dynamic modality weighting, and session-aware ranking, all within a single retrieval step. We will share how this approach has been deployed in production, powering retrieval in high-churn environments and contributing over $10M in incremental revenue through improved discovery and recommendation quality. To support adoption, we are also releasing open source datasets for benchmarking real-world information retrieval tasks, along with open source demo implementations that show how to apply the Mixture of Encoders to your own data and use cases.
—
This talk is sponsored by Superlinked.
