Streamlining Search Quality: Search Relevance Workbench
Session Abstract
Robust Search Evaluation is both a “must have” for any modern day Search team and an “after thought” that never gets the team’s full attention. This is especially true with the various open source search engines. Most teams build their own data collection and eval tools. Some use standalone open source tools. We present a better solution!
Session Description
In this talk we will lay out the history of Search Evaluation, why it’s critical in today’s AI powered world, and make the case for why Search Evaluation needs to be part and parcel of any modern Search Engine. We will share some lessons from building multiple Search Evaluation toolsets, including the popular open source tool Quepid, and why we felt we needed to build the Search Relevance Workbench as an integrated suite. We will show how SRW collects user click behavior using the User Behavior Insights open standard, and how click data is combined with labeled data to measure search quality. We’ll show how you can use that information to run optimizers like Learning to Boost and Hybrid Search Optimizers that replace traditional manually tuned algorithms. You will leave understanding how SRW is different from previous tools, and how you can take advantage of it with your own search engine (not just OpenSearch) as well.
