Search ranking test
Compares ranking search results by text relevance, by sales velocity, or by profit margin, scored on revenue after a search.
Authored by Sid Chaudhary
Founder & CEO at Intempt
Agentic workflow
Coming soon1 step Blu runs end to end. It drafts a plan and nothing runs until you approve it.
- 1
Set up the ranking test
Coming soonBlu builds a step
About this recipe
When to use it
Search ranking logic directly shapes what people buy, and relevance-only ranking often leaves revenue on the table. This tests relevance, popularity-weighted, and margin-weighted ranking against each other.
What it delivers
Runs a server-side experiment controlling the search backend, testing relevance-based, popularity-weighted, and margin-weighted product search ranking.
How you prompt this
Describe the outcome in your own words and Blu Agent maps it to this recipe, or run it by name. It drafts a plan first and nothing runs until you approve it.
Tell Blu Agent what you want
“Compares ranking search results by text relevance, by sales velocity, or by profit margin, scored on revenue after a search.”
recipe.md preview
Built from the published catalog. Blu Agent's full step prompts load in your workspace when you run it.
Inputs & outputs
What the recipe needs before it runs, and what it leaves in your workspace.
Inputs
- Nothing declared beyond an Intempt workspace. Blu asks for anything else it needs as it goes.
Outputs
How the recipe runs
Blu follows the steps in order and shows you each one before anything goes live.
- 01Set up the ranking testCreate a SERVER EXPERIMENT on /experiences titled "Search Ranking Optimization". ═══ PATH 1: Top-level configuration ═══ Experience type: server_experiment Flag Key: search_ranking_strategy Variants: - Control (34%): relevance-based (text match score only) - Variant B (33%): popularity-weighted (relevance × sales velocity) - Variant C (33%): margin-weighted (relevance × profit margin) Targeting: - Pages: any page where the search backend is called: typically search results pages (page URL contains "/search") and inline search-as-you-type widgets in the header - Devices: any (server-side search is device-agnostic; the ranking algorithm runs in the search backend regardless of client device) - Audience: users performing product searches (search query string is non-empty, OR Submit on the search form) - Display frequency: always (every search query call invokes getFlag and applies the assigned variant's ranking) Primary metric: goal_completed_in_experience where experience_id = <this> AND value > 0 (revenue from Placed order within session of a search) Secondary metrics: - Click on on search results (search-to-click conversion) - Average click position (which rank (1, 2, 3) got clicked) - Revenue per search (sum of Placed order.total_price / count of distinct searches) - Zero-result rate (searches where no Click on followed within 30 seconds) Guardrail: search-to-purchase conversion (the existing aggregate metric) must not drop >5% in any variant. Note: variant C (margin-weighted) is a known accuracy/profit tradeoff: guardrails catch it if the accuracy drop is too steep. Schedule: 21 days, 5,000 searches per variant minimum ═══ PATH 2: Variant JSON payload ═══ Variant: Control (relevance) { "ranking_strategy": "relevance", "weights": { "text_match_score": 1.0, "popularity": 0.0, "margin": 0.0 }, "max_results": 24, "tie_breaker": "alphabetical" } Variant: B (popularity-weighted) { "ranking_strategy": "popularity_weighted", "weights": { "text_match_score": 0.6, "popularity": 0.4, "margin": 0.0 }, "max_results": 24, "tie_breaker": "popularity" } Variant: C (margin-weighted) { "ranking_strategy": "margin_weighted", "weights": { "text_match_score": 0.7, "popularity": 0.0, "margin": 0.3 }, "max_results": 24, "tie_breaker": "margin" } SDK integration: const ranking = await intempt.getFlag('search_ranking_strategy', userId); const results = await search.query(query, { weights: ranking.weights, max: ranking.max_results }); Each rendered search result should have target_id = "search-result-{position}" so Click on per-position metrics roll up consistently. Taxonomy notes: - The search backend (search.query) is application-side and not part of canonical taxonomy. - "Click position" is the rank of the clicked result: typically captured as target_id ("search-result-3" to position 3) or as a property on Click on. - "Margin" data is required for variant C; if your product catalog doesn't track per-product margin, variant C cannot run faithfully. - "Display frequency: always": every search query for a given userId returns the same variant (sticky assignment) for the experiment's duration. The variant is computed once on first search and cached.
Install Intempt and run it
- 1
Sign up
Create a free Intempt workspace.
- 2
Connect your data
Add the sources this recipe reads from.
- 3
Ask Blu
Type /search-results-ranking-test in Blu once this recipe is live there.
- 4
Review and approve
Blu builds each step and waits for your OK before anything goes live.
Related recipes
Sid ChaudharyDiscovery call qualification fieldsGenerates an AI summary for Discovery calls from meeting data using the qualification framework in the summary recipe.View recipe Sid ChaudharyPost demo nurtureKeeps a demo warm for 90 days with a playbook, a case study, an ROI calculator and a customer story, and pulls the AE in early if they bite.View recipe Sid ChaudharyReturn rate by categoryShows which product categories get returned most, and which ones have got worse since last quarter.View recipe 