How Relevance Shapes Ad Auctions: Learnings from the FTC Lawsuit and Amazon's Response
How relevance can shape which Amazon ads are selected, how they rank, and what the winner pays, with interactive models.
Say a shopper searches Amazon for wireless headphones. Not every headphone ad is relevant to that search. An ad for wired headphones, for example, is a poor match. Amazon's response to the FTC lawsuit says its ad auctions take both bid and relevance into account. Our note on soft reserves and Amazon ad costs assumed, for simplicity, that ads were ranked by bid alone.
The questions below show how relevance can play a role in ranking (more relevant wireless headphone ads rank higher), pricing, and selecting which ads compete (only headphone ads), and which metrics measure it. The discussion is illustrative and drawn from public disclosures by Amazon and the FTC's recent lawsuit.
How does relevance affect ad ranking?
Amazon says it applies a ranking score to each eligible ad based on relevance and bid. Amazon frames relevance as better for shoppers and advertisers. It also fits the economics of pay-per-click ads: a high bid on an ad that shoppers rarely click earns little. The FTC complaint describes Amazon's relevance factor as based on, among other things, each ad's predicted click-through rate (CTR) and its conversion rate, the share of clicks expected to lead to a sale.1
Amazon does not publish its ranking formula. Our examples use a simple, standard rule from a 2023 paper by Amazon researchers, which builds on a seminal 2007 study of search ad auctions by economists Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz. We do not know whether Amazon uses this rule, and its actual score likely includes more factors.1
The score is what the ad would earn per impression if it paid its full bid. A $1.00 bid with an 8% CTR scores 8 cents. Picture each ad's score as a box, with the bid as its height and the CTR as its width. The score is the box's area, and the biggest box ranks first.
The ad with the highest bid often does not win. Amazon says that in 2024, "approximately 92% of selected Sponsored Products ads were not the highest bid," and that the average winning advertiser's bid typically ranked around 128th by amount. Amazon also says its formula has given more weight to relevance over time.2 Ranked by bid alone, the highest bid always wins. Ranked by bid and relevance, an ad can win with a higher bid, a higher CTR, or both, and a more relevant ad can beat a higher bid.
How does relevance affect what the winner pays?
An ad auction can charge the winner in one of two ways. In a first-price auction, the winning advertiser pays its bid. In a second-price auction, it pays just enough to beat the runner-up, which with bid-only ranking is the runner-up's bid. Amazon says it has priced clicks with a form of generalized second-price auction since it introduced ads in 2006. Once relevance enters the ranking, it enters the second price too: the winner pays the least it could have bid and still ranked first. In the box picture, that means lowering the winning box's height from its full bid until its area just matches the runner-up's box. The width, its CTR, stays the same. The height where the two areas match is the price per click:
In this simplified model, three patterns stand out:
- Your bid decides whether you win, and the runner-up sets the price. The winner's price depends only on the runner-up's score and the winner's own CTR. The winner's bid never enters the formula.
- Higher relevance means a lower price. Raise A's CTR, and its price falls even though its bid stays the same.
- The price is always below the winning bid. The winner's box started out bigger than the runner-up's, so it can always shrink a little and still match it.
When the top ads have similar CTRs, the CTRs cancel out and the price comes back to the runner-up's bid, the same price as ranking by bid alone.
Switch to first price to see the difference. The winner pays its own bid, so relevance decides who wins but does not set the price. At the same bids, first price always costs more. Advertisers who pay their own bid have a reason to bid below their value, a practice called bid shading, so in practice the gap is smaller than a same-bid comparison suggests.
The FTC alleges that soft reserves made Amazon's auctions closer to first price, with Sponsored Products advertisers paying their own winning bid close to 80 percent of the time by 2024. Amazon confirms it uses soft reserves and denies misleading advertisers. Our note on soft reserves and Amazon ad costs discusses this in detail.3
What metrics measure ad relevance?
Amazon does not disclose which relevance measures go into its ranking score or how they are weighted. Several kinds of metrics could be used. The FTC complaint names two: predicted click-through rate and conversion rate. Amazon describes relevance models that predict how likely a shopper is to find an ad useful, considering factors such as the content around the ad placement, the shopper's search, and how likely the shopper is to engage with the product.4
Both rates are predictions. The auction has to estimate them before the ad is shown, typically using historical data such as how often shoppers clicked the ad, and bought after clicking, on similar searches.
In pay-per-click search ads, CTR has traditionally been the main relevance measure because it ties directly to ad revenue. A platform paid per click earns the price times the chance of a click on each impression, so an ad that shoppers rarely click earns little no matter how high its bid. Conversion rate tracks what shoppers and advertisers care about most, a sale, but it does not directly change what the platform earns per impression.
How does relevance decide which ads enter the auction?
Relevance plays a role before any ranking happens, when Amazon decides which ads may compete. Amazon says that for a single search, potentially hundreds of thousands of ads could compete to be shown, and that it narrows them to fewer than a thousand candidates by removing less relevant ads.2 To include and exclude ads, Amazon uses advertisers' targeting clauses, the keywords, products, and categories they choose to target, and potentially other relevance measures.
✓ Kept as candidates
✕ Removed
Targeting clauses come in a few forms, and some leave the relevance call to Amazon:
- Keyword targeting. The advertiser picks the keywords. Exact match requires the search to match the keyword word for word, or its plural. Phrase match also allows longer searches that contain the keyword in order. Broad match can include synonyms and related terms, so Amazon decides which searches are close enough.5
- Automatic targeting. Amazon picks the searches and products, based on previous shopping queries and the product's information.5 Amazon decides whether a search is relevant to the product.
- Product detail pages. There is no search to match. Advertisers can target specific products or categories, and Amazon says keyword targeting can also help products appear on detail pages. For those ads, Amazon decides which ones fit the page.
Which ads make the cut matters for price as well as placement: in a second-price auction, the runner-up sets what the winner pays.
Notes and sources
1. The scoring rule. Amazon says it applies a ranking score based on relevance and bid, and the FTC complaint (paragraph 99) says the relevance factor is based on, among other things, predicted click-through rate and the conversion rate of clicks that result in sales. Neither discloses the formula. Our examples use the rule from Guo, Li, Nabi, Salhab, and Zhang (2023), Amazon researchers who model a second-price auction in which the winner has the highest bid times CTR and, when clicked, pays the second-highest score divided by its own CTR. They cite Edelman, Ostrovsky, and Schwarz (2007) for this auction. We do not know whether Amazon's auction uses this rule. A real ranking score may weigh conversion and other factors. The pricing logic is the same for any score of the form bid times a relevance factor: price per click equals the runner-up's score divided by the winner's relevance factor. ↩
2. Amazon's figures. From Amazon's public response to the FTC lawsuit. The 92% figure is for selected Sponsored Products ads in 2024. Amazon also says the mean winning advertiser's bid is typically about the 128th bid by amount, and that a single search could draw potentially hundreds of thousands of competing ads, which it narrows to fewer than a thousand candidates by removing less relevant ads before applying a ranking score. ↩
3. Soft reserves and simplifications. The FTC complaint (paragraph 76) alleges that soft reserve surcharges led Amazon to charge Sponsored Products advertisers their own winning bid close to 80 percent of the time. Amazon's response to the lawsuit says it has priced clicks with a form of generalized second-price auction since 2006, confirms that its auctions use hard and soft reserves, describes a soft reserve as a real-time minimum that seeks to reflect what each placement is actually worth, and denies misleading advertisers. The interactives leave reserves out, use one placement and two advertisers, ignore the one-cent increment, and treat CTR as the auction's prediction before the ad is shown. With several placements, search engines use a generalized second-price (GSP) auction, in which each ad's price is set by the ad ranked just below it, and bidding your full value is no longer always the best strategy (Edelman, Ostrovsky, and Schwarz 2007). Values are hypothetical. ↩
4. Relevance measures. The FTC complaint (paragraph 99) says the relevance factor is based on, among other things, predicted CTR and the conversion rate of clicks that result in sales. Amazon's response describes machine learning relevance models that predict how likely a shopper is to find an ad useful and consider factors including the content around the ad placement, the shopper's search query, and the likelihood that the shopper engages with the advertised product. Neither lists every measure or how each is weighted. ↩
5. Targeting options. From the Amazon Ads guide to Sponsored Products targeting. Amazon says automatic targeting is based on previous shopping queries and the advertised product's information, and matches ads to keywords and products similar to that product. It says exact match matches the search word for word and includes plurals, phrase match includes plurals, and broad match may include singulars, plurals, variations, synonyms, and related terms. It says keyword targeting can help products appear in shopping results and on product detail pages, and that product targeting lets advertisers target specific products, categories, brands, or other product features. Amazon does not say how it decides which keyword-targeted ads fit a given detail page. ↩
Sources. Amazon's response to the lawsuit; FTC v. Amazon complaint, W.D. Wash. No. 2:26-cv-03097, paragraphs 76 and 99; Amazon Ads, "Targeting with Sponsored Products"; Guo, Li, Nabi, Salhab, and Zhang, "MESOB: Balancing Equilibria & Social Optimality", MarbleKDD 2023 (Amazon); Edelman, Ostrovsky, and Schwarz, "Internet Advertising and the Generalized Second-Price Auction: Selling Billions of Dollars Worth of Keywords", American Economic Review 97(1), 2007.