How Facebook's Three Ad Relevance Rankings Correlate with Performance

A detailed look at how Ad Relevance Rankings correlate with actual performance.

On Facebook, Ad relevance diagnostics compare your ads to ads using the same optimization goal that are competing for the same audience. They’re broken up into three rankings: Quality Ranking, Engagement Ranking, and Conversion Rate Ranking. According to Facebook, they’re made available to help diagnose underperforming ad creative. Historically at Thesis we haven't paid much attention to these rankings, but we wanted to do a more formal analysis to understand how these rankings actually correlate to the performance metrics our clients care about.

An example of how the rankings are presented in the Facebook Ads UI


  • We looked at 10 ad accounts.
  • Date Range: August 22, 2011 - September 18, 2021
  • Ad Relevance Diagnostics are only available for the last 35 days, hence the shorter window. 
  • We only focused on Prospecting campaigns, meaning targeting that excluded any of the typical remarketing audiences (site visitor, add to carts, etc).
  • All of the ad accounts were optimized for web purchase events.
  • Spend data has been rounded to the nearest ten thousand dollars. 
  • Each one of these rankings has five possible values: Above Average, Average, Below Average (Bottom 35% of ads), Below Average (Bottom 20% of ads), or Below Average (Bottom 10% of ads)
  • None of the accounts we looked at had any ads that ranked in the bottom 20% or 10% of ads for any of the three available ranking types. For readability purposes, we’ve removed those rankings from the data below.
  • Notably, we've labeled ads that didn't receive a score as“Unscored Ads." Facebook says that they’re unable to provide scores for any ad that’s received less than 500 impressions. When digging into the data, we found this to be true, but there were a considerable number of ads with well over 500 impressions without any scores. What we did find is these ads lived in ad sets that were either still in the Learning Phase or included some sort of dynamic elements (multiple ad copy variations, asset placement customization, etc). Ads that had exited the Learning Phase or were in Learning Limited and weren’t using any dynamic elements did have a score.

Quality Ranking

In the above table we pulled performance metrics in aggregate for 10 brands, and then pulled those same CTR, CPA and ROAS metrics for the ads labeled as Above Average, Average, or Below Average. We don't see any clear trends based on this data set. We had hoped to to see better CPAs and/or ROAS on higher ranked ads... but that was clearly not consistent. Notably, we did notice that higher Quality Rankings seemed to correlate with lower CPMs. And that does make sense intuitively, as Facebook defines Ad Quality as "How your ad's perceived quality compared to ads competing for the same audience." So they appear to reward ads with higher Quality Rankings with lower CPMs.

Engagement Ranking

Engagement ranking takes into accounts actions such as clicks, reactions, comments, etc. to determine a score. When looking specifically at CPAs, we found that the ads ranked Average tended to have better CPAs and ROAS compared to the Above Average ads. While Above Average ads had a better CTR than Average ads in every account we looked at, we found looking at actual CTR provided more actionable insights, especially when comparing ads with the same Engagement Ranking. 

Conversion Rate Ranking

This ranking compares the expected conversion rate of ads against other ads targeting the same audiences. While these rankings generally correlated with higher conversion rates when we calculated them ourselves, we did find that in quite a few of these accounts ads ranked average often had higher conversion rates than ads ranked above average, even with the same targeting. Given some of those inconsistencies and the large amount of unranked ads in all of these accounts, we found creating a custom metric for conversion rate to be more effective than relying on the Conversion Rate Rankings. 

Unfortunately we were unable to find clear evidence that these rankings are more helpful than other KPIs (like CTR, CVR, Hook Rate etc) for performing day to day ad optimizations, especially when you consider that a large percentage of the ads we analyzed were entirely unranked. We imagine that with a much larger data set (like all Facebook advertisers) these rankings might be very useful, but at Thesis our practical takeaway is to not pay great attention to them.

Get in touch