Two luxury jewelry sites. One identical task. An unmoderated comparative usability benchmark that found the numbers pointing in opposite directions, and asked why.
Blue Nile and Brilliant Earth both occupy the luxury jewelry space with similar product categories, comparable price points, and a shared customer: someone shopping for something that matters. That made them a fair pair for a head-to-head unmoderated usability benchmark. Same task. Same participants. Same conditions. Anything different in the data comes from the sites themselves.
High-consideration e-commerce has almost no margin for confusion. A user shopping for a platinum necklace as a gift isn't browsing casually. They have a specific goal and limited patience for a site that gets in the way. This study measured exactly where each site helped or hurt that experience, using behavioral data rather than self-report alone.
| PARAMETER | DETAILS |
|---|---|
| Method | Unmoderated remote usability study. Participants completed tasks independently with no moderator present |
| Platform | Loop11 (participants required to install browser extension) |
| Sites Tested | Blue Nile (bluenile.com) and Brilliant Earth (brilliantearth.com) |
| Participants | 19 recruited · 15 fully completed · 4 incomplete. Pool included family members, classmates, and friends. |
| Throwaway Task | Find the company's physical store locations page. Used to orient participants before the comparison task |
| Comparison Task | You are shopping for a gift. Find a necklace made of platinum with diamonds and sapphires. Add it to your cart and note the price. |
| Task Type | Standard (URL-based start) · Page limit: 5 per site · Success criteria: shopping cart URL or specific product URL (Any rule) |
| Measures | Success rate, fail rate, abandon rate, lostness, page views, task duration, SUS score, click path analysis |
| Course | UX 60502 · User Experience Processes and Practice · Dr. Cathy Smith · December 2025 |
19 participants is small, and the pool skewed toward family members in one geographic area and classmates whose engagement patterns weren't consistent with real users. Loop11's browser extension requirement likely deterred participants and introduced selection bias toward more tech-comfortable users. Results should be read as directional benchmarking, not a definitive usability verdict. SUS respondent counts also differed: 17 for Blue Nile, 15 for Brilliant Earth, so scores aren't from equal sample sizes.
Brilliant Earth scored higher on SUS (67 vs. 64), lower lostness (0.35 vs. 0.47), and faster task completion for successful users (1:19 vs. 2:03). By those numbers, it's the more intuitive site. But it also had a 26% abandon rate versus Blue Nile's 10%, and a 15-point gap in success rate. The SUS scores don't reflect those gaps at all.
The users who succeeded on Brilliant Earth navigated it efficiently, but more than a quarter of participants gave up before getting there. A strong SUS score from the finishers doesn't speak for the ones who didn't make it. That's the core tension of this study, and it's a useful argument for why behavioral metrics and self-report scores need to be read together, not as substitutes for each other.
On Brilliant Earth, the average abandon time was 2:10, longer than the average successful completion time of 1:19. Participants who gave up weren't confused and leaving immediately. They were trying, getting stuck somewhere mid-task, and eventually quitting. That's a different problem than a confusing entry point. It points to something breaking down deeper in the flow.
| Metric | Blue Nile | Brilliant Earth |
|---|---|---|
| Success rate | 78% | 63% |
| Fail rate | 10% | 10% |
| Abandon rate | 10% | 26% |
| Lostness | 0.47 | 0.35 |
| Avg page views (all) | 8.1 | 6.6 |
| Avg page views (success) | 8.8 | 7.6 |
| Avg page views (fail) | 5.0 | 4.5 |
| Avg duration (all) | 2:14 | 1:23 |
| Avg duration (success) | 2:03 | 1:19 |
| Avg duration (fail) | 3:39 | 1:21 |
| Avg duration (abandon) | -- | 2:10 |
| Avg sec/page (success) | 13.97 | 10.39 |
| Avg sec/page (fail) | 43.8 | 18.0 |
| SUS score | 64 (n=17) | 67 (n=15) |
| Typical success path | 4 clicks | 6 clicks |
Lostness score per Loop11 dashboard screenshots. The written analysis document cited 0.42 for Blue Nile; Loop11 screenshots show 0.47. Screenshots are the authoritative source.
The most common failure on both sites was landing in a customization section instead of pre-designed jewelry. Neither site clearly separates these pathways at the category level. Users expecting to browse ready-made pieces followed labels that dropped them into build-your-own flows, a fundamentally different task.
Most common fail path on both sitesBrilliant Earth's successful users relied heavily on filters to reach the task URL. The same page URL appeared multiple times in click streams as participants adjusted settings without navigating away. When filters didn't surface the right result, there wasn't an obvious fallback, which likely contributed to the high abandon rate.
6-click avg success pathBrilliant Earth successful users were faster (1:19 vs. 2:03) with fewer page views (7.6 vs. 8.8). But Brilliant Earth failures spent less time per page (18 sec vs. 43.8), suggesting different failure modes. Blue Nile failures involved exploratory clicking across more pages. Brilliant Earth failures got stuck and stopped.
43.8 sec/page on Blue Nile fail vs. 18 sec on BE failThe close SUS scores (Blue Nile 64, Brilliant Earth 67) don't capture the 16-point gap in success rate or the abandonment difference. SUS reflects how the people who finished felt. It has no mechanism to account for the ones who left. Using SUS alone here would lead to the wrong conclusion.
3-point SUS gap; 15-point success gapBrilliant Earth abandoners averaged 2:10 before quitting, longer than the 1:19 average for successful completions. They weren't bailing immediately. Something downstream caused them to give up, likely a navigation dead end or a filter interaction that didn't return useful results.
Abandon avg 2:10 vs. success avg 1:19Blue Nile's 4-click success path suggests more direct navigation to product or cart. 93% of successful participants ended at the shopping cart URL. Whether this reflects better search, better IA, or better category labeling isn't determinable from this study alone, but the shorter path correlates with higher completion.
14/15 successful users hit the cart URL (93%)Navigation categories that separate custom jewelry (build-your-own, design studio) from pre-designed pieces. On Brilliant Earth, the fail path URL is explicitly /jewelry/necklace/diamond/design-your-own/, a legitimate section of the site for a different user goal. On Blue Nile, the fail path included a search results page filtered to all-diamond necklaces with no material refinement.
A filter-forward browsing experience where users can narrow by metal, stone, price, and style. Successful participants used this system effectively. The repeat page URL appearances in click streams show users adjusting filters without leaving the page. When it worked, it worked efficiently (7.6 avg page views for success).
Blue Nile's successful users took more pages (8.8 avg) and more time (2:03) than Brilliant Earth's successful users, but they also succeeded more often (78% vs. 63%). The fail pattern is also different: Blue Nile failures spent 43.8 seconds per page, versus 18 seconds for Brilliant Earth failures. Blue Nile users who failed were exploring. Brilliant Earth users who failed were stuck.