Back to main page
- Heuristic evaluation and comparative usability testing against a direct competitor
- Task-based testing with 16 participants split between industry professionals and first-time buyers
- Statistical analysis (t-tests, System Usability Scale) and a set of prioritized design recommendations
- Experts made up to ten times more errors than novices when the interface broke the conventions of the tools they use daily.
- First-time home buyers found the platform difficult to use.
- Industry professionals, despite finding the platform complex, preferred it over the competitor.
SUS Score (Novice VS Expert)
New home buyers
41.9Zealty SUS
78.7REW SUS
n=8 · p=0.001 · Significant - novice users strongly prefer REW over Zealty
Expert users
56.6Zealty SUS
78.4REW SUS
n=8 · p=0.015 · Significant - expert users prefer REW over Zealty
A System Usability Scale score of 68 is the general usability benchmark. Zealty fell below it for both groups, but the gap differs by audience. Experts scored closer to the B2B professional-product benchmark of 57, meaning Zealty performs roughly in line with typical professional software. Novices scored well below that, showing the platform's complexity was a much steeper barrier for first-time buyers than for industry professionals.
Novice - Task Time (Zealty vs REW)
Address Search
54.3sZealty
23.7sREW
p=0.088 · Not significant - no reliable difference between Zealty and REW
Amenity Check
25.2sZealty
11.3sREW
p=0.029 · Significant - REW is significantly faster than Zealty
Navigate & Sort
71.9sZealty
33.5sREW
p=0.006 · Significant - REW is significantly faster than Zealty
Complex Filtering
149sZealty
89.3sREW
p=0.010 · Significant - REW is significantly faster than Zealty
Expert - Task Time (Zealty vs REW)
Address Search
54.7sZealty
25sREW
p=0.174 · Not significant - no reliable difference between Zealty and REW
Amenity Check
45.4sZealty
15sREW
p=0.068 · Not significant - REW is faster, but the difference is not statistically reliable
Navigate & Sort
99.5sZealty
60.7sREW
p=0.139 · Not significant - no reliable difference between Zealty and REW
Complex Filtering
161.2sZealty
58.6sREW
p=0.000 · Highly significant - REW is much faster than Zealty
Both groups were tested on four tasks: finding an address, checking property details, navigating and sorting, and filtering. Filtering was the widest gap in the study - task 4 alone shows experts spending nearly three times longer on Zealty than on the competitor.
Novice - Avg Errors per Task (Zealty vs REW)
Address Search
1.38Zealty
0.25REW
Amenity Check
0.38Zealty
0.25REW
Navigate & Sort
0.50Zealty
0.38REW
Complex Filtering
1.50Zealty
0.63REW
Expert - Avg Errors per Task (Zealty vs REW)
Address Search
1.13Zealty
0.63REW
Amenity Check
0.50Zealty
0.00REW
Navigate & Sort
1.75Zealty
0.38REW
Complex Filtering
4.00Zealty
0.88REW
On Task 4, complex filtering, experts made 4.6 times more errors on Zealty than on the competitor. The gap wasn't complexity itself - it was the interface breaking conventions professionals rely on every day.