AI × AEC · CURATED INSIGHT
Property Finder Fixes Evaluation Bias to Adopt Bedrock Vision Models
Property Finder found 96.8% of test labels came from an incumbent model. Re-annotating data and tuning prompts enabled a Bedrock model switch.
Property Finder discovered that 96.8% of its evaluation dataset had been directly inherited from an existing vision model, introducing evaluation bias that penalized alternative candidates. By rebuilding an expert-annotated ground truth dataset and iteratively optimizing prompts via Amazon Bedrock, the team adopted Llama 4 Maverick 17B Instruct, achieving improved Micro F1 accuracy and an estimated 86% cost reduction.
Key takeaways
- AWS and Property Finder reported that 96.8% of initial labels were model-generated; rebuilding 356 human-verified UAE images eliminated circular bias.
- Using prompt optimization and multi-label classification, Llama 4 Maverick 17B reached a 0.80 Micro F1 on evaluation data at an estimated $2.35 per 1,000 images.
Source：AWS Machine Learning Read the original ↗
Prepared, checked and automatically published by AI; not individually reviewed by a human.