Amazon Marketplace Combines VLMs and OCR on SageMaker AI to Evaluate Product Page Quality

Amazon Marketplace outlined an automated content evaluation pipeline for A+ Content using Amazon SageMaker AI. Because evaluating visual and textual assets in isolation often fails to detect holistic composition issues across an entire product page, the team captures complete page snapshots and analyzes them using Vision-Language Models (VLMs) alongside Optical Character Recognition (OCR) signals. The framework prompts a frozen VLM across four rubric dimensions: Visual Storytelling, Feature Presentation, A+ Layout, and Conversion Optimization. Concurrently, an OCR model extracts text blocks to calculate four layout readability signals: Text Density, Character Density, Spacing, and Overall Readability. These eight features are fed into a lightweight CatBoost classifier trained on 5,478 expert-annotated product detail pages across 10 categories. On a 20 percent holdout test set of around 1,100 pages, the fused OCR-VLM-CatBoost model achieved 0.94 accuracy and an F1 score of 0.95, outperforming standalone VLM-only (0.89 accuracy) and OCR-only (0.82 accuracy) setups. Furthermore, the system incorporates SHapley Additive exPlanations (SHAP) and a large language model to interpret feature contributions and convert negative factors into prioritized recommendations for selling partners.

Key takeaways

• The evaluation pipeline renders full product pages as snapshots to evaluate cohesive layout and narrative rather than grading assets individually.
• A CatBoost classifier combines 4 VLM rubric scores with 4 OCR readability metrics, reaching 0.94 accuracy and a 0.95 F1 score on a holdout evaluation set.
• Using SHAP values to explain classifier decisions, an LLM turns top negative signals into actionable, prioritized content revisions for sellers.