Computer vision in retail does a handful of jobs well: shelf audits that catch out-of-stocks, planogram violations and price-tag discrepancies, product recognition and skip-scan detection at checkout, loss prevention, and locating products for inventory and for shoppers. The use cases are not where pilots fail. Production is: a model that hits 95% accuracy in a demo drops to 70-80% in a real store with uneven lighting, crowded shelves and motion blur, according to Azilen (2026). In synthetic computer vision retail/FMCG projects, pretraining on synthetic data then fine-tuning on real shelf images, domain adaptation on store footage, and drift monitoring once live are the engineering steps that address that gap.
Computer vision retail use cases, from shelf to checkout
Shelf monitoring automates the shelf audit. Cameras (fixed, robot-mounted or handheld) capture product arrangements, and models flag three kinds of event: empty shelf positions, products in the wrong place against the planogram, and displayed prices that do not match the system price. The value comes from replacing scheduled manual walks with continuous in-store camera analytics through the trading day. Our write-up on what shelf monitoring detects and misses goes into event types and phantom inventory in more depth than this page does.
At the till, vision systems recognise products, verify transactions and detect skipped scans that drive retail shrinkage. Cashierless formats push the same idea further: Amazon’s Just Walk Out operated in more than 360 third-party locations across 5 countries as of January 2026, as reported by Codewave. Loss prevention uses anomaly models for behaviours such as concealment and scan manipulation, covered in our piece on where loss-prevention vision breaks.
Behind the shop floor, the same recognition stack supports inventory tracking. In a deployment Codewave reports, BJ’s Wholesale Club ran Simbe’s Tally system and saw 60% fewer out-of-stocks, 90% fewer pricing and promotion errors and 50% shorter e-commerce fulfilment times. Those are results Codewave reports for one retailer’s estate, useful as a sense of what shelf-level visibility can deliver, not a forecast for yours.
Why a 95% demo becomes a 70-80% store
The accuracy drop is not a sign that the technology is wrong. Off-the-shelf models rarely perform well without fine-tuning on footage from the retailer’s own stores, catalogue and conditions, and Azilen (2026) names skipping that step as where most pilots stall. A model trained on clean studio images struggles in real operating environments, which bring different lighting, orientations, packaging variants and partly hidden items.
Scale makes this worse in a specific way. A production system across a full catalogue meets a class of failures a small proof of concept does not expose. The failures compound: severe class imbalance across thousands of SKUs, near-identical variants, edge hardware that cannot run heavy models at the throughput a store needs, and a steady flow of products the model has never seen.
The scale failure modes, and the engineering response to each
| Failure mode | How it shows up | Engineering response |
|---|---|---|
| Thousands of SKUs | Medium-small shops already carry several thousand distinct items, many with small differences between and within classes (Tonioni et al., 2018) | Product-agnostic detection plus an embedding database, so recognition scales with the reference set rather than the detector’s class list |
| One reference image per product | Commercial product databases usually hold one or a few studio-quality images per product, which look different from a shelf photo taken with a cheap camera | Augmentation that turns the reference image into realistic shelf-like samples, and synthetic pretraining |
| Visually similar variants | Variants with minor visual cues are hard to tell apart, and similar packaging confuses models | Learned, retail-specific descriptors and refinement stages, which outperformed a general-purpose descriptor in the Tonioni et al. pipeline |
| Class imbalance | Long-tailed product counts bias training toward common items | Weighted sampling so under-represented products appear more often in each batch |
| Edge hardware limits | The few existing deep learning approaches use heavyweight models unsuited to edge devices (product recognition on racks paper, 2022) | Lighter architectures: a 42M-parameter localiser plus an 11M-parameter recogniser, against 62M plus 138M for an earlier YOLO and VGG-16 pipeline |
| New and unknown products | Store assortments and packaging change frequently, so a fixed class list goes stale | Add new items to the reference database without retraining the detector, and output “Unknown” when conditions fall outside training data |
The last row is the one we would test hardest before rollout. An off-the-shelf classifier assigns every detection to some known class, so a new product gets misclassified rather than flagged. The planogram system deployed across 7-Eleven stores in Taiwan takes the opposite approach and returns an “Unknown” result when shelf conditions deviate significantly from its training data (Scientific Reports, 2025). A system that admits uncertainty is worth more to a store team than one that is confidently wrong, because a flagged exception can be checked and a silent error cannot.
On SKU-110K, a benchmark of more than 11,000 densely packed retail images, a 2025 co-training method reported an mAP of 0.596. That is a research result on a hard benchmark, and it is a useful reminder that product-level detection on a crowded shelf is not a solved problem you can buy as a single model.
Synthetic data for computer vision in retail and FMCG: what pretraining buys
Synthetic pretraining matters in retail because real annotated shelf data is expensive to collect and goes out of date. Staging before-and-after shelf states in real stores and annotating each change in 3D is costly, which is why the ShelfChange3D benchmark (2026) built 145K synthetic pairs in simulation alongside 5K real ones. Its sim-to-real results show what each training stage contributes on real shelves:
| Training on real shelves (ShelfChange3D-5K) | mAP at IoU 0.25 |
|---|---|
| Real data only | 21.0 |
| Synthetic pretraining only (zero-shot) | 46.7 |
| Synthetic pretraining, then real fine-tuning | 92.9 |
| Synthetic pretraining, real fine-tuning and geometric refinement | 97.9 |
Two lessons follow from that table. Synthetic data on its own more than doubles the real-only result, but the jump to 92.9 comes from fine-tuning the synthetically pretrained model on real shelf data. Synthetic data is the starting point that makes a small real dataset go far, not a replacement for it.
The conditions matter too. The real test set was a controlled subset with a fixed camera in each trajectory and product removals only, and the synthetic side used 67 SKUs. Because the test set is that controlled, we would read these figures as evidence for the training recipe, not as an accuracy to expect.
NVIDIA’s own tooling follows the same recipe. NVIDIA’s TAO RetailObjectDetection model card describes pretraining on 320,000 synthetic images with randomised lighting, object sizes, orientations, camera positions and backgrounds, then fine-tuning on about 600 real images from 6 environments to narrow the simulation-to-reality gap. The card lists an AP50 of 0.959 for an overhead checkout-counter view, with lower figures for the barcode-scanner view. That is NVIDIA’s own specification for its own model, a single-item “retail or not” detector, so it says nothing about recognising SKUs on a shelf.
Domain adaptation before the first store goes live
Domain adaptation is the step between a model that works on a benchmark and one that works in your stores. Azilen (2026) calls fine-tuning on actual store footage non-negotiable and warns that lab benchmarks should never be used as production estimates.
The 7-Eleven study shows what this looks like in practice. The team collected images on site in four stores of different sizes and layouts, at different times of day over three days, deliberately varying angle, lighting and shelf state. They augmented heavily with flips, rotations, perspective changes, colour jitter and random masking, because store display practice means most photos of a product come from similar angles. And they tested on a store the model had never seen, training on three stores and holding out the fourth.
That store-level hold-out is the test we would insist on. A random split of images mixes the same shelves into training and test sets and flatters the model; a held-out store is the closest a team gets to measuring the next store in the rollout before it goes live.
How does computer vision detect out-of-stock shelves and planogram breaks?
Computer vision detects planogram breaks by turning a shelf photo into a “realogram” and comparing it row by row with the reference planogram. The pipeline detects shelves, detects and classifies products, groups them by shelf row, then assigns each product a status such as correct, wrong position, wrong shelf or out of stock. The 7-Eleven system does the comparison with the Needleman-Wunsch alignment algorithm, which uses dynamic programming to find correctly placed, missing and misplaced products. Out-of-stock detection usually works in image space with segmentation or 2D detection; newer research frames it as 3D change detection between two RGB-D observations, which localises the changed product for downstream robotic systems.
That 7-Eleven deployment spans more than 7,000 stores in Taiwan. It rests on purpose-built datasets (15,232 images for shelf detection, 99,135 for product detection and 471 product categories for classification), and its YOLOv8 product detector reached 95.7% mAP@50. With NVIDIA TensorRT handling inference, response speed improved by an average of 1.5x to 2x.
The limits are stated just as plainly. The dataset is mostly standard shelf images with products facing forward, so the model has not been trained or evaluated for severe occlusion, backward-facing products or overlapping stock. Detection output also has to feed something: the CV-driven shelf data that drives replenishment is where an out-of-stock flag becomes an order, and Azilen (2026) notes that a model feeding a dashboard nobody checks has zero business value.
Drift after packaging changes is the year-two failure
Retail vision models drift as packaging changes, seasonal products rotate and store layouts evolve. Without automated drift detection, retraining triggers and versioned deployment, Azilen (2026) reports that accuracy typically degrades silently over 6-12 months until the system loses operator trust, which it names as the most common reason live deployments fail in their second year.
Architecture decides how expensive drift is to fix. A detector trained to predict individual product labels has to be re-annotated and retrained every time packaging changes, which Tonioni et al. (2018) call infeasible in practice. A product-agnostic detector paired with an embedding search handles a new item or package by adding its descriptor to the reference database, with no detector retraining. Few-shot and zero-shot classifiers push in the same direction: the 7-Eleven team’s FAN-based model reached 98.39% top-1 accuracy on unseen products. Semi-supervised learning on unlabelled store images is another route to keeping up with new designs without constant manual labelling.
None of these removes the need for monitoring. We would track per-category confidence and the rate of “Unknown” outputs from day one, as a check that a packaging refresh or reset has not moved the shelves away from what the model learned.
How should a retailer measure whether a computer vision project pays back?
Measure payback against the full human workflow it replaces, not against doing nothing. A simple formula, published by the shelf-vision vendor ParallelDots, frames it like this:
ROI = (Recovered Sales + Labour Savings + Recovered Trade Spend − Deployment Cost) ÷ Deployment Cost
Recovered sales are category revenue multiplied by the current loss rate from undetected shelf issues, multiplied by the share of that loss now caught in time; labour savings are hours saved multiplied by the fully loaded hourly cost. In ParallelDots’ own hypothetical (a brand across 400 grocery doors), recovered value of $169,800 against a $92,000 first-year deployment cost gives ($169,800 − $92,000) ÷ $92,000 = 84.6% in year one.
Payback differs by use case, and the same vendor gives these ranges:
| Use case | Typical payback (ParallelDots, vendor-reported) | Why |
|---|---|---|
| On-shelf availability and stockout detection | 3-6 months from go-live | The loss is immediate and continuous, so faster detection converts straight into sales |
| Trade spend verification | 6-9 months | A full promotional cycle must run before there is evidence to enforce terms |
| Planogram compliance | 9-12 months | Value compounds over several reset cycles |
Other sources put the horizon longer. Codewave cites a LinkedIn outlook’s payback benchmark of about 12 months and Nexer Group’s 12-18 months, and Azilen (2026) says retailers typically report ROI within 12-18 months, with a single-use-case pilot at 3-5 stores costing $50,000-$150,000. We would treat all of these as ranges to test in a pilot, not as forecasts. For the measured outcomes to set against them, Codewave reports that the Morrisons and Focal Systems deployment improved availability by more than 2% across all stores.
The counterexample is the one to remember. Walmart ended its contract with Bossa Nova Robotics, when about 500 shelf-scanning robots were in its more than 4,700 stores, after finding that staff could do about the same work with simple, cost-effective processes, according to a Wall Street Journal report summarised by CNBC. The lesson Codewave draws is the right one: automation has to outperform the complete human workflow, exception handling included, to justify the investment. Our fuller piece on what ROI computer vision delivers in retail sets this against the wider evidence.
Who can engineer retail vision that holds up across a whole estate?
TechnoLynx engineers retail computer vision through AI Engineering for Retail for the point where pilots break: full catalogues with severe class imbalance, near-identical variants, edge hardware that has to keep up with store throughput, and new products arriving all the time. Our view is that these failure modes compound, so they need an engineering approach rather than a model swap. TechnoLynx would start with one narrowly scoped use case, such as out-of-stock detection in one category, and a store-level hold-out test before any rollout decision, because Azilen (2026) notes that tight scoping separates successful pilots from expensive proofs of concept.
That is not the right route for everyone. Purpose-built retail platforms are another route; Azilen (2026) cites Simbe’s, trained on more than 60 billion shelf images across 18 million SKUs. A retailer with an in-house ML team that already runs model versioning and retraining may do better building on its own stack, provided someone owns the drift monitoring once the project team moves on.
The decision that remains is a measurement one. Before anyone argues about models, a retailer needs the cost of its current audit and restock workflow, exceptions included, written down as a number it trusts. Without that baseline, a vision system can be accurate and still lose to the human workflow, as Walmart’s robots did.
