Zebra Technologies Corporation, a global leader in digitising and automating workflows to deliver intelligent operations, recently unveiled how a team of its AI researchers have used AI and physics to overcome the simulation-to-real-world data gap when training computer vision models for warehouse shelf-storage, floor and pallet staging and truck loading.
By realistically simulating the physical damage, clutter, and chaos of these real-world settings, Zebra’s STRIPES (Sim-to-Real Industrial Perception for Warehouse Scenes) dataset enables computer vision models to perceive with improved levels of accuracy, compared to using only real-life images or traditional synthetic datasets.
“By releasing the STRIPES dataset, we’re providing a powerful new tool for academic and industrial communities, Zebra partners and customers to accelerate the development of the next generation of intelligent automation”, said Andrea Mirabile, Senior Director of AI Research, Zebra Technologies.
The framework behind STRIPES generates scenes that mirror the complex state of a real warehouse. The researchers developed a sophisticated pipeline with three key innovations:
- AI Image generation: Instead of using static 3D models, the researchers leveraged Generative AI to create a diverse library of 30,000 images with 3,000 unique box textures and geometries and one million labelled objects.
- Simulated wear and tear: The researchers applied realistic structural damage like crushing, dents, surface dust and plastic wrapping to mimic box degradation during handling and shipping.
- Physics-enabled clutter: The team simulated gravity to stack boxes in physically plausible and often chaotic arrangements. This naturally replicated the dense, tilted, and interlocked configurations found in three distinct operational phases: shelf storage, floor staging, and truck loading.
The dataset was benchmarked against state-of-the-art compact AI computer vision models optimised for real-time edge device deployment for warehouse applications.
STRIPES training with real-world fine-tuning yielded consistent improvements in object detection accuracy compared to real only and synthetic only by up to 4.9% average precision. In the future, STRIPES applications could be broadened to include robotic manipulation, grasping, and 3D world-modelling.
“Our research and solutions are closing the gap between the physical, data, intelligence, and execution layers”, said Mirabile. “We’re demonstrating how frontline AI can deliver better asset visibility and embed intelligence into workflows”.
The research was presented at The 19th European Conference on Computer Vision (ECCV), 8–12 September in Malmö, Sweden, one of the world’s leading computer vision conferences. Mirabile and team previously secured second place out of 650 teams in the prestigious Computer Vision Pattern Recognition (CVPR) RetailVision Workshop.
Image Credit: Zebra Technologies
Source: Tahawul Tech