Case Summary
A Kaufland store is a very big thing. It has a sales floor of up to 12.000 square meters and provides more than 30.000 products. A lot of events can occur on our shelves that are likely to be overlooked. Items can get sold out, other items might be placed on the wrong position or they are labeled in the wrong way.
Consequently, customers miss the products they need or relevant information about these products. This may impact customer satisfaction and in last consequence lead to missed revenue. But how to avoid this situation and get all the relevant information from the shelves in a very short time?
How about using a drone, flying through the store while capturing various relevant information within seconds? We can take care of the flying operations, so you can focus on the computer vision component.
Can you apply object detectors to locate empty spots in the shelves, barcode-readers for decrypting the labels and self-taught AI for detecting if the items are placed above the correct label? Do you have other ideas to analyze the products, the shelves or do things we can’t even imagine, feel free to test?
If you are interested in computer vision and autonomous flying, live up to the challenge and impress the world with your solution!
Research Problem
Checking shelf compliance means processing different types of information and comparing the results. So first you can check whether there is a product or not. If there is one it has to be checked what item it is. The third step is to compare this information with the item that shall be found at this place. Therefore, this information must be provided by another source like for example a database or a barcode located at the shelf.
Combining different information from shelves means analyzing different types of data by using different models. So, every datatype and model will lead you to different challenges. For example:
- gaps in shelves can differ a lot in shape, size and lighting conditions. They might be uniformly shaped if there is a big quadratic packaging missing or they might be small and irregular in case of flexible packaging. And please keep in mind: even if a shelf is fully stowed there might remain a gap between the items and the case board above your recognition should not detect!
- Recognizing an item by its appearance might be difficult too. Especially if there are items which look very similar.
- Barcode detection seems to be easy on the first look. But there are some challenges even here: Barcodes can be quite small. They also might be shown on digital labels and last but not least, barcodes might not be located exactly below the gaps that you are interested in.
If you analyze other information you get you might face other challenges.
The next question after choosing and building your models is how to deploy it. Do you want to build an application that completes different task one after the other? Or do you want to have several models that work in parallel and evaluate each other? And finally: how are you presenting your information in a form that shows all relevant information at once and that will affect us?
The Data
Test set images here… And XML files…
The dataset consists of approx. 350 images, containing around 60 different items and their store labels. Each image is accompanied by an XML file with the same name, containing the annotations of the objects (items and store labels) present in the image. We have split the dataset into 3 parts and will refer to the biggest one as “the ground truth”. In this set, all the articles are in place and have the correct store label below them. The <name> of each item in the annotations is represented by a unique 8 positioned number (example 00060444), and the <name> tag of the corresponding store label is preceded by “label_” (example “label_00060444”).
The “working” part of the dataset provides images in which some of the items are missing (i.e. the space above the corresponding label is missing). In this case, the annotations of the label and the item are extended with a trailing “_m” (“00060444_m” and “label_00060444_m”). Other articles have wrong labels or vice versa if you choose to take the challenge to detect the nightmare of each store manager – the misplaced item/label – keep in mind these combinations are marked with trailing “_w” in the <name> tags (example “00060444_w” and “label_00060440_w”).
And then the “evaluation” dataset is very similar to the “working” one but will be held out for the DSS to evaluate the designed approach to the problem.
The submission file has the following look. You should input 1 in the ‘Value’ column if there is any problem with the object (missing, wrong, etc.) and 0 if the object is fine. Leave ‘NaN’ or blank if you do not have prediction for a particular field.
Among the annotations, some of the objects are not completely visible, or other circumstances are impairing a good view to them – the <difficult> tag, in this case, is set to 1.
Example:
The communication channel for this case is: case_kaufland_april_2019 If you have further questions, please ask in the channel.
The Kaufland experts for the Datathon
Pencho Dobrev, a Data Scientist @ Kaufland, joined Kaufland’s Data Science team two years ago and is mainly exploring different methods for time series data forecasting. Covering broad techniques, from pure statistics to Deep Neural Networks, his tools of choice include PySpark, TensorFlow, Scikit-Learn and Keras. His Master’s degree at Sofia University St. Kliment Ohridski is focused on Statistics and Econometrics.
Always up for a Data Science challenge, he participated in competitions either as contestant or supporter.
Data.Chat Name: penchodobrev
Hendrik Lange, Industry Expert, Data Scientist @ Kaufland, has been working for Kaufland Information Systems (KIS) as a Data Scientist for two years. Currently, his work is focused on different approaches for the optimization of prices. Former projects focused on neural network approaches for a recommender system and a customer segmentation project. His favorite tools at the moment are Spark, H2O and Keras. Prior to joining KIS he worked at the University of Marburg in the department of Social Science in the area of electoral studies. At that time he mainly used generalized linear model approaches.
Data.Chat Name: @hl
Expected Output
The main focal point for presenting the results from the Datathon from each team is the written article. It would be considered by the jury and it would show how well the team has done the job.
Considering the short amount of time and resources in the world of Big Data Analysis it is essential to follow a time-tested and many-project-tested methodology CRISP-DM. You could read more at http://www.sv-europe.com/crisp-dm-methodology/
The organizing team has tried to do the most work on phases “1. Business Understanding” “2. Data Understanding”, while it is expected that the teams would focus more on phases 3, 4 and 5 (“Data Preparation”, “Modeling” and “Evaluation”), so that the best solutions should have the best results in phase 5. Evaluation.
Phase “6. Deployment” mostly stays in the hand of the case-study providing companies as we aim at the continuation of the process after the event.
Data Science Society’s team prepared an environment where you can run your Jupyter Notebook using R or Python using 32 GB of RAM and 8 CPUs. Stay tuned in the #announcement channel of the Data.Chat for more information.
Book your Datathon2019 ticket now!
