Fine-Grained Image Classification of Groceries for Assisting Visually Impaired People

Topic: Fine-Grained Image Classification of Groceries for Assisting Visually Impaired People

Speaker: Marcus Klasson, Division of Robotics, Perception, and Learning at KTH Royal Institute of Technology

Moderator: Kalle Åström, Professor, Mathematical Imaging Group, Lund University - Coordinator AI Lund

When: 12 October at 12.00-13.15

Where: Online

Spoken language: English

Abstract

In recent years, computer vision-based assistive technologies have enabled visually impaired people to use automatic image classification on their mobile phones. However, the in-built image classifiers often lack the ability to recognise fine-grained object information that could be important for the user. A particular application of such situations is classifying groceries, where the challenge is to classify visually similar items in high-variation environments. In this seminar, I will present a dataset with mobile phone images of groceries that simulates the scenario of using an image classifier to identify food items in grocery stores. Furthermore, I will illustrate how the mobile phone images can be combined with web-scraped images and text descriptions using a variational autoencoder to train more accurate classifiers, compared to training with mobile phone images only. Finally, I will introduce how to continually update the image classifier with new items without retraining from scratch, and discuss some future directions for the development of computer vision-based assistive mobile apps.