Fine-grained Image Text Alignment for Artworks

Image-Text Matching Process in SCAN

This project is still ongoing as part of my advanced MSc degree research in LIIR Lab, Department of Computer Science, KU Leuven.

Image-text alignment is a fundamental research topic in the inter-field of computer vision and natural language processing. It will save the intense labor to annotate the artworks for online digital artwork archives if we can automatically describe an artwork image/subimage with its textual attributes. Furthermore, this topic can help to boost the multimodal question answering performance in the cultural heritage domain by providing fine-grained image-text correspondence information. Therefore, it is interesting to explore the methods that can figure out the artwork image/subimage and text correspondence.

The goal of this project is to train a model that can find relative textual attributes given an artwork image/subimage.

Feiyang Tang
Feiyang Tang
Ph.D. Candidate in Machine Learning

Data Enthusiast, ENFJ-T. Travelling, hiking and crime series lover. Multilingual.

Related