Gamifying medical image annotation




UX researcher (team of 4)




3 months


Skin cancer is one of the most common types of cancer. It is often preventable by identifying the origin in an early stage, and machine learning algorithms can help with this by reviewing images at scale. However, these algorithms need to be trained using a dataset of medical images that have been correctly annotated. Having experts annotate thousands of images is costly and time-consuming. A solution to this problem is to make use of crowdsourcing.


MelaGo is a crowdsourcing platform that was developed in order to make the task of annotating images accessible to laymen. It allows users to annotate images anytime anywhere. In the application a tutorial is offered to train the users about annotating asymmetry, color and border of the images. In order to make the app more engaging, and keep users motivated to use it, gamified elements were added. The executed research was focused on how gamification affects the motivation of the users and the data gathered through the app.

How can gamification motivate non-expert online crowds to keep annotating melanoma images?

— focus question


This project focused on experimenting with gamification elements and researching their effect. From literature 'best practices' were gathered such as offering unlockables within the application, a competitive leaderboard among the best annotators or nurturing your own avatar.

The avatar represents a hypothetical patient. The objective was to motivate the users to annotate more and more images, which also claimed to 'influence' the state of the patient. In simple terms: the more images you annotate, the better your patient avatar becomes.

I designed and wired an interactive digital prototype to finalize the concept. The designed wireframes were developed into a web-application. In this project, I set up and conducted user tests to gather the data for analysis.

Measuring UX

The research was focused on how gamification elements affect the users' motivation to annotate more images. To measure that, a testing period was organised with 20 participants split into two groups over the course of a working week. Each annotation was logged in the application in order to learn from the insights.

From the data (analysed in Tableau) we found that the gamification features positively affected the quantity of annotations made. The figures show the total number of annotations made daily in each of the two groups. While initially both groups seemed to be making similar progress, the interest of the control group significantly dropped.

The research results were presented at Labels scientific conference in 2018, here is a link to the presented paper.