Towards Practical Complementary Label Learning: A Study with Real-World Human Annotators

Wei-I Lin (r10922076@ntu.edu.tw or empennage98@gmail.com)

This summer (Jun. - Aug. 2022)

Complementary label learning (CLL) is a weak type of multi-class classification problem where the learning algorithms have access to only complementary labels, classes that an instance does not belong to. We aim to understand how humans behave when asked to provide complementary labels in various aspects, including the time to annotate, quality of the annotations, bias in selecting complementary labels, etc.

• Conduct experiments on crowdsourcing platforms such as Amazon Mechanical Turk or host a crowdsourcing platform on your own.
• Collect labels and log relevant information such as time to annotate using the platform.
• Implement and test different CLL algorithms on the collected dataset.
• Release the collected dataset as possibly the world's first real-world benchmark dataset on CLL.

• Send an email to Hsuan-Tien Lin (htlin@csie.ntu.edu.tw) and cc the potential mentor above.
• Tell us about your background, why you are interested in the project, your expected workload, and your requested salary.
• If the position is still available, we'll contact you for an interview.