Jun
AI Lund lunch seminar: Prototype Softmax Cross Entropy: A New Perspective on Softmax Cross Entropy
Topic: Prototype Softmax Cross Entropy: A New Perspective on Softmax Cross Entropy
When: 14 June at 12.00-13.15
Speaker: Proff. Nicola Wolpert, Hochschule für Technik Stuttgart
Moderator: Maria Priisalu, PhD Student, Mathematics, Lund University
Where: Online - link by registration
Spoken language: English
Abstract
In the talk we consider supervised learning for image classification. Inspired by recent results in the field of supervised contrastive learning, we focus on the loss function for the feature encoder. We show that Softmax Cross Entropy (SCE) can be interpreted as a special kind of loss function in contrastive learning with prototypes. This insight provides a completely new perspective on cross entropy, allowing the derivation of a new generalized loss function, called Prototype Softmax Cross Entropy (PSCE), for use in supervised contrastive learning.
We prove both mathematically and experimentally that PSCE is superior to other loss functions in supervised contrastive learning. It only uses fixed prototypes, so no self-organizing part of contrastive learning is required, eliminating the memory bottleneck of previous solutions in supervised contrastive learning. PSCE can also be used equally successfully for both balanced and unbalanced data.
Prototype Softmax Cross Entropy: A New Perspective on Softmax Cross Entropy
Registration
To participate is free of charge, but please sign up at ai.lu.se/2023-06-14/registration and we will send you a link for access at the zoom-platform.
About the event
Location:
Online - link by registration
Contact:
Jonas [dot] Wisbrant [at] control [dot] lth [dot] se