Single-loss multi-task learning for improving semantic segmentation using super-resolution

We propose a novel means to improve the accuracy of semantic segmentation based on multi-task learning. More specifically, in our Multi-Task Semantic Segmentation and Super-Resolution (MT-SSSR) framework, we jointly train a super-resolution and semantic segmentation model in an end-to-end manner using the same task loss for both models. This allows us to optimize the super-resolution model towards producing images that are optimal for the segmentation task, rather than ones that are of high-fidelity. Simultaneously we adapt the segmentation model to better utilize the improved images and thereby improve the segmentation accuracy. We evaluate our approach on multiple public benchmark datasets, and our extensive experimental results show that our novel MT-SSSR framework outperforms other state-of-the-art approaches.

Recommended citation: Aakerberg, Andreas and Johansen, Anders S and Nasrollahi, Kamal and Moeslund, Thomas B (2021). "Single-loss multi-task learning for improving semantic segmentation using super-resolution." International Conference on Computer Analysis of Images and Patterns.
Download Paper