Semantic segmentation is a process in computer vision that involves assigning a class label to every pixel of an input image. The goal is to produce a dense pixel-wise segmentation map of an image, where each pixel is assigned to a specific class or object. There are different types of image segmentation tasks, including semantic segmentation, instance segmentation, and panoptic segmentation. Semantic segmentation is used in many applications such as automated driving, medical imaging, and industrial inspection.
In semantic segmentation, the model predicts for every pixel in the image, which is why it is commonly referred to as dense prediction. The task is to label each pixel of an image with a corresponding class of what is being represented. For example, an autonomous vehicle needs to identify vehicles, pedestrians, traffic signs, pavement, and other road features. Semantic segmentation can be used to differentiate different objects in an image, and it can be considered an image classification at a pixel level.
Semantic segmentation is different from other types of image segmentation tasks. Instance segmentation identifies, for every pixel, a belonging instance of the object, detecting each distinct object of interest in the image. Panoptic segmentation combines pixel-level semantic segmentation tasks, which assign a given category to each pixel, with instance-level semantic segmentation tasks, in which each individual in a given category must be uniquely identified.
