Semantic Segmentation
The goal of semantic segmentation is to predict what class every pixel in an image belongs to ex. background, cat, dog.
Semantic Segmentation is applied to a picture of people in a bike race, source
With the ability to see what everything in an image is, there are endless possibilities of what it can do. Semantic segmentation is the first "comprehensive" computer vision technology where one gets information about the whole image.
The upside is that it provides a clear boundary between two objects of different classes. Ex. road surfaces and the sidewalk when used for driving, in a way that object detection does not. The downside is that semantic segmentation does not separate objects of the same category that overlap.
Suppose one needs pixel-level predictions and the ability to distinguish between objects. In that case, instance segmentation might be worth looking into, or if one is primarily interested in identifying objects for example tracking object detection is worth a look.
The upside is that it provides a clear boundary between two objects of different classes. Ex. road surfaces and the sidewalk when used for driving, in a way that object detection does not. The downside is that semantic segmentation does not separate objects of the same category that overlap.
Suppose one needs pixel-level predictions and the ability to distinguish between objects. In that case, instance segmentation might be worth looking into, or if one is primarily interested in identifying objects for example tracking object detection is worth a look.
What is Semantic Segmentation used for?
- Autonomous vehicles: where one wants to detect things such as the road, cars, pedestrians, and the sidewalk.
- Medical imaging: where one wants to highlight regions in the image.
Uncertainty and Semantic Segmentation
Since we at Desupervised are focused on developing safe AI and having the AI know that it does not know things, we can enable proper uncertainty quantification in all our models. Here is an example of what that means for a self-driving vehicle.
On the left, we have the normal view captured by a camera, in the middle is the standard semantic segmentation prediction and on the right is the uncertainty estimation, the uncertainty estimation is brighter where it is more uncertain.
This means that the uncertainty estimation highlights the areas where the extra focus should be put, as those are the "dangerous" areas where the model is not certain and could change the prediction to a different class easily.
This means that the uncertainty estimation highlights the areas where the extra focus should be put, as those are the "dangerous" areas where the model is not certain and could change the prediction to a different class easily.