Here’s How Samsung Uses AI to Improve Portrait Photos on the Galaxy S21 Series

The Samsung Galaxy S21 is one of the first flagship phones of 2021 and it uses various AI algorithms to enhance images.


Smartphone cameras powered by artificial intelligence (AI) have become sort of a buzzword nowadays. Every company seems to be saying its new phone has an AI-powered camera. But what does this really mean? Well, Samsung seems to be trying to answer just that question. The company posted a detailed article on its blog, explaining how the AI camera on its new Samsung Galaxy S21 works. The blog is, of course, meant to promote Samsung’s new cameras but it’s a more in-depth explanation than you will find at many other places. “A lot happens behind the scenes,” says Samsung and well, we have always wanted to know what.

AI in selfies

According to Samsung, the front camera identifies faces in images and is capable of marking them for further processing, something the company calls “segmentation”. This is actually a term in AI research, which refers to the algorithm’s capability to differentiate between objects it’s meant to process and those it isn’t. The Samsung AI algorithm enhances details in eyes, hair and facial features, though the company doesn’t really explain what “enhancing” means. 

Samsung claims the algorithm adjusts white balance and creates natural skin tones in “any environment”. “The results are instantly shareable photos that don’t look overly processed or require further editing,” the blog post states.

AI in portrait mode

Segmentation is an important part of the company’s portrait mode feature as well. The algorithm differentiates between humans or pets in the scene and separates them from the background too. The background is separated from the main image using a “seed map”, which allows the algorithm to apply the effects you select to the subject only. There’s something called the “tri-map” too, which identifies areas where the background and the subject merge. Then, the “matte map” applies something called “matting” to the details in hair and facial features of subjects. Lastly, there’s “depth estimation”, which is the AI’s way of measuring depth in the photo and applying the background blur accordingly.

While Samsung says this is how the AI in its Galaxy S21 camera works, the process is actually quite similar to what everyone else uses too. If hardware remains the same, AI-based cameras are separated based on the quality of their algorithms. What Samsung is telling us about are the various algorithms that work within a larger algorithm, but the differentiating factor between different companies’ approaches is in training data and how accurately the AI can predict things.