Meta’s Nifty New AI Photo Segmentation Tool Cuts Anything Out of an Image

Meta has just revealed a beta version of an image segmentation tool that has implications for amateur photo editors but also for scientists, journalists, advertisers, and, well, just about everyone else. 

Meta’s Segment Anything Model (SAM) analyzes an image and uses an AI-based algorithm to split it into its various parts. This goes beyond something like Google’s Magic Eraser, as SAM splits up the entire image, pixel by pixel, to create accurate segmentation models. This means you could identify and remove any object from a photo, from an annoying person in the background to something small in the foreground. 

The model has been trained on a dataset of over one billion images and portions of images, the largest ever dataset for this type of editing tool. There is a demo to try for yourself and a full version to download, though the latter is only available for research purposes (for now.) The algorithm identifies the various objects in a photo, and then you click to choose which gets removed. Eventually, the company says they will pair this technology with AI-based language prompts, so you could just tell the tool something like, “Get rid of that stapler,” for instance.

This is not just a photo-editing tool, as the technology is easily applied to videos, though Meta says it will be a little while before we get to try that particular aspect of the model. 

This is going to be huge news in the world of photography, with some built-in ethical implications, but Meta says it expects these tools to really catch on with science photographers, journalists, and other professions in which quick image-based turnaround is a must. The company also envisions this technology becoming extremely useful for the self-driving and robotics industries, as each relies on image recognition.

For now, give these tools a whirl and imagine a future in which objects in photos and videos are as interchangeable as Lego blocks.