What Is Bixby Vision and What Does It Do?

Bixby Vision is a feature built into some Samsung Galaxy models that enhances the camera app with artificial intelligence. Using the phone’s camera, you can use Bixby Vision to identify places and objects the camera can see, shop for products you photograph, translate text, and more.

What is Bixby Vision?

Bixby Vision is a combination of several related technologies. Since it offers to take action on what the software detects in the camera, it is part personal assistant. In fact, it’s the imaging component of Bixby, Samsung’s personal assistant which is meant to work in much the same way as Apple’s Siri and Google’s Google Assistant.

It’s also partially an augmented reality application. Augmented reality superimposes graphics and information over real-world imagery. In the case of Bixby Vision, Bixby identifies things in the camera view and can change its appearance (such as when it translates foreign language text) or overlay options you can take action on, such as shopping for an item you point the camera at.

Behind the scenes, all this is possible because Bixby Vision is based on an artificial intelligence machine learning platform.

How to Get Started using the Bixby Vision App

Bixby Vision is fully integrated into the Camera app. In general, to use Bixby Vision:

1. Start the Camera app.

2. Tap Bixby Vision at the top left of the screen.

3. Point the camera at an object. Bixby Vision will try to identify it, and after a moment, one or more cards will appear with suggestion for what you can do with the object or what additional information the phone can display. Review and tap on any cards you like.

For some Bixby features, you need to take an additional step by launching the appropriate Bixby Vision app first.

What You Can Do With Bixby Vision

Bixby Vision includes a lot of different capabilities, and in general, the software is smart enough to show the appropriate options for whatever object you point the phone at. Here is a list of Bixby Vision’s major capabilities and how to get to them:

Use Bixby Vision to Extract Text From Images

Bixby Vision works like a scanner and Optical Character Recognition scanner software. Point them camera at some text and then tap Extract. It’ll be copied to the clipboard, where you can edit it or just copy and paste it into another document.

Use Bixby Vision to Shop With Amazon

When you point the camera at an object, Bixby Vision will offer to open the Amazon Assistant app and find that product (or a similar one) for sale. This feature works best when you point the camera at a product in its original retail packaging.

Find Information About Places With Bixby Vision

Tap the Place icon in the lower right corner of the window (it’s shaped like a map pin). Move the phone around, pointing in various directions. Bixby Vision tries to find information about the location and displays it in augmented reality mode as overlays around the screen.

Find Similar Images With Bixby Vision

Point the camera at any object, and tap Images. Bixby Vision displays any similar images it found online.

Translate With Bixby Vision

Bixby Vision can translate text in a foreign language and overlay English text over the foreign characters in the camera display. To translate text, tap the Translate button at the bottom left corner of the window and point the phone at the text you want to translate. Be sure to hold the phone steady for best results.

Use Bixby Vision for Wine Search

There’s a special wine search tool built in. Just point the camera at a wine bottle label, and then tap Search.

Additional Visual Apps

At the bottom of the screen, tap Apps.

You’ll see the Play and explore page, which lists several additional Bixby Vision apps. These are all self-contained experiences that use artificial intelligence and vision to perform special functions. Home décor, for example, is an AR app that lets you visualize placing furniture around a room in your home. Makeup uses the front-facing selfie camera to show you what your face looks like with various cosmetic products. There are a number of vision apps in this section to experiment with.