What is Google Lens?

google lens
Google Lens is a visual search engine that can analyze pictures and then provide useful information or perform specific tasks. Anton Petrus / Moment / Getty

Google Lens is an app that analyses images in order to bring up relevant information and perform other specific tasks. The app is integrated with Google Photos, Google Assistant, and the native Android camera app, and it leverages artificial intelligence and deep learning to work better, and faster, than earlier image recognition apps like Google Goggles. It was first announced alongside Google's Pixel 2 and Pixel 2 XL phones, with a wider release to the first generation Pixel phones, and other Android devices, to come later.

In addition to integration with apps like Google Assistant in select phones, Google Lens is also available as a standalone app that is compatible with a much wide range of devices.

Google Lens is a Visual Search Engine

Search has always been Google's flagship product, and Google Lens expands on that core competency in new and exciting ways. At a very basic level, Google Lens is a visual search engine, which means it can analyze the visual data of an image and then perform a number of different tasks based on the contents of the image.

Google, and most other search engines, have included image search functions for a long time, but Google Lens is a different animal.

While some regular search engines are capable of performing a reverse image search, which involves analyzing an image and then searching for similar content on the web, Google Lens goes a whole lot further than that.

One very simple example is that if you take a picture of a landmark, and then tap the Google Lens icon, it will recognize the landmark and pull up relevant information from the internet.

Depending on the specific landmark, this information can include a description, reviews, and even contact information if it's a business.

How Does Google Lens Work?

Google Lens is integrated into Google Photos and Google Assistant, so you can access it directly from those apps. If your phone is capable of using Google Lens, you'll see an icon, indicated by the red arrow in the above illustration, in your Google Photos app. Tapping that icon activates Lens.

When you use Google Lens, an image is uploaded from your phone to Google's servers, and that's when the magic starts. Using artificial neural networks, Google Lens analyses the image to determine what it contains.

Once Google Lens figures out the content and context of a picture, the app provides you with information or gives you the option to perform a contextually appropriate action.

For instance, if you see a book sitting on your friend's coffee table, snap a picture, and tap the Google Lens icon, it will automatically determine the author, title of the book, and provide you with reviews and other details.

What Else Can Lens Do?

google lens playing music from a concert poster
Google Lens can perform context-specific tasks, like playing music when you point it at a concert poster.

As Lens learns more, and becomes smarter, Google is able to add more and more functionality to the app. As a visual search engine, the most basic task that it's able to perform is to identify an object and return Google search results that are relevant to that object.

Depending on what you point Lens at, it can often go far beyond simply returning search results and basic information. For instance, if you point it at a concert poster, Lens is able to quickly identify the band and then start playing a relevant music video.

 

Using Google Lens to Capture Email Addresses and Other Information

Google Lens is also able to recognize and transcribe text, like business names on signs, phone numbers, and even email addresses.

This is sort of like old-school optical character recognition (OCR) that you may have used to scan documents in the past, but with far more utility and a great deal of accuracy thanks to help from Google DeepMind.

This feature is pretty easy to use:

  1. Aim your camera at something that includes text.
  2. Press the Google Lens button.

That's really all there is to it.

Depending on what you took a picture of, pressing the Google Lens button will bring up different options.

  • Call phone numbers.
  • Send email to email addresses.
  • Add phone numbers or email addresses to your contacts.
  • Copy the text to paste elsewhere or run a search.

Shopping With Google Lens

google lens shopping links
Google Lens can identify products like clothes, shoes, and decor, and provide shopping links.

Google. 

Lens also makes shopping a whole lot easier. If you find yourself out in the world, and you see an outfit you love, or the decor at a friend's house gives you some good ideas, just point Lens at whatever it is you're interested in.

If you point Lens at items like clothing or decor, it is able to identify those items, or visually similar items, and provide vital information like reviews and even shopping links.

Google Lens and Google Assistant

Google Assistant is, as the name implies, Google's virtual assistant that comes built right in to Android phones, Google Home, and many other Android devices. It's also available, in app form, on iPhones.

Assistant is primarily a way to interact with your phone by talking to it, but it also has a text option that allows you to type requests.

By speaking the wake word, which is "Okay, Google" by default, you can have Google Assistant place phone calls, check your appointments, search the Internet, or even activate your phone's flashlight function.

Google Assistant integration was announced alongside the initial Google Lens reveal. This integration allows you to use Lens directly from Assistant if your phone is capable of doing so, and it works by activating a live feed from the phone's camera.

When you tap a part of the image, Google Lens analyzes it, and Assistant provides information or performs a contextually relevant task.

Google Lens and Google Maps

google lens google maps augmented reality
Lens integration with Google Maps creates a futuristic augmented reality experience.

Google 

One of the coolest, and most useful, implementations of Lens is its integration with Google Maps. This integration creates a futuristic augmented reality experience that can provide real-time Street View directions and pull up information about local businesses.

This type of augmented reality isn't really designed for use when you're driving, but it can be tremendously useful when walking around an unfamiliar city.

How to Get Google Lens on Your Phone

google lens app
You may be able to run the Google Lens app on your phone if it isn't already integrated.

Lens was originally only available on the Pixel 2 and Pixel 2 XL, but it has been integrated into a much wider range of devices. If you see the Google Lens icon in your Photos, Assistant, or native camera app, then you already have it on your phone.

If you don't see the icon in any of those apps, you may still be able to experience the fun of visual search.

  1. Open the Play Store on your phone.
  2. Search for Lens.
  3. Click Google Lens.
  4. Click Install.

This will install the Lens app on your phone. To use it, you will need to launch the app.

It won't be integrated into your Google Assistant or any other apps if your phone isn't compatible, but you will still be able to perform visual searches from within the Lens app itself.

Note: The Lens app doesn't work on all Android phones, so you may not have the option to install it. If you have multiple devices, it may be easier to visit the Google Lens app page on the Play Store from a computer, and log in to your Google account.

If the message near the top of the page says This app is compatible with your device, or This app is compatible with some of your devices, then you can use Google Lens on one or more of your phones.