Internet, Networking, & Security Around the Web What Is a Search Engine? How search engines work and why they're useful By Tim Fisher General Manager, VP, Lifewire.com Tim Fisher has 30+ years' professional technology support experience. He writes troubleshooting content and is the General Manager of Lifewire. our editorial process Facebook Twitter LinkedIn Tim Fisher Updated October 29, 2019 Around the Web How to Get a VPN Tweet Share Email A search engine is a tool used to find data based on a particular input. Web search engines are one example where you can enter a word or phrase to find web pages that match those terms. Not every web search engine works the same way, but most are crawler-based, meaning that they actively seek the web for pages to add to their index. The search engine is what you use to quickly lookup information from the index and display the results on the page. Search engines are the primary method for browsing the web, and there are many different kinds that are built for finding specific information. Within each search engine are often advanced search options that are used to focus your search better and help you find what you're looking for. Although they're often used interchangeably, a web search engine is different than a web directory and a web browser. How a Search Engine Works Search engines automatically create website listings by using software, often called spiders or spiderbots, that "crawl" web pages. They follow a site's links to other pages and index the information in the process. The crawler bots behind the search engine scour the web not only by jumping from one link to the next but also by checking each site for a robots.txt file. This file contains a listing of which pages on the site a search engine should crawl. This is one way website owners can block a search engine from indexing a particular page. Software spiders return to pages that have already been crawled on a pretty regular basis to check for updates and changes, and everything they find goes back into the search engine database. Using a Search Engine Every search engine is different, but the common idea with all of them is to type something into a search box and wait for the results. Some search engines also have a reverse search option that lets you browse the web with something other than text, such as a sound clip or picture file. Many search engines include extra features beyond just a simple search box. They let you interact with the index using special text commands or buttons that can filter the results and remove items that aren't relevant to what you're looking for. For example, there are lots of advanced Google Search commands that can be used to get better search results from Google. Top 10 Tips and Tricks for Better Web Searches Examples of Search Engines Lots of search engines exist, but the one you choose to use should depend on what you want to do with it. Web page search engine: Often multi-purposed, they locate all sorts of data, from general web pages and news to help documents, online games, and usually more like images, videos, and files.Image search engines: Search for photos, drawings, clip art, wallpapers, etc.Video search engines: Find music videos, news videos, live streams, and more.People search engines: Locate people on the internet using their name, address, phone number, email address, etc.Mobile search engines: A regular search engine that's optimized for searching and displaying results on a smaller screen.Job search engines: Find job postings.Invisible web search engines: Tools that browse the invisible web. Early Search Engines: Whatever Happened to Them? Other Facts About Search Engines There's no need to manually update a search engine so that you're getting the most up-to-date results in your searches. If you're seeing really old results that you suspect should be updated, you might just need to clear your browser's cache. Search engines don't search the entire web. There are massive portions of the web that aren't crawlable by a search engine, and they're collectively known as the invisible/deep web. A search engine finds web pages on its own, so you don't necessarily need to tell it to index your website or to add a particular page to its database. However, for a variety of reasons, a search engine might not have crawled a particular web page. Some search engines include a tool that lets you explicitly request that they check the page and add it to the index so that other people can find it. Google's URL Inspection tool is one example. Search engine optimization (SEO) is something practiced by web content writers in an attempt to compete with similar content found through search engines. Specific algorithms are used by search engines to rank web pages, so it's one goal of an online content writer to ensure that the page reflects the topic accurately. This is often done through the use of specially crafted keywords and phrases. Boost Your Website Rank With These Insightful SEO Tools Due to the fact that search engines crawl the web for data, it's often an aim of website owners to strive for more inbound links so that crawlers find the pages more quickly and monitor them for changes more often. Technically, a search engine is a tool that searches through something else. With that definition, lots and lots of websites include search engines in the form of a simple search bar where you enter a word or two to find related content on that specific site. Anyone can add a search option to their own site, but they're not the same as a web search engine as described on this page.