Not The Same: Invisible Web and the Dark Web

What is the difference between dark and Invisible web?

Have you watched the news, your favorite TV show, or a hit movie recently, and heard the term "Dark Web", "Invisible Web", or "Deep Web"? These are subjects that are getting a lot of mentions lately, and many people are curious about them - and rightly so! Unfortunately, popular culture to the contrary, these terms are not interchangeable, and mean very different things. In this article, we're going to look at what the difference is between the Invisible Web and the Dark Web, as well as a term you might not have heard before - the Surface Web.

 

Different "layers" to the Web

It’s probably best to start by explaining that there are actually several “layers”, so to speak, of the Web: the Surface Web, the Invisible Web, and the Dark Web. The web that we’re all used to – the one that offers up our favorite sports websites, gossip news, online magazines, etc. – that is commonly known as the Surface Web. The Surface Web includes any content that is easily crawled, or indexed, by search engines.

The Invisible Web

However, there’s a limit to what search engines include in their indexes. That’s where the term “invisible web” comes into play.  The term "invisible web" mainly refers to the vast repository of information that search engines and directories don't have direct access to and are not including in their index, like databases, libraries and court records.

Unlike pages on the visible, or Surface Web (that is, the Web that you can access from search engines and directories), information in databases is generally inaccessible to the software spiders and crawlers that create search engine indexes.

There’s generally nothing nefarious going on here, and there are several different factors as to why a site would not be included in a search engine index, but basically  they simply boil down to technical barriers and/or deliberate decisions on the part of the site owner(s) to exclude their pages from search engine spiders.

For instance, university library sites that require passwords to access their information will not be included in search engine results, as well as script-based pages that are not easily read by search engine spiders. There’s also really large databases in there, both public and private; anything from  NASA, the Patent and Trademark Office,  U.S. National Oceanic and Atmospheric Administration to databases like LexisNexis, which require a fee to search.

How do you access the Invisible Web?

It used to be that these pages were hard to get to, but over the years, search engines have gotten pretty sophisticated and are including more and more of the content that was difficult to find in their indexes.  However, there are still many, many pages that are not making it into search engines for whatever reason; you can still find them directly if you know how. Basically, you can "piggyback", so to speak, on search engines to drill down into databases to find these pages. For example, if you did a search for “weather” and “database”, you’d come up with some pretty fascinating information. From this initial search query, you can drill down into the database's index to find what you’re looking for.

 

So the difference between the Dark Web and the Invisible Web is....

Now we can finally get to what the Dark Web – also known as DarkNet – really is. If the Surface Web is basically everything that a search engine offers up in its index, and the Invisible Web – which, incidentally, is estimated to be at least 500x times larger than the Surface Web – is basically information that a search engine does not or cannot include in its index, then the Dark Web is a relatively small portion of the Invisible or Deep Web, one that has a lot of different stuff going on, anything from drug trafficking to murder for hire to people who are looking to share information safely in an unsafe environment or culture, with complete freedom from censorship; in other words, it’s not all bad stuff going on there. 

Intrigued? Keep reading to find out more about the Dark Web here, or check out this Ultimate Guide to the Invisible Web to get a deep dive overview of how this all fits together.