The Internet is the first place most researchers turn to because of its sheer size and content; ironically, the size of the Internet is also the reason many researchers are slowly losing their minds.
Searching for information on the Web is a time sucker; you begin your search, look up at the clock and realize three hours have passed since you first hit the enter key. This is partly due to writers being a curious lot and going off on tangents; it is also due to reliable, useful, and relevant information being difficult to find. You don’t want to stop being curious, but you do want to start finding accurate information. To work out where to start your search, you need to know the difference between the Visible Web and the Invisible, or Deep Web.
The difference between the Visible Web and the Invisible Web
The Visible Web is made up of pages that search engine spiders can crawl and index. The Invisible Web is made up of pages that spiders cannot get into for various reasons; however, it is two to three times the size of the Visible Web. So if you think Google has a lot of information, just wait until you see this.
Where to begin the search
Before you start your search you need to decide where to search. You basically have three options; search engines, subject directories and the Invisible Web. Here is a brief description of each; you can find links to subject directories and Invisible Websites in the blogwell toolbox.
Search engines
Search engines like Google, Yahoo!, and Live have minimal human supervision; data is compiled by robots (a.k.a spiders and crawlers) that crawl the Web for information.
These are best if your topic is narrow; the more specific your search request, the more relevant the results.
Subject directories
Subject directories are organized into categories by people, not robots, so the quantity of information is not as great as that of search engines. However, depending on the directory, the information is generally of a higher quality than search engine results. *
Use subject directories if your topic is broad and you have the time to browse.
The Invisible Web
The Invisible Web is home to thousands of searchable databases that search engines cannot access. This is because these sites need some type of human interaction; you need to do something on the home page, that search engine spiders are incapable of doing.
For instance, to access the
Lexis Nexis database, you need to type in a password, or to access the
Librarians Internet Index you need to type in a search query (LII is both visible and invisible). Sadly, spiders are not quite bright enough to do this…yet.
You can find a list of my favorite
subject directories and
Invisible Web databases in the
blogwell toolbox.
If you are interested in reading more about the Invisible Web, take a look at:
Invisible or Deep Web: What it is, Why it exists, How to find it, and its inherent ambiguity
Those Dark Hiding Places: The Invisible Web Revealed
Deep Web at Wikipedia
Subscribe
