Deep Web
Deep Web, is also know as ‘Hidden Web’, ‘Dark Web’, or the ‘Invisible Web’. It is mostly generated by database-driven pages that are usually visible to authorized members. Who can access the hidden information after logging into the system.
Such data is totally hidden from search enging crawlers (or the ‘spiders’), that means Google, Yahoo, MSN, Altavista and other search endinges cannot search data from these pages.
Web crawlers of todays technology can only read ‘surface web‘.
March 20, 2009 at 4:39 PM
Deep Web Technologies (the company I represent) has built the search engines for a number of free public research portals, e.g. Science.gov, WorldWideScience.org, Scitopia.org. The Deep Web Technologies web-site is http://deepwebtech.com.
Readers may also want to check out the Federated Search Blog (http://federatedsearchblog.com). In that blog I write quite a bit about the deep web and the technology it takes to search it.