• Skip to main content
  • Skip to primary sidebar
  • Skip to footer

Deep Web

The Dark World

  • Deep Web
  • Deep Web Links
  • Best VPN
  • Tor
  • Hidden Wiki
  • News
You are here: Home / Deep Web Research Papers / Downloading Hidden Web Content

deepwebadmin / November 28, 2015

Downloading Hidden Web Content

Share1
Pin

Abstract:

An ever-increasing amount of information on the Web today is available only through search interfaces: the users have to type in a set of keywords in a search form in order to access the pages from certain Web sites. These pages are often referred to as the Hidden Web or the Deep Web. Since there are no static links to the Hidden Web pages, search engines cannot discover and index such pages and thus do not return them in the results.

However, according to recent studies, the content provided by many Hidden Web sites is often of very high quality and can be extremely valuable to many users.

In this paper, we study how we can build an effective Hidden Web crawler that can autonomously discover and download pages from the Hidden Web. Since the only “entry point” to a Hidden Website is a query interface, the main challenge that a Hidden Web crawler has to face is how to automatically generate meaningful queries to issue to the site.

Here, we provide a theoretical framework to investigate the query generation problem for the Hidden Web and we propose effective policies for generating queries automatically.

Our policies proceed iteratively, issuing a different query in every iteration. We experimentally evaluate the effectiveness of these policies on 4 real Hidden Web sites and our results are very promising.For instance, in one experiment, one of our policies downloaded more than 90% of a Hidden Website (that contains 14 million documents) after issuing fewer than 100 queries.

Download

Share1
Pin

Filed Under: Deep Web Research Papers Tagged With: deep web, deep web research papers, hidden internet, hidden web

Primary Sidebar

STAY ANONYMOUS

CyberGhost VPN Deep Web Access

Footer

Follow US

Recent Post

  • 11 Spine-Chilling and Nightmarish Deep Web Stories from Users
  • Deep Web Destinations – A Massive List of Places to Visit on the Deep Web
  • How Dark Web Whistleblowers Work
  • Money on the Dark Web: Bitcoin Fades as Monero Rises?
  • The Story of Deep Web Narcotics

Disclaimer

The information contained in this website is for general information purposes only. The information is provided by Deep Web Sites and while we endeavour to keep the information up to date and correct, we make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the website or the information, products, services, or related graphics contained on the website for any purpose. Any reliance you place on such information is therefore strictly at your own risk. Read more>>

© 2023 · Deep Web

  • Terms and Conditions
  • Privacy and Cookie policy
  • Disclaimer
  • Contact us