Architectural Design of Meta Crawler for Terrorist Network Mining

Nowadays world is flooded with the digital data as well as with different types of search services but none of the search service gives the complete view of internet data. Moreover there is scarcity of the Meta Crawlers that are application specific. This paper presents architectural design of Meta crawler designed especially for Terrorist Web Mining. This architectural design is use to create an effective Meta crawling service that span across search engines. Meta crawler collates the search results of multiple search engines to fetch the terrorist Networks related information. Search results from single search engine may prove insufficient and spamming affected hence there is need for Meta Crawler which filters the results aggregated. Meta Crawler proposed in this paper has a multithreaded architecture hence this Terrorist Meta Crawler design have advantages like performance efficiency and scalability. This architecture can be used to designed a terrorist network related dynamic Meta Crawler.

Download PDF