Projkt Rachna

Crawler and Profiler with Weight Algo

Transcript

1. DRDO News Reader A brief introduction to the DRDO news reader project and it’s technical implementation
2. Overview Page cache Keyword / Context Parser RSS Scrape r Web Spider Rule Engine Front-End Mongo MySql
3. Problems with generic scrapers Cricket, Cricket, Cricket India, Pakistan Header Missile Politics India…
4. Scraper Module Sources URL Patterns URL Patterns URL Pattern RSS Feed RSS Feed Root Page Search Page Path Regex Content Xpath
5. Scraper Module http://dawn.com/20 12/06/25/delhi-mu mbai-to-have-missile -defence-systems/ //div[@class=‘story’ ] dawn.com/dddd/d d/dd
6. Keyword Parser NEW DELHI: Delhi and Mumbai, the two most vital cities of India, have been chosen for Ballistic Missile Defence system of the Defence Research and Development Organisation (DRDO) that can be put in place at short notice.A detailed proposal is being prepared for final clearance by the Cabinet Committee on Security (CCS).Work one installing the BMD system in the two cities has already begun and the final proposal will be put before the government after detailed analysis of the project, sources have told PTI. NEW DELHI: Delhi and Mumbai, the two most vital cities of India, have been chosen for Ballistic Missile Defence system of the Defence Research and Development Organisation (DRDO) that can be put in place at short notice.A detailed proposal is being prepared for final clearance by the Cabinet Committee on Security (CCS).Work on installing the BMD system in the two cities has already begun and the final proposal will be put before the government after detailed analysis of the project, sources have told PTI. Text Cleaning Keyword Identificati on Keyword Weighing Text Stream and Ping Parsing (0.8 + 0.6) Missile, Agni, Defence, United, China Defence Research and Development Organisation Defence Agreements URL -> 5x Title -> 5x Keyword/Description -> 3x Context -> 1x Segment Breakdow n
7. Keyword Parser Context Detection Context Weighing Final Report India-C hina relation s India China LOAC LAC Tibet Laddak h Arunach al Maoist DRDO – 29.3 Missile – 12.4 India-China – 4.1 0 2 4 6 DRDO – 0.6 Missile Tech – 0.4 Variance Penalty 50 50 Column1 0 5 1 – 0.1