Wikipedia_Extractor

This Repository will help you get the content and links of all the article related to a particular keyword.

---

Usage:

This wiki_extractor.py file will take 3 parameter inputs:

• keyword:- It should be the word for which we want the related pages links to be extracted.
• num_urls:- Number of related pages ,related to a keyword, that we want.
• output:- The name of the json file in which the results would be stored.

C:\Users\username\Wikipedia_Extractor>python wiki_extractor.py --keyword="Indian Historical Events" --num_urls=10 --output="output.json"

Once the above command is executed successfully the output file will be generated ,as follows, in the current directory.

[
    {
        "url":"URL for related page no: 1",
        "content":"Content of the related page no: 1"
    },
    {
        "url":"URL for related page no: 2",
        "content":"Content of the related page no: 2"
    }
    ......
    ,
    {
        "url":"URL for related page no: 10",
        "content":"Content of the related page no: 10"
    }
]

Future Scope:

• Currently only upto 10,000 related pages can be requested using the wikipedia api, if anyone from the community knows how we can try to extract more related pages please create a PR for that, it would be really helpful.