Photon Library

Photon is available as a library for both Python 2 & Python 3.

To install photon as a library, you can simply do

pip install photon --user

Documentation

Most basic example

import photon
result = photon.crawl('http://example.com')

The crawl function returns data in json format. A sample output can be found here.

To make the crawling as flexible as possible, following optional arguments are present

Argument	Type	Default
level	int	2
threads	int	2
timeout	float	6
delay	float	0
regex	str	None
exclude	str	None
seeds	list	None
user_agent	list	random
cookies	dict	None
keys	boolean	False
only_urls	boolean	False

Please go through the Photon wiki for a detailed explanation of each option.

A more advanced example

import photon
result = photon.crawl('http://example.com', level=3, threads=10, keys=True, exclude='/blog/20[18|17]')