Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
indrajithi committed Jun 12, 2024
1 parent 6d1169e commit fae1f84
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

A simple and efficient web crawler in Python.

[![CI](https://github.com/indrajithi/tiny-web-crawler/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/indrajithi/tiny-web-crawler/actions/workflows/ci.yml)

## Features

- Crawl web pages and extract links
- Crawl web pages and extract links starting from a root URL and extract all the links found on each page
- Handle relative and absolute URLs
- Save crawl results to a JSON file
- Easy to use and extend
- Save the results of your crawl in a structured JSON format for easy analysis and processing
- Designed with simplicity in mind, making it easy to use and extend for various web crawling tasks

## Installation

Expand All @@ -22,7 +24,7 @@ pip install tiny-web-crawler
```python
from tiny_web_crawler.crawler import Spider

root_url = 'http://example.com'
root_url = 'http://github.com'
max_links = 2

spider = Spider(root_url, max_links)
Expand Down

0 comments on commit fae1f84

Please sign in to comment.