Skip to content

mtzfactory/real_estate_scraping

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

4 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

REAL ESTATE SCRAPING

This web app was the last one I made before joining the programming bootcamp, back in 2017. At that time I had already done two more apps in Angular, so I decided to make this third one to build up knowledge.

This was a personal project to build a real estate scraping app, which gathered data from different real estate web pages and stored information for further analysis later on.

The backend was run in Amazon AWS:

ยท The worker scripts written in Python were executed in an Amazon EC2 instance and scheduled by a cron service to scrape the real estate web pages.

ยท AWS Lambda was used to host the Api, which was also written in python.

ยท PostgreSql database was used to store all the data scraped from the web pages.

If you plan to run your own server, you can find a bunch of information about AWS at __ documentation folder, hope you find it useful, it helped me a lot.

Development server

This project was generated with Angular CLI version 1.1.0.

Clone the repo and do:

$ git clone https://github.com/mtzfactory/real_estate_scraping.git
$ cd real_estate_scraping
$ npm install

Run ng serve for a dev server. Navigate to http://localhost:4200/. The app will automatically reload if you change any of the source files.

Run ng build to build the project. The build artifacts will be stored in the dist/ directory. Use the -prod flag for a production build.

Screens

ยท Dashboard reporting statistics of last web scraping. m