Introduction to Collecting Web Data with Python

This repository contains the materials for a Python workshop held at the University of Pittsburgh.

This two-day workshop will teach the basics of the Python programming language with a focus on collecting data from web sites and social media platforms. Day one will introduce Python and the Jupyter Notebook. On day two we will cover Application Programming Interfaces (APIs) and web scraping. Participants will receive hands-on training and leave with introductory knowledge of Python.

Dates:

• March 31st: Introduction to Python, Jupyter, & basic command line
• April 7th: Introduction to APIs and Web Scraping with a little bit of Pandas

Location:

• Digital Scholarship Commons on the ground floor of Hillman Library (in the back to the left of the elevator bank).

Lead Instructors:

• Matt Burton ([email protected])
• Aaron Brenner ([email protected])

Student Instructors:

• Shadi Janansefat ([email protected])
• Qikun Wu ([email protected])

Pre-Workshop Software Drop-in

• There will be drop-in hours to help participants configure their laptops with the Anaconda Python distribution on the mornings of Wednesday March 29th and Thursday March 30th from 9am to 11am in the Digital Scholarship Commons.

Day One (March 31st)

9AM-4PM

Day one will focus on an introduction to the Python programming language.

• Last minute configuration help (8:30-9am)
• Welcome and Introductions (9-9:30am)
• Data Science with Python (9:30-10:00am)
• The Command Line & Jupyter Notebooks (10-10:30am)
• Introduction to Python (10:30am-12noon)
• Lunch - on your own (12noon-1pm)
• Introduction to Python (1-3pm)
• Open Q&A time (3-4pm)

Homework Assignment

Review:

• If you would like to review the fundamentals of Python that we introduced on day one, please visit Python for Everyone: Exploring Data with Python 3. Each chapter includes interactive examples and great explanations of the concepts of Python programming.

Videos:

• Data Analysis in Python with pandas - A playlist of short (around 10 minutes) videos introducing pandas and how to use it for common tasks.
• Introduction to Web scraping - A short introduction to the Python libraries we will be working with on Day 2

Written Tutorials:

• Pandas Tutorial: Data analysis with Python: Part 1
• Pandas Tutorial: Data analysis with Python: Part 2
• Python Web Scraping Tutorial using BeautifulSoup

Day Two (April 7th)

9AM-4PM

Day two includes an introduction to APIs and web scraping.

• Quick introduction to Pandas (9-10am)
• Introduction to APIs (10am-12noon)
• Lunch - on your own (12noon-1pm)
• Introduction to Web Scraping (1-3pm)
• Open work and Q&A time (3-4pm)