Master of Data Science for Public Policy   Master of International Affairs   Master of Public Policy  

Developing bots and scrapers with Python

This course is divided into three parts:

The first part will be an overview of how web pages work (html, css and javascript). Students will then be introduced to the client-server model along with a speedy review of python basics.

The second part will be a practical notebook session. Students will interact with a set of Jupyter Notebooks, allowing them to get hands-on experience with how to query web server APIs or plain html pages.

Finally, students will be asked to choose an API or website of their choice and develop a bot/scraper. They will be invited to explain to the group what they developed, which blocking points they encountered and what could be improved in the future regarding it. It will be followed with a live scraper development session to consolidate students’ learning.