Gotta Go Fast! Using Asyncio for More Efficient Web Scraping

30 Minute Talk
Sunday at 2:00 pm in Orchid Ballroom East

The internet has a wide variety of information available for human consumption. But this data is often difficult to access programmatically if it doesn't come in the form of a dedicated API. With tools like Beautiful Soup, Python developers can parse data directly from web pages.

There's just one problem: looping through HTTP requests for a bunch of URLs one by one can take forever! Luckily, we can speed things up dramatically by making these requests asynchronous. In this live-coding adventure we will explore how to use asyncio to improve the performance of your web scraping endeavors.

Presented by

Samuel Agnew