How to scrape files from the web using your web browser

Excel’s web scraping capabilities have improved in the last couple of years, thanks to an update to the Get Data From Web feature.

However, while it can now extract content from a much larger pool of websites, it still lacks many of the features required for more complex scraping tasks. And without relying on the antiquated VBA, you are severely limited in what you can do.

Forget about Excel, though. Your web browser is actually a far more powerful web scraper, thanks to its built-in console. This lets you run JavaScript code to perform almost any task you can think of.

Downloading files directly from a webpage is one of them.

Thankfully, you haven’t got to know a jot of JavaScript these days, as ChatGPT (other chatbots are available) can do almost all the work for you.

Example

Say you’re using Google Chrome and want to download each and every Excel template on this webpage.

Right-click on the first file link, Calorie-amortization-schedule1.xlsx, and select Inspect. This opens up the Elements tab of the Developer Tools panel.

At this point, press Windows key + Shift + S to take a region-specific screenshot of the webpage and the <a> element in the HTML markup.

Paste the screenshot into ChatGPT and prompt the following:

generate console code to download all the excel files from this webpage

It will duly respond with an explanation and a JavaScript snippet.

Copy this and then select the Console tab in Chrome.

To prevent endless save popups, go to Settings > Downloads and disable Ask where to save each file before downloading.

Paste in the code and press Return.

You may need to run allow pasting first to enable this feature.

The script will execute, and the files will start downloading one by one. Open your Downloads folder to see them stack up in real time!

Watch the video