How to Extract Data from News Articles and News Websites?

News websites have a lot of important data.

This type of data could be used to do financial analysis, sentiment analysis, much more.

Therefore, you might need to extract data from news websites and scrape it into the excel spreadsheet to do more analysis.

Using the web data scraper makes that an easy job to complete.

Easy and Free Web Scraping

For the given project, we will utilize a 3i Data Scraping scraper, a powerful data scraper, which can scrape data from all websites. You can download and install it now.

A web data scraper will permit you to extract website data you want to scrape as well as click on data that you wish to extract. Then, the scraper will automate the procedure and extract data into the excel spreadsheet. In this example, we will extract the news feeds pages from the Newsweek website.

Scraping News Article Data

Let’s start the data scraping project.

Ensure to download as well as install 3i Data Scraping scraper before you start.

1. Open 3i Data Scraping scraper and click on the “New Project”. Provide the URL that you wish to extract and we would submit Newsweek URL that we have chosen. 3i Data Scraping will render the site within the app.
2. Begin by clicking the title about the initial news article available on a page. This will be underlined in green and indicate that this has been chosen.
3. The rest headlines on a page would get highlighted in yellow color. Then click on the second one given on the page and choose them. They will now get highlighted with green color. On the left-hand sidebar, rename the selection with the headline.
4. After that, click on the PLUS (+) symbol given next to your selected headline and select the command, “relative select”.
5. Then, use the command ‘Relative Select’ and click on the Headline of the first article as well as then on a category given above that. One arrow will come to show an association that you’re making. Rename the selection with category.

Repeat steps 4-5 for adding an article’s byline. The project will now look like this:

Need to find out how to extract more data? Then check our in-depth guide about how to extract data from a website.

Adding Pagination

Now, 3i Data Scraping is scraping the data that you’ve chosen from the initial page of different news articles. Now, we will tell 3i Data Scraping to extract extra article pages.

1. After that, click on the PLUS (+) symbol next to the page selection as well as the pick the Select command.
2. Then scroll down to the bottom of a page as well as click on “next page” control. Rename the selection with next.
3. Utilize the icon subsequent to the next selection for expanding it.
4. Then, delete both the extractions with this command.
5. After that, click on the PLUS (+) symbol next to the next command as well as pick the click command.
6. The pop-up will come and ask you if it is the next page’s link. Then, click on the “yes” button and enter a total number of times you would want to repeat this procedure. Here, we would repeat that 5 or more times.

Run The Web Data Scraping Project

Now, it’s time to test the scraping project. For doing that, just click on the green button “Get Data” given on the left-hand sidebar.

There, you can test, run, as well as schedule the project. Here, we will run that straight away.

Now, 3i Data Scraping will go and get data that you’ve demanded from the site. When the scraping gets completed, you will get a notification.

Note: Remember that a few news websites may block some of your IPs for doing web scraping. To solve this, you may require to turn on the IP Rotation option in the 3i Data Scraping scraper.

Final Thoughts

When your run gets completed, you would be able to download that in an Excel or CSV format or in a JSON file.

Now, you understand how to extract data from news websites. In case, you face any problems while setting your project up, contact us via live chat or fill-up the form and we’ll happily assist you!