I am an academic researcher examining news for public companies. I have successfully created perl scripts (although quite crude and I am very new to this) to scrape data from websites before but this one is stumping me. I want to be able to take a list of search terms (a text file input with the stock exchange and ticker of each firm (e.g. NYSE WMT), select all news wires, news papers, and press releases (or each one of these at a time), and submit the search on [url removed, login to view] This is the part I really need help with, but to finish the code I want to then take the search results and collect all of the dates of the articles (and headlines would be a bonus) and output the tickers, exchanges, headlines, and dates for all of the results. This is the free service part of [url removed, login to view], but if it works I am considering purchasing a subscription. So the input would be a list that would look something like:
and the output would look something like this:
Titles: EXCHG TICKER SOURCE DATE HDLN
DATA: NYSE WMT NEWSWR 01AUG2007 Bla bla bla
NYSE WMT NEWSWR 30AUG2006 Bla bla bla 2
NYSE WMT PRESSRL 01AUG2007 Bla bla bla 3
NYSE IBM NEWSWR 01AUG2007 Bla bla bla 4
Of course adding in anything to be courteous and not bog down their server would be nice too.
So complete, or even partial help would be very nice!