I am interested in a program which can scrape the Alexa directory (yes, the entire thing) and either store it in a directory-based structure or MySQL (preferable).
I would prefer the programmer to us PHP, as I can reasonably understand this programming language and make minor edits later on if necessary.
This program must explore the entire Alexa directory.
Top > Shopping > Consumer Electronics > Video > Television
(can be found at [url removed, login to view];CategoryID=220503&mode=general&Start=1&SortBy=Popularity)
The program would scrape all 5 result pages, and store the title, description, site URL, and a link to "more info" (why I think MySQL would be preferable for this, though it could be possible to name each directory in a files system the name of the category, and make each file stored therein the title)
It should be able to scrape every directory, subdirectory, and subsubdirectory (etc.) and return all the results pages in an organized fashion.
If you need help in getting started with programming this, I would consult: [url removed, login to view]
I understand this could be timely to produce and will consider all fair bids, such as based on an hourly rate and expected time frame. This is a very serious project and I am willing to pay well.
Feel free to contact me with any questions! Thank you in advance.
5 freelancers are bidding on average $440 for this job
Dear Sir: I already scraped alexa (about 25.000 website rankings and other info per each domain) please check pm i have enough experience with sscrapers and I can develop your scraper working an entire week