A client need a Web Crawler , Details are given below :-
See the attached spec but this is a simple project to crawl and cache to a database single websites. We want to use it as an internal tool to help with our assessment and optimisation of websites for search engines.
The project will be picked up by our internal developers after completion who will add to the project to put out custom tests on it. This project is just for the crawling/caching part.
Please PM ME for detailed Specs !!