I am looking to have a web crawler created for my shopping comparison site. It will need to be modifiable to scrape various retailers, have an admin panel, and directly update the MySQL database.
The fields that need to be scraped are as follows: Product Code, Category, Title, Direct URL, Price, Description, Details, Model Info, Care Info, Colors, Sizes, Images
Not all of the retailers have all the above fields, but these are all to be gathered where possible. Example retailers include Forever 21, Delia's, ASOS, the Limited, Banana Republic, GAP, Old Navy. If your tool will not be customizable for adding future retailers, please include your additional cost per retailer in your bid.
The scraper will need to remove entries that no long exist, in addition to updating current and new products.
Each product will need to be reviewed and additional fields will need to be filled in before they go live on the site, so each entry will need to be set to off when added to the database, as well as not delete the additional fields when updating current products.
Also, please include your hourly rate for future work as things change.
Sorry, additional fields which were not included before: Retailer, Brand