Looking for web scrape expert to scrape a single site once a week. The site has an inventory number for each item and in turn a unique web page. If an item has been sold it no longer appears – I am trying to identify what has been sold.
I would like the complete inventory numbers and product items listed in excel each week. I would like a comparison between each week showing what was no longer available.
For now I am only looking at inventory numbers, version 2 of the project (or can be included if simple) may be to collect description details and price of the products and compare any changes week to week.
Is should be noted this website blocks certain countries access and we believe if there are too many requests from one IP in a certain time frame. There are approximately 8,000 items and we have heard of blocking occurring at 5,000.
This is a time sensitive project. I would look at staggered payment, delivery of the software and successful access first time, 2nd payment on successful scrape second week and final payment on third week scrape. Potentially a 50-25-25 split. If for any reason it stops working in the first three weeks an opportunity would of course be given to get it sorted but we would push payments by one week.
Thank you for looking.