Import.io Scrape/Crawl Contest
- Status: Closed
- Hadiah: $140
- Penyertaan diterima: 2
- Pemenang: benideas
Website to be mined: HTTP://[url removed, login to view]
Details: Create an extractor and crawler to pull all information on every product on SamsClub.com. Include a connector to search specific items. Information scraped includes item titles, item numbers, model numbers, item price, item shipping cost to zip code 90210, item description (html and static format), item specifications(html and static format), all item links, item categories/links, all item images (link).
-You must use [url removed, login to view] desktop software and provide login credentials for entry to be considered. We will not accept any entries using any other software/methods.
-You must provide [url removed, login to view] login information so that we may access your extractors and crawlers to judge the entries.
-You must provide the information requested in the contest details outlined above.
-Your crawler/extractors MUST navigate the infinite scroll autoloading pages.
-Your crawler/extractors must pull the correct information for every item. This will only work by using the manual overrides noted above as the location of the requested information changes within product groups and category groups.
***WINNING*** The winner of this contest will be the one who provides all of the requested information in the quickest, most detailed and organized manner. This includes keeping the number of crawlers/extractors as close to 1 as possible, but being able to navigate across all item and categories using the single crawler/extractor. Combination crawler/extractors may be used for crawling links and extracting data from said links, but crawling should start at high level and accurately navigate without duplicating products.