I need perl script for command line operation which scrapes click bank dot com
The output of the script would be a database containing a list of every item for sale on clickbank, along with their categories and subcategories, entered into a mysql 'items' database.
The database items fields:
iseq a record sequence number
pub_code a code extracted from link field - sub string between first and second '.' chars (example AWPROFITS from below) of link field
title item title extracted from page
cat item category extracted from page
subcat item sub category extracted from page
link item link extracted from page (example [url removed, login to view])
In order to obtain the category and subcategory information you may need to use the Clickbank Market Place page ([url removed, login to view]) and parse the pages by each category and subcategory.
+ database connect parameters would be stored in the script.
+ a web or admin interface is not needed. This is for command line operation only.
+ the script should be 'polite' and not place a heavy load on the clickbank server. The script should offer an editable parameter for how quickly it scrapes the site.
+ some items are in more than one category and that is fine.
+ clickbank offer a datafeed but the information is not suitable for this purpose.
+ the script should have lots of comments so I can see how it works and make minor changes myself.
+ the code does not need to be very clever or efficient, easier to understand is better.