Sedang Disiapkan

223558 perl scrape script

I need perl script for command line operation which scrapes click bank dot com

The output of the script would be a database containing a list of every item for sale on clickbank, along with their categories and subcategories, entered into a mysql 'items' database.

The database items fields:

iseq a record sequence number

pub_code a code extracted from link field - sub string between first and second '.' chars (example AWPROFITS from below) of link field

title item title extracted from page

cat item category extracted from page

subcat item sub category extracted from page

link item link extracted from page (example [url removed, login to view])


In order to obtain the category and subcategory information you may need to use the Clickbank Market Place page ([url removed, login to view]) and parse the pages by each category and subcategory.


Please note:

+ database connect parameters would be stored in the script.

+ a web or admin interface is not needed. This is for command line operation only.

+ the script should be 'polite' and not place a heavy load on the clickbank server. The script should offer an editable parameter for how quickly it scrapes the site.

+ some items are in more than one category and that is fine.

+ clickbank offer a datafeed but the information is not suitable for this purpose.

+ the script should have lots of comments so I can see how it works and make minor changes myself.

+ the code does not need to be very clever or efficient, easier to understand is better.

Kemahiran: Semua Boleh, MySQL, Perl

Lihat lagi: how to make sequence, first place bank, first market bank, first command bank, dot net list, site scrape, scrape web, scrape a web, parse a string, page scrape, load information from mysql, hop database, output script, net perl, editable mysql, database command line interface, command line parameter, sub script, mysql connect net, mysql containing

Tentang Majikan:
( 5 ulasan )

ID Projek: #1969794