Sedang Disiapkan

scraper to scrape vbulletin forum and record specific data about threads

Need a simple scraper created that will scrape a vbulletin forum.

Actions scraper will complete:

Every 10 minutes: Scrapes a subforum page that lists all threads in that subforum sorted by thread creation date (decending), adds new thread records to a watchlist as they are created.

Every 30 seconds: Scrapes a subforum page that lists all threads in that subforum sorted by last reply (decending), records the time, views, and replies for all threads on the watchlist that appear on that page.

Must include proxy support, rotating through a user entered list

USER INPUTS:

- user entered list of proxies

- user entered URLSs (2) to scrape

Output:

-scraper must keep a log of every time it scrapes, recording the time it scraped the page, also the number of views and replies at that time for each thread on the watchlist.

-export the log in csv format.

goal: to be able to record the rate of new posts per 30 seconds, and rate of new replies per 30 seconds, of a thread that is getting tons of views and replies.

Kemahiran: Perisian Forum, MySQL, Kejuruteraan Perisian, vBulletin, Pengikisan Web

Lihat lebih lanjut: log page, time record, threads, recording web page, export data web page, forum proxy, web proxies, list web proxies, simple web proxy, proxies vbulletin, vbulletin log user, views thread vbulletin, export vbulletin data, vbulletin thread views, web page proxy, rate posts vbulletin, vbulletin posts user, export vbulletin user data, proxy software support, vbulletin thread, views vbulletin

Tentang Majikan:
( 65 ulasan ) Chicago, United States

ID Projek: #4078051