Data collection / web page scraping

We need someone to create a script that will extract data from a few search and shopping engines for us. We need to scrape the top engines that have paid listings on a set of keywords in order to gather certain information. The same process will be used to gather this information from the shopping engines. This information collected will be extracted into a database. Project outline described briefly below.

A brief description of the requirements are as follow:

We need a program that can scrape the following information (title, description, URL, and placement (1st , 2nd, etc) for paid or sponsored listings from 5 different search engines, and 3 shopping engines. The program will grab the keywords that will be searched from a database or excel file. The program needs to multi-tread the queries to be efficient. The program will run (pulling from a list of keywords) until all the keywords have been exhausted. Keywords may be batched; IE 5000 keywords per day or something like that. The program needs to handle at least 5000 keywords per day. When complete with the entire list, it will start over from the beginning.

The results from the data collection / scrapping will be added to a database. The results from the scraping will need to be processed by the database.


I have attached an excell files(named [url removed, login to view]) which will show you what the output will include.

A single URL might have thousands of keywords. Along with the associated titles, descriptions, and placement.. We will need some reporting created to properly analyze the data.

About us.

We a online advertiser with thousands of keywords (keywords change and are added to daily). We want to see what other advertisers in our pace are doing, in order to be competitive.

This project needs to be completed ASAP

Kemahiran: .NET, Java, Perl, Visual Basic

Lihat lagi: we excell, web search engines list, web scraping process, web page scrapping, top data, to create web page, set data, search files on web, scraping data from web database, run java program online, online advertisers, online advertiser, multi program online, java start web, i want to create web page, excell program, data process online, data 0, daily advertiser, competitive set example, advertisers online, java data collector, webpage data collection, perl webpage scraping, scrape webpage perl

Tentang Majikan:
( 7 ulasan ) Las Vegas, United States

ID Projek: #89459

Dianugerahkan kepada:


Hello, Please look at the PMB. Thanks, Sergey

$300 USD dalam 5 hari
(26 Ulasan)

12 pekerja bebas membida secara purata $205 untuk pekerjaan ini


I can do with Perl. I can create demo for you

$270 USD dalam 14 hari
(116 Ulasan)

Please check PMB

$100 USD dalam 2 hari
(0 Ulasan)

i do it many time

$100 USD dalam 30 hari
(0 Ulasan)

I have already created a content harvestor kind of a program and is very much succeeded using perl. I think it will not take a lot of time for me and can meet the expected in this schedule.

$100 USD dalam 7 hari
(0 Ulasan)

i have already worked on this project.. please contact me for further details

$100 USD dalam 5 hari
(0 Ulasan)

Please see the pmb.

$190 USD dalam 0 hari
(0 Ulasan)

Lets Start, Please see us in PM so that we can discuss about project and our experience

$200 USD dalam 15 hari
(0 Ulasan)

Ready to start immediately on your project, as we are the experts in such things. Waiting for your selection. Thanks!

$300 USD dalam 7 hari
(0 Ulasan)

We are web design ,graphic design and website development company comprises a pool of professionals who deliver quality work on a wide range of communication web site design and web application projects. The web desi Lagi

$250 USD dalam 8 hari
(0 Ulasan)

Dear Sirs, This project seems to be very interesting for us, please see PM.

$300 USD dalam 15 hari
(0 Ulasan)

Hi, I have 5 years experience in VB,C#,ASP.NET,especially in VB. Just before several days I had left my original outsourcing IT company in SZ in order to have further time to pursuit personal career target, I can conce Lagi

$250 USD dalam 6 hari
(0 Ulasan)