Ditutup

web crawler, C#

Hi,

I need a crawler (written in C#) for a specific real estate website.

It should be a console application which can crawl data (every 12 hours) and store it in MS SQL database (image files can be stored locally in a folder).

It should check all available listings (~100 000 listings).

To get listings you need to apply filters

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building

- postal code (list with postal codes will be provided)

Spec:

1. Data to store (if available) for a single offer:

- whole html

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building)

- title

- price

- construction year

- rooms

- living area

- city

- street

- postal code

- property type

- floor

- Floors in the house

- cubic area

- plot area

- offer Id (Immocode)

- Floors in the house

- Availability

- Renovated in

- Condition

- Features

- Description

- pictures (store image urls and images in the db or locally)

- Advertiser details: Name, address, phone number

2. The crawler should not make any damage on the website (prevent from DoS)

3. The crawler should not be detectable

4. The crawler should crawl the website every 12 hours without being blocked.

5. All data should be updated in case of any changes.

6. Price change for listings should be stored separately to have visibility if it grows or drops.

7. The crawler should send alerts (email or slack notification) if it's blocked.

Long-term cooperation possible.

Kemahiran: .NET, Pengaturcaraan C#, Pengikisan Web

Lihat lagi: crawling websites c#, web crawler in c# codeproject, web scraping c# code project, selenium web crawler c#, web crawler sample code c#, c# web crawler htmlagilitypack, multithreaded web crawler c#, c# pull data from website, java web crawler image capture, web crawler cpu usage, write web crawler, web crawler names, web crawler system requirement, web crawler demo gathering information function, java spring web crawler, web crawler job description, web crawler description title, web crawler extract description php mysql, web crawler extract phone number, web crawler fax number

Tentang Majikan:
( 1 ulasan ) Lugano, Switzerland

ID Projek: #18542040

51 pekerja bebas membida secara purata $733 untuk pekerjaan ini

seaanddream

Hi, my name is Selim. I read your `web crawler, C#` project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go ahead as soon as we can clarify further pr Lagi

$900 USD dalam 10 hari
(254 Ulasan)
7.7
mhmhz

Hi I did all my projects in C# and using mult-threading and proxies to enhance the performance. Thanks

$400 USD dalam 3 hari
(146 Ulasan)
7.6
promactinfo

Hello, We have a very good experience in Dot Net Development and can offer you experienced developer(s) for your project. We have been very active using DotNet core as preferred platform. It allows application t Lagi

$3000 USD dalam 30 hari
(51 Ulasan)
7.7
asin1987

Dear Employer, Please message me for your job. I have scraped more than a dozen Real Estate sites and I can share samples of those as well. I understand your job quite well and will adhere to eaxh and point. Do t Lagi

$1000 USD dalam 15 hari
(157 Ulasan)
7.4
sergiuloghin

Hello! I can create the crowler using c#. the task is pretty much clear. Is 4-5 days delivery time ok for you? Can you send me the website? You will not be dissapointed, I have a big 33%rehire rate. thank you!

$777 USD dalam 5 hari
(137 Ulasan)
6.9
kyrbatow

Hi. I should test site and define, if it blocks crawlers. Can you send site url? Then I can prepare for 1-2 days sample script to be sure, that this site can be daily scraped. Thanks anyway.

$800 USD dalam 15 hari
(33 Ulasan)
6.6
zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within th Lagi

$611 USD dalam 5 hari
(75 Ulasan)
6.9
DavidLindon

what is the real estate site? a proxy subscription will be required to rotate IPs to prevent it being blocked.

$250 USD dalam 7 hari
(47 Ulasan)
6.1
knikolovdev

Greetings, My name is Constantine and I specialize in C# and web scraping and data processing applications. Would be happy to do this project for you and future ones as well. I have good portfolio of many similar tas Lagi

$600 USD dalam 3 hari
(51 Ulasan)
6.1
bianyunzhi95

Hello. I am really grateful to face your project. As I see, you 're looking for a senior c# developer and here is him. Confidentally I can say, I could be your wonderful choice on this project. Really look forward Lagi

$800 USD dalam 10 hari
(33 Ulasan)
5.8
schoudhary1553

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Lagi

$900 USD dalam 10 hari
(19 Ulasan)
5.9
DebasisB

Hi there, I am interested in the project. Can you please provide the link to the web site from where,data to be crawled? Thank you.

$611 USD dalam 10 hari
(63 Ulasan)
6.2
mmadi

Hello, Hope you are doing well! We at Flojics are most preferred agency for our client due to 100% recommendation and “5 STAR” rating from our clients which we achieve through an excellent client support as well as t Lagi

$500 USD dalam 12 hari
(8 Ulasan)
6.0
seefattechnologi

Hello!! As per the description I am highly interested to get started on this project. I am Asp.net Full-stack developer. I have 5+ Years experience in analysis and development of client/server, windows and web ap Lagi

$800 USD dalam 20 hari
(41 Ulasan)
6.2
winlancerinc

Hello There, I'm available to assist you to work on web crawling task and I will start immediately. I am 6+ years experienced Web scraping specialist holding excellent background with creating scraping tools, a Lagi

$611 USD dalam 10 hari
(15 Ulasan)
5.6
ig149

I have read your job description and I have done something like this in past it's a custom scrapper: LinkdIn: [login to view URL] I can certainly help you get this scra Lagi

$800 USD dalam 15 hari
(42 Ulasan)
5.4
narsim3128

With respect to this project I would like to present myself as a candidate for your consideration. I have more than 8 years of IT experience. I have successfully completed few projects which involved Programming, scrip Lagi

$555 USD dalam 5 hari
(34 Ulasan)
5.6
Nidhijain123

Hi, I have my own script for scrapping. I can modify the existing script to suit your need and give you the require data in excel sheet. My expertise are in the area of scrapping using C#, My SQL, PHP. Lookin Lagi

$611 USD dalam 10 hari
(18 Ulasan)
5.2
inthebush

Hi. I’m experienced developer with 5+ years in C#. I just read your description carefully. I have experience in all aspect of web and application based on C# using Asp.net MVC, Asp.net core and C# programming. I have Lagi

$800 USD dalam 10 hari
(8 Ulasan)
5.0
sagarvb2005

Hello Sir, I have read your whole project description and i believe that i will do it better. Could you please discuss more about it and provide some sample data if possible so i can give you proper Delivery time a Lagi

$930 USD dalam 10 hari
(21 Ulasan)
5.2