web crawler, C#
Bajet $100-1000 USD
Hi,
I need a crawler (written in C#) for a specific real estate website.
It should be a console application which can crawl data (every 12 hours) and store it in MS SQL database (image files can be stored locally in a folder).
It should check all available listings (~100 000 listings).
To get listings you need to apply filters
- offer type (buy or rent)
- offer category (house, apartment, business, parking space, land/plot, apartment building
- postal code (list with postal codes will be provided)
Spec:
1. Data to store (if available) for a single offer:
- whole html
- offer type (buy or rent)
- offer category (house, apartment, business, parking space, land/plot, apartment building)
- title
- price
- construction year
- rooms
- living area
- city
- street
- postal code
- property type
- floor
- Floors in the house
- cubic area
- plot area
- offer Id (Immocode)
- Floors in the house
- Availability
- Renovated in
- Condition
- Features
- Description
- pictures (store image urls and images in the db or locally)
- Advertiser details: Name, address, phone number
2. The crawler should not make any damage on the website (prevent from DoS)
3. The crawler should not be detectable
4. The crawler should crawl the website every 12 hours without being blocked.
5. All data should be updated in case of any changes.
6. Price change for listings should be stored separately to have visibility if it grows or drops.
7. The crawler should send alerts (email or slack notification) if it's blocked.
Long-term cooperation possible.
51 pekerja bebas membida secara purata $733 untuk pekerjaan ini
Hi, my name is Selim. I read your `web crawler, C#` project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go ahead as soon as we can clarify further pr Lagi
Hi I did all my projects in C# and using mult-threading and proxies to enhance the performance. Thanks
Hello, We have a very good experience in Dot Net Development and can offer you experienced developer(s) for your project. We have been very active using DotNet core as preferred platform. It allows application t Lagi
Hello! I can create the crowler using c#. the task is pretty much clear. Is 4-5 days delivery time ok for you? Can you send me the website? You will not be dissapointed, I have a big 33%rehire rate. thank you!
Hi. I should test site and define, if it blocks crawlers. Can you send site url? Then I can prepare for 1-2 days sample script to be sure, that this site can be daily scraped. Thanks anyway.
what is the real estate site? a proxy subscription will be required to rotate IPs to prevent it being blocked.
Greetings, My name is Constantine and I specialize in C# and web scraping and data processing applications. Would be happy to do this project for you and future ones as well. I have good portfolio of many similar tas Lagi
Hello. I am really grateful to face your project. As I see, you 're looking for a senior c# developer and here is him. Confidentally I can say, I could be your wonderful choice on this project. Really look forward Lagi
Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Lagi
Hi there, I am interested in the project. Can you please provide the link to the web site from where,data to be crawled? Thank you.
Hello!! As per the description I am highly interested to get started on this project. I am Asp.net Full-stack developer. I have 5+ Years experience in analysis and development of client/server, windows and web ap Lagi
Hello There, I'm available to assist you to work on web crawling task and I will start immediately. I am 6+ years experienced Web scraping specialist holding excellent background with creating scraping tools, a Lagi
With respect to this project I would like to present myself as a candidate for your consideration. I have more than 8 years of IT experience. I have successfully completed few projects which involved Programming, scrip Lagi
Hi, I have my own script for scrapping. I can modify the existing script to suit your need and give you the require data in excel sheet. My expertise are in the area of scrapping using C#, My SQL, PHP. Lookin Lagi
Hello Sir, I have read your whole project description and i believe that i will do it better. Could you please discuss more about it and provide some sample data if possible so i can give you proper Delivery time a Lagi
