Scrapy + Redis / Cassandra -- Price Monitoring Bot / Crawler / Scraping / Data Mining
$30-250 USD
Dibayar semasa penghantaran
Hello,
We are looking to utilize Scrapy (Scrapy+Redis or other option) to run on our internal servers to handling data mining for the purposes of price monitoring, alerting and adjustments.
We need a freelancer who is skilled with scrapy already building the environment from the ground up (its ok to use templates and such).
It needs to have the following functions at minimum:
1) Scrapy running from scaleable server environment
2) Documented method for business users to add websites/fields to be scraped
3) Store Data
4) Data Reporting
5) Anti-Blacklisting (IP Rotation, Proxies, Check [login to view URL], Throttling, Spoofing, Headless Browser)
6) Quality Assurance / Compliance
The project must be well documented in a understandable way in case you leave the project at some point. We also need you to stay on board long term to help maintain this environment.
We will provide all the infrastructure and associated items.
Payment:
50% milestone funded upfront, second 50% funded when working model shown. 100% released once you finish and it is fully functional with all items in scope of this project complete.
Any items not specifically mentioned in the original project are not valid.
If you bid on this project, your bid is final price for initial project.
ID Projek: #15890077
Tentang projek
6 pekerja bebas membida secara purata $218 untuk pekerjaan ini
There's not much to say, but I know I'll do a very good job Relevant Skills and Experience python
