Scaling up a Django + React Web Scraping Website

I am working on a platform that runs a Django/Postgres backend with a React frontend. The platform scrapes data from multiple sources with Scrapy, cross-relates them, and gives some basic analysis of them via the front-end and HighCharts.

I am running into some issues with scaling the platform and some strange issues getting the entire system to work together. For example, running Scrapy & Twisted on a worker server has been problematic at times and I am having issues using Python multi-processing in tandem with the Django ORM (objects appear to be created randomly).

Kemahiran: Python, Django, Pengikisan Web, NoSQL Couch & Mongo, Elasticsearch

Lihat lagi: web service website scraping, web scraping website pdf, web scraping data collection website, scrape website to csv, web scraping best practices, best web scraping services, web scraping applications, vpn for web scraping, python web scraping, web scraping access denied, python web scraping gui, web scraping website google analytics, web scraping django, methods extraction data website web scraping, manual web scraping website repost, freelance web scraping one website, python web scraping website automation, data entry, website search and web scraping, Website testing or Web Scraping, django web scraping project

Tentang Majikan:
( 1 ulasan ) Băilești, Romania

ID Projek: #26870645

10 pekerja bebas membida secara purata $136 untuk pekerjaan ini


Can help in scaling the scraper with scrapy. want to know more about it. These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantomjs, python. Have done testing and automation w Lagi

$253 USD dalam 2 hari
(55 Ulasan)

Greetings, Encourage Info-tech is a Team of Leading Python developers that specializes in the Scraping and Data mining [login to view URL] Previous work Concludes Scraping in the Retail and Gambling Industry to Lagi

$140 USD dalam 7 hari
(10 Ulasan)

Why not move your script to AWS ec2 instead if you want it to run smoothly. If that doesnt work then the problem is from the script not the server

$140 USD dalam 7 hari
(13 Ulasan)

Hello. I have specialized in web scraping using python/Django, scrapy, selenium, beautifulsoup, etc. Especially, I have experience in fixing above issues at before time. I can help you. Thanks, Irina.

$50 USD dalam 2 hari
(9 Ulasan)

⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ hello. how are you ? I am a python expert. i know you need some help from freelancer. so I can help you on your project in python. and also if you have any website of Django or Flask according to your requi Lagi

$140 USD dalam 2 hari
(1 Ulasan)

Hello! After reviewing the detail carefully i understood that you need Scaling up a Django + React Web Scraping Website and i think I am the perfect person for this job . I have a strong team having experience of more Lagi

$150 USD dalam 6 hari
(1 Ulasan)

Hi, With over 5 years of experience in Python and Django. I’ve gone through your complete project description. I am interested in this project as it is exactly within the scope of my skill. My main skills are as follow Lagi

$50 USD dalam 7 hari
(1 Ulasan)

All of our programming skills: *Front-end: - HTML, HTML5, JSON. - JAVASCRIPT (Ajax, , flutter, AngularJS / 2 / 4 / 5 / 6 / 7, ReactJS - Redux, Jquery). - CSS, CSS3, Twitter Bootstrap, Less, Sass, Scss, Responsive, Mat Lagi

$100 USD dalam 5 hari
(0 Ulasan)

Hi, I saw your project description carefully. So, I think you want to make interface to scrap data from websites using React and Scrapy code will be put using Django. I think this is good Idea. Python have good module Lagi

$200 USD dalam 7 hari
(0 Ulasan)

Hello there. I have carefully read your job description with great interests. I have experience in very SIMILAR project with yours. I have crawled several sites such as [login to view URL] [login to view URL] http Lagi

$140 USD dalam 7 hari
(0 Ulasan)