Lengkap

Scrapy/Selenium (Python) to extract texts and files(specific webpages) based on keywords.

I need to make a Python script (Scrapy or Selenium, I am up to suggestions) to extract information within some specific(I have around 12) websites - daily(auto) or manually.

The pages are in portuguese, but I can guide you into the key input-fields and key-pages to look for.

1. User input:

- Time period (if the page has this feature)

- Website to scrap.

- Keywords(can be a list of words) to look for.

- User chooses the local path to the files to be downloaded.

2. Back-end:

- Access the page.

- Searches the tabs that can have useful information(I will provide the specific parts for each webpage to make the queries) links inside that domain.

- Download(if the page serves files in doc, html or pdf) and look for the keywords.

- Extract all the related content (files or the text in html).

- Go around Captchas(if the page has captcha)

3. Logging:

- All the extracted content must have the URL which the information/file is available in the webpage - can be done by logs.

- All the extracted content must have the DATE which the scrapping has been made - can be done by logs.

4. Configuration:

- All key-fields (like CSSSelector for a date field) should be configurable for each spider.

- The URL to start scrapping each webpage should be configurable.

- If page contains Authentication(Login/Password), user will fill the configuration for it.

IMPORTANT:

1. My plan is to pay for each 4 mapped websites (so total project is for 3 "packs" of websites)

2. The content in few cases will need to be extracted from images.

3. Start your bid with the word forward, so I can know if you did read all the description.

4. If you can't extract properly the content I can give you another one to replace that one, so you still need to deliver 4 websites per milestone.

5. I WILL RELEASE THE MILESTONES ONLY AFTER YOU SEND ME THE CODE AND I AM TOTALLY SATISFIED (I WILL RUN TESTS TO CHECK FUNCTIONALITY).

I have many projects at hand and would be great to stablish a good relation with you, since I constantly need someone to work with me.

Thank you.

Kemahiran: Data Scraping, Python, Scrapy, Selenium Webdriver, Pengikisan Web

Lihat lagi: scrapy examples, python scrapy example, scrapy vs selenium, python web scraping, scrapy python 3, scrapy documentation, scrapy vs beautifulsoup, web scraping, extract dbx files, extract 3gp files, mapguide enterprise extract shp files, extract xml files website, python script text files, extract embedded files doc, extract bkf files systools bkf repair tool, testsuite example selenium python, extract perl files server, extract ole files rich text, extract mht files, test suites selenium python

Tentang Majikan:
( 5 ulasan ) FORTALEZA, Brazil

ID Projek: #19212037

Dianugerahkan kepada:

etuannv

Hi there, I am interested in your project. I would approach your project by using Python with Scrapy. The website will be written in Python with Django. Here is a demo project: Price tracking system: https://etuannv.c Lagi

$250 USD dalam 10 hari
(63 Ulasan)
6.2

36 pekerja bebas membida secara purata $579 untuk pekerjaan ini

Vlzinch

Hi! I’m experienced Python developer, and web-scraping is one of my main fields of knowledge, so I’m 100% confident that I can complete your project and extract data from the sites you need. Please contact me to d Lagi

$748 USD dalam 7 hari
(61 Ulasan)
7.7
mhmhz

Hi Can you provide the sites so i can analysis them? Thanks

$800 USD dalam 5 hari
(103 Ulasan)
7.4
zhangyingtai

forward Hello sir I have 9 years of experience about web scraping and have made 200+ crawlers with python. I have fully understood the project and I am confident. I can start the work right now. Best Regards, Lagi

$588 USD dalam 10 hari
(111 Ulasan)
7.5
$1000 USD dalam 7 hari
(93 Ulasan)
7.2
zekovicm

Forward Hi there,I am Python Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and fi Lagi

$705 USD dalam 10 hari
(91 Ulasan)
7.2
polarjin2017

Here is my selenium with python working result. [login to view URL] python selenium web driver app to scrap live data from the web site and export to excel file. This is just what I've done. I can do pytho Lagi

$250 USD dalam 3 hari
(49 Ulasan)
6.4
dreammate0621

Hello! Let's just rest a moment. <Actions speak louder than words!> Nice to meet You! I am a WEB expert! I am interested in Your project. I wanna work with You. If you hire me, I am gonna do my best for Your proj Lagi

$555 USD dalam 10 hari
(5 Ulasan)
6.3
C3guru

forward I've read your requirements about User Input,Back-end,Logging and Configuration. I have a good experience with selenium and python. Recently,I've developed B*T for Telegram. That acts like human 100% exactl Lagi

$1000 USD dalam 10 hari
(15 Ulasan)
5.8
lightingdavid

Hello. I have good skills in "Data Scraping, Python, Scrapy, Selenium Webdriver, Web Scraping". I have working for 7+ years in this field. I 'm very interest to your project. I have checked your project description Lagi

$250 USD dalam 3 hari
(31 Ulasan)
5.1
kunitsynartem

Hello! I have 2 years of experience in web scraping using Python and I'm interested in your project. I can use both Selenium and Scrapy depending on what is better for certain website. Also I can handle logins, file do Lagi

$600 USD dalam 10 hari
(27 Ulasan)
5.1
smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO Lagi

$444 USD dalam 10 hari
(49 Ulasan)
4.9
drishinfotech

forward HI, I read your job description and would like to assist you in website scraping task. I understand your conditions and will surely provide you the code after completion of the each task. Please share Lagi

$750 USD dalam 10 hari
(9 Ulasan)
4.7
albertpopov46

Dear, sir @I am fulltime freelancer@ I read your description in carefully. I am python expert and I have rich experience with scrapping. Also i have selenium experience . So I think that i can do your project in Lagi

$500 USD dalam 10 hari
(10 Ulasan)
4.2
yongbeauty1996

hello how are you? I am very interested in your project. I have read your description very carefully. I can do your job in time. kind regards

$555 USD dalam 10 hari
(4 Ulasan)
4.2
NIKE9

Hi, I am a senior selenium/python expert and I can build the script as requirements in the description. I have 7+ years of professional experiences in web development. I can start immediately, also finish your proje Lagi

$750 USD dalam 7 hari
(3 Ulasan)
3.6
KGeorgy

Hi, Thanks for your job posting. I've read your project description carefully. You are going to build scrapy that gets data based on keywords. As a senior scraping developer, I have rich experience in scrapy and pyt Lagi

$500 USD dalam 10 hari
(6 Ulasan)
3.6
chirag9700

I have more than 6+ years of experience into IT field. Since last 6 years, I am dealing with different kind of field such like : - Laravel, CI, YII - Angular.js - Node.js - Ionic Framework - PHP - HTML - Python - Djan Lagi

$666 USD dalam 10 hari
(2 Ulasan)
4.2
BoyVit85

How are you. Credit is my motto. I am expert web scraping. I can do your job with BS4 and Seleinum framework of python. I can do any project in your demand completely by my good experiences of last ago. I think thi Lagi

$555 USD dalam 10 hari
(3 Ulasan)
3.1
edison4mobile

HI, how are you? I have checked your description carefully. I can say I understood fully what you want. As I have rich experienced in python(2, 3) so that your project is not problem for me. I am really confident an Lagi

$777 USD dalam 10 hari
(1 Ulasan)
2.8
vorasiddh4it

We have 11+ years of experience in software development. We have developed 400+ projects and the research paper in the field of Machine Learning, Artificial Intelligence and Image processing (GIS), Network, SEO based W Lagi

$1000 USD dalam 10 hari
(4 Ulasan)
3.4