Find Jobs
Hire Freelancers

Web Scraping Needed - 22 Million Records

$30-250 USD

Dalam Kemajuan
Disiarkan lebih dari 4 tahun yang lalu

$30-250 USD

Dibayar semasa penghantaran
I need a freelancer that can crawl a list of 22,000,000 domains that I will share in four CSV files that you will download from Google Storage. The crawler needs to crawl ONLY THE LANDING PAGE - not the entire site - and capture the following and output a CSV file and stored to Dropbox: 1) Does URL have Google Analytics code - yes or no. Use a search for "Google Analytics" in the source of the page. 2) Is there a link to a privacy policy on the page - yes or no. Use a search for the word "Privacy" in the link text 3) How many unique internal URL links are present on the page. Return href count. 4) Is the URL secure (SSL) - yes or no. 5) Is the URL mobile-friendly - yes or no. Use a search for "meta name="viewport"" in the source of the page. 6) Is the domain parked - yes or no. Search for [login to view URL], [login to view URL], namecheap, uniregistry, adsense anywhere in the source code OR the words "for sale" or "See related links" the meta description. 7) Is a phone number present on the page? yes or no. Capture the phone number. 8) URL being crawled. 9) Does the domain resolve to an actual website (no timeouts, 404s, etc) You MUST have your own server or equipment to do the scraping. Project must be complete in 5 days.
ID Projek: 23751802

Tentang projek

11 cadangan
Projek jarak jauh
Aktif 4 tahun yang lalu

Ingin menjana wang?

Faedah membida di Freelancer

Tetapkan bajet dan garis masa anda
Dapatkan bayaran untuk kerja anda
Tuliskan cadangan anda
Ianya percuma untuk mendaftar dan membida pekerjaan
Dianugerahkan kepada:
Avatar Pengguna
Hi, I am a java programmer good at data scrapping. I can download the 22,000,000 landing pages by applying multiple-thread to improve the downloading speed. Once all the pages are downloaded, to generate the result you need is easy task. I am ready to work now . I can finish your work within 5 days.
$160 USD dalam 5 hari
5.0 (30 ulasan)
4.8
4.8
11 pekerja bebas membida secara purata $245 USD untuk pekerjaan ini
Avatar Pengguna
Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHON (Scrapy, Selenium) based web scraper as well as WINDOWS BASED web scraping software through which I have crawled many sites such as Craigslist, Amazon, Yelp and many others. I have also worked on complex site to bypass CAPTCHA with the use of PROXY IP bouncing techniques.. Let's work together :)
$222 USD dalam 5 hari
4.9 (94 ulasan)
5.7
5.7
Avatar Pengguna
Hello From your project I understand that you need a developer to web scraping. I want to inform you that we are a team of freelancers who have experience of over 7+ years. And after reading the description I can guarantee you the best outcome of the project if we are working on this together. Please give me a chance to prove my skills and you won’t regret it. Looking for a long term work relation with you. Thanks
$140 USD dalam 7 hari
5.0 (18 ulasan)
5.2
5.2
Avatar Pengguna
Dear Sir, I am interested in your project. I have gone through your requirement. I have gone through your requirement, we can complete your task within 5 days by sure. please ping me we can start right now. I will provide you a demo within 24 hours. I'm expert in Core Java, J2ee, Javafx, Spring boot, Spring websocket, Spring JPA, Hibernate, JSF, Angular, ReactJs, React native, web scraping, web automation, mysql, mongo, postgreSql. I assure, I will do my best to work with you on your project to present the best possible outcome for you and your customers. I will also do my best to correct any area of work where quality comes into question, I want to have pride in my service to your company and the final product provided. I look forward to working with you. Thanks & Regards
$750 USD dalam 5 hari
4.7 (30 ulasan)
5.5
5.5
Avatar Pengguna
I have over 7+ years of experience along with in-depth technical knowledge on Python, Flask, Node.js, JavaScript, PostgreSQL Administration, Django, MySQL, MongoDB, XML-RPC services, stores, ERP/CRM portals and implement the fully innovative and creative strategy to deliver with outstanding functionalities and user experiences. I have rich experience in web scrapping and web crawling and recently i have done similar kind of projects. >>I can extract data on any website using python. >> I can do Data RESEARCH from any platform for certain subjects. >>Expertise in python,django,flask. >>I can stretch my time according to your time zone as well. >> I can Find data from INTERNET RESEARCH/ WEB SCRAPING. Please initiate the chat for further discussion. Regards,
$250 USD dalam 5 hari
4.9 (6 ulasan)
5.2
5.2
Avatar Pengguna
Hi Hiring manager I have read your all given information in description and i am ready to do various type of work for you and available 24*7*365 for you. 1* I can do Data RESEARCH from any platform for certain subjects. 2* I can create the accurate DATABASE (EMAIL/CONTACTS/PERSONAL INFO etc*. 2* I can properly fill a Google Sheet/EXCEL/WORD. Have good working skills of Excel/PDF. 4* I am Expert in DATA ENTRY and do any kind of that work. 5* I can Find data from INTERNET RESEARCH/ WEB SCRAPING. 6* I can Work as a VIRTUAL ASSISTANT for daily any kind of task. 7* I can work as per client TIME ZONE if they want and provideS 15hrs services in a day. 8* I have 3YRS of good experience of as a VIRTUAL ASSISTANT and done various type of task. 9* I have the excellent skill of SEO/DIGITAL MARKETING/SOCIAL MEDIA MARKETING/MANAGEMENT. 10* I can create MEMES/posting,strategy for increasing likes,followers of social media. 11* I have expertise in WORDPRESS sites creating/product Uploading/WOOCOMMERCE. 12* I have Good skills in SHOPIFY, PRESTASHOP, MAGENTO etc for DROPSHIPPING. 13* I Can Write Optimized SEO friendly impressive articles for BLOG and get 1 rank. 14* I am fast Learner. Grab The instructions from client instantly and implement. 15* I can get 100% results and Give Guaranteed work satisfaction from client instructions. for work i am always available and take the challenges and never say no to work always be ready and be professional in work. thanks
$140 USD dalam 1 hari
4.9 (19 ulasan)
4.4
4.4
Avatar Pengguna
Hello There, Hope you are doing well ! Saw your job posting and had carefully gone through your requirements for website. I hastened to submit my proposal for your review. I have 4 years of experience in the field of design/development, Python, Django, PHP, Angular I promise the highest quality of work that suits your unique styling. Thanks and Regards, Tabish Adnan!!.
$250 USD dalam 3 hari
5.0 (3 ulasan)
2.6
2.6
Avatar Pengguna
Hi there! I have read your description carefully and I am confident I can full fill your requirement. I provide my skills in hopes of working with you. I am a web scraper and have 7+years of ample experienced in web scraping. I can extract data on any website using python and PHP script(beautiful soup, selenium, scrapy, phantomJS) I can save data scraped into xlsx, CSV, MySQL and etc using CSV module and xlsxwriter Question Would you like to let me know your target website URL? please feel free to contact me if you want. I look forward to working back with you. Best Regards.
$250 USD dalam 7 hari
0.0 (0 ulasan)
0.0
0.0
Avatar Pengguna
Hi, Having read your complete requirements, I would like to assure you that I will produce the most notable results for you, having had a myriad of previous experience with similar projects. I would like to discuss the thorough details with you to fully grasp the requirements. i have my own server - let's talk. fell free to contact me
$140 USD dalam 7 hari
0.0 (0 ulasan)
0.0
0.0
Avatar Pengguna
Hi. nice to meet you. i have experience in web scraping in php and asp.net. Let's discuss more details over chat. Thank you
$250 USD dalam 7 hari
0.0 (0 ulasan)
0.0
0.0

Tentang klien

Bendera UNITED STATES
Sheb, United States
4.7
20
Kaedah pembayaran disahkan
Ahli sejak Jun 16, 2009

Pengesahan Klien

Terima kasih! Kami telah menghantar pautan melalui e-mel kepada anda untuk menuntut kredit percuma anda.
Sesuatu telah berlaku semasa menghantar e-mel anda. Sila cuba lagi.
Pengguna Berdaftar Jumlah Pekerjaan Disiarkan
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Memuatkan pratonton
Kebenaran diberikan untuk Geolocation.
Sesi log masuk anda telah luput dan telah dilog keluar. Sila log masuk sekali lagi.