Python Web Scraper for Data Collection (Eastern Europe or Latin America)
$25-100 USD
Dibayar semasa penghantaran
*******
LOCATION
*******
You must live in Eastern Europe or Latin America.
*******
OVERVIEW
*******
I will conduct searches on Google like "Plumbers Minneapolis, Minnesota" or "Carpenters Charlotte, North Carolina."
Then I will run the script(s) you created for me.
The script(s) you create will save the Google search results in a .csv file. That is trivial and straightforward. The script(s) will also open the websites associated with the search results. That is also trivial and straightforward. The main challenge lies in locating and filling out contact forms on these websites with my first name, last name, email address, phone number, etc.
Most search results will include a link to the company’s website. For example, when I search on Google for… Jake The Plumber Minneapolis, MN … I see a link to Jake The Plumber’s website.
Your script(s) will not submit the forms; rather, I will review each form and then manually submit each myself. In other words, I will click on, for example, the “Send” or “Submit” button.
If I were to loop your script(s) X times, then your script(s) would open X tabs in Google Chrome. Each tab would contain a website for a particular business.
For example, if I were to loop your script(s) 89 times, then your script(s) would open 89 tabs in Google Chrome. Each tab would contain a website for a particular business.
Most such websites I have visited take about 100 MB of RAM. I have 16 GB of RAM. If I were to free up 12 GB of RAM, then I could open [takes out pencil and paper, calculates furiously for a few minutes] 120 websites.
That limitation would not bother me because I plan to scrape not more than a hundred pages each day. In other words, I do not plan to scrape hundreds of thousands, nor tens of thousands, nor even thousands of websites each day. I plan to use your scripts on a relatively small scale of not more than one hundred websites each day.
*******
TESTING
*******
You must test your script on Linux Mint 21.3 Cinnamon Edition to ensure that it runs properly on Python 3.10.12, on the current stable version of Google Chrome, on Linux Mint 21.3 Cinnamon Edition. I do not care if your script runs on any other operating systems or any other browsers.
If you typically use Microsoft Windows, please consider setting up a virtual machine to install Linux Mint 21.3 Cinnamon Edition so you can test your code properly.
In particular, if I were to send you a bug report, I would not like to receive a reply from you like, “The script works 100% perfectly for me on Google Chrome on Windows 11. Because I am a 100% perfect engineer, who always ships code that is 100% bug-free, I am 100% certain that the code is 100% cross-platform compatible; therefore, I am 100% certain that it is your fault that the script is not running properly.”
Of course I am being facetious. No engineer I've ever worked with has explicitly made such ludicrous statements. Nonetheless, I want you to understand the frustration I have had dealing with incompetent, self-professed software engineers who have shipped me buggy code which they failed to test on Linux, and then blamed me.
*******
SPECIFICATIONS
*******
I will send you complete specifications for this project.
*******
A MAIN SCRIPT THAT EXECUTES SUBORDINATE SCRIPTS
*******
To make it easier to modify (in particular, to debug) you will provide me with a main script named “Main script for scraping and filling in a contact [login to view URL]”
that executes subordinate scripts. For example, “Main script for scraping and filling in a contact [login to view URL]” might contain…
import subprocess
# Common base directory
base_dir = "~/abc/xyz/"
# Run script 1
[login to view URL](["python3", base_dir + "1 of 3 script for scraping and filling in a contact [login to view URL]"])
# Run script 2
[login to view URL](["python3", base_dir + "2 of 3 script for scraping and filling in a contact [login to view URL]"])
# Run script 3
[login to view URL](["python3", base_dir + "3 of 3 script for scraping and filling in a contact [login to view URL]"])
In this example, almost all of your code would have been in…
"1 of 3 script for scraping and filling in a contact [login to view URL]", and
"2 of 3 script for scraping and filling in a contact [login to view URL]", and
"3 of 3 script for scraping and filling in a contact [login to view URL]"
*******
THE NUMBER OF ITERATIONS AND THE LENGTH OF THE PAUSES
*******
I don’t want a Google algorithm to ban me for web scraping. Therefore, you will give me a script named, “Trigger main script for scraping and filling in a contact [login to view URL]” which will include something like the following…
import subprocess, random, time
# Define the Python script file to be executed
script_file = "Main script for scraping and filling in a contact [login to view URL]"
# Define the number of times to loop
number_of_iterations = 71
# Define the range for random interval (between X and Y seconds)
min_interval = 10 # Minimum interval in seconds
max_interval = 120 # Maximum interval in seconds
# Loop to execute the script multiple times with a random delay
for _ in range(number_of_iterations):
# Generate a random delay between min_interval and max_interval seconds
random_delay = [login to view URL](min_interval, max_interval)
# Execute the script
[login to view URL](["python", script_file])
# Pause execution for the random delay
[login to view URL](random_delay)
Obviously, this will enable me to manually change the number of iterations, and the length of the pause between each iteration.
You don’t have to use the precise code above; but your script(s) must contain the functionality above.
*******
QUESTION
*******
Which do you intend to use: Playwright or Selenium?
*******
FINAL NOTE
*******
Please do not begin working on this project for me unless I create a Milestone Payment for you on Freelancer.
ID Projek: #38010873
Tentang projek
19 pekerja bebas membida secara purata $94 untuk pekerjaan ini
I am a highly experienced Python Web Scraping developer, having expertise in creating scripts to collect Google search results and automate the process of filling out contact forms on websites. I ensure the script(s) w Lagi
Hello I can do this. Please share the details of the task so that I can check and confirm accordingly
With my proficiency in Python and extensive experience in web scraping and data extraction, I am confident that I am the best fit for your project. My portfolio showcases a wide range of successful data collection proj Lagi
Hello Your job description has piqued my interest. With a rich background in web development, I offer senior-level expertise with over 4 years of hands-on experience. My skill set includes: Laravel, NodeJs, Python, Rea Lagi
❤️❤️Hi Eric A.,❤️❤️ I am Artur, a seasoned Web Developer with nearly 6 years of experience in Python and Web Scraping. After reviewing your project details, I am confident in my ability to design and develop an Eye-C Lagi
Although my name is Adrian, most people refer to me as a "coding wizard" with over six years of experience under my belt. The bulk of my work has been in web/cloud development working with a variety of sectors, includi Lagi
As an experienced Python developer with a strong background in web scraping, I believe I am the perfect fit for your project. I have spent over 8 years working in the field of Software Development, specializing in Pyth Lagi
? Hello ? As an experienced Python developer with a strong background in web scraping and automation, I am confident in my ability to deliver scripts that meet your requirements. I have successfully completed similar Lagi
As a seasoned web developer with extensive knowledge in Python and web scraping, I am confident that I've got the skills and expertise you need to complete this project. With over a decade of experience in the industry Lagi
With over 8 years’ experience as a dynamic full stack developer, I have a comprehensive understanding of the skills, experience and dedication needed for your web scraping project. I’ve worked on numerous data scraping Lagi
Hello. Note: 25$/hour. I'm a Python developer. I can develop the various libraries like Selenium and so on. I have rich experiences about web scrapping. I'd like to work with you. Thanks for considering me.
