Web crawler for downloading images from [login to view URL] based on some parameters (search term, size and date).

The web crawler should be written in python and run on Windows from the command line.

The command line will have 2 parameters:

1. A name of a CSV file that configures which images to download.

2. An output directory (for saving the downloaded images).

Each line in the CSV configuration file will specify one search definition, and include:

1. search term (e.g.: "curtain", "hand", "man with glasses" etc.)

2. start date

3. end date

4. minimum size (if an image doesn't exists in this size - get the next bigger size)

5. max number of images

For each such line (defining a search), the crawler will open a sub-directory in the output folder,

and put the images for this search there.

Next to each downloaded image, the crawler will create a .txt file with the name of the image

(with a .txt extension), and in this text file it will put:

1. Image name

2. Image date (YYYY-MM-DD)

3. Size in pixels

4. Flickr tags

5. Flickr description

6. The search term used


1. The user should be able to stop the crawler, and then restart from it from the same state it was when stopped

(we intend to schedule it to run only during nights and weekends, and then pause during work days).

2. The crawler should run in several threads, in order to accelerate the performance.

Kemahiran: Python

Lihat lagi: flickr image crawler, crawler flickr, web crawler images flickr, web-crawler, flickr crawler python, python download file, download file python, python web, flickr, python create file, python file create, python download, python text image, web crawler python, max extension, python csv text, command web, search image folder, search python, python restart, search download images, max file extension, csv images, image python, user term

Tentang Majikan:
( 2 ulasan ) Israel

ID Projek: #4065396

Dianugerahkan kepada:


Hello, I've been working professionally with Python for about five years now and have substantial experience with web scraping and general application development. Please check your private message box for a more in-de Lagi

$500 USD dalam 14 hari
(19 Ulasan)

15 pekerja bebas membida secara purata $461 untuk pekerjaan ini


hi, dear I am expert in python, expert in scripting and web scrapper. I can do this easily. more details in bmp. thanks

$490 USD dalam 14 hari
(17 Ulasan)

I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.

$250 USD dalam 5 hari
(3 Ulasan)

Interested in this project.

$300 USD dalam 5 hari
(9 Ulasan)

Hi, I'm interested.

$600 USD dalam 17 hari
(1 Ulasan)

Sir, I have done many similar projects before. I can get this crawler coded for you quickly and efficiently. If you want, I can code a compact GUI with a start/stop button to accompany the crawler. I am ready to begin Lagi

$125 USD dalam 7 hari
(8 Ulasan)

Hi Ser. I've read your requirements, and I can deliver application You need. I'm an expert in Python programming, and this task will be easily completed. For the start, I've written test application that does most of Lagi

$250 USD dalam 5 hari
(1 Ulasan)

I can deliver the crawler you require for you. please check PMB

$1500 USD dalam 7 hari
(2 Ulasan)

We are the group of the experts to satisfy you with the quality service. Please send the full details of the project to proceed further. Thanks and Regards.

$1000 USD dalam 20 hari
(0 Ulasan)

Hello, thank you for the chance to bid on this project for you. I am a professional computer programmer with over 10 years experience. I would love the chance to develop a professional application for you.

$225 USD dalam 7 hari
(0 Ulasan)

Hello, I'm ready to start the work.

$800 USD dalam 20 hari
(0 Ulasan)

Expert here..Lets start...

$150 USD dalam 7 hari
(0 Ulasan)

Hi, I'm a professional developer. I can do this project for you.

$200 USD dalam 10 hari
(0 Ulasan)

Hello, I am highly interested in this kind of project. I can get the job done.

$400 USD dalam 20 hari
(0 Ulasan)

I have done this kind of project before in Python. I know how to use multiprocessing/gevent to achieve parallelism and asynchronous I/O (fetch multiple Flickr results at once, make smart use of computer resources).

$500 USD dalam 5 hari
(0 Ulasan)

I'm an experienced system administrator working with similar scripts daily. I'll be glad to do this for you.

$120 USD dalam 6 hari
(0 Ulasan)