Ditutup

create example scripts using celery and rabbidmq to run different tasks on different remote nodes

I'm creating a distributed system. But I'm not aware of Celery/RabbitMQ setups.

My plan is to create 5 different workers with different tasks (example):

- Worker1 - send icmp request

- Worker2 - scrap a url content

- Worker3 - execute a whois on a domain

- Worker4 - get a list of N urls from "somewhere" (could be the same rabbitmq?) and fetch the title tag.

- Worker5 - fetch all ips related with those 300 urls.

When the celery client send a task request through the broker:

- I want to scrap this url: a.b.c

The worker2 should respond and execute the task, and return the result back to the celery client. All the other workers should never receive this request, as those don't have the proper function/task.

When the celery client send a task request through the broker:

- I want to fetch the title tag from a list of 230 or 1000 websites

The Worker4 should respond, fetch the list of websites, execute the task and return the result back to the celery client.

Each worker runs on a different linux box on the internet.

I will also need the ability of have a pool of worker1 boxes. Let's say 5 linux boxes with the same worker1 software, and the first to respond will perform the task (Round-robin).

The celery client must be a Flask page with:

- a table with 10 domains and 5 urls

- When the user click on one of those domains, and hit the button 'run whois', the celery task must be triggered

- When the user select 3 of those urls and select the option "fetch title", the celery task must be triggered

The response must also be presented via flask webapp (can be a different page)

Kemahiran: Python, DevOps, Data Scraping, Scripting

Lihat lagi: create flyers trifolds using adobe photoshop, create floor plan using autocad, create online admission using php javascript html, create dyanamic website using php, create website voting using java mysql tomcat, web accounting example scripts, web banking example scripts php mysql, create contact form using formailpl, create simple banner using css, create registration form using mysql, create report php using pdf, create image swapping using flash, create xml file using mysql, vlc example scripts, create acknowledgement banner using scripts, example jboss create web service using annotation, example php code create order report using fpdf, create ear file using maven example, how to run selenium ide scripts using java eclipse, provide example scripts for automating the ui testing of a video player like youtube using selenium

Tentang Majikan:
( 0 ulasan ) Brazil

ID Projek: #28332447

19 pekerja bebas membida secara purata $559 untuk pekerjaan ini

lerashestakova02

Hello I saw that you need help with remote auto script . I have 5 years of experience working in these jobs. I believe I can help you with it. If you find me worthy of doing your job. Send a message, please Valeriya S

$750 USD dalam 8 hari
(17 Ulasan)
5.6
(15 Ulasan)
5.6
benliao

Hello I have read your job description carefully. Also, I have rich experience in python and web scraping So I am interested in and I am confident in your project. I always provide perfect results to my clients as you Lagi

$500 USD dalam 8 hari
(6 Ulasan)
4.2
(8 Ulasan)
4.2
DRG1990

Hi, I summarizes all descriptions and provides clear and concise explanations. I am an expert of Scraping with Python experience. I have done several similar projects in the backend development of numerous internet s Lagi

$250 USD dalam 5 hari
(5 Ulasan)
4.1
braincenter

Hello, Hope this message finds you well, I checked your details and I believe that my experience is what you are looking 4. I have been working on similar projects for the past eight years, and I have the essential sk Lagi

$500 USD dalam 7 hari
(10 Ulasan)
4.3
nikolai198061

Hi, client. I am a senior Python developer who has rich experiences. Especially, I have professional experiences in Web Scraping. I've successfully done a lot of projects to scrape data like email, URL, ads, contact, p Lagi

$500 USD dalam 7 hari
(3 Ulasan)
2.8
kholmogorovadev

Hi. I am a Flask expert I know Celery/RabbitMQ I can help you as well I'll provide you good result Please contact me best regards Aleksandra

$500 USD dalam 7 hari
(3 Ulasan)
2.9
sharmashivangi00

Hello, Hope you are doing well..!! I am a Full Stack Developer with over 4 years of experience in Python backend frameworks like Django and Flask along with the frontend frameworks like React.js and Angular. I have b Lagi

$500 USD dalam 7 hari
(3 Ulasan)
3.3
rostikhr111

Hi, How are you? I am very interested in your project. I have rich experience in web development using Flask and Django. Especially I have rich experience in Schedule, Asynchronous using Celery, AsyncIO. I am sure th Lagi

$500 USD dalam 7 hari
(1 Ulasan)
2.7
demydkasilov

Hi, sir I hope you are doing well and staying safely without COVID-19 effect. I have carfully checked your requirement. I am a web scraping expert and have good experience with scraping using python, node.js and php fo Lagi

$500 USD dalam 7 hari
(2 Ulasan)
2.4
Rustamdev

Hello. I have rich experiences in python scraping. So, if you hire me, I will do my best for you and for your project. Thanks. Rustam.

$250 USD dalam 7 hari
(1 Ulasan)
1.8
stepinnsolution

Hello man. I am a full stack developer and have around about 12+ years of experience in the development field. I am the CEO of Stepinnsolution. Stepin Solution to award leading the digital world. I want to tell you We Lagi

$1500 USD dalam 7 hari
(1 Ulasan)
0.6
AndreiiWebGats

Hello, Nice to meet you. I am very pleased to present this proposal for your consideration as a fullstack developer. I have deep knowledge in real-time data processing and experience in RabbitMQ, Redis, socket.io. One Lagi

$500 USD dalam 7 hari
(1 Ulasan)
0.3
shanarana001

Hello successful buyer I have go through your request and it is fully understand and i can do it perfectly and the work will be done professionally and perfectly because am an expert in this field Kindly contact me now Lagi

$250 USD dalam 7 hari
(0 Ulasan)
0.0
idevenster

Can I have a go at your project? I have been doing Python, Java, Node Js for the past 10 years. I think I can help you out. I don't need money as such for now

$500 USD dalam 3 hari
(0 Ulasan)
0.0
iwtechnocrats41

HI, Hope you are well! I have gone through your requirement properly and our company's main policy is to create and maintain long-term relationships rather than earning profit. This would be a win-win situation for Lagi

$500 USD dalam 30 hari
(1 Ulasan)
0.0
IWTechnocrats

Dear Sir/Ma'am, Thanks for sharing your project requirement with us. I have 6+ year of experience in web development. It will be our great pleasure to work on your project. I have checked your requirement regarding we Lagi

$500 USD dalam 7 hari
(1 Ulasan)
0.0
sarohirose

Hello, I have checked The requirement and understand that as well. I have created SIMILAR in past. Do you want to see the DEMO ??? Can will send you ?? Thanks.

$667 USD dalam 3 hari
(0 Ulasan)
0.0