Need Apache Nifi Cluster Active/Active Datacenter diagram

project summary - there are vendors who will send XML files to a staging area (file-system) every 15 minutes ( shown in the attached diagram ), when a XML file is written to the staging area, Nifi will fetch those XML files and write to HDFS, once a XML file is written to HDFS, we have a scala application JAR running in a wildfly server which will get notified ([login to view URL]), based on the notification, that jar will submit a spark application programmitically to YARN resource Manager, YARN resource manager then starts the spark application master on a selected spark node (out of 5 spark nodes) and run the spark driver and eventually run the spark jobs in spark executors which is distributed on the spark worker nodes

the issue we are having now is in Nifi cluster, since we have 2 datacenters, and clients are sending the xml files to a server location in one of the datacenter, how can Nifi fetch these XML files from staging area and proceed with the flow, if one datacenter goes down, how can we redirect users to the other nifi cluster in other datacenter and resume the processing, if so, how we can sync the two staging areas in both datacenters, this is for Active / Active datacenter deployment

Kemahiran: Pemprosesan Data, Apache Hadoop, Spark, Data Extraction, Sains Data

Lihat lagi: download free edt diagram creator project, download edt diagram creator project, active project, siemens plc ladder diagram conveyor project, diagram science project, erd diagram pharmacy project, network diagram visio project, circuit diagram microprocessor project, diagram mini project cryptography, context diagram example project tracker system, dataflow diagram online project management, need apache tomcat mobile application j2me, activity diagram website project, build uml diagram flex project, circuit diagram arm project, cluster management software student project, data flow diagram computer project, data flow diagram online project management, dfd diagram database project, er diagram j2ee project

Tentang Majikan:
( 0 ulasan ) Kegalle, Sri Lanka

ID Projek: #29430268

2 pekerja bebas membida secara purata $110 untuk pekerjaan ini


My self Fahim Ullah I am IT student 2 years experience relevant field I need job and money plz take me this project.

$20 USD dalam 7 hari
(0 Ulasan)

Hello There, I think I can help you in this project. Lets have a call to discuss more on the issues you are facing and requirements you want then we can proceed with the actual implementation. Thanks.

$200 USD dalam 4 hari
(0 Ulasan)