BigData: instalación, carga y consultas

Hacer 4 instalaciones Linux BDs Cloudera/Hadoop/Impala

Desarrollar 3 programa batch en Linux para carga de registros, leer desde archivos en fmto textoCSV y cargarlos al BigData. Cada archivo textoCSV tiene 500 registros y cada registro tiene 278 campos.

Implementar una herramienta de consultas sobre los registros del BigData.

Todo desarrollado sobre herramientas opensource sobre Linux.

Experience Level: Intermediate
Data Entry Features, Big Data records:
2.1. Registry Size: 554 alphanumerics positions
2.2. Number of fields into each record: 125 fields within the 554 positions.
2.3. Number of records in one day: 18* million records.
2.4. Daily files: 1,100* files in hexadecimal format.
2.5. Each file contains 13,000* records.
2.6. The structure and size record is constant and is a one type de record. (all requirement es for one type record)

Batch Process for: (maybe in Talend)
1.1.1. New files detector that arrive to the server for the charged process. A process outside requirement, leave hex files via FTP. This FileListener process once it detects that are a new files, you must process them to load their information BigDataBase. This Include the some rutine to not read/process two times the same file.
1.1.2. Data file reader in hexadecimal format to load into the BigData base. (could be with low level scripts .sh, perl, c+ or any tool for load)
1.1.3. Remove last data for purposes of query optimization.
1.1.4. Past data loader for purposes of reloads data that was deleted. Read in hexadecimal format too. (on demand)

4. BigDatabase
4.1. Suggest buy not a relational. NoSQL or Hadoop based or any BigDatabase.

3. Softwares:
3.1. All user interfaces (GUI) should be Web Enabled.
3.2. All solutions, softwares, programs and tools, must be on LINUX operating system (CentOSv7.X) and Opensource licensing.

5. Provide.
5.1. Four Installation of each Bigdatabase and ETL tools: (1) development, (2) training, (3) installation at the end customer - the productive environment, (4,5) on our demand.
5.2. Manuals step by step: of Installation of bigDatabase; Installation of ETL tool.
5.3. Session training for the support local team for this instalación. (by sky, remotely and by english) Manuals Step by Step by demo online:
5.3.1. Installation of the tools Big Database and ETL.
5.3.2. Maintenance operations, support and basic troubleshooting of the tools BigDatabase and ETL.

Kemahiran: Big Data Sales

Lihat lagi: consultas con web service oracle desde sharp, aspnet code using drag n drop, carga datos con java mysql, php designer 2007 registro key, instalaci, carga datos web desde mysql, como realizar count registro php report maker, freelancer consultas sql, campos daniel gustavo

Tentang Majikan:
( 2 ulasan ) Lima, Peru

ID Projek: #8838775

8 pekerja bebas membida secara purata $8575 untuk pekerjaan ini


--> Having 6 years of IT experience with 3 years on Big Data, Hadoop Stack and Hadoop Eco System --> Cloudera Certified Hadoop Developer for Apache Hadoop --> Good Experience in Writing Hadoop articles to Techincal b Lagi

$8888 USD dalam 60 hari
(4 Ulasan)

Hi sir, I have read your requirement carefully. I am interested to work for this project . If you wanna confirm me, please check my work history and portfolio kindly. Then, you can know about my ability that can Lagi

$9000 USD dalam 30 hari
(0 Ulasan)

Hola, puedo desarrollar tu proyecto por favor contáctese via chat para discutir el proyecto... Saludos, Ignacio, Austral Design

$8421 USD dalam 40 hari
(0 Ulasan)

Hi, I have read your post and understood your requirement. I have great experience in handling /PHP/MySQL/HTML5/jQuery/Wordpress/Magento/Joomla/Drupal/AngularJS/node.js/CSS3/Java/Python/Django/Javascript/iOS/Andr Lagi

$7731 USD dalam 75 hari
(0 Ulasan)

A proposal has not yet been provided

$7894 USD dalam 30 hari
(0 Ulasan)

Buenos días, Soy Jesús Moral, Director y fundador de Nakima Solutions, una empresa tecnológica dedicada a ofrecer servicios informáticos a empresas. Estamos especializados en sector FINTEC (tecnologías aplicadas Lagi

$8333 USD dalam 3 hari
(0 Ulasan)

Tenemos realizadas instalaciones de herramtas de Big Data y manejamos scripting. La mayoria de nuestros desarrollos son para servidores linux

$10000 USD dalam 90 hari
(0 Ulasan)

A proposal has not yet been provided

$8333 USD dalam 30 hari
(0 Ulasan)