
Open
Posted
•
Ends in 5 days
Paid on delivery
I need an experienced big data specialist to convert 5 TB of flat JSONL data stored in an S3 bucket on AWS into Parquet format with modulo hashing for optimized query performance. Requirements: - Use AWS EMR for the conversion - Handle flat JSON (key-value pairs) structure - Implement specific field hashing as per my requirements Ideal Skills and Experience: - Proficient in AWS EMR - Strong background in data format conversion - Experience with JSONL and Parquet - Knowledge of modulo hashing and query optimization Please provide a detailed approach and timeline.
Project ID: 39718103
16 proposals
Open for bidding
Remote project
Active 10 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
16 freelancers are bidding on average ₹7,325 INR for this job

I work in python/javascript/reactjs/AWS/AZURE/ETL I have good knowledge of database, cloud deployment and docker containarization of apps. I have worked with aws lambdas/ serverless/ fargate I worked with creating chat bots using chatgpt using rag pipeline I would like to work on this project. Lets discuss more on this project
₹7,000 INR in 7 days
4.9
4.9

As a seasoned Data Scientist and Programmer, I possess the technical prowess and depth of experience necessary to tackle your big data conversion project. Well acquainted with AWS EMR, JSONL, and Parquet data formats, I assure you of a smooth transition from your 5 TB flat JSONL data in an S3 bucket to optimized Parquet format using modulo hashing. My comprehensive understanding of Python and Big Data will enable me to handle your project expertly, ensuring minimal disruption and maximum efficiency. Over the years, I have accumulated extensive experience as a Data Analyst and Model Builder handling large datasets in Python, utilizing libraries such as Pandas and NumPy for thorough data exploration and manipulation. You can count on my skills to analyze the structure of your flat JSON files deftly while carefully preserving valuable key-value pairs during the conversion process. In conclusion, my knowledge extends beyond mere conversion. I guarantee a successful completion of this project that involves not just converting raw data but also optimizing it for efficient querying. I am dedicated to delivering high-quality solutions within stipulated timelines, ensuring that your investment translates into tangible value for your business. Partner with me for your big data conversion needs; together we'll unravel great insights from your vast data stores.
₹12,500 INR in 2 days
4.4
4.4

Hi there, I can convert your big data datasets into Parquet format, ensuring efficient storage, faster querying, and compatibility with analytics tools. The focus will be on data integrity, optimization, and scalability for large datasets. Once I have your current data sources, schema details, and target storage environment, I’ll deliver clean, optimized Parquet files ready for analytics or processing. Best regards, Waleed Saleem
₹3,000 INR in 3 days
2.5
2.5

I am a perfect fit for your project. I understand the need to convert 5 TB of flat JSONL data in an S3 bucket to Parquet format with specific modulo hashing for optimized queries. While I am new to freelancer, I have tons of experience and have successfully completed similar data conversion projects off-site. I possess expertise in AWS EMR, data format conversion, JSONL, Parquet, and query optimization. I can efficiently handle your requirements and deliver high-quality results. I would love to chat more about your project! Regards, Tiffany Pienaar
₹6,250 INR in 14 days
0.0
0.0

"I see a great alignment between the project needs and my capabilities! I have a strong background in data format conversion and specific expertise in handling JSONL data and Parquet format. Utilizing AWS EMR for the conversion and implementing modulo hashing align perfectly with my skill set. I may be new to Freelancer, but I have a strong background and a proven track record with projects I've completed off-platform. I'm confident in my ability to deliver a clean, professional, and optimized solution for your data conversion needs. I'd really love to hear more about your project! I'm happy to offer a free initial consultation to help get things started and see how I can best support your goals. Kind Regards, Juan K"
₹6,250 INR in 14 days
0.0
0.0

"I am a perfect fit for your project. I specialize in big data processing and can efficiently convert your 5 TB of flat JSONL data into Parquet format with modulo hashing for optimized query performance." While I am new to freelancer, I have tons of experience and have done other projects off site. I have extensive experience working with AWS EMR, handling various data structures, and implementing custom hashing techniques for query optimization. I am confident in my ability to deliver a clean, professional, and seamless conversion process tailored to your specific requirements. I would love to chat more about your project! Regards, Damian Badenhorst
₹6,250 INR in 20 days
0.0
0.0

❤️❤️❤️ Hi.❤️❤️❤️ Your project perfectly aligns with my skills and experience, and I’m ready to get started immediately. I’m an experienced data engineer with strong expertise in AWS EMR, JSONL → Parquet conversion, and modulo hashing. I can efficiently convert your 5 TB S3 dataset into optimized Parquet format with field-level hashing for fast queries. Approach: EMR-based parallel processing, schema validation, hashing for partitioning, and final output verification. I can start immediately and deliver a reliable, optimized solution quickly. Best regards, Melissa R
₹7,000 INR in 7 days
0.0
0.0

"Your idea is perfect for my expertise! I specialize in converting and optimizing large datasets.” I understand the importance of clean data transformation and seamless query performance. While I am new to freelancer.com, I have extensive experience in AWS EMR, JSONL to Parquet conversion, and modulo hashing. I would love to chat more about your project! Regards, Dylan
₹8,150 INR in 14 days
0.0
0.0

As a seasoned full stack developer with deep expertise in AWS and Python, I believe I'm the perfect fit for your Big Data Conversion to Parquet project. Over the years, I've developed an intimate familiarity with tools like EMR that are essential to the conversion process. I'm confident that I can handle the conversion of your 5 TB flat JSONL data in an efficient and accurate manner using modulo hashing. My experience extends to data format conversion, be it from flat JSON to optimized Parquet, where I ensure a structured approach for better query pipelines and overall performance. Being skilled in Python is another advantage I bring to the table, as it's a widely utilized language for effective big data management. What sets me apart is my passion for innovation and performance—a commitment that aligns perfectly with your need for optimized query performance. As your ideal freelancer, my goal is to deliver flawlessly functioning, efficient solutions on time and within budget. Save yourself from potential headaches and trust in my track record of consistent high-quality project delivery. Let's turn this big data challenge into an opportunity for tangible growth and optimization.
₹2,000 INR in 5 days
0.0
0.0

Hi, I can easily DO your work IN 24 HOURS, DM me now to get started, PRICE NEGOTIABLE 100% Work satisfaction is provided!
₹11,000 INR in 2 days
0.0
0.0

Hi there, I can help with converting 5 TB of flat JSONL data on AWS to Parquet format using EMR for optimized query performance. Proficient in AWS EMR, data format conversion, JSONL, Parquet, and modulo hashing. While I am new to Freelancer, I have years of related experience. Let's chat more about your project! Regards, Gordon
₹6,250 INR in 14 days
0.0
0.0

Hey, this looks interesting and right up my alley. I’ve done similar work before and can help you get this done quickly and properly. Let me know what you need most, and I’ll make it happen without any hassle. Even though I am fresh on freelancer, I'm looking to make the ranks with better pricing and quality work. Best Regards, CJ & TEAM
₹7,500 INR in 30 days
0.0
0.0

Skilled, reliable, and ready to bring your ideas to life. I understand your need to convert 5TB of flat JSONL data to Parquet format using AWS EMR and modulo hashing for optimized queries. While I am new to Freelancer, I have ample experience in data format conversion and query optimization off-site. I would love to chat more about your project! Regards, Leonard Boucher
₹9,400 INR in 14 days
0.0
0.0

I am a perfect fit for your project as an expert in Big Data conversion. I will seamlessly convert your 5 TB JSONL data to Parquet format on AWS EMR, implementing modulo hashing for optimal query performance. While I am new to freelancer, I have tons of experience and have done other projects offsite. I specialize in AWS EMR, data format conversion, JSONL, Parquet, and query optimization. Let's discuss how my skills can elevate your project's efficiency! Regards, Saint Sambo
₹6,250 INR in 14 days
0.0
0.0

I am a perfect fit for your project as a skilled big data specialist. I understand the need to convert 5 TB of flat JSONL data in an AWS S3 bucket to Parquet format with modulo hashing for optimized queries. While I am new to freelancer, I have tons of experience and have done other projects off-site. I have expertise in AWS EMR, data format conversion, JSONL, Parquet, and query optimization. I would love to chat more about your project! Regards, Brandon Mitchell
₹9,400 INR in 14 days
0.0
0.0

I have 1 year of industrial experience with hands-on expertise in working with databases on both AWS and Azure platforms. I have worked extensively with various data formats such as Parquet, Delta, JSON, and many more. Additionally, I am skilled at efficiently handling and processing large volumes of data with accuracy and speed.
₹9,000.09 INR in 7 days
0.0
0.0

RAMPUR, India
Payment method verified
Member since Dec 5, 2020
₹1500-12500 INR
₹12500-37500 INR
₹41000-61000 INR
₹1500-12500 INR
₹12500-37500 INR
$2-8 USD / hour
$2-8 USD / hour
$10-30 USD
$2-8 AUD / hour
$30-250 AUD
$15-25 USD / hour
₹1500-12500 INR
$50-500 USD
₹750-1250 INR / hour
$30-250 USD
₹1500-12500 INR
₹1250-2500 INR / hour
₹600-601 INR
₹1500-12500 INR
$250-750 USD
$30-250 USD
$750-1500 USD
$10000-20000 AUD
£250-750 GBP
$30-250 AUD