
Closed
Posted
I have several datasets coming in from different systems where the numeric fields are stored as plain ASCII files in some cases and UTF-8 exports in others. My goal is to run solid, reproducible data analysis on them, but first they need to be cleaned and standardised. Here is what I need from you: • Build a repeatable cleaning pipeline in Python (Pandas, NumPy) that detects the file encoding, converts everything to UTF-8 where necessary, and validates that all numeric columns are truly numeric—no stray characters, misplaced separators, or locale-specific quirks. • Where the raw files already sit in SQL Server, create T-SQL or Python-based routines that achieve the same checks and corrections in-database so I can schedule them in an Agent job. • Document each step clearly so I can port the logic later to R or even a small C/C++ utility if performance becomes critical. Acceptance criteria – The script ingests a folder full of mixed-encoding files and leaves a clean UTF-8 version plus a detailed log of any corrections made. – A sample table in SQL Server is fully cleansed using your routine, with before/after counts matching exactly. – Code is delivered via Git with concise README and in-line comments. If you are comfortable working with numeric data, encoding issues, and the usual Python + SQL toolkit, I’d love to see how you’d approach this.
Project ID: 40703752
134 proposals
Remote project
Active 2 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
134 freelancers are bidding on average $35 USD/hour for this job

Hello, I am Dr. Rajesh Rolen, PhD in Computer Science & Engineering, with experience of over 20+ years in SQL As a preferred freelancer in the top 1%, I have done 400+ projects here on freelancer.com, I have 4.9 ratings out of 5 on average, which showcases my quality of work and timely delivery. Key Highlights: - Free Hosting Support on the Cloud or any desired platform. - Free 3 months of post-delivery support to ensure that our client doesn’t face any challenges after the launch of the project. - Free Dedicated tester on projects to ensure quality delivery, so clients don’t need to act as a tester. - 10+ Years experience UI/UX team to ensure intuitive UI. Portfolio: https://www.freelancer.com/u/Microlent Please open the chat and send me a message, so we can have a more detailed discussion about the project to give you the project timeline and cost. Thank you for considering my services. I look forward to engaging in a productive conversation and understanding how I can be of assistance in bringing your project to life. Regards Rajesh Rolen
$25 USD in 40 days
7.7
7.7

Hello, I can build a reproducible numeric-data cleaning pipeline that handles mixed ASCII and UTF-8 exports, detects encoding issues, standardizes text to UTF-8, and validates numeric columns against stray characters, separators, and locale-specific formats. Using Pandas and NumPy, I’ll produce cleaned files alongside a detailed correction log and clear validation summaries. For data already stored in Microsoft SQL Server, I’ll implement equivalent T-SQL or Python routines suitable for scheduling through SQL Server Agent, including before/after row and validation counts to confirm no records are lost. The repository will include Git-ready code, a concise README, inline comments, and documentation structured so the logic can later be ported to R or C/C++. I’ll also test the workflow against a representative folder of mixed-encoding files and a sample SQL Server table, ensuring the acceptance criteria are demonstrably met. Which locale-specific numeric formats and file encodings should the pipeline prioritize, such as comma decimals, thousands separators, or legacy code pages? Regards, Muhammad Awais
$25 USD in 24 days
6.9
6.9

Hi, I am a Python data automation developer with 8 years of rich experience in software development, with a strong background in data cleaning, numeric validation, SQL Server, and reproducible ETL workflows. I am familiar with Python, Pandas, NumPy, SQL Server, T-SQL, encoding detection, UTF-8 conversion, numeric normalization, locale-specific separators, data validation, logging, Git, and automation. For your datasets, I can build a repeatable pipeline that detects and normalizes file encodings, cleans and validates numeric fields, logs every correction, and provides equivalent SQL Server routines that can be scheduled through SQL Server Agent. I'm an individual freelancer and can work on any time zone you want. Please contact me with the best time for you to have a quick chat. Thanks. Emile.
$25 USD in 40 days
5.8
5.8

Hello, We carefully studied the description of your project and we can confirm that we understand your needs and are also interested in your project. Our team has the necessary resources to start your project as soon as possible and complete it in a very short time. We are 25 years in this business and our technical specialists have strong experience in SQL, NumPy, Data Analysis, Pandas and other technologies relevant to your project. Please, review our profile https://www.freelancer.com/u/tangramua where you can find detailed information about our company, our portfolio, and the client's recent reviews. Please contact us via Freelancer Chat to discuss your project in details. Best regards, Sales department Tangram Canada Inc.
$30 USD in 5 days
5.9
5.9

Hi there, I went through your project description and understand you need a repeatable Python and SQL Server data-cleaning pipeline that will standardize mixed-encoding files and ensure numeric fields are consistently valid and analysis-ready. I will build the Python pipeline using Pandas and NumPy to detect file encodings, convert files to UTF-8, normalize numeric formats, identify stray characters, separators, locale-specific values, and log every correction. For SQL Server, I will create T-SQL or Python routines that apply equivalent validation and cleansing logic and can be scheduled through SQL Server Agent. I will validate the solution against mixed-encoding files and a representative SQL Server table, confirming before/after row counts and data integrity. I will also structure the code with reusable functions, concise inline comments, and a README explaining the processing logic so it can later be ported to R or C/C++. Could you confirm which encodings and numeric locale formats are most common in your incoming datasets? I will share a project blueprint within 24–48 hours of award and begin immediately. Kind Regards, Imran Ali
$25 USD in 40 days
5.2
5.2

You need a repeatable, auditable numeric cleaning workflow across mixed-encoding ASCII/UTF-8 inputs and (optionally) SQL Server, so I’ll deliver a Python pipeline that: 1) Scans each file, detects encoding, and converts to UTF-8 deterministically. 2) Applies strict numeric parsing to every declared numeric column using locale-robust normalization (e.g., handling thousands separators/decimal marks) and rejects/flags any stray characters. 3) Produces a cleaned UTF-8 output set plus a detailed correction log (cell/row/file context, rule applied, before/after values). 4) Includes testable, port-ready modules: parsing/normalization, validation, logging, and configuration-driven column rules. For SQL Server, I’ll provide equivalent in-database checks via T-SQL (or Python orchestration calling SQL) that mirror the Python logic so Agent jobs can run scheduled cleansing and produce exact before/after counts. Code will be delivered in Git with a concise README, inline comments, and runnable commands to reproduce results end-to-end.
$25 USD in 36 days
5.4
5.4

Hi, I’m a data analyst with 7+ years of experience in Python, Pandas, NumPy, SQL Server, and data cleaning. I can build a reproducible pipeline to detect and standardize file encodings, validate and clean numeric fields, and generate detailed correction logs. I can also implement the equivalent SQL Server routines for scheduled processing, with clear documentation and comments so the logic can be easily ported to R later. I’ll deliver clean UTF-8 outputs, validated SQL results, before/after reconciliation, and Git-ready code with a concise README. Best regards, Soufiane
$30 USD in 40 days
5.3
5.3

I am a data engineer with extensive experience in data cleaning, automation, and working with mixed-encoding datasets using Python and SQL. Proficient in libraries such as Pandas and NumPy, I have a proven track record of delivering robust data processing solutions that ensure accurate and consistent results for data analysis. I can build a repeatable cleaning pipeline in Python that automatically detects file encodings, converts data to UTF-8, and ensures all numeric columns are validated without errors like stray characters. Furthermore, I can implement T-SQL routines for SQL Server that perform similar validations, allowing these processes to run as scheduled Agent jobs. My background includes documenting procedures clearly, which aligns well with your requirement for easily porting logic to other languages like R or C/C++. I am interested in discussing your Numeric Data Cleaning Automation project further. Could you provide more details on the dataset sizes or any specific types of encoding quirks you've encountered? I would be happy to share more about my approach.
$25 USD in 40 days
5.3
5.3

Hi there, We can build a repeatable pipeline that detects file encoding, normalizes content to UTF-8, and validates numeric fields with explicit handling for stray characters, separators, and locale quirks. We will also mirror the same checks in SQL Server using a routine suitable for scheduled Agent execution, then document the logic so it can be ported later to R or C/C++. Our initial assumption is that the numeric formats and cleansing rules can be standardized from a representative sample. Best Regards, 8veer
$50 USD in 20 days
5.1
5.1

I would make the cleaning pipeline deterministic and auditable rather than silently coercing anything that happens to resemble a number. For the file workflow, I’ll normalize source encodings to UTF-8, validate numeric fields against configurable locale/separator rules, preserve exact row counts, and produce a detailed log showing every rejected or corrected value. Ambiguous values can be quarantined instead of being guessed. For SQL Server, I’ll reproduce the same validation logic using staged processing and TRY_CONVERT/T-SQL where appropriate, with the routine structured for SQL Server Agent scheduling. The final Git repository will include reusable Python/Pandas code, configuration, sample tests, before/after reconciliation and a concise README explaining the rules clearly enough to port later to R or C/C++. Rate: $35/hour. Estimated effort: 18 hours over 4 working days. I can start immediately after award.
$35 USD in 40 days
4.6
4.6

Hi There! I specialize in Python and SQL data-cleaning automation, with 9+ years of experience, a team of 62 professionals, 450+ completed projects, and Top 1% verified freelancer status. I can build a reproducible pipeline that detects mixed encodings, standardizes files to UTF-8, and validates numeric fields against stray characters, separators, and locale-specific formats. I’ll mirror the same validation and correction logic in SQL Server using T-SQL/Python, with detailed correction logs and before/after count checks so the process is safe to schedule through SQL Server Agent. The Git delivery will include a concise README and clear inline documentation designed for future R or C/C++ portability. Would you like the pipeline to preserve the original raw files untouched while writing the cleaned UTF-8 versions to a separate output folder?
$38 USD in 40 days
4.2
4.2

I am an experienced Python framework developer specializing in Django, Flask, and FastAPI with a strong track record of building secure, scalable, and high-performance applications. I develop powerful backend systems, RESTful APIs, automation tools, dashboards, and database-driven platforms with clean, optimized code. My focus is on speed, reliability, and long-term maintainability. I can handle complete project development, bug fixing, API integrations, deployment, and performance optimization efficiently. With strong problem-solving skills, fast communication, and commitment to deadlines, I am confident in delivering professional solutions that exceed expectations and help grow your business successfully. I appreciate the opportunity to submit this proposal and am excited about the possibility of working with you to bring your project to life. Thanks A.R.M MASUD
$38 USD in 40 days
4.6
4.6

Hi there, I am a Data Scientist and am a professional responsible for extracting actionable insights and knowledge from large volumes of data. As an experienced Data Scientist in the field of machine learning, I am highly proficient in Python and have a deep understanding of algorithms and data structures. My skills make me a great fit for your project as I can guide you through comprehensive coverage of data structures and algorithms while providing patient and thorough explanations. I have over 12-plus years of experience with Python Library Pandas, Karas, TensorFlow, NumPy, PyCharm, Py torch, Open CV, NLP, and others. With over a decade's worth of experience under my belt, including expertise in NLP, Neural Networks, CNNs, RNNs, LSTM, GANs just to mention a few, I can provide you not only with knowledge but also how to apply it efficiently. Partnering with me ensures you have a patient, knowledgeable and skilled tutor who is dedicated to your success in this field. My top priority is to provide a high quality of work, https://www.freelancer.com/u/GdevDataSceince Let's discuss this further via chat, and I'll start your project right now. Thanks Gdev
$38 USD in 40 days
4.4
4.4

Hi there, I appreciate the opportunity to help streamline your numeric data cleaning process. Your requirements for handling mixed-encoding files and ensuring robust, reproducible cleaning routines are right in my wheelhouse. With extensive experience in Python (Pandas, NumPy) and SQL Server, I've tackled similar challenges where data consistency and encoding can derail reliable analysis. My approach would be to develop a modular Python pipeline that auto-detects file encodings (using libraries like chardet), standardizes everything to UTF-8, and rigorously validates numeric columns—flagging and correcting non-numeric entries, misplaced separators, or locale-specific issues. Each cleaning step will be logged for transparency and auditability. For your SQL Server datasets, I can create efficient T-SQL scripts or Python routines (leveraging SQL Server's integration capabilities) to perform equivalent checks and corrections directly in-database, making them easy to schedule and maintain within SQL Agent jobs. All code will be versioned in Git, with a clear README and detailed inline documentation to facilitate future migration to R or C/C++, should you need higher performance. You can expect a process that ingests entire folders, outputs clean UTF-8 files, and provides detailed correction logs—meeting the acceptance criteria you've set. I'm excited to bring best practices in data engineering to your project and ensure your analysis starts with clean, trustworthy data. Best Regards! George
$25 USD in 14 days
4.0
4.0

Hi there, As a full-service technology and digital solutions agency, Aesthetic Logic specializes in precisely the kind of data cleaning and standardization tasks you need accomplished. With our comprehensive skill set and 4.9★ rating across 55 projects, we are adept at delivering on-time, quality solutions. Our expertise in Data Analysis combined with our deep knowledge of NumPy make us the ideal candidate for building your data cleaning pipeline in Python. We have successfully tackled similar projects in the past where mixed-encoding issues had to be resolved and text transformed to UTF-8. Further, our proficiency in SQL means we can provide a robust solution for cleaning and validating data within your SQL Server environment itself. Best regards, Mobasher Reza
$38 USD in 40 days
4.0
4.0

Hi, Your project is about making messy numeric data reliable before analysis begins , especially across mixed encodings and inconsistent formats. I can build a repeatable Python pipeline that detects file encoding, normalises everything to UTF-8, and validates numeric fields with clear logs for every correction. I’ve handled similar work with Pandas, NumPy, and SQL Server, including cleaning imported files, enforcing numeric rules, and preparing routines that can run inside Agent jobs. I’d keep the logic explicit and well documented so it can later be ported to R or a lightweight C/C++ tool without surprises. My approach would be to separate detection, cleansing, validation, and reporting into small steps, then mirror the same checks in T-SQL or Python for in-database processing. If you’d like, I can outline the pipeline and SQL workflow next. Best regards, Gabriel
$25 USD in 34 days
3.7
3.7

Hello, After reviewing your project requirements, I understand you need a reliable data cleaning pipeline to standardize mixed-encoding datasets, validate numeric fields, and create repeatable Python/SQL routines for future processing. I have experience with Python data analysis using Pandas and NumPy, SQL-based data processing, and building automated cleaning workflows. I can create a pipeline that detects file encodings, converts data into UTF-8 format, cleans numeric columns, handles separators and locale issues, and generates detailed correction logs. The main challenge will be ensuring the cleaning process is fully reproducible across different data sources. I will structure the solution with clear validation steps, SQL Server-compatible routines, and documentation so the workflow can be maintained or migrated later. I have a couple of quick questions: • What types of numeric inconsistencies appear most often in your datasets (decimal separators, missing values, special characters, etc.)? • Are the SQL Server tables using a fixed schema, or should the routine dynamically detect columns? I can deliver the scripts through Git with a clear README, comments, and test results showing before/after validation. Best regards, Carlos
$25 USD in 40 days
4.0
4.0

⭐⭐⭐⭐⭐ Built robust numeric‑data pipelines with encoding detection, strict validation and clean Python/T‑SQL automation ready for analysis.
$38 USD in 40 days
4.7
4.7

Hello we can use pandas Ai for making the pipelines more efficient would you like to talk a little ?
$29 USD in 10 days
3.6
3.6

Hi, Aashiq here from Cape Town, South Africa. This project instantly caught my eye, so I had to reach out. I see you’re looking for a solid pipeline in Python that handles mixed-encoding files and validates numeric fields. It’s clear you want a robust solution for data cleaning that easily integrates with SQL Server. I've successfully helped businesses streamline their data processes using Python and SQL, ensuring clean and accurate datasets. My experience includes creating repeatable cleaning pipelines and delivering detailed documentation. If you'd like, I can share samples of my previous work. Based on what you mentioned, here is how we would approach the project: - Analyze file encodings and convert them to UTF-8 - Validate and clean numeric columns with Pandas - Develop T-SQL routines for SQL Server integration - Provide thorough documentation for future adaptations You can count on me for clear communication and a seamless solution that’s user-focused and optimized for performance. Best Regards, Aashiq
$45 USD in 7 days
3.6
3.6

Riyadh, Saudi Arabia
Member since Jun 16, 2026
$15-25 USD / hour
$750-800 USD
₹12500-37500 INR
₹100-400 INR / hour
$250-750 USD
₹600-1500 INR
₹600-1500 INR
₹100-400 INR / hour
$10-16 USD / hour
₹1000-2000 INR / hour
₹12500-37500 INR
$15-25 USD / hour
$250-750 USD
₹12500-37500 INR
₹100-400 INR / hour
₹12500-37500 INR
₹12500-37500 INR
€20-50 EUR / hour
₹750-1250 INR / hour
$8-15 USD / hour
₹12500-37500 INR