ETL Jobs

I need help turning a backlog of paper-based and scanned financial records into clean, searchable data inside my PostgreSQL database. All source documents are organised by month in shared cloud folders; your job is to review each file, capture every relevant field, and insert it directly into the tables I specify (or propose a tighter schema if you spot improvements). Accuracy matters more than speed—balances must tie out and numeric formats must stay consistent with the existing data. I’ve already set up secure, limited-privilege access to the database, and can supply ER diagrams plus sample rows so you see the conventions to follow. If you prefer to stage the work in a local instance before pushing to production, that’s fine with me. Deliverables • Populated Post...

I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and database connectio...

Job Title: Senior Cloud Database Engineer – PostgreSQL Migration Architect Location: Remote Work Timing: UK Business Hours (GMT/BST) Engagement Type: Freelance (Long-Term Possible) Platform: Freelancer.com Project Overview We are seeking a highly skilled and experienced Senior Cloud Database Engineer with deep specialization in PostgreSQL migrations and distributed database infrastructure. This role is for a demanding enterprise client, so we require a proven track record of successful large-scale cloud database migrations with verifiable experience. You must be comfortable working during UK business hours and collaborating with cross-functional cloud and infrastructure teams. Role Summary Senior Cloud Database Engineer – PostgreSQL Migration Architect – Data Infras...

Job Description We need a disciplined SQL/data engineer to produce a deterministic, re-runnable script-based build (lean MVP, not enterprise infra). Goal Create a single canonical, one-row-per-company dataset anchored on UEI, then enrich it using exact UEI matches (no name guessing). Inputs (provided) USAspending extract / source tables (L24M window) DSBS export (CSV) with UEI + owner contact fields SAM Active Entity extract (CSV) with UEI + phone + officer names Written Segment B & C rules (explicit) Deliverables 1. USAspending segmentation + canonical vendor table >Filter to last 24 months >Vendor-level aggregation → one row per UEI >Apply Segment B & C logic exactly as written >Deterministic CSV output >Basic QA checks: a. no duplicate UEIs in outp...

I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and database connectio...

Job Title: Technical Specialist working US hours Reports To: Technical Services Manager Job Overview We are seeking a resourceful and proactive Technical Specialist to join our Technical Services Team. As part of a team that provides custom solutions for clients, including custom reports, scripts, integrations, and data migrations, you will play a key role in bridging the gap between clients' business needs and our development team’s technical expertise. This role requires strong analytical skills, initiative, and the ability to problem-solve independently. You will also work closely with developers, Customer Success Managers, and clients to deliver tailored solutions. Responsibilities • Data Migrations: Oversee data migration processes from legacy systems to our newest pr...

Overview We are looking for a Senior Data Engineer to join the team that owns our core data product, the platforms and infrastructure that support it, and the data services consumed by our customers. You will work closely with other engineers, product stakeholders, and customers to design, build, and operate production-grade data systems, focusing on reliable ingestion, transformation, and exposure of data through APIs and product integrations. As a senior member of the team, you will be expected to help set technical standards, improve data quality and operational reliability, and contribute to shared ownership of systems from design through to reliable operation in production environments. What you’ll do • Design, build, and maintain data ingestion pipelines • Develop and...

- **Core Architecture:** Spring Cloud + Kafka + Hadoop + Python Automation，This project requires a certain level of technical expertise.

I run a data platform built on Spark and Python, and I need an experienced engineer who can sit with me in Pune three to four times a week (hardly 2 hours in a day) to keep it running smoothly. Most of the immediate work revolves around tracking down bugs in existing PySpark jobs, but the role naturally extends to writing fresh code when gaps appear, tightening our data-pipeline orchestration, and mapping end-to-end data lineage so every downstream consumer stays confident in their numbers. Typical day-to-day work you will tackle: • Debug production PySpark jobs and accompanying Python utilities • Refactor or rewrite modules where quick fixes will not suffice • Optimise and monitor pipeline schedules (Palantir Foundry) • Document lineage and hand off clear, repro...

I have an existing analytics initiative that now needs a dedicated Redshift-based warehouse. The core objective is to design and implement a robust schema in Amazon Redshift, then ingest data coming from three different sources—our operational SQL databases, a set of RESTful APIs, and periodic flat-file drops in CSV or JSON. Here is what I’m aiming for: • A well-structured Redshift warehouse (star or snowflake schema, whichever is most appropriate) built to scale and documented clearly. • Reliable, automated ingestion pipelines for each source type. For SQL we currently use PostgreSQL and MySQL; for APIs the payloads are mostly JSON; the flat files live in S3. • Transformations that standardise data types, handle slowly changing dimensions, and enforce dat...

I’m standing up a new analytics environment in Azure and I need an expert who can take full ownership of three intertwined streams of work: • Data migration – Build end-to-end pipelines in Azure Data Factory that handle extraction, transformation, and loading from our on-prem sources into an Azure Data Lake. Azure Functions or Logic Apps can be used where they add value for event-driven steps or lightweight transformations. • Security & RBAC – Design and apply role-based access controls across the whole solution so that data in the lake, ADF, and any supporting services stays locked to the right teams. Policy-driven best-practice is a must. • CI/CD – Set up a clean, automated release flow (Azure DevOps or GitHub Actions—your choice) tha...

I have several disparate sources holding our inventory information and I want it all pulled together into one clean, well-structured Microsoft SQL Server database file. The job is straightforward in principle: extract every piece of inventory data you can reach, reconcile duplicate SKUs or mismatched item descriptions, then load the final result into a single .sql or .bak file that I can restore through SQL Server Management Studio. Please preserve every existing field that carries operational value—quantities on hand, reorder thresholds, location codes, cost figures, and any audit timestamps—and normalise the schema where it clearly improves reporting speed or data integrity. If a field appears under different names in different systems, map it consistently; if you uncover ga...

Estoy por arrancar una migración con Informatica PowerCenter y necesito apoyo directo en la construcción de los procesos ETL. El conjunto de datos con el que trabajaremos está en Oracle y lo extraeremos desde una base de datos SQL; el objetivo es una migración limpia, controlada y auditable hacia el nuevo esquema. Qué espero de ti • Diseñar y desarrollar mappings, sesiones y workflows en PowerCenter que cubran todo el flujo de extracción, transformación y carga. • Incluir manejo de errores, validaciones de calidad y registro detallado de eventos. • Documentar cada mapping en el repositorio y entregar un pequeño manual de despliegue/rollback. Aceptaremos el trabajo cuando: - Todos los procesos se ejecu...

Project Title: Hiring Experts for IT and Digital Transformation Roles Project Description: We are seeking highly qualified professionals to join our team for a comprehensive digital transformation initiative. The roles require expertise in IT, digital infrastructure, social protection, and legal frameworks. Below are the positions and their respective qualifications: 1. Senior Project Manager - PMP or PRINCE2 certification - 10 years of experience managing complex projects in IT/e-government 2. Principal IT Architect - Degree in Computer Engineering or equivalent, TOGAF certification - 10 years of experience as an enterprise architect with expertise in API, ETL, cloud technologies 3. Business Architects/Analysts - Degree or Master’s in Computer Security, Data Science, or Computer...