The practice of designing and building systems for collecting, storing, and analyzing data at scale. It is a broad field with applications in just about every industry. Organizations have the ability to collect massive amounts of data, and they need the right people and technology to ensure it is in a highly usable state by the time it reaches data scientists and analysts.

rn

Often deal with the unknown by using more advanced data techniques to make predictions about the future. They might automate their own machine learning algorithms or design predictive modeling processes that can handle both structured and unstructured data. This role is generally considered a more advanced version of a data analyst. Some day-to-day tasks might include: Gathering, cleaning, and processing raw data; Designing predictive models and machine learning algorithms to mine big data sets; Developing tools and processes to monitor and analyze data accuracy; Building data visualization tools, dashboards, and reports; Writing programs to automate data collection and processing

Daripada 1,195 ulasan, klien menilai Data Engineers 4.84 daripada 5 bintang.
Upah Data Engineers

Tapis

Carian terbaru saya
Tapis mengikut:
Bajet
hingga
hingga
hingga
Jenis
Kemahiran
Bahasa
    Status Pekerjaan
    6 tugasan ditemui, harga dalam USD

    Hi, I am one of the core members of an EduTech startup and we are building course contents on various tech stacks including Data Analytics, Data Science & Data Engineering. On DS & DA, we already have enough creators, we need some senior Data Engineers to create end to end video courses for us.

    $1365 (Avg Bid)
    $1365 Avg Bida
    4 bida

    I built an ETL pipeline to process terabytes of data. To achieve that goal, I setup a Spark Cluster (Scala) and MinIO server for object data storage. I can process and save 200 gigabytes in roughly 30 minutes using 10 virtual machines, for Spark Processing. The issue I have is that I am not able to scale that Processing. Meaning if I double the number of spark virtual machines, this does not affect processing time. I need a Data Architect who has enough expertise to help me identify the bottleneck and fix the issue. ARCHITECTURE SUMMARY. • I use virtual machines set up on-premises using VMWare ESXi 6 • Physical machines (which host VMs) are on a 1 GB network. • There is no over commitment for vCPU nor RAM • Spark VMs. 16VCPU, 64 GB RAM • MinIO (Storage). 16vCPU, ...

    $275 (Avg Bid)
    $275 Avg Bida
    1 bida

    Need anyone who has hands-on Experience in spark and python. Minimum of 2 years experience. Should be able to code in pyspark.

    $336 (Avg Bid)
    $336 Avg Bida
    7 bida

    Hi there I need somone who can help me with list of things to be considered before moving to databricks with code base already running locally. I need help with list of things to be considered before moving to databricks. I guess person with data engineering background can help me Thanks

    $80 (Avg Bid)
    $80 Avg Bida
    16 bida

    Hi there I need somone who can help me with list of things to be considered before moving to databricks with code base already running locally. I need help with list of things to be considered before moving to databricks. I guess person with data engineering background can help me Thanks

    $91 (Avg Bid)
    $91 Avg Bida
    14 bida
    AI Data Analysis 1 hari left
    DISAHKAN

    Hi I need an AI system to conduct data analytics Further information can be provided in chat

    $135 (Avg Bid)
    $135 Avg Bida
    34 bida

    Artikel yang Disyorkan Hanya untuk Anda

    Communication is the key to a successful project that generates great reviews.
    3 MIN READ
    Learn how to thrive and succeed as a digital nomad.
    3 MIN READ
    Freelancing while studying: make extra money & build your experience flexibly.
    4 MIN READ