W-2 Jobs Portal

  • W-2 Open Positions Need to be Filled Immediately. Consultant must be on our company payroll, Corp-to-Corp (C2C) is not allowed.
Candidates encouraged to apply directly using this portal. We do not accept resumes from other company/ third-party recruiters

Job Overview

  • Job ID:

    J36993

  • Specialized Area:

    BIG Data

  • Job Title:

    Big Data Engineer

  • Location:

    Costa Mesa,CA

  • Duration:

    12 Months

  • Domain Exposure:

    Government, Education, IT/Software

  • Work Authorization:

    US Citizen, Green Card, OPT-EAD, CPT, H-1B,
    H4-EAD, L2-EAD, GC-EAD

  • Client:

    To Be Discussed Later

  • Employment Type:

    W-2 (Consultant must be on our company payroll. C2C is not allowed)

  • Bench Recruiter:

    Thelma keller




Job Description

Primary responsibilities:

  • Designing Hive/HCatalog data model includes creating table definitions, file formats, compression techniques for Structured & Semi-structured data processing
  • Implementing Spark processing based ETL frameworks
  • Implementing Big data pipeline for Data Ingestion, Storage, Processing & Consumption
  • Modifying the Informatica-Teradata & Unix based data pipeline
  • Enhancing the Talend-Hive/Spark & Unix based data pipelines
  • Develop and Deploy Scala/Python based Spark Jobs for ETL processing
  • Strong SQL & DWH concepts

Key Requirements and Skills:

  • Cleanse, manipulate and analyze large datasets (Structured and Unstructured data XMLs, JSONs, PDFs) using Hadoop platform. o Develop Python, PySpark, Spark scripts to filter/cleanse/map/aggregate data.
  • Manage and implement data processes (Data Quality reports).
  • Develop data profiling, deduping logic, matching logic for analysis.
  • Programming Languages experience in Python, PySpark and Spark for data ingestion
  • Programming experience in BigData platform using Hadoop platform
  • Present ideas and recommendations on Hadoop and other technologies best use to management
  • 5+ years of experience in processing large volumes and variety of data (Structured and unstructured data, writing code for parallel processing, XMLS, JSONs, PDFs)
  • 3+ years of programming experience in Python, Spark for data processing and analysis.
  • Strong SQL experience is a must
  • 3+ years of experience using Hadoop platform and performing analysis. Familiarity with Hadoop cluster environment and configurations for resource management for analysis work
  • Detail oriented. Excellent communication skills (verbal and written)
  • Must be able to manage multiple priorities and meet deadlines
  • Degree in Statistics, Economics, Business, Mathematics, Computer Science or related field

Apply Now
Equal Opportunity Employer

DIGITAL TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. DIGITAL TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will DIGITAL TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract