Hadoop & Big Data
Healthcare, Education, IT/Software, Media/ Entertainment
US Citizen, Green Card, OPT-EAD, CPT, H-1B,
H4-EAD, L2-EAD, GC-EAD
To Be Discussed Later
W-2 (Consultant must be on our company payroll. C2C is not allowed)
• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
• Experience with integration of data from multiple data sources, Implementing ETL process using APACHE NIFI
• Monitoring performance and advising any necessary infrastructure changes
• Defining data retention policies
• Management of Hadoop cluster, with all included services such as Hive, HBase, mapReduce and Sqoop
• Cleaning data as per business requirements using streaming API’s or user defined functions.
• Build distributed, reliable and scalable data pipelines to ingest and process data in real-time, defining Hadoop Job Flows.
• Managing Hadoop jobs using scheduler.
• Apply different HDFS formats and structure like Parquet, Avro, etc. to speed up analytics.
• Work with various hadoop ecosystem tools like Hive, pig, Hbase , spark etc.
• Experience with Spark, SparkR, Python, Scala
• Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
• Reviewing and managing Hadoop log files.
• Assess the quality of datasets for a hadoop data lake.
• Fine tune Hadoop applications for high performance and throughput.
• Troubleshoot and debug any Hadoop ecosystem run time issues.
• Being a part of a POC effort to help build new Hadoop clusters.
DIGITAL TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. DIGITAL TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will DIGITAL TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract