Press ENTER to skip to the job description.
Data Engineer

Data Engineer
TX, Austin

Job Description

Position- Data Engineer

Responsibilities include:

  • Translate complex functional and technical requirements into detailed design.
  • Design for now and future success
  • Hadoop technical development and implementation.
  • Loading from disparate data sets. by leveraging various big data technology e.g. Kafka
  • Pre-processing using Hive, Impala, Spark, and Pig
  • Design and implement data modeling
  • Maintain security and data privacy in an environment secured using Kerberos and LDAP.
  • High-speed querying using in-memory technologies such as Spark.
  • Following and contributing best engineering practice for source control, release management, deployment etc.
  • Production support, job scheduling/monitoring, ETL data quality, data freshness reporting

Skills Required:

  • 5+years of Python development experience.
  • 3+ years of demonstrated technical proficiency with Hadoop and big data projects.
  • 5-8 years of demonstrated experience and success in data modeling.
  • Fluent in writing shell scripts [bash].
  • Writing high-performance, reliable and maintain able code.
  • Ability to write MapReduce jobs.
  • Ability to setup, maintain, and implement Kafka topics and processes.

Apply Now