Data Engineer 3(Big Data/Python)
Position: Data Engineer 3
Austin, TX (100% remote is fine)
Duration: 3 months Contract
We are looking for a senior data engineer with expertise in Big data technologies ( Hadoop, hive etc)
Translate complex functional and technical requirements into detailed design.
Design for now and future success
Hadoop technical development and implementation.
Loading from disparate data sets. by leveraging various big data technology e.g. Kafka
Pre-processing using Hive, Impala, Spark, and Pig
Design and implement data modeling
Maintain security and data privacy in an environment secured using Kerberos and LDAP
High-speed querying using in-memory technologies such as Spark.
Following and contributing best engineering practice for source control, release management, deployment etc
Production support, job scheduling/monitoring, ETL data quality, data freshness reporting
5+years of Python development experience
3+ years of demonstrated technical proficiency with Hadoop and big data projects
5-8 years of demonstrated experience and success in data modeling
Fluent in writing shell scripts [bash]
Writing high-performance, reliable and maintainablecode.
Ability to write MapReduce jobs
Ability to setup, maintain, and implement Kafka topics and processes