Principal Software Engineer - Big Data
Position: Principal Big Data Engineer
Do you love working with petabytes of data using big data technologies? Do you possess deep expertise in designing and building the Big Data systems of tomorrow? Do you want to be part of an exciting group using ML chartered to implement the next generation Data platform for one of the top e-commerce companies in the world? If you answered yes to the above, then we want to talk to you.
As the only Principal on the team, your first 6 months would be focused on...
-Our centralized computing system using Kafka., and having people use the system.
-Building out the big platform. How can we make the search better?
-Web analytics and marketing
A day in the life…
- As a Data Engineer, you must be an expert with Big Data technologies like Hadoop, Hive, Hbase, Spark, and various AWS technologies
- You will architect and drive the build out of our next generation data platform
- As a hands-on engineer, you will influence all architecture decisions
- You will build reusable code, with the ability to scale with very large data volumes
- Everything you build will need to scale and perform
- As a senior member on the team you will also mentor junior engineers on the team
You own this if you have…
- 10+ years of Data Engineering experience
- 5+ years of experience in Hadoop MapReduce, Hive, Pig, Spark, and Yarn
- 2+ years of experience in NoSQL technologies like Hbase and Cassandra
- 3+ years of experience in Data Pipelines like Kafka and Airflow
- 2+ years of experience working with either AWS or Azure technologies – EMR, HDInsight
- Experience in architecting, building and maintaining large-scale data infrastructures
- Code/Build/Deployment: GIT, Maven, Docker, Jenkins, Kubernetes, etc.
- A strong team player Ability to quickly triage and troubleshoot complex problems
- Bachelor’s degree in Computer Science