Big Data Software Development Manager
Role: Big Data Software Development Manager
Location: Stamford, CT
My client is a global entertainment company building out a data integration platform within their Data Engineering team to elevate their relationships with consumers and further optimize their business. It's a really exciting time to join them!
The Big Data Development Manager role is responsible for building the data integration platform -- managing a team of developers, production support engineers and operations for big data and legacy data management platforms. It will partner with the Data Analytics team.
Software Development Manager, Big Data
The Software development manager is responsible for building and orchestration of our client's data integration platform. Reporting to the head of Data Engineering, he/she will manage a team of developers, production support engineers and operations for our clients big data and legacy data management platforms This role will be key to the rollout of our client's Data platform and will partner with Data Analytics and other technology teams. Our environment is dynamic, fast-paced, and lots of fun.
- Design, develop and maintain our client's Consumer marketing, products, content and subscription data marts
- Drive test driven development approach across the data environments
- Develop using a CI/CD framework setting up best practices
- Manage the flow of data from various collection sources through final delivery of product
- Responsible for creating and maintaining inbound and outbound Data Delivery SLA’s
- Implement monitoring tools, measure metrics and improve performance for our client's data assets.
- Govern and maintain security, backup and recovery, and performance
- Manage Data Operations across legacy Enterprise Data Warehouse and Data management platform.
- Conduct POC’s for Data Integration tools, continuously assessing, automating and evolving data Integration roadmap
- Continuously re-assess the cloud compute capabilities to optimize environment, infrastructure and get the best value and performance out of database assets.
- Provide Support 24/7 support for operations
- 8 + years experience in Data Warehousing development and transformation
- 3 + year of technology management experience of onshore and offshore development team
- 2+ years of experience with Hadoop Ecosystem including Spark, Storm, HDFS, Hive, HBase and other NoSQL databases
- Experience in developing Spark Streaming applications analyzing the data through Spark (conducted ETL processes and connected to different SQL and Redshift databases)
- Experience in writing queries for moving data from HDFS to Hive and analyzing data
- Understanding Partitions, Hive Query optimization, Bucketing etc.
- Experience in Sqoop for moving data between RDBMS and HDFS
- Strong understanding of programming paradigms such as distributed architectures and multi-threaded program design
- Strong in algorithms, collection framework and the ability to build high performance engine to handle large amounts of data
- AWS Dev ops and Sys operations certification is a plus
- BS in Computer Science or similar technical degree
- Media industry experience is a plus