Data Engineer, Data Intelligence Engineering
We are seeking a Data Engineer for a full-time permanent position with our client in Yonkers, NY. The ideal candidate will have experience with B2B along with Java/Spring, Apache Spark, SQL/NoSQL/Hadoop and Linux. The client will transfer h1b visas. Must be able to attend at least 1 onsite interview.
The data engineer will help build a business intelligence data system from ground up. The engineer will design and establish the analytics database. The engineer will analyze various internal and external data sources and develop the data pipelines and ETLs to feed the databases systematically using modern Data-Engineering technologies.
- Design , Deploy, performance tune and monitor analytical databases
- Design, Build, and Launch new data models , data pipelines and ETLs that ingest, map, aggregate, and broadcast data feeds to internal and external groups
- Design, develop, and deploy various data visualizations
- Analyze business requirements, provide development estimates and feedback, and determine implementation/execution approaches.
- Work with other software engineers, database developers, infrastructure, operations and QA teams to design and implement quality software solutions
- Support, troubleshoot, monitor and optimize existing data engineering systems
- Help streamline software development by contributing to the implementation of a continuous integration process, automating manual processes, and eliminating defects
- Performs other duties as assigned but not at a higher level
- Experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large and complex datasets.
- Strong knowledge of DBMS concepts with hands-on experience architecting and deploying Nosql Databases (MongoDB), and working knowledge of Relational Databases (Oracle)
- Strong scripting experience using Python/Bash in Linux/UNIX environment to process and analyze large data sets
- Knowledge and Hands-on experience with object-oriented Programming languages and frameworks - Java/JEE, Spring, in development for modern data engineering systems
- Strong Knowledge and Hands-on experience in building ETL systems and data-pipelines.
- Experience with technologies such as Apache Spark, Kafka, Flume, Sqoop is a big plus
- Experience with data visualizations software (Tableau/Qlikview)
- Experience with AWS technologies - EC2, S3, EMR, etc.
- Experience working in projects involving building Data Warehouse, Data Lakes, Big data ecosystem (Hadoop, HDFS, Hive, and Hbase ) and Data Science is a plus
- Experience working with modern tools in the Agile software development life cycle - Version Control Systems (Ex. git, github, Stash/BitBucket), Knowledge Management (Ex. Confluence, Google Docs), Development Workflow (Ex. Jira), Continuous Integration (Ex. Bamboo, Jenkins), Real Time Collaboration (Ex. Hipchat, Slack)