Sr. Data Engineer - Analytics

Sr. Data Engineer - Analytics
IL, Oak Brook

Job Description

Sr. Data Engineer

Full-time/Direct-hire

Oakbrook, IL

The senior data engineer is a hands-on role responsible for the design and enhancement of the advanced analytics platform. The senior data engineer will design and implement architectures to support advanced analytics solutions and enrich data in the platform, creating data pipelines for a wide variety of internal and external data types and ensuring the quality and consistency of current and future data flows. The senior data engineer will collaborate with data scientists to wrangle data for the design, development, and deployment of advanced analytics solutions supporting strategic business priorities.

Essential Functions:

  • Collaborate with stakeholders and advanced analytics business partners to understand business needs and to identify and define advanced analytics solutions
  • Apply data management and architecture expertise in developing solution architecture and advanced analytics solution design, including reporting technology
  • Articulate tradeoffs, strengths, and weaknesses of various architectures and their components to determine the best design given use cases and constraints
  • Develop a complete understanding of data flows related to the analytics platform, identifying and analyzing potential issues
  • Design data flows, procedures, and schedules
  • Identify internal and external data sources for potential integration into platform
  • Lead data migration and integration projects
  • Create appropriate standards and processes to ensure quality and consistency of data in the platform
  • Outline best practices and establish standards for data strategy, data lifecycle, data ownership, data definition, and data classification
  • Maintain and advance deep technical skills and knowledge, including of the Hadoop ecosystem, its main components, knowledge of when to use each, and how they can be optimally combined, keeping up-to-date with latest architecture technologies and trends
  • Provide leadership to third-party contractors
  • Protect reputation by keeping information confidential.
  • Maintain professional and technical knowledge by attending educational workshops, professional publications, establishing personal networks, and participating in professional societies.

Minimum Qualifications:

  • Bachelor’s degree in computer science or related field of study
  • 3+ years of hands-on professional experience
  • Expert knowledge of data architectures, data pipelines, real-time data processing, streaming, networking, and security
  • Outstanding understanding of distributed computing, performance optimization, and system architecture
  • Hands-on experience planning and executing the migration and integration of legacy and new data sources to Hadoop-based data lakes
  • Advanced knowledge of big data querying tools
  • Proficient understanding of AI/machine learning
  • Experience with cloud development (e.g. Azure)
  • Experience working with Hive, Spark, and Scala
  • Experience working with Hadoop, Cassandra, or other NoSQL environment
  • Experience with SQL
  • Exceptional problem solving / analytical thinking and skills
  • Teamwork and collaboration
  • Strong communication skills
  • Excellent organizational skills
  • Natural sense of urgency, teamwork, and collaboration reflected in daily work ethic
  • Proficient in Microsoft Office

Preferred Qualifications:

  • Master’s degree in computer science or related field of study
  • Experience with Python
  • Experience working with Agile methodologies

 

Apply Now