Redshift Data Warehouse Engineer
The company's existing data warehouse runs on AWS-Redshift and currently supports the reporting & analytic needs for all departments. Data Engineering provides both operational (e.g. nightly updates, bug/fix) and new development support. Core technologies used are: Redshift (environment, DB), Matillion (ETL) and Python (procedural language).
The 1st release of a new data warehouse will be deployed Q2. It will run on the Google Cloud Platform (GCP) and use Google Service for core functions; i.e. GCS (data lake), Big Query (DB), Cloud Composer/Dataflow (scheduler/ELT/ETL). New releases are planned each quarter.
This position will support the Redshift Ops Manager with 1) day-to-day Redshift Data Warehouse operational support; i.e. ensure nightly updates process as expected & triage/remediate any load issues, bug/fixes; 2) re-engineer selected inefficient processes and some minor new development; and 3) migrate selected data & associated Matillion/Python processes from legacy Redshift to new data warehouse.
Applicant selected for this position will join a team rich in data domain expertise and will use his or her programming, data manipulation and analytical skills to:
- Maintain, create and troubleshoot curated datasets from a variety of Redshift databases, ETL processes and external sources
- Work with business partners from multiple disciplines (customer acquisition, marketing, product, pricing etc.) to understand business processes and ensure data quality
- Maintain and code appropriate data cleaning algorithms to prepare data for further analysis (e.g. identify and deal with missing data, inconsistent formatting and other anomalies)
- Assist Redshift users by answering questions and create new insights based on trends and patterns identified in the data; and to determine migration strategies
- Direct experience supporting data warehouse refresh processes and platform migrations
- Direct experience using cloud-based data repositories (AWS-S3/Redshift experience preferred)
- Strong, hands-on experience coding using SQL, procedural languages (Python preferred), and ETL tools (Matillion preferred)
- Knowledge and experience using software development tools to manipulate, combine, and clean multiple data sources, to include relational & columnar databases, documents (JSON, XML, CSV etc.), and data acquired from systems via SOAP and REST APIs
- Comfortable working in both Linux and Windows computing environments. A strong knowledge of Linux is a plus
- Strong attention to detail, demonstrated ability to document work and a commitment to utilize version control systems and task tracking systems
- Curiosity, love of complex problem solving and an enthusiasm to learn!