Pyspark Developer

Pyspark Developer
CT, Stamford

Job Description


I came across your profile and I feel you will be a great fit for the below position

Please let me know if you are interested.

Requirements / Responsibilities:

8+ years of experience in Data Modeling, Data architecture, Data Quality, Metadata, ETL and Data Warehouse methodologies and technologies.
3+ years of experience in any combination of the following: SQL, Hadoop, Hive, Linux, APIs, Spark, Python, Scala
2+ years of experience in implementing data warehousing solutions on AWS Cloud using services such as Redshift, S3, Lambda etc.
Experience in writing queries for moving data from HDFS to Hive and analyzing data
Understanding Partitions, Hive Query optimization, Bucketing etc.
Strong understanding of programming paradigms such as distributed architectures and multi-threaded program design
Should be very strong in algorithms, collection framework and build high performance engine to handle large amount of data
Contributor to the overall Data Product roadmap by working closely with our business partners to understand their challenges and develop analytical tools to help drive business decisions
Design & build data pipeline using big data technologies
Leverage prototyping methodologies to propose and design creative business solutions that exploit our broad toolset of technologies (Python, Spark, Redshift, Hive, AWS services)
Educate and inform business partners on architecture, capabilities, best practices and solutions to build out future enhancements
Assist in analyzing business requirements, source systems, understand underlying data sources, transformation requirements, data mapping, data model and metadata for reporting solutions
Self-driven/self-initiator and resourceful to achieve goals independently as well as in teams and promotes an open flow of information so that all stakeholders are well informed.
Ability and desire to work within a cross-functional team environment with people from multiple business units, vendors, countries and cultures.
Ability to work independently under minimum supervision and proactive in solving issues
BS in Computer Science or similar technical degree


 Mittal Panchal

Technical Recruiter

Harvey Nash - The Power of Talent

Office: 646 388 7026

Mobile: 917 562 4741

Fax: 212 481 1319

Apply Now