Enterprise Software Architect w/Big Data

Enterprise Software Architect w/Big Data
CT, Stamford

Job Description

We are seeking a Enterprise Software Architect with Big Data experience for a 6 month contract position with our client in Stamford, CT. The ideal candidate will have experience with Spark, AWS and Python, and integration using big data.  

 Please send qualified resumes to mary.abraham at harveynashusa.com.

  • Build solutions for ingesting data in real-time from front end apps, transform and push them via an API to a CRM system (sailthru) for real-time and optimized marketing.
  • Build solutions to ingest data from various Publisher APIs like YouTube, Instagram, Facebook but build in algorithms to account for quota limits, spikes in usage and so on. This will likely involve use of multi-threading and should have data quality and maintenance checks built in

Enterprise Software Architect / Solution Architect 

Key Responsibilities

  • Managing backend data ingestion/integration pipelines development lifecycle including architecture, design, development, testing, and deployment.
  • Explore and discover new data sources and quickly familiarize with the available APIs or other data acquisition methods like web-scraping to ingest data
  • Build quick proof of concepts of new data sources to showcase data capabilities and help analytics team identify key metrics and dimensions
  • Design, develop and maintain data ingestion & integration pipelines from various sources which may include contacting primary or third party-data providers to resolve questions, inconsistencies, and/or obtain missing data
  • Design, implement and manage a near real-time ingestion & integration pipelines
  • Analyze data to identify outliers, missing, incomplete, and/or invalid data; Ensure accuracy of all data from source to final deliverable by creating automated quality checks
  • Evangelize an extremely high standard of code quality, system reliability, and performance. 


  • Bachelor’s degree in Computer Science or Related Discipline
  • Minimum 10+ years of experience in building enterprise level software solutions
  • Minimum 4+ years of experience in architecting cloud-based software solutions
  • Minimum 4+ years of experience in APIs based development using Python, Java
  • Experience in architecting & building the secured, reliable and high-performance data pipeline using Python, Spark on AWS cloud
  • Experience in Python libraries such as Pandas and NumPy, SciPy, Flask, SQLAlchemy and/or Automation is a plus.
  • Experience in architecting solutions at scale to empower the business and support a wide variety of use cases, from experimental work to mission-critical production operations.
  • Experience in real-time data processing using Python, Spark and Spark-Streaming
  • Experience in ingesting and processing Social Media platforms data such as Facebook, Twitter, Instagram, Snapchat, Clickstream
  • Experience working with both Structured and Unstructured data including complex JSONs
  • Experience in AWS Kinesis Stream Processing, EMR, Redshift, S3, Lambda
  • Experience in the database systems such as AWS Redshift, BigQuery, SQL Server or Oracle
  • Experience with multi-threading and asynchronous event-driven programming
  • Experience with high volume, high availability distributed systems
  • Experience in coming up with the viable solutions to tough engineering problems
  • Knowledge of code versioning tools {{such as Git, Mercurial or SVN}}
  • Familiarity with the Sailthru APIs is a plus


Apply Now