Senior Data Engineer

Description

As a Senior/Tech Lead Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data platforms for our clients. Your primary responsibilities will revolve around data modeling, ETL development, and platform optimization, leveraging technologies such as EMR/Glue, Air Flow, Spark, using Python and various cloud-based solutions.

Key Responsibilities:

• Design, develop, and maintain ETL pipelines for ingesting and transforming data from diverse sources.
• Collaborate with cross-functional teams to ensure seamless deployment and integration of data solutions.
• Lead efforts in performance tuning and query optimization to enhance data processing efficiency.
• Provide expertise in data modeling and database design to ensure scalability and reliability of data platforms.
• Contribute to the development of best practices and standards for data engineering processes.
• Stay updated on emerging technologies and trends in the data engineering landscape.

Requirements

• Bachelor's Degree in Computer Science or related field.
• Minimum of 5 years of experience in tech lead data engineering or architecture roles.
• Proficiency in Python and PySpark for ETL development and data processing.
• AWS CLOUD at least 2 years
• Extensive experience with cloud-based data platforms, particularly EMR.
• Must have knowledge with Spark.
• Excellent problem-solving skills and ability to work effectively in a collaborative team environment.
• Leadership experience, with a proven track record of leading data engineering teams.
• The first point - it would be great if the candidate would confirm, that he/she is:
• Practical experience with performance optimization of the Spark job (MUST)
• Knowing what is SOLID and knowledge of OOD (MUST)