Data Engineer at Sourcefit

Position Summary:

We are seeking a highly skilled and experienced Data Engineer to join our team. The Data Engineer will be responsible for designing, developing, and maintaining scalable data pipelines and systems. This role is crucial for assembling complex data sets, improving internal processes through automation, and optimizing data infrastructure for scalability and efficiency. The ideal candidate will work closely with data scientists, analysts, and other stakeholders to support data analytics, business intelligence, and AI initiatives, ensuring high data quality and integrity throughout the organization.

Job Details:

• Monday to Friday | Night Shift
• Work from home

Responsibilities:

• Design, develop, and maintain scalable data pipelines and systems.
• Assemble large, complex data sets that meet functional and non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Build the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from various data sources using SQL and Azure technologies.
• Develop and maintain data architecture and systems to support data analytics, business intelligence, and AI initiatives.
• Collaborate with data scientists, analysts, and other stakeholders to assist with data-related technical issues and support their data infrastructure needs.
• Ensure data quality and integrity by performing data cleaning, validation, and analysis.
• Monitor and troubleshoot data pipeline performance and reliability.

Requirements:

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Proven experience as a Data Engineer or in a similar role.
• Strong knowledge of SQL and experience with relational databases (e.g., MySQL, PostgreSQL).
• Experience with big data tools and frameworks (e.g., Hadoop, Spark, Kafka).
• Proficiency in programming languages such as Python, Java, or Scala.
• Extensive experience with Azure cloud services, including Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
• Familiarity with data warehousing solutions, particularly Microsoft Fabric.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.
• Experience in data analytics and AI, including data visualization, statistical analysis, predictive modeling, and machine learning.