Offer summary

Qualifications:

Proficiency in programming languages for data processing such as Python, Scala, or Java., Strong experience with big data technologies like Hadoop and Spark, as well as ETL tools., Familiarity with data storage systems including SQL and NoSQL databases, and data lakes., Experience with cloud platforms and data services like AWS Redshift and Google BigQuery..

Key responsabilities:

Design, develop, and maintain robust data pipelines for machine learning workflows and GenAI applications.

Implement data ingestion, transformation, and storage solutions for both structured and unstructured data.

Ensure data quality, integrity, and consistency across the entire data pipeline.

Collaborate with ML engineers and data scientists for seamless integration of data pipelines with models and applications.

Job description

Octopus by RTG is on a mission of connecting top notch ogranizations around the globe with top notch talents. We are currently looking for a Senior Data Engineer.

Responsibilities:

Design, develop, and maintain robust data pipelines to support machine learning workflows and GenAI applications.
Implement data ingestion, transformation, and storage solutions for structured and unstructured data.
Ensure data quality, integrity, and consistency across the entire pipeline.
Optimize data infrastructure for scalability, performance, and cost-efficiency.
Implement real-time data processing workflows
Collaborate with ML engineers and data scientists to ensure seamless integration of data pipelines with models and applications.

Requirements

Proficiency in programming languages for data processing (e.g., Python, Scala, Java).
Strong experience with big data technologies (e.g., Hadoop, Spark) and ETL tools.
Familiarity with data storage systems (e.g., SQL databases, NoSQL databases, data lakes).
Strong Experience with vector databases and embedding stores
Experience with cloud platforms and data services (e.g., AWS Redshift, Google BigQuery, Azure Data Factory).
Knowledge of data modeling, warehousing, and real-time processing frameworks (e.g., Kafka, Flink).
Strong problem-solving skills and ability to work in cross-functional teams.

Required profile