Senior Data Engineer

Remote: 
Full Remote
Contract: 
Work from: 

Offer summary

Qualifications:

5+ years of Data Engineering experience, 3+ years of hands-on experience with large datasets using Spark and Delta Lake, Proficiency in Python, Scala, and SQL for data pipeline development, Experience with AWS services and data visualization tools is preferred..

Key responsabilities:

  • Design and develop robust data pipelines for healthcare data
  • Mentor early career developers and collaborate with cross-functional teams
  • Implement and manage data flows integrating information from various sources
  • Optimize datasets and models for analytics and data science needs.

Datavant logo
Datavant Computer Software / SaaS Startup https://www.datavant.com/
5001 - 10000 Employees
See all jobs

Job description

Datavant is a data platform company and the world’s leader in health data exchange. Our vision is that every healthcare decision is powered by the right data, at the right time, in the right format.

Our platform is powered by the largest, most diverse health data network in the U.S., enabling data to be secure, accessible and usable to inform better health decisions. Datavant is trusted by the world’s leading life sciences companies, government agencies, and those who deliver and pay for care. 

By joining Datavant today, you’re stepping onto a high-performing, values-driven team. Together, we’re rising to the challenge of tackling some of healthcare’s most complex problems with technology-forward solutions. Datavanters bring a diversity of professional, educational and life experiences to realize our bold vision for healthcare.

What We’re Looking For

We are seeking a talented and experienced Lead Data Engineer specializing in healthcare data pipelines to join our team. The ideal candidate will have expertise in designing, developing, and maintaining robust data pipelines for healthcare data including prescriptions, medical claims, MAO4, patient demographics, and FHIR data. This role will primarily focus on utilizing Databricks with Spark, Python, Scala, and SQL for building and optimizing data pipelines, as well as performing data engineering for machine learning and AI output. While healthcare data knowledge is preferred, a strong foundation in data engineering and a willingness to learn about healthcare data is essential. Close collaboration with software engineers, data integration engineers, product managers, and stakeholders as well as mentorship for other data engineers will be key to success in this role.

What You Will Do

  • ​​Deliver a world-class data platform from the ground up
  • Plan and delegate complex projects with broad scope
  • Mentor and grow early career developers or engineers, including offshore engineers
  • Designing and developing data engineering solutions.
  • Support analytics, data science and data platform teams and understand their unique needs and challenges.
  • Design, implement and manage data flows that integrate information from various sources into a common pool implementing data pipelines based on ELT and ETL models.
  • Develop datasets and models to meet the requirements of analysts, data scientists and key stakeholders; optimizing query compute and data storage patterns for cost and performance
  • Work closely with the other engineering teams to troubleshoot issues with API’s and data exchanges between multiple systems.

What You Need to Succeed

  • 5+ years Data Engineering experience
  • 3+ years hands-on experience working with large, structured/unstructured datasets using partitioned cloud storage architecture using query engines such as Spark, Delta Lake.
  • 3+ years experience designing, developing, deploying and testing in Databricks. 
  • 3+ years of hands-on experience in Python/Pyspark/SparkSQL.
  • 3+ years of experience with Scala programming language
  • 2+ years experience on Big data pipelines/DAG tools like Airflow and dbt
  • 2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data.
  • Experience in the AWS computing environment and storage services such as s3/glacier is required.
  • Experience with conceptual, logical and/or physical database designs is required.
  • Good knowledge in Linux and shell scripting is highly desired.
  • Experience with Data Visualization tools like Looker, Tableau is desired.
  • Strong communication skills to relay complex data integration requirements to team members.

What Helps You Stand Out

  • Experience with healthcare data is a plus, but not required; willingness to learn about healthcare data and domain knowledge is essential.
  • Experience with FHIR data
  • Experience working in an Agile development environment is preferred.
  • Familiarity with machine learning concepts and tools is a plus.

We are committed to building a diverse team of Datavanters who are all responsible for stewarding a high-performance culture in which all Datavanters belong and thrive. We are proud to be an Equal Employment Opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, disability, veteran status, or other legally protected status.

At Datavant our total rewards strategy powers a high-growth, high-performance, health technology company that rewards our employees for transforming health care through creating industry-defining data logistics products and services.

The range posted is for a given job title, which can include multiple levels. Individual rates for the same job title may differ based on their level, responsibilities, skills, and experience for a specific job.

The estimated total cash compensation range for this role is:
$160,000$200,000 USD

To ensure the safety of patients and staff, many of our clients require post-offer health screenings and proof and/or completion of various vaccinations such as the flu shot, Tdap, COVID-19, etc. Any requests to be exempted from these requirements will be reviewed by Datavant Human Resources and determined on a case-by-case basis. Depending on the state in which you will be working, exemptions may be available on the basis of disability, medical contraindications to the vaccine or any of its components, pregnancy or pregnancy-related medical conditions, and/or religion.

This job is not eligible for employment sponsorship.

Datavant is committed to a work environment free from job discrimination. We are proud to be an Equal Employment Opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, disability, veteran status, or other legally protected status. To learn more about our commitment, please review our EEO Commitment Statement here. Know Your Rights, explore the resources available through the EEOC for more information regarding your legal rights and protections. In addition, Datavant does not and will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay. 

At the end of this application, you will find a set of voluntary demographic questions. If you choose to respond, your answers will be anonymous and will help us identify areas for improvement in our recruitment process. (We can only see aggregate responses, not individual ones. In fact, we aren’t even able to see whether you’ve responded.) Responding is entirely optional and will not affect your application or hiring process in any way.

Datavant is committed to working with and providing reasonable accommodations to individuals with physical and mental disabilities. If you need an accommodation while seeking employment, please contact us at peopleteam@datavant.com. We will review your request for reasonable accommodation on a case-by-case basis.

For more information about how we collect and use your data, please review our Privacy Policy.

 

Required profile

Experience

Industry :
Computer Software / SaaS
Spoken language(s):
English
Check out the description to know which languages are mandatory.

Other Skills

  • Mentorship
  • Collaboration
  • Communication

Data Engineer Related jobs