Offer summary

Qualifications:

Proven experience in configuring Azure Databricks Workspaces., Strong experience with Azure DevOps, including GIT and Pipelines., Solid understanding of data engineering principles and practices., Excellent problem-solving and communication skills..

Key responsabilities:

Collaborate with cross-functional teams to design and manage Azure Databricks environments.

Deploy and configure Azure Databricks workspaces to meet project requirements.

Create and manage Databricks Asset Bundles (DAB) for CI/CD deployment processes.

Monitor and optimize Databricks performance for efficient resource utilization.

Job description

As an Azure Databricks Engineer, you will play a pivotal role in our technology solutions team, focusing on the deployment and management of Azure Databricks environments. This position requires a deep understanding of cloud-based data engineering and analytics, particularly within the Azure ecosystem. You will be responsible for implementing best practices in CI/CD deployment using Databricks Asset Bundles (DAB) and Azure DevOps, ensuring that our data solutions are robust, scalable, and efficient. Your expertise will help drive our clients' digital transformation initiatives by leveraging the power of data analytics and machine learning.

Job Responsibilities

Collaborate with cross-functional teams to design, implement, and manage Azure Databricks environments.
Deploy Azure Databricks and configure workspaces to meet project requirements.
Create and manage Databricks Asset Bundles (DAB) to facilitate CI/CD deployment processes.
Implement CI/CD pipelines using Azure DevOps, ensuring seamless integration and delivery of data solutions.
Manage Unity Catalog changes and workspace configurations to enhance data governance and accessibility.
Utilize Terraform for infrastructure as code (IaC) to automate the provisioning of Databricks resources.
Monitor and optimize Databricks performance, ensuring efficient resource utilization and cost management.
Provide technical guidance and support to team members and stakeholders regarding best practices in data engineering and analytics.
Stay updated with the latest trends and advancements in Azure Databricks and related technologies to continuously improve our solutions.

Requirements

Proven experience in configuring Azure Databricks Workspaces.
Expertise in creating Databricks Asset Bundles (DAB) for CI/CD deployment.
Strong experience with Azure DevOps, including GIT and Pipelines.
Demonstrated experience as an Azure Databricks Workspace Administrator.
Proficient in managing Databricks Unity Catalog for data governance.
Familiarity with Terraform for infrastructure automation is a plus.
Solid understanding of data engineering principles and practices.
Excellent problem-solving skills and the ability to work collaboratively in a team environment.
Strong communication skills to effectively convey technical concepts to non-technical stakeholders.

Required profile