Hiring.Camp

Data Engineer - Azure & Databricks Expert

Employees Systra

·

Jul 22, 2024

Location
Montréal, QC, CA
Type
Full-time
Department
Engineering
Experience
5+ years
Closing date
Today
Source
iCIMS

Description

SYSTRA Canada is part of the SYSTRA group, an international consulting and engineering group, a world leader in the design of transport infrastructures. SYSTRA Canada is an engineering and consulting firm whose primary focus is to offer transportation solutions, whether for passengers or goods: feasibility studies of building a new railway line, increasing the capacity of the existing infrastructure, privatization of a railway.

Context

As part of our data-driven strategy, we are looking for an experienced Data Engineer to design, develop, and optimize data pipelines on our Azure platform. You will play a key role in the ingestion, processing, transformation, and delivery of large-scale data through Azure Data Lake, Azure Data Factory, Azure Synapse, and especially Databricks.

Missions/Main Duties

• Design and develop data ingestion pipelines (batch and real-time) from various sources (API, SFTP, databases, etc.)• Set up and orchestrate data flows using Azure Data Factory or Azure Synapse Pipelines• Develop complex data processing workflows with Apache Spark through Databricks (PySpark or Scala)• Ensure data quality, traceability, and governance, particularly through Delta Lake• Optimize the performance of pipelines and Spark processing jobs• Collaborate with Data Science, BI, and Architecture teams to ensure platform consistency and scalability• Participate in the industrialization, documentation, and security of data flows• Contribute to the evolution of the data architecture and the implementation of DevOps/DataOps best practices (CI/CD, monitoring, testing, etc.)

Profile/Skills

Required Technical Skills:• Expertise in Microsoft Azure: Data Lake Storage Gen2, Data Factory, Synapse, Key Vault, etc.• Strong command of Databricks (PySpark / Spark SQL / Delta Lake)• Solid experience in developing data pipelines• Strong data modeling skills (structured and semi-structured data)• Proficiency with Avro, Parquet, and JSON formats• Knowledge of data security, quality, and governance principles• Experience with Git, CI/CD (Azure DevOps or GitHub Actions), and pipeline testing

 

Preferred Skills:• Scala, advanced SQL• Knowledge of event-driven architectures (Event Hub, Kafka)• Understanding of MLOps or experience collaborating with Data Scientists• Microsoft (DP-203) or Databricks certification is a plus

 

Required Experience:• At least 5 years of experience as a Data Engineer• Hands-on experience with large-scale data projects in an Azure & Databricks environment

 

Personal Qualities:• Analytical mindset, rigor, and autonomy• Ability to work in an agile and collaborative environment• Strong technical communication skills

SYSTRA Canada is committed to having a diverse and inclusive workforce. We are an Equal Opportunity Employer that values applications from all and offers employment equity based on the merits and abilities of candidates and personnel.

 

We use AI-enabled tools in our recruitment platform (iCIMS) for tasks like resume parsing and candidate matching. These tools do not make hiring decisions; all applications are reviewed by a human recruiter, and final decisions are made by our hiring team.For questions about our hiring process or AI tools, please contact [email protected].

 

Note: In this description, the masculine gender is used for easy reading purposes and includes the feminine.

Skills

ScalaAzureCI/CDSQLSparkDatabricksData ScienceGitGitHubDevOps

Similar Jobs

30

Data Engineer

Io Tech Solutions Limited · Hong Kong

Today

Data Engineer

Waste Management · Wellington, Wellington Region, New Zealand · Hybrid

Today

Data Engineer

Waste Management · Auckland, Auckland, New Zealand · Hybrid

Yesterday

Data Engineer

Nasstar · Remote, UK · Remote

Yesterday

Data Engineer

Iliad · Sede Milano - HQ, Italy

Yesterday

Data Engineer

Slihrms · GB.Bristol.The Hub, United Kingdom +1 · Remote, Onsite

Yesterday

Data Engineer

Theapexgroup · Makati City, Ayala Avenue, Philippines

Yesterday

Data Engineer

Accenture · Hyderabad, HDC3B, India

Yesterday

Data Engineer

Accenture · Pune, PDC5A, India

Yesterday

Data Engineer

Accenture · Pune, PDC5A, India

Yesterday

Data Engineer

Accenture · Bengaluru, BDC14A, India

Yesterday

Data Engineer

Accenture · Hyderabad, HDC3B, India

Yesterday

Data Engineer

Accenture · Chennai, CDC2A, India

Yesterday

Data Engineer

Accenture · Chennai, CDC2E, India

Yesterday

Data Engineer

Accenture · Bengaluru, BDC7B, India

Yesterday

Data Engineer

Accenture · Navi Mumbai, MDC5C, India

Yesterday

Data Engineer

Accenture · Mumbai, MDC2B, India

Yesterday

Data Engineer

Mayo Clinic · Rochester, MN, United States, US

Yesterday

Data Engineer

Vardaspace · El Segundo, California, United States · Onsite

Yesterday

Data Engineer

Beghouconsulting · Durham +7 · Hybrid

Yesterday

Data Engineer

EXL Talent Acquisition Team · Gurugram, Haryana, India · Hybrid

Yesterday

Data Engineer

EXL · Gurugram, Haryana, India · Hybrid

Yesterday

Data Engineer

WISE · London, United Kingdom

Yesterday

Data Engineer

Tgguk · Liverpool, UK

Yesterday

Data Engineer

Safran Cabin · Santa Maria, California, US

2 days ago

Data Engineer

Steampunk Inc. · McLean, VA, US

2 days ago

Data Engineer

Sandvik · Pune, India

2 days ago

Data Engineer

Thomson Reuters · India, Bengaluru, Karnataka +1 · Hybrid

2 days ago

Data Engineer

Sandvik employee without Access · Pune, India

2 days ago

Data Engineer

Analytica · Washington, DC · Remote

2 days ago
Data Engineer - Azure & Databricks Expert at Employees Systra | Hiring.Camp