- Location
- Ottawa, Ontario, Canada · published
- Type
- Contract
- Department
- Engineering
- Seniority
- Senior
- Source
- Workable
Description
49 Solutions is proactively building a team of experienced, security-cleared professionals in preparation for anticipated Databricks initiatives within the Government of Canada. We are connecting with qualified candidates in advance so that we have a strong team ready to respond as relevant opportunities arise.
Please note that these opportunities are currently prospective and are not associated with a guaranteed placement or awarded contract. Candidates who join our network will be considered for relevant opportunities as they become available.
49 Solutions is seeking Senior Databricks Data Engineers to design, develop and operationalize secure, scalable data pipelines and data products for Government of Canada environments using Databricks, Lakeflow, Delta Lake, Spark, Python and SQL.
Responsibilities
• Design, build and maintain enterprise data pipelines using Databricks.
• Build ingestion and transformation processes using Lakeflow, Lakeflow Connect, Jobs and Auto Loader.
• Develop batch and streaming solutions with Apache Spark and PySpark.
• Implement raw, cleaned and curated layers using Lakehouse and medallion patterns.
• Develop and optimize Delta Lake tables and enterprise data products.
• Connect Databricks with cloud, enterprise and on-premises sources.
• Modernize legacy ETL/ELT pipelines and data warehouse workloads.
• Implement data quality, validation, reconciliation, monitoring and Unity Catalog controls.
• Develop reusable engineering patterns and participate in CI/CD and automated deployments.
• Support testing, migration, production deployment, runbooks and client knowledge transfer.