Data Engineer

Summary of Role

The Data Engineer is responsible for designing, developing, and maintaining scalable data solutions that transform raw data into trusted, accessible, and actionable information. This role focuses on building and optimizing data pipelines, data architectures, and data platforms that support analytics, business intelligence, machine learning, and operational reporting requirements.

The successful candidate will work closely with Data Scientists, Analysts, Architects, and Business Stakeholders to design and implement modern data engineering solutions that enable data-driven decision-making across the organization.

Responsibilities

Data Engineering & Data Management

• Design, develop, and maintain scalable data pipelines and architectures to support business and analytical requirements.

• Build and optimize databases, data warehouses, and data lakes for performance, reliability, and scalability.

• Extract, transform, and load (ETL/ELT) data from a variety of structured and unstructured sources.

• Develop and implement automated processes for data acquisition, integration, cleansing, transformation, and storage.

• Define and implement data storage solutions based on business and technical requirements.

• Create and maintain physical and logical data models to support enterprise data initiatives.

• Support data migrations across multiple databases, servers, and cloud platforms.

• Ensure data quality, consistency, integrity, and governance across all data assets.

Analytics & Machine Learning Support

• Collaborate with Data Scientists and Analytics teams to operationalize machine learning models.
• Develop data pipelines that support predictive analytics, artificial intelligence, and machine learning workloads.
• Deploy machine learning models into production environments.
• Build robust testing frameworks to validate data transformations and model outputs.

Data Architecture & Solutions Development

• Design and implement secure, scalable, and highly available data platforms.
• Evaluate and recommend data solutions, technologies, and tools to address business requirements.
• Develop databases optimized for analytics, reporting, and advanced data processing.
• Build and manage cloud-based data solutions utilizing modern cloud technologies.

Quality Assurance & Change Management

• Perform root-cause analysis and troubleshooting for data-related issues.
• Design and execute testing scenarios to validate data quality and transformation accuracy.
• Assess, document, and implement changes in accordance with change management processes.
• Ensure release processes, procedures, and documentation are maintained and updated.
• Support configuration management and release management activities.

Stakeholder Engagement

• Collaborate with business, analytics, and technical teams to improve data accessibility and usability.
• Translate business requirements into technical data solutions.
• Provide recommendations regarding data architecture, integration, and optimization strategies.
• Communicate complex technical concepts effectively to both technical and non-technical stakeholders.

Skills

• Strong programming skills in Python

• Advanced SQL development skills, including:

• SQL Server

• MySQL

• Relational database technologies

• ETL/ELT processes

• Data pipelines

• Data warehouses

• Data lakes

• Experience working with large-scale structured and unstructured datasets.

• Knowledge of data governance, data quality, and metadata management practices.

• Experience building and maintaining highly available and scalable systems.

• Understanding of cloud platforms such as:

• Microsoft Azure

• Amazon Web Services (AWS)

• Google Cloud Platform (GCP)

• Hadoop

• Cassandra

• Storm

• Similar distributed processing frameworks

• Python

• Bash

• Shell Scripting

• Perl

Analytics & Reporting

• Working knowledge of Power BI for dashboards, reports, and analytical solutions.
• Understanding of:
• Data visualization
• Data virtualization
• Augmented analytics
• Business intelligence solutions

Qualifications

• Matric and a Bachelor's Degree in Computer Science, Information Technology

• Certified Data Engineer certification.

• Cloud certifications (Azure, AWS, Google Cloud).

• Professional data and analytics certifications.

Experience

• Proven experience in a Data Engineering role within a fast-paced technology environment.

• Experience developing modern data and analytics platforms that deliver actionable insights from large and complex datasets.

• Strong hands-on experience with Python development.

• Experience designing and supporting ETL/ELT frameworks and data integration solutions.

• Experience working with SQL Server, MySQL, and enterprise database solutions.

• Experience with cloud technologies, including SaaS, PaaS, and IaaS environments.

• Experience with automation, scripting, and data process orchestration.

• Experience building secure, scalable, resilient, and highly available data platforms.

• Experience supporting machine learning and advanced analytics initiatives.

• Experience with big data technologies and distributed computing environments.