Custom Software Engineer

Project Role : Custom Software Engineer
Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact.
Must have skills : Data Engineering
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary:

As a Data Engineer, a typical day involves designing, developing, and maintaining scalable data solutions on the Azure platform. The role requires building and optimizing ETL/ELT pipelines, implementing data processing frameworks, and ensuring data availability for business and analytical needs. Collaboration with cross-functional teams, troubleshooting data workflows, and following agile development practices are key aspects of the role to deliver reliable and high-performing data solutions.

Roles & Responsibilities:
- Expected to be an SME, collaborate and manage the team to perform.
- Engage with multiple teams and contribute to key technical decisions.
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, Python, and PySpark.
- Optimize Spark jobs, cluster configurations, and data processing workflows for performance and scalability.
- Develop and support data solutions leveraging Azure Storage Accounts, ADLS Gen2, and Delta Lake.
- Integrate Databricks with orchestration tools such as Apache Airflow for scheduling and monitoring data pipelines.
- Deploy and manage Databricks notebooks, jobs, workflows, and libraries across Development, QA, UAT, and Production environments.
- Troubleshoot data pipeline failures and ensure reliable data processing and delivery.
- Collaborate with stakeholders and development teams to deliver business-driven data solutions.

Professional & Technical Skills:
Must Have Skills:
- Strong hands-on experience in Azure Databricks with Python and PySpark.
- Good understanding of Apache Spark architecture, Spark configurations, performance tuning, and cluster management.
- Experience with Azure Storage Accounts, Azure Data Lake Storage (ADLS Gen2), and Delta Lake.
- Knowledge of authentication and authorization mechanisms, including Service Principals and Managed Identity.
- Experience integrating Databricks with Apache Airflow for job scheduling and monitoring.
- Understanding of Microservices architecture, REST APIs, and API integrations.
- Experience deploying Databricks notebooks, workflows, and libraries across multiple environments.
- Strong knowledge of Databricks Jobs, Workflows, and troubleshooting data pipeline failures.
- Ability to develop scalable ETL/ELT pipelines using PySpark and Python.
- Knowledge of ITSM processes and Production Support activities.

Additional Information:
The candidate should have a minimum of 8 years of experience in Data Engineering.
Experience in Azure Cloud-based Data Engineering solutions is preferred.
This position is based at our Bengaluru office.
A 15 years full-time education is required.

15 years full time education

About Accenture

Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities.

Visit us at www.accenture.com 

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.