Hiring.Camp

Big Data Engineer- Scala

Capco

·

2 weeks ago

Location
India - Pune · India
Department
Data & Analytics
Source
Greenhouse

Description

 

Job Title: Big Data Engineer - SCala

About Us

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the  British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. 

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

 

 

Job Description: Big Data Engineer – Scala

We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies.

Key Responsibilities

  • Design, develop, and maintain scalable Big Data applications and data pipelines.
  • Develop high-performance data-processing solutions using Scala and Apache Spark.
  • Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data.
  • Work with technologies such as Hadoop, HDFS, Hive, Kafka, and Spark SQL.
  • Perform Spark job performance tuning and optimization.
  • Develop reusable and maintainable data-processing frameworks.
  • Troubleshoot production issues related to data pipelines and distributed applications.
  • Implement data-quality checks, monitoring, and error-handling mechanisms.
  • Collaborate with data architects, analysts, business teams, and other engineering teams.
  • Follow software engineering best practices, including Git, code reviews, testing, and CI/CD.

Required Skills

  • Strong hands-on programming experience in Scala.
  • Strong experience with Apache Spark / Spark SQL.
  • Good understanding of Big Data and distributed computing concepts.
  • Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive.
  • Experience building large-scale ETL/data engineering pipelines.
  • Strong SQL skills.
  • Experience working with Kafka or similar messaging/streaming technologies.
  • Understanding of data formats such as Parquet, Avro, and JSON.
  • Experience with performance optimization and troubleshooting of Spark applications.
  • Good understanding of software development lifecycle and version-control systems.

Good to Have

  • Experience with AWS, Azure, or GCP.
  • Knowledge of Databricks.
  • Experience with Airflow or similar orchestration tools.
  • Exposure to Delta Lake or other modern lakehouse technologies.
  • Knowledge of Docker, Kubernetes, and CI/CD pipelines.
  • Experience working in Agile/Scrum environments.

Core Skills

Scala | Apache Spark | Spark SQL | SQL | Hadoop | HDFS | Hive | Kafka | ETL | Data Engineering | Big Data

 

Skills

ScalaAWSAzureGCPDockerKubernetesCI/CDSQLSparkHadoopAirflowDatabricksData EngineeringETLGitAgileScrum

Similar Jobs

30

Big Data Engineer

Bristlecone·Mumbai, Maharashtra

4d ago

Big Data Engineer

PROSTAFF Schweiz GmbH·Zürich, Zurich

1w ago

Big Data Engineer

Odaseva·Paris, Île-de-France·Hybrid

1mo ago

Big Data Engineer

Placerlabs·Ramat Gan, Tel Aviv

Big Data Engineer

EXL Talent Acquisition Team·Pune, Maharashtra·Hybrid

4mo ago

Big Data Engineer

EXL·Pune, Maharashtra·Hybrid

4mo ago

Big Data Engineer

Clear Fracture·New York City, New York·Remote

6mo ago

Big Data Engineer 

Inetum·Lisbon, Portugal

1y+ ago

Big Data Engineer

Kmccorp·Hyderabad, Telangana·Remote

1y+ ago

Big Data Engineer

Sequoiaconnect·Remote

1y+ ago

Big Data Engineer

Ocelot Consulting·Creve Coeur, Missouri

1y+ ago

Big Data Engineer

ByteCubed·Arlington, Virginia

1y+ ago

Big Data Engineer

Topcoder taas·New York City, New York

Big Data Engineer

Bluehilldev·Remote

Senior Lead Software Engineer - Big Data

JP Morgan Chase·Bengaluru, Karnataka

Today

Senior Lead Software Engineer - Big Data

JP Morgan Chase·Bengaluru, Karnataka

Today

Big Data Engineer (AVP)

Citi Bank·5900 HURONTARIO STREET MISSISSAUGA, Canada·Hybrid

4d ago

Software Engineer, Big Data Infrastructure

Benchling·San Francisco, California·Hybrid

4d ago

Big Data Engineer (AVP)

Citi Bank·Mississauga, Ontario

4d ago

Cloud Support Engineer - Big Data, AWS Support Engineering

Amazon Company

Software Engineer, Big Data (829)

Protegrity·Navi Mumbai, Maharashtra·Hybrid

1w ago

Data Engineer - Big Data – Scala, Spark, Azure Confirmé H/F

Inetum·Biot, Provence-Alpes-Côte d'Azur·Onsite

1w ago

Lead Site Reliability Engineer (Big Data)

Mastercard·O'Fallon, Missouri

1w ago

Software Engineer III - AWS BIG Data Engineer

JP Morgan Chase·Bengaluru, Karnataka

2w ago

Software Engineer III - AWS BIG Data Engineer

JP Morgan Chase·Bengaluru, Karnataka

2w ago

Senior Staff Engineer, Big Data Infrastructure

Stripe·New York City, New York

Software Engineer, Big Data, Tech Enablement

Ziprecruiter·Santa Monica, California·Remote

2w ago

Software Engineer, Big Data, Tech Enablement

Ziprecruiter·Santa Monica, California·Remote

2w ago

Senior Software Engineer, Big Data, Tech Enablement

Ziprecruiter·Santa Monica, California·Remote

2w ago

Senior Software Engineer, Big Data, Tech Enablement

Ziprecruiter·Santa Monica, California·Remote

2w ago