Hiring.Camp

Big Data and Cloud Data Engineer - Manager

Statestreet

·

4 days ago

Location
Hyderabad, India
Type
Full-time
Department
Engineering
Seniority
Manager
Experience
10+ years
Closing date
Today
Source
Workday

Description

Role Overview:

A Data Engineer in the cloud is primarily responsible for designing, building, and managing scalable data infrastructure and pipelines. They work to ensure that data is accessible, reliable, and processed efficiently for various business needs. This role involves a mix of software engineering, data warehousing, and big data technologies.

The key responsibilities include developing, constructing, testing, and maintaining data architectures (such as databases and large-scale processing systems). They are also responsible for aligning these architectures with business requirements, ensuring data quality and governance, and implementing data security measures.

In addition, Data Engineers collaborate closely with data scientists and analysts to understand their needs and provide them with the necessary data for analysis

Why this role is important to us:

The team you will be joining is a part of Compliance IT ,Compliance IT ensures that our operations adhere to the necessary regulations and standards, safeguarding our business from potential risks and liabilities. This role is not only about maintaining compliance but also about driving our mission forward by ensuring that our systems and processes are robust, secure, and efficient.

Your contributions will directly impact our company's ability to operate within legal parameters while also fostering a culture of integrity and accountability. We deeply value the expertise and dedication you'll bring to the table, and we are excited to see the positive influence you'll have on our team.

What you will be responsible for

As a Data Engineer , The key responsibilities include developing, constructing, testing, and maintaining data architectures (such as databases and large-scale processing systems). They are also responsible for aligning these architectures with business requirements, ensuring data quality and governance, and implementing data security measures.

These skills will help you succeed in this role:

  • Bachelor’s degree in computer science or engineering

  • Minimum 10+ years of hands-on experience with data focused application development such as ETL, Data engineering.

  • Experience with advanced Bigdata technologies like Hadoop, Spark, Hive, SQL and NoSQL.

  • Strong in programming languages with 3+ years of Hands-on experience: Scala and Python.

  • 2+ years of experience with public cloud technologies such as AWS, DataBricks is required.

  • Experience in build and optimize Spark jobs for performance and cost efficiency.

  • Prior experience in working with CI/CD pipelines and orchestration tools.

  • Able to monitor and troubleshoot production data pipelines.

  • Knowledge of ETL Tools, DEVOPS, GitHub is preferrable.

  • Exposure to Elasticsearch, Graph DB is a big plus

  • Exposure to machine learning is a plus.

  • Knowledge of Finance, custody services, data privacy, AML and related compliance domain is plus.

Additional requirements:

  • Prior experience with Finance and compliance domain.

  • Certifications in Databricks, Cloud environment.

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement

Skills

PythonScalaAWSCI/CDSQLElasticsearchMachine LearningSparkHadoopDatabricksData EngineeringETLGitHubDevOpsAMLCompliance

Similar Jobs

20

Senior Manager, Big Data and Analytics

Hp · WAV01 - Vancouver, Washington, US, United States of America +1

1 week ago

Big Data and AI Specialist

Vodafone · Iringa, Iringa Region,TZ, TZ

2 weeks ago

Insurance Big Data and Artificial Intelligence Policy Director (Analyst VI) - DORA: DOI Hybrid

State of Colorado · Denver, CO, CO, US · Hybrid

1 month ago

Big Data and Artificial Intelligence Policy Director (Analyst VI) - DORA: DOI Hybrid

State of Colorado · Denver, CO, CO, US · Hybrid

1 month ago

Big Data and Business Analyst (Temporary position)

Robert Bosch · Beograd, Serbia · Hybrid

3 months ago

Data Analyst - Big Data and Analytics Platform

Icf · Nationwide Remote Office (US99), United States of America · Remote

3 months ago

Lead Data Scientist - Big Data and Modelling

Cermati.com · Jakarta, Jakarta, Indonesia

5 months ago

Associate Lecturer - LAW2636 Legal Considerations of Artificial Intelligence, Big Data and Blockchain (HE7)

Sim · Clementi Campus, Singapore

6 months ago

Senior Software Developer (C++) in Bioinformatics and Big Data

National Center for Biotechnology Information (NCBI) · Bethesda, Maryland +1 · Remote, Onsite

1+ year ago

Software Development Manager - Instant Video - Playback Insights and Big Data

Comtech LLC · Seattle, WA, United States

1+ year ago

Experienced Big Data and Hadoop Trainer

SRP Systems Inc · Princeton, NJ, United States

1+ year ago

Marketing / Branding Expert needed for a Big Data and Machine Learning Company

SRP Systems Inc · Princeton, NJ, United States

1+ year ago

Tenure Track Faculty Position with a focus on Big Data Analytics and Aviation Safety in the School of Graduate Studies, Daytona Beach Campus

ERAU Careers · 341 College of Aviation, United States of America

3 months ago

Big Data Architect and Trainer

SRP Systems Inc · Princeton, NJ, United States

1+ year ago

Big Data Developer Pyspark and Scala

Citi Bank · 1/124, SHIVAJI GARDENS, MOONLI, India · Hybrid

2 weeks ago

Software Engineer III - Big Data Pyspark, Java And AWS

JPMorgan Chase · Wilmington, DE, United States, US

5 months ago

Big Data Keynote/Guest Speaker | Technology And Innovation [ESC1X201]

ProSidian Consulting · Texas City, TX, United States

1+ year ago

An external freelance lecturer for “Big Data & Analytics” (m, f, d) (Digital Business and Data Science, B.Sc.) in Hamburg, Germany

Global University Systems GmbH · Hamburg, Hamburg · Remote, Hybrid, Onsite

5 months ago

Exciting FinTech Startup - ETL Developer or Big Data analyst - Data Warehouse, AWS Redshift, Stitch and Segment

Oiga · Medellín, Antioquia, Colombia

1+ year ago

An external freelance lecturer for “Big Data & Analytics” (m, f, d) (Digital Business and Data Science, B.Sc.) in Hamburg, Germany

Global University Systems GmbH

5 months ago