- Type
- Contract
- Department
- Engineering
- Experience
- 8+ years
- Source
- RecruiterFlow
Description
Our client is seeking an experienced AWS Data Engineer to design and build cloud-native data pipelines on an open data platform, architecting Apache Iceberg tables and optimizing large-scale data processing across AWS and Snowflake environments.
Responsibilities & Qualifications
- Design and develop end-to-end data ingestion, transformation, and processing pipelines using AWS Glue, Apache Spark, and Amazon S3
- Build and manage Apache Iceberg tables to enable open, interoperable access across multiple compute engines
- Configure and integrate Apache Polaris with Iceberg REST Catalog for AWS and Snowflake environments
- Implement table maintenance, schema evolution, and partitioning strategies to optimize performance and cost
- Support platform security, access controls, and AWS networking infrastructure
- Conduct workload benchmarking and performance analysis comparing AWS Glue versus Snowflake for cost and efficiency
- Provide operational monitoring, troubleshooting, and optimization of cloud-based data platform infrastructure
Requirements
- 8–10 years of experience in data engineering and cloud-based data pipeline development
- Proven expertise with Amazon S3, AWS Glue, Apache Spark, and large-scale data processing
- Strong hands-on experience with Apache Iceberg architecture, including table design and maintenance
- Demonstrated proficiency with Apache Iceberg schema evolution and partitioning optimization
- Experience configuring and deploying Apache Polaris and Iceberg REST Catalog
- Solid understanding of AWS IAM, networking, and security best practices in cloud environments
- Experience with performance tuning, cost optimization, and cross-platform data tool evaluation
Skills
AWSSparkSnowflakeData EngineeringREST
