- Location
- New York
- Type
- Full-time
- Department
- Engineering
- Experience
- 7+ years
- Closing date
- Today
- Source
- CareersPage
Description
DataBricks Data Engineer requires:
- Must have functional and technical experience with Sales Incentive Compensation (SIC) including incentive plan design, quota allocation, attainment tracking, commission calculations, and payout processing.
- At least 7 years of experience in data engineering, with a minimum of 3 years hands-on experience building and managing Databricks solutions.
- Extensive experience with Databricks platform components: Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows.
- Strong proficiency in Python and/or Scala for data engineering; SQL expertise required.
- Strong experience with core data engineering practices including data ingestion, transformation (ETL/ELT), data modeling, and pipeline orchestration.
- Experience integrating Databricks with cloud platforms (Azure, AWS, or GCP) and data sources such as Azure Data Factory, Event Hubs, Kafka, or equivalent.
- Passion and ability to create holistic, end-to-end, integrated data solutions.
- Familiarity with SIC platforms (e.g., Anaplan, Oracle ICM, Varicent) and experience integrating them with data engineering platforms is a strong plus.
- Ability to understand complex incentive compensation rules and model them accurately in data pipelines and transformation logic
Skills
PythonScalaAWSAzureGCPSQLOracleDatabricksData EngineeringETL
