- Location
- Walnut Creek, CA
- Workplace
- Remote
- Type
- Contract
- Department
- Engineering
- Seniority
- Senior
- Experience
- 2+ years
- Source
- Breezy HR
Description
About Object Edge:
Object Edge is an AI-native, forward-deployed engineering firm. It builds Hive, a company brain for revenue, embeds engineers to build agentic workflows on it, and provides management engineering to lead the change.
Award-winning eCommerce consultancy that delivers end-to-end design and development solutions centered around our client’s goals and customer needs.
Our teams unite curious, intellectual advisors across the globe who creatively solve business problems and produce measurable results.
About the Position:
Senior Data Engineering (Databricks)
What You’ll Do
• Be a technical leader in teams with minimal oversight and direction to deliver innovative
solutions on the Databricks Platform using core cloud data lakehouse methodologies,
tools, distributed processing engines, event streaming platforms, and other modern
data-related technologies.
• Be part of the Databricks Center of Excellence.
• Build the next generation of data platforms and work with some of the most forward-
thinking organizations in data and analytics.
• Work under the direction of a Solution Architect to help design and implement
components of our clients’ data platform solution.
• Participate in design sessions and be able to break down complex development tasks,
and complete development items on time.
• Contribute to various COE initiatives to develop Databricks solution accelerators and
bring an innovative mindset.
Location: Remote - United States
What You’ll Bring:
As a Senior Engineer in the Databricks practice, you will bring a curious mindset and a passion for
exploring innovative solutions to address our clients’ most pressing data challenges. You are a self-
starter who excels at breaking down complex problems and eagerly shares insights with your team
and the broader Builder community.
Key Responsibilities:
• Develop and implement data solutions using Databricks, with hands-on experience in
specific Databricks platform features such as Delta Lake, Uniform (Iceberg), Delta Live
Tables (Lakeflow Declarative Pipelines), and Unity Catalog.
• Collaborate with cross-functional teams to design and optimize data pipelines for both
batch and streaming data, ensuring data quality and efficiency.
• Stay updated with emerging technologies, the latest Databricks platform features, and
continuously improve your skills in Databricks and other relevant tools.
Experience You’ll Bring:
• Strong data engineering experience, with at least 2 years of hands-on data pipeline
design and development experience with Databricks, including specific platform
features like Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative
pipelines), and Unity Catalog.
• Proficiency in designing and building robust, scalable, YAML configuration-driven data
pipelines with batch, micro-batch, and streaming data ingestion and processing patterns
using tools like Auto Loader, demonstrating a strong understanding of modern data
engineering practices.Experienced in building complex job/workflow orchestration patterns
using either Databricks jobs/workflows or other orchestration tools like Airflow, dbt
etc.
• Exposure to building robust data quality, pipeline audit/observability solutions with
Databricks native features and/or other data quality tools/frameworks like Great
Expectations, Collibra, dbt, etc.
• Proficiency in Big Data Platforms: Apache Spark, Presto, Amazon EMR.
• Experience with Cloud Data Warehouses: Amazon Redshift, Snowflake, Google
BigQuery.
• Strong programming skills using SQL, stored procedures, and object-oriented programming
languages (like Java, Python, PySpark, etc.).
• Familiarity with building DevOps, CI/CD pipelines using Databricks Asset Bundles with
automated validation and testing.
• Exposure to Infrastructure as Code (IaC) tools like Terraform is a big plus.
• Familiarity with NoSQL Databases and Container Management Systems.
• Exposure to AI/ML tools (like MLflow), prompt engineering, and modern data and AI agentic
workflows.
• An ideal candidate will have Databricks Data Engineering Associate and/or Professional
certification completed with multiple Databricks project delivery experience.
Compensation:
Salary is commensurate with experience based on department and role.
Object Edge provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. We believe in providing employees with a work environment free of unlawful discrimination and harassment. In addition to national law requirements, Object Edge complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.