Senior Data Engineer

Waverly Advisors’ primary goal is to serve our clients, one another, and our communities. We aren’t your typical wealth management firm. Our intense client focus is at the center of everything we do. We go far beyond just managing our clients’ investments, offering truly in-depth financial planning. We set ourselves apart by actually living and acting on our guiding principle, ‘Serve.’ It is the reason we go to work every day.   

In this role, you’ll be challenged to take on work that upholds our guiding principle and drives Waverly Advisors forward. We hope you’ll grow as a person and leader in your field and transform those around you as well.    

Waverly Advisors is building a modern data platform on Databricks. It brings together custodian, portfolio management, CRM, and planning data into one governed source of truth that BI, internal applications, and AI workloads consume. Everything is deployed as code and governed to the standards expected of a regulated financial services organization. 

We are looking to add a Senior Data Engineer to our team in our Birmingham, AL office. The role is an essential part of the Waverly Advisors team. As Senior Data Engineer, you will help design how data arrives, how it is validated, how it is modeled, and how the rest of the firm consumes it. Some sources arrive as files, others through APIs that were not designed for our convenience. You will extend the ingestion framework we have started so that onboarding a source becomes configuration and review rather than a new engineering project. 

 

Responsibilities: 

• Extend our ingestion framework across data contracts, schema validation, quarantine handling, freshness monitoring, and provenance. 

• Onboard data sources end to end, from contract authoring through production go-live and monitoring. 

• Design the master account, entity, and household model that conforms data from every source system. 

• Build the curated datasets the firm runs on, from conformed core tables through the reporting and analytics layers that BI, internal applications, and AI consume. 

• Set testing standards for data code, including unit tests, contract fixtures, and integration tests on masked data. 

• Build production data operations, including quarantine triage, reprocessing workflows, and data quality expectations. 

• Write the runbooks and decision records that let the next engineer pick up your work and let an examiner trace a number back to its source file. 

 

Required Qualifications and Experience:  

• Production-depth experience with Spark and Delta Lake, including Structured Streaming, Auto Loader, schema evolution, and idempotent pipeline design.  

• Hands-on Databricks experience, including jobs, serverless compute, and working within Unity Catalog as a data producer: schema and table design, tagging, row filters and column masks, and grants managed as code.  

• Databricks Asset Bundles or equivalent deployment tooling. Our jobs and pipelines are deployed as code through CI, not built in the workspace UI.  

• Experience building or substantially owning a framework, not just a collection of pipelines.  

• Real API integration experience, including rate limits, inconsistent pagination, expiring credentials, and vendor behavior that changes without notice.  

• Testable Python, with transform logic structured as functions and pytest as part of your normal workflow.  

• Strong SQL skills, including window functions and modeling patterns for slowly changing data.  

• Comfort inheriting an existing codebase, learning its conventions, and improving it.  

• Sound judgment about which technical decisions are easy to reverse and which ones are long-term commitments.  

 

Domain knowledge is a real advantage. If you have worked with custodial data, portfolio accounting, or trust and entity structures, tell us. If you came to engineering through the industry rather than the other way around, we are interested, provided the engineering depth above is genuinely there. 

 

Preferred Qualifications and Experience:  

• Lakeflow or Delta Live Tables for declarative pipelines. 

• Azure services as a consumer: ADLS storage paths, Key Vault-backed secret scopes, and Event Hubs for streaming sources. 

• Data modeling for financial or accounting systems, including entity, ownership, and hierarchy structures. 

• Reverse ETL or writing data back into operational business systems. 

• Experience in financial services or another regulated industry. 

 

Travel:  

• Minimal travel is required.  

 

Physical Requirements:  

• Prolonged periods of sitting at a desk and working on a computer.  

• Must be able to lift 15 pounds at times.  

• Must be able to access and navigate each department at the organization’s facilities.  

 

Benefits:   

• Comprehensive Health, Dental, and Vision coverage to support your overall well‑being.  

• 401(k) retirement plan with match and profit sharing to help you invest in your future.  

• Twelve paid holidays each year.  

• An extra vacation day during your birthday week—so you can celebrate you!  

• Responsible Time Off Policy giving flexibility without annual PTO limits, while balancing team responsibilities and business needs.    

• Paid sabbatical program: Enjoy four consecutive weeks of paid time off after seven years of service.   

• Compensation commensurate with experience.  

 

Legal:   

Waverly Advisors, LLC. is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, gender, genetic information, national origin, disability, uniform service, veteran status, age, or any other classification protected by federal, state, or local law.