Senior Software Consultant - Databricks / Snowflake - Afternoon

Company Overview 

10Pearls is an award-winning end-to-end digital innovation company that helps businesses imagine and build the future. We are proud to announce that 10Pearls was named the winner of the Best Tech Work Culture Timmy Award in Washington, D.C. by Tech in Motion, recognized on the Inc. 5000 Fastest-Growing Companies List, and was ranked the #1 Most Diverse Midsize Company in Greater Washington. We partner with businesses to help them transform, scale, and accelerate by adopting digital and exponential technologies. Our work has ranged from creating highly usable, secure digital experiences, mobile and software products, to helping businesses modernize through cloud adoption and development and the digitalization of their business processes. Our clientele is highly diverse, including Global 1000 enterprises, mid-market businesses, and even high-growth start-ups. But those are just facts. What makes us unique is that we have a true heart and soul. We have a strong focus on a double bottom line and actively support and engage with the communities where we live and work to make the world a better place. In a nutshell, we believe in doing well while doing good and know how to balance the two. 

Role

We are seeking a skilled Data Engineer with strong expertise in Databricks and Snowflake to design, build, and optimize scalable data pipelines. You will work on high‑performance data processing workflows that support our platform, with a focus on real‑time analytics, large‑scale data transformations, and efficient data modeling. If you have experience with distributed data systems, cloud‑based data platforms, and modern data engineering best practices, we’d love to hear from you.

Responsibilities

• Build and optimize data pipelines that ingest, validate, and transform core banking data (accounts, transactions, balances, parties, fees) from multiple source systems into our Databricks/Delta Lake lakehouse.
• Scale and evolve a multi‑tenant architecture, ensuring tenant isolation, efficient partitioning, and consistent schema evolution as we onboard new banks.
• Own CI/CD for the data platform, including GitHub Actions workflows, SQLMesh plan/apply lifecycle, and Databricks deployment automation.
• Develop and integrate ML models, including propensity scoring, churn prediction, segmentation, and customer scoring models that feed directly into analytics and decisioning layers.
• Ensure pipeline reliability through monitoring, alerting, and robust data validation across tenants and environments.
• Design and maintain 300+ SQL and Python data models across Bronze, Silver, and Gold layers using SQLMesh, with an emphasis on clean abstractions, reusability, and correctness.
• Own the metrics layer, defining and validating gold‑standard business metrics (revenue, attrition, household analytics, segmentation, balance projections) used by dashboards and APIs.
• Champion data quality by writing SQLMesh audits, unit tests, and enforcing schema contracts to ensure downstream consumers can trust the data.
• Collaborate with product and banking domain experts to translate business requirements into well‑modeled, documented, and performant data assets.
• Drive documentation and discoverability, ensuring data models are self‑describing and easily understood by analysts and product teams.

Requirement

• 8+ years of software engineering experience, with deep expertise in data engineering and strong exposure to analytics engineering or data modeling.
• Production experience with SQLMesh or dbt, including building, testing, and deploying transformation projects (SQLMesh strongly preferred).
• Hands‑on experience with Databricks or Snowflake, operating pipelines and warehouses in production environments.
• Advanced SQL skills, including complex window functions, CTEs, incremental logic, and performance‑optimized aggregations.
• Proficiency in Python, especially for PySpark transformations, data validation, and pipeline automation.
• Strong understanding of dimensional modeling, medallion/layered architectures, and data quality best practices.
• Experience with CI/CD for data, including automated testing, version control, and deployment pipelines.

Nice‑to‑Have Qualifications

• Experience building or operationalizing ML models (propensity, churn, segmentation) within a data platform.
• Background in banking, financial services, or fintech data domains.
• Familiarity with Azure services (ADLS Gen2, Azure SQL, Databricks on Azure).
• Experience with multi‑tenant SaaS data architectures, including schema isolation and tenant‑aware partitioning.
• Exposure to data mesh concepts and domain‑oriented data ownership.
• Familiarity with Databricks Unity Catalog, Auto Loader, or Databricks Workflows.
• Experience with Linear, GitHub Actions, or similar project management and CI/CD tooling.