Lead and mentor a data engineering team, architect data solutions using Azure and Databricks, implement governance protocols, and ensure data quality in healthcare domains.
Senior Data Engineering Lead - 10 + Years Experience required
Work Mode: Remote with PST, travel as needed, to Orange County, CA 20-30%
Role Overview
We are seeking an experienced Senior Data Engineering Lead to drive enterprise data platform architecture, lead data engineering initiatives, and mentor a growing team of engineers. This role combines strategic leadership with deep technical expertise in modern cloud data platforms, particularly the Azure ecosystem and Databricks Lakehouse architecture.
The ideal candidate will take ownership of data engineering standards, governance frameworks, and scalable data solutions while ensuring high data quality, security, performance, and regulatory compliance aligned with healthcare industry standards.
Key Responsibilities
Leadership & Team Mentoring
Lead, mentor, and develop a high-performing data engineering team.
Provide technical guidance, architectural oversight, and code reviews.
Establish engineering best practices, standards, and governance frameworks.
Foster collaboration across engineering, analytics, BI, and business teams.
Drive knowledge sharing, technical upskilling, and continuous improvement.
Data Platform Architecture & Engineering
Design, implement, and maintain scalable data pipelines across cloud and hybrid environments.
Architect modern lakehouse /data warehouse solutions using Azure and Databricks.
Develop robust ETL/ELT frameworks ensuring reliability, performance, and scalability.
Lead integration of data from enterprise systems, SaaS platforms, APIs, and on-premises sources.
Define data models optimized for analytics, reporting, and advanced analytics use cases.
Data Governance, Security & Quality
Implement enterprise-grade data governance using Databricks Unity Catalog.
Establish data lineage, access controls, metadata management, and security policies.
Ensure compliance with healthcare security and regulatory standards where applicable.
Build data quality monitoring, observability, and reliability frameworks.
Strategy & Stakeholder Engagement
Develop long-term data platform strategy aligned with organizational goals.
Drive cloud data modernization and optimization initiatives.
Collaborate with business stakeholders to translate requirements into scalable solutions.
Support BI and analytics teams with governed, high-quality datasets.
Mandatory Technical Requirements
Strong hands-on experience with:
Azure Data Factory
Azure Data Lake Storage
Azure Databricks
Azure Synapse Analytics
Proven experience designing enterprise-scale data pipelines and lakehouse architectures.
Mandatory expertise in Databricks Unity Catalog, including governance implementation, lineage tracking, access control policies, and metadata management.
Strong experience in ETL/ELT development, data warehousing, and dimensional data modeling.
Advanced proficiency in Python, SQL, Spark, JSON, and modern data engineering frameworks.
Experience integrating enterprise systems such as ERP, CRM, and healthcare/SaaS platforms where applicable.
Solid understanding of data governance, security protocols, RBAC, data lineage, and data quality practices.
Experience working in Agile environments with DevOps, CI/CD pipelines, and version control tools (Git, Jenkins, Jira).
Preferred Qualifications
Experience in healthcare, diagnostics, or regulated data environments.
Experience implementing large-scale cloud data modernization programs.
Exposure to real-time data pipelines, streaming platforms, and API-driven architectures.
Experience with BI/visualization tools such as Power BI, Tableau, or similar platforms.
Strong stakeholder communication and consulting experience.
Experience mentoring distributed or global teams.
Education
Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related field.
Key Competencies
Technical leadership and mentoring capability
Strategic thinking and data architecture expertise
Strong problem-solving and analytical skills
Excellent communication and stakeholder management
Ability to balance hands-on engineering with leadership responsibilities
Work Mode: Remote with PST, travel as needed, to Orange County, CA 20-30%
Role Overview
We are seeking an experienced Senior Data Engineering Lead to drive enterprise data platform architecture, lead data engineering initiatives, and mentor a growing team of engineers. This role combines strategic leadership with deep technical expertise in modern cloud data platforms, particularly the Azure ecosystem and Databricks Lakehouse architecture.
The ideal candidate will take ownership of data engineering standards, governance frameworks, and scalable data solutions while ensuring high data quality, security, performance, and regulatory compliance aligned with healthcare industry standards.
Key Responsibilities
Leadership & Team Mentoring
Lead, mentor, and develop a high-performing data engineering team.
Provide technical guidance, architectural oversight, and code reviews.
Establish engineering best practices, standards, and governance frameworks.
Foster collaboration across engineering, analytics, BI, and business teams.
Drive knowledge sharing, technical upskilling, and continuous improvement.
Data Platform Architecture & Engineering
Design, implement, and maintain scalable data pipelines across cloud and hybrid environments.
Architect modern lakehouse /data warehouse solutions using Azure and Databricks.
Develop robust ETL/ELT frameworks ensuring reliability, performance, and scalability.
Lead integration of data from enterprise systems, SaaS platforms, APIs, and on-premises sources.
Define data models optimized for analytics, reporting, and advanced analytics use cases.
Data Governance, Security & Quality
Implement enterprise-grade data governance using Databricks Unity Catalog.
Establish data lineage, access controls, metadata management, and security policies.
Ensure compliance with healthcare security and regulatory standards where applicable.
Build data quality monitoring, observability, and reliability frameworks.
Strategy & Stakeholder Engagement
Develop long-term data platform strategy aligned with organizational goals.
Drive cloud data modernization and optimization initiatives.
Collaborate with business stakeholders to translate requirements into scalable solutions.
Support BI and analytics teams with governed, high-quality datasets.
Mandatory Technical Requirements
Strong hands-on experience with:
Azure Data Factory
Azure Data Lake Storage
Azure Databricks
Azure Synapse Analytics
Proven experience designing enterprise-scale data pipelines and lakehouse architectures.
Mandatory expertise in Databricks Unity Catalog, including governance implementation, lineage tracking, access control policies, and metadata management.
Strong experience in ETL/ELT development, data warehousing, and dimensional data modeling.
Advanced proficiency in Python, SQL, Spark, JSON, and modern data engineering frameworks.
Experience integrating enterprise systems such as ERP, CRM, and healthcare/SaaS platforms where applicable.
Solid understanding of data governance, security protocols, RBAC, data lineage, and data quality practices.
Experience working in Agile environments with DevOps, CI/CD pipelines, and version control tools (Git, Jenkins, Jira).
Preferred Qualifications
Experience in healthcare, diagnostics, or regulated data environments.
Experience implementing large-scale cloud data modernization programs.
Exposure to real-time data pipelines, streaming platforms, and API-driven architectures.
Experience with BI/visualization tools such as Power BI, Tableau, or similar platforms.
Strong stakeholder communication and consulting experience.
Experience mentoring distributed or global teams.
Education
Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related field.
Key Competencies
Technical leadership and mentoring capability
Strategic thinking and data architecture expertise
Strong problem-solving and analytical skills
Excellent communication and stakeholder management
Ability to balance hands-on engineering with leadership responsibilities
Top Skills
Azure Data Factory
Azure Data Lake Storage
Azure Databricks
Azure Synapse Analytics
JSON
Python
Spark
SQL
Similar Jobs
.png)
Cloud • Insurance • Payments • Software • Business Intelligence • App development • Big Data Analytics
The Data Engineer will build and maintain data solutions, optimize data architectures, and ensure data quality while collaborating with cross-functional teams.
Top Skills:
BigQueryGoogle Cloud PlatformPythonSQL
Information Technology
The Senior Data Engineer will design and maintain scalable data architecture, optimize data workflows, lead the team, and align data solutions with business goals.
Top Skills:
AirflowAWSAzureBigQueryGCPKafkaPythonRedshiftSnowflakeSparkSQL
AdTech • Marketing Tech • Sales
The Data Engineer will design and maintain scalable data solutions, build ETL pipelines, manage data lakes, and collaborate with cross-functional teams for analytics support.
Top Skills:
AirflowAksSparkAzure DatabricksDatabricks WorkflowsPostgresPysparkPythonSpark Sql
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
