Lead the design and delivery of data platforms based on Databricks, optimizing ETL/ELT pipelines and ensuring data governance and security.
Seeking a Databricks Architect to lead the design and delivery of large-scale data platforms using Databricks.
Key Responsibilities
- Architect and implement Databricks lakehouse solutions (Delta Lake, Spark, Unity Catalog, MLflow).
- Design and optimize ETL/ELT pipelines, data ingestion, and real-time streaming frameworks.
- Integrate Databricks with cloud services (AWS, Azure, GCP) and enterprise data systems.
- Establish data governance, security, and compliance frameworks.
- Lead migration efforts from legacy platforms to Databricks.
- Collaborate with business stakeholders, data engineers, and data scientists to deliver end-to-end solutions.
Required Skills
- Proven hands-on experience as a Databricks Architect / Senior Data Engineer.
- Strong experience with data pipelines including Cribl, Confluent and other high capacity data interchange
- Strong expertise in Databricks ecosystem (Spark, Delta Lake, SQL Analytics, MLflow).
- Cloud experience (Azure Databricks preferred, AWS/GCP acceptable).
- Strong coding background in Python, PySpark, SQL, Scala.
- Knowledge of data security, compliance, and governance standards.
- U.S. work authorization; must be based onshore.
Preferred
- Databricks certifications.
- Experience with BI tools (Power BI, Tableau) and automation/DevOps (Terraform, GitHub Actions, Jenkins).
- Prior consulting/enterprise implementation experience.
Top Skills
AWS
Azure
Confluent
Cribl
Databricks
Delta Lake
GCP
Github Actions
Jenkins
Mlflow
Power BI
Pyspark
Python
Scala
Spark
SQL
Tableau
Terraform
Unity Catalog
Similar Jobs
Big Data • Analytics
The Solutions Architect will lead enterprise-scale data migrations and design cloud-native architectures for data platform modernization using Azure technologies, focusing on scalability and advanced analytics.
Top Skills:
Azure Data FactoryAzure Data LakeAzure DatabricksSynapseUnity Catalog
Big Data • Analytics • Business Intelligence • Big Data Analytics
The Databricks Architect will design and implement data architecture using Databricks, ensuring secure and scalable data platforms on AWS/Snowflake with a focus on MLOps and CI/CD practices.
Top Skills:
Apache IcebergAWSDatabricksDelta LakeMlflowPythonScalaSnowflakeSparkUnity Catalog
Big Data • Analytics • Business Intelligence • Big Data Analytics
The role involves architecting and designing scalable Databricks solutions, improving performance, and implementing DevOps processes for data ecosystems.
Top Skills:
AWSDatabricksHadoopSpark
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
