Design and lead Databricks Lakehouse solutions, optimize data pipelines, implement Delta Lake practices, and integrate cloud services.
Company Description
Role: Databricks Architect
Duration: 12 months
Location: Troy, MI- Remote
Minimum exp.- 14-15 yrs.
Role Overview
We are looking for a Databricks Architect to design and lead modern Lakehouse data platforms using Databricks. The role focuses on building scalable, high-performance data pipelines and enabling analytics and AI use cases on cloud-native data platforms.
Key Responsibilities
- Architect and implement Databricks Lakehouse solutions for large-scale data platforms
- Design and optimize batch & streaming data pipelines using Apache Spark (PySpark/SQL)
- Implement Delta Lake best practices (ACID, schema enforcement, time travel, performance tuning)
- Build and manage Databricks jobs, workflows, notebooks, and clusters
- Enable data governance using Unity Catalog (access control, lineage)
- Integrate Databricks with cloud data services (ADLS / S3, ADF, Synapse, etc.)
- Support analytics, BI, and AI/ML workloads (MLflow exposure is a plus)
- Lead solution design discussions and mentor data engineering teams
Must-Have Skills
- 10+ years in data engineering / data architecture
- 5+ years of strong hands-on experience with Databricks
- Expert in Apache Spark, PySpark, SQL
- Strong experience with Delta Lake & Lakehouse architecture
- Cloud experience on Azure Databricks / AWS Databricks
- Proven experience in designing high-volume, scalable data pipelines
Good-to-Have
- Unity Catalog, MLflow, Databricks Workflows
- Streaming experience (Kafka / Event Hubs)
- CI/CD for Databricks (Azure DevOps / GitHub)
All your information will be kept confidential according to EEO guidelines.
Top Skills
Spark
Aws Databricks
Azure Databricks
Azure Devops
Ci/Cd
Databricks
Delta Lake
Event Hubs
Git
Kafka
Mlflow
Pyspark
SQL
Unity Catalog
Similar Jobs
AdTech • Artificial Intelligence • Cloud • Digital Media • Marketing Tech • Analytics • Consulting
The Lead MarTech Product Strategist will architect MarTech roadmaps, lead Adobe projects, manage project teams, mentor talent, and ensure quality delivery for client business growth.
Top Skills:
Adobe Experience CloudAdobe Experience PlatformAdobe Journey OptimizerAdobe Target
Fintech • Financial Services
Lead and mature internal audit function with a risk-based audit plan. Oversee audits across financial, operational, and regulatory domains. Build and manage a high-performing audit team across multiple locations.
Top Skills:
Power BIPythonSQL
Fintech • Information Technology • Software • Financial Services
The Senior UI Developer will build high-performance frontend applications, improve code quality, and collaborate with teams to enhance user interfaces.
Top Skills:
Ag-GridCSSJavaScriptMuiPlaywrightReactStorybookTypescriptVitest
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
