Onebridge Logo

Onebridge

Sr. Data Engineer - Databricks

Reposted 2 Months Ago
Remote
Hiring Remotely in United States
Senior level
Remote
Hiring Remotely in United States
Senior level
The Sr. Data Engineer - Databricks designs and maintains scalable data solutions using Apache Spark and Databricks, ensuring data quality and performance for analytics and reporting.
The summary above was generated by AI

Onebridge, a Marlabs Company, is a global AI and Data Analytics Consulting Firm that empowers organizations worldwide to drive better outcomes through data and technology. Since 2005, we have partnered with some of the largest healthcare, life sciences, financial services, and government entities across the globe. We have an exciting opportunity for a highly skilled Sr. Data Engineer – Databricks to join our innovative and dynamic team. 

Sr. Data Engineer – Databricks | About You 

As a Sr. Data Engineer – Databricks, you are responsible for designing, building, and maintaining scalable data solutions that enable analytics, reporting, and data-driven decision-making across the organization. You bring strong experience working with modern data platforms and enjoy transforming complex data into reliable, high-quality datasets. You thrive in collaborative environments, partnering with analytics, engineering, and business teams to deliver impactful data pipelines. You are comfortable working in cloud-based ecosystems and consistently apply best practices in data engineering, data governance, and performance optimization. You take ownership of data solutions from design through deployment and ongoing optimization.

Sr. Data Engineer – Databricks | Day-to-Day 

  • Design, develop, and maintain robust and scalable data pipelines using Apache Spark and cloud-native data services.
  • Build, optimize, and support ETL and ELT workflows to enable analytics, reporting, and downstream applications.
  • Implement and manage data solutions using Databricks, Delta Lake, and Unity Catalog.
  • Ensure data quality, reliability, and performance across large-scale and complex datasets.
  • Collaborate with cross-functional teams to gather data requirements and translate them into effective technical solutions.
  • Apply data engineering best practices related to scalability, security, monitoring, and maintainability.
  • Support the continuous improvement of data architecture, pipeline performance, and operational stability in a cloud environment.

Sr. Data Engineer – Databricks | Skills & Experience 

  • 7+ years of experience in data engineering or a related technical role.
  • Strong hands-on experience with Databricks and Apache Spark.
  • Experience working with Delta Lake and Unity Catalog for data management and governance.
  • Solid understanding of data pipeline design, ETL and ELT patterns, and data modeling concepts.
  • Experience working with cloud data platforms such as Azure, AWS, or GCP.
  • Proven ability to design scalable, reliable, and high-performing data solutions.
  • Strong problem-solving skills and the ability to work effectively in collaborative, fast-paced environments.

Similar Jobs

8 Hours Ago
In-Office or Remote
2 Locations
102K-204K Annually
Senior level
102K-204K Annually
Senior level
Fitness • Healthtech • Retail • Pharmaceutical
Build and operate an Azure Databricks data platform using Python frameworks, configuration-driven pipelines, CI/CD, Terraform, and infrastructure as code. Support production reliability through monitoring, incident response, root cause analysis, runbooks, disaster recovery, and automation. Maintain data security and governance with Unity Catalog, enable domain teams to onboard data sources, and improve self-service platform capabilities in a regulated healthcare environment.
Top Skills: Adls Gen2SparkAzureAzure Key VaultCi/CdDatabricksDatabricks Asset BundlesDelta LakeDelta Live TablesGitGithub ActionsInfrastructure As CodeLakeflowMicrosoft Entra IdPysparkPytestPythonSQLTerraformUnity Catalog
One Month Ago
Remote or Hybrid
OH, USA
Senior level
Senior level
Financial Services
Build and operate scalable Databricks-on-AWS data pipelines using PySpark, Delta Lake, and lakehouse patterns. Optimize performance, implement data quality, monitoring, alerting, and automated remediation, and deliver curated datasets for BI and analytics partners. Collaborate with stakeholders on architecture and design while applying secure software engineering, CI/CD, agile, and operational stability practices. The role also uses AI-assisted development tools and supports workforce data analytics.
Top Skills: AlteryxAmazon AthenaAmazon EmrAmazon S3Apache AirflowApache IcebergSparkAutosysAWSAws CloudwatchAws GlueAws LambdaBitbucketClaudeDatabricksDatabricks WorkflowsDelta LakeDelta Live TablesGitGithub CopilotJavaJenkinsOracleParquetPysparkPythonScalaSigmaSpinnakerSQLTableau
One Month Ago
Remote
New Jersey, USA
Senior level
Senior level
Big Data • Cloud • Analytics • Consulting
Build scalable ETL/ELT pipelines in Databricks using PySpark, Spark SQL, Delta Lake, Delta Live Tables, Workflows, and Unity Catalog. Ingest batch and streaming data across medallion architecture layers, develop transformations and data quality rules, and implement orchestration, monitoring, alerting, and automation. The role requires cloud data engineering, data modeling, performance tuning, CI/CD, and strong knowledge of lakehouse architecture, governance, and distributed systems.
Top Skills: AirflowSparkAuto LoaderAWSAzureCi/CdData GovernanceData WarehousingDatabricksDatabricks WorkflowsDbtDelta LakeDelta Live TablesDimensional Data ModelingGCPGitLakeflow ConnectLakehouseMedallion ArchitectureMetadata ManagementPysparkPythonRagSpark SqlSQLStructured StreamingUnity Catalog

What you need to know about the Los Angeles Tech Scene

Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.

Key Facts About Los Angeles Tech

  • Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
  • Key Industries: Artificial intelligence, adtech, media, software, game development
  • Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
  • Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account