CargoSprint Logo

CargoSprint

Lead Engineer, Data Processing (Databricks)

Posted 5 Days Ago
Remote
Hiring Remotely in United States
Senior level
Remote
Hiring Remotely in United States
Senior level
Lead the Data Processing team, owning system design and implementation, driving modernization, and ensuring operational reliability and data quality through effective pipeline management.
The summary above was generated by AI

Empowering the people that make global commerce happen.


About CargoSprint

CargoSprint is made up of a world-class team of highly motivated individuals who are passionate about transforming the cargo industry. We have developed cutting-edge digital solutions that streamline cargo operations, enhance efficiency, and improve the overall experience for everyone involved. Our workplace fosters innovation, collaboration, and the drive to solve industry challenges.


CargoSprint is dedicated to delivering game-changing solutions that connect the cargo industry like never before, and we are looking for driven, enthusiastic people who share our vision of innovation and excellence.


If you think we are a great mutual fit, we want hear from you!

About You

You're a hands-on leader who can run a team and still ship. You care about operational reliability, clear ownership boundaries, and measurable throughput. You modernize systems safely, reduce toil, and build habits that make pipelines dependable. 
About the role

This role leads our Data Processing team. You'll own the pipelines and processing jobs that transform data for product and business use cases, with an emphasis on modernization, reliability, and data quality. Strong experience with Databricks (or similar Spark-based platforms) is required. 


What you'll do

  • Lead the Data Processing team: technical direction, delivery cadence, coaching, and hiring support (as needed). 
  • Own core processing systems end-to-end: design, implementation, quality, observability, and on-call readiness. 
  • Drive modernization of legacy processing: simplify flows, reduce failure modes, improve performance and cost. 
  • Build and operate pipelines (batch and/or streaming) with strong data quality, lineage, and backfill strategies. 
  • Establish SLAs/SLOs for key pipelines and improve monitoring, alerting, and incident response. 
  • Partner with application engineering, finance, and analytics to prioritize work and deliver dependable outputs. 
  • Raise the engineering bar via standards, PR reviews, runbooks, and pragmatic architecture decisions. 


Qualifications

  • 8+ years building production systems; 2+ years leading engineers or acting as a team tech lead. 
  • Strong experience with data processing systems (ETL/ELT, streaming, batch processing, orchestration). 
  • Hands-on experience with Databricks and Spark (PySpark/Scala) and strong SQL. 
  • Experience modernizing legacy systems safely and incrementally. 
  • Operational discipline: monitoring, alerting, incident response, and root cause analysis for pipelines. 
  • Ability to drive cross-team alignment and translate requirements into reliable delivery. 

Nice to have 

  • Lakehouse patterns (Delta Lake, Iceberg, or Hudi) and data catalog/governance tools. 
  • .Net or Java exprertise 
    Event-driven architectures (Kafka/Kinesis/PubSub) and streaming pipelines. 
  • FinTech, payments, billing, invoicing, or other high-volume transactional domains. 
  • Experience enabling self-serve analytics for business users. 
  • Logistics, cargo, or supply chain experience.  
  • Spanish language proficiency. 

 

Compensation and benefits 

We offer competitive pay and benefits designed to fuel our team's success. 

  • Health and Wellness: Medical, dental, and vision plans for you and your family 
  • Future-Ready: 401(k) with company match 
  • Work Life Balance: Generous flexible PTO program and paid holidays 
  • Grow With Us: Professional development opportunities 


#LI-Remote

Does this role sound like the next step in your career?

We’d love to hear from you! If you don’t meet all of the requirements exactly, we encourage you to use your cover letter to tell us about your unique experience—talent comes from many places, and skills are transferable.


Our Commitment to an Extraordinary Work Environment

At CargoSprint, we value diversity and inclusivity. We strive to create a welcoming and supportive community for employees from all backgrounds. Regardless of your gender, sexual orientation, physical ability, religion, ethnicity, race, or age, you will find a place where you can thrive and be your authentic self.

Our CargoSprint Recruitment Team personally reviews every application.

Top Skills

Databricks
Delta Lake
Hudi
Iceberg
Kafka
Kinesis
Pubsub
Pyspark
Scala
Spark
SQL

CargoSprint Los Angeles, California, USA Office

Los Angeles, CA, United States

Similar Jobs

3 Hours Ago
Remote or Hybrid
United States
123K-169K Annually
Senior level
123K-169K Annually
Senior level
Digital Media • Fintech • Information Technology • Mobile • Payments • Software • Financial Services
The Senior Solutions Consultant will demonstrate digital banking solutions to clients, establish relationships throughout the sales cycle, and collaborate with sales teams.
Top Skills: Digital Banking SolutionsMobile DevicesProjection/Mirroring SoftwareTabletsTechnical ArchitectureWireless Connectors
3 Hours Ago
Remote or Hybrid
United States
45K-45K Annually
Junior
45K-45K Annually
Junior
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The role involves providing exceptional customer service for Group Variable Universal Life insurance policies, responding to inquiries, and educating customers on policy features.
Top Skills: Computer Systems
3 Hours Ago
Remote or Hybrid
United States
110K-168K Annually
Senior level
110K-168K Annually
Senior level
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The Director - API Product Management leads API integrations within HR tech, overseeing product strategy, team management, and stakeholder collaboration to enhance customer experiences.
Top Skills: AgileAPIsHcmWorkday Pro

What you need to know about the Los Angeles Tech Scene

Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.

Key Facts About Los Angeles Tech

  • Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
  • Key Industries: Artificial intelligence, adtech, media, software, game development
  • Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
  • Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account