The Lead Data Engineer will design and build data pipelines using various technologies, working closely with data science and engineering teams to manage large datasets and build applications for data orchestration and monitoring.
Windfall is seeking a Lead Data Engineer to join our core data team. As a Lead Data Engineer on our data team, you will be building out the core data asset that everything else at Windfall is built on top of. Communication and collaboration are at the heart of Windfall, and you will work closely with our product, data science, and other engineering teams. You will personally design and build the pipelines for massive datasets, taking them all the way from inception to exploration to production and customer use.
We’re on a mission to change how organizations perceive and use people data. And we hold true to our core values of: (1) Be an excellent communicator; (2) Operate with transparency; (3) Provide leverage, not optimization; (4) Make a difference every day; and (5) Act with integrity and trust.
Responsibilities:
- Construct data pipelines to ingest and merge billions of individual entities into Windfall’s core data asset
- Work closely with our data science team to run ML models on top of billions of data points
- Build supporting data services and applications to orchestrate and monitor our data systems
Some technology you will use:
- Cloud platform - GCP
- Programming languages - Java, Python, and Kotlin
- Data warehouse & databases - BigQuery, Postgres, Scylla/Cassandra
- Distributed processing frameworks - Dataflow (Apache Beam) and Apache Spark
- Orchestration - Airflow
Requirements:
- 8+ years of professional data engineering experience
- Significant experience working with Apache Beam/Spark/Flink or MapReduce
- Strong Object-oriented programming ability in a JVM language
- Expert knowledge of distributed data processing
- Familiarity with different datastores, their differences, and appropriate usages
- Experience at a sub-200 person company
- You communicate as well as you code
- You can simplify complex problems into simple solutions
- You balance a strong sense of ownership and responsibility in your work with collaboration and team alignment
- You are comfortable making trade-offs between quality, complexity, and speed-of-delivery
Preferred Qualifications:
- Previous experience in a player-coach role
- Proven experience taking a large project from ideation to production
- Experience leading greenfield projects
- Working knowledge of cloud-native data engineering infrastructure
Additional Information:
The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across California and Colorado. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. We also offer a comprehensive benefits package, which you can explore on our careers site.
California: Salary range is $170k - $240k
Colorado: Salary range is $150k - $220k
About Windfall
Windfall is a people intelligence and AI company that gives go-to-market teams actionable insights. By democratizing access to people data, organizations can intelligently prioritize go-to-market resources to drive greater business outcomes. Powered by best-in-class machine learning and artificial intelligence, Windfall activates insights into workflows that engage the right people for each respective organization. More than 1,500 data-driven organizations use Windfall to power their business. For more information, please visit www.windfall.com.
We comply with CCPA. For more information on how we comply, review our privacy notice.
Top Skills
Airflow
Apache Beam
Spark
BigQuery
Cassandra
Dataflow
GCP
Java
Kotlin
Postgres
Python
Scylla
Similar Jobs
Payments
The Lead Data Engineer designs, develops, and maintains software solutions while mentoring junior team members and participating in agile practices.
Top Skills:
ClouderaHiveImpalaJavaKafkaLinuxNifiOoziePostgresScalaSparkSqoopUnix
Financial Services
The Senior Lead Software Engineer will design and develop high-performance transactional data systems, optimize OLTP platforms, and lead technical guidance within an agile team.
Top Skills:
Apache KafkaAWSAzureC++CockroachdbGCPGoJavaKinesisKubernetesMqMySQLOraclePostgresPythonSQL Server
Healthtech • Biotech
The Lead Data Engineer will improve data management and database infrastructure, work on system integration, and enhance performance and scalability for Elekta's Oncology Information Systems.
Top Skills:
Amazon Web ServicesCi/CdData ManagementDatabase TechnologiesGitGCPAzure
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
