Senior Data Pipeline Engineer

| Venice
System1 is looking for engineers with production experience using SQL and Python in a Linux on AWS environment to join our Data Platform team. Our team is the horizontal layer supporting business intelligence, optimization, machine learning, external & internal reporting, and data APIs. We process and report on hundreds of millions of events and user attributes per day, gathered from an extremely heterogeneous set of input streams.
Our bread and butter is Python and PostgreSQL, but we also utilize a range of technologies including Lambda, SNS, SQS, Redis, and Dynamodb for caching and mapping, as well as Redshift, Kinesis, and Spark for large dataset munging and ad hoc analytics.

The Role You Will Have

  • Developing new data processing infrastructure
  • Adding new reporting features (i.e. low latency data-marts, API reporting services, and other custom reporting solutions)
  • Building new, and scaling out existing, Spark pipelines and streaming apps
  • Building out new ingest pipelines
  • Working with the data-science team to productionize ML pipelines
  • Coordinating data models with other engineering teams
  • Working with the DevOps team to increase our monitoring and alerting coverage as needed
  • Discover scale bottlenecks and successfully overcome them
  • Contributing technical specification documents for data architecture projects
  • Contributing to ongoing maintenance of existing infrastructure and investigating issues and failures

What You Will Bring

  • 5+ years full-time Python development experience
  • 3+ years PostgreSQL experience, SQL fluency, and an understanding of relational data models is a must
  • Extensive experience building and managing ETL pipelines for cloud-based projects from initial inception to production
  • Experience ensuring a high degree of reliability and data integrity for mission critical systems
  • Prior experience working with Spark or other big data processing platform in high-volume environments (and running the big data solutions in AWS) is a plus
  • Experience with large migration projects (moving data and ETL pipelines from a tech stack to a new, different one) is a plus
  • Excellent communication skills and a collaborative mindset

What We Have to Offer

  • Free Uber/Lyft to and from work every day!
  • Collegial and collaborative team with highly intelligent and motivated coworkers
  • Cross-team lunches and demos to foster learning
  • Unlimited Paid Sick Time, Competitive PTO and Benefits package
  • Catered meals and fully stocked kitchen
  • Weekly happy hour at various bars, restaurants, and venues across Los Angeles
  • Weekly fitness class with private trainer: high intensity training, yoga, beach volleyball, beach soccer, ultimate frisbee
  • Competitive Relocation Package
  • Company parties and outings: Skyzone indoor skydiving, Medieval Times, Karaoke, etc.
Read Full Job Description

Technology we use

  • Engineering
    • JavascriptLanguages
    • PythonLanguages
    • SqlLanguages
    • ReactLibraries
    • AngularJSFrameworks
    • DjangoFrameworks
    • SparkFrameworks
    • MongoDBDatabases
    • MySQLDatabases
    • PostgreSQLDatabases

Location

1501 Main Street, Venice, CA 90291
1501 Main Street, Venice, CA 90291

What are System1 Perks + Benefits

System1 Benefits Overview

-Free Uber/Lyft to and from work every day!
-Collegial and collaborative team with highly intelligent and motivated coworkers
-Cross-team lunches and demos to foster learning
-Unlimited Paid Sick Time, Competitive PTO and Benefits package
-Catered meals and fully stocked kitchen
-Weekly happy hour at various bars, restaurants, and venues across Los Angeles
-Weekly fitness class with private trainer: high intensity training, yoga, beach volleyball, beach soccer, ultimate frisbee
-Competitive Relocation Package
-Company parties and outings: Skyzone indoor skydiving, Medieval Times, Karaoke, etc.

Health Insurance & Wellness Benefits
Flexible Spending Account (FSA)
Disability Insurance
Dental Benefits
Vision Benefits
Health Insurance Benefits
Life Insurance
Pet Insurance
Wellness Programs
Retirement & Stock Options Benefits
401(K)
401(K) Matching
Company Equity
Performance Bonus
Child Care & Parental Leave Benefits
Family Medical Leave
Adoption Assistance
Vacation & Time Off Benefits
Generous PTO
Paid Holidays
Paid Sick Days
Perks & Discounts
Casual Dress
Commuter Benefits
Company Outings
Daily Meals Provided
Stocked Kitchen
Some Meals Provided
Happy Hours
Parking
Relocation Assistance
Fitness Subsidy
Professional Development Benefits
Job Training & Conferences
Tuition Reimbursement
More Jobs at System119 open jobs
All Jobs
Finance
Data + Analytics
Design + UX
Dev + Engineer
Marketing
Product
Design + UX
new
System1
Developer
new
System1
Marketing
new
System1
Marketing
new
System1
Developer
new
System1
Developer
new
System1
Finance
new
System1
Developer
new
System1
Developer
new
System1
Marketing
new
System1
Data + Analytics
new
System1
Developer
new
System1
Developer
new
System1
Data + Analytics
new
System1
Data + Analytics
new
System1
Developer
new
System1