Lead Data Engineer
Who are we?
Sense360 is a Series A startup that is funded by prominent VC funds such as FirstMark Capital (Airbnb, Pinterest, Riot Games) and Upfront Ventures (Maker, TrueCar). Our data insights have been referenced by many publications including Bloomberg and CNBC. Sense360 was founded by successful repeat entrepreneurs.
We are transforming the way businesses make decisions by combining massive, disparate datasets and turning them into accessible, actionable, and accurate insights that drive the strategic decisions that brands make. See our Culture Deck here.
What will you do?
As a Lead Data Engineer you will own our data infrastructure. You will leading the evolution of our architecture to handle more data, more types of data, more client data delivery mechanisms and more internal users. You will be responsible for everything between researching technologies to coding to debugging production systems. Our data pipeline processes 2.5+ TB/day and that is built using Spark, AWS, Python, Airflow and Rails. Below is a list of the major initiatives that you will be helping with.
- Analyze and integrate new data sources: receipt data, POS data, product review data
- Move our surveying pipeline from batch to real-time
- Work closely with our data science team to build model training platforms
- Build client report generation and delivery system
- Scale and optimize our analytics database cluster
What do we look for?
Engineers who thrive at Sense360 have a few key traits:
- Driven to be the best version of themselves on a daily basis
- Care about their work, their team and their company
- Expect their ideas to be challenged because they believe that the best ideas can come from anywhere
- View feedback as a gift that they give and receive
- Positive and interesting
What qualifications do you need?
- 3+ years experience architecting distributed data processing systems (e.g. Spark, Storm, Presto, Samza, Flink, etc) processing TB's of data
- Proficient in Python, Java or Scala
- 3+ years experience working in a startup
- evangelist for CS, data and testing best practices
- built real-time data processing systems a plus
Willing to relocate if already within the US and sponsor visas.