Senior Data Engineer
CreatorIQ is looking for a Senior Data Engineer to join our growing analytics team. You’ll be responsible for designing and building a new database and data pipeline architecture. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support our data analysts, data scientists and data architects on data initiatives and will ensure optimal data architecture is consistent throughout ongoing projects. They must be self-directed, take initiative, and be a positive driving force that works well with stakeholders and members of the team to support the data team’s needs. The right candidate will be excited by the prospect of re-designing our company’s data architecture to support our next generation of products and data initiatives.
Who we are:
CreatorIQ is the leading cloud solution for global enterprises to manage and optimize influencer campaigns at scale. Our global client roster includes the likes of Mattel, Disney, Tiffany & Co, Airbnb, Ralph Lauren, CVS, Amazon, Dell, Edelman, Creative Artists Agency, and many more.
What you'll do:
- Develop, construct, test, and maintain data architectures
- Ensure architectures will support the business and analytics requirements. Work together with the Data Scientists and Data Analysts to ensure that architecture is optimized for analytics purposes.
- Work with stakeholders including the Analytics team to assist with data-related technical issues and support their data infrastructure needs.
- Discover opportunities for data acquisition
- Develop data set processes for modeling, mining, and production
- Assist with deploying large scale data models to run in real-time in the software
- Recommend ways to improve data reliability, efficiency, and accuracy
- Ensure the data is always accurate, consistent, useful, and concise
- Build out a new database that supports analytics needs that the data scientist can use to drive insights.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
Who you are:
- A team player that is responsive, a problem-solver, and good at working with other people
- Experience building and optimizing big data data pipelines, architectures and data sets.
- 5+ years of experience in a Data Engineer role
- Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
- Deep proficiency of MySQL and Python. Java, C++, Scala a plus
- Experience with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift