Perform a migration from Amazon Redshift to Databricks by moving tables, views, and stored procedures; build and maintain Databricks pipelines; work with AWS services (S3, Glue, Athena); support data lake architecture and apply data governance (lineage, catalogs, RBAC); collaborate with stakeholders. 5-6 month engagement.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Data Engineer to support a strategic data platform migration, moving tables, views, and stored procedures from Amazon Redshift to Databricks. You will build and support Databricks pipelines and jobs, work with AWS services including S3, Glue, and Athena, and apply data governance practices including data lineage, cataloging, and role-based access controls throughout the migration. The role requires 3–5 years of data engineering experience. This is a 5–6 month project engagement.
WHAT YOU WILL DO
- Migrate tables, views, and stored procedures from Amazon Redshift to Databricks.
- Build and support Databricks pipelines and jobs.
- Work with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Support data migration activities within a modern data lake architecture.
- Apply data governance and data management practices throughout the migration.
- Collaborate with project stakeholders to support migration activities.
MUST HAVES
- 3–5 years of experience as a Data Engineer.
- Hands-on experience with Databricks.
- Experience with Amazon Redshift.
- Strong SQL skills.
- Experience with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Experience building and supporting Databricks pipelines and jobs.
- Understanding of data lake architectures.
- Familiarity with data governance and data management practices, including data lineage, data catalogs, and role-based access controls (RBAC).
- Upper-intermediate English level
NICE TO HAVES
- Experience with Databricks Lakebridge transpiling capabilities.
PERKS AND BENEFITS
- Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
- Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
- Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Well-being & support: access local well-being programs and people-focused support tailored to your location
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead Samsara Mexico communications with a 360° strategy across external media, internal comms, and talent brand. Serve as primary media contact, develop localized storylines, manage executive visibility, support customer storytelling, run crisis comms, manage agencies, and measure program impact in coordination with marketing, legal, and global comms.
Top Skills:
LinkedIn
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
The Account Executive will engage with small to medium customers, focusing on IoT sales. Responsibilities include prospecting, closing deals, and building relationships. Requires high-volume cold calling and a collaborative approach.
Top Skills:
SFDC
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Manage and execute knowledge development projects for Global Support: implement knowledge processes and standards, collaborate with SMEs, produce AI-optimized content, monitor KPIs, and drive improvements in knowledge technology and operations.
Top Skills:
AgileAIAi ModelsConfluenceCrm SystemsJIRAKnowledge Management SystemsScrumTableauTicketing SystemsZendesk
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
