Modernize legacy IBM DataStage ETL/ELT workloads by re-engineering them as scalable Databricks and Spark pipelines on AWS. Responsibilities include migration assessment, pipeline development, testing, data validation, CI/CD automation, production cutover, stabilization, legacy decommissioning, documentation, and technical training. The role performs parity, unit, functional, UAT, regression, and performance testing while supporting application redeployment and operational readiness.
This is a remote position.
Onsite/Hybrid/Remote: Remote
Duration: 12 months
Rate Range: $68 in W2
Work Authorization: GC and US Citizens Only
Must Have:
- IBM DataStage development
- ETL/ELT migration and modernization
- Databricks and Apache Spark on AWS
- PySpark, Python, SQL, and Shell scripting
- Delta Lake, Unity Catalog, and Photon
- AWS Glue, Lambda, and Redshift
- CI/CD and automated ETL testing
- PyTest and XML/JSON parsing
Responsibilities:
- Assess existing ETL/ELT jobs, databases, and data warehouses for migration readiness.
- Re-engineer legacy IBM DataStage jobs using Databricks Spark on AWS.
- Develop, test, and deploy scalable cloud-based data pipelines.
- Perform parity checks, unit testing, functional testing, UAT, regression testing, and performance testing.
- Validate schemas, data quality, pipeline performance, and migration accuracy.
- Build CI/CD processes for automated development, testing, and deployment.
- Support migration cutover, production deployment, and hypercare.
- Coordinate application redeployment and end-to-end testing.
- Decommission legacy DataStage jobs and related artifacts.
- Prepare technical documentation and operational readiness materials.
- Conduct knowledge-transfer and training sessions for engineering and business teams.
Qualifications:
- Strong hands-on experience developing and maintaining IBM DataStage jobs.
- Proven experience modernizing large-scale ETL/ELT and data warehouse workloads.
- Experience building Databricks and Spark pipelines in an AWS environment.
- Advanced programming skills in PySpark, Python, SQL, and Shell scripting.
- Experience with Delta Lake, Unity Catalog, and Databricks Photon.
- Experience with GitLab or Azure DevOps for CI/CD.
- Knowledge of automated testing frameworks and ETL validation methods.
- Experience supporting production cutovers, stabilization, and application teams.
Nice to Have:
- Experience with AWS Glue, Lambda, and Redshift.
- Experience with JIRA and Agile delivery.
- Knowledge of data governance and cloud security standards.
- Experience creating operational documentation and delivering technical training.
aKube Inc Los Angeles, California, USA Office
Los Angeles, California, United States
Similar Jobs
Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
Develop and manage a branch-based wealth management practice by conducting client discoveries, creating tailored financial plans, driving new business development and referrals, conducting proactive outreach, and coordinating with internal specialists for investment, insurance, estate, and financial planning services.
Artificial Intelligence • Marketing Tech • Software
Own new-client implementations for Movable Ink’s personalization platforms, serving as the project manager and product expert. Responsibilities include developing project plans, coordinating clients and internal teams, designing technical solutions, configuring and QAing campaigns, delivering platform training, troubleshooting issues, and ensuring launches achieve client goals and KPIs. The role requires expertise in mobile messaging, digital marketing, web technologies, data infrastructure, and client or project management.
Top Skills:
APIsCloud Data WarehousesData LakesEmail Service ProvidersETLHTMLJavaScriptMmsPush NotificationsSms
Automotive • Big Data • Information Technology • Robotics • Software • Transportation • Manufacturing
Leads the strategy and execution of AI model performance and on-vehicle inference for autonomous driving. Responsibilities include managing engineering leaders, setting performance budgets, optimizing models and GPU systems, establishing benchmarking and profiling practices, detecting regressions, building performance dashboards, and guiding cross-functional architecture and production decisions. The role also oversees hiring, coaching, staffing, KPIs, technical leadership development, and trade-offs across latency, memory, throughput, accuracy, power, and numerical parity.
Top Skills:
Autonomous DrivingBenchmarkingC++Computer VisionCudaEdge HardwareGpu ProfilingGpu SystemsInference RuntimesKernel OptimizationKnowledge DistillationMachine LearningMemory OptimizationModel OptimizationNvidia Nsight ComputeNvidia Nsight SystemsPerformance DashboardsPruningPythonPyTorchPytorch ProfilerQuantizationReduced PrecisionRoboticsTelemetryTensorrt
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
