Design, build, and maintain AWS-based data pipelines and ETL processes using Glue, Spark/PySpark and EMR. Integrate multiple data sources into data lakes and Redshift, implement DevOps CI/CD practices, and work with Kafka, Hudi, and various AWS storage and analytics services.
This is a remote position.
- 3+ Years of Development / Data Engineering Experience
- 2+ Years Developing with AWS Cloud solutions
- 2+ Years Developing in Spark/PySpark
- 2+ Years Developing AWS Glue ETL
- 2 + Years AWS Storage models (S3 and DynamoDB)
Some hands-on developing with on-prem ETL tooling (Ab Initio preferred, Informatica)
Requirements
- Bachelor of degree in computer or any related field.
- Minimum of 12+ years’ experience in the said role.
- Strong experience in AWS GLUE, EMR, Hudi, experience extracting data from multiple sources & loading data into Data Lakes, AWS Redshift
- Experience working with AWS Elastic search, RDS, PostgreSQL preferred
- Experience w/ AWS services such as Lambda, EMR, SNS/SQS, Event Bridge, Lake Formation & Athena
- Experience w/ integrating applications/systems (data producers) with Enterprise Kafka topics (Confluent Kafka integration w/ AWS S3, Redshift)
- Experience in Requirements Analysis, Data Analysis, Application Design, Application Development & Integration Testing
- Working knowledge of on-prem Extraction, Transformation, Cleansing and Loading methodology and principles
- Experience implementing and adding to DevOps principles (GitLab, maintaining CI/CD pipelines)
- Experience in Java svcs is nice to have
N/A:- Please kindly note that this role is for USC and GC Visa holders only.
Interested and qualified candidates should send their resume to [email protected] using the job title as the subject of the mail.
Similar Jobs
Artificial Intelligence • Big Data • Machine Learning
Design, build, and maintain scalable AWS-based data pipelines, data lakes and warehouses. Implement data modeling and warehousing for Redshift/Athena, ensure data quality, optimize queries, and develop production-grade Amazon QuickSight dashboards with security (RLS), performance tuning, and stakeholder-driven reporting.
Top Skills:
Amazon EmrAmazon QuicksightApache AirflowAws AthenaAws GlueAws LambdaAws RedshiftAws S3Aws Step FunctionsDirect QueryPower BIPysparkPythonQlikRow-Level SecuritySpiceSQLTableau
Information Technology • Software
Own and scale AWS OpenSearch production infrastructure and PostgreSQL-to-OpenSearch data pipelines. Design indices and mappings, optimize queries and bulk indexing, monitor cluster health, implement ingestion reliability, perform reindexing/backfills, build reconciliation and recovery processes, and collaborate with backend, DB, and product teams on searchable schemas and runbooks.
Top Skills:
Amazon S3Amazon SqsAws DmsAws EcsAws EksAws LambdaAws Opensearch ServiceClickhouseCloudwatchElasticsearchGoIamJavaOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonTerraformTinybird
Information Technology • Software
Own and scale AWS OpenSearch infrastructure and PostgreSQL-to-OpenSearch pipelines. Design indices, mappings, analyzers, and retention policies. Optimize queries and bulk indexing, monitor cluster health, implement retries/idempotency, backfills, reconciliation, and disaster recovery. Build dashboards, alerts, runbooks, and participate in incident response while collaborating with backend, DB, infra, and product teams.
Top Skills:
Aws DmsAws LambdaAws Opensearch ServiceClickhouseCloudwatchEcsEksElasticsearchGoIamJavaJvmOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonS3SqsTerraformTinybird
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
