Designs and develops AWS data pipelines and automated document-processing workflows for structured and unstructured data. Builds chunking, embedding, vector indexing, semantic search, knowledge bases, and RAG integrations. Develops AI agents using Amazon Bedrock and Claude technologies, deploys them through AgentCore, and maintains GitHub Actions CI/CD pipelines. Optimizes systems for scalability, reliability, security, and performance. Requires remote work in western U.S. time zones and travel to client workshops or meetings.
Sr. Data Engineer (GenAI & AWS) Remote/Hybrid with Travel
The key is time zone coverage and so we can expand our search across the West Coast - remote/hybrid but necessarily located in the western time zones. You must be comfortable to travel to the client site for workshops/meetings.
- Senior Data Engineer (GenAI / AWS)
Overall experience: 12-20+ years Cloud Platform: AWS Primary Language: Python
Key Responsibilities
- Design and develop data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend.
- Build automated document-processing workflows for extracting, transforming, and processing structured and unstructured data.
- Implement document chunking, embeddings, vector indexing, and semantic search.
- Develop and maintain vector databases and knowledge bases to support Retrieval-Augmented Generation (RAG).
- Integrate knowledge bases with AI agents for contextual information retrieval.
- Build AI agents using Claude Agent SDK and deploy them on Amazon Bedrock AgentCore Runtime.
- Develop CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments.
- Optimize data pipelines for scalability, reliability, security, and performance.
Must-have vs. good-to-have skills
Must-have
Good-to-have
Python, SQL
TypeScript
AWS Step Functions, Lambda, S3
AWS Glue
Amazon Textract and Comprehend
Document AI / OCR
Vector databases and embeddings
OpenSearch, Pinecone, pgvector
Document chunking and indexing
Advanced retrieval and reranking
RAG and knowledge-base integration
LangChain / LlamaIndex
Amazon Bedrock
Claude Agent SDK
GitHub Actions CI/CD
Terraform / AWS CDK
AWS IAM and API integrations
AgentCore Runtime
Important: If this individual must personally build and deploy the Claude agents, move Claude Agent SDK and AgentCore Runtime into the must-have category.
Similar Jobs
Digital Media • Social Media
The Media Sales Rep will drive revenue through advertising and sponsorship, develop business partnerships, and manage a sales pipeline while collaborating cross-functionally and executing successful campaigns.
Top Skills:
Digital AdvertisingSocial Media
Healthtech
The Channel Leader drives Zerigo Health's channel strategy through partner relationships, facilitating sales, managing account health, and ensuring growth.
Top Skills:
Ai-Driven Personalized Care PlansFda-Cleared Smart Home Device
Other • Professional Services
Own complex software systems and critical workflows end-to-end while ensuring reliability, performance, security, and maintainability. Lead architectural decisions, mentor engineers, manage incident response, improve system health, and communicate effectively with technical and non-technical stakeholders. The role requires extensive Ruby on Rails, AWS or equivalent cloud infrastructure, Linux production operations, frontend development, relational databases, testing, CI/CD, and observability experience.
Top Skills:
AWSCi/CdIso 27001JavaScriptJIRALinuxMySQLNginxNistOauthReactRedisRuby On RailsSAMLScrumSoc 2Stimulus
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
