Senior PySpark developer responsible for building and maintaining large-scale ETL and data engineering pipelines using PySpark/Spark, Hadoop and related big-data tools. Work with SQL/NoSQL databases, cloud platforms, data lakes/warehouses, and support CI/CD and containerization. Collaborate on systems analysis, project delivery, and meet deadlines under pressure.
Requirements and Qualifications:
- 8+ years of relevant experience in the Data Engineering, ApacheSpark
Python, Pyspark, Unix, Hadoop Must have any 2 skill from below
( DB : Hive, SQL, MongoDB
GTH - Airflow, Gen-AI, ECS)
- Strong proficiency in PySpark and Apache Spark.
- Solid experience with SQL, NoSQL databases (e.g., Hive, HBase, Cassandra).
- Hands-on experience with big data frameworks (Hadoop, Kafka, etc.).
- Expertise in cloud platforms (AWS, Azure, or GCP).
- Proficiency in Python programming and data manipulation.
- Knowledge of ETL tools, data lakes, and data warehouses.
- Experience in CI/CD, containerization (Docker, Kubernetes) is a plus.
- Strong problem-solving and analytical skills.
- Excellent communication and teamwork skills.
- Experience in systems analysis and programming of software applications
- Experience in managing and implementing successful projects
- Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Compensation, Benefits and Duration
Minimum Compensation: USD 48,000
Maximum Compensation: USD 170,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Similar Jobs
Information Technology
Sell CDW Security services and products across the full sales lifecycle. Develop strategic account plans, build relationships with enterprise clients and partners, manage pipeline and forecasts in Salesforce, coordinate cross-functional resources, mentor junior sellers, represent CDW at events, and achieve bookings, revenue, and profit targets while tracking market trends and driving customer security program adoption.
Top Skills:
AICloud SecurityData PrivacyEmerging Security TechnologiesIdentity And Access ManagementPhysical SecuritySalesforceSecure InfrastructureThird-Party Risk ManagementThreat And Vulnerability Management
Information Technology
Sell CDW Security services and products, manage full sales lifecycle, develop strategic account plans, build and grow client and partner relationships, coordinate internal resources, maintain Salesforce pipeline and forecasts, mentor junior sellers, represent CDW at industry events, and meet bookings, revenue, and profit targets.
Top Skills:
AICloud SecurityCyber Workforce DevelopmentEmerging Security TechnologiesIdentity And Access ManagementPhysical SecurityPrivacyRisk And ComplianceSalesforceSecure InfrastructureThird-Party Risk ManagementThreat And Vulnerability Management
Information Technology
Sell CDW Security services and products through full sales lifecycle management: strategic account planning, opportunity qualification, relationship building, partnering with services and OEMs, pipeline forecasting, and closing. Mentor junior team members, engage partners and marketing, maintain Salesforce, represent CDW at industry events, and drive revenue, bookings, and gross profit targets.
Top Skills:
Salesforce
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
