Design and implement data integration and data warehouse solutions using big data technologies. Build cost-efficient, performant cloud data pipelines, apply data modeling and machine learning knowledge, and implement real-time streaming and aggregation patterns.
Role Big Data Engineer Responsibilities Implement Data integration and Data Warehouse based solutions using Big Data Technologies. Should be highly proficient in the use of Big Data / Open Source Technologies and standard techniques of Data Integration, Data Manipulation with hands-on contribution. Should be able to develop cost efficient and performant data pipelines in the cloud platform In-depth understanding of modern big data technology, including Data modeling and machine learning skills Knowledge of real time data streaming and aggregation architectural patterns and practice Essential Skills: 3+ Years hands on knowledge on SQL as well as SQL/NoSQL databases Proficient in programming languages such as Python and PySpark/Scala/Java
Similar Jobs
Information Technology • Consulting
Design and implement data integration and data warehouse solutions using Big Data and open-source technologies. Build cost-efficient, performant cloud data pipelines, apply data modeling and ML concepts, and implement real-time streaming and aggregation patterns. Hands-on SQL/NoSQL and programming experience required.
Top Skills:
Big DataCloudData IntegrationData ModelingData WarehouseJavaMachine LearningNoSQLPysparkPythonReal-Time Data StreamingScalaSQL
Insurance
Lead enterprise data architecture and platform engineering efforts using Databricks Lakehouse. Design scalable cloud-native data platforms, implement medallion architectures, enable data governance and AI/ML workloads, build automation and CI/CD for Databricks, and mentor teams while collaborating with stakeholders on strategy and delivery.
Top Skills:
SparkAutomated TestingAWSAzureCi/CdDatabricksDatabricks LakehouseDelta LakeFhirGoogle Cloud PlatformHedisHl7Infrastructure As CodeMedallion ArchitecturePysparkPythonSQLX12
Insurance
Designs and implements big-data data management solutions (Cloudera/Hortonworks/Databricks). Builds and automates reporting and analytics using SQL and Databricks, optimizes queries, manages ETL, integrates multiple data sources, enforces data governance, and supports payment integrity reporting requirements.
Top Skills:
ClouderaData WarehouseDatabricksDatabricks SqlETLHbaseHiveHortonworksImpalaJavaKafkaKylinMemsqlNoSQLPhoenixPolybasePrestoPythonRestScalaSoapSparkSpark-StreamingSQLStormTalend
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
