Software Engineer - Big Data
Beyond Limits is seeking a creative and confident big data engineer to join our team. As a data engineer, you will build data pipelines at scale and develop models for near real-time decisions and predictions. Your responsibilities include collecting, parsing, analyzing and visualizing large sets of data. You will work closely with cross departmental resources to meet requirements imposed by both Data Scientists and Software Engineers. A strong candidate will have solid command of SQL and ETL operations in addition to very strong analytical and critical thinking skills.
- BS/MS in Computer Science, Computer Engineering, Physics or related technical equivalent.
- Extensive knowledge in at least one programming (Go, Scala, or Java) and one scripting language (Python is preferred).
- 2+ years of experience building and maintaining large scale analytic systems.
- 2+ years of programming experience with Hadoop and/or Spark.
- Operational knowledge of NoSQL or RDBMS platforms (Postgres/MySQL and Cassandra/ElasticSearch).
- Extensive knowledge of the Hadoop processing stack including but not limited to HDFS, Yarn, Hive and Hbase.
- Knowledge of streaming data processing solutions (Apache Kafka and Spark Streaming)
- Experience designing efficient and robust ETL workflows.
- A great team player with excellent communication and documentation skills.
- Knowledge of the Berkeley Data Analytics Stack (BDAS)
- Experience with AWS and VMWare cloud environments
- Knowledge of analytic and statistical techniques using Python, PySpark, Pandas and related tools
- Docker and Kubernetes for cloud orchestration and deployment
About Beyond Limits
Beyond Limits is a pioneering Artificial Intelligence company with a technology legacy in Space Exploration. A spinoff of NASA's Jet Propulsion Lab and Caltech, Beyond Limits provides advanced intelligence solutions that go far beyond conventional AI. Our cognitive computing technology mimics human thought processes and provides autonomous reasoning to aid human-like decision-making.