back to job search

Linux Administrator ( Big Data)

Linux Administrator (Big Data/Hadoop)

You will be responsible for the care and feeding of our big data installation as well as the short- and long-term capacity planning based on our future growth plans.

RESPONSIBILITIES:

  • Collaborate with all other team members to ensure our big data platform is prepared to host new services and workloads as they are developed.
  • Work with the development team to understand architectural work to assist with evaluation, decision-making and sequencing of the key technological infrastructures to support your product.
  • Provide consistent performance evaluation and monitoring for the production Hadoop instance.
  • Participate in continuous integration, production release management, solution validation.

REQUIREMENTS:

  • Experience with installation and support of Enterprise Hadoop clusters (cloud and local datacenter).
  • Experience with administering physical clusters.
  • Experience with designing, capacity arrangement, cluster set up, security, performance fine-tuning, resource management, monitoring, troubleshooting, structure planning, scaling and administration of Hadoop/Spark clusters.
  • Experience with deploying and administering various cluster services like Zookeeper, YARN, HDFS, HBase, Hive, Oozie, Map Reduce, Kafka, Spark, Storm etc.
  • Good networking knowledge (will be dealing with a lot of applications/services on top of Linux and networking).
  • Solid Linux Administration Experience
  • Solid working experience of Linux security concerns in both cloud and bare metal hosting environments.
  • Experience with major cloud vendors like AWS and Azure and administering clusters in the cloud.
  • Physical and virtual networking experience.

NICE TO HAVE:

  • Knowledge of server hardware – storage, compute and networking components.
  • Knowledge of administering in-memory SQL-on-Hadoop engines like Impala, Hive-on-Spark etc.
  • Experience with administering Spark clusters and troubleshooting resource management issues.
  • Familiarity with deploying Notebook environments like Zeppelin, Jupyter etc.
  • Familiarity with deploying and administering BI platforms like Tableau Server.