Linux Administrator (Big Data/Hadoop)
You will be responsible for the care and feeding of our big data installation as well as the short- and long-term capacity planning based on our future growth plans.
- Collaborate with all other team members to ensure our big data platform is prepared to host new services and workloads as they are developed.
- Work with the development team to understand architectural work to assist with evaluation, decision-making and sequencing of the key technological infrastructures to support your product.
- Provide consistent performance evaluation and monitoring for the production Hadoop instance.
- Participate in continuous integration, production release management, solution validation.
- Experience with installation and support of Enterprise Hadoop clusters (cloud and local datacenter).
- Experience with administering physical clusters.
- Experience with designing, capacity arrangement, cluster set up, security, performance fine-tuning, resource management, monitoring, troubleshooting, structure planning, scaling and administration of Hadoop/Spark clusters.
- Experience with deploying and administering various cluster services like Zookeeper, YARN, HDFS, HBase, Hive, Oozie, Map Reduce, Kafka, Spark, Storm etc.
- Good networking knowledge (will be dealing with a lot of applications/services on top of Linux and networking).
- Solid Linux Administration Experience
- Solid working experience of Linux security concerns in both cloud and bare metal hosting environments.
- Experience with major cloud vendors like AWS and Azure and administering clusters in the cloud.
- Physical and virtual networking experience.
NICE TO HAVE:
- Knowledge of server hardware – storage, compute and networking components.
- Knowledge of administering in-memory SQL-on-Hadoop engines like Impala, Hive-on-Spark etc.
- Experience with administering Spark clusters and troubleshooting resource management issues.
- Familiarity with deploying Notebook environments like Zeppelin, Jupyter etc.
- Familiarity with deploying and administering BI platforms like Tableau Server.