Hadoop System Administrator

Job brief

Deploying a Hadoop cluster, maintaining a Hadoop cluster, adding and removing nodes using monitoring tools like Cloudera Manager, configuring the NameNode high availability, and keeping a track of all the running Hadoop jobs.

Requirements

• Maintain and administer computer networks and related computing environments including computer hardware, systems software, applications software, and all configurations. 
• Perform data backups and disaster recovery operations. 
• Diagnose, troubleshoot, and resolve hardware, software, or other network and system problems, and replace defective components when necessary.  
• Experience in importing the real-time data to Hadoop using Flume, Sqoop, Kafka, and implemented the Oozie job. ExperienceSchedule Recurring Hadoop Jobs with Apache Oozie.
• Install OS and administrated Hadoop stack withCDH5 (with YARN) Cloudera Distribution including configuration management, monitoring, debugging, and performance tuning Scripting   Hadoop package installation and configuration to support fully-automated deployments.
• Configure, monitor, and maintain email applications or virus protection software. 
• Operate master consoles to monitor the performance of computer systems and networks, and to coordinate computer network access and use. 
• Monitor network performance to determine whether adjustments need to be made, and to determine where changes will need to be made in the future. 
• Plan, coordinate, and implement network security measures to protect data, software, and hardware. 
• Analyze equipment performance records to determine the need for repair or replacement. 
• Confer with network users about how to solve existing system problems. 
• Recommend changes to improve systems and network configurations, and determine hardware or software requirements related to such changes. 
• Design, configure and test computer hardware, networking software, and operating system software. 
• Perform routine network startup and shutdown procedures, and maintain control records. 
• Load computer tapes and disks, and install software and printer paper or forms. 
• Maintain logs related to network functions, as well as maintenance and repair records. 
• Gather data pertaining to customer needs, and use the information to identify, predict, interpret, and evaluate the system and network requirements. 
• Coordinate with vendors and with company personnel to facilitate purchases. 
• Implement and provide technical support for voice services and equipment, such as private branch exchange, voice mail system, and telecom system. 
• Maintain an inventory of parts for emergency repairs. 
• Research new technologies by attending seminars, reading trade articles, or taking classes, and implement or recommend the implementation of new technologies.