Systems Engineer II (Remote)

NRTC is Member Driven and Technology Focused – providing solutions that help our 1,500 electric and telephone members bring all the advantages of today’s evolving technology to rural America. NRTC’s solutions are developed specifically to meet the needs of rural utilities and their customers, and include integrated broadband, managed services, smart grid, mobile solutions. We are a dynamic, growing cooperative with more than 800 employees operating out of eight office locations.

Job Summary 

NRTC System Engineers handle the installation, maintenance, and troubleshooting of servers and applications used by NRTC members to support their ISP networks as well as developing, maintaining, and implementing tools to import member end user data into NRTC’s systems. The System Engineers works with a team to remotely administer NRTC servers on-site at member locations providing services such as caching DNS, DHCP, network speed tests, and radius authentication. As a cooperative, NRTC members and the systems teamwork in unison to support and grow member ISP networks.

Responsibilities 

• Develop, maintain, and implement tools and scripts to import new member’s end user customer data into NRTC systems in a secure and efficient manner

• Manage NRTC’s remote Network Utility Servers deployed in member network to support ISP services

• Configure and manage DHCP IPv4 and IPv6 IP systems for ISP network including configuration of DHCP options for advanced logging and analysis

• Configure and manage radius authentication infrastructure for member ISP services including BNG and PPPoE configurations

• Provide members with best practices for supported systems

• Onboarding and management of member authoritative and caching DNS services

• Onboarding of member email hosting, email client configuration, email platform management 

• Provide support for engineering and professional services

• Monitor and maintain system uptime, security, virtual environments, and disaster recovery

• Support team members with the installation and configuration of Linux systems including updating and maintaining automatic installation tools

• Implement and support Linux environments for NRTC members, including troubleshooting and problem resolution for operating systems, application configuration, and hardware issues

• Troubleshoot and diagnose issues reported by members and internal technical support teams

• Work independently or as part of a team to diagnose and troubleshoot issues

• Exercises good judgment when seeking assistance to solve or troubleshoot most issues

• Support NRTC member change requests and conversions

• Create scripts and processes for automation of repetitive tasks

• Create, maintain up to date documentation on specific member configuration and troubleshooting steps

• Perform change management activities, including code upgrades to production systems

• Mentor teammates on current tools, processes, and changes to member system configurations

• Stay current on new technologies

• Other duties and responsibilities as assigned

Qualifications 

Knowledge, Skills, and Abilities 

• Advanced level knowledge of all aspects of administration for Linux operating systems including installation, management, application deployment, database management, and security
  Thorough understanding of DHCP, authoritative and caching DNS, AAA IP management (radius)

• Thorough understanding of email delivery, protocols, troubleshooting, and server applications (Postfix, Dovecot, Zimbra)

• Thorough understanding of container management (Docker, Kubernetes, Podman, etc.)

• Thorough understanding and experience with version control software (GIT, SVN, CVS, etc.)

• Strong experience troubleshooting, debugging and root cause analysis of system outages and slowdowns

• Experience managing virtualization technologies such as VMware, Xen and Hyper-V.

• Experience integrating with API interfaces

• Experience with major incident management

• Able to automate regular tasks using common scripting languages (Bash, Python, PHP, or Perl) 

• Able to understand and update existing scripts in common scripting languages (Bash, Python, PHP, or Perl) 

• Ability to communicate and solve problems efficiently and effectively with identified customers and IT personnel to ensure the implementation of accurate processes

• Experience administrating authoritative and caching DNS servers including advanced configurations such as split DNS

• Experience obtaining and deploying SSL certificates using automated tools such as Let’s Encrypt certbot

• Experience with monitoring software (Zabbix, Nagios, PRTG)

• Experience with automation tools (Ansible, Chef, Salt, etc.)

• Advanced ability to be proactive, detail-oriented, and self-motivated

• Strong teamwork skills

Minimum Education and Experience  

• Bachelor's degree in computer science or related field

• Five (5) years of experience managing Linux server infrastructure, including troubleshooting, physical installations, modifications, replacements and configuration changes and log analysis

• Linux Foundation Certified System Administrator or equivalent desired 

An equivalent combination of education and experience may be considered.

Physical Demands 

• Work is typically performed in an office and/or remote setting

• Participation in a weekly on-call rotation to provide 24x7 support for ISP services managed by NRTC

• This role will require the ability to team lift at least 80 pounds

• Minimum domestic travel may be possible, approximately 5%

Disclaimer: The above job description summary is intended to describe the general nature and level of work being performed by individuals assigned to this position. It is not intended to be an exhaustive list of required responsibilities, duties, and skills. The order in which responsibilities, duties, and skills are listed is not significant.  

#zr