Linux Interview - Just when you thought you knew it all.

I have recently found myself in a search for the perfect job. What might that be? Well, I would say a job where I get to operate 200 foot tall giant robots in some cool inter-galactic robo-monkey-gladiator-swordfight type sport. I'm reasonably sure that I won't be running into one of those anytime soon. So, failing that, I think the perfect job is a Linux Systems Administration gig, working for a company that is doing something really cool, really needed, and really open. I would want to be surrounded by a group of people who are there for the same reason I am: make those systems run at peak performance, find new ways of achieving those goals, and scaling them out as needed. I need the people around me to have ideas to share, and be happy to hear mine.

Well, that's enough of the "What I want in a perfect world". The meat of this article is really to list out some of the most common questions I have been asked along this journey (which I am still on as of this writing). There are things we, as Linux Administrators, do every day. Using ls, ps, df, du, who, w, last, top, vmstat, cat, blah blah blah... but there are things that we know, we just get caught offguard having to describe them to people who ask. Hopefully this list will help you along with your search.

Warning: If you do not know a certain level of Linux, Unix, BSD, or shell scripting, this is not a cheatsheet for you. I make a lot of assumptions here, and hope that you will go off and research how to write scripts on your own.

Standard questions/exercises:

• What is an inode? An inode is a data structure holding information about filesin a file system. There is an inode for each file, and a file is uniquely identified by the file system on which it resides and its inode number on that system.

• Write a script that does "X". Using the shell you are most familiar with will help you here. We all execute oneliners every day, several times a day. Simply string those commands together in a most unelegant manner - it should not matter to the interviewer; obviously you know what you are doing. Example - a script that counts up to 10 without using count:

#!/bin/bash
ten=0
while [ $ten -le 10 ]
do
echo $ten
ten=$((ten+1))
done

• What tool would you use to detect the OS of a remote machine, and what is the command line to do it? Tricky. Could use rsh in an insecure environment and do: rsh -l username hostname "uname -r" ... or, the equivelent in ssh, assuming ssh keys are in place; ssh hostname uname -r ... or, and this would be my first try - install nmap and do a sudo nmap -O -v hostname
• Show the failed attempts to login via ssh. It's dirty, but it's a first pass that gives an indication of who tried to pop the top off your box: sudo grep Failed /var/log/messages
• What system call signals are available in linux? If you can memorize and ramble off 64 available signals, and their names.. maybe you should get out more. I rely on "man kill" and "kill -l" (thats a lowercase "L", not a ONE)
  
• How do I find what network connections are active, and what program is using them? netstat -ap -or- netstat -anp (if there is a problem with name resolution, the "n" will use the ip rather than perform a hostname lookup)
• What are setuid/setgid in relation to file permissions? Yikes. I love the short questions with the long answers. Don't stumble, mutter, or ramble here - it's easy to do. setuid on an executable file allows a user to run that program with the permissions of the owner of the file rather than the person who executed it. setguid is the same thing; only with the permission of the group the file is a member of.
• What is init? Init is the program that runs after the kernel loads that spawns all other processes. It runs as PID 1, and on RH, CentOS, Fedora, looks to the rc#.d/* files for stuff to launch. The # is the current runlevel, and the * is anything executable that is in there - either symlinked by hand or using something like chkconfig --level ##### PROGRAM on/off
• What are the standard runlevels? This is totally tricky, there's only 3 runlevels that are the same across all UNIXs/Linux's/BSD's - 0(Halt), 1(Single user), and 6(Reboot). Typical of the newish modified systems (Redhat and its derivitives), standard runlevels 3 (no X), and 5 (X) are default - set in inittab.

I need to get to a few things.. I didn't realize how long this list was going to end up being..