Recently, I was looking for some (mostly) Linux interview questions, and frankly I was disappointed. The top 10-20 google hits (mostly from Indian sites) were mediocre, boring crap, and even worse a mere copycat of each other, without a single, truly genuine question.

Most of them share questions like “What’s the difference between BASH and DOS?“, or “Unix and Linux“. Or “What’s a microprocessor?” Really? But my favourite bullshit question found in every one of them is “What are the different modes of Network bonding in Linux?“

Ok, perhaps I felt pissed off, because I didn’t use bonding in Linux, and I didn’t know the answer. Anyway, I’ve compiled my top 20 or so Linux interview questions, with some brief hints what I’d like to hear in the answers.

1. Describe swap partition

Please mention virtual memory (physical memory + swap), paging out to swap, that it’s much slower, than RAM.

2. How do you make ssh daemon more secure?

A few tips:

• Use protocol version 2, which is the default for any recent distros
• Use firewalls, iptables, (hosts.allow and deny files, anyone?) whatever to limit access to the daemon
• Disable password based authentication, allow ssh keys only
• Disable root login, if possible
• Some people suggest to use a high port other than 22 to evade scans
• Use safer MACs (=message authentication code), ciphers and key exchange algorithms
• Disable port forwarding, if it makes sense

3. What’s an inode?

An inode is a structure holding all info about the file except the filename, eg. ownership, permissions, timestamps (last access, modification, …), size, file type, and link count (see the next question). Bonus for man 2 stat.

4. Why can’t you make a hard link between partition?

A hard link is merely a name for the given file in the directory entries. So you can assign many names for the same file, pointing to the same inode. However, inodes are unique only on the same partition, that’s why you’ll get a similar error:

sj@thorium:~$ ln /boot/vmlinuz-4.15.0-52-generic aaa
ln: failed to create hard link ‘aaa’ => ‘/boot/vmlinuz-4.15.0-52-generic’: Invalid cross-device link

5. Why would anyone use LVM?

Let’s say you have a database server with a single disk with 100 GB. Sooner or later you find it almost filled. Then you need to extend /var/lib/mysql (supposing it’s a mysql server). The simplest solution is to create a logical volume (when installing the server), and you can add another disk to the same volume group, and extend the logical volume, and increase the partition size.

6. Describe ACL, what commands will you use to set and check ACL on a file?

Standard unix permissions consist of ‘owner’, ‘group’ and ‘other’ (or world) categories. Sometime you need a finer, more granular approach to set permissions. You may use setfacl and getfacl commands. Also ls command will display ‘+’ sign to indicate that the given file has ACL set.

7. An (CLI) application segfaults. How do you start troubleshooting it?

Start it using gdb, and check its output.

8. How do you check if an application leaks memory?

Start the application using valgrind.

9. Assuming bash and uid=0, how would you prevent rm -rf $A/bin to misbehave if you forgot to set A=/path/to/somewhere?

Use “set -o nounset”

10. What kind of signals can we send via the kill command?

TERM: terminate the process gracefully, ie. allowing it to shutdown some descriptors, free memory, say goodbye to users, etc.
KILL: terminate forcefully
HUP: usually used to re-read its config
USR1: user provided signal to do some task
ALRM: do some timed job
See man 7 signal for more

11. Enumerate some widely used port numbers and the associated services

20, 21: ftp (Bonus points for mentioning active and passive modes)
22: ssh
25: smtp
53: dns
80: http
110: pop3
143: imap
443: https
…

12. What happens in the background when you do “wget https://index.hu/”? What network requests, protocols are used?

I’d like to hear about the following steps:
– dns resolution (it’s udp, sending more packets, how dns works)
– some http protocol stuff (please include the 3-way TCP handshake)
describe a basic http request and 200 OK response
– please mention some crypto stuff as well (eg. server certificate, why you want a signed certificate vs. a self signed one, public keys, private keys, key exchange, negotiating encryption algorithm, etc.)
– bonus points for mentioning certificate based authentication

13. What’s the difference between a forked and a threaded process?

A forked process becomes independent from its parent (own application state, memory, descriptors), but it’s more expensive than a thread. Threads share the same state and memory space, no isolation from each other. Bonus point for mentioning COW.

14. You have accidentally removed executable right from all files in /bin. How could you list file?

/lib64/ld-linux-x86-64.so.2 /bin/ls

15. Describe the purpose of fsck utility. How could you use it?

fsck stands for file system check. When the system boots, it has the chance to run fsck to fix any file system issues (eg. after an unclean shutdown).
You can also run fsck to heal the partition. Usually it’s a good idea to umount it first. Bonus point for lost+found.

16. Describe the setuid and setgid flags on an executable file. Enumerate a few setuid or setgid binaries in Linux. What ‘find’ command (with parameters please) would find these files in /bin?

The setuid flag is set on a program which needs to run with the permissions of its owner (and not the user’s running it). Let’s say you want to change your password. The shadow file can be modified by root only. So a regular user can update his own password if he becomes root temporarily.

Some examples for setuid / setgid files:
– passwd
– ping (for using the raw socket)
– mount, umount
– su
– sudo

find /bin -type f -perm 4755 -print

17. What’s the loopback device, what would you use it for?

Linux has lo interface usually with 127.0.0.1 (feel free to use 127.0.0.0/8) which means the local host. You may use it to bind network services you want to  access only yourself, eg. a local dns resolver. Note that Debian variants tend to bind mysql to 127.0.0.1.

18. How would you mitigate the risk of stealing passwords on a Linux server?

Once I setup a server which had password for only root. Administrators (ie. users using their own usernames) had ‘*’ (or ‘!’) as their passwords. Now they could login via ssh using ssh keys, and could use passwordless sudo to gain elevated privileges. So virtually there were no passwords to steal.

19. What are the network and broadcast addresses of 10.1.2.3/25?

10.1.2.0 and 10.1.2.127 respectively.

20. How do you install ‘an average’ open source application that is not found in your repo from a git repo?

git clone https://github.com/someproject
cd someproject
./configure
make
sudo make install

21. Explain git rebase

Rebase puts your commit to the HEAD of the given branch.

22. Describe some best practices for building docker images

Add only the minimum necessary layers to the image (ADD/COPY, ENV, RUN)

Chain the commands to a single RUN directive

Install the bare minimum of packages (eg. –no-install-recommends in case of debian variants)

Use a dedicated docker image to build your stuff requiring devel packages, then use a much slimmer image to package your runtime.

Use smart tagging

Use a private docker registry, if it makes sense

Don’t include any secrets to the image

Expose only the absolute necessary network ports to the outside

Use official images only as starting point of your new image

After building the image, be sure to use some vulnerability scanner tool