Static code check for a Gerrit refspec

I wanted to introduce static code check for our repo in Gerrit. Gerrit allows you to fetch the given refspec, then you may use git diff-tree to see what files are in the given change. The trouble is that you already need the repo cloned to do so, which is not that efficient for a larger repo. Fear not, the Gerrit ssh API is to the rescue. Let’s how to make it work.

Let’s say you set up a Jenkins job to be triggered from Gerrit whenever a patchset is created. Gerrit sends a bunch of parameter about the given commit to Jenkins, eg.

GERRIT_CHANGE_ID=I95e8231824c3d63d063b10c8a1d638cc42e3dd44
GERRIT_CHANGE_NUMBER=40175
GERRIT_PROJECT=myrepo
GERRIT_REFSPEC=refs/changes/75/40175/4
Now let’s get the files from the patchset using the Gerrit SSH API (you need the jq utility to process the json output):
ssh -p $GERRIT_PORT $GERRIT_HOST gerrit query –files –format JSON change:$GERRIT_CHANGE_NUMBER –current-patch-set | jq -r “select(.project==\”$GERRIT_PROJECT\”)|.currentPatchSet.files[].file”
The above command produces the following output:
/COMMIT_MSG
aaa.sh

bbb.groovy

Now we have to filter /COMMIT_MSG, since it’s not an actual file in the commit, rather the commit message which we are not interested in this time. So we have two files in the patchset: ‘aaa.sh’ and ‘bbb.groovy’.

Once we have the affected files we use git archive to get only these specific files even from a large repo:

git archive –format=tar –remote=ssh://${GERRIT_HOST}:${GERRIT_PORT}/${GERRIT_PROJECT} “$GERRIT_REFSPEC” “${FILES[@]}” | tar xf –

The FILES array is supposed to contain “aaa.sh” and “bbb.groovy” strings. Note that I omitted the –prefix parameter, so it dumps the files to the current directory, ie. to $WORKSPACE

Finally we run a docker container to do some checks on these files:

docker run –rm -u $(id -u):$(id -g) –name “${JOB_NAME}-${BUILD_NUMBER}” -v “${WORKSPACE}:${WORKSPACE}” -w “$WORKSPACE” -e PATTERN=”.*(sh|inc|groovy)” “static_check:latest”

Notice the PATTERN variables in which we may define what file extensions we want to include in the static code check. I recommend to make this a job variable.

The actual entrypoint in the container executes the following code:

#!/bin/bash

set -o nounset
set -o errexit
set -o pipefail

PATTERN=”${PATTERN:-.*(sh|inc|groovy)}”

error() {
echo “$*”
exit 1
}

check_for_trailing_whitespace() {
local f=”$1″

if grep ‘[[:blank:]]$’ “$f”; then
error “Trailing whitespace in the above lines”
fi
}

while read -r f; do
echo “Checking ${f}”

check_for_trailing_whitespace “$f”

if [[ “$f” =~ .sh$ || “$f” =~ .inc$ ]]; then
shellcheck “$f”
fi
done < <(find . -type f -regextype posix-extended -regex “$PATTERN”)

The above code check every file for trailing white characters, and runs shellcheck for each file with .sh or .inc extensions.

Top 20 or so interview questions

Recently, I was looking for some (mostly) Linux interview questions, and frankly I was disappointed. The top 10-20 google hits (mostly from Indian sites) were mediocre, boring crap, and even worse a mere copycat of each other, without a single, truly genuine question.

Most of them share questions like “What’s the difference between BASH and DOS?“, or “Unix and Linux“. Or “What’s a microprocessor?” Really? But my favourite bullshit question found in every one of them is “What are the different modes of Network bonding in Linux?

Ok, perhaps I felt pissed off, because I didn’t use bonding in Linux, and I didn’t know the answer. Anyway, I’ve compiled my top 20 or so Linux interview questions, with some brief hints what I’d like to hear in the answers.

1. Describe swap partition

Please mention virtual memory (physical memory + swap), paging out to swap, that it’s much slower, than RAM.

2. How do you make ssh daemon more secure?

A few tips:

  • Use protocol version 2, which is the default for any recent distros
  • Use firewalls, iptables, (hosts.allow and deny files, anyone?) whatever to limit access to the daemon
  • Disable password based authentication, allow ssh keys only
  • Disable root login, if possible
  • Some people suggest to use a high port other than 22 to evade scans
  • Use safer MACs (=message authentication code), ciphers and key exchange algorithms
  • Disable port forwarding, if it makes sense

3. What’s an inode?

An inode is a structure holding all info about the file except the filename, eg. ownership, permissions, timestamps (last access, modification, …), size, file type, and link count (see the next question). Bonus for man 2 stat.

4. Why can’t you make a hard link between partition?

A hard link is merely a name for the given file in the directory entries. So you can assign many names for the same file, pointing to the same inode. However, inodes are unique only on the same partition, that’s why you’ll get a similar error:

sj@thorium:~$ ln /boot/vmlinuz-4.15.0-52-generic aaa
ln: failed to create hard link ‘aaa’ => ‘/boot/vmlinuz-4.15.0-52-generic’: Invalid cross-device link

5. Why would anyone use LVM?

Let’s say you have a database server with a single disk with 100 GB. Sooner or later you find it almost filled. Then you need to extend /var/lib/mysql (supposing it’s a mysql server). The simplest solution is to create a logical volume (when installing the server), and you can add another disk to the same volume group, and extend the logical volume, and increase the partition size.

6. Describe ACL, what commands will you use to set and check ACL on a file?

Standard unix permissions consist of ‘owner’, ‘group’ and ‘other’ (or world) categories. Sometime you need a finer, more granular approach to set permissions. You may use setfacl and getfacl commands. Also ls command will display ‘+’ sign to indicate that the given file has ACL set.

7. An (CLI) application segfaults. How do you start troubleshooting it?

Start it using gdb, and check its output.

8. How do you check if an application leaks memory?

Start the application using valgrind.

9. Assuming bash and uid=0, how would you prevent rm -rf $A/bin to misbehave if you forgot to set A=/path/to/somewhere?

Use “set -o nounset”

10. What kind of signals can we send via the kill command?

TERM: terminate the process gracefully, ie. allowing it to shutdown some descriptors, free memory, say goodbye to users, etc.
KILL: terminate forcefully
HUP: usually used to re-read its config
USR1: user provided signal to do some task
ALRM: do some timed job
See man 7 signal for more

11. Enumerate some widely used port numbers and the associated services

20, 21: ftp (Bonus points for mentioning active and passive modes)
22: ssh
25: smtp
53: dns
80: http
110: pop3
143: imap
443: https

12. What happens in the background when you do “wget https://index.hu/”? What network requests, protocols are used?

I’d like to hear about the following steps:
– dns resolution (it’s udp, sending more packets, how dns works)
– some http protocol stuff (please include the 3-way TCP handshake)
describe a basic http request and 200 OK response
– please mention some crypto stuff as well (eg. server certificate, why you want a signed certificate vs. a self signed one, public keys, private keys, key exchange, negotiating encryption algorithm, etc.)
– bonus points for mentioning certificate based authentication

13. What’s the difference between a forked and a threaded process?

A forked process becomes independent from its parent (own application state, memory, descriptors), but it’s more expensive than a thread. Threads share the same state and memory space, no isolation from each other. Bonus point for mentioning COW.

14. You have accidentally removed executable right from all files in /bin. How could you list file?

/lib64/ld-linux-x86-64.so.2 /bin/ls

15. Describe the purpose of fsck utility. How could you use it?

fsck stands for file system check. When the system boots, it has the chance to run fsck to fix any file system issues (eg. after an unclean shutdown).
You can also run fsck to heal the partition. Usually it’s a good idea to umount it first. Bonus point for lost+found.

16. Describe the setuid and setgid flags on an executable file. Enumerate a few setuid or setgid binaries in Linux. What ‘find’ command (with parameters please) would find these files in /bin?

The setuid flag is set on a program which needs to run with the permissions of its owner (and not the user’s running it). Let’s say you want to change your password. The shadow file can be modified by root only. So a regular user can update his own password if he becomes root temporarily.

Some examples for setuid / setgid files:
– passwd
– ping (for using the raw socket)
– mount, umount
– su
– sudo

find /bin -type f -perm 4755 -print

17. What’s the loopback device, what would you use it for?

Linux has lo interface usually with 127.0.0.1 (feel free to use 127.0.0.0/8) which means the local host. You may use it to bind network services you want to  access only yourself, eg. a local dns resolver. Note that Debian variants tend to bind mysql to 127.0.0.1.

18. How would you mitigate the risk of stealing passwords on a Linux server?

Once I setup a server which had password for only root. Administrators (ie. users using their own usernames) had ‘*’ (or ‘!’) as their passwords. Now they could login via ssh using ssh keys, and could use passwordless sudo to gain elevated privileges. So virtually there were no passwords to steal.

19. What are the network and broadcast addresses of 10.1.2.3/25?

10.1.2.0 and 10.1.2.127 respectively.

20. How do you install ‘an average’ open source application that is not found in your repo from a git repo?

git clone https://github.com/someproject
cd someproject
./configure
make
sudo make install

21. Explain git rebase

Rebase puts your commit to the HEAD of the given branch.

 

22. Describe some best practices for building docker images

Add only the minimum necessary layers to the image (ADD/COPY, ENV, RUN)

Chain the commands to a single RUN directive

Install the bare minimum of packages (eg. –no-install-recommends in case of debian variants)

Use a dedicated docker image to build your stuff requiring devel packages, then use a much slimmer image to package your runtime.

Use smart tagging

Use a private docker registry, if it makes sense

Don’t include any secrets to the image

Expose only the absolute necessary network ports to the outside

Use official images only as starting point of your new image

After building the image, be sure to use some vulnerability scanner tool

 

 

“Top 3 Best Email Archiving Solutions” – how not write a review

I’ve just stumbled in an article about the “Top 3 Best Email Archiving Solutions”. I was curious, so I engaged, and read the article on the topic. What I found was a ridiculous conclusion, read it for yourself below.

Our favorite email archiver is Clean Email. This modern and easy to use inbox cleaner is a perfect companion for anyone who relies on email and wants to be more organized.

So basically the people of Clean Email concluded that Clean Email was the best solution to archive emails. It’s clearly a biased “review”.

 

Changing the background colour for on remote server via SSH

Once I managed to shutdown a server. At the end of the day I wanted to shutdown my notebook, so I switched to the workspace with my terminals open, and typed sudo halt.

I noticed pretty soon that my notebook didn’t halt, instead I lost the connection to a remote host, because I typed the command in the wrong terminal.

The solution is to make the terminal background colour somewhat different, eg. to make it reddish. The following command does the trick:

printf ‘\033]11;#9e0e2a\007’

You have several options.

1. Add it to the .profile on the remote host

2. Create an alias for ssh:

alias s=”printf ‘\033]11;#9e0e2a\007’; ssh”

You can have an alias to revert the bgcolor back to black:

alias black=”printf ‘\033]11;black\007′”

HP Enterprise adatkozponti mernok = vicc kategorias ajanlat a Braining Hub-tol

A jelek szerint a Hewlett Packard Enterpise ill. a partnerei is munkaero (hiany okozta) gondokkal kuszkodnek. Igy a HP Enterprise Braining Hub elhatarozta, hogy adatkozponti mernokoket fog kepezni.

Fel ev, munka mellett is vegezheto, majd 400 kontaktora, 600 ezer Ft + AFA osszegert egy egeszen jo tematikan ragjak magukat vegig a hallgatok.

Azonban a kepzes utan kotelezettsegek is varjak a vegzetteket, amiben az szerepel, hogy

A képzés elvégzése után a Hewlett Packard Enterprise partnercégei által felajánlott pozíciót köteles vagy elfogadni, amennyiben az minimum 250 000 Ft nettó fizetéssel jár. 1 éves időtartamra.

Csak 1 kerdesem van: Maaargit, noooormalis? Magyarorszagon, ahol ma lasszoval kell fogni IT szakembereket, ez komoly, hogy brutto 380 kHUF koruli osszeggel akarjak kiszurni a szemuket? Foleg, hogy a vegzetteket nyilvan nem emberbarati szeretetbol, hanem kokemeny profitert fogjak kikozvetiteni. Nonszensz.

Skype vs. Firefox as an unsupported browser

Today I wanted to login to skype web client. Unfortunately web.skype.com showed me the following nice bullshit:

I was using skype web a few days ago, and now firefox is not supported.  Shall I use Microsoft Edge? Really? Shame on you, damned fools!

Fear not however, because Mozilla has a nice addon to set the User-Agent. So I’ve installed the addon, then faked Windows / Edge 17, and finally I was in, and could use skype web just like before.

Removing an S3 bucket

I tried to remove a bucket from an S3 compatible object store. The trouble is that it wasn’t empty, there were lots of objects inside. I tried minio’s client:

mc rm --recursive --dangerous --force exo/somebucket

But it just seemed to stall after removing ~100 objects. The solution is to run it using timeout in a loop, eg.

while true; do timeout 10 mc rm --recursive --dangerous --force exo/somebucket; done