linux-server-setup/exercises/h7.md

Linux servers - Exercice 7
==============

*Disclaimer:*
--------------

This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).

*Table of contents:*
--------------

- [a) **Practical lab test** Solve a previously published final major test of this school course (You can find them with DuckDuckGo or Google).](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#a-solve-a-previously-published-final-major-test-of-this-school-course-you-can-find-them-with-duckduckgo-or-google)

    - [Initial set-up for remote control of a server](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#initial-set-up-for-remote-control-of-a-server)

    - [Security set-up for a company](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#security-set-up-for-a-company)

    - [System statistics](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#system-statistics)

    - [PHP from abroad](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#php-from-abroad)

    - [iot12tools - IoT tools for users](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#iot12tools---iot-tools-for-users)

    - [Hello Python 3](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#hello-python-3)

- [c) **Free-will Linux usage** (optional) Use Linux outside the course scope.](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#c-optional-use-linux-outside-the-course-scope)

    - [About my Linux usage](https://github.com/Fincer/linux_server_setup/blob/master/exercises/h7.md#about-my-linux-usage)

--------------

**a)** Solve a previously published final major test of this school course (You can find them with DuckDuckGo or Google).
--------------

**Answer:**

Let's pick up our previous major test [ict4tn021-1 autumn 2016](http://terokarvinen.com/2016/arvioitava-laboratorioharjoitus-%e2%80%93-linux-palvelimet-ict4tn021-1-uusi-ops-alkusyksylla-2016).

----------------------------

### Initial set-up for remote control of a server

**Issue:** Prepare to control the server from abroad. Secure the server with firewall

We shall install SSH server daemon to the targeted server computer. It is assumed that local or other direct access to the server command line is available since SSH server daemon is not yet installed, thus preventing direct, remote SSH control of the server.

Once having access to the server command line, we shall test that the computer can reach the internet. Test the following commands:

```
ifconfig
ping www.duckduckgo.com
ip -br link show
```

Command explanations:

- [ifconfig](https://www.linux.fi/wiki/Ifconfig) = Has IP address determined to a network interface on the server?

- [ping](https://fi.wikipedia.org/wiki/Ping) = Does a site respond to ICMP requests?

- ip = Is the network interface, which should have connection to the internet, up and active (UP)?

When the internet connection is established, we shall proceed by installing the following packages, assuming that the server uses a Debian-based Linux distribution: `openssh-server`, `openssh-sftp-server`, `openssh-client`

On some distributions, those packages can be installed simply issuing:

```
sudo apt-get update
sudo apt-get -y install ssh
```

We shall confirm that the SSH server daemon starts during the server boot-up. We shall also confirm that the SSH server daemon is up and running and its status is `active`:

```
sudo systemctl enable ssh.service
systemctl is-active ssh.service
```

**NOTE:** Alternative commands can also be used, like:

```
sudo systemctl enable sshd
systemctl status sshd
```

Has our firewall program installed on the system?

```
which ufw
which iptables
```

If both of those executable files ('ufw' and 'iptables') are found on the system, we shall accept network traffic to port `22`, protocol TCP (INPUT 22/TCP). Let's apply these rules to our firewall:

```
sudo ufw allow 22/tcp
```

Make sure our firewall program `ufw` (Uncomplicated Firewall) is enabled and turned on:

```
sudo ufw enable
sudo systemctl enable ufw.service
```

**NOTE:** By default, Linux firewall blocks all input traffic. Therefore, SSH input traffic must separately be allowed like described above.

----------------------------

### Security set-up for a company

**Issue:** Install remotely working security tools for our company. (In this assignment, you can assume that installing a package or packages from our repository is secure)

The following commands have been pre-determined in the assignment:

- download `.deb` package file with `wget` command

- install the downloaded `.deb` package with command `sudo dpkg -i` which extract a new repository file `terorep.list` into `/etc/apt/sources.list.d/`)

- update package databases with command `sudo apt-get update`

- install package `terowatch` which is made available by the new repository. The package is available for Ubuntu distribution, version 16.04 LTS. The repository file `terorep.list` itself contains string `deb http://terokarvinen.com/qrs/terorep/ precise main`

**NOTE:** `terowatch` package uses network interface `eth0` by default. However, we haven't defined such interface in our system configuration (this can be fixed by adding `net.ifnames=0` in udev rules or in kernel boot parameters in Syslinux or GRUB2). Instead, we use default network interface `enp4s0`.

Proper fix to this issue would be patching the code and applying the patch into the deb package. Another solution would be making the proper fix directly to the source code. In this assignment, we directly modify the executable file, written in Python. This method is not recommended but for the extent of this assignment, it is sufficient solution to proceed.

```
dpkg -L terowatch
sudo sed -i 's/eth0/enp4s0/g' /usr/bin/qrsc
```

Command `terowatch` gives desired text output “TeroWatch is installed” (the string is defined in shell executable `/usr/bin/terowatch`)

----------------------------

### System statistics

**Issue:** Collect workload statistics of various system resources (CPU, RAM) while doing the assignment. The data collection must be started before proceeding in the assignment. Write a short analysis of the collected statistics after you've finished other parts of the assignment.

We shall install `sysstat`:

```
sudo apt-get update && sudo apt-get -y install sysstat
```

Let's run `sar` command with 2 second interval for undefined time period. Print output of the command to file `$HOME/sysstat.file`

```
sar 2 -o $HOME/sysstat.file
```

The file can be analysed afterwards with `sar` command in the end of the assignment.

![sysstat_cpu-mem](https://github.com/Fincer/linux_server_setup/blob/master/images/sar_stats_cpu-ram.png)

Short analysis, based on the picture above:

- CPU (command: `sar 10 -f $HOME/sysstat.file`): CPU has been in moderate workload. User processes have not depleted resources that much, in contrast to system processes which stress the CPU many times more. 'iowait' value tells us that the processor has waited for mass memory device. During the data collection, some data was transferred from a hard disk to another. CPU capacity limits were not reached.

- Mem/RAM (command: `sar -r 10 -f $HOME/sysstat.file`): Memory consumption has been significant during the data collection period (`%memused` and `%commit`). The system has 4GB DDR3 memory of which majority has been in use. Amount of free memory has been 100MB. Two main operations have affected the memory usage level: web browser usage and file transfer/copy operations which were performed during the data collection period. Active memory was used 2.5GB on average, passive (inactive) memory usage was 700MB. Amount of memory which has waited for to be written varies between 5MB-200MB (`kbdirty`).

----------------------------

### PHP from abroad

**Issue:** Install us necessary tools for remote PHP website development.

- SSH server daemon has already been installed, port `22` is opened -> OK

- We shall install HTTP daemon (Apache web server) and open the default port `80` for it in our firewall. Additionally, we shall install necessary PHP (7.0) packages:

```
sudo apt-get update && sudo apt-get -y install apache2 php7.0 libapache2-mod-php7.0 && sudo ufw allow 80/tcp
```

Let's enable Apache server daemon, check the status of it (must be `active`) and tell the system that Apache should automatically be started during the server boot-up:

```
systemctl is-active apache2
sudo systemctl enable apache2
```

Let's check that the port `80` is opened in our firewall:

```
xubuntu@xubuntu:/home$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
80/tcp (v6)                ALLOW       Anywhere (v6)
```

(you don't need to execute command `sudo systemctl start apache2` after the Apache installation because the Apache web server daemon is automatically enabled during the server boot-up by default. This information can be found with command `systemctl status apache2`)

We shall enable Apache's PHP module while using module `userdir`. This can be done by commenting the lines between `<IfModule mod_userdir.c>` tags in file `/etc/apache2/mods-available/php7.0.conf`. Right after we should enable Apache's 'userdir' module and restart the Apache web server:

```
sudo a2enmod userdir
sudo systemctl restart apache2.service
```

Let's check that Apache web server is still up and running by issuing command `systemctl status apache2.service`. Let's check that we can access our default localhost website (IP address `127.0.0.1`) which indicates whether the Apache server works as intended:

```
xdg-open http://localhost
```

'xdg-open' command definition:

```
[12/03/2018 22:31:11 - fincer: ~ ]$ whatis xdg-open
xdg-open (1)         - opens a file or URL in the user's preferred application
```

We must check that we can access our website from a remote network. In a test laboratory, this test would be done checking output of command `ifconfig`, looking for a relevant IP address and login to the server from another computer using the grabbed IP address and SSH client program (command syntax using default SSH port `22` would be: `ssh server-user@server-ip`). In order to access the server from a remote network, the relevant IP address for connecting the server must be known, and confirmation for succeeded remote access must exist. Any router & NAT/PAT configurations between a client and the server must work.

**Issue:** Our users are as follows: Maija Mehilälinen, Peter Ö, Oskar Jäärä, John Do, Verner Vrij, Mikko Möttönen, Jalmari Ähkä, Håkan Swarz and Maija Maitoparta. Create a sample website for each user with PHP.

**Issue:** List all user accounts and passwords in file lab.txt (`/home/*/lab.txt`). Protect the file so that other users can not read it.

We shall generate all required passwords (9 for users + admin) with `pwgen`. Password length is 20 characters, randomized, at least one uppercase character included and secure parameter for the command is used:

```
sudo apt-get update && sudo apt-get install pwgen
pwgen 20 10 -sc1
```

We shall store additional admin password into a separate, secure place (of your choice). Admin account (or user) username is `admin`.

We haven't created admin user `admin` yet. Let's do it. We shall also grant `sudo` group permissions to this new administration user:

```
sudo adduser admin
sudo usermod -aG sudo admin
su admin
```

The last command `su admin` switched our shell view to `admin` shell. We should lock the server root account by issuing command `sudo usermod --lock root` (which adds an exclamation mark right before root's password in file `/etc/shadow`). We could increase system's overall security by blocking usage of various TTY sessions (commenting out lines in file `/etc/securetty`) and by adding more restrictions to the system's PAM policy.

For securing our SSH connections, we shall add the following line in `/etc/ssh/sshd_config` file (use `sudoedit /etc/ssh/sshd_config` or `sudo nano /etc/ssh/sshd_config` command):

```
PermitRootLogin no
```

After having altered `/etc/ssh/sshd_config`, restart SSH server daemon by issuing command 

```
sudo systemctl restart sshd
```

Create file `$HOME/lab.txt` (where `$HOME` is `/home/admin`) with the following contents:

```
mmehilal    RWjM8r2fyq8gQq3wnzq0        # Maija Mehilälinen
po          a5B60cmTyrwPLS2mJ9uQ        # Peter Ö
ojaara      P1KkLl0XaV5JpqntxdJG        # Oskar Jäärä
jdo         ZgMb5TrU4Ee3bT48eCff        # John Do
vvrij       RWjM8r2fyq8gQq3wnzq0        # Verner Vrij
mmottone    GpuyZqp2pGsmPINJPO1h        # Mikko Möttönen
jahka       nFTellIyRjwiC0YKtPwq        # Jalmari Ähkä
hswarz      u5qfJCC2jZMGQPWwPZLV        # Håkan Swarz
mmaitopa    OCeBmKYWWgFIXubdFc6j        # Maija Maitoparta
```

Restrict permissions for this file by issuing the following command (as user `admin`):

```
chmod og-rwx,u-x $HOME/lab.txt
```

Output:

```
admin@xubuntu:~$ ls -l lab.txt
-rw------- 1 admin admin 407 Mar 12 11:42 lab.txt
```

Let's create a generic, pretty simple PHP file `index.php` into the home directory of `admin` (and as `admin`).

```
nano /home/admin/index.php
```

Add the following contents:

```
<?php
phpinfo();
?>
```

**NOTE:** Make sure you have enabled `php` & `userdir` modules on Apache at this point!

After that, we shall copy the PHP file for each user into a user-specific subdirectory `public_html` (`/home/*/public_html`).

The user-specific directory `public_html` and contents of it should have correct permissions set up for each user.

On Debian-based Linux distributions, a perl script '[adduser](https://alioth.debian.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=adduser/adduser.git;a=tree)' is provided for creating new users. Let's do the following:

- 1) Create required user accounts with the generated passwords

- 2) Copy the pre-created `index.php` file into each user's `$HOME/public_html` folder

- 3) For each user, test that their `index.php` is showing expected and correct output

Run the following with any system user who belongs to `sudo` group:

```
sudo apt-get update && \
sudo apt-get -y install curl && \
for user in mmehilal po ojaara jdo vvrij mmottone jahka hswarz mmaitopa; do \
echo -e "User is: $user\n" && \
sudo adduser $user && \
sudo mkdir -p /home/$user/public_html/ && \
sudo cp /home/admin/index.php /home/$user/public_html/ && \
sudo chown -R $user:$user /home/$user && \
[[ $(curl -s http://localhost/~${user}/index.php | grep -i "404 not found" | wc -l) -ne 0 ]] && \
echo -e "\nPHP test site not found for '${user}'\n" || \
echo -e "\nPHP test site is OK for user\n"; \
done

```

**NOTE:** To test site on a graphical web browser, you need to keep in mind that user `xubuntu` is the only user having access to graphical display on X server right now (see `.Xauthority` and environment variable `DISPLAY` for details). Thus, the following commands should be used in our current setup:

```
sudo passwd xubuntu
su xubuntu
for user in mmehilal po ojaara jdo vvrij mmottone jahka hswarz mmaitopa; do xdg-open http://localhost/~${user}; done
```

**NOTE:** We can exit the shell view of the user `admin` simply issuing command `exit` on that shell view.

**NOTE:** Consider the following in a production environment, instead of doing nasty things with administration users:

- You can (but don't have to) modify the current user, grant administration rights, modify username, home directory and groups (`usermod` command) instead of adding a separate account for an administrator

- You can create a new main/admin user and remove an old one (in our case, we should delete `xubuntu` account). You can delete users with command `sudo userdel <account>`.

- You can have clearly restricted and well named groups to separate users with various permissions on a system. Apply your user and group policy so that specific users belong to the right groups.

- Check usage of various environment variables in your shell view

Screenshot of user-specific PHP sites on the configured server environment:

![apache-php-multiple](https://github.com/Fincer/linux_server_setup/blob/master/images/apache_phpinfo_multiple.png)

Checking permissions for each `index.php` files & `public_html` directories:

```
for user in mmehilal po ojaara jdo vvrij mmottone jahka hswarz mmaitopa; do ls -lR /home/$user | sed '/total [0-9]/d'; done
```

Output:

```
/home/mmehilal:
drwxr-xr-x 2 mmehilal mmehilal 60 Mar 12 11:50 public_html

/home/mmehilal/public_html:
-rw-r--r-- 1 mmehilal mmehilal 20 Mar 12 12:00 index.php
/home/po:
drwxr-xr-x 2 po po 60 Mar 12 12:00 public_html

/home/po/public_html:
-rw-r--r-- 1 po po 20 Mar 12 12:00 index.php
/home/ojaara:
drwxr-xr-x 2 ojaara ojaara 60 Mar 12 12:01 public_html

/home/ojaara/public_html:
-rw-r--r-- 1 ojaara ojaara 20 Mar 12 12:01 index.php
/home/jdo:
drwxr-xr-x 2 jdo jdo 60 Mar 12 12:01 public_html

/home/jdo/public_html:
-rw-r--r-- 1 jdo jdo 20 Mar 12 12:01 index.php
/home/vvrij:
drwxr-xr-x 2 vvrij vvrij 60 Mar 12 12:01 public_html

/home/vvrij/public_html:
-rw-r--r-- 1 vvrij vvrij 20 Mar 12 12:01 index.php
/home/mmottone:
drwxr-xr-x 2 mmottone mmottone 60 Mar 12 12:01 public_html

/home/mmottone/public_html:
-rw-r--r-- 1 mmottone mmottone 20 Mar 12 12:01 index.php
/home/jahka:
drwxr-xr-x 2 jahka jahka 60 Mar 12 12:02 public_html

/home/jahka/public_html:
-rw-r--r-- 1 jahka jahka 20 Mar 12 12:02 index.php
/home/hswarz:
drwxr-xr-x 2 hswarz hswarz 60 Mar 12 12:02 public_html

/home/hswarz/public_html:
-rw-r--r-- 1 hswarz hswarz 20 Mar 12 12:02 index.php
/home/mmaitopa:
drwxr-xr-x 2 mmaitopa mmaitopa 60 Mar 12 12:02 public_html

/home/mmaitopa/public_html:
-rw-r--r-- 1 mmaitopa mmaitopa 20 Mar 12 12:02 index.php
```

----------------------------

### iot12tools - IoT tools for users

**Issue:** Install and configure metapackage `iot12tools` for us. The metapackage should install the following development tools for 'Internet of Things': arduino IDE, gedit (text editor), gedit-plugins, curl (CLI browser) and Python 3 programming language.

Run the following commands as `admin`:

```
sudo apt-get update && sudo apt-get -y install equivs
mkdir ~/iot12tools
cd ~/iot12tools
equivs-control iot12tools

```

Edit contents of file `io12tools` to look like the following:

```
### Commented entries have reasonable defaults.
### Uncomment to edit them.
Section: misc
Priority: optional
Standards-Version: 3.9.2

Package: iot12tools
Version: 0.1
Maintainer: Pekka Helenius <fincer89@hotmail.com>
Depends: arduino, gedit, gedit-plugins, curl, python3
Architecture: all
Description: A meta package which provides basic IoT development tools
 Installs Gedit text editor, Gedit plugins, Arduino IDE and Python 3

```

Run the following command in directory `~/iot12tools`:

```
equivs-build iot12tools
```

Install the generated metapackage (which installs required packages for us):

```
sudo apt install -f ./iot12tools_0.1_all.deb
```

----------------------------

### Hello Python 3

**Issue:** Jalmari wants to develop in Python 3. Create a simple Python 3 file in his home root directory. The file should print output 'Hello World'.

Install necessary python3 packages as the user `admin`:

```
sudo apt-get update && sudo apt-get install -y python3
```

Switch your shell view to user `jahka` (Jalmari Ähkä) and go to his home directory:

```
su jahka
cd
```

**NOTE:** We can make sure that we are in his home directory (`/home/jahka`) by issuing command 'pwd'.

Let's create a new file `~/helloworld.py` with the following contents:

```
#!/usr/bin/env python3

print("Hello World");

```

Modify file permissions, grant executable permission for the user `jahka`, remove from others (including the group who owns the file):

```
chmod u+x,og-x helloworld.py
```

The file permissions are now:

```
-rwxrw-r-- 1 jahka jahka 47 Mar 12 12:59 helloworld.py
```

Test the code:

```
jahka@xubuntu:~$ python3 ~/helloworld.py 
Hello World

```

The output is as desired, `Hello World`, indicating that we have successfully installed Python3 development environment for the required user.

**c)** (optional) Use Linux outside the course scope.
--------------

**Answer:**

The following pictures demonstrate [Arch Linux ARM](https://archlinuxarm.org/platforms/armv6/raspberry-pi) running on my Raspberry Pi 1 Model B. The solution is quite rushed but it works technically:

![rpi1b_archlinux-1](https://github.com/Fincer/linux_server_setup/blob/master/images/rpi1b_archlinux_1.jpg)

![rpi1b_archlinux-2](https://github.com/Fincer/linux_server_setup/blob/master/images/rpi1b_archlinux_2.jpg)

- Display: [Lilliput 869GL](http://www.lilliputuk.com/monitors/hdmi/869gl/)

- SDHC card: Transcend 16GB

- HDD: 750GB in [Icy Box IB-3640SU3](http://www.raidsonic.de/products/external_cases/external_cases_35/index_en.php?we_objectID=480) hard disk external case.

- Desktop environment: LxQt

### About my Linux usage

My first touch to Linux world was back in early spring, 2011. The first Linux distribution I installed was Ubuntu 10.04 LTS, and after that I have tried out many distributions, including Linux Mint, Fedora, OpenSUSE and Arch Linux.

The first computer I ever had a Linux installation was [Asus Eee PC 1215N](https://www.asus.com/Laptops/Eee_PC_1215N/) mini laptop. My primary operating system on that laptop was [EasyPeasy](https://sourceforge.net/projects/ubuntu-eee/), a variant of Ubuntu 10.04 LTS which uses Gnome 2 desktop environment and additional customization package 'ubuntu-netbook-launcher' (known as [Linux Netbook Remix](https://en.wikipedia.org/wiki/Ubuntu_Netbook_Edition)), predecessor of Canonical's Unity desktop.

After the Asus Eee PC 1215N broke, I moved to [Asus N56JR laptop](https://www.asus.com/Laptops/N56JR/) for which I installed Linux Mint with Cinnamon desktop environment at first. Soon after I moved to Arch Linux where I used Cinnamon desktop for a short while, as well. I moved quickly from Cinnamon to KDE desktop because I Cinnamon desktop had mysterious and unacceptable freeze events on my personal use.

Why did I choose KDE desktop over multiple alternatives, such as XFCE, LXDE, Mate, Gnome 3, Budgie, Pantheon or LxQt? I consider several things when choosing a desktop environment: stability, long-term development history, suitability for efficient multitasking and configuration possibilities (source code + GUI + conf files). I have been happy Arch Linux + KDE desktop user for many years - and I still am. Of course, I have tested various other desktop environments but always returned to KDE.

In general, I have multiple Linux distributions, MS Windows 7 and Android on Virtual Machines for various purposes.

More or less, I have participated in various open source project and done Windows software testing on Wine. I like to create scripts for daily usage, mainly to automate routine-like, boring tasks and processes. One of my key interests have been in digital image processing and raw image analysis, for which both Linux suits very well, offering a solid, adaptable and cost-effective platform. I set up my first LAMP server several years back from now on my Raspberry Pi 1 Model B.