Browse Source

Initial commit

master
Fincer 3 years ago
commit
c56fbe98ff
39 changed files with 6581 additions and 0 deletions
  1. +16
    -0
      README.md
  2. +35
    -0
      exercises/exam.md
  3. +438
    -0
      exercises/h1.md
  4. +316
    -0
      exercises/h2.md
  5. +475
    -0
      exercises/h3.md
  6. +595
    -0
      exercises/h4.md
  7. +1027
    -0
      exercises/h5.md
  8. +127
    -0
      exercises/h6.md
  9. +507
    -0
      exercises/h7.md
  10. +24
    -0
      exercises/h_all.md
  11. BIN
      images/apache_phpinfo_multiple.png
  12. BIN
      images/apina-site.png
  13. BIN
      images/bmicalc-sample.png
  14. BIN
      images/empty-page-sample.png
  15. BIN
      images/h2-exiftool.png
  16. BIN
      images/h2-fakeroot.png
  17. BIN
      images/h2-lynx.png
  18. BIN
      images/pekkahh-website.png
  19. BIN
      images/php-example-rainfall.png
  20. BIN
      images/phpmyadmin-2-resized.png
  21. BIN
      images/phpmyadmin-resized.png
  22. BIN
      images/rpi1b_archlinux_1.jpg
  23. BIN
      images/rpi1b_archlinux_2.jpg
  24. BIN
      images/sar-iostats.png
  25. BIN
      images/sar-stats_ipv4.png
  26. BIN
      images/sar-stats_memusage.png
  27. BIN
      images/sar_stats_cpu-ram.png
  28. BIN
      images/workprice-site_example.png
  29. +2274
    -0
      other/apache-log-sample
  30. +52
    -0
      patches/patch_apache_servertokens.patch
  31. +94
    -0
      scripts/compile-xcmenu-screenshot.sh
  32. +94
    -0
      scripts/file-and-dirs-owner.sh
  33. +112
    -0
      scripts/fix-xfce4-mousefocus-window-edges.sh
  34. +51
    -0
      scripts/list-commands-of-a-package.sh
  35. +29
    -0
      scripts/processid.sh
  36. +14
    -0
      scripts/processid_kill.sh
  37. +10
    -0
      scripts/remove-multiple-pkgs-in-debian.sh
  38. +174
    -0
      scripts/spoof_mac_address.sh
  39. +117
    -0
      scripts/sysstat_command.sh

+ 16
- 0
README.md View File

@ -0,0 +1,16 @@
README - Basics of setting up a single Linux server/client environment
==============
- This repository is related to single-server/client Linux (server) environments.
- This repository, as for now, doesn't contain any advanced set-up & maintenance instructions for multi-client/server environment techniques such as Puppet or Salt.
- This repository doesn't contain instructions to set up containers, such as Docker or Systemd-nspawn.
- The repository is mainly set up as a requirement by a school cource in Haaga-Helia University of Applied Sciences, Helsinki, Finland.
- The repository contains various exercises, originally presented in Finnish but translated into English due to practical purposes.
- All bourne shell files (sh) presented in this repository are not directly related to the exercises but they are useful in some situations.
- **NOTE!** Instructions in this repository may not directly apply to your setting. The instructions have not been tested in a production environment, so if you decided to use them in your system configuration, you should apply them with care. I do not take any responsibility in a case of failure or broken system configurations.

+ 35
- 0
exercises/exam.md View File

@ -0,0 +1,35 @@
0) Check critical commands
1) Set up SSH & Apache
- Test SSH
- Harden SSH login policy
- Customize SSH port (get ssh port from a separate variable)
- Use public key authentication method
- Test Apache
- Remove Apache default site
- Strict Apache security policy
- Apply custom patch to the source code
- Remove root login
- Restrict root accessibility
- Add a default user with sudo permissions
- Enable userdir for this new user
2) Set up PHP, phpMyAdmin & MySQL
- Set Apache PHP environment (get php version from a separate variable)
- Set MySQL database
- Configure phpMyAdmin
3) Set up firewall configuration with ufw
4) Set up a PHP website that takes user input and writes it into MySQL database
5) Set up and install Wordpress
- Points to different DNS (/etc/hosts)
6) Check /var/log/auth.log for any suspicious SSH login attempts

+ 438
- 0
exercises/h1.md View File

@ -0,0 +1,438 @@
Linux servers - Exercice 1
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Create your own Linux USB live stick. Try it out on a non-school computer. This section does not require installation of Linux. However, if you decided to install Linux, you should take proper backups first.
--------------
**Answer:** I did this with [Rufus Windows application](https://rufus.akeo.ie/) on Microsoft Windows 7 which was run inside [Oracle VirtualBox](https://fi.wikipedia.org/wiki/VirtualBox) as a virtual machine. Alternative methods include [UNetbootin](https://unetbootin.github.io/), [ISO Image Writer](https://github.com/KDE/isoimagewriter) or dd command (default location on Linux: /usr/bin/dd) which is a part of [Coreutils](http://www.gnu.org/software/coreutils/coreutils.html). Rufus allows you to create a live USB stick using either ISO or DD method which differ from each other, as the developer has explained [on his GitHub site](https://github.com/pbatard/rufus/issues/843#issuecomment-255680876).
Let's take a detailed look for two of these approaches: Rufus on Windows 7 and dd command on Linux (any Linux distribution applies here).
**Alternative 1 - Rufus (Windows, GUI-based)**
You can use either Windows 7 or 10 (any student/enterprise/server subversion applies here). You can run Windows either in a virtual machine or as physically installed in your mass media storage (SSD/HDD). Whichever method you choose to run Windows, it doesn't really matter. The most important part is that you have a working operating system.
1. Launch Microsoft Windows
2. Download Rufus application from [here](https://rufus.akeo.ie/), under the title "Download"
3. You can use either portable or installable version of Rufus. In the following example, we use portable version v. 2.18.
4. Download any Linux distribution you prefer. In further parts, Debian-based Ubuntu 16.04.3 LTS is used. What comes to alternative download methods, you can choose either ISO image file (downloaded directly from servers) or distributed torrent method. Torrent method usually requires a client application (such as [Transmission](https://transmissionbt.com/download/)) to work correctly. The Linux distribution of choice can be anything you prefer in your working environment and which requirements you have set up for your Linux server environment.
Let's download a widely used Xubuntu 16.04.3 LTS (x86_64 processor architecture), which is based on Ubuntu 16.04.3 LTS but uses XFCE desktop environment instead of Unity. In Ubuntu version syntax, 16 refers to release year (2016), 04 refers to release month (April), 3 is subversion and LTS stands for Long-Term Support (preferred in server environments).
[Xubuntu 16.04.3 LTS - Torrent download](http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/xubuntu-16.04-desktop-amd64.iso.torrent)
[Xubuntu 16.04.3 LTS - ISO image file directly from a server - 1.2G](http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/xubuntu-16.04-desktop-amd64.iso)
**Comparing checksums:**
If you wanted to make sure the ISO image you just downloaded is valid, you should check its checksum (mainly for security reasons). In this way you make sure that no malicious party has altered the image you trust and the download process has succeeded. You should take a closer look on sha256sum and md5sum values of your ISO image file. The value must match the one which is published on the Ubuntu download site ([sha256sum](http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/SHA256SUMS), [md5sum](http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/MD5SUMS)).
On Linux: check checksums with commands *sha256sum xubuntu-16.04-desktop-amd64.iso* and *md5sum xubuntu-16.04-desktop-amd64.iso*
On Windows: Use [Microsoft File Checksum Integrity Verifier](https://www.microsoft.com/en-us/download/details.aspx?id=11533) or check a CertUtil-based solution [given on superuser.com](https://superuser.com/questions/245775/is-there-a-built-in-checksum-utility-on-windows-7/898377#898377)
**NOTE:** Checking checksums is highly recommended. For instance, Linux Mint network servers were hacked due to outdated server software in 2016, resulting to maliciously injected ISO images uploaded by hostile attackers. In addition, the Linux Mint forum software was affected, as well (user credentials leaked). More about the incident is explained [on Linux Mint blog](https://blog.linuxmint.com/?p=2994).
5. Plug in an empty and large enough USB stick into your computer (4GiB should be sufficient). It is highly recommended that you have formatted the stick in advance because Rufus overwrites any existing files existing in the USB stick in order to place required Linux files in the stick. If your stick contained any important files, they should be moved from the stick to somewhere safe location before proceeding here.
6. Once the stick is emptied (and probably formatted), launch Rufus application. Select the USB stick as a target device in *Device* menu. All other settings can be left in their default values. Choose downloaded Xubuntu 16.04.3 LTS ISO image file from the dropdown menu located at the right of *Create a bootable disk using* part. The dropdown menu is presented as a disc icon without any texts.
7. Click *Start*, choose ISO method as a writing method once asked. Read the warnings. If all steps have been successfully accomplished so far, proceed with the writing operation.
8. Wait until the ISO image file has been written into the USB stick. Once done, reboot your computer with the USB stick attached. Immediately the computer has rebooted, press a button which you can access your BIOS/UEFI/Boot options menu with. Choose the Xubuntu 16.04.3 LTS USB stick (which should appear in the boot list). Start the operating system boot-up process bt selecting *Try without installing* option (this step varies between Linux distribution. More generally, you can also choose *Install <whateverdistro>* option and select installation on the live USB Linux desktop environment). Wait until the operating system has been booted up. Start installation and follow the instructions.
The installation process is not defined in detail here because installation steps varies from Linux distribution to distribution. However, basic principle is to install the operating system with good and strong passwords. Encrypted home folder or HDD/SSD partition options should also be considered. After you have successfully installed Linux, detach the live USB stick and reboot your computer. You should be able to access the installed Linux distribution without the USB stick.
When writing and applying passwords, please pay attention to correct keyboard layouts. In my case, I use a Swedish (Finnish) keyboard layout which differs from the default en_US layout. Therefore, mistyping characters can easily happen. Many Linux distributions allow you to change the keyboard layout during the installation process. Unless you're not able to do it in installation window, you can change the keyboard layout temporarily by executing *setxkbmap fi* in a local shell environment (NOTE: graphical environment is required since the command is a part of graphical X11 environment!). You can open up a local shell by opening a terminal emulator while running Linux from your USB stick. NOTE: changing passwords must be done before writing them in the installation steps!
**Alternative 2 - dd command (Linux, CLI-based)**
In case you use any Unix-based operating system (Linux, GNU/Linux, Apple MacOS, FreeBSD), you can create a USB live stick by using shell-based dd command. This method does not require a graphical desktop environment. **NOTE!** Commands on Apple MacOS may not be exactly equal to Linux ones if you use extra parameters in the following commands.
The following instructions assume that you have only a command line environment and you have a working network connection.
1. At first, you need to download an ISO image of your Linux distribution of choice. There are usually two options: download ISO image from a centralized server or download ISO image using torrent protocol. In the following steps, we download Xubuntu 16.04.3 LTS (x86_64).
a. Centralized server:
- requires that *wget command* exists in the system
- download the ISO image with the command:
```
wget --directory-prefix=$HOME --show-progress http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/xubuntu-16.04-desktop-amd64.iso
```
b. Torrent download:
- requires that *transmission-cli command* exists in the system
- download the ISO image with the command:
```
transmission-cli -w $HOME http://cdimage.ubuntu.com/xubuntu/releases/16.04.3/release/xubuntu-16.04-desktop-amd64.iso.torrent
```
2. Plug in an empty and large enough USB stick into your computer (4GiB should be sufficient). It is highly recommended that you have formatted the stick in advance but you can format the stick with *fdisk* command, as well ([util-linux](https://www.kernel.org/pub/linux/utils/util-linux/)). When using fdisk, please follow [these instructions](https://www.cyberciti.biz/faq/linux-how-to-delete-a-partition-with-fdisk-command/), for example. In case you have important files to be copied from your USB stick, copy the files with the following command sequence. We assume that the stick is not yet mounted on the system. Command sequence is expected to be executed with the default user (UID = 1000) which belongs to the default user group (GID = 1000). Correct USB stick device name and partition number (*/dev/XdYZ*) must be find out in advance by executing command *sudo fdisk -l*. Y stands for the device's alphabetical ID, Z stands for partition number (*/dev/sda1*, for example). X can have value *h* (IDE-based HDD/SSD's) or *s* (SATA-based HDD/SSD's). Newer NVMe-based storage solutions such as Samsung EVO series 860/960 use syntax */dev/nvme0n1p1* in which 1 stands for the device number and *p1* stands for the partition number.
```
sudo mkdir -p /mnt/livestick && \
mkdir $HOME/livestick-backup && \
sudo mount /dev/sdXY /mnt/livestick && \
sudo mv /mnt/livestick/* $HOME/livestick-backup/ && \
sudo chown -R 1000:1000 $HOME/livestick-backup
```
**NOTE:** If you have any USB stick partitions mounted already on the system, you should **not** use */mnt/livestick** location in above commands. Instead, use the path you can extract for the correct USB stick partition with command *mount*.
3. When you have backed up your important files earlier stored in the USB stick, you have confirmed the USB stick is empty and you have successfully downloaded your Linux distribution ISO image, you can start proceeding with the writing operations.
**NOTE!** You must be extra careful with dd command. If used wrongly, the command can destroy any data you have in your HDD/SSD or in any storage device. Recovering files is a headache and you really don't want to end up having one! One key principle is to triple check all locations given for the dd command and check that used locations are in correct places (so that input and output operations are being executed for correct devices and the devices don't mix up).
4. Let's begin with Xubuntu 16.04.3 LTS USB writing operations. In this section we assume that you have successfully downloaded the ISO image and you definitely know the correct path for your USB stick device (syntax: /dev/sda). **NOTE!** I don't refer to USB stick partition path (syntax: /dev/sda1). The device path syntax is as stated: */dev/sda*, */dev/sdb*, etc. dd command is executed as the root user (Linux main user/super user). dd command doesn't confirm the command syntax is correct so you must write it correctly in the beginning. If you are not sure about the correct syntax, please contant an expert.
The command syntax (NOTE: not likely the exact command in your environment!) is as follows:
```
sudo dd status=progress if=$HOME/xubuntu-16.04-desktop-amd64.iso of=/dev/sdb
```
where /dev/sdb is your USB stick as seen on Linux system. **Use correct, your exact device identifier instead of the one shown above**.
If you want more instructions how to use the 'dd' command, it is recommended to read the following wiki, for example: [Wikipedia - dd (Unix)](https://en.wikipedia.org/wiki/Dd_(Unix))
5. Wait until the ISO image file has been written into the USB stick. Once done, reboot your computer with the USB stick attached. Immediately the computer has rebooted, press a button which you can access your BIOS/UEFI/Boot options menu with. Choose the Xubuntu 16.04.3 LTS USB stick (which should appear in the boot list). Start the operating system boot-up process bt selecting *Try without installing* option (this step varies between Linux distribution. More generally, you can also choose *Install <whateverdistro>* option and select installation on the live USB Linux desktop environment). Wait until the operating system has been booted up. Start installation and follow the instructions.
The installation process is not defined in detail here because installation steps varies from Linux distribution to distribution. However, basic principle is to install the operating system with good and strong passwords. Encrypted home folder or HDD/SSD partition options should also be considered. After you have successfully installed Linux, detach the live USB stick and reboot your computer. You should be able to access the installed Linux distribution without the USB stick.
When writing and applying passwords, please pay attention to correct keyboard layouts. In my case, I use a Swedish (Finnish) keyboard layout which differs from the default en_US layout. Therefore, mistyping characters can easily happen. Many Linux distributions allow you to change the keyboard layout during the installation process. Unless you're not able to do it in installation window, you can change the keyboard layout temporarily by executing *setxkbmap fi* in a local shell environment (NOTE: graphical environment is required since the command is a part of graphical X11 environment!). You can open up a local shell by opening a terminal emulator while running Linux from your USB stick. NOTE: changing passwords must be done before writing them in the installation steps!
**b)** List hardware components of your new Linux computer (*sudo lshw -short -sanitize*).
--------------
**Answer:** *sudo lshw -short -sanitize* gives the following information (assuming that you have package *lshw* installed and proper commands *sudo* and *lshw* are available on the system):
Detailed comments about the listed devices have been included after each terminal output line (therefore, the output is not exactly as written by the command to stdout/output).
H/W path Device Class Description
- system N56JR (ASUS-NotebookSKU)
- /0 busN56JR
**Comment:** *Computer manufacturer & model identifier*
- /0/0 memory 64KiB BIOS
**Comment:** *Memory size of motherboard's BIOS firmware, 64 kibibytes*
- /0/8 processor Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz
**Comment:** *Processor manufacturer, model and clock rate*
- /0/8/9 memory 1MiB L2 cache
**Comment:** *Level 2 cache of the processor, size of 1 mebibytes*
- /0/8/a memory 256KiB L1 cache
**Comment:** *Level 1 cache of the processor, size of 256 kibibytes*
- /0/8/b memory 6MiB L3 cache
**Comment:** *Level 3 cache of the processor, size of 6 mebibytes*
- /0/c memory 16GiB System Memory
**Comment:** *Installed computer RAM memory size in gibibytes*
- /0/c/0 memory 8GiB DIMM DDR3 Synchronous 1600 MHz (0.6 ns)
**Comment:** *Information of RAM module 0. DIMM (dual in-line memory module), DDR3 (Double Data Rate 3), clock rate 1600 megahertz, latency 0.6 nano seconds*
- /0/c/1 memory DIMM [empty]
**Comment:** *Information of RAM module 1. Empty*
- /0/c/2 memory 8GiB DIMM DDR3 Synchronous 1600 MHz (0.6 ns)
**Comment:** *Same than RAM module 0 desribed above*
- /0/c/3 memory DIMM [empty]
**Comment:** *Information of RAM module 3. Empty*
- /0/100 bridge Xeon E3-1200 v3/4th Gen Core Processor DRAM Controller
**Comment:** *Memory bus controller information. Processor family Xeon E3-1200. So called main controller for any device below in tree*
- /0/100/1 bridge Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller
**Comment:** *PCIe x16 bus controller*
- /0/100/1/0 displayGK106M [GeForce GTX 760M]
**Comment:** *Graphics Card Gerforce GTX 760M, belonging to PCIe x16 bus*
- /0/100/2 display4th Gen Core Processor Integrated Graphics Controller
**Comment:** *Secondary Graphics adapter (integrated).*
- /0/100/3 multimedia Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller
**Comment:** *Audio controller belonging to Intel Xeon E3-1200 family*
- /0/100/16 communication 8 Series/C220 Series Chipset Family MEI Controller #1
**Comment:** *[Intel Management Engine Interface -kontrolleri](https://www.kernel.org/doc/Documentation/misc-devices/mei/mei.txt)*
- /0/100/1a bus8 Series/C220 Series Chipset Family USB EHCI #2
**Comment:** *[Enhanced Host Controller Interface](https://en.wikipedia.org/wiki/Host_controller_interface_(USB,_Firewire)) of Intel C220- chipset family number 2 for USB gateways*
- /0/100/1a/1usb1busEHCI Host Controller
**Comment:** *Host controller for the above mentioned device*
- /0/100/1a/1/1 busUSB hub
**Comment:** *[USB extension hub](https://en.wikipedia.org/wiki/USB_hub) splitted to multiple gates*
- /0/100/1a/1/1/1communication Bluetooth USB Host Controller
**Comment:** *Bluetooth controller*
- /0/100/1a/1/1/3multimedia ASUS USB2.0 Webcam
**Comment:** *ASUS USB-based web camera*
- /0/100/1b multimedia 8 Series/C220 Series Chipset High Definition Audio Controller
**Comment:** *Audio controller of C220 chipset*
- /0/100/1c bridge 8 Series/C220 Series Chipset Family PCI Express Root Port #1
**Comment:** *Port which operates at motherboard's North Bridge, making it possible to use PCIe devices, to track their hierarchy and opening up communication channels between them ([source](https://www.techwalla.com/articles/what-is-a-pci-express-root-port))*
- /0/100/1c.2bridge 8 Series/C220 Series Chipset Family PCI Express Root Port #3
**Comment:** *Equal to the previous device*
- /0/100/1c.2/0 wlan0 networkAR9485 Wireless Network Adapter
**Comment:** *WLAN network adapter Atheros AR9485. Interface identifier wlan0 which is seen by the OS commands (can be altered via changing Linux kernel boot parameters in grub/syslinux configuration files)
- /0/100/1c.3bridge 8 Series/C220 Series Chipset Family PCI Express Root Port #4
- /0/100/1c.3/0 eth0networkRTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
**Comment:** *Ethernet controller RealTek 8111/8168/8411. Interface identifier eth0, which can be checked by using *ifconfig command*, for example
- /0/100/1c.4bridge 8 Series/C220 Series Chipset Family PCI Express Root Port #5
- /0/100/1c.4/0 genericRTS5227 PCI Express Card Reader
**Comment:** *RealTek RTS5227 PCIe card reader*
- /0/100/1d bus8 Series/C220 Series Chipset Family USB EHCI #1
**Comment:** *[Enhanced Host Controller Interface](https://en.wikipedia.org/wiki/Host_controller_interface_(USB,_Firewire)) of Intel C220 chipset family number 1 for USB gateways*
- /0/100/1d/1usb2busEHCI Host Controller
**Comment:** *Host controller for the above mentioned device*
- /0/100/1d/1/1 busUSB hub
**Comment:** *[USB extension bus](https://en.wikipedia.org/wiki/USB_hub) splitted to multiple gates*
- /0/100/1d/1/1/2busHighSpeed Hub
**Comment:** *Centralization of high speed bus*
- /0/100/1d/1/1/2/3 input USB-PS/2 Optical Mouse
**Comment:** *Optical HID mouse device attached to USB bus*
- /0/100/1d/1/1/2/4 input Logitech Illuminated Keyboard
**Comment:** *HID keyboard device attached to USB bus*
- /0/100/1f bridge HM86 Express LPC Controller
**Comment:** *[Low Pin Count controller](https://en.wikipedia.org/wiki/Low_Pin_Count) which attaches all low-level legacy devices to the motherboard*
- /0/100/1f.2storage8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode]
**Comment:** *SATA controller of C220 chipset family for SATA-based mass media storage inputs*
- /0/100/1f.3bus8 Series/C220 Series Chipset Family SMBus Controller
**Comment:** *Power management [System Management Bus controller](https://en.wikipedia.org/wiki/System_Management_Bus)*
P.S. If you frequently listed your hardware on a Linux computer, you should consider creating a *bash alias command* for this purpose (assuming your default shell environment is bash). Basic idea is to simplify commands you need to type each time you want to do something. For listing hardware, write a the following in your _$HOME/.bashrc_ file (it's a hidden file):
```
alias listhw='sudo lshw -short -sanitize'
```
Next time you launch a bash shell session, you should be able to list your hardware by simply executing command *listhw*, defined in your _$HOME/.bashrc_ file.
If you want to create this command globally accessible for all users, you can write the above code in /etc/bash.bashrc file as root. Alternatively, create a file _/usr/bin/local/listhw_ as root with the following contents:
```
#!/bin/bash
sudo lshw -short -sanitize
```
Use command _sudo nano _/usr/bin/local/listhw_ for that.
The file is not executable by default. In order to make it executable, apply the following command:
```
sudo chmod +x /usr/bin/local/listhw
```
Alternative and additional commands to list hardware are *lspci* (list PCI devices), *hwinfo* (defailed output), *lsusb* (list USB devices). Availability of these commands vary and you may need to install extra packages in your Linux distribution. On Ubuntu, execute the following command:
```
sudo apt-get update && sudo apt-get install hwinfo pciutils usbutils
```
**c)** Install three new programs which are new to you. Try them out in their main target environment and primary use.
--------------
**Answer:**
Let's install three new programs: *[Thunar](http://thunar.xfce.org/)* (file manager), *[Gedit](https://wiki.gnome.org/Apps/Gedit)* (text editor), *[GParted](http://gparted.sourceforge.net/)* (mass storage partition editor, alternative [GitHub link](https://github.com/GNOME/gparted)
)
**Thunar:**
![Thunar](https://winaero.com/blog/wp-content/uploads/2017/09/Thunar-in-XFCE4.png)
Thunar is a user-friendly and easily understandable [GTK+-based](https://en.wikipedia.org/wiki/GTK+) file manager. It's not as customizable as [Qt-based](https://en.wikipedia.org/wiki/Qt_%28software%29) [Dolphin](https://www.kde.org/applications/system/dolphin/) but sufficient for daily basic usage. Thunar is the default file manager for [XFCE desktop environment](https://en.wikipedia.org/wiki/Xfce).
**Gedit:**
![Gedit](http://mattbaker.eu/imgs/2017-08/10686759591_gedit-editor-for-windows.jpg)
Gedit is a simple GTK+-based text editor, targeted to [GNOME desktop environment](https://en.wikipedia.org/wiki/GNOME). Alternative text editors are, for example, [Mousepad](https://github.com/codebrainz/mousepad), [Kate](https://www.kde.org/applications/utilities/kate/), [KWrite](https://www.kde.org/applications/utilities/kwrite/) and [Geany](https://en.wikipedia.org/wiki/Geany).
**GParted:**
![GParted](http://intodarkness.eu/imgs/2017-08/23065514301_gparted-live-0-9-1-1-iso.jpg)
GParted is a popular GTK+-based graphical front-end for CLI-based, GPL3-licensed [GNU Parted](http://www.gnu.org/software/parted/parted.html) back-end. Similar programs include a Qt-based [KDE Partition Manager](https://www.kde.org/applications/system/kdepartitionmanager/), targeted mainly to [KDE](https://en.wikipedia.org/wiki/Kde) and [LXQt](https://en.wikipedia.org/wiki/LXQt) desktop environments.
**Installation of Thunar, Gedit and GParted on Debian-based distributions**
**Via official package repositories:**
On Debian-based distributions (like Xubuntu), you can install the selected three programs with a single command, like:
```
sudo apt-get update && sudo apt-get install -y thunar gedit gparted
```
where _sudo apt-get update_ stands for "Update and synchronize package database list to correspond with newer available program versions", _&&_ stands for "If the previous command was successful, proceed with the following command", _sudo apt-get install -y_ stands for "Install the following packages without confirming (-y parameter)", and finally, _thunar gedit gparted_ stands for "Application names found in the available repositories listed in _/etc/apt/sources.list_ file and in files in _/etc/apt/sources.list.d/_ directory and which are parameters for _apt-get_ command".
_sudo_ part before the forementioned commands stands for "Grant me temporary root privileges to execute the following commands as root". It is essential that the current user is in _sudo_ group and all members of group _sudo_ are allowed to gain temporary root privileges (uncomment line _%sudo ALL=(ALL) ALL_ in file _/etc/sudoers_ as root).
By executing the above command, we assume that the system has established internet connection, [Dpkg](https://en.wikipedia.org/wiki/Dpkg) works as intended (it has not already locked file */var/lib/dpkg/lock*, for example) and no other installation process is being executed on.
You can execute the command without sudo in root terminal, but accessing root terminal is not highly recommended in server environments. In root terminal, the command would be:
```
apt-get update && apt-get install -y thunar gedit gparted
```
You can search for any packages with *apt-cache search package-name* command. It may not be the primary approach but you can search for packages on official [Ubuntu repositories website](https://packages.ubuntu.com/) on Linux Ubuntu distributions (Xubuntu, Ubuntu, Lubuntu, Kubuntu, Ubuntu Bugdie and so on). Alternatively, you can also install packages with *aptitude* command if you have installed package *aptitude*. Command syntax for installing packages with aptitude is
```
sudo aptitude install package-name
```
**Via local packages or from source code:**
If you consider installing local packages, keep in mind that you need to track all required package dependencies. This can easily lead to [Dependency hell](https://en.wikipedia.org/wiki/Dependency_hell) situation. Therefore, installing local packages suits best for commercial, separately distributed software which are bundled with their dependencies and can be installed out-of-the-box without worrying about dependencies.
All installable packages on Debian-based Linux distributions are known as _deb packages_ with suffix _.deb_. Therefore, local packages can be installed with the following command:
```
sudo dpkg -i my-deb-package.deb
```
If any unsatisfied dependencies are present, you should do as usually instructed:
```
sudo apt-get -f install
```
Please note that this does not always work, can break your system and make things just too troublesome to sort out. For example, your program may depends on a previous version of a package which is already installed or your system or/and not available on the official repositories. Installing a previous version can seriously break your system. In some rare cases, packages can be offered with two versions where the older version is known as _compatibility package_, usually installing older library versions to your system, co-existing with newer ones. However, keep in mind that your _.deb_ package must be aware of this _compatibility version_ (so that it depends exactly on that compatibility version instead of the officially named one. This is just a naming issue).
You can force package installation on the system, ignoring package dependencies. In this case, you **must** know what you're doing, and this approach is really not recommended for Linux novices.
If you want to compile software directly from source, you essentially need *[dh_make](https://manpages.debian.org/jessie/dh-make/dh_make.8.en.html)* and *[dpkg-buildpackage](http://man7.org/linux/man-pages/man1/dpkg-buildpackage.1.html)* commands available on your Debian-based operating system. Usually *[cmake](https://cmake.org/)*, *[make](https://en.wikipedia.org/wiki/Make_(software))* and *gcc* ([GNU Compiler Collection](https://gcc.gnu.org/)) are required as well. Not forgetting any extra _build time dependencies_, usually marked with _-dev_ suffix (development headers) in Debian-based Linux distributions. _Runtime dependencies_ can be described in package description.
Any package dependencies (buildtime or runtime) are defined in Debian control file which can be supplied with the package or you need to create and configure it by your own.
On Ubuntu, you can install most essential and widely used build time tools with:
```
sudo apt-get install build-essential
```
if you considered installing a remotely available package directly from source, you should find out required runtime and buildtime dependencies with (assuming you have aptitude installed):
```
aptitude show package-name
```
where package-name is the package of your choice. Replace this string with the correct name,
Additionally, you can investigate required dependencies by checking any header (_.h_) files required in C/C++ source files, if the program is based on these languages. By tracking down required headers it is possible to figure out required dependencies in some extent. In addition, check out official websites of the program (GitHub, BitBucket or another homepage), possibly contant the developer and/or check out any supplied library files (suffix is _.so_) and their dependencies with [ldd command](https://en.wikipedia.org/wiki/Ldd_%28Unix%29). On Ubuntu, you can also check out [packages.ubuntu.com](https://packages.ubuntu.com/) for dependencies if the package is available on official repositories.
Please see my practical [example 1](PLACEHOLDER) and [example 2]() for compiling software on Linux.
You can also apply patches directly to source code of a program by using [diff](https://www.gnu.org/software/diffutils/manual/html_node/Example-Unified.html) and [patch](https://en.wikipedia.org/wiki/Patch_%28Unix%29) commands. Patches must be applied before compiling the program.
**Removal of packages:**
You can effectively remove packages on Debian-based Linux distributions with:
```
sudo apt-get purge --remove package1 package2 package3
```
In some cases, package manager suggests you to remove any orphaned packages with:
```
sudo apt-get autoclean
```
You can freely explore more _apt-get commands_ with:
```
apt-get --help
```
and by reading command-related manual page:
```
man apt-get
```
Manual pages in any commands are highly supported in multiple Linux distributions. The command syntax is always the same: _man my_command_. Additionally, _--help_ paremeter is widely accepted by many Unix/Linux CLI programs.
**NOTE:** Although Linux distributions don't have any similar to Windows registry hell (unclean registry after software removal), keep in mind that any configuration files created _afterwards_ by a Linux program are not handled by Linux package managers. In other words: package managers can't track files created afterwards by your program. This is mostly a case with personal configuration files in user's home directory on Linux. In addition, any backup files (suffix: .old, ~, .bak, .pacnew...) are not included in the package according to your package manager and, therefore, they may not be uninstalled by simply removing a program. [Please see my script for tracking down those files]()
**Other Linux distributions:**
Main differences between various Linux distributions are:
- version modeling: For example, Arch Linux follows [rolling release model](https://en.wikipedia.org/wiki/Rolling_release) whereas Ubuntu follows [time based release model](https://wiki.ubuntu.com/TimeBasedReleases)
- package managers: OpenSUSE uses [zypper](https://en.opensuse.org/Portal:Zypper), Fedora uses [dnf](https://lists.opensuse.org/opensuse-factory/2017-08/msg00369.html), Arch Linux uses [pacman](https://wiki.archlinux.org/index.php/pacman)
- package configurations (configuration files, features available for a program by default)
- compilation methods for source codes: For example, Arch Linux uses [makepkg](https://wiki.archlinux.org/index.php/makepkg), Ubuntu uses dpkg-buildpackage
- package availability in official repositories: For example, some packages available on Arch Linux are not available on Ubuntu
- package versioning syntax: For example, Arch Linux uses version 59.0.1-1 and Ubuntu uses 59.0.1 for a same package
- package versions: For example, Arch Linux has newer package version in the official repositories than Ubuntu does.
You can read more at [Arch Linux Wiki - Arch compared to other distributions](https://wiki.archlinux.org/index.php/arch_compared_to_other_distributions).
**d)** What licenses have been used in the programs of your choice? Explain briefly which kind of rights and duties these licenses have.
--------------
**Answer:**
License information can be retrieved from multiple various sources. Licensing terms have usually been mentioned in source code or documentation, help manual, program website or retriavable from equivalently trusted source.
- Thunar is a part of Xfce project and licensed under GPL ([GNU General Public License](https://www.gnu.org/licenses/licenses.html#GPL)) and LGPL2.1 ([Lesser GNU General Public License](https://www.gnu.org/licenses/licenses.html#LGPL)). Detailed information about the licensing policy can be found on [Arch Linux package repository](https://www.archlinux.org/packages/extra/x86_64/thunar/), for example.
- Gedit is a part of GNOME project and licensed under GPLv2. Licensing terms are mentioned in [Gedit source code](https://git.gnome.org/browse/gedit/tree/COPYING).
- Gparted is a part of GNOME project and uses GPLv2+ license, according to [the official website](http://gparted.sourceforge.net/), bottom section. Exact statement is as follows: "GParted is distributed under the GNU General Public License version 2 or (at your option) any later version."
GPL license is targeted to distribute free sofware under several terms. The license grants permission to use, copy, modify and share a program and its source code. License is accumulating in nature so that any program initially released under GPL and developed further source code must be published and any further restrictions to usage of the program are forbidden. License allows commercial usage of a program. More about the license on [GNU project homepage](https://www.gnu.org/licenses/quick-guide-gplv3.html)
[LGPL license](https://en.wikipedia.org/wiki/GNU_Lesser_General_Public_License) differs from GPL license so that some non-GPL licensed component of a program can be linked to otherwise GPL-licensed program. Therefore, because the program is not fully open source, it can still use LGPL license but not GPL license. LGPL licensed program has less efficient [copyleft](https://en.wikipedia.org/wiki/Copyleft).
**e)** List programs you use (MS Word, for example), purpose of these programs (for example, text editing) and introduce equivalent open source Linux program (for example, LibreOffice Writer). List closed source programs even if they do not have open source equivalent for Linux.
--------------
**Answer:**
Syntax: Closed source program, open source program, purpose of the selected program
- Adobe Photoshop - *Open source:* Krita, GIMP = **Image manipulation**
- Adobe Lightroom - *Open source:* Darktable, RawTheRapee, digikam = **Photo editing**
- Microsoft Media Player - *Open source:* VLC, mplayer = **Media player**
- Notepad - *Open source:* Kate, Leafpad, gedit = **Text editor**
- Windows File Manager - *Open source:* Nautilus, Thunar, Dolphin = **File manager**
- Autodesk 3DS Max - *Open source:* Blender = **3D editor**
- Microsoft Paint - *Open source:* KolourPaint = **Rasterized image editing**
- Skype - *Open source:* Pidgin Internet Messenger = **Chat**
- Microsoft Windows - *Open source:* Wine = **Operating system (note! this can be debated. ReactOS, too)**
- Windows Task Manager - *Open source:* KSysGuard, LXTask = **Process and system statistics manager**
- Microsoft Outlook - *Open source:* Thunderbird, Kmail = **Email client**
- Windows Volume Control - *Open source:* PulseAudio Volume Control & alsamixer = **Audio controlling**
- Windows Control Panel - *Open source:* KDE System Settings (systemsettings5) = **General settings of an operating system**
- Adobe Acrobat - *Open source:* Evince = **PDF document viewer**
- Windows Image Viewer - *Open source:* Gwenview = **Photo previewing**
- Microsoft Internet Explorer - *Open source:* Firefox, Palemoon, Konqueror = **Internet browser**
- Microsoft Office, WPS Office - *Open source:* LibreOffice, OpenOffice, Calligra = **Set of programs to process office documents**
- Riegl RiScan Pro - *Open source:* CloudCompare, MeshLab = **LiDAR processing software**
- MathWorks MATLAB - *Open source:* FreeMat, GNU Octave = **Mathematical & physical calculations**
- Adobe Audition - *Open source:* Audacity = **Audio editor**
- Tux Racer = **Driving game (open source, mentioned just for fun...)**
- Bentley MicroStation = **City/Urban planning oriented CAD program (closed source, only for Windows)**
- Valve Steam = **Game publishing platform (closed source, available for Linux, too)**
- Pitney Bowes MapInfo Professional = **Map and GIS planning (closed source, only for Windows)**
- IrfanView = **Image previewer (closed source, only for Windows)**
- etc...
**f)** Optional extra task: backup your files (you can use an external USB storage, for example)
--------------
**Answer:** Depending on the final goal/purpose, alternative methods can be used. Creating a backup of whole HDD/SSD's is possible with a single dd command (recommended parameter 'status=progress' with large partitions) and with a USB live stick so that this partition/HDD/SSD is not mounted. More sophisticated tools include [Clonezilla](https://en.wikipedia.org/wiki/Clonezilla), for example. It is recommended to backup a server with slightly different methods.
If the purpose is to copy single files from a location to another location, use *cp* or *rsync* command.
```
cp -Rp /mnt/my_hdd1/somefilefolder/* /mnt/my_hdd2/backupfolder/
```
where -R stands for "recursive copy" and "-p" stands for "preserve file & folder permissions". Asterix symbol (*) stands for "include all files".
Rsync:
```
rsync -axHAWXS --numeric-ids --info=progress2 /mnt/my_hdd1/somefilefolder/ /mnt/my_hdd2/backupfolder/
```
**NOTE:** If you copy files between ext2/3/4 and NTFS file systems, keep in mind that any permissions stored in ext2/3/4 partition are not preserved in NTFS partition. NTFS is widely used on Microsoft Windows environments whereas ext2/3/4 are widely used on Unix-based (POSIX) environments.
In addition, you must consider any [symbolic links](https://en.wikipedia.org/wiki/Symbolic_link) included in copy process. Keep in mind to watch out user permissions as well (does the current user have permissions to copy selected files?).

+ 316
- 0
exercises/h2.md View File

@ -0,0 +1,316 @@
Linux servers - Exercice 2
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Create two different log events: One successful event and one failed or forbidden event. Analyze the log lines in detail.
--------------
**Answer:**
**Successful event example - Apache server**
```
phelenius@my-machine:~$ cat /var/log/apache2/access.log
127.0.0.1 - - [24/Jan/2018:19:14:47 +0200] "GET / HTTP/1.1" 200 3525 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1 (KHTML, like Gecko) Version/11.0 Safari/605.1 Ubuntu/16.04 (3.18.11-0ubuntu1) Epiphany/3.18.11"
127.0.0.1 - - [24/Jan/2018:19:14:47 +0200] "GET /icons/ubuntu-logo.png HTTP/1.1" 200 3623 "http://localhost/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1 (KHTML, like Gecko) Version/11.0 Safari/605.1 Ubuntu/16.04 (3.18.11-0ubuntu1) Epiphany/3.18.11"
```
**Analysis:** In this event, user's client program (Web browser) has tried to access site *[localhost](https://en.wikipedia.org/wiki/Localhost)* for which Apache web server (HTTP daemon) has responded with OK code. The log snap reveals
- client's IP address, in this case it is the same computer (*localhost* / *127.0.0.1*)
- user ID (just a line in this case)
- logging time
- used HTTP method ([GET](https://www.w3schools.com/tags/ref_httpmethods.asp))
- retrieved contents, using server defined root directory as root path (*/* and */icons/ubuntu-logo.png*)
- code 200 ([HTTP_OK](https://ci.apache.org/projects/httpd/trunk/doxygen/group__HTTP__Status.html#ga02e6d59009dee759528ec81fc9a8eeff))
- target size 3623 (reported to the client)
- [HTTP Referer](https://en.wikipedia.org/wiki/HTTP_referer) (client)
- [User agent header](https://en.wikipedia.org/wiki/User_agent) reported by the client. According to the agent string, client browser has been gecko-based [Epiphany web browser](https://en.wikipedia.org/wiki/Epiphany_(GNOME)), using x86_64 processor architecture. This string can be manipulated in the client end. For example, a desktop client web browser can pretend to be a mobile browser.
Default syntax for Apache log files follow the layout which is described [here](https://httpd.apache.org/docs/2.4/logs.html) under section 'Common Log Format'.
**Failed event example - Apache server**
```
phelenius@my-machine:~$ cat /var/log/apache2/access.log
127.0.0.1 - - [24/Jan/2018:22:30:50 +0200] "GET /this-page-dont-exist HTTP/1.1" 404 510 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1 (KHTML, like Gecko) Version/11.0 Safari/605.1 Ubuntu/16.04 (3.18.11-0ubuntu1) Epiphany/3.18.11"
127.0.0.1 - - [24/Jan/2018:22:30:50 +0200] "GET /favicon.ico HTTP/1.1" 404 500 "http://localhost/this-page-dont-exist" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/605.1 (KHTML, like Gecko) Version/11.0 Safari/605.1 Ubuntu/16.04 (3.18.11-0ubuntu1) Epiphany/3.18.11"
```
**Analysis:** In this event, user's client program (Web browser) has tried to access site *http://localhost/this-page-dont-exist* for which Apache web server (HTTP daemon) has responded with error code. The log snap reveals
- client's IP address, in this case it is the same computer (*localhost* / *127.0.0.1*)
- user ID (just a line in this case)
- logging time
- used HTTP method ([GET](https://www.w3schools.com/tags/ref_httpmethods.asp))
- retrieved contents, using server defined root directory as root path (*/this-page-dont-exist* and */favicon.ico*)
- error code 404 ([HTTP_NOT_FOUND](https://ci.apache.org/projects/httpd/trunk/doxygen/group__HTTP__Status.html#gabd505b5244bd18ae61c581484b4bc5a0))
- target sizes 500 & 510 (reported to the client)
- [HTTP Referer](https://en.wikipedia.org/wiki/HTTP_referer) (client)
- [User agent header](https://en.wikipedia.org/wiki/User_agent) reported by the client. According to the agent string, client browser has been gecko-based [Epiphany web browser](https://en.wikipedia.org/wiki/Epiphany_(GNOME)), using x86_64 processor architecture. This string can be manipulated in the client end. For example, a desktop client web browser can pretend to be a mobile browser.
Error logs of an Apache server can be found (by default) at _/var/log/apache2/error.log_. HTTP status codes with brief description are defined on [Apache homepage - Common HTTP Status Codes](https://wiki.apache.org/httpd/CommonHTTPStatusCodes).
**Other error situations - libraries & typos**
The most common errors I have encountered on Linux desktop usage as a system administrator are various [Permission denied events](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Step_by_Step_Guide/s1-navigating-ownership.html) mostly as a consequence of a human error (forgetting sudo or something similar) or various typos in commands. I have compiled a lot of programs from source code, making various library errors practically unavoidable in long-term. The syntax of these errors is: [Cannot open shared object file: No such file or directory](https://stackoverflow.com/questions/480764/linux-error-while-loading-shared-libraries-cannot-open-shared-object-file-no-s/480786) where an executable can't find a requested _.so_ library file or version from library path. Both of these errors can be avoided by typing commands precisely and avoiding compiling of software (and relying on official repositories and system-wide package upgrades, considering that installed packages are maintained actively).
**Other error situations - unsupported Linux OS versions & breaking libraries**
In long-term, using Linux distributions such as Ubuntu (or any variant that relies on Ubuntu package repositories, such as Linux Mint) become unusable with newer software. The newer versions require more recent library versions which older Ubuntu releases can't possibly provide anymore because official repositories have been shutted down. In this state, either older software must be used (not recommended) on older operating system or the system must be upgraded to more recent version where the newer packages are still available. In some extent, compiling programs from source on an older operating system version is possible and thus life cycle of an old operating system installation can be extended for some programs. This approach, however, leads easily to security risks and bugs which have not been fixed, and especially the security part is essential on Linux server installations.
I have personal experience on using older Ubuntu 10.04 LTS after the official support dropped down. This operating system was running on desktop use, not in server environment. Thus, I accepted some security risks that may have been present in this approach. In software level, keeping the old Ubuntu installation led to major conflicts with libraries required by modern programs. I had older versions of some of these libraries installed but the newer versions introduced some major changes in function calls and methods so that compiling modern programs against these libraries was practically almost impossible (without major code patches). The most common library which broke first was [glibc](https://www.gnu.org/software/libc/). Ironically, some modern programs I couldn't use ran via [Wine](https://www.winehq.org). That's right: I couldn't use Linux versions but Windows versions ran flawlessly, on my Ubuntu 10.04 LTS.
Since then, I have moved from Ubuntu to Arch Linux long time ago. Major reason for this is the rolling release model which Arch Linux uses. Compiling software from source is more flexible and, in theory, I don't have to reinstall my operating system never again due to "unsupported operating system version" issues.
Based on my personal experience, rolling release model is not trouble free and can easily cause program/library conflicts but resolving these conflicts is flexible for an advanced Linux user. As an Arch Linux user, I must be aware of various conflicts which different updates and packages can cause, and I need to know it before I do anything crucial on my operating system. This means that updating software must never be done blindly and I need to be able to solve various error situations that might be present in the system. Usually these errors are poorly or misleadingly documented or not documented at all. Tracking down the root cause of some errors can be very tricky because of multiple components used by various programs.
In general, some troublesome situation on Arch Linux consist of transitions from older API versions to a newer one. This has happened me once: from Qt4 to Qt5. The issue here is this: some packages are updated more frequently than the others, leading to mixing of two APIs in a system and therefore resulting to broken user experience.
**Other error situations - a tricky error without clear identification**
For example, I recently bought a new laptop. I use Firefox on KDE 5 environment. My laptop has Optimus graphics. I have updated all Xorg packages, Mesa drivers, Nvidia drivers and Plasma workspace packages, not forgetting the web browser itself. In some cases, the browser just crashes, and not only the browser but the whole desktop and may lead to laptop freeze so that only hard-reset is a viable solution. In this case, I must try to figure out whether it is Optimus, Mesa, Nvidia, Plasma workspace (or any library in it) or browser component (add-on, compile parameters, hardware acceleration) which cause the crash. This issue is very poorly documented. My best bet to track down the issue is in this case is to check any logs, especially Xorg & Journalctl log events, check all browser add-ons and hardware acceleration, try to check if similar issues for related packages have been reported in various Bugzilla sites for specific package versions and if patches are available, try them out if needed. Reporting this kind of problem to developers is very troublesome because I must track down the exact component which actually causes the crash. Reporting the issue for random developers as "My plasma workspace/kwin crashes while using a browser with Optimus laptop" says practically nothing and may lead to frustation, ignorance and false assumptions. Therefore, I must pin down the root cause myself before any reporting.
**Other error situations - hardware issues**
Linux driver support. There are arguments forth and against. Is it good or not? Well, simple answer: it completely depends on used hardware. On my recently purchased laptop, at least Linux kernel 4.15 is needed. In older kernel versions, the laptop doesn't simply boot on Linux. Therefore, at least Ubuntu 18.04 LTS or any other distribution providing this kernel must be installed or I must compile this kernel version (or newer) from source to the target hardware by myself.
The main rule here: check which components you need support to (yes, components, individual hardware components), try to find out which Linux kernel module they may use or does the hardware vendor provide any driver support (or does the vendor generally have no interest in Linux in which case you can assume no driver support is available for their newer hardware models either unless implemented in Linux kernel or drivers are provided by open source community). In some cases, it's just pure trial-and-error. If you have physical access to the hardware and are permitted to try Linux out on it, your situation is good. In general, older the software, better chance for it to have Linux support for all hardware components.
If you buy very modern or customized hardware, please consider using a distribution with rolling release model.
**Other error situations - random errors**
A good friend of mine contacted me recently. He stated that he couldn't log in to his [Gentoo](https://en.wikipedia.org/wiki/Gentoo_Linux) installation. After a short talk on phone and various approaches I figured out that the problem must be on his LightDM manager so I adviced him to check his _/var/log/lightdm/_ log files. It was proven that the problem was caused by faulty installation of LightDM package files (wrong permissions) or authentication errors oin his [PAM policy](https://en.wikipedia.org/wiki/Privileged_identity_management). To conclude, all these errors can be very random and results of various reasons.
**Other error situations - a tricky error tracked down**
Sample of [journalctl](https://www.loggly.com/ultimate-guide/using-journalctl):
- Boot events logged (kernel, dbus, system daemon, other daemon processes, etc...)
- Each log event has a timestamp and the hostname (my-machine).
- In the next part, journalctl introduces the name of a system component responsible for the event (process/kernel).
- For each process, [process identifier](https://en.wikipedia.org/wiki/Process_identifier) is represented. This identifier is useful for sending various signals like SIGKILL/SIGSTOP to processes, for example ([SIGKILL/SIGSTOP...](https://en.wikipedia.org/wiki/Signal_(IPC))).
- After PID identifier, the message of the responsible component is represented. Erroneous messages are usually, but not always, marked as bolded red.
Log events can usually be found either on user's home directory (custom paths used by programs executed only with user permissions) or in system-wide directory _/var/log/_. For example, journalctl log events can be saved in _/var/log/journal_ directory whereas user's [X11](https://en.wikipedia.org/wiki/X_Window_System) errors are logged either in _$HOME/.xsession-errors_ ([.xsession-errors](https://stackoverflow.com/questions/10222449/what-is-xsession-errors)) file or in _$HOME/.local/share/Xorg/_ directory. Some programs use simply user's home directory as their log directory (for example: bash shell logs user history in hidden _$HOME/.bash_history_ file).
The following journalctl snapshot tells that there is an issue with the bluetooth daemon (used computer is a laptop known as Asus N56JR).
```
...
1 Jan 24 17:14:04 my-machine systemd[1]: Started LSB: daemon to balance interrupts for SMP systems.
2 Jan 24 17:14:04 my-machine kernel: Bluetooth: BNEP (Ethernet Emulation) ver 1.3
3 Jan 24 17:14:04 my-machine kernel: Bluetooth: BNEP filters: protocol multicast
4 Jan 24 17:14:04 my-machine kernel: Bluetooth: BNEP socket layer initialized
5 Jan 24 17:14:04 my-machine dbus[918]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
6 Jan 24 17:14:04 my-machine bluetoothd[916]: Bluetooth management interface 1.10 initialized
7 Jan 24 17:14:04 my-machine bluetoothd[916]: Failed to obtain handles for "Service Changed" characteristic
8 Jan 24 17:14:04 my-machine whoopsie[985]: [17:14:04] Using lock path: /var/lock/whoopsie/lock
9 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
10 Jan 24 17:14:04 my-machine bluetoothd[916]: Error adding Link Loss service
11 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
12 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
13 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
14 Jan 24 17:14:04 my-machine bluetoothd[916]: Current Time Service could not be registered
15 Jan 24 17:14:04 my-machine bluetoothd[916]: gatt-time-server: Input/output error (5)
16 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
17 Jan 24 17:14:04 my-machine bluetoothd[916]: Not enough free handles to register service
18 Jan 24 17:14:04 my-machine bluetoothd[916]: Sap driver initialization failed.
19 Jan 24 17:14:04 my-machine bluetoothd[916]: sap-server: Operation not permitted (1)
20 Jan 24 17:14:04 my-machine cron[991]: (CRON) INFO (Running @reboot jobs)
21 Jan 24 17:14:04 my-machine systemd[1]: Starting Hostname Service...
...
```
Bluetooth error messages: lines 7-19. Line 8 is not an error message.
**journalctl analysis - regression test approach**
The problem seem to be widely known among Ubuntu & Arch Linux users, according to a short search in the internet. There is a [launchpad report](https://bugs.launchpad.net/ubuntu/+source/bluez/+bug/1490349) but not a simple answer is given there. There is a suggestion to downgrade bluez package on [Arch Linux forum](https://bbs.archlinux.org/viewtopic.php?id=195886). [AskUbuntu](https://askubuntu.com/) website has many threads for this issue.
This kind of problems must be examined carefully. Preferred method, as the software component has been tracked down, is regression test for this component. In regression test, the commit or commits which break the desired functionality are hunted down. Tricky part in regression test is to do it as soon as the problem has been identified: more the working package version and faulty one differ from each other, more effort is required to pin down the commits.
Another dilemma are various dependencies: in this case, we can blame bluez package. But is it actually bluez which broke the functionality? Or is it a dependency required by bluez package? Or a dependency of a dependency? If tracking down the root cause of a problem lead to this situation, it's like looking for a correct path in a giant tree or a dense bush. One crucial examination can still be done: is the problem present in some other program, too? If it is, then we can exclude all other packages from our search instead of common packages required by two individual programs. Of course, this is not foolproof either because program A can depend only on small code part of a dependency package (simple function call) whereas program B can utilize many more functions of the same dependency package.
Basic principle of a regression test is to compile suspected packages from source, two different versions: one where the problem is present, and another one where it is not. We continue compiling these two versions as long as we can pick up individial commits which likely cause the breakage in program functionality. To put it simple: it is an iteration process. [Git bisect](https://git-scm.com/docs/git-bisect) method provides a way to do this, for example.
Once the problematic commit/commits are known, we can fix the code ourselves and, for projects where the code is available to use, we can make a [pull request](https://help.github.com/articles/about-pull-requests/) which is accepted by verified developer, or in some cases, we can make our own [fork repository](https://confluence.atlassian.com/bitbucket/forking-a-repository-221449527.html) of the package/component/program. Therefore, commiting [a patch](https://www.thegeekstuff.com/2014/12/patch-command-examples/) or patchset to source code is possible.
**journalctl analysis - our bluez package**
In bluez error case, we should check all relevant error messages written by bluetooth daemon to STDERR output. By analysing [BlueZ source code](https://github.com/pauloborges/bluez) ([alternative link](https://git.kernel.org/pub/scm/bluetooth/bluez.git)) we can see that the message present in journalctl log snapshot can be found in source code as well. In more detail, the following source code files (C code):
```
phelenius@my-machine:~/bluez-master$ grep -Ril "Not enough free handles to register service"
profiles/proximity/reporter.c
plugins/gatt-example.c
attrib/gatt-service.c
```
Tracking down the problem even further can require analysis of the relevant parts of these files in addition to regression testing. Of course, hardware components where the error is present must be taken into account, too.
**Other error situations - A simple test case (command which succeeds as root but fails otherwise)**
The following command leads to an error situation (STRERR) if executed with normal user rights. If you run this command in root shell, it succeeds (STDOUT). Switch to root shell by executing _su root_ in your shell environment.
**NOTE!** If you use root shell for running any script, be **aware** and be **always** sure what you are about to do. You can log out from root shell with _exit_ command. Do not run scripts from untrusted sources as root.
```
for i in {0..4}; do touch $HOME/$i; mv $HOME/$i /usr/ 2>/dev/null && echo "Okay, you can do that. You are the god of the system" || echo "Sorry, no way. Do not even try that. You are not a god on \"$(hostname)\"" && if [[ -f $HOME/$i ]]; then rm $HOME/$i; fi ; done
```
Command description:
1. Command creates five individual files in a foor loop (0, 1, 2, 3, and 4) in the current user's home directory. For root, the folder path is simply _/root_
2. For each created file, move it from the $HOME directory to system folder _/usr_. In a case of error, write default error message (STDERR) to _null_ device _/dev/null_ which basically means that we don't print the default error message. Number _2_ stands for STDERR in this shell environment. _>_ means "write to". The main idea behind this is that we don't want to see the default error message if command _mv_ fails. Instead, we handle error situations in ways described in steps 4 and 5.
3. For each created file, if _mv_ command succeeds (STDOUT), do as it's stated after _&&_ and before _||_ (echo "Okay..."). More about these syntaxes: [Bash Reference Manual](http://www.gnu.org/software/bash/manual/bashref.html#Lists)
4. For each created file, if _mv_ command fails (STDERR), execute as it's stated after _||_ symbols (echo "Sorry..."). Practically, you end up to this scenario if the current user don't have a home folder defined in $HOME variable or if the current user doesn't have writing permissions to the system folder _/usr_. The most common cause for error is the latter one (Permission denied) because the writing permission in _/usr_ directory is reserved only for _root_ user.
5. For each file, in error situation, there is a command call _$(hostname)_ included. This command call prints out the output of _hostname_ command which, in our case, is the computer name. [Craracter escapes](http://www.gnu.org/software/bash/manual/bashref.html#Escape-Character) (slashes _\_) must be used if we want to use double quotes inside double quotes of our _echo_ command. With escapes, we basically tell the _echo_ command that " symbol, if preceded by \, must be included in the clause itself and it is not the end point of our clause. Alternative writing method for the _echo_ command is as follows: echo 'my "clause"'. Note single quotes here. Here we tell for the _echo_ command that the clause starts and ends with single quotes and, therefore, we don't have to escape our double quotes. This is a basic principle for any bash shell commands, including _grep_ (for example: cp --help | grep -i "\-R") etc.
Why do we write _$(hostname)_ and not simply _hostname_? We simply use bash internal [Command Substitution feature](http://www.gnu.org/software/bash/manual/bashref.html#Command-Substitution).
6. For each file, we do our last part. We check whether the file exists in the current user's home directory. If the file ($HOME/$i) exists, remove it. We end up to this situation mostly because of failures in moving the created file into _/usr_ directory.
7. Because all steps 2-6 have been executed inside the for loop, we stop our loop sequence by writing simply _done_.
Semicolons (;) are used because the script is so called _one liner_ script, meaning that multiple bash commands are stacked together, not written into multiple lines. The script can also be written in multiple lines and saved into _myscript.sh_ file for further usage. Execution rights for the script file are been given with _chmod +x myscript.sh_ command in a case where user's [umask values](https://wiki.archlinux.org/index.php/umask) do not grant execution permission for a file by default (this is the default case). In addition, we must include a shell definition in the first line in our _myscript.sh_ file. Practically, the definition is some of the following strings: *#!/bin/bash, #!/bin/sh, #!/bin/zsh, #!/bin/tcsh, #!/bin/csh, #/bin/ksh*, etc. In our case, it is _#!/bin/bash_
More about bash shell functionality can be found here: [Bash Reference Manual](http://www.gnu.org/software/bash/manual/bashref.html). The site is highly recommended for anyone writing advanced bash scripts.
**b)** Optional task, not teached yet: Install SSH daemon. Try some of the following commands on your own SSH server: ssh-copy-id, sshfs, scp or git. (The easiest command might be scp: ‘scp foo.txt tero@example.com:’)
--------------
**Answer:**
Execute the following
```
sudo apt-get update && sudo apt-get install openssh-server openssp-sftp-server git sshfs && sudo systemctl start ssh.service && systemctl status ssh.service | grep "Active:"*
```
If SSH daemon service is currently active, we do as follows:
Let's execute a command sequence where we connect to Haaga-Helia's SSH server ([Secure Shell (developer Tatu Ylönen, 1995)](https://fi.wikibooks.org/wiki/SSH)) and upload a local file _nullfile_ to the SSH server's current user home directory. We create this local file with _touch nullfile_ command. In the following output, all critical address information is masked with character pattern XXXX.
```
phelenius@my-machine:~$ ssh XXXXXX@haaga-helia.fi
The authenticity of host 'haaga-helia.fi (XX.XX.XX.XX)' can't be established.
RSA key fingerprint is XXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'haaga-helia.fi,XX.XX.XX.XX' (RSA) to the list of known hosts.
XXXXXX@haaga-helia.fi's password:
Last login: Tue Oct 31 19:25:16 2017 from XX-XX-XX-XX-XXXX.fi
[XXXXXX@haaga ~]$ pwd
/u10/XXXXXX
[XXXXXX@haaga ~]$ exit
logout
Connection to haaga-helia.fi closed.
phelenius@my-machine:~$ touch nullfile
phelenius@my-machine:~$ scp nullfile XXXXXX@haaga-helia.fi:/u10/XXXXXX/
XXXXXX@haaga-helia.fi's password:
nullfile
100%0 0.0KB/s 00:00
phelenius@my-machine:~$ ssh XXXXXX@haaga-helia.fi
XXXXXX@myy.haaga-helia.fi's password:
Last login: Wed Jan 24 23:41:44 2018 from XX-XX-XX-XX-XXXX.fi
[XXXXXX@haaga ~]$ ls |grep nullfile
nullfile
[XXXXXX@haaga ~]$ exit
logout
Connection to haaga-helia.fi closed.
phelenius@my-machine:~$
```
Other local ssh commands (etc.) can be found with command *ls /usr/bin |grep ssh*
**c)** Create an apt-get command of your dreams: one single command or one-liner which installs your favorite applications.
--------------
**Answer:**
Let's install the following packages (with apt-get install) and equally, remove the following packages (with apt-get purge) and possible orphaned packages (with apt-get autoclean)
```
sudo apt-get update && sudo apt-get -y install \
apache2 \
wireshark-gtk \
fail2ban \
ettercap-text-only \
varnish \
mariadb-server \
mariadb-client \
php7.0 \
epiphany-browser \
openssh-server \
openssh-sftp-server \
phpmyadmin \
gnome-terminal \
git \
cmake \
gedit \
aptitude \
build-essential \
vlc \
gpaint \
transmission-cli \
wcalc && \
sudo apt-get -y purge --remove firefox firefox-locale-en mousepad xfce4-terminal catfish onboard gnome-mines gnome-sudoku && \
sudo apt-get autoclean
```
**d)** Install three new command line programs using your command line package manager. Try each of these programs in their target environment and purpose.
--------------
**Answer:**
Let's pick up the following CLI (command line) programs:
- *[lynx](https://en.wikipedia.org/wiki/Lynx_(web_browser)) - classic non-graphical (text-mode) web browser*
- *[libimage-exiftool-perl](https://en.wikipedia.org/wiki/Exiftool) - library and program to read and write meta information in multimedia files*
- *[fakeroot](https://wiki.debian.org/FakeRoot) - tool for simulating superuser privileges*
The abovementioned package descriptions can be extracted with _aptitude show package_ command once the target package name is known and _aptitude_ is installed.
Let's install those target packages:
```
sudo apt-get install lynx libimage-exiftool-perl fakeroot
```
**Lynx** is a client based web browser. **Exiftool** is a CLI tool for processing, examining and manipulating exif metadata present in audio and image files. **fakeroot** is, as the name stands for, a program which is tricks the current user by giving a false assumption of being root user.
**Example runtime screenshots of each program:**
1. [Lynx (https://www.hs.fi - front page)](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/h2-lynx.png)
2. [Exiftool (examined CR2 file metadata)](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/h2-exiftool.png)
3. [fakeroot (asked who am i, accessed fakeroot environment, created a new file in fakeroot environment and asked who am i again. It is claimed that the current user is root although this is not true, as can be seen when exiting the fakeroot environment)](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/h2-fakeroot.png)

+ 475
- 0
exercises/h3.md View File

@ -0,0 +1,475 @@
Linux servers - Exercice 3
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Install Apache web server and set up a working web environment for each system user (http://example.com/~tero). Test the correct functionality with a sample website.
--------------
**Answer:**
```
#!/bin/sh
############
# List package names of Uncomplicated Firewall ja Apache HTTP web server in a new variable PROGRAMS
#
PROGRAMS="ufw apache2"
############
# Define new function 'install'
#
# Function 'install' - BEGIN
function install() {
############
# Look for executable 'apt-get', silent error printing (stderr), print count of matching strings.
# Define these checks in a new variable APT_CHECK so that these checks can easily be performed later in the script.
#
# grep part could have been excluded but it is still good to check that this file is not compromised and is truly
# an executable binary file.
#
# EXAMPLE If the system is compromised and the attacker replaces file /usr/bin/apt-get with his/her own version, the following
# APT_CHECK might successfully pass without additional grep pipe (check additional notes below).
#
# Demonstration is as follows (in root shell, because we assume that the attacker has access to root account).
# It is pretty clear that if the attacker has root access, almost all hope is lost because the attacker can likely access and modify almost any part in the system. However, we don't speculate with this option here.
#
# root@my-machine:/home/phelenius# echo -e "#\!/bin/sh\necho \"Attacker's code\"\n" > /usr/bin/malicious
# root@my-machine:/home/phelenius# chmod +x /usr/bin/malicious
# root@my-machine:/home/phelenius# which malicious
# /usr/bin/malicious
#
# The file /usr/bin/malicious is identified as executable. However, if we check its mimetype, we can find out the following:
# root@my-machine:/home/phelenius# file --mime-type $(which malicious)
# /usr/bin/malicious: text/plain
#
# Therefore we can find out the executable we though would be a binary is simply a text file. In similar way, the following APT_CHECK command would pass without grep pipe which fills the security gap a little bit. However, there's still a hole. A major risk still exist because attacker could have replaced the valid apt-get executable with his/her own malicious binary file (with correct mimetype application/x-sharedlib). In this scenario, a checksum validation should seriously be considered (compare against a trusted apt-get binary file).
#
APT_CHECK=$(file --mime-type $(which apt-get) 2> /dev/null | grep -o application | wc -w)
############
# The next if statement has the following conditions:
# if apt-get command is found in the system (APT_CHECK variable check), and...
# ...if the current user is not root (UID is not 0), and...
# ...the current user belongs to sudo group
#
if [[ $APT_CHECK -eq 1 ]] && [[ $(id -u) -ne 0 ]] && [[ $(groups | grep -o sudo | wc -w) -eq 1 ]]; then
############
# Update packages from sources listed in /etc/apt/sources.list file and in /etc/apt/sources.d/ directory
#
sudo apt-get update
############
# Install programs, which have been defined in the variable PROGRAMS
#
sudo apt-get -y install $PROGRAMS
############
# If the previous command succeeded, pass and execute the commands inside if statement
#
if [[ $? -eq 0 ]]; then
############
# Enable Apache2 specific userdir module
#
sudo a2enmod userdir
############
# Enable "ServerName" parameter with value 'www.example.com' in Apache's default page configuration file (000-default.conf)
#
sudo sed -i 's/#ServerName www\.example\.com/ServerName www\.example\.com/' /etc/apache2/sites-enabled/000-default.conf
############
# Add "ServerAlias" after "ServerName" in Apache's default page configuration file (000-default.conf)
#
sudo sed -i '/ServerName www\.example\.com/a\ \ \ \ \ \ \ \ ServerAlias example\.com' /etc/apache2/sites-enabled/000-default.conf
############
# Add new local virtual host example.com to /etc/hosts file
#
echo -e "127.0.0.1\texample.com" | sudo tee -a /etc/hosts
############
# Restart Apache2 HTTP web server daemon
#
sudo systemctl restart apache2.service
fi
fi
}
# Function 'install' - END
############
# Execute the function 'install', described above
#
install
############
# Create public_html directory in the current user's home directory (either $HOME or ~ can be used)
#
mkdir -p ~/public_html/
############
# Create a new file index.html inside $HOME/public_html with the following html contents:
#
echo -e '\
<!DOCTYPE html>\n \
<html>\n \
\t<head>\n \
\t\t<title>Test</title>\n \
\t</head>\n \
\t<body>\n \
\t\t<h1>Test</h1>\n \
\t</body>\n \
</html>\n \
' \
> ~/public_html/index.html
# About the echo command:
# \t stands for tabulator
# \n stands for a newline
# \ stands for a newline for multiline input in shell environment
# > stands for writing the contents of the echo command into the file/output mentioned after this symbol.
# -e stands for "enable interpretation of backslash escapes" (meaning that you can use \t \n \r \s....symbols in echo command)
#################################
# Is the following procedure secure? However, something found in the internet:
#
# "Basically, the Apache server does not only require read permissions of all files it serves, but the execution permission of all directories in the path of your virtual host."
# https://askubuntu.com/questions/451922/apache-access-denied-because-search-permissions-are-missing
#
chmod 701 $HOME
############
# Test the created site with the default graphical web browser as the current user (retrieved with whoami command + bash internal command substitution method)
#
xdg-open http://example.com/~$(whoami)/
```
**b)** Browse and explore the websites on your local host. Find out any hints of successful (200 ok) website load events and failed load events (for example, 404 not found). Analyse the relevant lines.
--------------
**Answer:**
**NOTE!** My detailed answer to this assignment is in the previous exercise 2, section a) (successful & failed event + analysing them). Link: [Exercise 2]()
In my answer I have analysed Apache web server log lines. The logic is exactly same than required in this assignment. Apache logs each event when a client (web browser, for example) try to access a file or location in server.
**c)** Create a purposeful error in any code line which is executed by Apache web server (PHP or Python, for example). Find out any relevant log lines and analyse them in detail.
--------------
**Answer:**
Let's install PHP 7.0 with Apache2, and execute a shell script which does as required.
```
#!/bin/sh
############
# Look for executable 'apt-get', silent error printing (stderr), print count of matching strings.
# Define these checks in a new variable APT_CHECK so that these checks can easily be performed later in the script.
APT_CHECK=$(file --mime-type $(which apt-get) 2> /dev/null | grep -o application | wc -w)
############
# Look for executable 'wget', silent error printing (stderr), print count of matching strings.
# Define these checks in a new variable WGET_CHECK so that these checks can easily be performed later in the script.
WGET_CHECK=$(file --mime-type $(which wget) 2> /dev/null | grep -o application | wc -w)
############
# Print count of matching strings which have retrieved by checking any packages containing 'apache2' on Debian system.
# Define this check in a new variable APACHE2_CHECK so that this check can easily be performed later in the script.
APACHE2_CHECK=$(dpkg --get-selections | grep apache2 | wc -l)
############
# External PHP code download link (NOTE! Downloading untrusted files is not recommended in security oriented system environments!)
# Define this command in a new variable SAMPLE_CODE so that it can easily be called later in the script.
SAMPLE_CODE="https://gist.githubusercontent.com/pchatterjee/3756368/raw/40c241c344c3e8d2333cc0c496e2782d9a1e6d93/calculator_v2.php"
############
# The next if statement has the following conditions:
# if apt-get command is found in the system (APT_CHECK variable check), and...
# ...if the current user is not root (UID is not 0), and...
# ...the current user belongs to sudo group
#
if [[ $APT_CHECK -eq 1 ]] && [[ $(id -u) -ne 0 ]] && [[ $(groups | grep -o sudo | wc -w) -eq 1 ]]; then
############
# If apache2 is installed in the system, then...
#
if [[ $APACHE2_CHECK -gt 0 ]]; then
############
# install PHP 7.0 and relevant Apache2 PHP 7.0 modules
#
sudo apt-get install -y php7.0 libapache2-mod-php
############
# Comment the following lines in file /etc/apache2/modules-available/php7.0.conf, so that PHP works for with userdir module.
#
# TODO Better method should be coded. The current method has a risk that it replaces wrong lines, especially if there are multiple similar lines in a file.
# Better approach would be to replace lines by starting from line which includes pattern <IfModule and ending to line matching pattern </IfModule
# However, because content structure of php7.0.conf file is not complicated at this moment, the current solution "works".
# Better approach must be implemented if handling any more complex php7.0.conf file.
#
sed -i 's?<IfModule mod_userdir\.c>?#<IfModule mod_userdir\.c>?' /etc/apache2/mods-enabled/php7.0.conf
sed -i 's?<Directory /home/\*/public_html>?#<Directory /home/\*/public_html>?' /etc/apache2/mods-enabled/php7.0.conf
sed -i 's?php_admin_flag engine Off?#php_admin_flag engine Off?' /etc/apache2/mods-enabled/php7.0.conf
sed -i 's?</Directory>?#</Directory>?' /etc/apache2/mods-enabled/php7.0.conf
sed -i 's?</IfModule>?#</IfModule>?' /etc/apache2/mods-enabled/php7.0.conf
############
# If command wget is available on the system, then...
#
if [[ $WGET_CHECK -eq 1 ]]; then
############
# Download sample PHP code into directory $HOME/public_html/
# NOTE! Can we trust the downloaded code?
#
echo -e "\nDownloading sample PHP code for testing purposes (requires internet connection, security risk exists! Use local trusted code if system hardening means anything to you...)\n"
#
wget -P $HOME/public_html/ $SAMPLE_CODE
############
# Let's generate a purposeful error in file $HOME/public_html/calculator_v2.php by altering the code with sed command (switch -> switchasd)
#
sed -i 's/switch($op) {/switchasd($op) {/' $HOME/public_html/calculator_v2.php
############
# Let's try to open the erroneous PHP site with the default graphical web browser.
# xdg-open command refers to default program defined to open specific mimetypes or protocols in Linux operating system.
#
xdg-open http://example.com/~$(whoami)/calculator_v2.php
############
# Command 'wget' is not found in the system
#
else
echo -e "\nInstall application 'wget' before downloading a sample code"
fi
############
# Packages matching string 'apache2' can't be found in the system by doing the check via Debian package manager
#
else
echo -e "\nYou need to install Apache web server before PHP 7.0\n"
exit
fi
fi
```
Apache web server prints out the following error message in _/var/log/apache2/error.log_ while trying to open local website _example.com/~phelenius/calculator_v2.php_:
```
[Wed Feb 07 00:20:02.923994 2018] [:error] [pid 14260] [client 127.0.0.1:38430] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
[Wed Feb 07 00:20:03.203824 2018] [:error] [pid 14259] [client 127.0.0.1:38432] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
[Wed Feb 07 00:20:03.684123 2018] [:error] [pid 14258] [client 127.0.0.1:38434] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
[Wed Feb 07 00:20:03.932480 2018] [:error] [pid 14268] [client 127.0.0.1:38436] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
[Wed Feb 07 00:20:04.269771 2018] [:error] [pid 14261] [client 127.0.0.1:38438] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
[Wed Feb 07 00:20:04.470184 2018] [:error] [pid 14262] [client 127.0.0.1:38440] PHP Parse error: syntax error, unexpected 'case' (T_CASE) in /home/phelenius/public_html/calculator_v2.php on line 20
```
Apache web server complains multiple times about a PHP syntax error in /home/phelenius/public_html/calculator_v2.php file, on line 20.
Mentioned PIDs (Process IDs) belong to apache2 process:
```
phelenius@my-machine:~$ ps aux |grep -E "14261|14262|14268|14258|14259|14260"
www-data 14258 0.0 0.5 253844 11648 ? S 00:16 0:00 /usr/sbin/apache2 -k start
www-data 14259 0.0 0.5 253844 11648 ? S 00:16 0:00 /usr/sbin/apache2 -k start
www-data 14260 0.0 0.5 253844 11648 ? S 00:16 0:00 /usr/sbin/apache2 -k start
www-data 14261 0.0 0.7 254136 14372 ? S 00:16 0:00 /usr/sbin/apache2 -k start
www-data 14262 0.0 0.5 253844 11648 ? S 00:16 0:00 /usr/sbin/apache2 -k start
www-data 14268 0.0 0.5 253844 11648 ? S 00:16 0:00 /usr/sbin/apache2 -k start
```
IP address 127.0.0.1 refers to local host, numbers 38430, 38432, 38434, 38436, 38438 and 38440 to the ports where the connection has been established from.
The equivalent log events in _/var/log/apache2/access.log_:
```
127.0.0.1 - - [07/Feb/2018:00:20:02 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
127.0.0.1 - - [07/Feb/2018:00:20:03 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
127.0.0.1 - - [07/Feb/2018:00:20:03 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
127.0.0.1 - - [07/Feb/2018:00:20:03 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
127.0.0.1 - - [07/Feb/2018:00:20:04 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
127.0.0.1 - - [07/Feb/2018:00:20:04 +0200] "GET /~phelenius/calculator_v2.php HTTP/1.1" 500 185 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3300.0 Iron Safari/537.36"
```
- User's client program (address 127.0.0.1/localhost + client is web browser in this case) tries to retrieve server website _/~phelenius/calculator_v2.php_ for which the Apache web server has responded with error code 500 ([HTTP_INTERNAL_SERVER_ERROR](https://ci.apache.org/projects/httpd/trunk/doxygen/group__HTTP__Status.html#ga5d9777e02c26063c2985e39ef71091d2)).
- User ID is simply a line symbol
- Log time is "07/Feb/2018:00:20:04 +0200" etc
- HTTP method used is [GET](https://www.w3schools.com/tags/ref_httpmethods.asp)
- has [HTTP Referer](https://en.wikipedia.org/wiki/HTTP_referer)
- size of the object is 185 (reported to the client program)
- [User agent header](https://en.wikipedia.org/wiki/User_agent) reported by the client. According to the agent string, client browser has been gecko-based [Epiphany web browser](https://en.wikipedia.org/wiki/Epiphany_(GNOME)), using x86_64 processor architecture. This string can be manipulated in the client end. For example, a desktop client web browser can pretend to be a mobile browser.
Default syntax for Apache log files follow the layout which is described [here](https://httpd.apache.org/docs/2.4/logs.html) under section 'Common Log Format'.
**e)** Install and try out PhpMyAdmin or similar out-of-the-box web-based database management solution.
--------------
**Answer:**
**NOTE!** We assume that the established Linux system has a working Apache HTTP daemon web server and PHP 5.0 or PHP 7.0 script language packages installed as Apache modules (see tasks a) and c) above).
1. [phpMyAdmin](https://en.wikipedia.org/wiki/PhpMyAdmin) requires MySQL/MariaDB database back-end server. Let's install MySQL database server with its runtime dependencies on Debian-based distribution (Ubuntu 16.04.3 LTS in this case).
```
sudo apt-get -y install mysql-server
```
2. Local administrator (assumed to belong to sudo group) is asked to set a new password for root MySQL database root account:
**NOTE!** This description applies to Ubuntu 16.04 LTS. Newer OS releases may not ask you to set MySQL password after MySQL server installation. If you don't see a blue screen stating "While not mandatory...", run the following command instead:
```
sudo mysql_secure_installation
```
```
Configuring mysql-server-5.7
While not mandatory, it is highly recommended that you set a password for the MySQL administrative "root" user.
If this field is left blank, the password will not be changed.
New password for the MySQL "root" user:
```
We should choose a strong password ('Hu8aS1n?tI23' or 'Tu$pAR!iMu65' etc. Can be generated with a password generator program _pwgen_ as well). We should use a password which we don't use in other environments.
**NOTE!** If unsure, check and configure your keyboard layout so that the password you supply will be correctly set up.
When you have entered your password, press _TAB_ and then press _<Ok>_ (or just _ENTER_). Type the password again and continue.
3. Let's install [phpMyAdmin](https://en.wikipedia.org/wiki/PhpMyAdmin) with its runtime dependencies on Debian-based distribution (Ubuntu 16.04.3 LTS in this case).
```
sudo apt-get install -y phpmyadmin
```
Accept the installation with extra dependencies.
4. On Debian-based systems, the next screen states:
```
Configuring phpmyadmin
Please choose the web server that should be automatically configured to run phpMyAdmin.
Web server to reconfigure automatically:
[] apache2
[] lighttpd
```
Because we assume that you have Apache2 and not lighttpd, we should select apache2. Press _SPACE_ so that option _apache2_ has asterix in the brackets ([*]). After that, press _TAB_ to move the selection to _<Ok>_ and press _ENTER_.
5. The next screen states:
```
Configuring phpmyadmin
The phpmyadmin package must have a database installed and configured before it can be used. This can be optionally handled with dbconfig-common.
If you are an advanced database administrator and know that you want to perform this configuration manually, or if your database has already been installed and configured, you should refuse this option. Details on what needs to be done should most likely be provided in /usr/share/doc/phpmyadmin. │
Otherwise, you should probably choose this option.
Configure database for phpmyadmin with dbconfig-common?
```
The most common selection in this step is _<Yes>_ (alternative to _<No>_). For choosing a right choice, the following manuals can be used:
[Configure the database with dbconfig-common](https://docs.openstack.org/ocata/fr/install-guide-debconf/debconf/debconf-dbconfig-common.html)
[dbconfig-common.pdf](https://www.debian.org/doc/manuals/dbconfig-common/dbconfig-common.pdf)
[Stackoverflow - What means dbconfig-common](https://stackoverflow.com/questions/31403273/what-means-dbconfig-common)
Additionally, for example:
![dbconfig-common_1](https://docs.openstack.org/ocata/fr/install-guide-debconf/_images/dbconfig-common_keep_admin_pass.png)
![dbconfig-common_1](https://docs.openstack.org/ocata/fr/install-guide-debconf/_images/dbconfig-common_used_for_remote_db.png)
Let's choose option _<Yes>_ because we don't have earlier database in our system. More secure, although more troublesome, approach would be selecting _<No>_, according to the material references above.
**NOTE!** dpkg tool _dpkg-reconfigure_ makes it possible to change package-related configuration in Debian-based systems afterwards (for example, _sudo dpkg-reconfigure phpmyadmin_). However, pay special attention and use extra care if you do any configurations afterwards.
6. The next screen asks system administrator to set up MySQL database-related password for phpmyadmin MySQL user:
```
Configuring phpmyadmin
Please provide a password for phpmyadmin to register with the database server. If left blank, a random password will be generated.
MySQL application password for phpmyadmin:
```
Let's pick up a strong password for phpmyadmin. Take a look on step 2. for setting up a strong password. Use different password here.
7. If you are not asked to set up a default phpmyadmin username or password, please run _sudo dpkg-reconfigure phpmyadmin_. The default phpmyadmin username is usually 'phpmyadmin' and the login password is the one you write down by executing the command mentioned earlier in this step.
8. Let's try accessing phpMyAdmin front page:
```
xdg-open http://localhost/phpmyadmin
```
If the installation has been successful, the web browser view should look like this:
![phpmyadmin-loginpage](https://rootninja.files.wordpress.com/2010/10/image38.png)
9. Log in to phpMyAdmin with the following credentials:
- **user name:** phpmyadmin (or any other defined during phpmyadmin installation)
- **password:** password defined in step 7.
10. If successfully logged in, the web browser view should be as follows (phpMyAdmin - Front page):
![phpmyadmin-frontpage](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/phpmyadmin-resized.png)
phpMyAdmin - Database configuration sample:
![phpmyadmin-dbconfpage](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/phpmyadmin-2-resized.png)
**f)** Create a web application which executes a simple calculation for the user (BMI calculator, for example)
--------------
**Answer:**
**NOTE!** We assume that the Linux installation has a working Apache HTTP daemon web server with correct PHP support enabled. Apache server has 'userdir' module enabled, and some local user has _public_html_ folder created.
1. Let's create the following, very simple PHP code and save it to ~/public_html/rainfall.php (current user)
```
<?php
$rainfalls = array(54.2, 43.1, 37.5, 25.6, 10.2, 15.7, 29.4, 41.9, 55.6, 69.1, 77.2, 64.3);
// Print output
echo "Total rainfall is " . array_sum($rainfalls) . " millimeters in a year.";
?>
```
2. Let's open the php file with the default graphical web browser found in the local Linux system:
```
xdg-open http://example.com/~$(whoami)/rainfall.php
```
Result:
![php-sample-image-1](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/php-example-rainfall.png)

+ 595
- 0
exercises/h4.md View File

@ -0,0 +1,595 @@
Linux servers - Exercice 4
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Make it possible to create home pages with normal user privileges in your virtual server environment.
--------------
**Answer:**
Virtual server IP address is _174.138.2.190_. The server is hosted on [DigitalOcean](https://www.digitalocean.com/community/tutorials/how-to-create-your-first-digitalocean-droplet) which provides hosting services for users. Ubuntu 16.04 LTS (without DE) is used as a server platform.
Ubuntu distribution hosted on DigitalOcean has custom configurations compared to Canonical version of clean Ubuntu environment. For example, DigitalOcean has predefined different repository sources in package manager sources file _/etc/apt/sources.list_ (http://mirrors.digitalocean.com/ubuntu/) and has customized configuration for Apache web server by default. Can you trust these repositories and predefined configurations as a system administrator? Can you be sure the program source codes are "clean" and do not contain malicious code patches? Are the used package repositories updated and which flags have been used to compile the binary software available there? What differences are there between the official Canonical and DigitalOcean repositories?
It is good to stop to think the previous issues before looking for a third party hosting service for your server environment. Do available markets have better virtual hosting service providers and in which criteria? In small business, it can be safer to hold all the aces and set up a minimal self-hosted server using a computer (suited for your needs) such as Raspberry Pi and claim a Domain name for it from a DNS provider. If you do larger business, you need to consider your server capacity again. It can be more profitable and comfortable to buy some server space from a virtual hosting server provider.
Of course, if you ever want to, you can install the server environment from scratch using rolling releases such as Arch Linux, Gentoo or 'Linux from Scratch'. However, this is not recommended approach in server environments requiring system stability over newer software.
Anyway,
1. Do initial configuration for your server following the guide by Tero Karvinen:
[Tero Karvinen - First Steps on a New Virtual Private Server – an Example on DigitalOcean and Ubuntu 16.04 LTS](http://terokarvinen.com/2017/first-steps-on-a-new-virtual-private-server-an-example-on-digitalocean)
2. Connect to your new virtual private server once you have set it up. I use a predefined user name _newuser_ in my server environment:
```
phelenius@my-machine:~$ ssh newuser@174.138.2.190
newuser@174.138.2.190's password:
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
7 packages can be updated.
0 updates are security updates.
Last login: Tue Feb 13 14:49:24 2018 from XXX.XXX.XXX.XXX
newuser@goauldhost:~$
```
3. Open default SSH TCP port 22 in your firewall:
**NOTE!** Default policy for Ubuntu firewall is to deny/drop all input connections so we need to allow traffic into protocol/daemon specific ports to open up communication between this server and clients.
```
newuser@goauldhost:~$ sudo ufw allow 22/tcp && sudo ufw enable
```
Once you have established SSH connection to your remote server, install _apache2_ and open port 80 for it doing the following:
```
newuser@goauldhost:~$ sudo apt-get install apache2 && sudo ufw allow 80/tcp
```
4. Enable Apache _userdir_ module:
```
newuser@goauldhost:~$ sudo a2enmod userdir
```
5. Some PHP and userdir module related configurations seem to be predefined in packages provided on DigitalOcean server environment (such as # symbols in /etc/apache2/mods-enabled/php7.0.conf in order to enable PHP for user sites), unlike stated in [Exercise 3](https://github.com/Fincer-altego/basics-of-a-linux-server-school-course-/blob/master/h3.md).
6. Restart Apache HTTP daemon (HTTPD) after enabling _userdir_ module:
```
newuser@goauldhost:~$ sudo systemctl restart apache2.service
```
Recheck HTTP daemon state (should be **Running**):
```
newuser@goauldhost:~$ systemctl status apache2.service
```
If everything seem to be running and working you can try to establish connection to the HTTP daemon server using _your local computer_ (do not try this in your virtual private server environment but with an external client computer):
```
phelenius@my-machine:~$ xdg-open http://174.138.2.190:80
```
The IP address here refers to the virtual private server IP address.
7. If the connection test is successful, you should see something similar to this picture:
![apache2-defaultpage](https://www.dedyprastyo.com/wp-content/uploads/2017/10/Install-Apache-di-Ubuntu-17.04-www.dedyprastyo.com_-400x400.jpg)
Let's get back to the virtual server environment in our SSH session.
8. We should hide any extra server information which is visible for clients. Add the following lines in _/etc/apache2/apache2.conf_ (with sudoedit):
```
TraceEnable Off
ServerSignature Off
ServerTokens Prod
```
Description of these lines [here](https://www.petefreitag.com/item/419.cfm).
Restart Apache HTTP daemon (web server):
```
newuser@goauldhost:~$ sudo systemctl restart apache2.service
```
9. Create a new user _monkey_ (finnish: apina). We do not add this user to _sudo_ groups, therefore denying all root access for this user:
```
newuser@goauldhost:~$ sudo adduser monkey
```
Change to the default shell of user _monkey_ (you can check it by executing: _grep monkey /etc/passwd_):
```
newuser@goauldhost:~$ su monkey
Password:
monkey@goauldhost:/home/newuser$ cd
monkey@goauldhost:~$ mkdir public_html
```
Therefore we have created a new home site folder for the user _monkey_. Contents of this folder should be available like shown in the following picture:
![monkeysite-samplepic](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/monkey-site.png)
**NOTE!** As you can see, Apache doesn't give any server information in the website view, thanks for the configurations done in step 8.
User _monkey_ can add any content to his/her personal site. By default, Apache looks for any of the following files, defined in _/etc/apache2/mods-available/dir.conf_ (symbolic link in folder _/etc/apache2/mods-enabled/dir.conf_):
```
newuser@goauldhost:~$ ls -l /etc/apache2/mods-enabled/dir.conf
lrwxrwxrwx 1 root root 26 Feb 8 10:40 /etc/apache2/mods-enabled/dir.conf -> ../mods-available/dir.conf
newuser@goauldhost:~$ cat /etc/apache2/mods-enabled/dir.conf |grep -i directoryindex | awk '{$1 = ""; print $i}'
index.html index.cgi index.pl index.php index.xhtml index.htm
```
Permissions of folder $HOME/public_html of the user _monkey_ are as follows:
```
monkey@goauldhost:~$ ls -l
total 4
drwxrwxr-x 2 monkey monkey 4096 Feb 13 15:35 public_html
```
drwxrwxr-x
123456789 10
d = directory
r = read (value 4)
w = write (value 2)
x = execute (value 1)
where
...symbols 2-4 (rwx) indicate permissions of the user _monkey_ to the folder
...symbols 5-7 (rwx) indicate permissions of the (primary) group where the user _monkey_ belongs to. In this case, the group is _monkey_
...symbols 8-10 (r-x) indicate permissions granted for other system users to the folder
Permissions can be written in numeric form but also in symbolic form. For example,
775 = rwxrwxr-x (4+2+1, 4+2+1, 4+1)
ug=rwx,o=rx = rwxrwxr-x
Take a look on the following links to get more information about Unix permissions:
- [Arch Wiki - File permissions and attributes](https://wiki.archlinux.org/index.php/File_permissions_and_attributes)
- [Wikipedia - Notation_of_traditional_Unix_permissions](https://en.wikipedia.org/wiki/File_system_permissions#Notation_of_traditional_Unix_permissions)
- [Arch Wiki - umask](https://wiki.archlinux.org/index.php/Umask)
Changing permissions is recommended to be done in symbolic mode because individual permissions can't be as flexibly be changed in numeric mode.
ls command shows a user (monkey) first after which group (monkey) is shown.
The folder is accessible in public network via address _http://174.138.2.190:80/~monkey/_
You can change permissions with _chmod_ command.
Numeric form:
chmod 775 ~/public_html
Symbolic form:
chmod ug=rwx,o=rx ~/public_html
You can change permissions recursively using _-R_ parameter (chmod -R ...)
**EXTRA: Deleting Server field from HTTP header by updating Apache source code on Debian-based Linux distributions**
--------------
Including Server field in HTTP header by Apache HTTP daemon (web server) is debated. [HTTP/1.1 standard specification](https://tools.ietf.org/html/rfc2616#page-141) states the following:
> Note: Revealing the specific software version of the server might allow the server machine to become more vulnerable to attacks against software that is known to contain security holes. Server implementors are encouraged to make this field a configurable option.
**NOTE!** Apache is compiled from source code.
Removal of Server field from HTTP header is debated in Apache developers' community (The key question here: does removal of the field actually improve any security?). More about this topic: [StackoverFlow (Can't remove Server: Apache header)](https://stackoverflow.com/questions/35360516/cant-remove-server-apache-header).
**NOTE!** The following method does not work with automatic system updates via package repositories (usage of _sudo apt-get update_ and _sudo apt-get upgrade_ commands) because the patches binary files would be replaced by ones available in the official repositories. Therefore, each time you want to update your Apache server, you need to recompile it from source applying the patch file provided in this GitHub repository. _This method can be troublesome for system administration or contain unaccepted policy in your company. Consider using puppet or salt or any relevant automation method here._
**NOTE!** You must be aware what you are doing each time you compile software from source. Trust the source, trust the patch files (diff or patch) and do not do anything that can lead to unknown or troublesome bugs, malicious code execution or create new security risks.
**NOTE!** We are unable to sign the package with the official maintainer key because we update Apache with our specific patch file and compile the software ourselves. Therefore we ignore any signing requirements in _dpkg-buildpackage_ command (or alternatively we can use our own keys).
**NOTE** Consider any policy that determines your production and/or server environment requirements. For example: Am I allowed to install software from source code? Does my method break a (critical) component in working server environment? Etc.
**NOTE!** _Again. Know exactly what you are about to do. As a system administrator you hold responsibility here._
We have already improved security of our Apache web server by removing critical information sent to client. However, the server still gives information about its name as follows:
```
newuser@goauldhost:~/source_codes/apache2$ curl -I http://localhost
HTTP/1.1 200 OK
Date: Sat, 17 Feb 2018 13:27:11 GMT
Server: Apache
Content-Type: text/html;charset=UTF-8
```
We want to remove field _Server: Apache_. Multiple approaches were tested (such as modsecurity2 module and writing line _Header unset Server_ into file _/etc/apache2/apache2.conf_) without success. Therefore I ended up patching the source code of Apache web server so that the wanted field can really be removed.
Download Apache source code on your Debian-based Linux distribution:
```
newuser@goauldhost:~$ mkdir -p ./source_codes/apache2 && cd ./source_codes/apache2
newuser@goauldhost:~/source_codes/apache2$ apt-get source apache2
```
After which add [source code patch file](https://github.com/Fincer-altego/basics-of-a-linux-server-school-course-/blob/e60bfe23814f5b72315daec144c31a75bc7f1ef0/patch_apache_servertokens.patch) into created _$HOME/source_codes/apache2_ folder.
**NOTE!** I have personally created the patch file with Unix tool _diff_. The patch file is not downloaded from any suspicious website. You can always check the patch file code yourself if still hesitating.
If you have a working Apache HTTP daemon (web server) environment on your Linux, please check the version of your Apache software version with the following command:
```
newuser@goauldhost:~$ dpkg -s apache2 |grep -i version
Version: 2.4.18-2ubuntu3.5
```
In that way we can be sure version of the downloaded source code matches with our already-installed Apache environment.
Once you have downloaded the source code, go to the following folder (which contains the code):
```
newuser@goauldhost:~/source_codes/apache2$ cd apache2-2.4.18/
```
Implement the patch file changes into the Apache source code:
```
newuser@goauldhost:~/source_codes/apache2/apache2-2.4.18$ patch -Np1 -i ../patch_apache_servertokens.patch
```
Before compiling Apache web server, you must install the following build time dependencies:
```
newuser@goauldhost:~/source_codes/apache2/apache2-2.4.18$ sudo apt-get install debhelper libaprutil1-dev libapr1-dev libpcre3-dev zlib1g-dev libssl-dev liblua5.1-0-dev libxml2-dev autotools-dev build-essential
```
Compile and install the Apache web server:
newuser@goauldhost:~/source_codes/apache2/apache2-2.4.18$ dpkg-buildpackage -rfakeroot -b -us -uc
...
<compiling source code>
...
<source code compiled and archived as new deb packages>
```
If Apache HTTP daemon is running, stop it:
```
newuser@goauldhost:~/source_codes/apache2$ sudo systemctl stop apache2.service
```
It is essential to check which apache2 packages have been installed into the target system. We want to install only specific deb packages already found in the system, as multiple deb packages have been compiled by the previous command. All found Apache2 packages in the system should be replaced by the ones compiled from the Apache2 source code.
System has the following Apache2 packages:
```
newuser@goauldhost:~/source_codes/apache2/apache2-2.4.18$ cd ..
newuser@goauldhost:~/source_codes/apache2$ dpkg --get-selections |grep apache | awk '{print $1}'
apache2
apache2-bin
apache2-data
apache2-utils
libapache2-mod-php
libapache2-mod-php7.0
```
Then we compare the above list to the compiled deb packages:
```
newuser@goauldhost:~/source_codes/apache2$ ls |grep deb
apache2_2.4.18-2ubuntu3.5_amd64.deb
apache2_2.4.18-2ubuntu3.5.debian.tar.xz
apache2-bin_2.4.18-2ubuntu3.5_amd64.deb
apache2-data_2.4.18-2ubuntu3.5_all.deb
apache2-dbg_2.4.18-2ubuntu3.5_amd64.deb
apache2-dev_2.4.18-2ubuntu3.5_amd64.deb
apache2-doc_2.4.18-2ubuntu3.5_all.deb
apache2-suexec-custom_2.4.18-2ubuntu3.5_amd64.deb
apache2-suexec-pristine_2.4.18-2ubuntu3.5_amd64.deb
apache2-utils_2.4.18-2ubuntu3.5_amd64.deb
```
... as a result we can see that the following deb packages should be installed with _dpkg -i_ command:
```
newuser@goauldhost:~/source_codes/apache2$ sudo dpkg -i apache2_2.4.18-2ubuntu3.5_amd64.deb apache2-bin_2.4.18-2ubuntu3.5_amd64.deb apache2-data_2.4.18-2ubuntu3.5_all.deb apache2-utils_2.4.18-2ubuntu3.5_amd64.deb
```
If everything has succeeded you should have a working, patched Apache web server in your target system. Because the patches web server software supports _ServerTokens None_ option now, we shall add this option into _etc/apache2/apache2.conf_.
```
newuser@goauldhost:~$ sudoedit /etc/apache2/apache2.conf
```
Add the following lines (// just replace ServerTokens Prod with None):
```
TraceEnable Off
ServerSignature Off
ServerTokens None
```
**NOTE!** Any settings in _/etc/apache2/conf-available/security.conf_ overrides these configuration changes.
Restart Apache2 server (you must apply the patch file before doing this because the default Apache software does not implement 'None' for ServerTokens):
```
newuser@goauldhost:~$ sudo systemctl start apache2.service
```
Check whether the configuration works:
```
phelenius@my-machine:~$ curl -I http://174.138.2.190
HTTP/1.1 200 OK
Date: Sat, 17 Feb 2018 14:02:20 GMT
Last-Modified: Wed, 14 Feb 2018 00:06:44 GMT
ETag: "20b-56520e2f88f4a"
Accept-Ranges: bytes
Content-Length: 523
Vary: Accept-Encoding
Content-Type: text/html
```
Therefore we have successfully deleted Server field from HTTP header.
You can additionally set and unset HTTP header fields sent to a client as follows:
```
Header unset Last-Modified
Header unset Accept-Ranges
Header unset Vary
Header unset ETag
```
You must be careful when unsetting fields because it affects behavior of client programs and efficiency of your server environment (performance, for example). Remember that factors such as field order, formatting and error messages can give hints about the used server environment as well (for example, 404 not found message).
**NOTE!** The header options mentioned above work only if module 'headers' has been activated (run command _sudo a2enmod headers_ and restart the server)
More about HTTP header in [Wikipedia](https://en.wikipedia.org/wiki/List_of_HTTP_header_fields). More articles in [ETag](https://en.wikipedia.org/wiki/HTTP_ETag), [Vary: Accept-Encoding](https://blog.stackpath.com/accept-encoding-vary-important), etc.
**s)** Set user default website to be the default website for Apache in your virtual server environment.
--------------
**Answer:**
Let's begin from the final state of the previous answer. We have created a user _monkey_ in our server computer. This user has a website URL 174.138.2.190:80/~monkey
1. Remove default webpage of Apache web server, and move _DocumentRoot_ to point to directory *$HOME/public_html* of user *monkey*, after which restart Apache service daemon.
```
newuser@goauldhost:~$ sudo sed -i 's?DocumentRoot /var/www/html?DocumentRoot /home/monkey/public_html?' /etc/apache2/sites-available/000-default.conf
```
We can disable folder path _/var/www_ commenting out the following lines (inserting # symbols) in file */etc/apache2/apache2.conf* (use command _sudoedit_, for example):
```
#<Directory /var/www/>
# Options Indexes FollowSymLinks
# AllowOverride None
# Require all granted
#</Directory>
```
Restart Apache web server daemon:
```
newuser@goauldhost:~$ sudo systemctl restart apache2.service
```
Go to the following IP address with your _local computer_ (use HTTP port 80):
```
xdg-open http://174.138.2.190:80
```
The opening view should be as follows:
![emptypage-sample](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/empty-page-sample.png)
2. Create a new file _index.html_ in the directory $HOME/public_html of user _monkey_ (where $HOME=/home/monkey/)
```
newuser@goauldhost:~$ su monkey
Password:
monkey@goauldhost:/home/newuser$ cd
monkey@goauldhost:~$ echo -e '<!DOCTYPE html>\n <html>\n \t<head>\n \t\t<title>Testi</title>\n \t</head>\n \t<body>\n \t\t<h1>Testi</h1>\n \t</body>\n </html>\n' > ~/public_html/index.html
```
3. We should redirect all index.html traffic to the folder root _/home/monkey/public_html/. This can be done by creating hidden page-related control file _.htaccess_ file to the directory root.
```
monkey@goauldhost:~$ cd public_html
monkey@goauldhost:~/public_html$ touch .htaccess
monkey@goauldhost:~/public_html$ echo -e '<IfModule mod_rewrite.c>\n\tRewriteEngine On\n\tRewriteBase /\n\tRewriteRule ^index\.html$ / [NC,R,L]\n</IfModule>' | tee -a ./.htaccess
```
Reactivate Apache module 'rewrite'. We shall switch our user because user _monkey_ doesn't have _sudo_ rights at this point:
```
monkey@goauldhost:~/public_html$ su newuser
Password:
newuser@goauldhost:/home/monkey/public_html$ sudo a2enmod rewrite
Enabling module rewrite.
To activate the new configuration, you need to run:
service apache2 restart
newuser@goauldhost:/home/monkey/public_html$ sudo systemctl restart apache2.service
```
Your Apache web server should redirect all traffic of _http://174.138.2.190:80/index.html_ to address _http://174.138.2.190:80/_
**y)** Find clues of possible penetration attempts to your web server. You can find more information about suspicious IP address without connecting them by using commands ipcalc, geoiplookup and whois, for instance.
--------------
**Answer:**
At the time of writing this answer, the Apache web server was running bit over a week period. However, there were no any webpage deployed during that time, and therefore my web server hadn't created any major log entries. However, I noticed one suspicious connection attempt to setup.php file of phpMyAdmin software, although the software was not installed. The lookup was done by checking web server log file _var/log/apache2/access.log_. The log entry itself was as follows:
```
66.118.142.165 - - [13/Feb/2018:14:43:58 +0000] "HEAD /phpmyadmin/scripts/setup.php HTTP/1.0" 404 159 "-" "-"
```
We shall analyze the logged IP address (+ download a georeferred IP database to determine more specific geolocation of the source by using _wget_ command):
```
newuser@goauldhost:~$ sudo apt-get -y install geoip-bin
newuser@goauldhost:~$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
newuser@goauldhost:~$ gunzip GeoLiteCity.dat.gz
newuser@goauldhost:~$ OBSIP=66.118.142.165
newuser@goauldhost:~$ echo -e "Server Geolocation:\n$(geoiplookup -f /home/newuser/GeoLiteCity.dat $OBSIP)\n\n$(nslookup $OBSIP)\n\nDNS Name:\n$(dig +short -x $OBSIP)" && unset OBSIP
```
Output is as follows:
```
Server Geolocation:
GeoIP City Edition, Rev 1: US, FL, Florida, Tampa, 33611, 27.886700, -82.511703, 539, 813
Server: 67.207.67.2
Address: 67.207.67.2#53
Non-authoritative answer:
165.142.118.66.in-addr.arpa name = 66-118-142-165.static.sagonet.net.
Authoritative answers can be found from:
DNS Name:
66-118-142-165.static.sagonet.net.
```
The log entry tells us that connection to page _http://174.138.2.190:80/phpmyadmin/scripts/setup.php_ was tried to establish. The server responded with code 404 (HTTP_NOT_FOUND), indicating that the address couldn't be found. It is seen that user agent string of _66.118.142.165_ is empty (just a line). It is known that the user agent string can be very easily customized.
HTTP [HEAD method](http://condor.depaul.edu/dmumaugh/readings/handouts/SE435/HTTP/node14.html) were used in the connection attempt. The HTTP HEAD method is more suitable for quick file existence look-ups than HTTP GET method because only the file existence is checked, leading to decreased data transfer rates between a server and a client. The HEAD method is usually used for caching documents (data). A client program just tries to download metadata of the request document from a server. [Apache web server does not return the message response body while answering to the client program with the HTTP HEAD method.](https://hc.apache.org/httpclient-3.x/methods/head.html).
the HEAD method can be "disabled" by adding the following lines into 1) file *~/public_html/.htaccess* (in the case of this assignment) 2) or into file */etc/apache2/sites-available/000-default.conf* 3) or any site-specific configuration file in Apache's _sites-available_ folder 4) or any _.htaccess_ file in a website directory root:
```
RewriteEngine on
RewriteCond %{THE_REQUEST} !^(POST|GET)\ /.*\ HTTP/1\.1$
RewriteRule .* - [F,L]
```
The original answer [here](https://www.linuxquestions.org/questions/linux-security-4/disabling-head-options-http-methods-in-apache-webserver-763347/#post4511023)
It is understandable that line _RewriteEngine on_ doesn't need to be determined twice and adding this code requires re-enabling of Apache rewrite module.
I have had a Debian-based web server (LAMP) environment in the past years. I have included Apache access.log parts of that web server environment here (year 2014):
[Apache - access.log, example](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/apache-log-sample)
The log file reveals many suspicious connection attempts from Thailand and Netherlands, for example.
The most memorable log entry from the past years was, however, a penetration attempt by Romanian hacker/bot group and it looked like the following:
```
4.125.148.79 - - [07/Aug/2013:20:53:35 +0400] "GET /w00tw00t.at.blackhats.romanian.anti-sec:) HTTP/1.1" 404 142 "-" "ZmEu"
```
**y)** Create a set of websites on your local computer and copy the sites to your web server with scp command.
--------------
**Answer:**
1. Let's create the required websites locally, after which the upload is done with user _newuser_. I have used pre-created websites in this assignment. The upload is done with my Arch Linux computer using required SSH protocol:
```
[13/02/2018 21:04:16 - fincer: ~ ]$ scp /home/fincer/Documents/website/website_1.03_fincer.zip newuser@174.138.2.190:./
The authenticity of host '174.138.2.190 (174.138.2.190)' can't be established.
ECDSA key fingerprint is SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '174.138.2.190' (ECDSA) to the list of known hosts.
newuser@174.138.2.190's password:
website_1.03_fincer.zip 100% 656KB 655.8KB/s 00:01
[13/02/2018 21:04:32 - fincer: ~ ]$
```
2. Modify Apache default webpage address to point to _/home/newuser/public_html/_:
```
newuser@goauldhost:~$ sudo sed -i 's?DocumentRoot /home/monkey/public_html?DocumentRoot /home/newuser/public_html?' /etc/apache2/sites-available/000-default.conf
newuser@goauldhost:~$ sudo systemctl restart apache2.service
```
3. Because /home/newuser/public_html/ is empty (checked with ls command), we shall extract uploaded _website_1.03_fincer.zip_ to that directory.
```
newuser@goauldhost:~$ sudo apt-get update && sudo apt-get install unzip
newuser@goauldhost:~$ mv website_1.03_fincer.zip ./public_html/ && cd ./public_html
newuser@goauldhost:~$ unzip website_1.03_fincer.zip
```
4. It should be possible to open the website using URL _174.138.2.190_, and the website should look like the following:
![website-sample](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/pekkahh-website.png)
5. Let's copy another website sample into folder $HOME/public_html/ of user _monkey_:
```
[13/02/2018 22:01:40 - fincer: ~ ]$ scp /home/fincer/Documents/server_site.tar.xz monkey@174.138.2.190:./public_html/
```
6. Let's establish a new SSH connection to the virtual server computer with user _monkey_:
```
[13/02/2018 22:04:23 - fincer: ~ ]$ ssh monkey@174.138.2.190
```
7. In the virtual web server computer terminal, go to _$HOME/public_html/ of user _monkey_, extract _server_site.tar.xz_ and put the extracted files into correct places in the directory hierarchy. Rename old _index.html_ file to _index.html.old_
```
monkey@goauldhost:~/public_html$ mv index.html index.html.old
monkey@goauldhost:~/public_html$ tar xf server_site.tar.xz
monkey@goauldhost:~/public_html$ mv ./server_site/* ./
monkey@goauldhost:~/public_html$ rm -Rf ./{server_site,server_site.tar.xz}
```
(you could have just used one * symbol in the previous rm command because the equal name syntax)
The deployed website can be viewed in URL address _http://174.138.2.190:80/~monkey_, and they look as follows:
![workprice-samplesite](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/workprice-site_example.png)
**y)** Set up a simple PHP webpage on your web server. For instance, you can print a remote address of the user ( $_SERVER['REMOTE_ADDR'] ) etc. Be careful if you use input forms of any kind.
--------------
**Answer:**
In this answer, I add a PHP-based BMI calculator (Body Mass Index) to my website. For this purpose, two files were created: _bmicalc.html_ and _bmicalc.php_ which both are uploaded to the virtual web server with the following command, executed at the _local computer_:
```
phelenius@my-machine:~$ scp $HOME/public_html/bmi-index/{bmicalc.php,bmicalc.html} newuser@174.138.2.190:./public_html/
newuser@174.138.2.190's password:
bmicalc.php 100% 3051 3.0KB/s 00:00
bmicalc.html 100% 523 0.5KB/s 00:00
```
The following image demonstrates a web browser view of URL address _174.138.2.190/bmicalc.html_. On the left side: the source code of the HTML page. On the right side: server-side "raw" PHP source code which is not seen by the client program, our web browser in this case (PHP code = server-side execution/only web server sees the code, JavaScript code = client-side execution/client can see the code, too):
![bmicalc-sample](https://raw.githubusercontent.com/Fincer-altego/basics-of-a-linux-server-school-course-/master/bmicalc-sample.png)
**Edit** The following changes has been done after answering the assignment:
- PHP-related HTTP method GET has been changed to POST method afterwards because usage of the GET method leads to visible input values in a web browser URL field. This doesn't happen when using the POST method.
- Implementation of server-side solutions which restrict user accessibility to other web server directories and files.
- PHP code has been merged with the HTML document, i.e. there is no additional .php file in the web server anymore.

+ 1027
- 0
exercises/h5.md
File diff suppressed because it is too large
View File


+ 127
- 0
exercises/h6.md View File

@ -0,0 +1,127 @@
Linux servers - Exercice 6
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Write and execute "Hello world" in three code languages of your choice. Set up a necessary development environments.
--------------
**Answer:**
Let's write "Hello World" in the following three languages: perl, python 3 and C. Let's install needed development tools:
```
sudo apt-get -y install perl python gcc
```
(GCC = [Gnu Compiler Collection](https://gcc.gnu.org/) + [Wikipedia](https://en.wikipedia.org/wiki/GNU_Compiler_Collection))
Let's write the codes into a subfolder _hello-world_ in user's home dir:
```
[newuser@goauldhost: ~ ]$ mkdir ~/hello-world
[newuser@goauldhost: ~ ]$ cd hello-world/
[newuser@goauldhost: hello-world ]$ touch {hello.py,hello.pl,hello.c}
```
- hello.py = "Hello World" written in Python 3
- hello.pl = "Hello World" written in Perl
- hello.c = "Hello World" written in C
Hello World program written in three languages (nano editor used):
**Perl (hello.pl):**
```
#!/usr/bin/env perl
# Declare runtime environment above
# Include these to find common warnings, syntax errors etc in the code
import strict;
import warnings;
# Print Hello World! with a newline
print "Hello World!\n";
```
**Python (hello.py):**
```
#!/usr/bin/env python3
# Declare runtime environment above
# Print Hello World!
print("Hello World!");
```
**NOTE!** Pay attention when referring to python executable. On some Linux distributions, python refers to python2, and on some, python refers to python3. Practices differ. It can be safer to use 'python3' or 'python2' instead of just 'python'.
In a simple program like "Hello world" this python issue doesn't really matter but if any python libraries are imported into the code, you must know which python environment to use, Python 2 or Python 3.
**C (hello.c):**
```
// Include Standard Input Output Library (stdio.h core library included in)
#include <stdio.h>
// Declare main function for the program. Int for returning a integer
int main()
{
// Print Hello World! (stdout in CLI)
printf("Hello World!");
// Return boolean value "true" to the execution environment
return 0;
}
```
hello.c requires compiling from source code to executable binary file. Therefore, we compile the source code with the following command in _hello-world_ folder:
```
gcc -o hello hello.c
```
after which our 'binary' can be executed with
```
~/hello-world/hello
```
All programs give stdout/output string "Hello World!" in our shell environment:
```
[newuser@goauldhost: hello-world ]$ pwd
/home/newuser/hello-world
[newuser@goauldhost: ~ ]$ python hello.py
Hello World!
[newuser@goauldhost: ~ ]$ perl hello.pl
Hello World!
[newuser@goauldhost: ~ ]$ ./hello
Hello World!
```
**b)** (optional) Write a simple program for some practical purpose with each language. Ideas: utilize the key benefits of your language of choice. Take an input value from a user, generate a calculation and print the output (input-processing-layout).
--------------
**Answer:**
This could have been very interesting assignment to work out. Unfortunately, my working laptop was broken so that I was not effectively able to write new code. I have worked with a perl program which generated random port and TCP/UDP sequences for knockd daemon. In addition, I have worked with some updates to Wine Configuration window (winecfg, part of Wine program) in C language:
![winecfg-update](https://i.imgur.com/SihmPUA.jpg)
Some Python and bash based program-related scripts and code updates are available here:
- [PlayOnLinux patches](https://github.com/Fincer/linux-patches-and-scripts/tree/master/playonlinux).
- Little CMake script targeted for compiling C++ code can be found [here](https://github.com/Fincer/linux-patches-and-scripts/tree/master/xclipshow)
- Some basic scripting to convert videos with ffmpeg in KDE DE: [ffmpeg-fileconversion-video](https://github.com/Fincer/linux-patches-and-scripts/blob/master/kde-servicemenus-multimediatools/ffmpeg-fileconversion-video.sh)
etc.

+ 507
- 0
exercises/h7.md View File

@ -0,0 +1,507 @@
Linux servers - Exercice 7
==============
*Disclaimer:*
--------------
This exercise is a part of [Linux servers (ICT4TN021, spring 2018) // Linux-palvelimet (ICT4TN021, kevät 2018)](http://www.haaga-helia.fi/fi/opinto-opas/opintojaksokuvaukset/ICT4TN021) school course organized as a part of Information Technology studies in Haaga-Helia university of Applied Sciences, Helsinki, Finland. Course lecturer [Tero Karvinen](http://terokarvinen.com/) has defined the original assignment descriptions in Finnish presented in this document in English. Answers and translations have been written by Pekka Helenius (me, ~ Fincer).
**a)** Solve a previously published final major test of this school course (You can find them with DuckDuckGo or Google).
--------------
**Answer:**
Valitaan harjoitus [ict4tn021-1 alkusyksyllä 2016](http://terokarvinen.com/2016/arvioitava-laboratorioharjoitus-%e2%80%93-linux-palvelimet-ict4tn021-1-uusi-ops-alkusyksylla-2016).
----------------------------
**Initial set-up for remote control of a server**
_Prepare to control the server from abroad. Secure the server with firewall_
We shall install SSH server daemon to the targeted server computer. It is assumed that local or other direct access to the server command line is available since SSH server daemon is not yet installed, thus preventing direct, remote SSH control of the server.
Once having access to the server command line, we shall test that the computer can reach the internet. Test the following commands:
```
ifconfig
ping www.duckduckgo.com
ip -br link show
```
Command explanations:
- [ifconfig](https://www.linux.fi/wiki/Ifconfig) = Has IP address determined to a network interface on the server?
- [ping](https://fi.wikipedia.org/wiki/Ping) = Does a site respond to ICMP requests?
- ip = Is the network interface, which should have connection to the internet, up and active (UP)?
When the internet connection is established, we shall proceed by installing the following packages, assuming that the server uses a Debian-based Linux distribution: openssh-server, openssh-sftp-server, openssh-client
On some distributions, those packages can be installed simply issuing:
```
sudo apt-get update
sudo apt-get -y install ssh
```
We shall confirm that the SSH server daemon starts during the server boot-up. We shall also confirm that the SSH server daemon is up and running and its status is 'active':
```
sudo systemctl enable ssh.service
systemctl status ssh.service
```
**NOTE!** Alternative commands can also be used, like:
```
sudo systemctl enable sshd
systemctl is-active sshd
```
etc.
Has our firewall program installed on the system?
```
which ufw
which iptables
```
If both of those executable files ('ufw' and 'iptables') are found on the system, we shall accept network traffic to port 22, protocol TCP (INPUT 22/TCP). Let's apply these rules to our firewall:
```
sudo ufw allow 22/tcp
```
Make sure our firewall program 'ufw' (Uncomplicated Firewall) is enabled and turned on:
```
sudo ufw enable
sudo systemctl enable ufw.service
```
**NOTE!** By default, Linux firewall blocks all input traffic. Therefore, SSH input traffic must separately be allowed like described above.
----------------------------
**Security set-up for a company**
_Install remotely working security tools for our company. (In this assignment, you can assume that installing a package or packages from our repository is secure)_
The following commands have been pre-determined in the assignment:
- download .deb installer file with 'wget' command
- install the downloaded .deb package with command 'sudo dpkg -i' which extract a new repository file 'terorep.list' into '/etc/apt/sources.list.d/')
- update package databases with command 'sudo apt-get update'
- install package 'terowatch' which is made available by the new repository. The package is available for Ubuntu distribution, version 16.04 LTS. The repository file itself contains string 'deb http://terokarvinen.com/qrs/terorep/ precise main'
**NOTE!** 'terowatch' package uses network interface 'eth0' by default. However, we haven't defined such interface in our system configuration (this can be fixed by adding 'net.ifnames=0' in udev rules/kernel boot parameters in syslinux or grub). Instead, we use network interface 'enp4s0'.
Proper fix to this issue would be patching the code and applying the patch into the deb package. Another solution would be making the proper fix directly to the source code. In this assignment, we directly modify the executable file, written in Python. This method is not recommended but for the extent of this assignment, it is sufficient solution to proceed.