Labs, Pr1_en

12. Lab: Linux and CLI

Remote lab
Note: In 2024 autumn, this lab is intended to be solved remotely. Check for more information on MM.

Lab materials

Help on commands

If you need help with some of the commands, there are three main ways to get information on the commands. Third and the slowest would be using the internet, so we’ll skip this one. The other two sources of information are available quickly from your command line

tldr (simplified community-driven man pages)

This is a quick “handbook” composed by daily Linux users. It’s main purpose is to quickly and simply describe the most common ways to use each of the programs, as well as some less common, but important ones. The downside to this utility is that it often lacks detailed information and explanations, however it will get you the information you need or at least a starting point most of the time

The tool does not come as default, but we have installed it on the lab computers.

To use in the command line type tldr command

Alternatively, you can use the web version from https://tldr.sh

I.e.. to learn how to copy files, write tldr cp

man pages

man stands for manual and comes native with Linux. It includes manuals for commands, libraries, system calls etc. It includes manuals on how to use most programs. To use a manual, just type man command .

E.g. to check out how to copy files, you can run  man cp.

To close the manual, hit the letter  q . To search within a manual, hit   / , followed by what you are searching from. All other hotkeys and additional help can be read by hitting the key  h .

The advantage of  man  compared to  tldr  is that its very detailed and has a variety of different types of information. This can also be a downside for new users, as they are very throughout and might discourage or even terrify new users.  However, once you get used to reading information in this way, it becomes irreplaceable.

Lab tasks

There are two lab tasks this week. Both of them end up with a web form that you must fill! You do not need to present us your solutions after completion. Solutions are presented by filling out web based forms.

Task 1: working on the command line

The following task will introduce you to the command line and some basics working on it.

The task will end with a web form that must be filled. The submission of the web form confirms the completion of the task.

Pick your method

Before you can start solving you will need access to Linux and a terminal window. Some options that would work for this task:

  1. Solve the task in the lab computer
  2. Solve the task by using a remote desktop connection to the lab
    Guide: Remote access using RDP
  3. Solve the task by connecting to the lab computers through an SSH tunnel
    Guide: SSH connection guide

Note: Don’t use your own Linux for this task as there are some parts of it that can only be done with the university mount points.

Solving the task

The task is written as a step-by-step workflow that you should follow. All actions should be done using the command line. In the end, you will need to submit your command history. Prepare yourself enough time to solve the task 1 from start to finish, as you will need to submit your command history in the end of the task.

Step 1: Pick your working directory

In this step, you need to choose the location for the files that you will create during this lab. Most likely it should be somewhere on your P drive – i.e. ~/P/iax0583/lab11 .

The following commands will help you navigate:

  • Use  pwd  to check your current working directory
  • Use cd  to navigate from and to directories
  • Use  mkdir  to create a directory
  • Use  ls  to check the contents of the directory. Use additional arguments for more details.
  • Use mv  to rename directories and files  (if you happened to make a mistake)
  • Use tldr command  and man command  to get help on the commands.

Change the directory to the one you intend to solve your lab task

Step 2: Download the test program

The program wget  is most often used to download files off the internet. Use this to download a tester program by running the following command

Step 3: Unpack the tester and execute it

You’ve just downloaded a zip archive. Use the program called unzip to unpack it. First learn how to use it. Then pay close attention to the output that this program will give you in order to understand what was unpacked and where! Double-checking everything with ls  doesn’t hurt either. You need to understand what was unpacked and where!

You may need to manually make the test program executable

Once you have everything unpacked and found the testing program, you may need to add execute permission to it. Note, that it may also already be set by default. If you need to add the execute permission, use chmod  to change file access permissions (including adding execution privilege)

Execute the testing program

From this point on you can try to run the program provided. Try it now! Execute the tester!

The program will stop when an unmet requirement is encountered. You should retry the testing program  after every step from this point beyond.

You should see the following output:

Execute the tester after each step in this task.

Step 4: Download VIM configuration file

NB! You should skip this step if you already have one!

Vim is one of the more commonly used command line based text editors. It’s a lot more configurable and capable compared to basic text editors such as nano.

Lets download a configuration file that will make it a little bit nicer. I.e. introduce syntax highlighting, automatic tabs etc. The file needs to be stored in your home folder – that’s were Vim will be looking for it.

1. Go to your home directory. To do this, use the command cd .  Shorthand for your home directory is ~

2. Download the configuration file. Use wget  to download the following file: blue.pri.ee/.vimrc

Use the tester to verify completion of this step!

Step 5: Copy over an additional data file with the secret

Go to the M folder in your home directory (~/M/). Find your lecturer’s public directory and go to it. In there you will find a hidden directory (it starts with a dot).

Inside you will find a .dat file. Copy that file over to the same folder as the test program. For this you will need to use the cp  command. Specify the file you want to copy and the path you want to copy it to.

To test if this worked, run the program and see if it finds the secret!

Step 6: Next up, you need to create a file with your matricula

The file you need to create is named matricula . It needs to be saved into the same folder as the rest of the files so far. To create it, I’ll provide two possible options.

Option 1: Open up your preferred command line text editor, write a file called “matricula”

Vim guide:

    • Start by writing vim matricula  to create a file called matricula and open it in Vim.
    • Press ‘i’ to enter writing mode.
    • Now write your matricula number into the file.
    • Press ‘esc’ to exit write mode.
    • write :wq  in normal mode to write the file and exit

Option 2: This is what most people who use terminal regularly would likely do. They would use the echo  command, that just prints the passed text back on the screen and then redirect that output to a file, e.g. echo "text" > file

Step 7: Write a program that prints “Hello world!”

NB! Before continuing, remind yourself what is a file extension. What extension is used by C source code and what is used by programs!

Again use a command line text editor. We recommend Vim, as it supports C code, but you can use any other editor as well.

Once done, compile the program. Use -o hello  to specify the output (name of the program) as “hello”.

NB! Do not write the name of your source code file after -o. This will overwrite your source code!

Now run the test program that you downloaded from us. If everything was successful, it will create a new file with instructions.
Follow those instructions to complete the task.

Task 2: filtering logs and web home

In this task we will look at a tool that is widely used for digging through large amounts of data. We will also take a look at piping the standard output stream of one program into the standard input stream of another program. Lastly, we’ll do a quick introduction to your web home.

Input data

I’ve generated a file with someone random data that would simulate a log file from a robot. These kind of log files are typically extremely long. Usually we only look into them when something goes wrong and even then we are usually interested in only a fraction of what it contains.

Log file is at my web home: http://www.tud.ttu.ee/web/Risto.Heinsar/sensors.txt

Filtering data

To filter long text files we use a tool called grep . Grep is a pattern matching tool that will try to match entered the pattern and print all the lines containing it. Other lines will be skipped.

The patterns can be simple such as a single letter words or part of a string, e.g.  SENSOR , ERROR , firefox etc. In this case we would run the program as  grep pattern .

The patterns can also be more complex, containing regular expressions. To enable the use of regular expressions, we add a an argument -E  when executing. E.g.  grep -E sensor[1-3]:  will display us all the lines containing either  sensor1: , sensor2:  or sensor3: . Notice that the colon was also included in the pattern.

You can test your regular expressions here: https://regex101.com

Hint 1: You can grep the output of grep again – you don’t need to do everything with one execution if that is too difficult. Though you should definitely try!

Hint 2: the output of grep can be written to a file using output stream redirection.

Hint 3: You can also do regular expressions that include logical operations (e.g. one or the other).

Submitting the task (base and advanced)
  1. Decide whether you wish to solve the base or advanced task and solve it.
  2. Copy the resulting output file to your web home and figure out the public address of it
  3. Fill the following web form: https://forms.office.com/r/kb72jTqeXu

NB! If you have technical issues with the web home, let us know after filling out the form. Show us the location of your file and the addresses you tried.

Base task requiremets

Your task is to find all the lines in the file given by me which have the severity level of ERROR and which are related to the sensor number derived from your matricula number using the formula matricula % 10 .

Example:

Student code 123456IACB -> matricula 123456.
123456 % 6 is 6

So the output has to be about SENSOR6. Some examples:

The task can be solved either in one go or in parts – it’s up to you.

Write this output to a text file using stream redirection. The name of the created file should be your student_code.txt. E.g. if your student code is 123456IACB, then your file name should be called 123456iacb.txt.

Extra task requirements

Your task is to find all lines containing the severity level ERROR. The lines you are looking for should contain the sensors matricula % 10  and motor number matricula % 4 .

Example:

Student code 123456IACB, matricula 123456
123456 % 10 -> 6
123456 % 4 -> 0

This means we are looking for SENSOR6 and MOTOR0. Some examples of the lines:

The task can be solved either in one go or in parts – it’s up to you.

Write this output to a text file using stream redirection. The name of the created file should be your student_code.txt. E.g. if your student code is 123456IACB, then your file name should be called 123456iacb.txt.

Using the web drive

All students have their own web drive where they can hold and publish simple web pages and files. It’s the W drive. Copy the output of your filtered data to that drive.

More information on the web drive (Estonian only): https://taltech.atlassian.net/wiki/spaces/ITI/pages/38994529/li+pilase+veebikataloog+Online+student+directory

Copy the resulting file to your web drive. You can use cp  to copy files.

Now the file should be accessible under one of the following links:

  • http://www.tud.ttu.ee/web/uni-id/student_code.txt
  • http://www.tud.ttu.ee/web/Firstname.Lastname/student_code.txt

Now fill out the form!

Additional content