--- title: 'Linux Basics' description: 'A basic introduction to Linux' date: '2024-06-03' tags: ['linux', 'cli'] toc: true url: basic-linux --- This is a very brief introduction to working with Linux on the command line. It's a collection of commands and tips&tricks that I have collected through out my Linux journey. Some of it spesific to Debian derivatives. Maybe you find something usefull too. I will probably update this from time to time. You can look at the history if you want to see whats changed. ## Find OS information Print information about your Linux distro. ```console lsb_release -a cat /etc/os-release uname -a ``` See how long your machine has been running since last boot. ```console claw0ry@lnx:~$ uptime 11:36:40 up 4:02, 1 user, load average: 0.00, 0.00, 0.00 ``` ## Built-in documentation with manpages In Linux we can use the **man** command to read built-in documentation. ```plaintext man COMMAND ``` To list all available manpages. ```console man -k . ``` You can combine it with grep to filter the result if you are not quite sure what the manpage entry is called. ```console claw0ry@lnx:~$ man -k . | grep ssh ssh (1) - OpenSSH remote login client ssh-add (1) - adds private key identities to the OpenSSH authentication agent ssh-agent (1) - OpenSSH authentication agent ssh-argv0 (1) - replaces the old ssh command-name as hostname handling ssh-copy-id (1) - use locally available keys to authorise logins on a remote machine ssh-keygen (1) - OpenSSH authentication key utility ssh-keyscan (1) - gather SSH public keys from servers ssh-keysign (8) - OpenSSH helper for host-based authentication ssh-pkcs11-helper (8) - OpenSSH helper for PKCS#11 support ssh-sk-helper (8) - OpenSSH helper for FIDO authenticator support ssh_config (5) - OpenSSH client configuration file sshd (8) - OpenSSH daemon sshd_config (5) - OpenSSH daemon configuration file ~:$ man sshd ``` ## Manage files and directories ### List files and directories ```console # list name of files and directories under /var ls /var # show permissions, owners, size, modification date etc for each file and directory in /var ls -l /var # same as above, but also recursivly ls -lR /var # '-h' lists sizes in human readble format, '-F' will append a '/' to all directories ls -lhF /var # '-a' will also list hidden files and directories (that starts with '.') ls -la $HOME ``` ### Finding files ```console # list all files and directories in /var find /var # list all files and directories in /var in 'ls' style find /var -ls # find all files that ends with '.log' in /var and execute 'type' on each file find /var -name "*.log" -exec type {} \; ``` ### Create files ```console touch file.txt vim file.txt nano file.txt echo "Some content" > file.txt cat file.txt > another_file.txt ls -lR /var > dir_list.txt ``` Write directly from **stdin** to file (also work with append). ```console claw0ry@lnx:~$ cat > file.txt Write some lines of text claw0ry@lnx:~$ cat file.txt Write some lines of text ``` ### Search content of files We can use **grep** to search for spesific phrases/words or patterns in a file or files. ```console grep 'Failed' /var/log/auth.log ``` ### Search logfiles Debian derivatives now use **journalctl** to display logs. Here are some basic filtering techniques. - '--since': Filter on time. Can use 'yesterday', 'today' or a datetime '2024-07-09', '2024-07-09 18:00:00' - '--grep': Filter using grep on the `MESSAGE=` field - '--unit': Filter on the service, e.g ssh, nginx, apache2 etc ```console journalctl --since yesterday --grep 'failed' --unit ssh ``` ### Archiving files and directories In Linux we use the **tar** command to create an archive. In most cases we also want to compress it with zip. - 'c': Create an archive - 'v': Verbose - 'z': Compress the archive with zip - 'f': Specify output file ```console # backup my home directory to mybackup.tar.gz tar -cvzf mybackup.tar.gz /home/claw0ry ``` If we omit the 'z' flag, tar would create an archive with the exact same size as my homefolder. We can list the contents of a tar archive (or tarball). ```console tar -tvf mybackup.tar.gz ``` To decompress or unpack an archive we also use **tar**. - 'x': Extract an archive - 'C': Use another directory than current working directory ```console # extract to current working directory tar -xf mybackup.tar.gz # or to a specific directory tar -xf mybackup.tar.gz -C /tmp/backup ``` ### Transfer files We can transfer files to and from other computers/servers with **scp** or **rsync**. NOTE: If we have the same username on both ends, we dont need to specify the USER. Using **scp**. ```plaintext scp [src] [dest] # from local to remote scp [local-path] [USER@]HOST:[remote-path] # from remote to local scp [USER@]HOST:[remote-path] [local-path] ``` Using **rsync**. ```plaintext rsync -aP [src] [dest] # from local to remote rsync -aP [local-path] [USER@]HOST:[remote-path] # from remote to local rsync -aP [USER@]HOST:[remote-path] [local-path] # optionally we can specify -n to do a dry-run to see what would happen rsync -naP [local-path] [USER@]HOST:[remote-path] ``` Real world examples. ```console # copy my local '.bashrc' to a lab server scp /home/claw0ry/.bashrc lab_user@lab1.example.com:/home/lab_user/.bashrc # sync my blog folder to webserver rsync -aP /home/claw0ry/code/blog web.claw0ry.net:/var/www ``` ### Change permissions, owners and groups Which files and directories can be accessed, modified etc by whom is determined by permissions and owners. We can see the permissions and owners of a file/directory with **ls**. ```console claw0ry@lnx:~$ ls -la /home/cloud_user total 20 drwxr-xr-x 3 cloud_user cloud_user 4096 Jul 9 08:25 . drwxr-xr-x 4 root root 4096 Aug 24 2023 .. -rw-r--r-- 1 cloud_user cloud_user 1 Feb 29 19:38 .bash_history -rw-r--r-- 1 cloud_user cloud_user 0 Feb 29 19:16 .cloud-locale-test.skip -rw------- 1 cloud_user cloud_user 57 Jul 9 08:17 .lesshst drwx------ 2 cloud_user cloud_user 4096 Aug 24 2023 .ssh -rw-r--r-- 1 cloud_user cloud_user 0 Aug 28 2023 .sudo_as_admin_successful ``` The first character idicates what kind of item it is: - '-': file - 'd': directory - 'l': symbolic link Next we have the permissions divided into three groups. The first three are the owners permissions, then we have the groups permissions and lastly others permissions. - '-': none - 'r': read - 'w': write - 'x': execute The owner of the file is specified in the first column that you see `cloud_user`, and the group is specified in the column next to it. In this example we can see that all files and directories is owned by the `cloud_user` user and `cloud_user` group execpt for the parent directory which is owned by `root` user and `root` group. #### Change owners and groups We can change the owner and group with the **chown** command. ```plaintext chown [-R] [OWNER][:GROUP] FILE # only change owner chown OWNER FILE # only change group chown :GROUP FILE # change both, but not the same chown OWNER:GROUP FILE # change both to the same chown OWNER: FILE ``` - 'R': Recursive (will also change owner for all subfiles and directories) ```console claw0ry@lnx:~$ cd /home/cloud_user claw0ry@lnx:~$ touch file.txt claw0ry@lnx:~$ ls -l file.txt -rw-r--r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt claw0ry@lnx:~$ chown another_user:another_user file.txt claw0ry@lnx:~$ ls -l file.txt -rw-r--r-- 1 another_user another_user 0 Jul 9 08:37 file.txt claw0ry@lnx:~$ chown cloud_user file.txt -rw-r--r-- 1 cloud_user another_user 0 Jul 9 08:37 file.txt claw0ry@lnx:~$ chown :cloud_user file.txt -rw-r--r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt ``` #### Change permissions To set permissions we use the **chmod** command. There are two modes to set permissions with; symbolic and octal. ##### Octal mode ```plaintext chmod OCTAL FILE ``` - read(r): 4 - write(w): 2 - execute(x): 1 To calculate the permissions bits we just need to add the permissions together. So for read and write access it will be `4+2=6`. For all permissions it will be `4+2+1=7`. Let's say we have `file.txt` and we want the owner to have full permissions, the group to have read and write and others to have none. This would calculate to `760`. We can set these permissions with the following command. ```console claw0ry@lnx:~$ chmod 760 file.txt claw0ry@lnx:~$ ls -l file.txt -rwxrw---- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt ``` These are the most common permissions used: - 777 (everyone has full permissions) - 644 (owner has read+write, and everyone else has read) - 750 (owner has full permissions, groups has read+execute, and everyone else has none) - 600 (owner is the only user that has access) ##### symbolic mode ```plaintext chmod MODE FILE ``` - '+': add permission - '-': remove permission - '=': set permisions - 'u': owner (user) - 'g': group - 'o': others ```console # add read permissions for the owner chmod u+r file.txt # remove execute for owner chmod u-x file.txt # set owners permissions to read, write, execute chmod u=rwx file.txt ``` We can combine permissions with `,`. ```console claw0ry@lnx:~$ ls -l file.txt -rwxrw---- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt claw0ry@lnx:~$ chmod u=rwx,g=rw,o=r file.txt claw0ry@lnx:~$ ls -l file.txt -rwxrw-r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt ``` ## Input/output redirection In Linux shells there is a concept of three streams. - The standard input (stdin), which takes the users input. - The standard out (stdout), which is the output of a command. It is usually displayed in your terminal. - The standard error (stderr), which is the error messages from a command. This is also usually displayed in your terminal alongside stdout. Write errors to file instead of console. ```console # no redirection claw0ry@lnx:~$ ls -l nonexisting ls: nonexisting: No such file or directory # with redirection of stderr claw0ry@lnx:~$ ls -l nonexisting 2> ls_error.txt claw0ry@lnx:~$ cat ls_error.txt ls: nonexisting: No such file or directory ``` Redirect **stdout** to a file and only show errors in the console. ```console # these two are equivalent find /root 1> dirs_i_can_read.txt find /root > dirs_i_can_read.txt ``` Redirect both **stdout** and **stderr** to the same destination. ```console find /root 2>&1 root_dirs.txt ``` This tells Linux to redirect **stderr** to **stdout** and then **stdout** to a file. ## Reboot and PowerOff ```plaintext shutdown [OPTIONS] [TIME] [WALL...] ``` ```console # poweroff poweroff shutdown -P shutdown -P 20:00 shutdown -P +5 shutdown -P +5 # reboot reboot shutdown -r [TIME] shutdown -r 20:00 shutdown -r +5 'Server will be rebooted for maintenance!' ``` ## Manage users and groups ```console # Add a new user adduser LOGIN # add user to group gpasswd -a USER GROUP # give sudo access cat /etc/sudoers gpasswd -a USER sudo ``` ## Disk space List filesystem space usage. ```console claw0ry@lnx:~$ df -h Filesystem Size Used Avail Use% Mounted on udev 467M 0 467M 0% /dev tmpfs 96M 484K 95M 1% /run /dev/nvme0n1p1 20G 2.1G 17G 11% / tmpfs 477M 0 477M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/nvme0n1p15 124M 12M 113M 10% /boot/efi tmpfs 96M 0 96M 0% /run/user/1001 tmpfs 96M 0 96M 0% /run/user/1002 ``` List directory space usage. ```console claw0ry@lnx:~$ du -hs /home/cloud_user 36K /home/cloud_user claw0ry@lnx:~$ sudo du -hs /var/log 55M /var/log ``` If we omit the `-s` we will also see subdirectories. ```console claw0ry@lnx:~$ sudo du -h /var/log 4.0K /var/log/runit/ssh 8.0K /var/log/runit 52K /var/log/apt 4.0K /var/log/private 53M /var/log/journal/ec228e8f22cbefcdced18ace3b891949 53M /var/log/journal 16K /var/log/unattended-upgrades 32K /var/log/amazon/ssm/audits 852K /var/log/amazon/ssm 856K /var/log/amazon 55M /var/log ``` You can also sort the result to find what takes the most space. Since we are using the `-h` flag on **du** to get human readable sizes, we also provide the same flag to **sort** so that I will sort correctly. ```console claw0ry@lnx:~$ sudo du -h /var/log | sort -h 4.0K /var/log/chrony 4.0K /var/log/private 4.0K /var/log/runit/ssh 8.0K /var/log/runit 36K /var/log/apt 136K /var/log/nginx 440K /var/log/sysstat 25M /var/log 25M /var/log/journal 25M /var/log/journal/b8d53a40a48e4a9baaf67b1d19735980 ``` To get the size of a file(s), you can use the **ls** command with `-lh` to get the size in human readable format. Another alternative is **ncdu** which is an interactive version of `du` built with ncurses. It does not come pre-installed. ```console sudo apt install ncdu # replace /home with whatevery starting path you want ncdu /home ``` Now you can navigate and see the size of files and directories interactivly. ## Processes Dashboard overview of your processes, CPU and memory usage. ```console htop ``` Snapshot of the current process state. ```console ps aux ``` Kill a process. ```console pkill -9 PID ``` ## Schedule tasks In Linux we use **cron** with a `crontab` file to scedule runs of commands or scripts. The crontab template looks like this. ```plaintext # +----------- minute (0 - 59) # | +--------- hour (0 - 23) # | | +------- day of month (1 - 31) # | | | +----- month (1 - 12) # | | | | +--- day of week (0 - 6) (starts at sunday) # | | | | | # * * * * * COMMAND # # NOTES # The 'month' and 'day of week' can be represented by either a number, name or shortname. # e.g 1, January, Jan # e.g 1, Monday, Mon # # Command can be either a script or a command. You can seperate them with ';' to run # multiple commands. # # Visit https://crontab.guru for a visual representation of cron schedule ``` To edit your crontab you use the **crontab** command. ```console crontab -e ``` You can also specify the user to edit crontab for if you want it to run as another user. ```console crontab -u USER -e ``` > NOTE: When adding commands to your crontab, make sure you use full paths for both commands and scripts. You can check the cron logs using **journalctl**. ```console journalctl --unit cron ``` ## Manage and update packages ### dpkg ```console # install .deb package from file dpkg -i FILE.deb # list installed packages dpkg -l # remove package dpkg -r NAME # search installed packages dpkg -S PATTERN ``` NOTE: When intalling packages with `dpkg -i` it does not perform any dependency checks. If a dependency is missing it will fail. ### apt ```console # update package list apt update # upgrade installed packages apt upgrade # install package(s) apt install NAME # remove package apt remove NAME # remove package and config apt purge NAME # remove unwanted packages apt autoremove # search packages apt search PATTERN # show package details apt show NAME # list installed packages apt list --installed # list packages that can be upgraded apt list --upgradeable ```