diff options
Diffstat (limited to 'content/posts/basic_linux.md')
| -rw-r--r-- | content/posts/basic_linux.md | 624 |
1 files changed, 624 insertions, 0 deletions
diff --git a/content/posts/basic_linux.md b/content/posts/basic_linux.md new file mode 100644 index 0000000..ff30544 --- /dev/null +++ b/content/posts/basic_linux.md @@ -0,0 +1,624 @@ +--- +title: 'Linux Basics' +description: 'A basic introduction to Linux' +date: '2024-06-03' +tags: ['linux', 'cli'] +toc: true +url: basic-linux +--- + +This is a very brief introduction to working with Linux on the command line. It's a collection of commands and tips&tricks that I have collected through out my Linux journey. Some of it spesific to Debian derivatives. Maybe you find something usefull too. I will probably update this from time to time. You can look at the history if you want to see whats changed. + +<!--more--> + +## Find OS information + +Print information about your Linux distro. +```console +lsb_release -a +cat /etc/os-release +uname -a +``` + +See how long your machine has been running since last boot. + +```console +claw0ry@lnx:~$ uptime + 11:36:40 up 4:02, 1 user, load average: 0.00, 0.00, 0.00 +``` + +## Built-in documentation with manpages + +In Linux we can use the **man** command to read built-in documentation. + +```plaintext +man COMMAND +``` + +To list all available manpages. + +```console +man -k . +``` + +You can combine it with grep to filter the result if you are not quite sure what the manpage entry is called. + +```console +claw0ry@lnx:~$ man -k . | grep ssh +ssh (1) - OpenSSH remote login client +ssh-add (1) - adds private key identities to the OpenSSH authentication agent +ssh-agent (1) - OpenSSH authentication agent +ssh-argv0 (1) - replaces the old ssh command-name as hostname handling +ssh-copy-id (1) - use locally available keys to authorise logins on a remote machine +ssh-keygen (1) - OpenSSH authentication key utility +ssh-keyscan (1) - gather SSH public keys from servers +ssh-keysign (8) - OpenSSH helper for host-based authentication +ssh-pkcs11-helper (8) - OpenSSH helper for PKCS#11 support +ssh-sk-helper (8) - OpenSSH helper for FIDO authenticator support +ssh_config (5) - OpenSSH client configuration file +sshd (8) - OpenSSH daemon +sshd_config (5) - OpenSSH daemon configuration file +~:$ man sshd +``` + +## Manage files and directories + +### List files and directories + +```console +# list name of files and directories under /var +ls /var + +# show permissions, owners, size, modification date etc for each file and directory in /var +ls -l /var + +# same as above, but also recursivly +ls -lR /var + +# '-h' lists sizes in human readble format, '-F' will append a '/' to all directories +ls -lhF /var + +# '-a' will also list hidden files and directories (that starts with '.') +ls -la $HOME +``` + +### Finding files + +```console +# list all files and directories in /var +find /var + +# list all files and directories in /var in 'ls' style +find /var -ls + +# find all files that ends with '.log' in /var and execute 'type' on each file +find /var -name "*.log" -exec type {} \; +``` + +### Create files + +```console +touch file.txt + +vim file.txt + +nano file.txt + +echo "Some content" > file.txt + +cat file.txt > another_file.txt + +ls -lR /var > dir_list.txt +``` + +Write directly from **stdin** to file (also work with append). + +```console +claw0ry@lnx:~$ cat > file.txt +Write some +lines +of text +<CTRL-D> +claw0ry@lnx:~$ cat file.txt +Write some +lines +of text +``` + + +### Search content of files + +We can use **grep** to search for spesific phrases/words or patterns in a file or files. + +```console +grep 'Failed' /var/log/auth.log +``` + +### Search logfiles + +Debian derivatives now use **journalctl** to display logs. Here are some basic filtering techniques. + +- '--since': Filter on time. Can use 'yesterday', 'today' or a datetime '2024-07-09', '2024-07-09 18:00:00' +- '--grep': Filter using grep on the `MESSAGE=` field +- '--unit': Filter on the service, e.g ssh, nginx, apache2 etc + +```console +journalctl --since yesterday --grep 'failed' --unit ssh +``` + +### Archiving files and directories + +In Linux we use the **tar** command to create an archive. In most cases we also want to compress it with zip. + +- 'c': Create an archive +- 'v': Verbose +- 'z': Compress the archive with zip +- 'f': Specify output file + +```console +# backup my home directory to mybackup.tar.gz +tar -cvzf mybackup.tar.gz /home/claw0ry +``` + +If we omit the 'z' flag, tar would create an archive with the exact same size as my homefolder. + +We can list the contents of a tar archive (or tarball). + +```console +tar -tvf mybackup.tar.gz +``` + +To decompress or unpack an archive we also use **tar**. + +- 'x': Extract an archive +- 'C': Use another directory than current working directory + +```console +# extract to current working directory +tar -xf mybackup.tar.gz + +# or to a specific directory +tar -xf mybackup.tar.gz -C /tmp/backup +``` + +### Transfer files + +We can transfer files to and from other computers/servers with **scp** or **rsync**. + +NOTE: If we have the same username on both ends, we dont need to specify the USER. + +Using **scp**. + +```plaintext +scp [src] [dest] + +# from local to remote +scp [local-path] [USER@]HOST:[remote-path] + +# from remote to local +scp [USER@]HOST:[remote-path] [local-path] +``` + +Using **rsync**. + +```plaintext +rsync -aP [src] [dest] + +# from local to remote +rsync -aP [local-path] [USER@]HOST:[remote-path] + +# from remote to local +rsync -aP [USER@]HOST:[remote-path] [local-path] + +# optionally we can specify -n to do a dry-run to see what would happen +rsync -naP [local-path] [USER@]HOST:[remote-path] +``` + +Real world examples. + +```console +# copy my local '.bashrc' to a lab server +scp /home/claw0ry/.bashrc lab_user@lab1.example.com:/home/lab_user/.bashrc + +# sync my blog folder to webserver +rsync -aP /home/claw0ry/code/blog web.claw0ry.net:/var/www +``` + +### Change permissions, owners and groups + +Which files and directories can be accessed, modified etc by whom is determined by permissions and owners. + +We can see the permissions and owners of a file/directory with **ls**. + +```console +claw0ry@lnx:~$ ls -la /home/cloud_user +total 20 +drwxr-xr-x 3 cloud_user cloud_user 4096 Jul 9 08:25 . +drwxr-xr-x 4 root root 4096 Aug 24 2023 .. +-rw-r--r-- 1 cloud_user cloud_user 1 Feb 29 19:38 .bash_history +-rw-r--r-- 1 cloud_user cloud_user 0 Feb 29 19:16 .cloud-locale-test.skip +-rw------- 1 cloud_user cloud_user 57 Jul 9 08:17 .lesshst +drwx------ 2 cloud_user cloud_user 4096 Aug 24 2023 .ssh +-rw-r--r-- 1 cloud_user cloud_user 0 Aug 28 2023 .sudo_as_admin_successful +``` +The first character idicates what kind of item it is: + +- '-': file +- 'd': directory +- 'l': symbolic link + +Next we have the permissions divided into three groups. The first three are the owners permissions, then we have the groups permissions and lastly others permissions. + +- '-': none +- 'r': read +- 'w': write +- 'x': execute + +The owner of the file is specified in the first column that you see `cloud_user`, and the group is specified in the column next to it. In this example we can see that all files and directories is owned by the `cloud_user` user and `cloud_user` group execpt for the parent directory which is owned by `root` user and `root` group. + +#### Change owners and groups + +We can change the owner and group with the **chown** command. + +```plaintext +chown [-R] [OWNER][:GROUP] FILE + +# only change owner +chown OWNER FILE + +# only change group +chown :GROUP FILE + +# change both, but not the same +chown OWNER:GROUP FILE + +# change both to the same +chown OWNER: FILE +``` + +- 'R': Recursive (will also change owner for all subfiles and directories) + +```console +claw0ry@lnx:~$ cd /home/cloud_user +claw0ry@lnx:~$ touch file.txt +claw0ry@lnx:~$ ls -l file.txt +-rw-r--r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt +claw0ry@lnx:~$ chown another_user:another_user file.txt +claw0ry@lnx:~$ ls -l file.txt +-rw-r--r-- 1 another_user another_user 0 Jul 9 08:37 file.txt +claw0ry@lnx:~$ chown cloud_user file.txt +-rw-r--r-- 1 cloud_user another_user 0 Jul 9 08:37 file.txt +claw0ry@lnx:~$ chown :cloud_user file.txt +-rw-r--r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt +``` + +#### Change permissions + +To set permissions we use the **chmod** command. + +There are two modes to set permissions with; symbolic and octal. + +##### Octal mode + +```plaintext +chmod OCTAL FILE +``` + +- read(r): 4 +- write(w): 2 +- execute(x): 1 + +To calculate the permissions bits we just need to add the permissions together. So for read and write access it will be `4+2=6`. For all permissions it will be `4+2+1=7`. + +Let's say we have `file.txt` and we want the owner to have full permissions, the group to have read and write and others to have none. This would calculate to `760`. We can set these permissions with the following command. + +```console +claw0ry@lnx:~$ chmod 760 file.txt +claw0ry@lnx:~$ ls -l file.txt +-rwxrw---- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt +``` + +These are the most common permissions used: + +- 777 (everyone has full permissions) +- 644 (owner has read+write, and everyone else has read) +- 750 (owner has full permissions, groups has read+execute, and everyone else has none) +- 600 (owner is the only user that has access) + +##### symbolic mode + +```plaintext +chmod MODE FILE +``` + +- '+': add permission +- '-': remove permission +- '=': set permisions +- 'u': owner (user) +- 'g': group +- 'o': others + +```console +# add read permissions for the owner +chmod u+r file.txt + +# remove execute for owner +chmod u-x file.txt + +# set owners permissions to read, write, execute +chmod u=rwx file.txt + +``` + +We can combine permissions with `,`. + +```console +claw0ry@lnx:~$ ls -l file.txt +-rwxrw---- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt +claw0ry@lnx:~$ chmod u=rwx,g=rw,o=r file.txt +claw0ry@lnx:~$ ls -l file.txt +-rwxrw-r-- 1 cloud_user cloud_user 0 Jul 9 08:37 file.txt +``` + +## Input/output redirection + +In Linux shells there is a concept of three streams. + +- The standard input (stdin), which takes the users input. +- The standard out (stdout), which is the output of a command. It is usually displayed in your terminal. +- The standard error (stderr), which is the error messages from a command. This is also usually displayed in your terminal alongside stdout. + +Write errors to file instead of console. + +```console +# no redirection +claw0ry@lnx:~$ ls -l nonexisting +ls: nonexisting: No such file or directory + +# with redirection of stderr +claw0ry@lnx:~$ ls -l nonexisting 2> ls_error.txt +claw0ry@lnx:~$ cat ls_error.txt +ls: nonexisting: No such file or directory +``` + +Redirect **stdout** to a file and only show errors in the console. + +```console +# these two are equivalent +find /root 1> dirs_i_can_read.txt +find /root > dirs_i_can_read.txt +``` + +Redirect both **stdout** and **stderr** to the same destination. + +```console +find /root 2>&1 root_dirs.txt +``` + +This tells Linux to redirect **stderr** to **stdout** and then **stdout** to a file. + + +## Reboot and PowerOff + +```plaintext +shutdown [OPTIONS] [TIME] [WALL...] +``` + +```console +# poweroff +poweroff +shutdown -P +shutdown -P 20:00 +shutdown -P +5 +shutdown -P +5 + +# reboot +reboot +shutdown -r [TIME] +shutdown -r 20:00 +shutdown -r +5 'Server will be rebooted for maintenance!' +``` + +## Manage users and groups + +```console +# Add a new user +adduser LOGIN + +# add user to group +gpasswd -a USER GROUP + +# give sudo access +cat /etc/sudoers +gpasswd -a USER sudo +``` + +## Disk space + +List filesystem space usage. + +```console +claw0ry@lnx:~$ df -h +Filesystem Size Used Avail Use% Mounted on +udev 467M 0 467M 0% /dev +tmpfs 96M 484K 95M 1% /run +/dev/nvme0n1p1 20G 2.1G 17G 11% / +tmpfs 477M 0 477M 0% /dev/shm +tmpfs 5.0M 0 5.0M 0% /run/lock +/dev/nvme0n1p15 124M 12M 113M 10% /boot/efi +tmpfs 96M 0 96M 0% /run/user/1001 +tmpfs 96M 0 96M 0% /run/user/1002 +``` + +List directory space usage. + +```console +claw0ry@lnx:~$ du -hs /home/cloud_user +36K /home/cloud_user +claw0ry@lnx:~$ sudo du -hs /var/log +55M /var/log +``` + +If we omit the `-s` we will also see subdirectories. + +```console +claw0ry@lnx:~$ sudo du -h /var/log +4.0K /var/log/runit/ssh +8.0K /var/log/runit +52K /var/log/apt +4.0K /var/log/private +53M /var/log/journal/ec228e8f22cbefcdced18ace3b891949 +53M /var/log/journal +16K /var/log/unattended-upgrades +32K /var/log/amazon/ssm/audits +852K /var/log/amazon/ssm +856K /var/log/amazon +55M /var/log +``` + +You can also sort the result to find what takes the most space. Since we are using the `-h` flag on **du** to get human readable sizes, we also provide the same flag to **sort** so that I will sort correctly. + +```console +claw0ry@lnx:~$ sudo du -h /var/log | sort -h +4.0K /var/log/chrony +4.0K /var/log/private +4.0K /var/log/runit/ssh +8.0K /var/log/runit +36K /var/log/apt +136K /var/log/nginx +440K /var/log/sysstat +25M /var/log +25M /var/log/journal +25M /var/log/journal/b8d53a40a48e4a9baaf67b1d19735980 +``` + +To get the size of a file(s), you can use the **ls** command with `-lh` to get the size in human readable format. + +Another alternative is **ncdu** which is an interactive version of `du` built with ncurses. It does not come pre-installed. + +```console +sudo apt install ncdu + +# replace /home with whatevery starting path you want +ncdu /home +``` + +Now you can navigate and see the size of files and directories interactivly. + +## Processes + +Dashboard overview of your processes, CPU and memory usage. +```console +htop +``` + +Snapshot of the current process state. +```console +ps aux +``` + +Kill a process. + +```console +pkill -9 PID +``` + +## Schedule tasks + +In Linux we use **cron** with a `crontab` file to scedule runs of commands or scripts. + +The crontab template looks like this. + +```plaintext +# +----------- minute (0 - 59) +# | +--------- hour (0 - 23) +# | | +------- day of month (1 - 31) +# | | | +----- month (1 - 12) +# | | | | +--- day of week (0 - 6) (starts at sunday) +# | | | | | +# * * * * * COMMAND +# +# NOTES +# The 'month' and 'day of week' can be represented by either a number, name or shortname. +# e.g 1, January, Jan +# e.g 1, Monday, Mon +# +# Command can be either a script or a command. You can seperate them with ';' to run +# multiple commands. +# +# Visit https://crontab.guru for a visual representation of cron schedule +``` + +To edit your crontab you use the **crontab** command. + +```console +crontab -e +``` + +You can also specify the user to edit crontab for if you want it to run as another user. + +```console +crontab -u USER -e +``` + +> NOTE: When adding commands to your crontab, make sure you use full paths for both commands and scripts. + +You can check the cron logs using **journalctl**. + +```console +journalctl --unit cron +``` + +## Manage and update packages + +### dpkg + +```console +# install .deb package from file +dpkg -i FILE.deb + +# list installed packages +dpkg -l + +# remove package +dpkg -r NAME + +# search installed packages +dpkg -S PATTERN +``` + +NOTE: When intalling packages with `dpkg -i` it does not perform any dependency checks. If a dependency is missing it will fail. + +### apt + +```console +# update package list +apt update + +# upgrade installed packages +apt upgrade + +# install package(s) +apt install NAME + +# remove package +apt remove NAME + +# remove package and config +apt purge NAME + +# remove unwanted packages +apt autoremove + +# search packages +apt search PATTERN + +# show package details +apt show NAME + +# list installed packages +apt list --installed + +# list packages that can be upgraded +apt list --upgradeable +``` |