1 - Intro to DevOps

Today I'm excited to start a DevOps bootcamp. I'm targeting backend development (Python for now), and cloud engineering (at least AWS, Docker, K8S, Terraform...).

For now I'm not planning to do too much frontend dev, that's why I'll stick with Google Sites for now!

The bootcamp is from TechWorld with Nana.

Nov 22. 2023

2 - Operating Systems & Linux Basics

Hypervisor, Type 1 & Type 2 virtual machines..., a great introduction to virtualization. I've downloaded Oracle VirtualBox, created a Linux VM, and installed Ubuntu 22.04.3. I also installed the VirtualBox Extension Pack and enabled bidirectional copy/paste between host and guest.

Users & Groups

Linux has 3 types of users:

The root user
User accounts
Service accounts (for server distributions)

List all users:

cat /etc/passwd

The format of each line is:

username:password:UID:GID:GECOS:HOMEDIR:SHELL

List all groups:

cat /etc/group

Changing password:

passwd <user>

Switch user

su - <user>

Switch to root user

su -

Adding a group

sudo groupadd <group>

Adding a user

sudo useradd <user>

sudo useradd -G <group> <user>

Adding user to group:

sudo usermod -g <group> <user>

Adding several groups for a user

overwrite existing ones

sudo usermod -G <group1>,<group2> <user>

append to existing ones

sudo usermod -aG <group1>,<group2> <user>

Deleting a group

sudo delgroup <group>

List all groups for a user

groups <user>

Remove user from group

sudo gpasswd -d <user> <group>

Ownership & Permissions

Every file (and folders since folders are files in Linux) is owned by

an owner user (usually the creator of the file)
an owner group (usually the primary group of the owner user)

Changing ownership:

sudo chown <user>:<group> <file>

sudo chown <user> <file>

sudo chgrp <group> <file>

drwxrwxrwx

d for directory or - for files

rwx permissions for owner user

rwx permissions for users of owner group

rwx permissions for all other users

Different ways to add, remove, and set permissions:

sudo chmod u+w script.sh

sudo chmod g-x script.sh

sudo chmod o+r script.sh

sudo chmod g=r-x script.sh

sudo chmod 740 script.sh

That last one is simply the decimal value of the 3-digit binary value rwx:

--- 0

--x 1

-w- 2

-wx 3

r-- 4

r-x 5

rw- 6

rwx 7

Bash scripting

sh: Bourne shell: /bin/sh.
bash: Bourne Again shell: /bin/bash. Improved version of sh and the new default in most distributions.

All shell scripts have a .sh extension. The shebang line at the top of the file tells the OS which intepreter to use:

#!/bin/bash or #!/bin/sh

Variables

declaration:

var = "whatever"

storing the output of a command in a variable:

var = $(ls <directory>)

using the variable:

echo "the value is $var"

Conditional

if [ -d "dir" ] (checks if dir is a directory)

then

...

elif [...]

then

...

else

...

Testing numeric variables:

if [ "$var" -eq 10 ] (other operators: -lt -le -gt -ge -ne)

Testing strings:

if [ "$var" = "test" ] (= for Posix, == for Bash)

Brackets:

if [...] for Posix, if [[...]] for Bash

Arguments

var = $1 : Getting the first argument

var = $* : Getting all arguments

var = $# : Getting the number of arguments

User input

read -p <prompt> <var>

Loops

for <var> in <list of values, for example $*>

echo $var

done

while true

if [...]

then

break (exits the loop)

...

done

Addition vs. concatenation

var = $var1 + $var2 : concatenation
var = $(($var1 + $var2)): addition

Functions

function myfunction () {

var = $1 (to get first argument passed to function)

...

return $var (optional, to return a value)

}

When calling the function:

myfunction arg1

result = $! (stores what the function returned inside result)

Environment variables

All uppercase: USER, HOME,...

To list all environment variables:

printenv

To see only env. variable VAR:

printenv VAR

echo $VAR

We can create our own, useful for authentification.

For use only in current terminal session:

export VAR = myvalue

Persistent but only for current user, add export function in ~/.bashrc

source .bashrc to apply changes

System-wide:

/etc/environment

Adding a directory to PATH but only for current user:

PATH = $PATH:/home/user/directory inside .bashrc

Networking

LAN - Router - WAN: the router IP address is the Gateway

Subnet: IP address start + Subnet mask

For example: 192.168.0.0 + 255.255.255.0 -> the first 24 bits are fixed, also noted 192.168.0.0/24.

IP addresses of LAN are not visible to the outside of the subnet. Instead, the router pulic IP address is sent.

NAT: Network Address Translation: adds security + allows reuse of IP addresses

Firewall: which IP addresses are accessible

Ports are like doors in a building.

Typical ports: 80 (web servers), 3306 (mySQL), 5432 (PostgreSQL)

1 application per port.

DNS - Domain Name Service

Root domain .

TLD (Top-Level Do.) .edu, .com, .org...

Geograpic domains .fr, .us, .de...

Other domains .biz, ...

Regulated by ICANN: Internet Corporation for Assigned Names and Numbers

Fully qualified URL: 'course.ebersoft.com.'

DNS entries are cached for efficiency

SSH: Secure Shell

Secure way to access remote servers.

2 types of Authentification:

Username + password
SSH key pair (more secure)

Key pair: private key + public key

Services can also have their own username / key pair, not just people.

Typical port for SSH is 22.

Powerful -> should be restricted to specific IP addresses!

Quick List of Essential Linux commands

File&Directory

pwd

mkdir

touch

file

zip, tar

nano, vi, vim

File Manipulation

cat

grep

sed

head, tail

awk

sort

cut

diff

tee

locate

find

User&Permission

sudo

chmod

chown

useradd

userdel

System

top, htop

uname

hostname

time

systemctl

watch

jobs

kill

shutdown

Networking

traceroute

nslookup

dig

ping

wget

curl

scp

rsync

ifconfig

netstat

Miscellaneous

history

cal

man

echo

alias, unalias

Git

After dedicating December 2023 to Linux, I am starting 2024 with Git. As a LabVIEW developer, I had always stuck with Subversion since we dealt mostly with large binary files. I am excited to tame the great features Git has to offer.

Windows Subsystem for Linux

Since I want to use Git command line, I want to stay in a Linux terminal so I can keep working with Linux commands.

Until now I was using the VirtualBox virtual machine, but it does add extra steps to start and stop the VM, and I wanted the same files to be visible from both OS. So I was excited to find and activate the WSL.

Visual Studio Code

VS Code is my tool of choice for programming and eventually I might perform Git operations directly from its integrated environment. For now I will use the Git command line until I am familiar with it enough.

I also installed the WSL extension which allows VS Code to access files stored under the Linux files system.

Linux file system

Windows and Linux still have their own file system. To access the Windows files from Linux, the /mnt/ syntax is used. For the other way around, the screenshot above shows how straightforward it is!

For performance reasons, it is not recommended to access files of an OS from the other. So I will work mostly in the Linux file system.

Commands

git init: initializes a local repo from existing local code (creates .git folder)

git add <files>: adds files to staging area

git add . : stages all files

git commit: commits all changes in the staging area to the local repo

git commit -a: to also commit changes not yet added to staging area

git commit -m "message": to add the commit message

git stash: temporarily hides changes that we are not ready to commit yet

git stach pop: restores the hidden changes

git log: displays the commit history of the current branch

git push [remote][branch]: Updates remote repo with recent commits from local copy

git pull: Updates local copy with recent commits from remote repo

git reset: discards a commit. HEAD~1 for last commit.

By default (--soft), changes are still there. --hard to also delete the changes. If the commit was already pushed to the remote repo, it can also be discarded with git push --force after the local reset. This should only be done on branches where we are the single developer since other developers like bugfix and feature branches. For master branch, use git revert.

git revert <hash>: creates a new commit that reverts the changes made by a previous commit, without removing that commit from the log.

git rebase: takes the changes that were committed on one branch and replay them on a different branch.

git checkout: switches the local copy to a different branch or older commit of the current branch.

git branch: lists 'local' branches, inside .git/refs/heads

git branch -r: lists 'remote-tracking' branches, inside .git/refs/remotes

git branch "name": creates the new branch "name"

git branch -d <branch>: deletes the local branch

git merge <source>: merges commits from source branch into current branch inside local repo (then git push to remote repo)

git clone <url>: clones a repository into a new directory

git remote add <name> <url>: adds a remote <name> for the repository at <url>

git fetch <remote>: updates remote-tracking branches under .git/refs/remotes/<repo>

Page updated

Report abuse