LPIC - 102 Manage Software

Objective

# Rpm & Yum

Pkg general

Foundation that all programs rely on: Linux kernel
Package system maintain a database of installed files (with exact files & file locations)
Different ways of tracking pkgs & files:
- Application files (track each individual file)
- Library dependencies
- Application version

1 - RPM

RedHat pkg manager

Naming of the package:

1	Package name + Version number + Build number ()+ CPU architecture

A pkg designed for one distro, may have unmet dependencies in another distro

✗: upgrade dependencies may break other pkgs

✗ : (Servers) distro-specific scripts / config files

✗ / ✓: rebuild target pkg from source (not guaranteed, if pkg naming is different for dependencies)

Commands

# Install 
rpm -Uvh samba.rpm

# Verify pkg (-qi / -V)
rpm -qi samba.rpm

# display all installed pkgs
rpm -qa

# More options
-i    install (only if not present)    <==
-U    install new & upgrade existing (without manually uninstall old one)
-e    uninstall (erase)

-F    upgrade for existing pkgs only
-b    build binary pkg
-vh   show progress

-qc   configuration files of installed pkg
-qR   deps of installed pkg
-qRp  deps of uninstlled** pkg

-ivh  install new 
-Uvh  install or upgrade

--nodeps    no dependency check
--test      dry run
--rebuild   build binary pkg (or use: rpmbuild)

Extract data

rpm files = modified cpio archives

Usage

# install this
sudo apt install rpm2cpio

# convert to cpio
rpm2cpio samba.src.rpm > samba.src.cpio

# extract data
cpio -i --make-directories < samba.src.cpio

# Use pipe to convert & extract to CURRENT dir (no intermediate cpio file)
rpm2cpio samba.src.rpm | cpio -i --make-directories

RPM config Files

Main rpm config file: /usr/lib/rpm/rpmc

Mostly related to CPU optimization

Optimize code from your CPU model, by passing appropriate compiler options
Make global changes: edit file /etc/rpmrc

Build source rpm into binary rpm:

1
2
3

# athlon: AMD athlon processor
# rpm pass (-02 -g -march=-686) to compiler, whenever building athlon
optflags: athlon -02 -g -marchi686

Rpm can determine system architecture,

but in /etc/rpmrc, the buildarchtranslate lines cause rpmbuild to use one set of optimizations for a CPU family . e.g. For x86 systems
1
2
3
4
buildarchtranslate: athlon: i386
buildarchtranslate: i686:   i386
buildarchtranslate: i386:   i386
...
By specifying below, will have a slight performance boost, but reduced portability
1
buildarchtranslate: athlon: athlon

2 - Yum

RPM meta-pkg, Yum (Yellow Dog Updater - Modified)

Pro: Group pkgs together for distribution

Commands

yum install

# upgrade pkg (these 2 are nearly identical)
yum update
yum upgrade

# check if updates are available
yum check-update

# also remove dependencies
yum remove

# display pkg info
yum list
yum info

# search pkg
yum search

# clean cache di
yum clean

# enter shell mode
yum shell

# show the file belongs to what pkg
yum provides

# display pkg matching specified dependency
yum resolvedep

# display deps list
yum deplist

Obtain pkg: yumdownloader (download to the current dir)

GUI tools:

yum install yumex
yum install kyum

Yum config files

Main file: /etc/yum.conf
Additional conf: /etc/yum.repos.d/

Yum use files in /etc/yum.repos.d to locate repos.
So adding new repos: add files to this directory manually, or installing an rpm (just like apt)

3 - ZYpp

openSUSE’s own pkg management

Commands

# show all available plg update
zypper list-updates

# show repo info
zypper lr

# install
zypper in xx

# uninstall (remove)
zypper re xx

# search pkg
zypper se xx

# Deb pkg

apt-cache

apt-get

dpkg

Text-based & GUI tools ( dselect, aptitue, synaptic )

1 - `apt-cache`

Provide info about Debian pkg database (package cache)

# show pkg info
apt-cache showpkg xx

# show stats
apt-cache stats

# show unmet deps
apt-cache unmet

# show deps
apt-cache depends xx

# show pkgs that depend on xx
apt-cache rdepends xx

# find all pkgs that begin with sa
apt-cache pkgnames sa

2 - `apt-get`

Config file: /etc/apt/sources.list

Controls apt & dselect options: /etc/apt/apt.conf

Commands

# after dselect
apt-get dselect-upgrade

# smart conflict resolution
apt-get dist-upgrade

# get newest available source pkg (in sources.list)
apt-get source xx

# check for broken source pkg
apt-get check

# housekeeping the list
apt-get clean
apt-get autoclean 	 # only clean pkgs that cannot be downloaded

# get copies of dpkg files
apt-get download xx

Options

# fix broken
apt-get -f install / remove xx

# dry run
apt-get -s install xx

# compile source pkg (in sources.list)
apt-get -b source xx

3 - `dpkg`

Format: dpkg [option] [action] [pkg]

Options

# ignore deps info
dpkg --ignore-depends=[pkg] -i / -r xx

# dry run
dpkg --no-act -i / -r xx

# install all that match wildcard name
dpkg --recursive -i xx

Actions

Basic

# install
dpkg -i xx

# remove pkg only (Do this before UPGRADE pkg)  <==
dpkg -r xx

# remove pkg & config file
dpkg -P / --purge xx

# search for partially installed pkg
dpkg -C / --audit

Advanced

# show all CURRENTLY installed pkg   <==
dpkg --get-selections | sort -k 2 -r

# list all installed pkgs that match name pattern (here: jq)
dpkg -l jq

# show info about installed pkg
dpkg -p xx

# show info about **uninstalled pkg
dpkg -I xx

# check status & verify deps
dpkg -s / --status

Other

# reconfig installed pkg
dpkg --configure xx
dpkg-reconfigure xx

# show pkg 
sudo debconf-show cups

4 - Tools

dselect (high level pkg browser)
aptitude (combines interactive dselect & cmd-line options of apt-get)
- aptitude search xx
- aptitude install xx
synaptic (GUI tool)

Compare Deb pkg & other pkg formats

Deb source pkg are groups of files
- source tarball
- patch file (support only one patch file) | Rpm pkg can contain multiple patch files
- .dsc file (digital signature, for pkg verification)

Configure dpkg

Main dpkg config file: /etc/dpkg/dpkg.cfg

1
2
3

# alwasy dry run before install
# add this line in dpkg.cfg
no-act

dpkg rely on files in /var/lib/dpkg (list of available & installed pkgs)

This is the Debian installed file database

5 - Pkg deps & Conflicts

Convert between pkg formats (alien)

Convert between: rpm, deb, tarballs

alien requires both rpm & dpkg to be installed

# default is to .deb (--to-deb)
alien xx.rpm

# deb to rpm
alien --to-rpm xx.deb

# deb to tarball
alien --to-tgz xx.deb

Convert tarball to rpm

Convert a tarball: convert files in the dir structure of original tarball, use system’s root dir as base

# unpack
tar xvzf xx.tar.gz

# rename dir
# when create new tarball, will have file in correct locations such as /usr/bin, /usr/lib
mv xx-files usr

# pack
tar cvzf xx.tgz usr

# remove folder
rm -r usr

# convert to rpm
alien --ro-rpm xx.tgz

Deps problems

Missing libs / support programs
Incompatible libs / support programs
Duplicate files / features
Mismatched names

Workarounds

Force installation

# ignore failed deps
rpm -i xx --nodeps

# ignore other errors
rpm -i xx --force

# for dpkg
dpkg --ignore-depends=[pkg] -i xx
dpkg --force-depends -i xx
dpkg --force-conflicts -i xx

Upgrade / replace the depend-on pkg
Rebuild problematic pkg
1
rpmbuild --rebuild xx.src.rpm
Locate another version of the pkg

Startup Scripts Problem

This problem only affect servers
Old Linux use SysV startup scripts
local startup scripts:
- /etc/rc.d/rc.local
- /etc/rc.d/boot.local

# Manage shared libs

Libraries are chosen by programmers, NOT by users.

Cannot substitute one library for another.

1 - Overview

Linux Libraries

Multiple application that use same functions, can share same library files
Two types of libs:
- Static Libs (statically linked libs, functions copied into applications when it’s compiled)
- Shared Libs (dynamic libs, functions loaded into memory and bound to application, when program is launched)

Library Principles

Provide commonly used program fragments (use one copy multiple times)
- Linux widget sets: Qt, GTK+ (GIMP Tool Kit)
- Linux C library: glibc (GNU C library, /lib/x86_64-linux-gnu/libc.so.6)
Most programs use libs as shared libraries (dynamic libraries)
- Executable include references to shared lib files
May increase program load time, and other issues:
- programs must be able to locate shared libs
- If an important shared lib become inaccessible (due to overwritten), system might not even boot

Linux shared libs are similar to Windows DLLs (Dynamic Link Libraries, .dll)

Linux shared libs: .so (shared objects )
Linux static libs: .a (used by linkers for inclusion in programs)

2 - Locate lib files

System search for function’s lib file in a specific order

LD_LIBRARY_PATH env var
Program’s PATH env var
Directory /etc/ld.so.conf.d
File /etc/ld.so.conf
Directory /lib*/ and /usr/lib*/

Note:

/etc/ld.so.conf loads config file from directory /etc/ld.so.conf.d

/lib*/ directories are for libs needed by system utilities, that reside in /bin/ and /sbin/
/usr/lib*/ are for libs needed by additional software (e.g. MySQL DB utilities)

If another library is located in /etc/ld.so.conf, and listed above include,
Then system will search that lib directory, before files in /etc/ld.so.conf.d

1 2	cat /etc/ld.so.conf >> include /etc/ld.so.conf.d/*.conf

Configure library path (global config file / env var)

# original /etc/ld.so.conf file (load all file in ld.so.conf.d)
include /etc/ld.so.conf.d/*.conf
# then add lib path under this line

# after adding, update config
ldconfig

Change path temporarily

1 2	# add 2 dirs export LD_LIBRARY_PATH="/usr/loca/testlib:/opt/newlib"

Correcting problems

# If system cannot find the lib file, usually because lib isn't installed
# If lib file is available, need to add to LD_LIBRARY_PATH

# Another way: create a symbolic link (if there's difference in version)
sudo ln -s xxlib.so.5.2 xxlib.so.5
ldconfig

3 - Loading Dynamically

When program starts, dynamic linker is responsible for finding program’s needed lib functions.

After they’re located, dynamic linker will copy them into memory, and bind to programs.

# locate dynamic linker executable
locate ld-linux

# manually load a program & its libs
/usr/lib64/ld-linux-x86-64.so.2 /usr/bin/echo "Hello!"

# View program's needed libs (for trouble shooting) **
ldd /usr/bin/echo

Summary

Shared libs (dynamic libs)
Load library: search LD_LIBRARY_PATH first
Update lib cache: ldconfig
View libs required by a program: ldd

4 - Commands to manage libs

ldd : show program’s shared lib

ldconfig: update system’s cache & links

Show shared lib deps

ldd attempts to find true library (denote by =>)

When us ldd to track down problems,
ensure to check the needs of program’s all libs, and all libs used by first-tier libs

ldd /bin/ls

# display version info
ldd -v /bin/ls

Update lib cache

Library cache is a catalog of lib directories & all the various libs contained within.

Manage lib loading: ld.so, ld-linux.so

Programs above don’t read from ld.so.conf every time, but from cached list (/etc/ld.so.cache)

Need to update cache, every time when add / remove libs (ldconfig)

rpm & dpkg will run ldconfig automatically, after install / remove pkgs

# print current info
ldconfig -p

# verbose (for trouble shooting)
ldconfig -v

# Example (-v)
ldconfig -v 2> /dev/null | grep libmysqlclient

Develop New Libs

Add path to env var LD_LIBRARY_PATH

1	export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/kk/devs

After testing, move it to /usr/lib*/
Create lib config file in /etc/ld.so.conf.d/, which points to lib file’s location
Update lib cache (sudo ldconfig)

# Manage Process

uname

ps

top

jobs, nice, kill

1 - Overview

When Linux system first boots, a special process is started (init process).

This is the core of Linux system.

Understand Kernel

uname: print system info

# node name
uname -n

# kernel version
uname -vr

# machine (CPU code)
uname -m

# print all
uname -a

2 - Multiple Screens

In a text-based system, use terminal multiplexer to perform operations on multiple screens
(use pts terminal, pseudo-terminal)

screen

tmux

screen

Prefix shortcut: Ctrl + A (C - A)

Need to create window in each focus

# see screen list
screen ls

# detach
C-A   D

# reattach
screen -r [pid]

# ---------------- #

# vertical split
C-A   |

# horizontal split
C-A   Shift + S

# create window
C-A   C

# jump to next window
C-A   Tab

# kill all window
C-A   \

Example: Steps to create screen with 3 windows

# 1. vertical split
C-A   |

# 2. jump & create window
C-A   Tab
C-A   C

# 3. jump back
C-A   Tab

# 4. horizontal split
C-A   Shift + S

# 5. jump & create window
C-A   Tab
C-A   C

Stress test system’s CPU & memory (stress-ng)

1 2	stress-ng --class cpu -a 10 -b 5 -t 3m \ --matrix 0 -m 3 --vm-bytes 256m

tmux

screen 2.0 version

Prefix shortcut: Ctrl + B (C - B)

# see window list
tmux ls

# detach
C-B   D

# reattach
tmux attach-session -t [session number]

# -------------- #

# vertical split
C-B   %

# horizontal split
C-B   "

# next window
C-B   O

# help
C-B   ?

Example: Steps to create tmux with 3 windows

# 1. vertical split
C-B   %

# 2. jump back
C-B   O

# 3. horizontal split
C-B   "

3 - Process Status

ps auxwf | grep bash

ps supports 3 different styles of cmd-line options (historical reasons)

BSD style (xx)
Unix style (-xx)
GNU long options (--xx)

# all process by current shell (default)
ps

# all running process
ps -ef

# all process owned by a specific user
ps U kk

# all process associated with tty terminal
ps a

# extra info
ps u

# process tree view
ps f

# wide output (complete command, more than 80 char)
ps w

Select Process with ps

During trouble shooting: View only a selected subset of processes

# remove restriction of association with terminal
# usually used with "ps a"
ps x

# only show processes running command in cmd-list
ps -C [cmd-list]

# only show running processes
ps -r 

# only show processes associated with current tty terminal
ps -T

# ------------------- #

# only show processes, whose current real group is in the list
ps -G [list]

# only show processes, whose current effective group is in the list
ps -g [list]

Real & effective groups / users:

REAL (upper case): This is the user / group the account is associated with, when log into system
effective (lower case): user / group uses temporary alternative user / group ID (SUID / GUID)

If you want to see all process, use both effective & real options.

1	ps -u kk -U kk

Displayed info

PPID : parent process ID
TTY : teletype (identify a terminal)
C: processor utilization over process lifetime
STIME: system time (when process starts)
CPU Priority: default is 0. Negative values have higher priority (need to set as root, for values < 0)
RSS : resident set size (memory used by the program & its data)
CMD: commands to launch the process

Process States

ps -ef : Some CMD are shown in brackets.

These processes are currently swapped out from physical memory into virtual memory on the hard drive (Process state: sleeping).

Linux kernel puts a process to sleep mode, while process is waiting for an event. When event triggers, kernel sends process a signal.

Interruptible sleep: process receives signal immediately, and wake up
Uninterruptible sleep: process only wakes up based on external event (e.g. hardware becomes available)

Zombie process: Parent process does not recognize process’ termination signal

4 - top

top is the dynamic ps variant

GUI variant; kpm, gnome-system-monitor

Only to check system uptime:

Shows system load, but not CPU

uptime

Benchmark: familiar with purposes & normal habits of programs running on your system

top sorts processes based on %CPU by default

# refresh rate (default is 5 sec)
top -d 10

# by pid
top -p [pid]

# -------- In the top interface -------- 

# color mode
z

# highlight sort field
x

# toggle columns sort & set priority
f

# reverse sort
R

# sort by CPU
P

# sort by memory
M

# ---------------- 

# show full command
c

# show threads
H

# show idle processes
i

# show specific user process
u

# kill process
k

# write current setting to config file
W

# show cumulative CPU
S

load average: can equal to number of CPU cores, before competition for CPU time begins (1-min, 5-min. 15-min)

e.g. Quad-core CPU (can be as high as 4.0)

It’s common for 1-min load to be high (short burst of activities)

If the 15-min load average is high, system might be in trouble

CPU info

Process owner (user / system process)
Process state (running / idle / waiting)

Columns

PR: process priority
NI: nice value
VIRT: total virtual memory
RES: total physical memory
SHR: total memory shared with other processes
%CPU: share of CPU time
%MEM: share of available physical memory
TIME+: total CPU time used, since process starts
S: process status
- D: interruptible sleep
- I: idle
- R: running
- S: sleeping
- T: traced / stopped
- Z: zombie

Use watch for monitoring (refresh every 2 sec).

1	watch uptime

5 - jobs

Summarize process launched from your current shell
Provide job ID numbers
Ensure all programs have terminated, before logging out

No background process is running, that’s launched from the current shell

jobs command

-l	# check job PID
-p	# only list job PIDs

-n	# only list jobs that have status change
-r	# only list running jobs
-s	# only list stopped jobs

fg & bg

Move paused program to foreground:

# pause program
Ctrl + Z

# check job number
jobs

# check job PID
jobs -l

# move to foreground
fg [number]

# restart job in foreground (use job number)
fg 2

Send running program to background:

Each background process is tied to the session’s terminal 📌

# pause program
Ctrl + Z

# check job number
jobs

# check job PID
jobs -l

# move to background  (% + job number)
bg %[number]

# restart job in background (use job number)
bg 2

Note:

+ sign: last job added to bg stack
- sign: second-to-last job added to bg stack

All previously added jobs will show no sign.

Other commands:

Redirect nohup output messages - nohup.out

# run a job in background
gedit &

# bring bg jobs to fg  (% + job number)
fg %2

# stop bg job
kill %[number]

# continue running after log out
nohup [program] &

nohup will force the application to ignore any input from stdin.
It dis-associates process from terminal, and the process loses the output link to the monitor.

By default, stdout and stderr are redirected to ~/nohup.out

6 - Process priorities

nice

renice

nice

Range: [-20, 19] (nice default is 10)
Only root may launch program with negative priority value
Default for program run without nice is 0

# assign 12 to program, and pass to txt
nice -12  program  data.txt

# equivalent
nice -12
nice -n 12 
nice --adjustment=12

renice

Only root can modify other user’s process priority

Only root can decrease priority value

1 2	# assign 7 to pid 2390, for all process owned by kk renice 7 2390 -u kk

Summary: Normal user cannot set value < 0 in nice, cannot decrease value in renice

7 - Kill Process

Process Signals

Signals are also written as SIG + Name.

e.g. TERM -> SIGTERM

Number	Name	Description
1	`SIGHUP`	Hang Up
2	`SIGINT`	Ctrl + C - Interrupt
9	`SIGKILL`	`kill` - Force kill
15	`SIGTERM`	Soft kill
11	`SIGSEGV`	Segments violation
3	`SIGQUIT`	Stop running
20	`SIGTSTP`	Ctrl + Z - Stop (but does not terminate, program is still in memory)
18	`SIGCONT`	Restart a stopped process

Kill

1 SIGHUP : terminates interactive program, cause daemons to re-read config file

# view all numbered signals
kill -l

# kill process (default: 15 SIGTERM)
kill -s 15 [pid]

# server with this pid to reload its config file
kill -s 1 [pid]

Kill a process

Try TERM first
Then HUP / INT
Use KILL as last resort (might lead to corrupted files)

Killall: Variant of kill (kills process based on name, but not pid)

# kill all process with name vim
# -i: confirmation, before killing
killall -i vim

# check open files before killall
lsof | grep [pid]

Pkill: Variant of kill (use selection criteria)

Use pgrep to test out selection criteria prior to sending signals

# all process attached to terminal 2
pgrep -t tty2

# kill all process attached to terminal 2 (SIGTERM by default)
pkill -t tty2

# Practice

Source pkg & Binary pkg

Source pkg requires more time to compile (must be compiled prior to installation)
Both rpm & dpkg provide tools to create binary pkg (can be installed directly) from source code

Binary pkg creation is useful, if you’re running Linux on a peculiar CPU
(But cannot recompile for different CPU architecture)
Tarballs are preferred, when distribute for other platforms (universal)

More About Source pkg & Binary pkg

Source package include a tarball of the application’s source code, and instructions on building it
When you install the package, it builds and compiles everything on-site, then installs
[FAST] Binary packages have everything already built, and installing the package just takes everything out of it

Rpm & Dpkg

Databases for these two are separate. Hence these two are incompatible with one another
Pkg management system don’t share info, and databases don’t actively conflict
Dpkg often provides more extensive initial setup options
For problematic pkgs:
- rpm can rebuild from source (rpmbuild --rebuild)
- Debian source pkgs are rare. Less likely to rebuild dpkg from source

# Rpm & Yum

1 - RPM

2 - Yum

3 - ZYpp

# Deb pkg

1 - apt-cache

2 - apt-get

3 - dpkg

Options

Actions

4 - Tools

5 - Pkg deps & Conflicts

# Manage shared libs

1 - Overview

2 - Locate lib files

3 - Loading Dynamically

4 - Commands to manage libs

# Manage Process

1 - Overview

2 - Multiple Screens

screen

tmux

3 - Process Status

4 - top

5 - jobs

fg & bg

6 - Process priorities

nice

renice

7 - Kill Process

Process Signals

Kill

# Practice

1 - `apt-cache`

2 - `apt-get`

3 - `dpkg`