Linux Troubleshooting Technique

Linux Troubleshooting Technique


--USE Method (by Brendan Gregg)

For every resource, check utilization, saturation, and errors.
resource: all physical server functional components (CPUs, disks, busses, ...)

Utilization: the average time that the resource was busy servicing work
Saturation: the degree to which the resource has extra work which it can't service, often queued
Errors: the count of error events


--Quick Check

Check system resource usage and running processes by running the ten commands
# uptime
# dmesg | tail
# vmstat 1
# mpstat -P ALL 1
# pidstat 1
# iostat -xz 1
# free -m
# sar -n DEV 1
# sar -n TCP,ETCP 1
# top


--Logs Check

Common logs location
/var/log/auth.log : Authentication logs
/var/log/secure or /var/log/auth.log : Authentication log
/var/log/messages : General message and system related stuff
/var/log/kern.log : Kernel logs
/var/log/cron.log : Crond logs (cron job)
/var/log/maillog : Mail server logs
/var/log/qmail/ : Qmail log directory (more files inside this directory)
/var/log/httpd/ : Apache access and error logs directory
/var/log/lighttpd/ : Lighttpd access and error logs directory
/var/log/boot.log : System boot log
/var/log/mysqld.log : MySQL database server log file
/var/log/utmp or /var/log/wtmp : Login records file
/var/log/yum.log : Yum command log file.

# less /var/log/messages
# more /var/log/messages

Continuous log fail observation
#tail -f /var/log/messages

Search  for a term in a file
#grep -i error /var/log/messages


--Services check

Show all installed services
#systemctl list-unit-files

#service --status-all
#service --status-all | more
#service --status-all | less
#service --status-all | grep ntpd


--System info check

Show version
#lsb_release -a

# lshw
# lscpu
# lsblk
# lspci
# fdisk -l

Show kernel  modules
#lsmod

List containing your BIOS version and supported modes, your motherboard manufacturer and model number, your CPU technical details and your RAM characteristics
#sudo dmidecode -q