Difference between revisions of "Category:Linux troubleshooting"

Revision as of 04:13, 11 March 2007

This category will contain a collection of articles on troubleshooting Linux. It will be highly biased towards SuSE Linux (version 10.1 or later), as that is my primary OS. I am also using a 64-bit (x86_64) kernel, so my articles will also be biased towards these systems.

Note: Most of the following have also been tested on Mandriva Linux.

System information

% dmesg
% vmstat  # to quickly monitor CPU, memory, and I/O usage and decide which is the bottleneck

% ps -ef | egrep '^root ' | gawk '{print $2}'  # method 1
% pgrep -u root                                # method 2

% cat /proc/cpuinfo
% cat /proc/partitions
% cat /proc/meminfo
% cat /proc/sys/vm/swappiness  # number from 0 - 100; the higher the number the more the system will swap
% cat /proc/interrupts   # inspect your /proc/interrupts file for multiple devices having the same interrupt

% uname -a  # system architecture
% grep ^VERSION /etc/SuSE-release  # To see which SuSE Linux version you are using
% cat /etc/mandriva-release        # To see which Mandriva Linux version you are using
% pstree
% lsof | grep TCP                  # list open files
% lsof | grep ' root ' | awk '{print $NF}' | sort | uniq | wc -l  # list number of open files for a user

% getconf   # print system configuration variables
% getconfig # get configuration information for the Xorg server
% systool   # view system device information by bus, class, and topology
% bind -P   # print keyboard bindings

% cat /proc/scsi/scsi
WDC WD2000JD-22H Rev: 08.0
SATA-I, 200 GB, 150 MB/s, 8 MB Cache, 7200 RPM

Managing modules / devices / libraries / etc

% lspci
% lsmod
% depmod
% modprobe  # tail /var/log/messages (to check success / failure)
% rmmod

% ldd /usr/bin/python   # print shared library dependencies
       linux-gate.so.1 =>  (0xffffe000)
       libpython2.5.so.1.0 => /usr/lib/libpython2.5.so.1.0 (0xb7e2e000)
       libpthread.so.0 => /lib/libpthread.so.0 (0xb7e16000)
       libdl.so.2 => /lib/libdl.so.2 (0xb7e12000)
       libutil.so.1 => /lib/libutil.so.1 (0xb7e0e000)
       libm.so.6 => /lib/libm.so.6 (0xb7de7000)
       libc.so.6 => /lib/libc.so.6 (0xb7cb9000)
       /lib/ld-linux.so.2 (0xb7f71000)

Default runlevel

It is a good idea to make the defaul runlevel for your machine "3" (i.e. full multiuser mode without X11). This will prevent your system from hanging if something is wrong with your X11 settings (the graphics).

To change the default runlevel, edit your /etc/inittab file and change the line that reads

id:5:initdefault:

to

id:3:initdefault:

Now, everytime you turn on your machine (or reboot it), you will be taken to a CLI. Login as a user (not root!) and enter the following:

% startx

see also: wikipedia:init

Linux networking

see: SuSE wireless card configuration

% hostname -i           # show current IP address
% hostname -d           # show current domain name
% domainname            # show full domain name
% cat /etc/resolv.conf  # show nameserver(s); one per line

% /etc/init.d/network restart
% route add 20.0.xxx.xxx gateway foo
% /etc/rc.local
% /etc/sysconfig/network-scripts
% /sbin/ifconfig

% netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
20.0.xxx.xx     20.0.xx.xx      255.255.255.0   UGH       0 0          0 eth0

% cat /proc/net/arp
IP address       HW type     Flags       HW address            Mask     Device
192.168.xxx.xxx  0x1         0x2         00:00:00:00:00:00     *        eth0
192.168.xxx.xxx  0x1         0x2         00:00:00:00:00:00     *        eth0

External resources

• Linux Networking-HOWTO
• Wireless LANs with SuSE Linux
• Quick HOWTO : Ch14 : Linux Firewalls Using iptables
• Ndiswrapper Wiki
• the rt2x00 Open Source Project

Display (Monitor / Graphics Card)

% cat /etc/X11/xorg.conf
% xdpyinfo | grep dimen   # for screen dimensions

If you are having trouble (in SuSE) getting your monitor to display anything (either from an initial boot or from adding a new monitor), try the following:

• Reboot in Failsafe mode
• Login as root
• At the command prompt type: sax2 -m 0=vesa
• Configure video settings and test them (it is important to test your settings first!)
• Reboot in normal mode.

Backing up the MBR

It is easy to backup and restore the master boot record (MBR) in Linux. However, caution must be exorcised when performing any of the following commands.

• to backup

dd if=/dev/xxx of=mbr.backup bs=512 count=1

• to restore

dd if=mbr.backup of=/dev/xxx bs=512 count=1

where xxx is the device, which can be hda, sda, or any other.

Sound problems

Note, my sound card specs: Intel Corporation 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller

• Un-mute PCM sound
• Check the following:

% lsmod | grep snd
% cat /etc/modprobe.conf
% vi /etc/modprobe.d/sound

(change "snd-intel8x0" to "snd_intel8x0")

% ./sbin/lspci

(list sound specs / Multimedia audio controller)

% modprobe snd-via82xx

Hacked?

• Check for failed logins in: /var/log/messages
• Regularly monitor:

  • zcat /var/log/auth.log.*.gz | grep refused

  • grep -i failed /var/log/auth.log

  • last
  • w and/or who
  • uptime
• Verify that /etc/passwd hasn't changed.
• Check fuser for ports.
• Search for portscans in server report.
• Check for weird processing hogging the CPU.
• Use fail2ban, DenyHosts, etc.

Notes

• pstree — display a tree of processes
• lsmod — program to show the status of modules in the Linux Kernel
• modprobe — program to add and remove modules from the Linux Kernel
• netstat — Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships
• lspci — list all PCI devices
• more /usr/share/pci.ids — A list of all known PCI ID's (vendors, devices, classes, and subclasses). Maintained at The Linux PCI ID Repository, use the update-pciids utility to download the most recent version.

External links

• SuSE wiki
• How to get specific programs to run under Dapper Drake 64-bit edition
• the (R)ecovery (I)s (P)ossible Linux rescue system
• Restrict Linux users to their home directories only
• Howto: Formatting Linux Filesystem
• Quick HOWTO : Ch29 : Remote Disk Access with NFS
• DebianHelp
• Unix Host Security
• 28 Steps on how to harden your linux server