Wednesday, December 4, 2013

RHCSA/RHCE Preparation in brief

Red Hat Enterprise Linux v.6


The following notes were prepared during Red Hat Rapid Track RHCE training in August 2012 (RHEL 6.2), and cover both the RHCSA and RHCE exams. 
These notes were prepared before sitting the exam, and do not contain any exam material, in accordance with the exam legal agreement.

=== High probability of being on the exam ===

* Booting into/rescuing an unbootable system
* Set up a yum.repos.d repository on a web server

=== Low probability of being on the exam ===

* NTP server (not client)

=== Procedural ===

* Doesn't matter how you do the exam (CLI or GUI), as long as it works
* Use server and host numbers corresponding to desktop number
** desktop9.example.com
** server9.example.com
** host9.example.com
* Login: student:student
* su to root:redhat
* Use Virtual Machine Manager to start and log in to VM

=== Tips ===

* Run makewhatis before anything else to build the apropos database so man -k works
** This is particularly useful for finding all daemon-specific SELinux documentation:
 man -k _selinux
* Reboot the system, then re-test everything

=== Unit  1: Software management ===

* Register with RHN
* Using third-party repos
* Using yum
* Handling non-repository third-party software

Additional software channels

* Supplementary: closed source, e.g. Flash and Java
* Optional: open source but unsupported
** generally supported in another RHEL variant

* RHN can use Redhat's own servers, or a Redhat Satellite Server
* Requires Redhat subscription key

Registering

 rhn_register

Adding a third-party repository

* Add a new /etc/yum.repos.d/*.repo file
* Import public keys with rpm --import

yum

 yum list [regex] name search
 yum search [regex] description search
 yum provides [regex] search for packages that provide a given file
 yum info package like apt-cache show package
 yum install package
 yum install package.rpm remember the --nogpgcheck parameter
 yum remove package
 yum update like apt-get update && apt-get upgrade
 yum update package like apt-get update && apt-get install package
* Logs in /var/log/yum.log
* yum does not automatically remove unneeded packages
** E.g. yum install gnuplot installs 4, yum remove gnuplot removes 1
** But packages depending on a removed package will be removed automatically
* First update of a package may prompt about importing a local GPG key
** This is OK
* As with apt-get, yum install package and yum update package are equivalent

rpm

 rpm -qa list all installed packages
 rpm -q package list package
 rpm -ql package list files in package
 rpm -qi package show info about package
 rpm -q --scripts package show scripts in package
 rpm -qc package show config files in package
 rpm -qd package show doco files in package

=== Unit  2: Network management ===

* Configuration files
* Troubleshooting toolkit
* Interface configuration
** IP aliases
** Bonding
* Tuning kernel parameters

Checking configuration

* ifconfig, ip addr
* netstat -r, ip route
* hostname
* dig
* cat /etc/resolv.conf
* cat /etc/hosts
* Or use the GUI, right-click on the Network Manager icon
** May wish to disable Network Manager on servers that should have static addresses
** Definitely do so if configuring aliases (and presumably bonding)

Changing configuration

* /etc/sysconfig/network
* /etc/sysconfig/network-scripts/ifcfg-eth0
** Do not configure DNS directly in /etc/resolv.conf, as this will be overwritten by Network Manager
* /etc/sysconfig/network-scripts/route-eth0 for static routes
* ifup, ifdown
* Or again, use the GUI, right-click on the Network Manager icon

ifcfg-eth0 and similar interface config files

* NM_CONTROLLED: set to "no" to prevent Network Manager from changing the config

Troubleshooting

* ping, traceroute, mtr
* host, dig, nslookup

IP aliases

 ip addr add 1.1.1.1/24 dev eth0 label eth0:0

* For a permanent alias, create a /etc/sysconfig/network-scripts/ifcfg-eth0:0:

 DEVICE=eth0:0 # optional -- if not present, filename is used
 IPADDR=10.1.1.250
 PREFIX=24
 ONPARENT=yes

* Turn off Network Manager before configuring aliases

Bonding

* Modes: 0, 1, 3, 4:
** Mode 0: round robin (requires configuration on switch)
** Mode 1: active/backup
** Mode 3: broadcast (requires configuration on switch)
** Mode 4: 802.3ad (LACP 802.3ad, often used in Cisco environments)
** Modes 2, 5, and 6 also supported
* For a permanent bond, create a /etc/sysconfig/network-scripts/ifcfg-bond0:

 DEVICE=bond0
 IPADDR=10.10.10.10
 NETMASK=255.255.255.0
...

 Then configure each of the ifcfg-eth? files similar to the following

 DEVICE=eth0
 BOOTPROTO=none
 ONBOOT=yes
 MASTER=bond0
 SLAVE=yes
 USERCTL=no

Tuning kernel parameters

* Install documentation:
 yum install kernel-doc
* Query parameters:
 sysctl -a | grep keyword
* Set parameter:
 sysctl -w parameter=value
* To set persistently, add parameter = value to /etc/sysctl.conf
* Run the following to re-read the /etc/sysctl.conf file:
 sysctl -p

=== Unit  3: Storage management ===

* Partitions and file systems
* File system encryption
* Swap space
* iSCSI

Adding partitions

* fdisk, parted
* partx -a /dev/sda to get the kernel to rescan a partition table

Mounting

* Add the filesystem to /etc/fstab using the UUID obtained via blkid:
 UUID="1aEE6y..." /mountpoint ext4 defaults 1 2

Encryption

 cryptsetup luksFormat /dev/sda2
 cryptsetup luksOpen /dev/sda2 name
 mkfs.ext4 /dev/mapper/name
 mkdir /secret && mount /dev/mapper/name /secret
 ...
 umount /secret && cryptsetup luksClose name

Persistently mounting encrypted partitions

* Make an entry in /etc/crypttab:
 secret /dev/sda2 /root/secret.key
* Make an entry in /etc/fstab:
 /dev/mapper/secret /secret ext4 defaults 1 2
* Create the /root/secret.key file and make it readable only by root
* Make LUKS aware of the key
 cryptsetup luksAddKey /dev/sda2 /root/secret.key

Swap Partition 

# mkswap
# blkid

* Add to /etc/fstab using the UUID="" device, for great justice
* Recommended minimum swap space: square root of RAM size in GB

1| To access a new target with an iSCSI initiator

* install iscsi-initiator-utils
* Set initiator's IQN in /etc/iscsi/initiatorname.iscsi
* Discover iSCSI targets provided by the server:
 iscsiadm -m discovery -t st -p 192.168.0.254
* Log in to one or more iSCSI targets on the server:
 iscsiadm -m node -T iqn... -p 192.168.0.254 -l
* Identify which device is the iSCSI target:
** Look at the output of dmesg or /var/log/messages, run service iscsi status, or:
 ls -l /dev/disk/by-path/*iscsi*
*** Ignore any sr/sg devices unless that's what you want

2| To make iscsi mounts permanent

 Start the iscsi daemon

 #chkconfig iscsi on
 #service start iscsi

** If this gives an error due to an existing login from above, log out:
 iscsiadm -m node -T iqn... -p 192.168.0.254 -u
* Add /etc/fstab entries with _netdev in the options
** This prevents automatic mounting until the network subsystem has been started

=== Unit  4: Logical volume management === 

►►TAKE A LOOK AT THIS POST "RHCSA/RHCE LOGICAL VOLUME MANAGEMENT"◄◄

* pvcreate: create physical volumes
* PVs are made up of extents
*Default VG physical extent size is 4 MiB
*Limit of 65,536 extents per LV
* Larger VGs are automatically created with larger physical extents
*PV extent size not selected until added to VG

 #pvcreate /dev/sda3

* vgcreate: create volume group

 #vgcreate td /dev/sda3

* vgextend: import PVs into an existing VG
* lvcreate: create logical volumes within a VG

 #lvcreate -n data -L 64G td

Extending a filesystem

1| Verify available space in the VG

 #vgdisplay

* Extend the LV

* -L size is the size to extend to

 #lvextend -L 96G 

2| Extend the filesystem

 #resize2fs -p /dev/td/data

Reducing a filesystem

* Unmount the filesystem
* fsck the filesystem
* Shrink the filesystem

# resize2fs -p /dev/td/data 48G

* Shrink the LV
# lvreduce -L 48G /dev/td/data

* Remount the FS
# mount -a

Extending a volume group

* Prepare a new partition of type 0x8E (Linux LVM)
* Use pvcreate /dev/sda4 to create a PV
* Extend the volume group onto the new PV:

 #vgextend td /dev/sda4

Reducing a volume group

* Relocate physical extents on the PV to other PVs in the VG
* Remove the PV from the VG:
 #vgreduce td /dev/sda4

=== Unit  5: Account management ===

1|Password ageing

* chage
* -d last change date
* -m min days
* -M max days
* -W warn days
* -I inactive days

2|Account modification

* usermod
** -L lock the account

ACLs

* getfacl
* setfacl
*-m modify

#setfacl -m u:friend:rw filename
#setfacl -m d:g:grads:rwx directory (default ACL, a la setgid)
* -x remove

#setfact -x u:friend filename

* user and group masks determine the maximum permissions exerted via ACLs
* updated automatically by chmod

* File systems created during OS install have ACLs enabled
* File systems created after OS install have ACLs disabled
* Use tune2fs to enable them

 #tune2fs -o acl,user_xattr /dev/sda3

=== Unit  6: Authentication management ===

* LDAP
* Kerberos
* SSSD
* Network mounting home directories

LDAP

* Provides a user database and authentication
** Can optionally use Kerberos for authentication instead
* system-config-authentication enables the sssd service, which looks up and caches LDAP
* Verify that LDAP is working:

 #getent passwd [username]

Kerberos configuration

 #yum groupinstall directory-client
 #yum install openldap-clients krb5-workstation

* Then use the system-config-authentication GUI tool, or authconfig from the CLI

* sssd can be configured to not network-authenticate certain accounts

Network mounting home directories

* Check for NFS exports:
 #showmount -e nfsserver.domain

* Confirm the local mountpoint:
 #getent passwd [username]

* /etc/auto.master:
 /home/guests /etc/auto.guests

* /etc/auto.guests:
 ldapuser1 -rw instructor.example.com:/home/guests/ldapuser1

* Or we can use wildcards:

 * -rw instructor.example.com:/home/guests/&   (* is all users)

*Options: -rw,hard,intr
* read-write, hard-mounted (no timeouts), user interruptable
* Get autofs to re-read its config:
 #service autofs reload

=== Unit  7: Installation, Kiskstart and virtualisation ===

* Creating a Kickstart file by modifying a template
* Introduction to KVM
* Guest installation
* Management

Kickstart

* /root/anaconda-ks.cfg records the configuration of an installation
** Can be edited using system-config-kickstart
*** Also manually, which is required if LVM or individual packages are needed
* Use ksvalidator to check the syntax in order to prevent delays due to typos
* Useful options:
 zerombr

KVM

* Use virt-manager to manage virtual machines
* Or the virsh utility:

 #virsh list
 #virsh start name
 #virsh shutdown name ( graceful shutdown)
 #virsh destroy name   ( power off, does not delete)
 #virsh console name  ( ^] to escape)
 #virsh autostart name

=== Unit  8: Boot management ===

* Resolve GRUB issues
* Reconfigure GRUB
* Change the default runlevel
* Single user mode
* The boot process
* Rescue mode

* The single kernel parameter does not require login, not even the root password
* If passwd doesn't work in single user mode, this is due to a bug in SELinux; run the following:
 setenforce 0
 passwd root
 setenforce 1

* During service startup:
** Press Alt+Tab to display service start messages
** Press Shift+I to select services interactively

Repairing GRUB

* Start GRUB from the CLI within Linux, or open the GRUB CLI from GRUB
* Identify the /boot partition
 root (hd0,0)
* Install the first stage into the MBR:
 setup (hd0)

=== Unit  9: SELinux Management ===

* Three modes:
* Disabled
* Permissive
* Violations permitted, but logged
* Enforcing
* Violations not permitted

* To display SELinux contexts on files:
 #ls -Z

* There are rules that apply contexts automatically to files when they are created
* Based on parent directory?
* Moving files will not update the context
* They should be copied instead
* But restorecon can be used to reset the contexts to appropriate values
# restorecon -Frvv [directory]

* To view or modify the rules that restorecon uses to set default contexts:
 #semanage fcontext

* To check and set SELinux mode:
 #getenforce
 #setenforce [ Enforcing | Permissive | 1 | 0 ]

* To make change persistent, edit /etc/sysconfig/selinux
* This is a symlink to /etc/selinux/config
* Disabling or enabling outright is only via this file, therefore requires a reboot
* SELinux context type, the third field in the SELinux part of an ls -Z display, is the most relevant
* To manage SELinux:
 #semanage

* Or use the SELinux Administration GUI tool
* Note that the tool has a number of bugs, including:
* Add File Labeling will sometimes not complete
* fcontexts can be created which apply to all files, but these can subsequently not be deleted
* It's probably better to look up the semanage commands in the relevant man pages where possible
* e.g. man ftpd_selinux

* SELinux booleans are rules that determine the security policy
 #getsebool -a | grep ftp
 #semanage boolean -l | grep -i ftp ( more detail, including state)
 #setsebool -P ftp_home_dir on (-P: persistent across reboots)

* setroubleshoot daemon logs errors to /var/log/messages
* These can be debugged by passing their UUID to sealert:
# sealert -l [UUID]

=== Unit 10: Firewall management ===

* To enable packet forwarding, set sysctl net.ipv4.ip_forward = 1

 Targets
* ACCEPT
* DROP
* REJECT ( reject with ICMP port unreachable message)
* LOG

* system-config-firewall is a useful GUI tool for getting started
* Will tell you if the firewall service is disabled
* But cannot create or modify advanced firewall configurations

* Display filter table:
 #iptables -L

* Display NAT table:
 #iptables -L -t nat

* Make the current iptables settings persistent across reboots:
 #service iptables save

* Remove all rules:
 #iptables -F

* Add a REJECT target to the end of the INPUT chain:
#iptables -A INPUT -j REJECT

* This will reject all inbound packets, so don't run this without console access
* Add an ACCEPT target for localhost to the start of the INPUT chain:
 #iptables -I INPUT -s 127.0.0.1 -j ACCEPT

* Allow all connections from the local subnet:
# iptables -I INPUT -s 192.168.0.0/24 -j ACCEPT

* Allow all new connections:
# iptables -I INPUT -m state --state NEW -j ACCEPT

* Allow all established connections:
# iptables -I INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

* Can't merge the above two rules for some reason?
* Insert a LOG target in the second position of the INPUT chain:
# iptables -I INPUT 2 -m state NEW -j LOG

** Try not to log everything, as this causes considerable load

 Various iptables flags

 -i {eth0,...} inbound network interface
 -o {eth0,...} outbound network interface
 -s 192.168.0.0/24 source IP address or range
 -d 192.168.0.0/24 destination IP address or range
 -p {icmp,tcp,upd} protocol

 --icmp-type {echo-reply,...}
 -m state stateful packet filtering
 --state {NEW,INVALID,...} state to match
 --sport source port
 --dport destination port

 NAT

 Forward port 80 to a different server

 sysctl -w net.ipv4.ip_forward=1
# iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.0.254
# iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE

* In this configuration, the PREROUTING chain is used to forward traffic to the web server, while the POSTROUTING chain is used to allow the web server access to the outside network via the NAT host

=== Unit 11: NTP ===

* restrict keywords:
* default
* ignore: server packets disallowed, but not client packets
* kod: throttle access-violating clients with a kiss of death
* nomodify
* noquery
* nopeer
* notrap
* or use an IP address with optional mask

=== Unit 12: System logging service ===

* rsyslogd confgured in /etc/rsyslog.conf
* Uncomment appropriate lines for UDP or TCP syslog reception
* Rules section contains entries in the following format:

 service.loglevel {logfile,@udp_host,@@tcp_host}

* service and loglevel can each also be 

* To test the configuration:
 logger Hello,\ World\!

=== Unit 13: Web service ===

* SSL
* VHosts
* CGI
* User-based authentication
* Troubleshooting Apache

 Install the doco
 #yum install httpd-manual

Configuring SSL
 1| First install the SSL module:
 #yum install mod_ssl

* Or simply:
 #yum groupinstall web-server

2| Install certificate and key in /etc/pki/tls/{certs,private}
3| Configure in /etc/httpd/conf.d/ssl.conf:
 #SSLCertificateFile /etc/pki/tls/certs/localhost.crt
 #SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

4| Name-based virtual hosting

 Create a new NameVirtualHost entry in /etc/httpd/conf/httpd.conf or /etc/httpd/conf.d/hostname.conf, e.g.:

 NameVirtualHost *:80 # enable on given interface

 <VirtualHost *:80> # host on matching interface
  ServerName www.example.com
  ServerAlias example.com
  ServerAdmin webmaster@example.com
  DocumentRoot /var/www/example.com/html
 </VirtualHost>

 If the document root is outside /var/www, update the SELinux context:

 semanage fcontext -a -t httpd_sys_content_t '/www/virtual(/.*)?'
 restorecon -Frvv /www/virtual

Or use the SELinux Administration GUI tool

CGI

* Add the following to an existing VirtualHost:
 #ScriptAlias /cgi-bin/ "/var/www/example.com/cgi-bin/"

* CGI scripts must output as a minimum a Content-type: header

User-based authentication
* Create .htpasswd file:
# htpasswd -cm .htpasswd bob ( create)
 #htpasswd -m .htpasswd fred ( modify)

 Configure VHost to use it by adding the following to an existing VirtualHost:

  <Directory /var/www/html/private>
  AuthName "Secret Stuff"
  AuthType basic
  AuthUserFile /etc/httpd/.htpasswd
  Require valid-user
  Options +Indexes # permits directory listing
  </Directory>

=== Unit 14: Basic SMTP configuration ===

* Use postfix as an MTA, mutt as an MUA
* Postfix logs to /var/log/maillog
* Postfix configuration in /etc/postfix/main.cf
* Reconfigure to accept mail on all interfaces instead of just localhost:
 inet_interfaces = all

* Have hosts masquerade as the domain:
 myorigin = example.com

* This only affects locally sent mail, so it's OK to set on inbound MTAs
* Default host to send mail to:
 relayhost = smtp.example.com

* Can be overridden with the optional transport table
* The list of domains that this machine considers itself the final destination for:
 mydestination = $myhostname, example.com, foobar.com

* Set on the inbound MTA only
* How messages addressed to $mydestination should be delivered:
 local_transport = error:local delivery is disabled

* Enables local delivery by default; disable as above to force sending via $relayhost
* Trusted SMTP clients that we will relay from:
 mynetworks = 127.0.0.0/8, 192.168.0.69

* If postfix has unsent messages in its defer queue, send it a HUP signal to get it to retry, or run:
 postfix flush

=== Unit 15: Caching-only DNS server ===

* The bind daemon is called named under RHEL
* Configured in /etc/named.conf
* To configure a caching-only name server, edit it as follows:

 listen-on port 53 { any; };
 allow-query { 192.168.0.0/24; };
 forwarders { 192.168.0.1; };

* Then change resolv.conf, but remember that permanent changes must be made in /etc/sysconfig/network-scripts/ifcfg-eth0

=== Unit 16: File sharing with NFS ===

* NFSv4 improvements:
* Only uses one port, 2049
* Can mount the root of the NFS server's shares, i.e. all shares under one directory
* /etc/exports format:
 /shared/directory 192.168.0.0/24(ro,async)

* Can also use (*) in place of hostname/subnet
* async caches writes for improved performance
* Separate per-client cache control when a client mounts a share
* Check the export list:
 #showmount -e

=== Unit 17: File sharing with CIFS ===

* Connect to an SMB share using the GUI filesystem browser, or the CLI smbclient
# smbclient //server/share -U user%passwd

* Common smb.conf options:
* workgroup
* hosts allow
* security
* Useful smb.conf share parameters:
* write list = user, @group
* But Unix permissions still apply, i.e. even if the user or group is in the write list, they will be restricted by the directory's permissions

* If a user only needs to access samba shares, they don't need a login shell
* Create a samba password for them:
 #smbpasswd -a winuser

* If home directories are to be exported, SELinux boolean needs to be enabled:
# setsebool -P samba_enable_home_dirs on

* Home directories are exported at the server root level, not under /home
* Other directories need to have file contexts set:
 #semanage fcontext -a -t public_content_t '/shared(/.*)?'
 #semanage fcontext -a -t samba_share_t '/shared/doc(/.*)?'

* Also remember to open firewall ports if required:
 #system-config-firewall

=== Unit 18: File sharing with FTP ===

* FTP drop-box anonymous upload
* Create a directory belonging to group ftp, permissions 730
* Set security accordingly:
 #semanage fcontext -a -t public_content_rw_t '/var/ftp/incoming(/.*)?'
 #restorecon -Frvv /var/ftp/incoming
 #setsebool -P allow_ftpd_anon_write on

* Install vsftpd and configure /etc/vsftpd/vsftpd.conf:

 anon_upload_enable=YES
 chown_uploads=YES
 chown_username=daemon ( chown to a user without login privs)
 anon_umask=077 (default)

* Modify /stc/sysconfig/iptables-config to permit inbound FTP connections:
 IPTABLES_MODULES="nf_conntrack_ftp nf_nat_ftp"

* Presumably only PASV mode will work without this
* Use iptables to permit connections:
# iptables -A INPUT -p tcp --dport 21 -j ACCEPT
# iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

=== Unit 19: SSH service ===

* Generate a new SSH private:public key pair:
 #ssh-keygen

* Not providing the optional passphrase will permit passwordless login
* Copy the public key to another server:
 #ssh-copy-id -i .ssh/id_rsa.pub root@server


= Red Hat Certification =

== 4 Certification ==

RHEL 4 versions were certified using the Red Hat Ready v.2 certification test suite (rhr2).  But as of RHEL 4.6, new certifications are to be done with the certification test suite used for RHEL 5 - Red Hat Hardware Test Suite (hts-5.1).  See below for discussions on the Hardware Test Suite.

== Red Hat Enterprise Linux v.5 Certification ==

All certifications for RHEL 5.1 and RHEL 4.6 and later are to be done with the Red Hat Hardware Test Suite (hts-5.1). 

=== Planning for Certification ===

==== Understand certification requirements and policies ====

* For an overview of the Red Hat policies, be sure and review the Hardware Certification Policy document listed below in the section on Red Hat Certification Documentation.  
* Any new server or IO card for which Red Hat Linux support is being added will need to be certified for each major Red Hat release (RHEL 4, RHEL 5, etc.).   Note that once hardware has been certified on a major release that same hardware does not need to be re-certified for any updates to that release.
* Server firmware updates do not need to be re-certified
* For certification tests, only the fastest/largest cache and highest feature set cpu needs to be used for this test. For example, a server that has both Montecito and Montvale processors, will need to have its certification tests run with the Montvale processors only.
* One of the items that Red Hat requires is a pointer (URL) to the QuickSpec document for every server that is being certified.  Red Hat will review this document and expect certification test results for every device that is listed as Configured To Order (CTO) for that server. That is any device that can be factory integrated with that server.
* Make sure that you have all the equipment that will be listed in the QuickSpec so that it can be tested.

==== Develop certification and leverage plans ====

* When multiple servers are being certified that support the same IO card, the certification test for that IO card only needs to be run on one of the servers and that test result can be leveraged to the other servers.
* This IO card leverage could also be used when the same IO chip is used in core IO (or embedded) on multiple servers, but I usually do not utilize that leverage unless it using the exact same IO card like the rx3600 and rx6600 do.
* When there are multiple servers that need to be certified, I find it useful to complete a leverage matrix like this one: [http://linux.fc.hp.com/projects/testing/RedHatCert/leverage-template.xls  Leverage Matrix (Excel version)]  Note that if you send this to Red Hat you will want to convert it into either an Open Office format, HTML, or PDF. Here's the [http://linux.fc.hp.com/projects/testing/RedHatCert/rhel5-phaseI-cert.sxc  matrix] that I used for RHEL 5.0 certification.  It is in Open Office format.

==== Work with the HP Red Hat Program manager and HP One-Voice team to prioritize certification plans with other HP organizations ====

* When there will be a large number of systems certified at one time, for example when a major release of Red Hat will occur, the list of servers that will be certified and an estimate of when those tests will be run should be given to the One-Voice team early in the testing cycle.  The One-Voice team will combine that list with the lists of other test teams and work with Red Hat to develop a priority list for Red Hat to review the test results.

==== Communicate the leverage plans to Red Hat Technical Account Manager (TAM) ====

* The Red Hat TAM no longer requires the leverage matrix.  But, it does not hurt to keep them aware of what the team will be certifying and leveraging

=== Performing Certification Tests ===

==== Be sure you have the latest certification test suite version ====

* Check on Red Hat Network for the most recent certification test suite and download it.
* Confirm with the Red Hat TAM that there are not newer certification test suites available from other sources such as partners.redhat.com
* In Fort Collins, the RHEL 5 certification test suites are kept in linuxqa:/pbfd/test/cert/redhat/rhel5

==== Install target distribution on the server under test (SUT) and an additional certification server (if doing network card certifications) ====

* The certification tests must be run on the released (GA) version of the Red Hat Enterprise Linux being certified with.  (With major version releases - not updates - Red Hat will sometimes permit the use of the Release Candidate version).
* The certification test suite has a large number of dependencies.  Most of these can resolved by installing :
    RHEL 5 - Software Development, Virtualization, and Web Development package sets
    RHEL 4 - Everything package set (found under the custom installation menu)
*For RHEL 5, most of the tests need to be done on the dom0.  (For the core/memory tests, dom0 must have the same number of vcpus as physical CPUS and 1GB less than the maximum physical memory
*During the installation (or at first boot for RHEL 5), be sure and disable the firewall.  But do not disable SELinux.

==== Install certification suite on SUT and certification server ====

* There is a shell script that can help in installing the hts certification test suite.  In Fort Collins, it can be found in '''linuxqa:/pbfd/test/cert/redhat/rhel5/SetupCert'''. Copy the script to the /root directory of the SUT.
* Run:
    SetupCert hts-5.1-16 Model_number Release_type[betas|release] Release_tag(ie rhel5.1) myusername
* ... or the test suite can be installed by hand:
* First install the mt-st rpm from the Red Hat distribution (the lmbench package has a dependency on this and it is not installed during the Red Hat installation.
* Then copy the latest version of the hts test suite to the SUT. 
* Install the packages; lmbench, dt, stress, and hts
* Copy the Linux install DVD and CD disk 1 ISO images to /tmp for the CD and DVD tests
* Run:
    hts discover
* Edit the file /var/hts/config.xml as appropriate.  Set the ''vendor'', ''model'', and ''make''.  For example:
    &lt;vendor&gt;Hewlett-Packard&lt;/vendor&gt;
    &lt;model&gt;rx7640 dom0&lt;/model&gt;
    &lt;make&gt;Integrity&lt;/make&gt;
* After the hts test suite has been installed and configured, to generate a 'test plan', run 
    hts plan
* This plan can be viewed using:
    hts print --report summary

==== Run the appropriate tests on the SUT ====

These would be the tests from your certification and leverage plan. See the Red Hat documentation on hts users guide and the Tips and Tricks section below for running the tests.
* I find it useful to run just one or or a set of tests at a time.  Running all the tests at once makes it difficult and time-consuming to deal with test failures.  A single test may be run by:
    hts certify --test test_name
* There are other ways to invoke tests, see the Red Hat hts documentation for details on these other methods

=== Submitting Certification Results ===

* Test results are packaged into rpms by running:
    hts submit
**Test results can be packaged one rpm per test run by running hts certify after each test has been run.
**All test results can be packaged into a single rpm by running all the tests prior to running hts submit.
*Copy the test result rpms from the SUT to an archive system.  In Fort Collins, that would be:
    linuxqa:/pbfd/test/cert/redhat/RESULTS/Linux_version/Server_model
*Logon to the Red Hat Certification system: [https://hardware.redhat.com/list.cgi?GoAheadAndLogIn=1 Red Hat Certification Login]
*Choose the ''Create'' function from the left panel
*Enter the appropriate information and include one of the test results rpm, then submit
*This will create a new ''certification entry'' which is actually a bugzilla record.
*Red Hat will review this information based on their availability and the priority of the server for certification.  They do not need an entry with all test results posted to start reviewing.
*When Red Hat begins review, they will add a 'test plan' to the certification record.  This test plan can be updated by the submitter as well - to include leverage information on test results that can be applied to this record but were submitted to other certification records

=== Certification Review by Red Hat ===

''' Red Hat's responses '''

Red Hat will indicate when a certification record has been reviewed and accepted by changing the status of the  certification record to ''Certified (Supported)''.  Since this is a bugzilla record, the submitter of the record will receive email when this occurs.

''' Communicating review status '''
Once a certification has been approved, there are a number of teams that need to be aware of the results of the certification; 
*The HP Red Hat program manager (currently [mailto:jeanne.colon-bonet@hp.com Jeanne Colon-Bonet])
*The OSLO marketing representative who owns the [http://h71028.www7.hp.com/enterprise/cache/321101-0-0-0-121.html  HP certification web site] (currently [mailto:john.rinehart@hp.com John Rinehart])

== Red Hat Certification Documentation ==

* [http://linux.fc.hp.com/projects/testing/RedHatCert/RHEL5Cert/Hardware_Certification_Policy.pdf  Local copy of RHEL 5.1 cert policy (must read)]
* [http://linux.fc.hp.com/projects/testing/RedHatCert/RHEL5Cert/HwCert51PartnerSlides.pdf  Local copy of RHEL 5.1 cert policy updates]
* [http://linux.fc.hp.com/projects/testing/RedHatCert/RHEL5Cert/hts-5.1-doc/index.html Local copy of RHEL 5.1 cert docs - hts-5.1]
* [https://hardware.redhat.com/page.cgi?id=faq.html Red Hat FAQ on certification]

== Tricks / Tips ==

* When running the memory certification test, the Out-of-Memory (OOM) killer may be invoked because the test will consume all the system's memory.  This will usually result in a failed test.  To avoid this, be sure and configure the system that will be running the memory test with at least 1.5 times the amount of swap space as there is memory to be tested.  It may even be necessary to use 2 times that amount of swap as memory.

* For RHEL 5.1 certs, maximum system tests must be done on dom0.  To configure dom0 to be maximum edit the append line in the elilo.conf. This example is for a system with 16 cpus and 192GB (leave 1GB available for overhead.
             append="dom0_max_vcpus=16 dom0_mem=191G -- debug"

* If you are running more than one certification test on a SUT, you may want to change the name of the rpm files to keep better track of the test they relate to.  When the hts submit command is run, the test results will be packaged into an rpm and then written to the /var/hts directory on the SUT.  The rpm file name will have the company, model, and make in the name.  You do not want to change those. But the only unique part of the name is a number.  Change that number to be the name of the test or IO card that was tested.
* Running the cert program over serial console: There is a bug in the implementation of pseudo serial console of the Management Processor (MP). It does not supply modem handshake lines to the driver and the RHEL4 expects to see them (like Carrier Detect).  This results in a hang in some output operations like echo foo > /dev/console.  There are times like NETWORK cert where redhat-ready must be run from the serial console.  The workaround is to edit the getty line in /etc/inittab.

    co:2345:respawn:/sbin/agetty -L ttyS0 9600 vt100-nav

Signal the init process to re-read the inittab file - kill -HUP 1  and exit to restart the getty.

You should also add
    rhrconsole=ttyS0
to the boot line (at ELILO: prompt or in the elilo.conf file)

* Individual RHEL4 cert tests can be run from the command line by:
    redhat-ready adventure TEST
where TEST is the same name as in the adventure test menu, like STORAGE, NETWORK, etc.
Note that if you are running a test to send the results in, be sure and invoke the INFO
test along with the test you are running - eg.  
    redhat-ready adventure INFO STORAGE


* Precautions when running the RHEL NETWORK test.  There have been incidences in the past where the NETWORK test would cause a packet flood.  A good way to prevent this is to connect all the ethernet interfaces on the client and the test server to the same network switch and then put those ports in a VPN on that switch.  Even after doing that, it's still a good idea to monitor net traffic from a different system on the 10.net using the ethereal program or the one I use - tcpdump ip and host not wilson.rihes. The expressions in tcpdump line - not wilson.rihes - are there to reduce the output you see, so that it's easier to catch a packet storm kicked off by the network test.


No comments:

Post a Comment