Archive for the ‘ Technical ’ Category

Quick postfix queue depth script

Or counting files in any folder(s)

for i in `ls -al /var/spool/postfix |grep '^d'|awk '{print $9}'|grep -v '\.$'`; do c=`find $i|wc -l`; echo "$c $i"; done;

FreeIPA Server/Client setup on CentOS 6.5

So I’ve been dorking around with 389-ds a LOT at work lately and it’s a bitch to setup, especially when it comes to the certs.  As a hackathon project I decided to setup FreeIPA, which is the Free version of Redhat Identity Manager as a more comprehensive and easy to manage solution.  I have this setup at home as well in my personal lab.  Some pre-requisites first… make sure you have DNS and REVERSE LOOKUPS for all servers and clients and if you’re running iptables or a firewall on your hosts then make sure you have the following ports open, TCP/UDP: 888/444 for kerberos and 389/636 for ldap.

Here are the following specifics for our setup:

  • Domain:                   
  • Realm:                               EXAMPLE.COM
  • Server1:                   
  • Server2(replica):       
  • Client:                      

FreeIPA Server Setup

This setup is stupid easy, you just have yum install the ipa-server, then set it up with ipa-server-install.  Note you can just call run #ipa-server-install without any flags and it will ask you realm, domain, etc.  It will automatically setup your certificates, kerberos, etc… etc…

# yum -y install ipa-server
# ipa-server-install --realm=EXAMPLE.COM

Next let’s see if IPA is working correctly by requesting a ticket for the admin user

#kinit admin

There shouldn’t be any output, let’s validate that the ticket was issued

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin@EXAMPLE.COM

Valid starting     Expires            Service principal
02/12/14 16:09:06  02/13/14 16:09:03  krbtgt/EXAMPLE.COM@EXAMPLE.COM
02/12/14 16:49:46  02/13/14 16:09:03  host/

Replication Setup

On the master(Server1):


Copy the gpg file that was just created at /var/lib/ipa to Server2(the replica) and run the following command on Server2:

#ipa-replica-install /var/lib/ipa/

Client Setup

I’ll specify that I find it rather important to specify the –mkhomedir flag. If you don’t then setting it up later can be a bitch.

#ipa-client-install --realm=EXAMPLE.COM -p admin  --password=<password> --mkhomedir

Enable WebUI access from anywhere

By default the WebUI is only accessible from authenticated IPA clients. This means that non-ipa clients can not access the WebUI to manage FreeIPA. While the extra security is nice, in a dev or lab setting it might be overkill. Here’s the workaround I found so you can access the webui from any computer. How the workaround actually works is that enables kerberos authentication through the web browser itself. Note that you have to do this on all freeipa servers.

On the server(s) open the ipa.conf file used by the Apache web service.

#vim /etc/httpd/conf.d/ipa.conf

In the <Location “/ipa”> location definition, change the KrbMethodK5Passwd attribute from off to on.

KrbMethodK5Passwd on

Restart the httpd service:

# service httpd restart

One small issue I found is the default admin user doesn’t seem to work if you try to access from a non-ipa client. I had to create another user(webadmin) give it admin privileges and log in to a machine as that user before I could access the WebUI from non-ipa clients.


Yum and kernels, removing old one’s and limiting how many kernels yum keeps around

Check installed kernels:
# rpm -q kernel

Remove old kernels:
# package-cleanup –oldkernels –count=2

Make it permanent:
#vi etc/yum.conf

Working with volume groups that have the same name (cloned disk, recoveries, etc)

First, we attach the vmdk to the vm and then scan the bus to see it in linux:
# echo “- – -” > /sys/class/scsi_host/host0/scan

In this scenario, /dev/sda2 and /dev/sdb2 have the same volume group name of VolGroup00, let’s rename /dev/sdb2 to VolGroup01:
# vgimportclone –basevgname VolGroup01 /dev/sdb2

next let’s find the new VolumeGroup:
# vgscan
# vgchange -a y

Now let’s mount the LV we want from the new volume group:
# mount /dev/VolGroup01/LogVol02 /mnt/restore/

Once we’re done, we want to unmount the LV and remove reference to the VG
# umount /mnt/restore
# vgchange -an VolGroup01 (make VG unavailable)
# sync

Remove the drive, wait a few minutes and let’s do a rescan:
# vgscan

Finding biggest directories in linux

find . -type d -print0 | xargs -0 du -s | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

How to SSH tunnel a VNC connection and launch a Gnome desktop

For remote graphical access to a linux server I generally prefer to use NX aka nomachine, however I’ve been doing admin work on some shared servers that I don’t want to install NX on for a few different reasons.  All of the other admins here use VNC with SSH tunneling to access these boxen so I figured I would toe the party line and use VNC as well.  Here’s a quick guide as to what I did in order to get VNC with SSH tunneling working, complete with accessing a gnome desktop.

I am using Putty for SSH on Windows, and a TightVNC client to access a CentOS 6.x box that is running a Gnome desktop.

Putty Setup

In the putty session for the host you are connecting to, go to SSH –> Tunnels and set the port that you are going to forward.  VNC ports start @ 5900 by default and vary depending on the session number you choose.  In my example we’re going to use session number 66, so we’re going to forward port 5966 to localhost:5966 and click add, then save the configuration.

















VNC Server Configuration

In this section we’re going to configure the VNC server, define ~/.vnc/xstartup to launch Gnome when your VNC session is started and actually launch a VNC session to connect to.

SSH to the vncserver using the session with the tunnel defined from the last step.  Once you’re there create ~/.vnc/xstartup if it is not already defined and replace the contents with the following:

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
/usr/bin/gnome-session & gnome-terminal &

Mainly what we’re looking for here is the last line, which tells X to launch a gnome session when a new VNC session is created.  In this example I also have it launching gnome-terminal, so you can see how you could launch multiple apps every time a session is initiated.  You could add whatever apps you want to this, such as firefox or eclipse.  When you’re done editing the file, make sure it is executable by doing a chmod +x.

Finally, we’re going to launch a vnc session  and then connect to it with tightvnc.  If this is your first time launching a VNC session then it will ask you to define a password to secure your current and future sessions.  Here, :66 defines the session number.

$vncserver :66

It is important that your session number match the port that is forwarded… vncserver :10 would launch a session on port 5910, vncserver :22 would launch a session on port 5922.  Whatever session number/corresponding port is launched has to be defined as a tunnel in putty.

Now that the session is launched, we just open TightVNC(or your favorite VNC client) and connect it to localhost:66 (session number), enter the VNC password that you defined and voila!






That’s it!  we now have our connection to our Linux server with Gnome launched!











VCP5 Certified!

I’m now a VCP5!  The test was hard and I barely passed it, but I passed it on my first try which most people don’t do.  The exam was pretty difficult, with a lot of specific and hardly worded questions.  Lots of “Pick 3 answers that apply” kinds of questions.  I recommend just taking VMware’s practice exam(at until your brain hurts, then moving on to SLOG’s practice exams and do the same thing.  The Exam Cram is a nice list of technical bullet points to memorize before you head into the exam.  Here’s the resources that I used for studying:


SLOG: VCP5 Practice Exams

Cosonok’s Exam Cram How to pass the VCP5 exam


Working With LVM In Linux

Creating a new volume group, adding a disk to it and making it usable

Scan HBA for new LUN’s:
#echo ‘- – -‘ > /sys/class/scsi_host/hostX/scan

#fdisk /dev/sda5 (or /dev/mapper/mpathx if multipathing) … create new partition, type lvm (8e), write changes to disk
#kpartx -a /dev/mapper/mpathX if it’s a multipathed device using dm-mulipath, otherwise skip this step
#pvcreate /dev/sda5 or /dev/mapper/mpathXpX  (initializes partition for LVM)
#vgcreate vg02 /dev/sda5 or /dev/mapper/mpathXpX (or vgextend vg02 /dev/sda5 or /dev/mapper/mpathXpX to add to a volume group)
#lvcreate -L 500G -n lvora_backup vg02  (or lvextend to add)
#mkfs -V -t ext3 /dev/mapper/vg02-lvora_backup  (or resize2fs to extend the fs)
#mount /dev/mapper/vg02-lvora_backup /ora_backup

edit /etc/fstab:

/dev/vg02/lvora_backup  /ora_backup             ext3    defaults        1 2

Extending a logical volume if the vg has available space

lvextend -L +512M /dev/rootvg/lvtmp
resize2fs /dev/rootvg/lvtmp (If it’s ext3, if not then use your specific filesystem tools)

If it’s GFS2, Find what it’s mounted as using cat /proc/mounts, we’ll look for /home3 in this example:

[root@linuxserver ~]# cat /proc/mounts |grep /home3
/dev/dm-54 /home3 gfs2 rw,noatime,nodiratime,hostdata=jid=0,localflocks,data=writeback 0 0

Next we’ll do a test run to make sure we don’t bugger anything up:

[root@linuxserver ~]# gfs2_grow -T /home3
(Test mode–File system will not be changed)
FS: Mount Point: /home3
FS: Device:      /dev/dm-54
FS: Size:        31457278 (0x1dffffe)
FS: RG size:     65535 (0xffff)
DEV: Size:       51132416 (0x30c3800)
The file system grew by 76856MB.
gfs2_grow complete.

Looks good, let’s run it without the -T flag:

[root@linuxserver ~]# gfs2_grow /home3
FS: Mount Point: /home3
FS: Device:      /dev/dm-54
FS: Size:        31457278 (0x1dffffe)
FS: RG size:     65535 (0xffff)
DEV: Size:       51132416 (0x30c3800)
The file system grew by 76856MB.
gfs2_grow complete.

How to find the scsi id of any device on linux

#scsi_id -g -u -s /block/sdx

If it’s a vm
#cat /proc/scsi/scsi

if it’s a /dev/cciss device(HP SAS) then use

#cciss_id /dev/cciss/cXdX


Enabling Round Robin and MPIO on vSphere4

The purpose of this article is explain how to enable round robin and multipathing on an ESXi4 cluster.

Our environment consists of:

  • 48 HP BL460cG1 servers running ESXi4 embedded with qlogic fc mezzanine cards(2 hba’s/host)
  • 3 C7000 Chassis with VC-Enet modules and Cisco MDS9124 switches
  • SAN fabric connected to an HP EVA 8400

These instructions are primarily from HP and are SPECIFICALLY FOR THE EVA!  Check with your SAN vendor for their recommendations.  The shared storage in our environment is all fibre channel, so these instructions will most likely not work on iSCSI or shared storage over other protocols.  This article assumes you have two hba’s per host as well.  Also make sure that your SAN or LUN’s are setup for an active/active configuration otherwise you’ll have problems with LUN trespassing.  Most newer SAN’s are active/active by default, but some SAN’s such as some of the older EMC CX series are setup for active/passive and you have to use powerpath or a vendor specific product in order to setup true multi-pathing on a host.  Perform these steps at your own risk! If you’re not comfortable with any part of this then do some research, reference the sources at the bottom of the page, or call VMWare support before you go ahead with this. Now that we’ve got the disclaimers out of the way, let’s get down to the good stuff.  The whole process consists of approximately 3 steps: enabling round robin on all LUN’s on all hosts in the cluster, setting each host to use both preferred and non-preferred paths, and finally telling each host how many iops before it switches paths, utilzing both paths more effectively and helping to spread the load across both controllers on your SAN.

Enabling Round Robin
Set multi-path policy to Round Robin on all LUN’s on all hosts in a cluster using PowerCLI:

Get-VMHost -Location <Clustername>|Get-ScsiLun -LunType "disk"|where {$_.MultipathPolicy –ne "RoundRobin"}|Set-ScsiLun -MultipathPolicy "RoundRobin"

Check to see if it took:

Get-VMHost -Location <Clustername>|Get-ScsiLun

The following steps are run in the “unsupported console”.  Google to see how to enable ssh on each host.

Set the default Path Selection Policy(PSP) to Round Robin and SATP to VMW_SATP_ALUA on each host

esxcli nmp satp setdefaultpsp --satp VMW_SATP_ALUA --psp VMW_PSP_RR

Set the LUN’s to use preferred and non-preferred paths
Login to each host and type in the following command:

for i in `ls /vmfs/devices/disks/ | grep naa.600` ; do esxcli nmp roundrobin setconfig --useANO 1 --device $i ;done

You might get some errors, but run this command to see if it took:

esxcli nmp device list |grep ANO=

Set amount of iops before it switches paths

for i in `ls /vmfs/devices/disks/ | grep naa.600` ; do esxcli nmp roundrobin setconfig --type "iops" --iops=1 --device $i ;done

By default this is set to 1000, and you’ll have to write a script that runs on startup as the setting doesn’t keep over a reboot. In fact it seems that if you touch the iops= setting, then after a reboot it’s replaced with a random number.
Check the sources below for more detailed information, especially the top link which is the HP “Official” best practices document for this scenario.