Simple example of how to perform an operation on every file in a directory:

for file in `ls *.pdf`

do

 convert -density 300 $file `echo $file | sed 's/\.pdf$/\.jpg/'`

done

Posted: November 13th, 2010 under Linux.
Comments: none

Complete the installation of SLES 10 first as per http://dan.obarr.us/2010/01/sles-10-installation-and-setup.html.

Install OES Components and Configure eDirectory:

• Start the OES install:
  • YaST: Software, Add-on Product

• Select Local Directory or NFS (depending on where your ISO images are)
• If using NFS, specify the source server’s fully qualified name address
• Check the ISO Image box and set the path to /install/OES2-SP1-x86_64-CD1.iso

iManager/ConsoleOne Tasks:

• Add an eDirectory replica to new server:

• • In iManager, go to Partition and Replica Management, Replica View
  • Enter SITE.XXXX, click OK
  • Click Add Replica. Select the new server and click OK.

Configure NSS (not needed if only re-installing the OS for a failed server – following these steps will delete existing NSS pool/volumes):

• Run nssmu from the command shell
• Go to Devices, select the logical drive for the NSS pool, and hit F3 to Initialize it (be careful, make sure you have selected the correct drive!)
• Create new pool named DATA using all of the logical drive
• Create new volumes on DATA pool

Turn off oplocks:

• Edit /etc/opt/novell/nspserv.conf and make sure it has these two lines:
  • OPLOCK_SUPPORT_LEVEL 0
  • CROSS_PROTOCOL_LOCKS 1
• This change requires a restart of NCP server (reboot)

Register and Patch the server:

• Register the server with Novell:
  • At the command prompt:
    suse_register -a email=xxx@xxxxx -a regcode-sles=XXXXXXXXXXXX -a regcode-oes=XXXXXXXXXXXX

• Update the server using YaST, Software, Online Update (this works in text mode with screen)

Posted: March 22nd, 2010 under Linux.
Comments: 1

Posted: January 20th, 2010 under Linux.
Comments: none

Conceptual Overview:

This document describes the steps necessary to deploy a new web application via a reverse HTTPS proxy on the Blue Coat Proxy SG. It involves setting up a virtual IP address on the Blue Coat that is servicing the web servers, and using the proxy to terminate an SSL connection from the client to that address. It will then forward the request to the servers themselves, in a load balanced manner. In order to provide full acceleration from other locations, we will need to set up a SSL intercept on each remote proxy, causing it to intercept the connection and forward it across the ADN to the master that is actually servicing the (virtual) website.

Set up Virtual IP (VIP) Address on Servicing Blue Coat Proxy:

• Use the Infoblox Grid Manager to choose an unused IP address in the app server subnet, and add a host record with that address and the corresponding URL for your virtual host, such as virtualserver.example.com
  • (I will add a link here to the Infoblox instructions when I post them.)
• Configuration -> Network -> Advanced -> VIPs
• Click New, enter the virtual IP address
• Click OK and Apply to save the configuration
• You should now be able to ping the virtual IP address by DNS name (ping virtualserver.example.com)

Set up Forwarding Hosts:

• Configuration -> Forwarding -> Forwarding Hosts -> New
• Enter the following values:
  • Alias: a simple descriptive name, such as the host name (webserver1)
  • Host: the fully qualified host name (webserver1.example.com)
  • Ports: check HTTP and enter the host server port (in this case, we use 7778 since the web site is hosted on Oracle HTTP server)
• Click OK
• Repeat the above steps for each host that will service this virtual host
• Click Apply to save

Set up Forwarding Group:

• Configuration -> Forwarding -> Forwarding Groups -> New
• Enter a descriptive group name
• Select each host server that will service the website and click Add
• Click OK and Apply to save

Set up SSL Access Policy in VPM:

• Configuration -> Policy -> Visual Policy Manager -> Launch
• Click on SSL Access Layer tab. If this did not already exist, you would add one by:
  • Click on Policy -> Add SSL Access Layer
  • Give it a descriptive name (SSL Access Layer, for example)
  • Click OK
• Click Add Rule
• If necessary, use the Move Up or Move Down buttons to change the order of this rule
• Right-click on the word Any under the Destination column in the rule you just added and select Set
• Click New, select Server URL
• Set up your matching parameters and click Add, Close, OK
  • In our current example, we’d use Simple Match, and enter the virtual host URL, virtualserver.example.com
  • Note: You could also create a new Combined Destination Object to create a list of URLs
• Right-click on the word None under the Action column in the rule you just added and select Set
• Select Allow and click OK to save

Set up Forwarding Policy in VPM:

• If not already in the VPM, go to Configuration -> Policy -> Visual Policy Manager -> Launch
• Click on the Forwarding Layer tab. If this did not already exist, you would add one by:
  • Click on Policy -> Add Forwarding Layer
  • Give it a descriptive name (Forwarding Layer, for example)
  • Click OK
• Click Add Rule
• If necessary, use the Move Up or Move Down buttons to change the order of this rule
• Right-click on the word Any under the Destination column in the rule you just added and select Set
• Select the same Destination URL that you added under the SSL Access Policy and click OK
• Right-click on the word Send Direct under the Action column in the rule you just added and select Set
• Click New, select Select Forwarding
• Enter a descriptive name (no spaces)
• Under Forward to:, select the forwarding group you created earlier and click Add, OK, OK
• Click Install Policy, then close the VPM

SSL Notes:

You have two options to create the SSL private key and your certificate signing request (CSR). The Blue Coat interface is easier to understand, but openSSL is sometimes more compatible. (NOTE: iManager would not issue a wild card certificate from a CSR generated on the Blue Coat, requiring me to use openSSL.)

Create New Keyring and Generate Private Key:

These instructions assume you are creating the key with the Blue Coat management interface.

• Start by creating a new keyring: Configuration -> SSL -> Keyrings -> Create
• Enter a keyring name.
  • For all of our internal web apps, we are using a wildcard certificate for (*.example.com), so we use something like wildcard.example.com-key as the keyring name.
• If you want to back up the private key or use it on any other device, select Show key pair or Show key pair to director. For greatest security, leave it set to Do not show key pair.
• For all internal purposes, we use 2048 bits.
  • 1024 bits is virtually unbreakable without a supercomputer or large cluster of computers, so it is not insecure, we just use the same number of bits as the default SSH keys on SuSE.
  • If you will be requesting a certificate from a recognized certificate authority, the number of bits should correspond to the certificate product you will order.
• Click OK, then Apply to save the new keyring.

Generate Private Key with openSSL:

• (I will insert a link when I post these instructions.)
• Create a new keyring as described above, but instead of creating a new key, select Import
• Copy and paste in your private key generated with openSSL

Generate Certificate Signing Request (CSR):

These instructions assume you are creating the CSR with the Blue Coat management interface.

• Click on the new keyring and click Edit
• Under Certificate Signing Request, click Create
• Fill out the fields similar to the following:
  • State/Province: Nevada
  • Country Code: US
  • City/Locality: Las Vegas
    
  • Organization Name: Examples-R-Us, Inc.
  • Unit: IT Department
    
  • Common Name: virtualserver.example.com
  • Challenge: (Blue Coat requires an entry, openSSL does not, since this is optional)
  • Email Address: mail@example.com
  • Company: Examples-R-Us
    
• Click OK, Close and then Apply. This will save the new keyring and generate the CSR.
• Click on the keyring and click Edit.
• Copy the contents of Certificate Signing Request box and save to a file (virtualserver.example.com, for example).

Generate CSR with openSSL:

• (I will insert a link when I post these instructions.)
• It is possible to import your CSR into the keyring if desired. This would be handy if you wanted to use the same CSR later to generate another certificate (perhaps with a new expiration date).

Issue Certificate from iManager and Import:

• (I will insert a link when I post these instructions.)

Set Up SSL Intercept Service:

• Configuration -> Services -> Proxy Services -> New Service
• Enter a Name for your service, such as Example SSL Proxy
• Under Service Group, select the group (this is just for organizational purposes, it’s not a critical step)
• Under Proxy settings, Proxy, select HTTPS Reverse Proxy from the drop-down
• Under the Keyring drop-down, select the keyring you created above
• At the bottom, click New to create a new listener
• Choose Destination host or subnet
• Enter the virtual IP address (VIP) you created in the beginning
• Click OK, OK, Apply to save

If you have done everything correctly, you should now be able to open a web browser and type in https://virtualserver.example.com/ and get your website!

Optional – Set Up URL Rewriting:

The Blue Coat Proxy allows you to re-write the request URL. So for example, you can allow the user to enter a “pretty” URL, and hide the actual request string that is sent to to the host server.

• Install the re-write policy by going to Policy -> Policy Files -> Install Local File From -> Text Editor-> Install
• Sample rewrite policy:
• define action virtualserver_example_com_portal
  rewrite(url, “^https://virtualserver.example.com/$”,”https://virtualserver.example.com/prod/”)
  end
  
  define action force_uncompressed
  delete(request.header.Accept-Encoding)
  end

  <Proxy>
  url.host=virtualserver.example.com action.virtualserver_example_com_portal(yes)

  <Cache>
  action.force_uncompressed(yes)

Posted: January 12th, 2010 under Blue Coat Proxy SG.
Comments: 4

• Storage setup for a standard file and print server:
  • Need two logical drives, RAID 1, 5 or 10
  • First logical drive is for /boot, swap and / partitions (recommend 50GB)
  • Second logical drive is for NSS pool (size accordingly)
• Installation options:
  • Local DVD install: Boot SLES 10 SP1 DVD and start a new install
  • Network install: Boot SLES 10 SP1 CD1 and use the following options:
    • hostip=xx.xx.xx.xx/24
    • gateway=xx.xx.xx.xx
    • nameserver=xx.xx.xx.xx
      
    • nfs://servername/install/SLES-10-SP2-DVD-x86_64-GM-DVD1.iso
  • To perform install remotely with VNC (DVD or network), add these options:
    • vnc=1
    • vncpassword=password
• Select Language, Accept License, select New Installation
• Set Clock and Time Zone info:
  • Set Hardware Clock Set To UTC
  • Region: USA, Time Zone: Pacific
  • Set Time and Date to current at your location (look at your watch)
• Click Partitioning:
  • Create Custom Partition Setup, Custom Partitioning (for experts)
  • You should have two logical drives available, sda and sdb:
    • Create 1GB primary partition on sda, format as ext2, mount as /boot
    • Create 2GB primary partition on sda, format as swap
    • Create a primary partition on sda, use remaining space, format as ext3, mount as /
    • sdb will hold the NSS pool, which will be configured after OES installation
• You can leave the defaults for software selection.
• Click Accept to start installation – Server will format partitions, install software, and reboot.

Staging Step 2 – Final Setup After Reboot:

• Enter password for root
• Enter hostname and domain (use lower case)
  • Uncheck Change Hostname via DHCP
• Network Configuration:
  • Click to Enable Firewall for SSH
    
  • Make sure VNC Remote Administration is enabled
  • Click Network Interfaces, Edit:
    • Add Name Servers
    • Add search domains
• Don’t check for Internet connectivity
• Accept defaults on CA Management and click Next
• Leave Authentication Method on Local, click Next
• Don’t create a local user, just click Next, and Yes that it’s OK
• Next on Release Notes
• Next on Hardware Configuration
• Uncheck Clone This System for Autoyast
• Base install is finished at this point. If you’re doing a VNC install, your installation session will die here. You can reconnect to the new server at this point if needed.

Staging Step 3 – Other Settings:

• If it is a normal file server, copy SLES and OES source ISO’s to server:
  • Create /install
  • Copy the following ISO’s to /install:
    • SLES-10-SP2-DVD-x86_64-GM-DVD1.iso
    • OES2-SP1-x86_64-CD1.iso
• Delete CD/DVD or network install source and add local source for SLES:
  • YaST: Software, Installation Source

• YaST: Network Services, NTP Configuration:
  • Change Automatically Start NTP Daemon to start During Boot
  • Go to Advanced Configuration
  • Add server ntp.doc.nv.gov and Enable for initial sync
• Configure and start SNMP:
  • Copy /etc/snmp/snmpd.conf from another server
  • Enter chkconfig snmpd on to set daemon to auto-start
  • Enter rcsnmpd start to start the daemon

• Enable Systat:

• Move on to http://dan.obarr.us/2010/03/oes-2-installation-and-setup.html
  

Posted: January 6th, 2010 under Linux.
Comments: 1

Setup for Backup Server:

• Configure forwarding of root’s email to GroupWise System Admin resource:
  echo user@test.com > /root/.forward
• Create/modify /etc/rsyncd.conf. See example Backup Server rsyncd.conf.
• Enable rsync daemon: chkconfig rsyncd on
• Start rsyncd: rcrsyncd start
• Change into the backup volume/directory:
  • For a NSS BACKUP volume: cd /media/nss/BACKUP
  • For a XFS /backup directory: cd /backup
• Create the backup folders for the primary server:
  mkdir primaryservername
  mkdir primaryservername/APPS
  mkdir primaryservername/backup
  mkdir primaryservername/boot
  mkdir primaryservername/etc
  mkdir primaryservername/GW
  mkdir primaryservername/mbr
  mkdir primaryservername/SHARED
  mkdir primaryservername/USERS
• Create the backup folders for the backup server:
  mkdir backupservername
  mkdir backupservername/backup
  mkdir backupservername/boot
  mkdir backupservername/etc
  mkdir backupservername/mbr
• Modify /etc/exports to export the BACKUP volume and /install directory:
  • /install *(ro,sync,root_squash) 10.1.1.0/255.255.255.0(rw,no_root_squash,sync) 10.1.2.0/255.255.255.0(rw,no_root_squash,sync)
  • For NSS BACKUP Volume:
    /media/nss/BACKUP primaryservername(rw,no_root_squash,sync,fsid=20) backup1(rw,no_root_squash,sync,fsid=20)
  • For XFS /backup directory:
    /backup primaryservername(rw,no_root_squash,sync) backup1(rw,no_root_squash,sync)
• Enable nfsserver: chkconfig nfsserver on
• Start nfsserver: rcnfsserver start
• Create backup script bkp_backupservername.sh. See example Backup Server Backup Script.
• Set executable permission: chmod +x bkp_backupservername.sh
• Create lst file. See example exclude.lst.
• Use crontab -e to edit the cron jobs and add the backup script:
  • For a NSS BACKUP volume: @daily /media/nss/BACKUP/bkp_backupservername.sh
  • For a XFS /backup directory: @daily /backup/bkp_backupservername.sh

Backup Setup for Primary Server:

• Configure forwarding of root’s email to GroupWise System Admin resource:
  echo user@test.com > /root/.forward
• Create the backup folder: mkdir /backup
• Change into backup folder: cd /backup
• Create the mount folder: mkdir primaryservername
• Modify /etc/fstab to auto-mount the backup volume:
  • NSS BACKUP volume:
    backupservername:/media/nss/BACKUP/primaryservername /backup/primaryservername nfs rw,intr,proto=tcp,rsize=32768,wsize=32768 0 0
  • XFS /backup directory:
    backupservername:/backup/primaryservername /backup/primaryservername nfs rw,intr,proto=tcp,rsize=32768,wsize=32768 0 0
• Mount the backup volume: mount -a
• Create backup script bkp_primaryservername.sh. See example Primary Server Backup Script.
• Set executable permission: chmod +x bkp_primaryservername.sh
• Create lst file. See example exclude.lst.
• Use crontab -e to edit the cron jobs and add:
  @daily /backup/bkp_primaryservername.sh

Example: Backup Server rsyncd.conf (NSS BACKUP volume):

uid = root
gid = root
read only = false
use chroot = true
transfer logging = false
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = primaryservername.test.com backup.test.com
slp refresh = 300

[primaryservername_APPS]
path = /media/nss/BACKUP/primaryservername/APPS

[primaryservername_boot]
path = /media/nss/BACKUP/primaryservername/boot

[primaryservername_etc]
path = /media/nss/BACKUP/primaryservername/etc

[primaryservername_GW]
path = /media/nss/BACKUP/primaryservername/GW

[primaryservername_USERS]
path = /media/nss/BACKUP/primaryservername/USERS

[primaryservername_SHARED]
path = /media/nss/BACKUP/primaryservername/SHARED

Example: Backup Server rsyncd.conf (XFS /backup directory):

uid = root
gid = root
read only = false
use chroot = true
transfer logging = false
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = primaryservername.test.com backup1.test.com
slp refresh = 300

[primaryservername_APPS]
path = /backup/primaryservername/APPS

[primaryservername_boot]
path = /backup/primaryservername/boot

[primaryservername_etc]
path = /backup/primaryservername/etc

[primaryservername_GW]
path = /backup/primaryservername/GW

[primaryservername_USERS]
path = /backup/primaryservername/USERS

[primaryservername_SHARED]
path = /backup/primaryservername/SHARED

Example: Backup Server Backup Script:

#!/bin/bash
# Version: 2008-02-27 03:10pm
# Modified By: Dan O.

StartTime=`date`
Today=`date +%F`
Yesterday=`date --date=yesterday +%F`
PurgeDate=`date --date='31 days ago' +%F`

echo "**********************************************************************"
echo "Starting backup of system files on $HOSTNAME at $StartTime..."

echo -e "\nChanging into backup directory '/backup/$HOSTNAME'..."
cd /backup/$HOSTNAME

echo -e "\nPurging backup folders older than 30 days..."
echo -e "backup
boot
etc
mbr" |
while read DirName
do
  find $DirName/* -type d ! -name . -prune -mtime +30 -exec rm -rf {} \;
done

echo -e "\nCreating new backup directories for $Today:"
echo -e "backup
boot
etc
mbr" |
while read DirName
do
  mkdir -v $DirName/$Today
done

echo -e "\nRotating files from $Yesterday/ to $Today/..."
# No rotation for /backup/ scripts or mbr (too small to worry about differential)
echo -e "boot
etc" |
while read DirName
do
  mv $DirName/$Yesterday/* $DirName/$Today/
done

echo -e "\nSaving backup scripts..."
cp /backup/* backup/$Today/
touch backup/$Today

echo -e "\nSaving Master Boot Record for sda & sdb..."
dd if=/dev/sda of=mbr/$Today/sda-mbr bs=512 count=1
dd if=/dev/sdb of=mbr/$Today/sdb-mbr bs=512 count=1
touch mbr/$Today

echo -e "\nBacking up /boot/ directory:"
rsync -azb --stats --delete --backup-dir boot/$Yesterday/ /boot/ boot/$Today/
touch boot/$Today

echo -e "\nBacking up /etc/ directory:"
rsync -azb --stats --delete --backup-dir etc/$Yesterday/ /etc/ etc/$Today/
touch etc/$Today

EndTime=`date`

echo -e "\n\nBackup of system files on $HOSTNAME completed at $EndTime."
echo "**********************************************************************"

Example: exclude.lst:

._NETWARE
~DFSINFO.8-P
VOLDATA.TDF
.Trash-root
Thumbs.db
thumbs.db
THUMBS.DB

Example: Primary Server Backup Script:

#!/bin/bash
# Version: 2008-02-27 03:00pm
# Modified By: Dan O.

StartTime=`date`
Today=`date +%F`
Yesterday=`date --date=yesterday +%F`
PurgeDate=`date --date='31 days ago' +%F`
BackupHost="backupservername"

echo "**********************************************************************"
echo "Starting backup from $HOSTNAME to $BackupHost at $StartTime..."

echo -e "\nMaking sure backup directory is mounted with 'mount -a'..."
mount -a
# Would rather do a check to make sure it's already mounted. If not, try to mount. If it fails, abort.

echo -e "\nChanging into backup directory '/backup/$HOSTNAME'..."
cd /backup/$HOSTNAME
# Need to add check to make sure this directory change was successful, or abort.

echo -e "\nPurging backup folders older than 30 days..."
echo -e "backup
boot
etc
GW
mbr
SHARED
USERS" |
while read DirName
do
#  rm -rf $DirName/$PurgeDate
#  find $DirName/* -type d ! -name . -prune -mtime +30 -exec echo {} \;
  find $DirName/* -type d ! -name . -prune -mtime +30 -exec rm -rf {} \;
done

# Creating a new new "$Today" directory, then moving $Yesterday/* $Today/
# keeps the timestamp on $Yesterday/, and makes the script better able to handle
# manual backups (no nested $Today/$Yesterday, which can happen on a restart)
echo -e "\nCreating new backup directories for $Today:"
echo -e "backup
boot
etc
GW
mbr
SHARED
USERS" |
while read DirName
do
  mkdir -v $DirName/$Today
done

echo -e "\nRotating files from $Yesterday/ to $Today/..."
# No rotation for /backup/ scripts or mbr (too small to worry about differential)
echo -e "boot
etc
GW
SHARED
USERS" |
while read DirName
do
  mv $DirName/$Yesterday/* $DirName/$Today/
done

echo -e "\nSaving backup scripts..."
cp /backup/* backup/$Today/
touch backup/$Today

echo -e "\nSaving Master Boot Record for sda & sdb..."
dd if=/dev/sda of=mbr/$Today/sda-mbr bs=512 count=1
dd if=/dev/sdb of=mbr/$Today/sdb-mbr bs=512 count=1
touch mbr/$Today

echo -e "\n\nBacking up APPS volume:"
rsync -az --stats --delete --exclude-from=/backup/exclude.lst /media/nss/APPS/ "$BackupHost"::"$HOSTNAME"_APPS/
touch APPS/

BackupOptions="-azb --stats --delete --exclude-from=/backup/exclude.lst --backup-dir /$Yesterday/"
echo -e "\nBackup options set to '$BackupOptions'..."

echo -e "\nBacking up /boot/ directory:"
rsync $BackupOptions /boot/ "$BackupHost"::"$HOSTNAME"_boot/$Today/
touch boot/$Today

echo -e "\nBacking up /etc/ directory:"
rsync $BackupOptions /etc/ "$BackupHost"::"$HOSTNAME"_etc/$Today/
touch etc/$Today

echo
/etc/init.d/grpwise stop
echo "Backing up GW volume:"
rsync $BackupOptions /media/nss/GW/ "$BackupHost"::"$HOSTNAME"_GW/$Today/
/etc/init.d/grpwise start
touch GW/$Today

echo -e "\n\nSaving Trustees for USERS volume..."
/opt/novell/nss/sbin/metamig save USERS -m t > /media/nss/USERS/trustees.xml
echo "Backing up USERS volume:"
rsync $BackupOptions /media/nss/USERS/ "$BackupHost"::"$HOSTNAME"_USERS/$Today/
touch USERS/$Today

echo -e "\n\nSaving Trustees for SHARED volume..."
/opt/novell/nss/sbin/metamig save SHARED -m t > /media/nss/SHARED/trustees.xml
echo "Backing up SHARED volume:"
rsync $BackupOptions /media/nss/SHARED/ "$BackupHost"::"$HOSTNAME"_SHARED/$Today/
touch SHARED/$Today

EndTime=`date`

echo -e "\n\nBackup of $HOSTNAME to $BackupHost completed at $EndTime."
echo "**********************************************************************"

Posted: March 7th, 2008 under Linux, Uncategorized.
Comments: none