Converting Red Hat Network channels to yum repos

It is possible to download the entire contents of Red Hat Network channels and convert them into yum repositories. This is useful if the number of Red Hat Enterprise Linux (RHEL) systems in your environment is greater than the number of your Red Hat Network subscriptions.

Before proceeding, please read the excellent article YUM: Setup and Usage, particularly the section up2date to yum.

In this scenario, we have four servers:

yum_server is the yum server.
rhel_4_server is a server subscribed to a RHEL 4 channel.
rhel_3_server is a server subscribed to a RHEL 3 channel.
rhel_2_1_server is a server subscribed to a RHEL 2.1 channel.

These steps assume that Apache is correctly configured on the yum server and that the RHEL servers are subscribed to the appropriate RHEL channels.

In the following steps, substitute your own directory names where appropriate.

RHEL 4 and RHEL 3 servers

On these servers, create a directory named /rhn/base, and copy the contents of /RedHat/RPMS from each of the RHEL CDs/ISOs into this directory.

Create a directory named /rhn/updates. This directory will be populated the first time you run get_rhn_updates below.

Download every package from the RHN channel that is not already in /rhn/base or /rhn/updates using get_rhn_updates. This script should be executed periodically from cron.

Contents of get_rhn_updates:

#!/bin/sh

# Retrieve packages from subscribed RHN channel that are not already present
# in $EXCLUDE_DIRS. If package is already present in $EXCLUDE_DIRS, up2date
# will create a symlink to it in storageDir (/var/spool/up2date by default).
# Remove these symlinks, and move updates packages to $UPDATES_DIR.

BASE_DIR="/rhn/base/"
UPDATES_DIR="/rhn/updates/"
EXCLUDE_DIRS="$BASE_DIR:$UPDATES_DIR"

# Remove any manually downloaded packages
rm -f /var/spool/up2date/*.rpm

up2date --showall | xargs up2date -k $EXCLUDE_DIRS --get

find /var/spool/up2date -type l -print0 | xargs -0 rm -f
find /var/spool/up2date -type f -iname '*.rpm' -exec mv {} $UPDATES_DIR \;

RHEL 2.1 servers

Because the up2date --get flag is not available in the RHEL 2.1 version of up2date, this is the most challenging, and dangerous, section.

On this server, create a directory named /rhn/base, and copy the contents of /RedHat/RPMS from each of the RHEL 2.1 CDs/ISOs into this directory.

Create a directory named /rhn/updates. This directory will be populated the first time you run get_rhn_updates below.

up2date will not allow you to download packages that are already installed and at the latest version, so we will create an empty but valid RPM database to allow us to download all available updates from the RHN channel.

By running up2date against this empty RPM database, we can download all files not already in /rhn/base (base packages) and /rhn/updates (package updates), similar to the --get flag used in later versions of up2date.

# mkdir /rhn/empty_rpm_database
# cp /var/lib/rpm/* /rhn/empty_rpm_database
# for RPM in `rpm -qa --dbpath /rhn/empty_rpm_database/` ; do rpm --dbpath /rhn/empty_rpm_database/ --justdb -e $RPM --nodeps ; done

Note: Running the above for loop without --dbpath or (especially) --justdb would be catastophic!

up2date needs the redhat-release package installed in order to determine the version of Red Hat running (more information), so install it (not the actual package, just an entry in the RPM database) into the heretofore empty database.

# rpm --dbpath /rhn/empty_rpm_database/ --justdb -ivh /rhn/base/redhat-release-as-2.1AS-20.i386.rpm --nodeps

up2date's --download command cannot contain package version or release information, so we have to modify the output of --showall to exclude this information; this step is not needed with up2date --get.

These steps do not have to be executed manually, but they will appear in the get_rhn_updates script for RHEL 2.1.

Example:
# up2date --showall | tail -n5
zebra-0.91a-11.21AS
zip-2.3-10.1
zlib-1.1.4-8.2.1AS
zlib-devel-1.1.4-8.2.1AS
zsh-4.0.2-2

Remove package version and release:
# up2date --showall | perl -ne 'print "$1\n" if /(.*?)-\d.*-.*/' | tail -n5
zebra
zip
zlib
zlib-devel
zsh

The kernel* packages are skipped with up2date's default configuration. This will create an unresolvable chain of dependencies when using the empty (except for redhat-release) RPM database with up2date.

To fix this, change pkgSkipList=kernel*; to pkgSkipList=; in /etc/sysconfig/rhn/up2date.

The following is the RHEL 2.1 version of get_rhn_updates. The script should be executed periodically from cron.

#!/bin/sh

# Retrieve packages from subscribed RHN channel that are not already present
# in $EXCLUDE_DIRS. If package is already present in $EXCLUDE_DIRS, up2date
# will create a symlink to it in storageDir (/var/spool/up2date by default).
# Remove these symlinks, and move updates packages to $UPDATES_DIR.

PATH=/bin:/usr/bin

BASE_DIR="/rhn/base"
UPDATES_DIR="/rhn/updates/"
EXCLUDE_DIRS="$BASE_DIR:$UPDATES_DIR"
RPM_DB_PATH="/rhn/empty_rpm_database/"

# Remove any manually downloaded packages
rm -f /var/spool/up2date/*.rpm

up2date --dbpath $RPM_DB_PATH --showall | perl -ne 'print "$1\n" if /(.*?)-\d.*-.*/' | xargs up2date --dbpath $RPM_DB_PATH -k $EXCLUDE_DIRS --download

# The kernel-* packages do not get downloaded with the above command
up2date --dbpath $RPM_DB_PATH -k $EXCLUDE_DIRS --download kernel-BOOT \
kernel-debug kernel-doc kernel-enterprise kernel-headers kernel-pcmcia-cs \
kernel-smp kernel-source kernel-summit kernel-utils

find /var/spool/up2date -type l -print0 | xargs -0 rm -f
find /var/spool/up2date -type f -iname '*.rpm' -exec mv {} $UPDATES_DIR \;

Bringing it together with rsync

At this point, the RHEL 4, 3, and 2.1 servers are retrieving the latest updates from their respective RHN channel using get_rhn_updates. We'll use rsync on the yum server to retrieve these updates that will be made available to the yum clients.

Before proceeding, please read the excellent article Using Rsync and SSH. A large part of the below process was obtained from that article.

On the yum server, create an rsync user. Become the rsync user, and create an ssh version 2 rsa key with no passphrase.

$ ssh-keygen -t rsa

On the RHEL 4, 3, and 2.1 servers, create an rsync user, and copy the contents of yum_server:~rsync/.ssh/id_rsa.pub into ~rsync/.ssh/authorized_keys.

On the same line as the key you just added to authorized_keys, prepend the following, assuming ~rsync is /home/rsync.

from="yum_server","command=/home/rsync/validate-rsync"

This will only allow public key ssh connections as the rsync user from yum_server and will only allow the command /home/rsync/validate-rsync to be executed using this public key. validate-rsync ensures that only rsync commands are executed using the rsync user's ssh public key; all other commands are rejected.

Create ~rsync/validate-rsync and make it executable by the rsync user.

#!/bin/sh

case "$SSH_ORIGINAL_COMMAND" in
   *\&*)
   echo "Rejected"
   ;;
   *\(*)
   echo "Rejected"
   ;;
   *\{*)
   echo "Rejected"
   ;;
   *\;*)
   echo "Rejected"
   ;;
   *\<*)
   echo "Rejected"
   ;;
   *\`*)
   echo "Rejected"
   ;;
   rsync\ --server*)
   $SSH_ORIGINAL_COMMAND
   ;;
   *)
   echo "Rejected"
   ;;
esac

On the yum server, make the appropriate yum updates repositories group-writable by the rsync user, and periodically transfer the updates.

# su rsync -c 'rsync -av rhel_4_server:/rhn/updates/* /yum/RHEL/4AS/updates/'
# createrepo -q /yum/RHEL/4AS/updates/
# yum-arch -q /yum/RHEL/4AS/updates/
# su rsync -c 'rsync -av rhel_3_server:/rhn/updates/* /yum/RHEL/3AS/updates/'
# yum-arch -q /yum/RHEL/3AS/updates/
# su rsync -c 'rsync -av rhel_2_1_server:/rhn/updates/* /yum/RHEL/2.1AS/updates/'
# yum-arch -q /yum/RHEL/2.1AS/updates/

Note: RHEL 4 and later supports versions of yum (2.4+) that use repodata instead of individual headers. createrepo populates repodata, and yum-arch populates headers.

The above commands should be placed in a shell script and periodically executed from cron, ideally shortly after the RHN-subscribed systems run get_rhn_updates.

Back to brandonhutchinson.com.