Owl River Company

Your IP is: 38.107.191.104

Building Source RPM as non-root under CentOS^*

Some background

In the bad old days, when dinosaurs roamed the earth, *nix system admininstrators ('sysadmins') compiled the code they needed from random 'tarballs' -- compressed images of computer source code. This was needed for many reasons, but had the nice side effect of simplifying code portability, and support of varying vendors' hardware, back in the days of the 'Unix wars'.

The RPM Package Manager is one of several approaches to adding better tools to manage the use of the code tarballs may produce, for more systematic (and thus, accurate) system administration. It allows managing and solving at a high level, complex package interdependencies (shared libraries and such), to permit validation that a system has not been tampered with, and to make possible complex provisioning, and life-cycle build qualification, security, 'Change Control', and maintenance.

A preliminary note on 'why as non-root' and 'why in a minimal build environment'

One bad habit which novice sysadmins have is to do too much with the all-powerful 'root' account's rights and permissions. Without belaboring the point, it is very easy to unknowingly over-write or change a file which causes remote, or non-obvious negative side effects. Building source code, as is part of the process of compiling tarballs is one of those acts which potentially has such negative side effects. This is because the instructions for building a given tarball may silently and invisibly change a shared library, and cause much damage to a system.

A second bad habit and practice is to want to make an installation an 'Everything install' so that it is "easier to build', as one does not have to pay attention to retrieving the needed elements for the build environment. This has subtler, but just as potentially damaging effects.

Consider the following example, made with a package from a reputable packager, and a reputable archive. We have marked the potential problem in red:
[herrold@new spamassassin]$ rpmbuild --rebuild perl-Mail-SPF-2.004-1.rf.src.rpm Installing perl-Mail-SPF-2.004-1.rf.src.rpm warning: user dag does not exist - using root warning: group dag does not exist - using root warning: user dag does not exist - using root warning: group dag does not exist - using root Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.75053 + umask 022 + cd /home/herrold/rpmbuild/BUILD + cd /home/herrold/rpmbuild/BUILD + rm -rf Mail-SPF-2.004 + /bin/gzip -dc /home/herrold/rpmbuild/SOURCES/Mail-SPF-2.004.tar.gz + tar -xvvf - drwxr-xr-x julian/users 0 2007-01-19 20:18:54 Mail-SPF-2.004/ -r--r--r-- julian/users 626 2007-01-19 20:18:50 Mail-SPF-2.004/INSTALL ... + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.8915 + umask 022 + cd /home/herrold/rpmbuild/BUILD + cd Mail-SPF-2.004 + /usr/bin/perl Makefile.PL INSTALLDIRS=vendor PREFIX=/var/tmp/perl-Mail-SPF-2.004-1.rf-root/usr This module requires Module::Build to install itself. Install Module::Build now from CPAN? [y] Can't locate CPAN.pm in @INC (@INC contains: _build/lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl //usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi ///usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at ///Makefile.PL line 16, <STDIN> line 1. error: Bad exit status from /var/tmp/rpm-tmp.8915 (%build) RPM build errors: user dag does not exist - using root group dag does not exist - using root user dag does not exist - using root group dag does not exist - using root Bad exit status from /var/tmp/rpm-tmp.8915 (%build) [herrold@new spamassassin]$
If the CPAN perl module had been present in this minimal build environment, AND if we had been building as root, we would have just installed a new collection of perl modules, wholly unkown to the package manager; and laid the trap for causing great confusion. To regain comfort that the box is being properly updated, we would probably need to wipe and reinstall the machine, to regain certainty as to the safety of the box' versions.

In our example we were using 'rpmbuild' as the build engone, but the exact same analysis applies with a more traditional unmanaged:

./configure && make && sudo make install

Because we were not building as root, and because we are building in a sparse build environment, no such damage occurred.

.. and applied to Source RPM building

Not unexpectedly, then, a tool to manage the use of the binary (executable) code produced by a tarball, also needs within its problem space, to handle creating, safely, the content. In recent years (it was not generally true in the early days of 'RPM' usage practice), distribution vendors, and the independent packagers have come up with practice and packaging solutions which have essentially completely done away with the need to build packages (from source RPMs) as 'root'. In doing so, it also 'designs out' many of the the possibilities for inadventently modifying the general underlying buildhost.

Other approaches, for those few cases where it may be necessary to build something as root, there are techniques, such as 'chroot' builds in a defined build environment, and tools, to detect (basically by comparing the pre-, and post- build images) just what a given 'tarball' has done during its build.

So, let's see an example

We will set up a build environment. We populate our build host with some commonly needed compiler packages. We will test build a package. We will discuss some ways to make life easier for a builder. Some common failures occur during the build process -- missing build tools, strange messages, and the like, and we will mention some.

Set up a build box -- As one is getting started, nothing beats having a separate computer to make mistakes in. That way, if things get too messed up, or if one is experimenting on an approach which turns out to be a dead end, one can wipe and reinstall it. We discuss automated PXE based install techniques, setting up a local install (using loop mounted ISOs [install-from-iso.sh]) and update [TBD] mirror. Then, use of yum to add just the desired packages into a base install. A good base set, which will pull in many of the additional items a basic environment needs is (as root):
# yum -y install rpm-build make m4 gcc-c++ autoconf automake \ redhat-rpm-config

Of course, we need to set up an end user to build in. That account does not need to have 'sudo' rights, if we are willing to have two terminals open on the build host: one with root rights available to add any additional missing packages needed by the build, and the other to do the building in. Alternatively, adding the build account to /etc/sudoers and using root right thoughtfully is another approach.
We have published a simple little script (local copy) to set up a non-root user build environment under RPM. We can set up that environment (still as a non-root user) thus:
$ wget http://www.oldrpm.org/hintskinks/buildtree/RPM-build-tree.txt $ chmod 755 RPM-build-tree.txt $ ./RPM-build-tree.txt and in examining the results, we will find that the filesystem tree at: ~/rpmbuild/ matches root's tree at: /usr/src/redhat/

Additionally, the file: ~/.rpmmacros has been prepopulated with a '%_topdir' value. This value is a very important one, as it is the cornerstone of the RPM build macros, and is the base point for several other macros. You may see this thus:
[herrold@dhcp-233 ~]$ rpm --showrc | grep topdir -14: _builddir %{_topdir}/BUILD -14: _desktopdir %{_datadir}/applications -14: _rpmdir %{_topdir}/RPMS -14: _sourcedir %{_topdir}/SOURCES -14: _specdir %{_topdir}/SPECS -14: _srcrpmdir %{_topdir}/SRPMS -14: _topdir /home/herrold/rpmbuild [herrold@dhcp-233 ~]$

Keeping a clean build laboratory, and a 'lab notebook' -- We build many, many tarballs, for testing purposes and as an independent RPM packager. As a matter of good traceability practice, we have several conventions, permitting us to remember what we have used in solving a given package's build chain. We make a master directory: ~/build/, and then a sub directory for a given package: ~/build/{packagename}/. Within that directory, we copy a working copy of the 'SRPM' - the source RPM - so that we can find it later.

We also create a README file, to hold notes to ourself, as to the site from which we retrieved a given tarball of package, and any issues in the build which we need to remember (special build requirements, sequence of how to build a complete set of packages, and such). See our notes at: ftp://ftp.owlriver.com/pub/mirror/ORC/xfce4/README which describe the complex series of ordered dependencies for the XFCE4 X window manager series.
Let's build something -- As a simple example, we have a local SRPM copy of the 'joe' text editor at a permanent location (backup is here). Let's get a copy and build it as non root [we have numbered the lines, so that we may refer to specifics after we are done]: 1 [herrold@dhcp-233 ~]$ cd 2 [herrold@dhcp-233 ~]$ mkdir build 3 [herrold@dhcp-233 ~]$ cd build 4 [herrold@dhcp-233 build]$ mkdir joe 5 [herrold@dhcp-233 build]$ cd joe 6 [herrold@dhcp-233 joe]$ joe README 7 [herrold@dhcp-233 joe]$ wget ftp://ftp.owlriver.com/pub/mirror/ORC/joe/joe-2.9.8-4.src.rpm 8 [herrold@dhcp-233 joe]$ rpmbuild --rebuild joe-2.9.8-4.src.rpm In reviewing the commands above, we did this:
1. Move to the home directory
2. Make the ~/build/ subdirectory
3. Move into the ~/build/ subdirectory
4. Make the ~/build/joe/ subdirectory
5. Move into ~/build/joe/ subdirectory
6. Update our notes in the README
7. Retrieve the SRPM using wget
8. Build the binary RPM
When we inspect: ~/rpmbuild/RPMS/i386/ we find the binary file: joe-2.9.8-4.i386.rpm

Kick it up a notch -- We can let the *nix command line tools read just the parts of the many [450] lines of content whch a build produces, and look just for the indication of the fruit of the binary package build being written:
[herrold@dhcp-233 joe]$ rpmbuild --rebuild joe-2.9.8-4.src.rpm 2>&1 | wc 450 2689 20449 [herrold@dhcp-233 joe]$ rpmbuild --rebuild joe-2.9.8-4.src.rpm 2>&1 | \ grep ^Wrote Wrote: /home/herrold/rpmbuild/RPMS/i386/joe-2.9.8-4.i386.rpm [herrold@dhcp-233 joe]$

The proof of the pudding ... -- Part of the process of building is to test that a package built will install and be usable. We can uninstall the current version, and install the admittedly stale version of the joe editor at our test site. Note here that we have additionally added the end user account into the /etc/sudoers group of users permitted to perform root level operations, after a bit of thought and provision of their (non-root) password:
[herrold@dhcp-233 joe]$ rpm -q joe joe-3.1-6 [herrold@dhcp-233 joe]$ sudo rpm -e joe We trust you have received the usual lecture from the local System Administrator. It usually boils down to these two things: #1) Respect the privacy of others. #2) Think before you type. Password: [herrold@dhcp-233 joe]$ sudo rpm -Uvh /home/herrold/rpmbuild/RPMS/i386/joe-2.9.8-4.i386.rpm Preparing... ########################################### [100%] 1:joe ########################################### [100%] [herrold@dhcp-233 joe]$ rpm -q joe joe-2.9.8-4 [herrold@dhcp-233 joe]$ A couple of messages might appear; each is harmless, but one needs to be addressed before we can build. A tester found that he saw these:
Installing joe-2.9.8-4.src.rpm warning: InstallSourcePackage: Header V3 DSA signature: NOKEY, key ID 897da07a and later installing the resultant binary: error: Failed build dependencies: ncurses-devel is needed by joe-2.9.8-4.i386 The first message, containing the line with 'NOKEY' is a warning, saying that the package signing key used for that source RPM is not present in RPM's keyring collection. It is harmless, and may be ignored for the purposes of this training example. If one were in a distributor environment, or building code of unknown pedigree, it may well be that your release process requires that the key be know, or that the md5sum of tarballs in question be manually validated, or any included patches, or the .spec file be further vetted.

The second message, stating that a package (here, 'ncurses-devel') is needed for the binary package, seems very curious. 'rpm' does not enforce the presence of 'Requires' or 'BuildRequires' at the time of that it unpacks a .src.rpm; that task is done at build time by rpmbuild, which will decline to build, unless over-ridden a build time '--nodeps' option; later in installing as root, a binary RPM, missing Requires are enforced as well. [We note that our essay: "Just say 'No' to --nodeps" applies to installing binary RPM's, not to building source RPM's]. In any event, simply using 'yum' to install the missing package solves the issue:

	$ sudo yum -y install ncurses-devel

and the build, or install, may proceed.

In testing, joe shows its version at the bottom, and indeed, it is as expected:

IW .rpmmacros Row 1 Col 1 1:00 Ctrl-K H for help +----------------------------------------------------------------------------- | | %_topdir /home/herrold/rpmbuild | %make make | | [<snip>] | +----------------------------------------------------------------------------- ** Joe's Own Editor v2.9.8 ** Copyright (C) 2003 **

Then we clean up after ourselves:

[herrold@dhcp-233 ~]$ sudo yum -y upgrade Setting up Upgrade Process Setting up repositories local-proprietary 100% |=========================| 951 B 00:00 update 100% |=========================| 951 B 00:00 local-active-base 100% |=========================| 951 B 00:00 base 100% |=========================| 1.1 kB 00:00 addons 100% |=========================| 951 B 00:00 local-active-updates 100% |=========================| 951 B 00:00 extras 100% |=========================| 1.1 kB 00:00 Reading repository metadata in from local files Resolving Dependencies --> Populating transaction set with selected packages. Please wait. ---> Package joe.i386 0:3.1-6 set to be updated --> Running transaction check Dependencies Resolved ============================================================================= Package Arch Version Repository Size ============================================================================= Updating: joe i386 3.1-6 local-active-base 204 k Transaction Summary ============================================================================= Install 0 Package(s) Update 1 Package(s) Remove 0 Package(s) Total download size: 204 k Downloading Packages: Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Updating : joe ######################### [1/2] Cleanup : joe ######################### [2/2] Updated: joe.i386 0:3.1-6 Complete! [herrold@dhcp-233 ~]$

Note: We had a question asking why we used 'upgrade' rather than 'update' as the operator for 'yum'; this is an artifact of when the early drafts of this tip were written; there is no difference in the currently available versions of 'yum'; in prior versions, and in our sysadmin practice, 'upgrade' then had the favorable characteristic of removing outdated binary RPMs, and we got into the habit of its use.

Concluding remarks
As we noted at the start of this piece, one historical reason for building from sources was the fact that the resulting binary packages were well suited, with respect to the library versions used, to the build target *nix environment; by having a nice and portable SRPM, all of the advantages of a custom local binary are available on several architectures, and between several vendor's distributions from a single source package. The only price is that of setting up a sufficient build environment, and a little time. Porting and bootstrapping a toolset into a new environment is greatly simplified.

To some degree the naming conventions for a given package may vary between, say, a Netwinder Linux environment, and the latest bleeding edge release from Fedora or Novell. Sometimes, the names are wholly different; we have a series of 'dummy' packages, to address such matters as how /bin/sh is indicated as present on Solaris; or the renamings of build environment elements between libX11-devel and x.org; see: dummy-bin-sh, and dummy-libX11-devel, each of which exist so that 'rpmbuild' does not need the use of '--nodeps' to proceed. This facilitated hands off operation in a dependency solving autobuilder project we managed, in building the first, bootstrapping Linux distribution on a new processor architecture.

^* This writeup is done in the context of CentOS, but there is nothing peculiar to that Linux distribution rebuild effort. The techniques work from the earliest days of rpm, with minor changes. That is, it will work fine on: Red Hat Linux, Netwinder Linux, Alpha Linux, Aurora Linux, YellowDog Linux, cAos Linux, Red Hat Fedora and any other distribution which uses the RPM Package Manager.

Thanks to our reviewer, J. Norment, for corrections and suggestions for clarification.
See also:
Adding a patch to SRPM packaged content
Old RPM website
New RPM5 website

Please report any errors, unclarity, or suggestions for improvement to: info+non-root@owlriver.com . Please confirm that you are viewing the most recent copy of this site, and mention the revision date below.

-- Russ Herrold

Copyright (C) 2006 R P Herrold
      herrold@owlriver.com  NIC: RPH5 (US)
   My words are not deathless prose,
      but they are mine.

       Owl River Company
   "The World is Open to Linux (tm)"
   ... Open Source LINUX solutions ...
      info@owlriver.com
         Columbus, OH

rev 060413 RPH
http://www.owlriver.com/tips/non-root/

Up More Tips