Using Pbuilder for making Debian and/or Ubuntu Packages

Today’s post is for setting up an environment so that you can build Ubuntu or Debian packages in a “clean-room” environment with the help of pbuilder. We utilize this method to make clean packages for the LibreEMS suite (including Megatunix) and other things for various Debian/Ubuntu releases. Pbuilder will be used to make “clean” base images for each OS and architecture you are targeting, When it goes to build your package it’ll download and install whatever build-depends are defined in your package, and gather up the proper dependencies when it builds the resulting binaries. Its an excellent tool to make clean room builds and test out your package rules, to make sure you’re not missing important components.  It will report errors if you’ve made mistakes. See the manual pages on pbuilder for more information. We are leveraging this tool into out continuous integration system to do verified builds on committed changes.

Requirements:

  1. 64 Bit machine with 64 bit OS (in order to make 32 or 64 bit packages), a 32 bit machine can only make 32 bit packages.
  2. apt-cacher-ng, which saves you from having to download packages multiple times
  3. pbuilder
  4. configuration files (detailed below)

Special Notes: pbuilder will require significant amounts of disk space under /var/cache, so make sure you have enough free space (5-10 GB or more)

Steps:

    1. apt-get install pbuilder debootstrap quilt
    2. mkdir /usr/lib/pbuilder/hooks
    3. mkdir /var/cache/pbuilder/repo
    4. chmod 777 /var/cache/pbuilder/repo
    5. (Optional) Create /usr/lib/pbuilder/hooks/C10shell with the following content
      #!/bin/sh
      # invoke shell if build fails.
      
      apt-get install -y --force-yes vim less bash
      cd /tmp/buildd/*/debian/..
      /bin/bash < /dev/tty > /dev/tty 2> /dev/tty
    6. (Optional) Make it executable with: chmod +x /usr/lib/pbuilder/hooks/C10shell
    7. Create /usr/lib/pbuilder/hooks/D05deps with the following contents:
      #!/bin/sh
      (cd /var/cache/pbuilder/repo; apt-ftparchive packages . > Packages && gzip -c Packages >Packages.gz)
      apt-get update
    8. Make it executable with: chmod +x /usr/lib/pbuilder/hooks/D05deps
    9. Create /etc/pbuilderrc with the following contents, :
      # this is your configuration file for pbuilder.
      # the file in /usr/share/pbuilder/pbuilderrc is the default template.
      # /etc/pbuilderrc is the one meant for editing.
      #
      # read pbuilderrc.5 document for notes on specific options.
      
      BASETGZ=/var/cache/pbuilder/base.tgz
      BUILDPLACE=/var/cache/pbuilder/build/
      OTHERMIRROR="deb file:///var/cache/pbuilder/repo ./"
      USEPROC=yes
      USEDEVPTS=yes
      USEDEVFS=no
      BUILDRESULT=/var/cache/pbuilder/result/
      
      REMOVEPACKAGES="lilo"
      HOOKDIR="/usr/lib/pbuilder/hooks"
      # make debconf not interact with user
      export DEBIAN_FRONTEND="noninteractive"
      DEBEMAIL=""
      # for pbuilder debuild (sudo -E keeps the environment as-is)
      BUILDSOURCEROOTCMD="fakeroot"
      PBUILDERROOTCMD="sudo -E"
      
      # command to satisfy build-dependencies; the default is an internal shell
      # implementation which is relatively slow; there are two alternate
      # implementations, the "experimental" implementation,
      # "pbuilder-satisfydepends-experimental", which might be useful to pull
      # packages from experimental or from repositories with a low APT Pin Priority,
      # and the "aptitude" implementation, which will resolve build-dependencies and
      # build-conflicts with aptitude which helps dealing with complex cases but does
      # not support unsigned APT repositories
      PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends"
      
      #Command-line option passed on to dpkg-buildpackage.
      DEBBUILDOPTS=""
      
      #APT configuration files directory
      APTCONFDIR=""
      # the username and ID used by pbuilder, inside chroot. Needs fakeroot, really
      BUILDUSERID=1234
      BUILDUSERNAME=pbuilder
      
      # BINDMOUNTS is a space separated list of things to mount
      # inside the chroot.
      BINDMOUNTS="/var/cache/pbuilder/repo"
      
      # Set the debootstrap variant to 'buildd' type.
      # DEBOOTSTRAPOPTS[0]='--variant=buildd'
      # or work around bug in debootstrap 3.0.0 (314858)
      unset DEBOOTSTRAPOPTS
      
      # Set the PATH I am going to use inside pbuilder: default is "/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
      export PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin"
      
      # SHELL variable is used inside pbuilder by commands like 'su'; and they need sane values
      export SHELL=/bin/bash
      
      # The name of debootstrap command.
      DEBOOTSTRAP="debootstrap"
      
      # default file extension for pkgname-logfile
      PKGNAME_LOGFILE_EXTENTION="_$(dpkg --print-architecture).build"
      
      # default PKGNAME_LOGFILE
      PKGNAME_LOGFILE=""
    10. Create your personal ~/.pbuilderrc as well with the following.   NOTE: if you have an apt proxy (squid-deb-proxy or apt-cacher-ng,  uncomment and adjust the first few lines abotu the proxy setup:
      # Proxy configuration
      # Uncomment the next line if using apt-cacher-ng on the local machine
      #export http_proxy=http://localhost:3142/
      # Uncomment the next line if using squid-deb-proxy on the local machine.
      #export http_proxy=http://localhost:8000/
      
      # Codenames for Debian suites according to their alias. Update these when needed.
      UNSTABLE_CODENAME="sid"
      TESTING_CODENAME="jessie"
      STABLE_CODENAME="wheezy"
      STABLE_BACKPORTS_SUITE="$STABLE_CODENAME-backports"
      
      # List of Debian suites.
      DEBIAN_SUITES=($UNSTABLE_CODENAME $TESTING_CODENAME $STABLE_CODENAME
      "unstable" "testing" "stable")
      
      # List of Ubuntu suites. Update these when needed.
      UBUNTU_SUITES=("saucy" "raring" "quantal" "precise" "trusty")
      
      # Mirrors to use. Update these to your preferred mirror.
      DEBIAN_MIRROR="ftp.us.debian.org"
      UBUNTU_MIRROR="mirrors.kernel.org"
      
      # Optionally use the changelog of a package to determine the suite to use if
      # none set.
      if [ -z "${DIST}" ] && [ -r "debian/changelog" ]; then
      DIST=$(dpkg-parsechangelog | awk '/^Distribution: / {print $2}')
      # Use the unstable suite for certain suite values.
      if $(echo "experimental UNRELEASED" | grep -q $DIST); then
      DIST="$UNSTABLE_CODENAME"
      fi
      fi
      
      # Optionally set a default distribution if none is used. Note that you can set
      # your own default (i.e. ${DIST:="unstable"}).
      : ${DIST:="$(lsb_release --short --codename)"}
      
      # Optionally change Debian release states in $DIST to their names.
      case "$DIST" in
      unstable)
      DIST="$UNSTABLE_CODENAME"
      ;;
      testing)
      DIST="$TESTING_CODENAME"
      ;;
      stable)
      DIST="$STABLE_CODENAME"
      ;;
      esac
      
      # Optionally set the architecture to the host architecture if none set. Note
      # that you can set your own default (i.e. ${ARCH:="i386"}).
      : ${ARCH:="$(dpkg --print-architecture)"}
      
      NAME="$DIST"
      if [ -n "${ARCH}" ]; then
      NAME="$NAME-$ARCH"
      DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
      fi
      BASETGZ="/var/cache/pbuilder/$NAME-base.tgz"
      # Optionally, set BASEPATH (and not BASETGZ) if using cowbuilder
      # BASEPATH="/var/cache/pbuilder/$NAME/base.cow/"
      DISTRIBUTION="$DIST"
      BUILDRESULT="/var/cache/pbuilder/$NAME/result/"
      APTCACHE="/var/cache/pbuilder/aptcache/$NAME/"
      BUILDPLACE="/var/cache/pbuilder/build/"
      
      if $(echo ${DEBIAN_SUITES[@]} | grep -q $DIST); then
      # Debian configuration
      MIRRORSITE="http://$DEBIAN_MIRROR/debian/"
      COMPONENTS="main contrib non-free"
      # This is for enabling backports for the Debian stable suite.
      #if $(echo "$STABLE_CODENAME stable" | grep -q $DIST); then
      # EXTRAPACKAGES="$EXTRAPACKAGES debian-backports-keyring"
      # OTHERMIRROR="$OTHERMIRROR | deb http://www.backports.org/debian $STABLE_BACKPORTS_SUITE $COMPONENTS"
      #fi
      elif $(echo ${UBUNTU_SUITES[@]} | grep -q $DIST); then
      # Ubuntu configuration
      MIRRORSITE="http://$UBUNTU_MIRROR/ubuntu/"
      COMPONENTS="main restricted universe multiverse"
      else
      echo "Unknown distribution: $DIST"
      exit 1
      fi
    11. Now we need to initialize the “Packages” file for the empty repo so we can work the first time:
      dpkg-scanpackages /var/cache/pbuilder/repo > /var/cache/pbuilder/repo/Packages
    12. (OPTIONAL)  If you have lots of RAM (more than 4 GB) putting the pbuilder “build” chroot on tmpfs will speed it up immensely.  so add the following to /etc/fstab (it should be all on one line starting with “tmpfs” and ending with the second zero.
      tmpfs   /var/cache/pbuilder/build       tmpfs   defaults,size=2400M 0 0
      and then mount it with "sudo mount /var/cache/pbuilder/build"
    13. Now we need to initialize the pbuilder images for each OS variant, run the following command (64 bit machines only), on a fast internet connection this takes about 30-60 minutes (longer with a slower internet connection or older workstation) with an empty apt proxy cache (either apt-cacher-ng or squid-deb-proxy), if the cache is already primed (you did this before), it takes about 10 minutes to re-make them all, esp if using the tmpfs trick in step 12:
      for arch in `echo i386 amd64` ; do for dist in `echo sid wheezy jessie saucy raring quantal precise trusty` ; do sudo DIST=${dist} ARCH=${arch} pbuilder --create --architecture ${arch} --distribution ${dist} ; done ; done |tee /tmp/baseimage_create.log

      For 32 bit machines run:

      for dist in `echo sid wheezy jessie saucy raring quantal precise trusty ` ; do sudo DIST=${dist} pbuilder --create --architecture i386 --distribution ${dist} ; done |tee /tmp/baseimage_create.log
    14. Now to build any debian source package that has a “debian” directory within it, run the following, making sure your in the dir that CONTAINS the “debian” directory (typically the top level of the package):
      pdebuild --architecture <i386|amd64> --buildresult /tmp --pbuilderroot "sudo DIST=<sid|wheezy|jessie|trusty|precise|quantal|raring|saucy> ARCH=<i386|amd64>"
    15. To UPDATE your distroot’s to current patches do the following:
      for arch in `echo i386 amd64` ; do for dist in `echo sid wheezy jessie saucy raring quantal precise trusty` ; do sudo DIST=${dist} ARCH=${arch} pbuilder --update ; done ; done |tee /tmp/baseimage_update.log
    16. Enjoy!

How to install MegaTunix on your Mac

Requirements:

The 13 step sequence to go from nothing to MegaTunix!:

  1. Download and Install Xcode from Apple, Note: Make sure the command line Xcode tools are installed as part of the Xcode installation process.
  2. Download and install MacPorts from the above URL.
  3. Open a terminal (Applications->Utilities->Terminal)  I suggest dragging this icon to your dock, as you’ll learn to appreciate the terminal in the long run.
  4. In the terminal run “sudo port -v install git-core gtk2 gtkglext libglade2″,  when/if it prompts you for a password, enter the password you normally login with.
  5. Go watch a movie, take a walk, drink your favorite beverage, etc.  (This takes a while depending on how fast your computer really is)
  6. When done, still within the terminal, cd to your home directory with “cd” followed by Enter/Return
  7. run “git clone git://git.libreems.org/libreems-suite/megatunix.git”
  8. run “cd megatunix”
  9. run “./autogen.sh –enable-debug”
  10. run “make”
  11. run “sudo make install”, Entering your login password if asked.
  12. run “sudo ldconfig”
  13. run “megatunix”

Post comments/questions/thoughts below.  If you get a crash while running MegaTunix,  consider posting a bug report on The MegaTunix Issue Tracker

Welcome…

Welcome to the MegaTunix blog,  This will be The Place to learn about all things related to MegaTunix and perhaps some other things as well.  Enjoy yourselves and be sure to tip your waiters and waitresses well.

Check back often for upcoming articles for how to install MegaTunix, what to do if it blows it’s noodles,  information to provide when submitting bugs and so on.

 

MegaTunix has a new and improved issue tracker

The MegaTunix author has implemented it’s own issue tracker now available at https://redmine.libreems.org/projects/megatunix  Any other trackers for MegaTunix are now out of date, and will no longer be utilized.  If you have an open issue that isn’t represented here on this tracker,  please re-submit it!