Buildroot

Buildroot usage and documentation by Thomas Petazzoni. Contributions from Karsten Kruse, Ned Ludd, Martin Herren and others.

$LastChangedDate$

About Buildroot

Buildroot is a set of Makefiles and patches that allow to easily generate both a cross-compilation toolchain and a root filesystem for your target. The cross-compilation toolchain uses uClibc (http://www.uclibc.org/), a tiny C standard library.

Buildroot is useful mainly for people working with embedded systems. Embedded systems often use processors that are not the regular x86 processors everyone is used to have on his PC. It can be PowerPC processors, MIPS processors, ARM processors, etc.

A compilation toolchain is the set of tools that allows to compile code for your system. It consists of a compiler (in our case, gcc), binary utils like assembler and linker (in our case, binutils) and a C standard library (for example GNU Libc, uClibc or dietlibc). The system installed on your development station certainly already has a compilation toolchain that you can use to compile application that runs on your system. If you're using a PC, your compilation toolchain runs on an x86 processor and generates code for a x86 processor. Under most Linux systems, the compilation toolchain uses the GNU libc as C standard library. This compilation toolchain is called the "host compilation toolchain", and more generally, the machine on which it is running, and on which you're working is called the "host system". The compilation toolchain is provided by your distribution, and Buildroot has nothing to do with it.

As said above, the compilation toolchain that comes with your system runs and generates code for the processor of your host system. As your embedded system has a different processor, you need a cross-compilation toolchain: it's a compilation toolchain that runs on your host system but that generates code for your target system (and target processor). For example, if your host system uses x86 and your target system uses ARM, the regular compilation toolchain of your host runs on x86 and generates code for x86, while the cross-compilation toolchain runs on x86 and generates code for ARM.

Even if your embedded system uses a x86 processor, you might interested in Buildroot, for two reasons:

You might wonder why such a tool is needed when you can compile gcc, binutils, uClibc and all the tools by hand. Of course, doing so is possible. But dealing with all configure options, with all problems of every gcc or binutils version it very time-consuming and uninteresting. Buildroot automates this process through the use of Makefiles, and has a collection of patches for each gcc and binutils version to make them work on most architectures.

Obtaining Buildroot

Buildroot is available as daily SVN snapshots or directly using SVN.

The latest snapshot is always available at http://buildroot.uclibc.org/downloads/snapshots/buildroot-snapshot.tar.bz2, and previous snapshots are also available at http://buildroot.uclibc.org/downloads/snapshots/.

To download Buildroot using SVN, you can simply follow the rules described on the "Accessing SVN"-page (http://buildroot.uclibc.org/subversion.html) of the uClibc buildroot website (http://buildroot.uclibc.org), and download the buildroot SVN module. For the impatient, here's a quick recipe:

 $ svn co svn://uclibc.org/trunk/buildroot

Using Buildroot

Buildroot has a nice configuration tool similar to the one you can find in the Linux Kernel (http://www.kernel.org/) or in Busybox (http://www.busybox.org/). Note that you can build everything as a normal user. There is no need to be root to configure and use Buildroot. The first step is to run the configuration assistant:

 $ make menuconfig

For each entry of the configuration tool, you can find associated help that describes the purpose of the entry.

One of the key configuration items is the PROJECT which determines where some board specific packages are built and where the results are stored.

Once everything is configured, the configuration tool has generated a .config file that contains the description of your configuration. It will be used by the Makefiles to do what's needed.

Let's go:

 $ make

This command will download, configure and compile all the selected tools, and finally generate a target filesystem. The target filesystem will be named root_fs_ARCH.EXT where ARCH is your architecture and EXT depends on the type of target filesystem selected in the Target options section of the configuration tool. The file is stored in the "binaries/$(PROJECT)/" directory

Creating your own board support

Once a package has been unpacked, it is possible to manually update configuration files. Buildroot can automatically save the configuration of buildroot, linux, busybox, uclibc and u-boot in "local/$(PROJECT) by using the command: 

 $ make saveconfig

Once a buildroot configuration has been created by saveconfig, the default "$(TOPDIR)/.config" file can be overridden by

 $ make BOARD=<project>

Buildroot will then use "local/<project>/<project>.config" instead of ".config".

If you want to modify your board, you can copy the project configuration file to ".config" by using the command:

 $ make BOARD=<project> getconfig

You can share your custom board support directory between several buildroot trees by setting the environment variable BUILDROOT_LOCAL to this directory,

Offline builds

If you intend to do an offline-build and just want to download all sources that you previously selected in "make menuconfig" then issue:

 $ make source

You can now disconnect or copy the content of your dl directory to the build-host.

Environment variables

Buildroot optionally honors some environment variables that are passed to make :

An example that uses config files located in the toplevel directory and in your $HOME:

$ make UCLIBC_CONFIG_FILE=uClibc.config BUSYBOX_CONFIG_FILE=$HOME/bb.config

If you want to use a compiler other than the default gcc or g++ for building helper-binaries on your host, then do

$ make HOSTCXX=g++-4.3-HEAD HOSTCC=gcc-4.3-HEAD

Using auto-completion

If you are lazy enough that you don't want to type the entire make menuconfig command line, you can enable auto-completion in your shell. Here is how you can do that using bash:

$ complete -W menuconfig make

Then just enter the beginning of the line, and ask bash to complete it for you by pressing the TAB key:

$ make me<TAB>

will result in bash to append nuconfig for you!

Alternatively, some distributions (of which Debian and Mandriva are but an example) have more powerful make completion. Depending on you distribution, you may have to install a package to enable completion. Under Mandriva, this is bash-completion, while Debian ships it as part of the bash package.

Other shells, such as zsh, also have completion facilities. See the documentation for your shell.

Customizing the target filesystem

There are a few ways to customize the resulting target filesystem:

Customizing the Busybox configuration

Busybox is very configurable, and you may want to customize it. You can follow these simple steps to do it. It's not an optimal way, but it's simple and it works.

  1. Make a first compilation of buildroot with busybox without trying to customize it.
  2. Invoke make busybox-menuconfig. The nice configuration tool appears and you can customize everything.
  3. Run the compilation of buildroot again.

Otherwise, you can simply change the package/busybox/busybox-<version>.config file if you know the options you want to change without using the configuration tool.

If you want to use an existing config file for busybox, then see section environment variables.

Customizing the uClibc configuration

Just like BusyBox, uClibc offers a lot of configuration options. They allow to select various functionalities, depending on your needs and limitations.

The easiest way to modify the configuration of uClibc is to follow these steps :

  1. Make a first compilation of buildroot without trying to customize uClibc.
  2. Invoke make uclibc-menuconfig. The nice configuration assistant, similar to the one used in the Linux Kernel or in Buildroot appears. Make your configuration as appropriate.
  3. Copy the .config file to toolchain/uClibc/uClibc.config or toolchain/uClibc/uClibc.config-locale. The former is used if you haven't selected locale support in Buildroot configuration, and the latter is used if you have selected locale support.
  4. Run the compilation of Buildroot again

Otherwise, you can simply change toolchain/uClibc/uClibc.config or toolchain/uClibc/uClibc.config-locale without running the configuration assistant.

If you want to use an existing config file for uclibc, then see section environment variables.

How Buildroot works

As said above, Buildroot is basically a set of Makefiles that download, configure and compiles software with the correct options. It also includes some patches for various software, mainly the ones involved in the cross-compilation tool chain (gcc, binutils and uClibc).

There is basically one Makefile per software, and they are named with the .mk extension. Makefiles are split into four sections:

Each directory contains at least 2 files :

The main Makefile do the job through the following steps (once the configuration is done):

  1. Create the download directory (dl/ by default). This is where the tarballs will be downloaded. It is interesting to know that the tarballs are in this directory because it may be useful to save them somewhere to avoid further downloads.
  2. Create the shared build directory (build_ARCH/ by default, where ARCH is your architecture). This is where all non configurable user-space tools will be compiled.When building two or more targets using the same architecture, the first build will go through the full download, configure, make process, but the second and later builds will only copy the result from the first build to its project specific target directory significantly speeding up the build process
  3. Create the project specific build directory (project_build_ARCH/$(PROJECT) by default, where ARCH is your architecture). This is where all configurable user-space tools will be compiled. The project specific build directory is neccessary, if two different targets needs to use a specific package, but the packages have different configuration for both targets. Some examples of packages built in this directory are busybox and linux.
  4. Create the project specific result directory (binaries/$(PROJECT) by default, where ARCH is your architecture). This is where the root filesystem images are stored, It is also used to store the linux kernel image and any utilities, boot-loaders etc. needed for a target.
  5. Create the toolchain build directory (toolchain_build_ARCH/ by default, where ARCH is your architecture). This is where the cross compilation toolchain will be compiled.
  6. Setup the staging directory (build_ARCH/staging_dir/ by default). This is where the cross-compilation toolchain will be installed. If you want to use the same cross-compilation toolchain for other purposes, such as compiling third-party applications, you can add build_ARCH/staging_dir/usr/bin to your PATH, and then use arch-linux-gcc to compile your application. In order to setup this staging directory, it first removes it, and then it creates various subdirectories and symlinks inside it.
  7. Create the target directory (project_build_ARCH/root/ by default) and the target filesystem skeleton. This directory will contain the final root filesystem. To setup it up, it first deletes it, then it uncompress the target/generic/skel.tar.gz file to create the main subdirectories and symlinks, copies the skeleton available in target/generic/target_skeleton and then removes useless .svn/ directories.
  8. Add the TARGETS dependency. This should generally check if the configuration option for this package is enabled, and if so then "subscribe" this package to be compiled by adding it to the TARGETS global variable.

Building several projects in the same buildroot source tree

BACKGROUND

Buildroot has always supported building several projects in the same tree if each project was for a different architecture.

The root file system has been created in the "build_<ARCH>/root" directory which is unique for each architecture. Toolchains have been built in "toolchain_build_<ARCH>".

It the user wanted to build several root file systems for the same architecture, a prefix or suffix could be added in the configuration file so the root file system would be built in "<PREFIX>_build_<ARCH>_<SUFFIX>/root" By supplying unique combinations of "<PREFIX>" and "<SUFFIX>" each project would get a unique root file system tree.

The disadvantage of this approach is that a new toolchain was built for each project, adding considerable time to the build process, even if it was two projects for the same chip.

This drawback has been somewhat lessened with gcc-4.x.y which allows buildroot to use an external toolchain. Certain packages requires special features in the toolchain, and if an external toolchain is selected, this may lack the neccessary features to complete the build of the root file system.

A bigger problem was that the "build_<ARCH>" tree was also duplicated, so each package would also be rebuilt once per project, resulting in even longer build times.

PROJECT TO SHARE TOOLCHAIN AND PACKAGE BUILDS

Work has started on a project which will allow the user to build multiple root file systems for the same architecture in the same tree. The toolchain and the package build directory will be shared, but each project will have a dedicated directory tree for project specific builds.

With this approach, most, if not all packages will be compiled when the first project is built. The process is almost identical to the original process. Packages are downloaded and extracted to the shared "build_<ARCH>/<package>" directory. They are configured and compiled.

Package libraries and headers are installed in the shared $(STAGING_DIR), and then the project specific root file system "$(TARGET_DIR)" is populated.

At the end of the build, the root file system will be used to generate the resulting root file system binaries.

Once the first project has been built, building other projects will typically involve populating the new project's root file system directory from the existing binaries generated in the shared "build_<ARCH>/<>" directory.

Only packages, not used by the first project, will have to go through the normal extract-configure-compile flow.

IMPLEMENTATION

The core of the solution is the introduction of two new directories:

Each of the directories contain one subdirectory per project. The name of the subdirectory is configured by the user in the normal buildroot configuration, using the value of:

Project Options ---> Project name

The configuration defines the $(PROJECT) variable.

The default project name is "uclibc".

"package/Makefile.in" defines: 

    PROJECT_BUILD_DIR:=project_build_$(ARCH)/$(PROJECT)
    BINARIES_DIR:=binaries/$(PROJECT)

It also defines the location for the target root file system: 

    TARGET_DIR:=$(PROJECT_BUILD_DIR)/$(PROJECT)/root

I.E: If the user has choosen "myproject" as the $(PROJECT) name:

will be created.

Currently, the root file system, busybox and an Atmel customized version of U-Boot, as well as some Atmel specific bootloaders like at91-bootstrap and dataflashboot.bin are built in "$(PROJECT_BUILD_DIR)"

The resulting binaries for all architectures are stored in the "$(BINARIES_DIR)" directory.

SUMMARY

The project will share directories which can be share without conflicts, but will use unique build directories, where the user can configure the build.

THINGS TO DO

  1. Linux

    2. The current Linux implementation is flawed. It only works if the user chooses to use one of the few kernels selected as base for the kernel-headers. While the Makefile seems to have hooks, allowing the developer to specify whatever version he/she wants in the target/device/*/* Makefiles, the build will fail if another kernel version is choosen.

    The reason for this is that the kernel patches are not applied by the "target/linux/linux.mk" build script fragment. They are only applied by the "toolchain/kernel-headers/*.makefile" build script fragments

    If the kernel-header version and the linux version differs, there will be two "linux-2.6.X.Y" directories in "build_<ARCH>/<>", each with its own set of patches.

    The solution in the works, is to move the build of Linux to "project_build_<ARCH>/<project name>/linux-2.6.X.Y" combined with method to configure which patches can be applied. Possibly, the linux source tree used to generate the kernel headers will be moved to the "toolchain_build_<ARCH>" directory

    The user will be able to select from three different Linux strategies:

    The current kernel patches can be configured to be applied to the linux source tree even if the version differs from the kernel header version.

    Since the user can select any kernel-patch he/she will be able to select a non-working combination. If the patch fails, the user will have to generate a new proprietary kernel-patch or decide to not apply the kernel patches

    Other optional patches will be board specific or architecture specific patches.

    There will also be a way for the user to supply absolute or relative paths to patches, possibly outside the main tree. This can be used to apply custom kernel-header-patches, if the versions available in buildroot cannot be applied to the specific linux version used

    Maybe, there will also be a possibility to supply an "URL" to a patch available on Internet.

  2. Configurable packages

    3. Many packages can, on top of the simple "enable/disable build", be further configured using Kconfig. Currently these packages will be compiled using the configuration specified in the ".config" file of the first project demanding the build of the package.

    If another project uses the same packages, but with a different configuration,these packages will not be rebuilt, and the root file system for the new project will be populated with files from the build of the first project

    If multiple project are built, and a specific package needs two different configuration, then the user must delete the package from the "build_<ARCH>" directory before rebuilding the new project.

    A long term solution is to edit the package makefile and move the build of the configurable packages from "build_<ARCH>" to "project_build_<ARCH>/<project name>" and send a patch to the buildroot mailing list.

  3. Naming conventions

    4. Names of resulting binaries should reflect the "project name"

  4. Generating File System binaries

    5. Packages which needs to be installed with the "root" as owner, will generate a ".fakeroot.<package>" file which will be used for the final build of the root file system binary.

    This was previously located in the "$(STAGING_DIR)" directory, but was recently moved to the "$(PROJECT_BUILD_DIR)" directory.

    Currently only three packages: "at", "ltp-testsuite" and "nfs-utils" requests fakeroot.

    The makefile fragments for each file system type like "ext2", "jffs2" or "squashfs" will, when the file system binary is generated, collect all present ".fakeroot.<package>" files to a single "_fakeroot.<file system>" file and call fakeroot.

    ".fakeroot.<package>" files are deleted as the last action of the Buildroot Makefile.

    It needs to be evaluated if any further action for the file system binary build is needed.

Using the uClibc toolchain

You may want to compile your own programs or other software that are not packaged in Buildroot. In order to do this, you can use the toolchain that was generated by Buildroot.

The toolchain generated by Buildroot by default is located in build_ARCH/staging_dir/. The simplest way to use it is to add build_ARCH/staging_dir/usr/bin/ to your PATH environnement variable, and then to use arch-linux-gcc, arch-linux-objdump, arch-linux-ld, etc.

For example, you may add the following to your .bashrc (considering you're building for the MIPS architecture and that Buildroot is located in ~/buildroot/) :

export PATH="$PATH:~/buildroot/build_mips/staging_dir/usr/bin/"

Then you can simply do :

mips-linux-gcc -o foo foo.c

Important : do not try to move a gcc-3.x toolchain to an other directory, it won't work. There are some hardcoded paths in the gcc configuration. If the default toolchain directory doesn't suit your needs, please refer to the Using the uClibc toolchain outside of buildroot section.

If you are using a current gcc-4.x, then use --sysroot and -isysroot since these toolchains have fully functional sysroot support. No hardcoded paths do exist in these configurations.

Using the uClibc toolchain outside of buildroot

By default, the cross-compilation toolchain is generated inside build_ARCH/staging_dir/. But sometimes, it may be useful to install it somewhere else, so that it can be used to compile other programs or by other users. Moving the build_ARCH/staging_dir/ directory elsewhere is not possible if using gcc-3.x, because there are some hardcoded paths in the toolchain configuration. This works, thanks to sysroot support, with current, stable gcc-4.x toolchains, of course.

If you want to use the generated gcc-3.x toolchain for other purposes, you can configure Buildroot to generate it elsewhere using the option of the configuration tool : Build options -> Toolchain and header file location, which defaults to $(BUILD_DIR)/staging_dir/.

Location of downloaded packages

It might be useful to know that the various tarballs that are downloaded by the Makefiles are all stored in the DL_DIR which by default is the dl directory. It's useful for example if you want to keep a complete version of Buildroot which is know to be working with the associated tarballs. This will allow you to regenerate the toolchain and the target filesystem with exactly the same versions.

If you maintain several buildroot trees, it might be better to have a shared download location. This can be accessed by creating a symbolic link from the dl directory to the shared download location.

I.E:

ln -s <shared download location> dl

Another way of accessing a shared download location is to create the BUILDROOT_DL_DIR environment variable. If this is set, then the value of DL_DIR in the project is overridden. The following line should be added to "~/.bashrc". 

export BUILDROOT_DL_DIR <shared download location>

Extending Buildroot with more software

This section will only consider the case in which you want to add user-space software.

Package directory

First of all, create a directory under the package directory for your software, for example foo.

Config.in file

Then, create a file named Config.in. This file will contain the portion of options description related to our foo software that will be used and displayed in the configuration tool. It should basically contain :

config BR2_PACKAGE_FOO
        bool "foo"
        help
	  This is a comment that explains what foo is.

	  http://foosoftware.org/foo/

Of course, you can add other options to configure particular things in your software.

The real Makefile

Finally, here's the hardest part. Create a file named foo.mk. It will contain the Makefile rules that are in charge of downloading, configuring, compiling and installing the software. Below is an example that we will comment afterwards. 

     1  #############################################################
     2  #
     3  # foo
     4  #
     5  #############################################################
     6  FOO_VERSION:=1.0
     7  FOO_SOURCE:=foo-$(FOO_VERSION).tar.gz
     8  FOO_SITE:=http://www.foosoftware.org/downloads
     9  FOO_DIR:=$(BUILD_DIR)/foo-$(FOO_VERSION)
    10  FOO_BINARY:=foo
    11  FOO_TARGET_BINARY:=usr/bin/foo
    12
    13  $(DL_DIR)/$(FOO_SOURCE):
    14          $(WGET) -P $(DL_DIR) $(FOO_SITE)/$(FOO_SOURCE)
    15
    16  $(FOO_DIR)/.source: $(DL_DIR)/$(FOO_SOURCE)
    17          $(ZCAT) $(DL_DIR)/$(FOO_SOURCE) | tar -C $(BUILD_DIR) $(TAR_OPTIONS) -
    18          touch $@
    19
    20  $(FOO_DIR)/.configured: $(FOO_DIR)/.source
    21          (cd $(FOO_DIR); rm -rf config.cache; \
    22                  $(TARGET_CONFIGURE_OPTS) \
    23                  $(TARGET_CONFIGURE_ARGS) \
    24                  ./configure \
    25                  --target=$(GNU_TARGET_NAME) \
    26                  --host=$(GNU_TARGET_NAME) \
    27                  --build=$(GNU_HOST_NAME) \
    28                  --prefix=/usr \
    29                  --sysconfdir=/etc \
    30          )
    31          touch $@
    32
    33  $(FOO_DIR)/$(FOO_BINARY): $(FOO_DIR)/.configured
    34          $(MAKE) CC=$(TARGET_CC) -C $(FOO_DIR)
    35
    36  $(TARGET_DIR)/$(FOO_TARGET_BINARY): $(FOO_DIR)/$(FOO_BINARY)
    37          $(MAKE) prefix=$(TARGET_DIR)/usr -C $(FOO_DIR) install
    38          rm -Rf $(TARGET_DIR)/usr/man
    39
    40  foo: uclibc ncurses $(TARGET_DIR)/$(FOO_TARGET_BINARY)
    41
    42  foo-source: $(DL_DIR)/$(FOO_SOURCE)
    43
    44  foo-clean:
    45          $(MAKE) prefix=$(TARGET_DIR)/usr -C $(FOO_DIR) uninstall
    46          -$(MAKE) -C $(FOO_DIR) clean
    47
    48  foo-dirclean:
    49          rm -rf $(FOO_DIR)
    50
    51 #############################################################
    52 #
    53 # Toplevel Makefile options
    54 #
    55 #############################################################
    56 ifeq ($(strip $(BR2_PACKAGE_FOO)),y)
    57 TARGETS+=foo
    58 endif

First of all, this Makefile example works for a single binary software. For other software such as libraries or more complex stuff with multiple binaries, it should be adapted. Look at the other *.mk files in the package directory.

At lines 6-11, a couple of useful variables are defined :

Lines 13-14 defines a target that downloads the tarball from the remote site to the download directory (DL_DIR).

Lines 16-18 defines a target and associated rules that uncompress the downloaded tarball. As you can see, this target depends on the tarball file, so that the previous target (line 13-14) is called before executing the rules of the current target. Uncompressing is followed by touching a hidden file to mark the software has having been uncompressed. This trick is used everywhere in Buildroot Makefile to split steps (download, uncompress, configure, compile, install) while still having correct dependencies.

Lines 20-31 defines a target and associated rules that configures the software. It depends on the previous target (the hidden .source file) so that we are sure the software has been uncompressed. In order to configure it, it basically runs the well-known ./configure script. As we may be doing cross-compilation, target, host and build arguments are given. The prefix is also set to /usr, not because the software will be installed in /usr on your host system, but in the target filesystem. Finally it creates a .configured file to mark the software as configured.

Lines 33-34 defines a target and a rule that compiles the software. This target will create the binary file in the compilation directory, and depends on the software being already configured (hence the reference to the .configured file). It basically runs make inside the source directory.

Lines 36-38 defines a target and associated rules that install the software inside the target filesystem. It depends on the binary file in the source directory, to make sure the software has been compiled. It uses the install target of the software Makefile by passing a prefix argument, so that the Makefile doesn't try to install the software inside host /usr but inside target /usr. After the installation, the /usr/man directory inside the target filesystem is removed to save space.

Line 40 defines the main target of the software, the one that will be eventually be used by the top level Makefile to download, compile, and then install this package. This target should first of all depends on all needed dependecies of the software (in our example, uclibc and ncurses), and also depend on the final binary. This last dependency will call all previous dependencies in the correct order.

Line 42 defines a simple target that only downloads the code source. This is not used during normal operation of Buildroot, but is needed if you intend to download all required sources at once for later offline build. Note that if you add a new package providing a foo-source target is mandatory to support users that wish to do offline-builds. Furthermore it eases checking if all package-sources are downloadable.

Lines 44-46 define a simple target to clean the software build by calling the Makefiles with the appropriate option. The -clean target should run make clean on $(BUILD_DIR)/package-version and MUST uninstall all files of the package from $(STAGING_DIR) and from $(TARGET_DIR).

Lines 48-49 define a simple target to completely remove the directory in which the software was uncompressed, configured and compiled. The -dirclean target MUST completely rm $(BUILD_DIR)/ package-version.

Lines 51-58 adds the target foo to the list of targets to be compiled by Buildroot by first checking if the configuration option for this package has been enabled using the configuration tool, and if so then "subscribes" this package to be compiled by adding it to the TARGETS global variable. The name added to the TARGETS global variable is the name of this package's target, as defined on line 40, which is used by Buildroot to download, compile, and then install this package.

Conclusion

As you can see, adding a software to buildroot is simply a matter of writing a Makefile using an already existing example and to modify it according to the compilation process of the software.

If you package software that might be useful for other persons, don't forget to send a patch to Buildroot developers !

Resources

To learn more about Buildroot you can visit these websites: