Fr

ee

Embedded Linux Projects Using Yocto Project Cookbook

This book begins with the installation of a professional embedded Yocto setup, then advises you on
best practices, and finally explains how to quickly get hands on with the Freescale ARM ecosystem and
community layer, using the affordable and open source Wandboard embedded board.

What this book will do

for you...
Optimize your Yocto setup to speed up

development and debug build issues

Embedded Linux Projects

Using Yocto Project Cookbook

The embedded Linux world is standardizing around Yocto Project as the best integration framework
to create reliable embedded Linux products. Yocto Project effectively shortens the time it takes to
develop and maintain an embedded Linux product, and it increases its reliability and robustness by
using proven and tested components.

Sa

pl
e

Introduce development workflows for the

U-Boot and the Linux kernel, including

debugging and optimization methodologies
Customize your root filesystem with both

already supported and new Yocto packages

Understand the open source licensing

requirements and how to comply

with them when cohabiting with
proprietary programs
Bring professional embedded Yocto products

to market in a timely manner

reducing the size of both the Linux kernel
and root filesystems

A selection of the most important tasks

and problems

Carefully organized instructions for solving

the problem efficiently

Clear explanations of what you did

Apply the solution to other situations

$ 49.99 US
32.99 UK

community experience distilled

P U B L I S H I N G

Quick answers to common problems

A straightforward and easy-to-follow format

Alex Gonzlez

Optimize your production systems by

Inside the Cookbook...

Over 70 hands-on recipes for professional embedded Linux

developers to optimize and boost their Yocto know-how
Foreword by Javier Viguera, Embedded Software Engineer

Prices do not include

local sales tax or VAT
where applicable

P U B L I S H I N G

Visit www.PacktPub.com for books, eBooks,

code, downloads, and PacktLib.

Embedded Linux Projects

Using Yocto Project Cookbook

Alex Gonzlez

In this package, you will find:

The author biography

A preview chapter from the book, Chapter 1 'The Build System'
A synopsis of the books content
More information on Embedded Linux Projects Using Yocto
Project Cookbook

About the Author

Alex Gonzlez is software engineering supervisor at Digi International and one of the
maintainers of the Digi Embedded Yocto distribution.
He started working professionally with embedded systems in 1999 and the Linux kernel
in 2004, designing products for voice and video over IP networks, and followed his
interests into machine-to-machine (M2M) technologies and the Internet of Things.
Born and raised in Bilbao, Spain, Alex has his second home in the UK, where he lived for
over 10 years and received his MSc in communication systems from the University of
Portsmouth. He currently lives in La Rioja, where he enjoys photography and a good
Riojan wine.

Embedded Linux Projects Using

Yocto Project Cookbook
The Linux kernel is at the heart of a large number of embedded products being designed
today. Over the last 10 years, this operating system has developed from dominating the
server market to being the most used operating system in embedded systems, even those
with real-time requirements. On the way, Linux has evolved, and the embedded industry
has realized it has some key and unique characteristics:

Linux is quick to adapt to new technologies and it's the place where innovation
happens first
It is robust, and the development community is quick to react to problems
It is secure, and vulnerabilities are discovered and dealt with in a much quicker
way than in competing proprietary products
It is open, which means your company is able to own, modify, and understand
the technology
Finally, Linux is free

All of these make it a very compelling choice for embedded development.

But at the same time, an embedded Linux product is not only the Linux kernel.
Companies need to build an embedded system over the operating system, and that's
where embedded Linux was finding it difficult to make its placeuntil Yocto arrived.
The Yocto Project brings all the benefits of Linux into the development of embedded
systems. It provides a standard build system that allows you to develop embedded
products in a quick, reliable, and controlled way. Just as Linux has its strong points for
embedded development, Yocto has its own too:

Yocto is secure, as it uses recent sources and provides the means to quickly apply
security vulnerabilities to your products
It is robust, as it is used by a large community, which is quick to react to
problems
It is open, so your company can own the technology, understand it, and make it
fit for specific needs
It is free

With the Yocto Project's 6-month stable release process, package management updates,
and flexibility, you will be able to focus on your embedded application, knowing that you
are building it on top of a trusted system. You will speed up your development cycles and
produce outstanding products.

But Yocto is a new technology, and developers need to adapt to it. This books aims to
provide a practical guide for readers with basic knowledge of Linux and Yocto to develop
a production-ready industrial system based on the ARM architecture.

What This Book Covers

Chapter 1, The Build System, describes the use of the Poky build system and extends it
with the Freescale BSP community layer. It also describes common build system
configurations and features used to optimize the building of target images.
Chapter 2, The BSP Layer, guides you through the customization of the BSP for your
own product. It then explains how to configure, modify, build, and debug the U-Boot
bootloader, the Linux kernel, and its device tree.
Chapter 3, The Software Layer, describes the process of creating a new software layer to
hold new applications, services, or modifications to existing packages, as well as
discussing a release process for license compliance.
Chapter 4, Application Development, starts with toolchains and the Application
Development Toolkit (ADT), and deals with application development in detail, including
development environments such as Eclipse and Qt creator.
Chapter 5, Debugging, Tracing, and Profiling, discusses debugging tools and techniques,
and explores the tracing functionalities offered by the Linux kernel, along with some of
the user-space- tracing and profiling tools that make use of them.

The Build System

In this chapter, we will cover the following recipes:

Setting up the host system

Installing Poky

Creating a build directory

Building your first image

Explaining the Freescale Yocto ecosystem

Installing support for Freescale hardware

Building Wandboard images

Troubleshooting your Wandboard's first boot

Configuring network booting for a development setup

Sharing downloads

Sharing the shared state cache

Setting up a package feed

Using build history

Working with build statistics

Debugging the build system

The Build System

Introduction
The Yocto project (http://www.yoctoproject.org/) is an embedded Linux distribution
builder that makes use of several other open source projects.
The Yocto project provides a reference build system for embedded Linux, called Poky, which
has the BitBake and OpenEmbedded-Core (OE-Core) projects at its base. The purpose of
Poky is to build the components needed for an embedded Linux product, namely:

A bootloader image

A Linux kernel image

A root filesystem image

Toolchains and software development kits (SDKs) for application development

With these, the Yocto project covers the needs of both system and application developers.
When the Yocto project is used as an integration environment for bootloaders, the Linux
kernel, and user space applications, we refer to it as system development.
For application development, the Yocto project builds SDKs that enable the development of
applications independently of the Yocto build system.
The Yocto project makes a new release every six months. The latest release at the time of this
writing is Yocto 1.7.1 Dizzy, and all the examples in this book refer to the 1.7.1 release.
A Yocto release comprises the following components:

Poky, the reference build system

A build appliance; that is, a VMware image of a host system ready to use Yocto

An Application Development Toolkit (ADT) installer for your host system

And for the different supported platforms:

Prebuilt toolchains

Prebuilt packaged binaries

Prebuilt images

The Yocto 1.7.1 release is available to download from http://downloads.yoctoproject.

org/releases/yocto/yocto-1.7.1/.

Chapter 1

Setting up the host system

This recipe will explain how to set up a host Linux system to use the Yocto project.

Getting ready
The recommended way to develop an embedded Linux system is using a native Linux
workstation. Development work using virtual machines is discouraged, although they may
be used for demo and test purposes.
Yocto builds all the components mentioned before from scratch, including the cross-compilation
toolchain and the native tools it needs, so the Yocto build process is demanding in terms of
processing power and both hard drive space and I/O.
Although Yocto will work fine on machines with lower specifications, for professional
developer's workstations, it is recommended to use symmetric multiprocessing (SMP)
systems with 8 GB or more system memory and a high capacity, fast hard drive. Build servers
can employ distributed compilation, but this is out of the scope of this book. Due to different
bottlenecks in the build process, there does not seem to be much improvement above 8 CPUs
or around 16 GB RAM.
The first build will also download all the sources from the Internet, so a fast Internet
connection is also recommended.

How to do it...
Yocto supports several distributions, and each Yocto release will document a list of the
supported ones. Although the use of a supported Linux distribution is strongly advised, Yocto
is able to run on any Linux system if it has the following dependencies:

Git 1.7.8 or greater

Tar 1.24 or greater

Python 2.7.3 or greater (but not Python 3)

Yocto also provides a way to install the correct version of these tools by either downloading
a buildtools-tarball or building one on a supported machine. This allows virtually any Linux
distribution to be able to run Yocto, and also makes sure that it will be possible to replicate
your Yocto build system in the future. This is important for embedded products with long-term
availability requirements.
This book will use the Ubuntu 14.04 Long-Term Stable (LTS) Linux distribution for all
examples. Instructions to install on other Linux distributions can be found on the Supported
Linux Distributions section of the Yocto Project Development Manual, but the examples will
only be tested with Ubuntu 14.04 LTS.
3

The Build System

To make sure you have the required package dependencies installed for Yocto and to follow
the examples in the book, run the following command from your shell:
$ sudo apt-get install gawk wget git-core diffstat unzip texinfo gccmultilib build-essential chrpath socat libsdl1.2-dev xterm make
xsltproc docbook-utils fop dblatex xmlto autoconf automake libtool
libglib2.0-dev python-gtk2 bsdmainutils screen

Downloading the example code

You can download the example code files for all Packt books you have
purchased from your account at http://www.packtpub.com. If you
purchased this book elsewhere, you can visit http://www.packtpub.
com/support and register to have the files e-mailed directly to you.
The example code in the book can be accessed through several GitHub
repositories at https://github.com/yoctocookbook. Follow the
instructions on GitHub to obtain a copy of the source in your computer..

How it works...
The preceding command will use apt-get, the Advanced Packaging Tool (APT),
command-line tool. It is a frontend of the dpkg package manager that is included in the
Ubuntu distribution. It will install all the required packages and their dependencies to support
all the features of the Yocto project.

There's more...
If build times are an important factor for you, there are certain steps you can take when
preparing your disks to optimize them even further:

Place the build directories on their own disk partition or a fast external drive.

Use the ext4 filesystem but configure it not to use journalism on your Yocto-dedicated
partitions. Be aware that power losses may corrupt your build data.

Mount the filesystem in such a way that read times are not written/recorded on
file reads, disable write barriers, and delay committing filesystem changes with the
following mount options:
noatime,barrier=0,commit=6000.

Do not build on network-mounted drives.

These changes reduce the data integrity safeguards, but with the separation of the build
directories to their own disk, failures would only affect temporary build data, which can be
erased and regenerated.

Chapter 1

See also

The complete Yocto project installation instructions for Ubuntu and other supported
distributions can be found on the Yocto Project Reference Manual at http://www.
yoctoproject.org/docs/1.7.1/ref-manual/ref-manual.html

Installing Poky
This recipe will explain how to set up your host Linux system with Poky, the Yocto project
reference system.

Getting ready
Poky uses the OpenEmbedded build system, and as such, uses the BitBake tool, a task
scheduler written in Python which forked from Gentoo's Portage tool. You can think of BitBake
as the make utility in Yocto. It will parse the configuration and recipe metadata, schedule a
task list, and run through it.
BitBake is also the command-line interface to Yocto.
Poky and BitBake are two of the open source projects used by Yocto. The Poky project
is maintained by the Yocto community. You can download Poky from its Git repository at
http://git.yoctoproject.org/cgit/cgit.cgi/poky/.
Development discussions can be followed and contributed to by visiting the development
mailing list at https://lists.yoctoproject.org/listinfo/poky.
BitBake, on the other hand, is maintained by both the Yocto and OpenEmbedded
communities, as the tool is used by both. BitBake can be downloaded from its Git repository at
http://git.openembedded.org/bitbake/.
Development discussions can be followed and contributed to by visiting the development
mailing list at http://lists.openembedded.org/mailman/listinfo/bitbake-devel.
The Poky build system only supports virtualized QEMU machines for the following
architectures:

ARM (qemuarm)

x86 (qemux86)

x86-64 (qemux86-64)

PowerPC (qemuppc)

MIPS (qemumips, qemumips64)

The Build System

Apart from these, it also supports some reference hardware Board Support Packages
(BSPs), representative of the architectures just listed. These are those BSPs:

Texas Instruments Beaglebone (beaglebone)

Freescale MPC8315E-RDB (mpc8315e-rdb)

Intel x86 based PCs and devices (genericx86 and genericx86-64)

Ubiquiti Networks EdgeRouter Lite (edgerouter)

To develop on different hardware, you will need to complement Poky with hardware-specific
Yocto layers. This will be covered later on.

How to do it...
The Poky project incorporates a stable BitBake release, so to get started with Yocto, we only
need to install Poky in our Linux host system.
Note that you can also install BitBake independently
through your distribution's package management system.
This is not recommended and can be a source of problems,
as BitBake needs to be compatible with the metadata used
in Yocto. If you have installed BitBake from your distribution,
please remove it.

The current Yocto release is 1.7.1, or Dizzy, so we will install that into our host system. We will
use the /opt/yocto folder as the installation path:
$ sudo install -o $(id -u) -g $(id -g) -d /opt/yocto
$ cd /opt/yocto
$ git clone --branch dizzy git://git.yoctoproject.org/poky

How it works...
The previous instructions will use Git (the source code management system command-line
tool) to clone the Poky repository, which includes BitBake, into a new poky directory on our
current path, and point it to the Dizzy stable branch.

Chapter 1

There's more...
Poky contains three metadata directories, meta, meta-yocto, and meta-yocto-bsp, as
well as a template metadata layer, meta-skeleton, that can be used as a base for new
layers. Poky's three metadata directories are explained here:

meta: This directory contains the OpenEmbedded-Core metadata, which supports

the ARM, x86, x86-64, PowerPC, MIPS, and MIPS64 architectures and the QEMU
emulated hardware. You can download it from its Git repository at http://git.
openembedded.org/openembedded-core/.

Development discussions can be followed and contributed to by visiting the

development mailing list at http://lists.openembedded.org/mailman/
listinfo/openembedded-core.

meta-yocto: This contains Poky's distribution-specific metadata.

meta-yocto-bsp: This contains metadata for the reference hardware boards.

See also

There is documentation about Git, the distributed version control system, at

http://git-scm.com/doc

Creating a build directory

Before building your first Yocto image, we need to create a build directory for it.
The build process, on a host system as outlined before, can take up to one hour and need
around 20 GB of hard drive space for a console-only image. A graphical image, like coreimage-sato, can take up to 4 hours for the build process and occupy around 50 GB of space.

How to do it...
The first thing we need to do is create a build directory for our project, where the build
output will be generated. Sometimes, the build directory may be referred to as the project
directory, but build directory is the appropriate Yocto term.
There is no right way to structure the build directories when you have multiple projects, but
a good practice is to have one build directory per architecture or machine type. They can
all share a common downloads folders, and even a shared state cache (this will be covered
later on), so keeping them separate won't affect the build performance, but it will allow you to
develop on multiple projects simultaneously.

The Build System

To create a build directory, we use the oe-init-build-env script provided by Poky. The
script needs to be sourced into your current shell, and it will set up your environment to use
the OpenEmbedded/Yocto build system, including adding the BitBake utility to your path.
You can specify a build directory to use or it will use build by default. We will use qemuarm
for this example.
$ cd /opt/yocto/poky
$ source oe-init-build-env qemuarm

The script will change to the specified directory.

As oe-init-build-env only configures the current
shell, you will need to source it on every new shell. But, if
you point the script to an existing build directory, it will set
up your environment but won't change any of your existing
configurations.

BitBake is designed with a client/server abstraction, so we can

also start a memory resident server and connect a client to it.
With this setup, loading cache and configuration information each
time is avoided, which saves some overhead. To run a memory
resident BitBake that will always be available, you can use the
oe-init-build-env-memres script as follows:
$ source oe-init-build-env-memres 12345 qemuarm

Here 12345 is the local port to be used.

Do not use both BitBake flavors simultaneously, as this can be a
source of problems.
You can then kill the memory resident BitBake by executing the
following command:
$ bitbake -m

How it works...
Both scripts call the scripts/oe-setup-builddir script inside the poky directory to
create the build directory.
On creation, the build directory contains a conf directory with the following three files:

bblayers.conf: This file lists the metadata layers to be considered for this project.

local.conf: This file contains the project-specific configuration variables. You can
set common configuration variables to different projects with a site.conf file, but

this is not created by default.

8

Chapter 1

templateconf.cfg: This file contains the directory that includes the template

configuration files used to create the project. By default it uses the one pointed to by
the templateconf file in your Poky installation directory, which is meta-yocto/
conf by default.
To start a build from scratch, that's all the build directory needs.
Erasing everything apart from these files will recreate your build from
scratch.
$ cd /opt/yocto/poky/qemuarm
$ rm -Rf tmp sstate-cache

There's more...
You can specify a different template configuration file to use when you create your build
directory using the TEMPLATECONF variable; for example:
$ TEMPLATECONF=meta-custom/config source oe-init-build-env <builddir>

The TEMPLATECONF variable needs to refer to a directory containing templates for both
local.conf and bblayer.conf, but named local.conf.sample and bblayers.
conf.sample.
For our purposes, we can use the unmodified default project configuration files.

Building your first image

Before building our first image, we need to decide what type of image we want to build.
This recipe will introduce some of the available Yocto images and provide instructions to
build a simple image.

Getting ready
Poky contains a set of default target images. You can list them by executing the following
commands:
$ cd /opt/yocto/poky
$ ls meta*/recipes*/images/*.bb

The Build System

A full description of the different images can be found on the Yocto Project Reference Manual.
Typically, these default images are used as a base and customized for your own project needs.
The most frequently used base default images are:

core-image-minimal: This is the smallest BusyBox-, sysvinit-, and udev-based

console-only image

core-image-full-cmdline: This is the BusyBox-based console-only image with

full hardware support and a more complete Linux system, including bash
core-image-lsb: This is a console-only image that is based on Linux Standard

Base compliance

core-image-x11: This is the basic X11 Windows-system-based image with a

graphical terminal

core-image-sato: This is the X11 Window-system-based image with a SATO theme

and a GNOME Mobile desktop environment

core-image-weston: This is a Wayland protocol and Weston reference compositor-

based image
You will also find images with the following suffixes:

dev: These images are suitable for development work, as they contain headers and
libraries.

sdk: These images include a complete SDK that can be used for development on
the target.

initramfs: This is an image that can be used for a RAM-based root filesystem,

which can optionally be embedded with the Linux kernel.

How to do it...
To build an image, we need to configure the MACHINE we are building it for and pass its name
to BitBake. For example, for the qemuarm machine, we would run the following:
$ cd /opt/yocto/poky/qemuarm
$ MACHINE=qemuarm bitbake core-image-minimal

Or we could export the MACHINE variable to the current shell environment with the following:
$ export MACHINE=qemuarm

But the preferred and persistent way to do it is to edit the conf/local.conf configuration
file to change the default machine to qemuarm:
- #MACHINE ?= "qemuarm"
+ MACHINE ?= "qemuarm"

10

Chapter 1
Then you can just execute the following:
$ bitbake core-image-minimal

How it works...
When you pass a target recipe to BitBake, it first parses the following configuration files:

conf/bblayers.conf: This file is used to find all the configured layers

conf/layer.conf: This file is used on each configured layer

meta/conf/bitbake.conf: This file is used for its own configuration

conf/local.conf: This file is used for any other configuration the user may have

for the current build

conf/machine/<machine>.conf: This file is the machine configuration; in our

case, this is qemuarm.conf

conf/distro/<distro>.conf: This file is the distribution policy; by default, this is

the poky.conf file

And then BitBake parses the target recipe that has been provided and its dependencies.
The outcome is a set of interdependent tasks that BitBake will then execute in order.

There's more...
Most developers won't be interested in keeping the whole build output for every package, so it
is recommended to configure your project to remove it with the following configuration in your
conf/local.conf file:
INHERIT += "rm_work"

But at the same time, configuring it for all packages means that you won't be able to develop
or debug them.
You can add a list of packages to exclude from cleaning by adding them to the RM_WORK_
EXCLUDE variable. For example, if you are going to do BSP work, a good setting might be:
RM_WORK_EXCLUDE += "linux-yocto u-boot"

Remember that you can use a custom template local.conf.sample configuration file in
your own layer to keep these configurations and apply them for all projects so that they can be
shared across all developers.
Once the build finishes, you can find the output images on the tmp/deploy/images/
qemuarm directory inside your build directory.

11

The Build System

By default, images are not erased from the deploy directory, but you can configure your
project to remove the previously built version of the same image by adding the following to
your conf/local.conf file:
RM_OLD_IMAGE = "1"

You can test run your images on the QEMU emulator by executing this:
$ runqemu qemuarm core-image-minimal

The runqemu script included in Poky's scripts directory is a launch wrapper around the
QEMU machine emulator to simplify its usage.

Explaining the Freescale Yocto ecosystem

As we saw, Poky metadata starts with the meta, meta-yocto, and meta-yocto-bsp layers,
and it can be expanded by using more layers.
An index of the available OpenEmbedded layers that are compatible with the Yocto project is
maintained at http://layers.openembedded.org/.
An embedded product's development usually starts with hardware evaluation using a
manufacturer's reference board design. Unless you are working with one of the reference
boards already supported by Poky, you will need to extend Poky to support your hardware.

Getting ready
The first thing to do is to select which base hardware your design is going to be based on. We
will use a board that is based on a Freescale i.MX6 System on Chip (SoC) as a starting point
for our embedded product design.
This recipe gives an overview of the support for Freescale hardware in the Yocto project.

How to do it...
The SoC manufacturer (in this case, Freescale) has a range of reference design boards for
purchase, as well as official Yocto-based software releases. Similarly, other manufacturers that
use Freescale's SoCs offer reference design boards and their own Yocto-based software releases.
Selecting the appropriate hardware to base your design on is one of the most important design
decisions for an embedded product. Depending on your product needs, you will decide to either:

Use a production-ready board, like a single-board computer (SBC)

Use a module and build your custom carrier board around it

Use Freescale's SoC directly and design your own board

12

Chapter 1
Most of the times, a production-ready board will not match the specific requirements of an
professional embedded system, and the process of designing a complete carrier board using
Freescale's SoC would be too time consuming. So, using an appropriate module that already
solves the most technically challenging design aspects is a common choice.
Some of the characteristics that are important to consider are:

Industrial temperature ranges

Power management

Long-term availability

Precertified wireless and Bluetooth (if applicable)

The Yocto community layers that support Freescale-based boards are called meta-fsl-arm
and meta-fsl-arm-extras. The selection of boards that are supported on meta-fslarm is limited to Freescale reference designs, which would be the starting point if you are
considering designing your own carrier board around Freescale's SoC. Boards from other
vendors are maintained on the meta-fsl-arm-extras layer.
There are other embedded manufacturers that use meta-fsl-arm, but they have
not integrated their boards in the meta-fsl-arm-extras community layer. These
manufacturers will keep their own BSP layers, which depend on meta-fsl-arm, with specific
support for their hardware. An example of this is Digi International and its ConnectCore 6
module, which is based on the i.MX6 SoC.

How it works...
To understand Freescale Yocto ecosystem, we need to start with the Freescale community
BSP, comprising the meta-fsl-arm layer with support for Freescale reference boards,
and its companion, meta-fsl-arm-extra, with support for boards from other vendors,
and its differences with the official Freescale Yocto releases that Freescale offers for their
reference designs.
There are some key differences between the community and Freescale Yocto releases:

Freescale releases are developed internally by Freescale without community

involvement and are used for BSP validation on Freescale reference boards.

Freescale releases go through an internal QA and validation test process, and they
are maintained by Freescale support.

Freescale releases for a specific platform reach a maturity point, after which they
are no longer worked on. At this point, all the development work has been integrated
into the community layer and the platforms are further maintained by the Freescale
BSP community.

Freescale Yocto releases are not Yocto compatible, while the community release is.
13

The Build System

Freescale's engineering works very closely with the Freescale BSP community to make sure
that all development in their official releases is integrated in the community layer in a reliable
and quick manner.
Usually, the best option is to use the Freescale BSP community release but stay with the
U-Boot and Linux kernel versions that were released as part of the manufacturer's stable
BSP release.
This effectively means that you get the latest updates to the Linux kernel and U-Boot from the
manufacturer while simultaneously getting the latest updates to the root filesystem from the
community, extending the lifetime of your product, and making sure you are up to date with
applications, bug fixes, and security updates.
This takes advantage of the manufacturer's QA process for the system components that
are closer to the hardware, and makes it possible to use the manufacturer's support
while simultaneously getting user space updates from the community. The Freescale BSP
community is also very responsive and active, so problems can usually be worked on with
them to benefit all parts.

There's more...
The Freescale BSP community extends Poky with the following layers:

meta-fsl-arm: This is the community layer that supports Freescale reference

designs. It has a dependency on OpenEmbedded-Core. Machines in this layer will
be maintained even after Freescale stops active development on them. You can
download meta-fsl-arm from its Git repository at http://git.yoctoproject.
org/cgit/cgit.cgi/meta-fsl-arm/.

Development discussions can be followed and contributed to by visiting the

development mailing list at https://lists.yoctoproject.org/listinfo/
meta-freescale.
The meta-fsl-arm layer pulls both the Linux kernel and the U-Boot source from
Freescale's repositories using the following links:

Freescale Linux kernel Git repository: http://git.freescale.com/

git/cgit.cgi/imx/linux-2.6-imx.git/

Freescale U-Boot Git repository: http://git.freescale.com/git/

cgit.cgi/imx/uboot-imx.git/

Other Linux kernel and U-Boot versions are available, but keeping the manufacturer's
supported version is recommended.

14

Chapter 1
The meta-fsl-arm layer includes Freescale's proprietary binaries to enable some
hardware features most notably its hardware graphics, multimedia, and encryption
capabilities. To make use of these capabilities, the end user needs to accept
Freescale's End-User License Agreement (EULA), which is included in the metafsl-arm layer. To accept the license, the following line needs to be added to the
project's conf/local.conf configuration file:
ACCEPT_FSL_EULA = "1"

meta-fsl-arm-extra: This layer adds support for other community-maintained

boards; for example, the Wandboard. To download the layer's content, you may visit

https://github.com/Freescale/meta-fsl-arm-extra/.
meta-fsl-demos: This layer adds a metadata layer for demonstration target
images. To download the layer's content, you may visit https://github.com/
Freescale/meta-fsl-demos.

Freescale uses another layer on top of the layers above for their official software releases:
meta-fsl-bsp-release.

meta-fsl-bsp-release: This is a Freescale-maintained layer that is used in the

official Freescale software releases. It contains modifications to both meta-fsl-arm
and meta-fsl-demos. It is not part of the community release.

See also

For more information, refer to the FSL community BSP release notes available at
http://freescale.github.io/doc/release-notes/1.7/

Installing support for Freescale hardware

In this recipe, we will install the community Freescale BSP Yocto release that adds support for
Freescale hardware to our Yocto installation.

Getting ready
With so many layers, manually cloning each of them and adding them to your project's
conf/bblayers.conf file is cumbersome. The community is using the repo tool developed
by Google for their community Android to ease the installation of Yocto.
To install repo in your host system, type in the following commands:
$ sudo curl http://commondatastorage.googleapis.com/git-repodownloads/repo > /usr/local/sbin/repo
$ sudo chmod a+x /usr/local/sbin/repo

The repo tool is a Python utility that parses an XML file, called manifest, with a list of Git
repositories. The repo tool is then used to manage those repositories as a whole.
15

The Build System

How to do it...
For example, we will use repo to download all the repositories listed in the previous recipe to
our host system. For that, we will point it to the Freescale community BSP manifest for the
Dizzy release:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<default sync-j="4" revision="master"/>
<remote fetch="git://git.yoctoproject.org" name="yocto"/>
<remote fetch="git://github.com/Freescale" name="freescale"/>
<remote fetch="git://git.openembedded.org" name="oe"/>
<project remote="yocto" revision="dizzy" name="poky"
path="sources/poky"/>
<project remote="yocto" revision="dizzy" name="meta-fsl-arm"
path="sources/meta-fsl-arm"/>
<project remote="oe" revision="dizzy" name="meta-openembedded"
path="sources/meta-openembedded"/>
<project remote="freescale" revision="dizzy" name="fslcommunity-bsp-base" path="sources/base">
<copyfile dest="README" src="README"/>
<copyfile dest="setup-environment" src="setupenvironment"/>
</project>
<project remote="freescale" revision="dizzy" name="meta-fsl-armextra" path="sources/meta-fsl-arm-extra"/>
<project remote="freescale" revision="dizzy" name="meta-fsldemos" path="sources/meta-fsl-demos"/>
<project remote="freescale" revision="dizzy"
name="Documentation" path="sources/Documentation"/>
</manifest>>

The manifest file shows all the installation paths and repository sources for the different
components that are going to be installed.

How it works...
The manifest file is a list of the different layers that are needed for the Freescale community
BSP release. We can now use repo to install it. Run the following:
$ mkdir /opt/yocto/fsl-community-bsp
$ cd /opt/yocto/fsl-community-bsp
$ repo init -u https://github.com/Freescale/fsl-community-bspplatform -b dizzy
$ repo sync
16

Chapter 1
You can optionally pass a -jN argument to sync if you have a multicore machine for
multithreaded operations; for example, you could pass repo sync -j8 in an 8-core
host system.

There's more...
To list the hardware boards supported by the different layers, we may run:
$ ls sources/meta-fsl*/conf/machine/*.conf

And to list the newly introduced target images, use the following:
$ ls sources/meta-fsl*/recipes*/images/*.bb

The community Freescale BSP release introduces the following new target images:

fsl-image-mfgtool-initramfs: This is a small, RAM-based initramfs image

used with the Freescale manufacturing tool

fsl-image-multimedia: This is a console-only image that includes the

gstreamer multimedia framework over the framebuffer, if applicable

fsl-image-multimedia-full: This is an extension of fsl-image-multimedia,

but extends the gstreamer multimedia framework to include all available plugins

fsl-image-machine-test: This is an extension on fsl-image-multimediafull for testing and benchmarking

qte-in-use-image: This is a graphical image that includes support for Qt4 over

the framebuffer

qt-in-use-image: This is a graphical image that includes support for Qt4 over the
X11 Windows system

See also

Instructions to use the repo tool, including using repo with proxy servers, can be
found in the Android documentation at https://source.android.com/source/
downloading.html

Building Wandboard images

Building images for one of the supported boards (for example, Wandboard Quad) follows the
same process we described earlier for the QEMU machines, with the exception of using the
setup-environment script, which is a wrapper around oe-init-build-env.

17

The Build System

How to do it...
To build an image for the wandboard-quad machine, use the following commands:
$ cd /opt/yocto/fsl-community-bsp
$ mkdir -p wandboard-quad
$ MACHINE=wandboard-quad source setup-environment wandboard-quad
$ bitbake core-image-minimal

The current version of the setup-environment script only

works if the build directory is under the installation folder; in
our case, /opt/yocto/fsl-community-bsp.

How it works...
The setup-environment script will create a build directory, set up the MACHINE variable,
and prompt you to accept the Freescale EULA as described earlier. Your conf/local.conf
configuration file will be updated both with the specified machine and the EULA acceptance
variable.
Remember that if you close your terminal session, you will
need to set up the environment again before being able to
use BitBake. You can safely rerun the setup-environment
script as seen previously, as it will not touch an existing conf/
local.conf file. Run the following:
$ cd /opt/yocto/fsl-community-bsp/
$ source setup-environment wandboard-quad

The resulting image, core-image-minimal.sdcard, which is created inside the build

directory, can be programmed into a microSD card, inserted into the primary slot in the
Wandboard CPU board, and booted using the following commands:
$ cd /opt/yocto/fsl-community-bsp/wandboardquad/tmp/deploy/images/wandboard-quad/
$ sudo dd if=core-image-minimal.sdcard of=/dev/sdN bs=1M && sync

Here, /dev/sdN corresponds to the device node assigned to the microSD card in your
host system.
Be careful when running the dd command, as it could harm
your machine. You need to be absolutely sure that the sdN
device corresponds to your microSD card and not a drive on
your development machine.
18

Chapter 1

See also

You can find more information regarding the repo tool on Android's documentation at
https://source.android.com/source/using-repo.html

Troubleshooting your Wandboard's first boot

If you have problems booting your image, follow this recipe to troubleshoot.

Getting ready
1. Without the microSD card inserted, plug in a microUSB-to-USB cable to the USB
OTG interface of your Wandboard. Check the lsusb utility on your Linux host to see
whether the Wandboard appears as follows:
Bus 002 Device 006: ID 15a2:0054 Freescale Semiconductor, Inc.
i.MX6Q SystemOnChip in RecoveryMode

If you don't see this, try a different power supply. It should be 5V, 10W.
2. Make sure you connect a NULL modem serial cable between the RS232 connector
in your Wandboard target and a serial port on your Linux host. Then open a terminal
program like minicom with the following:
$ minicom -D /dev/ttyS0 -b 115200

You will need to add your user to the dialout group, or try to run
the command as sudo. This should open a 115200 8N1 serial
connection. The serial device may vary in your Linux host. For
example, a USB-to-serial adapter may be detected as /dev/
ttyUSB0. Also, make sure both hardware and software flow
control are disabled.

How to do it...
1. Insert the microSD card image to the module slot, not the base board, as the latter
is only used for storage and not for booting, and power it. You should see the U-Boot
banner in the minicom session output.
2. If not, you may have a problem with the serial communication. By default, the
Ethernet interface in the FSL community BSP image is configured to request an
address by DHCP, so you can use that to connect to the target.
Make sure you have a DHCP server running on the test network where the target is.
19

The Build System

You can use a packet sniffer like Wireshark to capture a network trace on your Linux
host and filter packages like the bootp protocol. At the least, you should see some
broadcasts from your target, and if you use an Ethernet hub, you should also see the
DHCP replies.
Optionally, you can log in to your DHCP server and check the logs to see if a new IP
address has been assigned. If you see an IP address being assigned, you might want
to consider adding an SSH server, like Dropbear, to your core-image-minimal image
so that you can establish a network connection with the target. You can do this by
adding the following line to the conf/local.conf configuration file:
IMAGE_INSTALL_append = " dropbear"

Note the space after the initial quote.

After building and reprogramming, you can then start an SSH session to the
Wandboard from your Linux host with:
$ ssh root@<ip_address>

The connection should automatically log in without a password prompt.

3. Try to program the default microSD card images from http://www.wandboard.
org/index.php/downloads to make sure the hardware and your setup is valid.
4. Try to reprogram your microSD card. Make sure you are using the correct images for
your board (for example, do not mix dual and quad images). Also, try different cards
and card readers.
These steps will have your Wandboard start booting, and you should have some output in your
serial connection.

There's more...
If everything else fails, you can verify the position of the bootloader on your microSD card. You
can dump the contents of the first blocks of your microSD card with:
$ sudo dd if=/dev/sdN of=/tmp/sdcard.img count=10

You should see a U-Boot header at offset 0x400. That's the offset where the i.MX6 boot ROM
will be looking for the bootloader when bootstrapped to boot from the microSD interface. Use
the following commands:
$ head /tmp/sdcard.img | hexdump
0000400 00d1 4020 0000 1780 0000 0000 f42c 177f

You can recognize the U-Boot header by dumping the U-Boot image from your build. Run the
following commands:
$ head u-boot-wandboard-quad.imx | hexdump
0000000 00d1 4020 0000 1780 0000 0000 f42c 177f
20

Chapter 1

Configuring network booting for a

development setup
Most professional i.MX6 boards will have an internal embedded MMC (eMMC) flash memory,
and that would be the recommended way to boot firmware. The Wandboard is not really
a product meant for professional use, so it does not have one. But neither the eMMC nor
the microSD card are ideal for development work, as any system change would involve a
reprogramming of the firmware image.

Getting ready
The ideal setup for development work is to use both TFTP and NFS servers in your host system
and to only store the U-Boot bootloader in either the eMMC or a microSD card. With this setup,
the bootloader will fetch the Linux kernel from the TFTP server and the kernel will mount the
root filesystem from the NFS server. Changes to either the kernel or the root filesystem are
available without the need to reprogram. Only bootloader development work would need you
to reprogram the physical media.

Installing a TFTP server

If you are not already running a TFTP server, follow the next steps to install and configure a
TFTP server on your Ubuntu 14.04 host:
$ sudo apt-get install tftpd-hpa

The tftpd-hpa configuration file is installed in /etc/default/tftpd-hpa. By default, it

uses /var/lib/tftpboot as the root TFTP folder. Change the folder permissions to make
it accessible to all users using the following command:
$ sudo chmod 1777 /var/lib/tftpboot

Now copy the Linux kernel and device tree from your build directory as follows:
$ cd /opt/yocto/fsl-community-bsp/wandboardquad/tmp/deploy/images/wandboard-quad/
$ cp zImage-wandboard-quad.bin zImage-imx6q-wandboard.dtb
/var/lib/tftpboot

Installing an NFS server

If you are not already running an NFS server, follow the next steps to install and configure one
on your Ubuntu 14.04 host:
$ sudo apt-get install nfs-kernel-server

21

The Build System

We will use the /nfsroot directory as the root for the NFS server, so we will "untar" the
target's root filesystem from our Yocto build directory in there:
$ sudo mkdir /nfsroot
$ cd /nfsroot
$ sudo tar xvf /opt/yocto/fsl-community-bsp/wandboardquad/tmp/deploy/images/wandboard-quad/core-image-minimal-wandboardquad.tar.bz2

Next, we will configure the NFS server to export the /nfsroot folder:
/etc/exports:
/nfsroot/ *(rw,no_root_squash,async,no_subtree_check)

We will then restart the NFS server for the configuration changes to take effect:
$ sudo service nfs-kernel-server restart

How to do it...
Boot the Wandboard and stop at the U-Boot prompt by pressing any key on the serial console.
Then run through the following steps:
1. Get an IP address by DHCP:
> dhcp

Alternatively, you can configure a static IP address with:

> setenv ipaddr <static_ip>

2. Configure the IP address of your host system, where the TFTP and NFS servers have
been set up:
> setenv serverip <host_ip>

3. Configure the root filesystem mount:

> setenv nfsroot /nfsroot

4. Configure the Linux kernel and device tree filenames:

> setenv image zImage-wandboard-quad.bin
> setenv fdt_file zImage-imx6q-wandboard.dtb

5. If you have configured a static IP address, you need to disable DHCP on boot
by running:
> setenv ip_dyn no

22

Chapter 1
6. Save the U-Boot environment to the microSD card:
> saveenv

7.

Perform a network boot:

> run netboot

The Linux kernel and device tree will be fetched from the TFTP server, and the root filesystem
will be mounted by the kernel from the NFS share after getting a DHCP address from your
network (unless using static IP addresses).
You should be able to log in with the root user without a password prompt.

Sharing downloads
You will usually work on several projects simultaneously, probably for different hardware
platforms or different target images. In such cases, it is important to optimize the build times
by sharing downloads.

Getting ready
The build system runs a search for downloaded sources in a number of places:

It tries the local downloads folder.

It looks into the configured premirrors, which are usually local to your organization.

It then tries to fetch from the upstream source as configured in the package recipe.

Finally, it checks the configured mirrors. Mirrors are public alternate locations for
the source.

If a package source is not found in any of the these four, the package build will fail with an
error. Build warnings are also issued when upstream fetching fails and mirrors are tried, so
that the upstream problem can be looked at.
The Yocto project maintains a set of mirrors to isolate the build system from problems with
the upstream servers. However, when adding external layers, you could be adding support for
packages that are not in the Yocto project's mirror servers, or other configured mirrors, so it is
recommended that you keep a local premirror to avoid problems with source availability.
The default Poky setting for a new project is to store the downloaded package sources on the
current build directory. This is the first place the build system will run a search for source
downloads. This setting can be configured in your project's conf/local.conf file with the
DL_DIR configuration variable.

23

The Build System

How to do it...
To optimize the build time, it is recommended to keep a shared downloads directory
between all your projects. The setup-environment script of the meta-fsl-arm layer
changes the default DL_DIR to the fsl-community-bsp directory created by the repo tool.
With this setup, the downloads folder will already be shared between all the projects in your
host system. It is configured as:
DL_DIR ?= "${BSPDIR}/downloads/"

A more scalable setup (for instance, for teams that are remotely distributed) is to configure
a premirror. For example, adding the following to your conf/local.conf file:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "http://example.com/my-source-mirror"

A usual setup is to have a build server serve its downloads directory. The build server can be
configured to prepare tarballs of the Git directories to avoid having to perform Git operations
from upstream servers. This setting in your conf/local.conf file will affect the build
performance, but this is usually acceptable in a build server. Add the following:
BB_GENERATE_MIRROR_TARBALLS = "1"

An advantage of this setup is that the build server's downloads folder can also be backed up
to guarantee source availability for your products in the future. This is especially important in
embedded products with long-term availability requirements.
In order to test this setup, you may check to see whether a build is possible just by using the
premirrors with the following:
BB_FETCH_PREMIRRORONLY = "1"

This setting in your conf/local.conf file can also be distributed across the team with the
TEMPLATECONF variable during the project's creation.

Sharing the shared state cache

The Yocto project builds everything from source. When you create a new project, only the
configuration files are created. The build process then compiles everything from scratch,
including the cross-compilation toolchain and some native tools important for the build.
This process can take a long time, and the Yocto project implements a shared state cache
mechanism that is used for incremental builds with the aim to build only the strictly necessary
components for a given change.

24

Chapter 1
For this to work, the build system calculates a checksum of the given input data to a task. If
the input data changes, the task needs to be rebuilt. In simplistic terms, the build process
generates a run script for each task that can be checksummed and compared. It also keeps
track of a task's output, so that it can be reused.
A package recipe can modify the shared state caching to a task; for example, to always
force a rebuild by marking it as nostamp. A more in-depth explanation of the shared state
cache mechanism can be found in the Yocto Project Reference Manual at http://www.
yoctoproject.org/docs/1.7.1/ref-manual/ref-manual.html.

How to do it...
By default, the build system will use a shared state cache directory called sstate-cache on
your build directory to store the cached data. This can be changed with the SSTATE_DIR
configuration variable in your conf/local.conf file. The cached data is stored in directories
named with the first two characters of the hash. Inside, the filenames contain the whole task
checksum, so the cache validity can be ascertained just by looking at the filename. The build
process set scene tasks will evaluate the cached data and use it to accelerate the build if valid.
When you want to start a build from a clean state, you need to remove both the sstatecache directory and the tmp directory.
You can also instruct BitBake to ignore the shared state cache by using the --no-setscene
argument when running it.
It's a good practice to keep backups of clean shared state caches (for example, from a build
server), which can be used in case of shared state cache corruption.

There's more...
Sharing a shared state cache is possible; however, it needs to be approached with care. Not
all changes are detected by the shared state cache implementation, and when this happens,
some or all of the cache needs to be invalidated. This can cause problems when the state
cache is being shared.
The recommendation in this case depends on the use case. Developers working on Yocto
metadata should keep the shared state cache as default, separated per project.
However, validation and testing engineers, kernel and bootloader developers, and application
developers would probably benefit from a well-maintained shared state cache.
To configure an NFS share drive to be shared among the development team to speed up the
builds, you can add the following to your conf/local.conf configuration file:
SSTATE_MIRRORS ?= "\
file://.* file:///nfs/local/mount/sstate/PATH"
25

The Build System

The expression PATH in this example will get substituted by the build system with a directory
named with the hash's first two characters.

Setting up a package feed

An embedded system project seldom has the need to introduce changes to the Yocto build
system. Most of the time and effort is spent in application development, followed by a lesser
amount in maybe kernel and bootloader development.
As such, a whole system rebuild is probably done very few times. A new project is usually built
from a prebuilt shared state cache, and application development work only needs to be done
to perform full or incremental builds of a handful of packages.
Once the packages are built, they need to be installed on the target system for testing.
Emulated machines are fine for application development, but most hardware-related work
needs to be done on embedded hardware.

Getting ready
An option is to manually copy the build binaries to the target's root filesystem, either copying
it to the NFS share on the host system the target is mounting its root filesystem from (as
explained in the Configuring network booting for a development setup recipe earlier) or using
any other method like SCP, FTP, or even a microSD card.
This method is also used by IDEs like Eclipse when debugging an application you are working
on. However, this method does not scale well when you need to install several packages and
dependencies.
The next option would be to copy the packaged binaries (that is, the RPM, deb, or ipk
packages) to the target's filesystem and then use the target's package management system
to install them. For this to work, your target's filesystem needs to be built with package
management tools. Doing this is as easy as adding the package-management feature
to your root filesystem; for example, you may add the following line to your project's
conf/local.conf file:
EXTRA_IMAGE_FEATURES += "package-management"

So for an RPM package, you will copy it to the target and use the rpm or smart utilities to
install it. The smart package management tool is GPL licensed and can work with a variety
of package formats.

26

Chapter 1
However, the most optimized way to do this is to convert your host system package's output
directory into a package feed. For example, if you are using the default RPM package format,
you may convert tmp/deploy/rpm in your build directory into a package feed that your
target can use to update.
For this to work, you need to configure an HTTP server on your computer that serves
the packages.

Versioning packages
You also need to make sure that the generated packages are correctly versioned, and that
means updating the recipe revision, PR, with every change. It is possible to do this manually, but
the recommendedand compulsory way if you want to use package feedsis to use a PR server.
However, the PR server is not enabled by default. The packages generated without a PR
server are consistent with each other but offer no update guarantees for a system that is
already running.
The simplest PR server configuration is to run it locally on your host system. To do this, you
add the following to your conf/local.conf file:
PRSERV_HOST = "localhost:0"

With this setup, update coherency is guaranteed for your feed.

If you want to share your feed with other developers, or you are configuring a build server
or package server, you would run a single instance of the PR server by running the following
command:
$ bitbake-prserv --host <server_ip> --port <port> --start

And you will update the project's build configuration to use the centralized PR server, editing
conf/local.conf as follows:
PRSERV_HOST = "<server_ip>:<port>"

Also, if you are using a shared state cache as described before, all of the contributors to the
shared state cache need to use the same PR server.
Once the feed's integrity is guaranteed, we need to configure an HTTP server to serve the feed.

How to do it...
We will use lighttpd for this example, as it is lightweight and easy to configure. Follow
these steps:
1. Install the web server:
$ sudo apt-get install lighttpd
27

The Build System

2. By default, the document root specified in the /etc/lighttpd/lighttpd.conf
configuration file is /var/www/, so we only need a symlink to our package feed:
$ sudo mkdir /var/www/wandboard-quad
$ sudo ln -s /opt/yocto/fsl-community-bsp/wandboardquad/tmp/deploy/rpm /var/www/wandboard-quad/rpm

Next, reload the configuration as follows:

$ sudo service lighttpd reload

3. Refresh the package index. This needs to be done manually to update the package
feed after every build:
$ bitbake package-index

4. Then we need to configure our target filesystem with the new package feeds:
# smart channel --add all type=rpm-md \
baseurl=http://<server_ip>/wandboard-quad/rpm/all

# smart channel --add wandboard_quad type=rpm-md \

baseurl=http://<server_ip>/wandboard-quad/rpm/wandboard_quad

# smart channel --add cortexa9hf_vfp_neon type=rpm-md \

baseurl=http://<server_ip>/wandboardquad/rpm/cortexa9hf_vfp_neon

5. Once the setup is ready, we will be able to query and update packages from the
target's root filesystem with the following:
# smart update
# smart query <package_name>
# smart install <package_name>

To make this change persistent in the target's root filesystem, we can configure the package
feeds at compilation time by using the PACKAGE_FEED_URIS variable in conf/local.conf
as follows:
PACKAGE_FEED_URIS = "http://<server_ip>/wandboard-quad"

See also

More information and a user manual for the smart utility can be found at
https://labix.org/smart/

28

Chapter 1

Using build history

When maintaining software for an embedded product, you need a way to know what has
changed and how it is going to affect your product.
On a Yocto system, you may need to update a package revision (for instance, to fix a security
vulnerability), and you need to make sure what the implications of this change are; for
example, in terms of package dependencies and changes to the root filesystem.
Build history enables you to do just that, and we will explore it in this recipe.

How to do it...
To enable build history, add the following to your conf/local.conf file:
INHERIT += "buildhistory"

The following enables information gathering, including dependency graphs:

BUILDHISTORY_COMMIT = "1"

The preceding line of code enables the storage of build history in a local Git repository.
The Git repository location can be set by the BUILDHISTORY_DIR variable, which by default
is set to a buildhistory directory on your build directory.
By default, buildhistory tracks changes to packages, images, and SDKs. This is
configurable using the BUILDHISTORY_FEATURES variable. For example, to track only image
changes, add the following to your conf/local.conf:
BUILDHISTORY_FEATURES = "image"

It can also track specific files and copy them to the buildhistory directory. By default, this
includes only /etc/passwd and /etc/groups, but it can be used to track any important
files like security certificates. The files need to be added with the BUILDHISTORY_IMAGE_
FILES variable in your conf/local.conf file as follows:
BUILDHISTORY_IMAGE_FILES += "/path/to/file"

Build history will slow down the build, increase the build size, and may also grow the Git
directory to an unmanageable size. The recommendation is to enable it on a build server for
software releases, or in specific cases, such as when updating production software.

29

The Build System

How it works...
When enabled, it will keep a record of the changes to each package and image in the form of
a Git repository in a way that can be explored and analyzed.
For a package, it records the following information:

Package and recipe revision

Dependencies

Package size

Files

For an image, it records the following information:

Build configuration

Dependency graphs

A list of files that includes ownership and permissions

List of installed packages

And for an SDK, it records the following information:

SDK configuration

List of both host and target files, including ownership and permissions

Dependency graphs

A list of installed packages

Looking at the build history

Inspecting the Git directory with the build history can be done in several ways:

Using Git tools like gitk or git log.

Using the buildhistory-diff command-line tool, which displays the differences in a

human-readable format.

Using a Django-1.4-based web interface. You will need to import the build history data
to the application's database after every build. The details are available at http://
git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/tree/README.

30

Chapter 1

There's more...
To maintain the build history, it's important to optimize it and avoid it from growing over time.
Periodic backups of the build history and clean-ups of older data are important to keep the
build history repository at a manageable size.
Once the buildhistory directory has been backed up, the following process will trim it and
keep only the most recent history:
1. Copy your repository to a temporary RAM filesystem (tmpfs) to speed things up.
Check the output of the df -h command to see which directories are tmpfs
filesystems and how much space they have available, and use one. For example, in
Ubuntu, the /run/shm directory is available.
2. Add a graft point for a commit one month ago with no parents:
$ git rev-parse "HEAD@{1 month ago}" > .git/info/grafts

3. Make the graft point permanent:

$ git filter-branch

4. Clone a new repository to clean up the remaining Git objects:

$ git clone file://${tmpfs}/buildhistory buildhistory.new

5. Replace the old buildhistory directory with the new cleaned one:
$ rm -rf buildhistory
$ mv buildhistory.new buildhistory

Working with build statistics

The build system can collect build information per task and image. The data may be used to
identify areas of optimization of build times and bottlenecks, especially when new recipes are
added to the system. This recipe will explain how the build statistics work.

How to do it...
To enable the collection of statistics, your project needs to inherit the buildstats class
by adding it to USER_CLASSES in your conf/local.conf file. By default, the fslcommunity-bsp build project is configured to enable them.
USER_CLASSES ?= "buildstats"

You can configure the location of these statistics with the BUILDSTATS_BASE variable, and
by default it is set to the buildstats folder in the tmp directory under the build directory
(tmp/buildstats).
31

The Build System

The buildstats folder contains a folder per image with the build stats under a timestamp
folder. Under it will be a subdirectory per package in your built image, and a build_stats
file that contains:

Host system information

Root filesystem location and size
Build time
Average CPU usage
Disk statistics

How it works...
The accuracy of the data depends on the download directory, DL_DIR, and the shared state
cache directory, SSTATE_DIR, existing on the same partition or volume, so you may need to
configure them accordingly if you are planning to use the build data.
An example build-stats file looks like the following:
Host Info: Linux agonzal 3.13.0-35-generic #62-Ubuntu SMP Fri Aug
15 01:58:42 UTC 2014 x86_64 x86_64
Build Started: 1411486841.52
Uncompressed Rootfs size: 6.2M /opt/yocto/fsl-communitybsp/wandboard-quad/tmp/work/wandboard_quad-poky-linuxgnueabi/core-image-minimal/1.0-r0/rootfs
Elapsed time: 2878.26 seconds
CPU usage: 51.5%
EndIOinProgress: 0
EndReadsComp: 0
EndReadsMerged: 55289561
EndSectRead: 65147300
EndSectWrite: 250044353
EndTimeIO: 14415452
EndTimeReads: 10338443
EndTimeWrite: 750935284
EndWTimeIO: 816314180
EndWritesComp: 0
StartIOinProgress: 0
StartReadsComp: 0
StartReadsMerged: 52319544
StartSectRead: 59228240
StartSectWrite: 207536552
StartTimeIO: 13116200
StartTimeReads: 8831854
StartTimeWrite: 3861639688
StartWTimeIO: 3921064032
StartWritesComp: 0
32

Chapter 1
These disk statistics come from the Linux kernel disk I/O stats (https://www.kernel.org/
doc/Documentation/iostats.txt). The different elements are explained here:

ReadsComp: This is the total number of reads completed

ReadsMerged: This is the total number of adjacent reads merged

SectRead: This is the total number of sectors read

TimeReads: This is the total number of milliseconds spent reading

WritesComp: This is the total number of writes completed

SectWrite: This is the total number of sectors written

TimeWrite: This is the total number of milliseconds spent writing

IOinProgress: This is the total number of I/Os in progress when reading /proc/diskstats

TimeIO: This is the total number of milliseconds spent performing I/O

WTimeIO: This is the total number of weighted time while performing I/O

And inside each package, we have a list of tasks; for example, for ncurses-5.9-r15.1,
we have the following tasks:

do_compile

do_fetch

do_package

do_package_write_rpm

do_populate_lic

do_rm_work

do_configure

do_install

do_packagedata

do_patch

do_populate_sysroot

do_unpack

Each one of them contain, in the same format as earlier, the following:

Build time

CPU usage

Disk stats

33

The Build System

There's more...
You can also obtain a graphical representation of the data using the pybootchartgui.
py tool included in the Poky source. From your project's build folder, you can execute the
following command to obtain a bootchart.png graphic in /tmp:
$ ../sources/poky/scripts/pybootchartgui/pybootchartgui.py
tmp/buildstats/core-image-minimal-wandboard-quad/ -o /tmp

Debugging the build system

In the last recipe of this chapter, we will explore the different methods available to debug
problems with the build system and its metadata.

Getting ready
Let's first introduce some of the usual use cases on a debugging session.

Finding recipes
A good way to check whether a specific package is supported in your current layers is to
search for it as follows:
$ find -name "*busybox*"

This will recursively search all layers for the BusyBox pattern. You can limit the search to
recipes and append files by executing:
$ find -name "*busybox*.bb*"

Dumping BitBake's environment

When developing or debugging package or image recipes, it is very common to ask BitBake to
list its environment both globally and for a specific target, be it a package or image.
To dump the global environment and grep for a variable of interest (for example, DISTRO_
FEATURES), use the following command:
$ bitbake -e | grep -w DISTRO_FEATURES

Optionally, to locate the source directory for a specific package recipe like BusyBox, use the
following command:
$ bitbake -e busybox | grep ^S=

You could also execute the following command to locate the working directory for a package or
image recipe:
$ bitbake -e <target> | grep ^WORKDIR=
34

Chapter 1

Using the development shell

BitBake offers the devshell task to help developers. It is executed with the following
command:
$ bitbake -c devshell <target>

It will unpack and patch the source, and open a new terminal (it will autodetect your terminal
type or it can be set with OE_TERMINAL) in the target source directory, which has the
environment correctly setup.
While in a graphical environment, devshell opens a new terminal
or console window, but if we are working on a non-graphical
environment, like telnet or SSH, you may need to specify screen
as your terminal in your conf/local.conf configuration file
as follows:
OE_TERMINAL = "screen"

Inside the devshell, you can run development commands like configure and make or invoke
the cross-compiler directly (use the $CC environment variable, which has been set up already).

How to do it...
The starting point for debugging a package build error is the BitBake error message printed on
the build process. This will usually point us to the task that failed to build.
To list all the tasks available for a given recipe, with descriptions, we execute the following:
$ bitbake -c listtasks <target>

If you need to recreate the error, you can force a build with the following:
$ bitbake -f <target>

Or you can ask BitBake to force-run only a specific task using the following command:
$ bitbake -c compile -f <target>

Task log and run files

To debug the build errors, BitBake creates two types of useful files per shell task and stores
them in a temp folder in the working directory. Taking BusyBox as an example, we would
look into:
/opt/yocto/fsl-community-bsp/wandboard-quad/tmp/work/cortexa9hfvfp-neon-poky-linux-gnueabi/busybox/1.22.1-r32/temp

35

The Build System

And find a list of log and run files. The filename format is
log.do_<task>.<pid>

and run.do_<task>.<pid>.
But luckily, we also have symbolic links, without the pid part, that link to the latest version.
The log files will contain the output of the task, and that is usually the only information we
need to debug the problem. The run file contains the actual code executed by BitBake to
generate the log mentioned before. This is only needed when debugging complex build issues.
Python tasks, on the other hand, do not currently write files as described previously, although
it is planned to do so in the future. Python tasks execute internally and log information to
the terminal.

Adding logging to recipes

BitBake recipes accept either bash or Python code. Python logging is done through the bb
class and uses the standard logging Python library module. It has the following components:

bb.plain: This uses logger.plain. It can be used for debugging, but should not

be committed to the source.

bb.note: This uses logger.info.

bb.warn: This uses logger.warn.

bb.error: This uses logger.error.

bb.fatal: This uses logger.critical and exits BitBake.

bb.debug: This should be passed log level as the first argument and uses logger.
debug.

To print debug output from bash in our recipes, we need to use the logging class
by executing:
inherit logging

The logging class is inherited by default by all recipes containing base.bbclass, so

we don't usually have to inherit it explicitly. We will then have access to the following bash
functions, which will output to the log files (not to the console) in the temp directory inside the
working directory as described previously:

bbplain: This function outputs literally what's passed in. It can be used in debugging

but should not be committed to a recipe source.

bbnote: This function prints with the NOTE prefix.

bbwarn: This prints a non-fatal warning with the WARNING prefix.

bberror: This prints a non-fatal error with the ERROR prefix.

36

Chapter 1

bbfatal: This function halts the build and prints an error message as with bberror.

bbdebug: This function prints debug messages with log level passed as the first

argument. It is used with the following format:

bbdebug [123] "message"

The bash functions mentioned here do not log to the console

but only to the log files.

Looking at dependencies
You can ask BitBake to print the current and provided versions of packages with the
following command:
$ bitbake --show-versions

Another common debugging task is the removal of unwanted dependencies.

To see an overview of pulled-in dependencies, you can use BitBake's verbose output by
running this:
$ bitbake -v <target>

To analyze what dependencies are pulled in by a package, we can ask BitBake to create DOT
files that describe these dependencies by running the following:
$ bitbake -g <target>

The DOT format is a text description language for graphics that is understood by the GraphViz
open source package and all the utilities that use it. DOT files can be visualized or further
processed.
You can omit dependencies from the graph to produce more readable output. For example, to
omit dependencies from glibc, you would run the following command:
$ bitbake -g <target> -I glibc

Once the preceding commands have been run, we get three files in the current directory:

package-depends.dot: This file shows the dependencies between runtime targets

pn-depends.dot: This file shows the dependencies between recipes

task-depends.dot: This file shows the dependencies between tasks

There is also a pn-buildlist file with a list of packages that would be built by the
given target.

37

The Build System

To convert the .dot files to postscript files (.ps), you may execute:
$ dot -Tps filename.dot -o outfile.ps

However, the most useful way to display dependency data is to ask BitBake to display it
graphically with the dependency explorer, as follows:
$ bitbake -g -u depexp <target>

The result may be seen in the following screenshot:

Debugging BitBake
It is not common to have to debug BitBake itself, but you may find a bug in BitBake and want
to explore it by yourself before reporting it to the BitBake community. For such cases, you
can ask BitBake to output the debug information at three different levels with the -D flag. To
display all the debug information, run the following command:
$ bitbake -DDD <target>

38

Chapter 1

Error reporting tool

Sometimes, you will find a build error on a Yocto recipe that you have not modified. The first
place to check for errors is the community itself, but before launching your mail client, head
to http://errors.yoctoproject.org.
This is a central database of user-reported errors. Here, you may check whether someone else
is experiencing the same problem.
You can submit your own build failure to the database to help the community debug the
problem. To do so, you may use the report-error class. Add the following to your
conf/local.conf file:
INHERIT += "report-error"

By default, the error information is stored under tmp/log/error-report under the build
directory, but you can set a specific location with the ERR_REPORT_DIR variable.
When the error reporting tool is activated, a build error will be captured in a file in the errorreport folder. The build output will also print a command to send the error log to the server:
$ send-error-report ${LOG_DIR}/error-report/error-report_${TSTAMP}

When this command is executed, it will report back with a link to the upstream error.
You can set up a local error server, and use that instead by passing a server argument. The
error server code and setting up details can be found at http://git.yoctoproject.org/
cgit/cgit.cgi/error-report-web/tree/README.

There's more...
Although you can use Linux utilities to parse Yocto's metadata and build output, BitBake lacks
a command base UI for common tasks. One project that aims to provide it is bb, which is
available at https://github.com/kergoth/bb.
To use it, you need to clone the repository locally by executing the following command:
$ cd /opt/yocto/fsl-community-bsp/sources
$ git clone https://github.com/kergoth/bb.git

Then run the bb/bin/bb init command, which prompts you to add a bash command to
your ~/.bash_profile file.
You can either do that or execute it in your current shell as follows:
$ eval "$(/opt/yocto/fsl-community-bsp/sources/bb/bin/bb init -)"

39

The Build System

You will first need to set up your environment as usual:
$ cd /opt/yocto/fsl-community-bsp
$ source setup-environment wandboard-quad

Some of the commands only work with a populated work directory,

so you may need to remove the rm_work class if you want to use
bb.

Some of the tasks that are made easier by the bb utility are:

Exploring the contents of a package:

$ bb contents <target>

Searching for a pattern in the recipes:

$ bb search <pattern>

Displaying either the global BitBake environment or the environment for a specific
package and grepping for a specific variable:
$ bb show -r <recipe> <variable>

40

Get more information Embedded Linux Projects Using Yocto Project Cookbook

Where to buy this book

You can buy Embedded Linux Projects Using Yocto Project Cookbook from the Packt
Publishing website.
Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and most internet
book retailers.
Click here for ordering and shipping details.

www.PacktPub.com

Stay Connected: