Deploying test images using LXC

Containers are lightweight virtualization technology. LXC is a userspace interface for the Linux kernel containment features. Through a powerful API and simple tools, it lets Linux users easily create and manage system or application containers. The container provides a lightweight method to allow custom software to be used on the dispatcher. The container is used to provide transparent access.

LAVA supports LXC containers both as a standalone device type and as dynamic transparent environments in order to interact with external devices. In either case the LXC protocol is used.

Using LXC as Device Type

LXC is a device type of its own and devices could be added to dispatchers under this device type. A device of LXC device type is created within the dispatcher in which the device is configured, as illustrated in the following figure:

LXC standalone

The LXC device type uses the LXC protocol in order to share data elements across different actions within the job.

Protocol elements

protocols:
  lava-lxc:
    name: lxc-device
    distribution: fedora
    release: '23'
    arch: amd64

 actions:
 - deploy:
   timeout:
     minutes: 5
   to: lxc
   os: fedora

Sample Job Definition

device_type: lxc
job_name: lxc-pipeline
timeouts:
  job:
    minutes: 15
  action:
    minutes: 5
priority: medium
visibility: public

metadata:
  source: https://git.linaro.org/lava-team/refactoring.git
  path: lxc-fedora.yaml

protocols:
  lava-lxc:
    name: pipeline-lxc-test
    distribution: fedora
    release: '23'
    arch: amd64

actions:
- deploy:
    timeout:
      minutes: 30
    to: lxc
    os: fedora

- boot:
    prompts:
    - '[root@(.*) /]#'
    timeout:
      minutes: 5
    method: lxc

- test:
    timeout:
      minutes: 5
    definitions:
    - repository: git://git.linaro.org/people/senthil.kumaran/test-definitions.git
      from: git
      path: fedora/echo-pass-fail-error-test.yaml
      name: echo-pass-fail-error-test

Using the LXC protocol to support Android

LAVA Android Naming Conventions

  • production image - a build of Android which, when deployed to a device, means that the device is not visible to adb. This is typically how a device is configured when first sold to the consumer.
  • developer image - a build of Android which, when deployed to a device, means that the device is visible to adb. Devices configured this way will be able to have the image replaced using any machine, just by connecting a suitable cable, so these images are not typically deployed onto hardware which will be sold to the customer without having this image replaced with a production image.

Introduction

Installing tools like adb and fastboot on the dispatcher can be problematic. Some of these issues arise from the need to put many different types of devices onto a single dispatcher, other issues arise from needing to use different versions of the build on the devices. Testing an old system may require downgrading support like openjdk, new devices or new builds may require upgrading the same support. Containers isolate this variation so that each testjob can have a suitable container instead of needing to deal with changes on the dispatcher:

  1. Shared lock issues - Tools can require use of flock and similar methods to distinguish a connection to one device from another.
  2. Version disparities - different device versions, different OS versions, may require different support in debug tools like adb and fastboot.
  3. hardware issues - USB hub variability.

See also

Deploying LXC devices for more information on the administration of LXC for LAVA.

LXC in LAVA

Using the lava-lxc protocol, a Lava Test Shell is provided inside the LXC to support installing and configuring whatever tools, packages and files which the testjob will need to use. Installing adb in this test shell removes the need to have a POSIX type shell on the device. Files can be pushed and pulled from the device and executed using the Android support in the image.

Requirements and Limitations

  1. The image deployed to the device must enable the Android Debug Bridge, i.e. a developer image. This means enabling developer access over USB or TCP. This rules out the use of production images.
  2. A list of packages to install into the bare container to provide the necessary tools to communicate with the device.
  3. The LXC depends on underlying kernel architecture. For armel, armhf, etc. dispatcher should run on these architectures.
  4. Each distro has its own template and the templates do not have common options. It can be difficult to have generic support for all distros.
  5. namespaces to relate different job actions to run in the LXC and for the device.

Namespaces

Namespaces were introduced to handle use-cases specific to LXC, but it can be expanded to other use-cases as and when required. The primary purpose of namespaces is to tie related actions together. In a typical job definition where more than one deploy, boot and test actions are specified there should be a mechanism to relate which deploy is connected with a boot and test action. This is important since an overlay created during a deploy action will be consumed by a test action somewhere down the job definition. A namespace comes into place to connect these actions together.

Protocol elements

protocols:
  lava-lxc:
    name: pipeline-lxc-test
    template: debian
    distribution: debian
    release: sid
    arch: amd64
    mirror: http://ftp.us.debian.org/debian/
    security_mirror: http://mirror.csclub.uwaterloo.ca/debian-security/

 actions:
 - deploy:
   namespace: tlxc
   timeout:
     minutes: 5
   to: lxc
   packages:
   - android-tools-adb
   - android-tools-fastboot
   - systemd
   - systemd-sysv
   os: debian

Differences between LXC releases

The release specified in the lava-lxc protocol will determine some of the packages which will need to be installed in the container. In particular, any container based on a Debian later than jessie will need two packages to be added to the setup of the container before the container can be used: systemd and systemd-sysv. These two packages must be specified in the deployment list.

In addition, some packages will have been renamed between releases. For example, android-tools-adb exists in Debian unstable but it is an old build and will at some point be replaced by adb which is also available in unstable but not in jessie.

Caution

Always check the availability of the packages needed for particular releases by using a local chroot or VM. Only packages which are included in the specified release can be installed using the deployment list. Packages from other repositories will have to be installed using the test definition.

Sample Job Definition

device_type: hi6220-hikey
job_name: lxc-hi6220-hikey
timeouts:
  job:
    minutes: 60
  action:
    minutes: 15
  connection:
    minutes: 2
priority: medium
visibility: public

metadata:
  source: https://git.linaro.org/lava-team/refactoring.git
  path: hi6220-hikey.yaml

protocols:
  lava-lxc:
    name: lxc-hikey-test
    template: debian
    distribution: debian
    release: sid
    arch: amd64

actions:
- deploy:
    namespace: tlxc
    timeout:
      minutes: 5
    to: lxc
    packages:
    - android-tools-adb
    - android-tools-fastboot
    - systemd
    - systemd-sysv
    os: debian

- boot:
    namespace: tlxc
    prompts:
    - 'root@(.*):/#'
    - 'hikey:/ $'
    timeout:
      minutes: 5
    method: lxc

- deploy:
    timeout:
      minutes: 15
    to: fastboot
    namespace: droid
    connection: lxc
    images:
      ptable:
        url: http://builds.96boards.org/snapshots/reference-platform/components/uefi/latest/release/hikey/ptable-aosp-8g.img
      boot:
        url: http://builds.96boards.org/snapshots/hikey/linaro/aosp-master/147/boot.img.tar.xz
        compression: xz
        archive: tar
      cache:
        url: http://builds.96boards.org/snapshots/hikey/linaro/aosp-master/147/cache.img.tar.xz
        compression: xz
        archive: tar
      userdata:
        url: http://builds.96boards.org/snapshots/hikey/linaro/aosp-master/147/userdata.img.tar.xz
        compression: xz
        archive: tar
      system:
        url: http://builds.96boards.org/snapshots/hikey/linaro/aosp-master/147/system.img.tar.xz
        compression: xz
        archive: tar
    os: debian

- boot:
    namespace: droid
    connection: serial
    prompts:
    - 'root@(.*):/#'
    - 'hikey:/ $'
    timeout:
      minutes: 15
    method: uefi-menu
    commands: fastboot

- test:
    namespace: tlxc
    connection: lxc
    timeout:
      minutes: 10
    definitions:
    - from: inline
      repository:
        metadata:
          format: Lava-Test Test Definition 1.0
          name: check-devices
          description: "LXC dev list"
        install:
          deps:
          - nmap
          - gawk
        run:
          steps:
          - /sbin/ifconfig
          - nmap -n -sn 10.15.63.0/24
      name: lxc-dev-inline
      path: inline/lxc-dev.yaml
    - repository: git://git.linaro.org/people/senthil.kumaran/test-definitions.git
      from: git
      path: debian/get-adb-serial.yaml
      name: get-adb-serial