File format

Files use YAML 1.1 for the sake of human and machine readability and write-ability. As a good practice, the files should contain a header such as:

%YAML 1.1
# ROS index|distribution|release-build|source-build|doc-build file
# see REP 141: https://ros.org/reps/rep-0141.html
---

Index file

The index file acts as a single configuration entrypoint which lists all available ROS distributions, along with references pointing to further process specific information. The references can be either relative to the index file or absolute.

The information stored in the index file is:

• distributions: a list of ROS distributions

  For each distribution further information are referenced:

  • distribution: reference to the distribution file
  • release_builds: list of references to the release-build files used to build the binary packages
  • release_cache: reference to a release cache. Whether this field is a dictionary, a list or a scalar is left as an implementation detail. The following examples will assume that the implementation necessitates an url.
  • source_builds: list of references to the source-build files used to run the tests
  • doc_builds: list of references to the doc-build files used to run the documentation

• type: must be 'index'

• version: version number, this REP describes version 2 (instead of version 1 described in REP 137 [6])

Example

An index file referencing multiple distribution with their release, source and doc files and their corresponding build files.

%YAML 1.1
# ROS index file
# see REP 141: https://ros.org/reps/rep-0141.html
---
distributions:
  groovy:
    distribution: groovy/distribution.yaml
    release_builds: [groovy/release-build-ubuntu.yaml, releases/release-build-arm.yaml]
    release_cache: http://www.example.com/groovy-cache
    source_builds: [groovy/source-build.yaml]
    doc_builds: [groovy/doc-build.yaml]
  hydro:
    ...
type: index
version: 2

Distribution file

A distribution is identified by the code name of the ROS distribution in the index file. Each distribution file contains the following information:

• repositories: a list of repositories which are identified by unique names

  • release: all release repository related information are nested under this key

    • url: the URL of the release git repository The URL must be anonymously readable and (with the appropriate credentials) writable.

    • version: version number for which packages are released. This can be a tag, branch or hash.

    • packages: an optional list of package names. If no package is specified, one package with the name of the repository is assumed.

    • tags: a dict of tags which can be used by tools like bloom and the rosinstall generator, preventing the need for those tools to guess about tags in release repositories, e.g. 'release/package/1.2.3' vs 'release/groovy/package/1.2.3-4'. This provides a useful future proofing mechanism for tools which use tags in the release repository, rather than guessing the format of the tags they can infer them directly from the tag templates.

      • tag_name {release, debian, etc...}: Format of tags are strings with {template_variables} e.g. 'release/groovy/{package}/{version}' A non-exhaustive list of possible template tags:

        • package - name of the package which this release tag corresponds
        • version - full version of the package being released, e.g. 1.2.3-4
        • upstream_version - upstream version of package being released, e.g. 1.2.3
        • debian_distro - target debian distro codename
        • debian_package_name - name of package, with any prefix and sanitized for debian

        The only required tag_name is 'release', others like 'debian' are optional.

  • source: all source repository related information are nested under this key

    • type: the type of SCM in use. Typically 'git', 'svn'...
    • url: the URL of the source repository The URL must be anonymously readable.
    • version: For git and hg this is the tag, branch or hash to be checked out. For svn the version should not be set since the branch/tag is encoded in the url.

  • doc: all doc repository related information are nested under this key

    • type: the type of SCM in use. Typically 'git', 'svn'...
    • url: the URL of the source repository The URL must be anonymously readable.
    • version: For git and hg this is the tag, branch or hash to be checked out. For svn the version should not be set since the branch/tag is encoded in the url.
    • blacklist_packages: an optional list of package names which should not be documented. (default: [])
    • depends: list of repository names. Other repositories to perform cross referencing in the documentation. This is only necessary if the packages from the dependent repositories are not released. (default: [])

  • status: defines the status of the packages of the repository. Can be one of the following: developed, maintained, unmaintained, end-of-life

  • status_description: an optional field describing in a short sentence the current status of the repository. For example detailing the reason for EOL and the recommended upgrade path.

  • status_per_package: an optional list of package names. For each package name a dictionary with the following optional keys allows to override the status information:

    • status: overrides the repository-wide status
    • status_description: overrides the repository-wide status description

• release_platforms: a dict of target platforms for which packages are released. Each key contains the OS name while the value is a list of OS code names. These OS names and OS code names are determined by rospkg.os_detect [4]. Each target platform will result in a different bloom release.

• type: must be 'distribution'

• version: version number, this REP describes version 1

Example: A distribution file listing repositories and the release platforms. All changes to REP 137 are commented on inline.

%YAML 1.1
# ROS distribution file
# see REP 141: https://ros.org/reps/rep-0141.html
---
# prefixed with 'release_' since these are the platforms we release for
release_platforms:
  debian: [wheezy]
  ubuntu: [precise, quantal, raring]
repositories:
  catkin:
    # each repo contains the (optional) sections for 'doc', 'release' and 'source'
    doc:
      type: git
      url: https://github.com/ros/catkin.git
      version: groovy-devel
    release:
      tags:
        release: release/hydro/{package}/{version}
      url: https://github.com/ros-gbp/catkin-release.git
      version: 0.5.77-0
    source:
      type: git
      url: https://github.com/ros/catkin.git
      version: groovy-devel
  genmsg:
    doc:
      type: git
      url: https://github.com/ros/genmsg.git
      # specified separately from 'source' since it might use different version
      version: 0.4.23
    release:
      tags:
        release: release/hydro/{package}/{version}
      url: https://github.com/ros-gbp/genmsg-release.git
      version: 0.4.23-0
    source:
      type: git
      url: https://github.com/ros/genmsg.git
      version: hydro-devel
  roscpp_core:
    doc:
      # optional blacklist packages to avoid documentation of these
      blacklist_packages: [test_rostime]
      depends: [genmsg]
      type: git
      url: https://github.com/ros/roscpp_core.git
      version: hydro-devel
    release:
      # the packages are populated by bloom
      # the subfolder is not stored anymore
      packages: [cpp_common, roscpp_core, roscpp_serialization, roscpp_traits, rostime]
      tags:
        release: release/hydro/{package}/{version}
      url: https://github.com/ros-gbp/roscpp_core-release.git
      version: 0.3.16-0
    source:
      # blacklisting packages should be done with CATKIN_IGNORE files
      # rather than in this file to also cover users checking out the repo
      type: git
      url: https://github.com/ros/roscpp_core.git
      version: hydro-devel
    # status also covers 'doc' and 'source' (not only release, e.g. if not even released)
    status: maintained
    status_description: 'Very actively maintained'
    # status can be overridden per package
    status_per_package:
      roscpp_core:
        status: end-of-life
        status_description: 'Metapackage is not necessary anymore'
      rostime:
        status: unmaintained
type: distribution
version: 1

Release build file

A release build file contains the information necessary to build packages of the packages specified in the release file:

• package_whitelist: a list of package names to build. If this is omitted all packages specified in the release part of the distribution file are built. Any upstream packages are implicitly included.
• package_blacklist: a list of package names excluded from build. If this is omitted no packages are excluded. Any downstream package are implicitly excluded. The blacklist overrides the whitelist.
• notifications: An optional section to configure email notifications.
  • emails: a list of email addresses to which to send all notification emails. (default: empty)
  • maintainers: a boolean flag used to enable email notification to listed maintainers. (default: false)
  • committers: a boolean flag used to enable email notification to committers. (default: false)
• targets: a nested dict of targets for which packages are built. The first level key contains the OS name. The second level key contains the OS code name. The third level key contains the CPU architecture. The OS names and OS code names specified must be listed as a release platform in the corresponding distribution file. Each level can contain a key _config which can contain a dict with arbitrary data. These configurations can be used to specify target specific information (e.g. apt_target_repository or yum_repository).
• jenkins_url: the url to the associated Jenkins master
• jenkins_sourcedeb_job_timeout: the timeout in minutes for the sourcedeb jobs (optional).
• jenkins_binarydeb_job_timeout: the timeout in minutes for the binarydeb jobs (optional).
• sync: specify the criteria that need to be fulfilled for packages to be synced. Any of the following options can be set and all of them must be fulfilled:
  • package_count: integer. Specifies how many packages need to be successfully built to perform a sync. (default: 0)
  • packages: list of package names. All the packages of this list must be successfully built to perform a sync. (default: [])
• type: must be 'release-build'
• version: version number, this REP describes version 1, since the specification is the same as in REP 137

Example: A build file selecting a subset of packages from the release part of a distribution file and specifying the platforms and architectures.

%YAML 1.1
# ROS release-build file
# see REP 141: https://ros.org/reps/rep-0141.html
---
package_whitelist: [ros_tutorials, common_tutorials]
package_blacklist: [turtlesim]
notifications:
  emails: [buildfarm-admin@example.com]
  maintainers: true
  committers: false
targets:
  _config:
    apt_target_repository: http://archives.example.com/ros
    apt_mirrors: [http://archives.example.com/ros, http://packages.foo.org/repos/example]
  ubuntu:
    oneiric:
      amd64:
      i386:
    precise:
      amd64:
      i386:
      armel:
jenkins_url: http://jenkins.example.com:8080
sync:
  packages: [ros_tutorials]
type: release-build
version: 1

Release cache file

Collection of all meta information of the ROS distribution, including all the information from the package.xml files. The cache must reference the release file and store a hash of the release file it was build from to be able to detect if the cache is invalid. The format of that cache is considered an implementation detail and is not specified in this REP.

Source build file

The source build file uses a specification similar to the release build file, but does not have sync information. Also the white- and blacklist is on a repository level and does not consider any kind of dependencies.

• repository_whitelist: a list of repository names to build. If this is omitted all repositories specified in the source part of the distribution file are built.
• repository_blacklist: a list of repository names excluded from build. If this is omitted no repositories are excluded. The blacklist overrides the whitelist.
• notifications: as specified for the release build file.
• targets: as specified for the release build file.
• jenkins_url: as specified for the release build file.
• jenkins_job_timeout: the timeout in minutes for each job (optional).
• type: must be 'source-build'
• version: version number, this REP describes version 1, since the specification is the same as in REP 137

Documentation build file

The documentation build file uses a specification similar to the source build file.

• repository_whitelist: a list of repository names to build. If this is omitted all repositories specified in the doc part of the distribution file are built.
• repository_blacklist: a list of repository names excluded from build. If this is omitted no repositories are excluded. The blacklist overrides the whitelist.
• notifications: as specified for the source build file.
• targets: as specified for the source build file. But the list of targets must only have one entry.
• jenkins_url: as specified for the source build file.
• jenkins_job_timeout: the timeout in minutes for each job (optional).
• doc_tag_index_repository: a repository storing the tag index
  • type: the type of SCM in use. Must be 'git'
  • url: the URL of the release git repository The URL must be writable (with the appropriate credentials).
  • version: This is the branch to be checked out and committed to.
• type: must be 'doc-build'
• version: version number, this REP describes version 1, since the specification is the same as in REP 137

Reference implementation

This REP is to be implemented in version 0.3 of the Python module rosdistro. This module will serve as a reference implementation for this REP. Any ROS tool requesting information defined in this REP should use this reference API implementation.

Affected tools

For all tools it must be ensured that the used API of rosdistro (even if deprecated) is still working flawlessly:

• bloom [1]
• buildfarm
• jenkins_scripts
• jenkins_tools
• prerelease_website
• rosdep [2]
• rosinstall_gen
• rosinstall_generator
• roslocate [3]

After this REP has been deployed 'rosinstall_generator' can be modified to enable not only checking out released packages but also source repositories.

The other tools should be updated in the future to use the newly introduced rather then the deprecated API. After some migration period the deprecated API might be removed from 'rosdistro'.