|Title:||Incorporation of Changelogs into Package Source Tree|
|Author:||Tully Foote, Thibault Kruse, Mirza Shah, Dirk Thomas, William Woodall|
This REP describes incorporating package changelogs (i.e. a list of changes made to the package in each release) as part of the package source tree rather than being maintained separately on the ROS wiki. This is to address shortcomings with maintaining a separate changelog list.
In ROS, package changelogs have traditionally been maintained as their own page within a package's ROS wiki page. These changelogs have had to be maintained separately from their code bases. This approach has had some drawbacks:
The format of the changelog should have the following properties:
The changelog document must be valid RST. The document will be processed by an RST parser (ignoring file inclusion directives, as those also do not work on GitHub).
The changelog file will use one section without subsections for each changelog entry, and within that other elements from the RST specification  are allowed:
Inline markup may not be preserved when transforming the log entries for deb/rpm format. The following elements of the RST specification  are explicitly forbidden in the version sections of the changelog:
A changelog entry is defined by a heading which contains a version number followed by the timestamp in parenthesis. The version number consists of three positive integers separate by single dots, e.g. 1.2.3 as specified in . The timestamp must be parseable by the Python dateutil module, i.e. dateutil.parser.parse(...). The timestamp should at least contain a full date (YYYY-MM-DD, ISO 8601 format). The rest of the elements in the timestamp are optional, but should be included in order, e.g. you should not specify seconds without minutes and hours. Additional elements are: hours and minutes (together), seconds, and timezone offset.
A changelog version section may not contain subsections, but it may be a subsection itself.
The following example shows a few valid changelog entries:
1.2.3 (2013-01-23) ------------------ * Change 1 Optional Text . . . * Change n Optional Text 1.2.2 (2013-01-22 08:35) ------------------------ * Change 1 * Change 2 1.2.1 (2013-01-22 06:35 +0100) ------------------------------ * Change
This is a standard example, but more variations are possible with the allowed RST elements. The following example uses a lot of variations to show what possibilities exist:
^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package foo ^^^^^^^^^^^^^^^^^^^^^^^^^ 0.1 === Free form text about this minor release. 0.1.27 (forthcoming) -------------------- * Great new feature 0.1.26 (2012-12-26) ------------------- * Utilizes caching to improve query performance (fix https://github.com/ros/ros_comm/pull/2) * Simplified API calls based on (https://github.com/ros/robot_model) Note that these changes are based on REP 192 * Fixed synchronization issue on startup .. not mentioning secret feature on purpose 0.1.25 (2012-11-25) ------------------- - Added thread safety - Replaced custom XML parser with `TinyXML <http://www.grinninglizard.com/tinyxml/>`_. - Fixed regression introduced in 0.1.22 ---- The library should now compile under Win32 0.1.0 (2012-10-01) ------------------ First public *stable* release 0.0 === 0.0.1 (2012-01-31) ------------------ 1. Initial release 2. Initial bugs
In the above example, the header (enclosed by ^ characters) and the sections with only MAJOR.MINOR are additional visual markup to make the changelog more readable / structure for humans to read, but will be ignored by tools which process the changelogs, e.g. bloom will not insert these elements into the debian changelog file.
Similarly the section 0.1.27 (forthcoming) does not match the format of the version section title, so it ignored when being parsed by tools like bloom. This is recommended practice to state changes which are already committed but not yet released.
The ROS wiki shall have a macro which users can point directly to the URL of the CHANGELOG.rst file. The ROS wiki will also have the changelog automatically embedded by rosdoc-lite when the package is parsed by the documentation indexer, such that the macro should only be needed for packages which are not indexed by the documentation system.
There are several rules which are good ideas and strongly encouraged, but either shouldn't be or cannot be enforced.
One rule is to have a valid version section for each version of a package released. For example, it is considered bad form to have changelog sections for 1.0.0 and 1.0.1 but not 1.0.2. This is confusing to anyone who is reading the raw CHANGELOG.rst file and requires tools like rosdoc-lite and bloom to fill the gap. This rule is highly recommended and if the developer chooses to skip a version, the release tools will protest, but allow it. This rule comes up because packages which share a single source repository must maintain the same version, so when changes in one package cause the version to increase in its peers, the peers may have no new changes to report. In this case the recommendation is to have an empty section like this:
0.3.5 (2013-05-09 16:36:55 -0700) --------------------------------- 0.3.4 (2013-04-09 16:36:55 -0700) --------------------------------- - More changes - Changed a thing
Alternatively you can give a simple message why this section is blank:
0.3.5 (2013-05-09 16:36:55 -0700) --------------------------------- - Released along with other packages, no changes 0.3.4 (2013-04-09 16:36:55 -0700) --------------------------------- - More changes - Changed a thing
The proposed format has the following properties that help to meet the design requirements:
While the toolchain could use the current time when a release is made the information is missing when reading the changelog in the source repository. Without a timestamp it is also not clear if the version has already been release (but the maintainer did not provided a timestamp) or is forthcoming (where the maintainer did not add an annotation for that).
This would make the decision if a headline is a valid changelog entry more difficult. On the other hand free form text could be either placed before a changelog entry or inside the changelog entry which should be sufficient to add additional information. Therefore the specification does not allow that.
How can a full changelog with multiple versions be generated?
The available information from the current package.xml and changelog file can be used to generate the changelog for the current version. Older changelog can not be generated since the information from the package.xml file at that point in time might have been different (i.e. other maintainer). Anyway a full changelog can be constructed based on the changelog of the previous changelog and the changelog of the current version if desired.
How to link to tickets/issues in bug tracker without having to give full URL?
Would be nice if GitHub did this for us on their website, but currently it does not.
- Outside section entries, no reason to forbid full RST
- Inside section entries, we only want to support things that can easily be transformed into deb/rpm format, though some loss of quality might be acceptable. Things to consider:
- Substitutions http://docutils.sourceforge.net/docs/ref/rst/directives.html#replacement-text
- References http://docutils.sourceforge.net/docs/ref/rst/directives.html#references
- Inclusion of other files (disabled on GitHub)
- Nested lists
- Definition lists (could also be used for version!)
- Directives, such as . note:: foo
REP now states some definitely allowed and forbidden elements. More may be allowed if users demand that and they can be easily supported.
Not urgent, leave out for now.
An early suggestion "ChangeList.txt" was rejected due to similarity to CMake "CMakeLists.txt".
The RST extension makes it possible for GitHub to render the file, and allows us to later possibly also support other markup flavors.
The package root is a common default way for such meta information, a "doc" sub folder is useful for static documentation. Sphinx does not allow to refer to documents outside the doc folder via toc-trees, but it does allow inclusion of files like this:.. include:: ../CHANGELOG.rst
So we went for CHANGELOG.rst in root as ideal place. Alternatives are not planned to have a single location to check for the existence of a changelog.
The prototype library could handle such complex README files. Though no technical reason is known that would prevent this, there was too much doubt on possible unknown problems with that approach, and user confusion over multiple alternatives, so for now it was decided to not go ahead with this.
- Forbid all inline markup
- Support some inline markup nicely, forbid all that we do not transform
- Support some inline markup nicely, treat other markup as raw source
- Support all inline markup nicely
The actual transformations to happen are for other tools to decide. For now, we shall support some markup nicely (references), and treat other markup as raw source.
- Wiki display: We could display the changelog in the wiki as raw text, try to render the RST, display what goes into the deb, or merely link to the source file in its home repo.
- raw display is quickest for the users and easiest for us, maybe
- rendered display is nicer to the eye, allows following embedded references
- link to the source location is a bit worse for the users (navigating separate sites, but may be least effort)
For rosbuild stacks the CHANGELOG.rst file could be placed beside the stack.xml file. However, that won't be a priority to implement in the near future and might require contributions from the community.
package (version) distribution(s); urgency=urgency [optional blank line(s), stripped] * change details more change details [blank line(s), included in output of dpkg-parsechangelog] * even more change details [optional blank line(s), stripped] -- maintainer name <email address>[two spaces] date
The Debian Policy manual  goes further to describe the maintainer as:
The maintainer name and email address used in the changelog should be the details of the person uploading this version. They are not necessarily those of the usual package maintainer.
* Fri Jun 23 2006 Jesse Keating <email@example.com> - 0.6-4 - And fix the link syntax. * Fri Jun 23 2006 Jesse Keating <firstname.lastname@example.org> 0.6-4 - And fix the link syntax. * Fri Jun 23 2006 Jesse Keating <email@example.com> - 0.6-4 - And fix the link syntax. * Wed Jun 14 2003 Joe Packager <joe at gmail.com> - 1.0-2 - Added README file (#42).
A prototype implementation of a library that parses any RST document and extracts changelog entries as described here is provided as ongoing effort here .
Since then an official implementation has been merged into catkin_pkg which will be used by rosdoc-lite and bloom: 
|||(1, 2) Debian Package Changelog Requirements (http://www.debian.org/doc/debian-policy/ch-source.html)|
|||Fedora RPM Package Changelog Requirements (http://fedoraproject.org/wiki/Packaging:Guidelines#Changelogs)|
|||Tully Foote Proposal for Stack Changelogs (9-03-2010) (http://code.ros.org/lurker/message/20100903.213420.d959fddc.en.html)|
|||(1, 2) reStructuredText (RST) (http://docutils.sourceforge.net/rst.html)|
|||(1, 2) Github Markup languages (https://github.com/github/markup)|
|||(1, 2) Prototype python script (https://github.com/tkruse/changelog_rst.git)|
|||Buildsystem SIG discussion (https://groups.google.com/d/msg/ros-sig-buildsystem/L3nE9X0T2Jk/ML_1JsHLuF0J)|
|||REP 127 Specification of package manifest format (https://github.com/ros-infrastructure/rep/blob/master/rep-0127.rst#version)|
|||(1, 2) Implementation in catkin_pkg (https://github.com/ros-infrastructure/catkin_pkg/blob/master/src/catkin_pkg/metapackage.py)|
This document has been placed in the public domain.