APT Source

This part of curtin is meant to allow influencing the apt behaviour and configuration.

By default - if no apt config is provided - it does nothing. That keeps behavior compatible on upgrades.

The feature has an optional target argument which - by default - is used to modify the environment that curtin currently installs (@TARGET_MOUNT_POINT).

Features

  • Add PGP keys to the APT trusted keyring
    • add via short keyid
    • add via long key fingerprint
    • specify a custom keyserver to pull from
    • add raw keys (which makes you independent of keyservers)
  • Influence global apt configuration
    • adding ppa’s
    • replacing mirror, security mirror and release in sources.list
    • able to provide a fully custom template for sources.list
    • add arbitrary apt.conf settings
    • add arbitrary apt preferences
    • provide debconf configurations
    • disabling suites (=pockets)
    • disabling components (multiverse, universe, restricted)
    • per architecture mirror definition

Configuration

The general configuration of the apt feature is under an element called apt.

This can have various “global” subelements as listed in the examples below. The file apt-source.yaml holds more examples.

These global configurations are valid throughput all of the apt feature. So for exmaple a global specification of a primary mirror will apply to all rendered sources entries.

Then there is a section sources which can hold any number of source subelements itself. The key is the filename and will be prepended by /etc/apt/sources.list.d/ if it doesn’t start with a /. There are certain cases - where no content is written into a source.list file where the filename will be ignored - yet it can still be used as index for merging.

The values inside the entries consist of the following optional entries

  • source: a sources.list entry (some variable replacements apply)
  • keyid: providing a key to import via shortid or fingerprint
  • key: providing a raw PGP key
  • keyserver: specify an alternate keyserver to pull keys from that were specified by keyid

The section “sources” is is a dictionary (unlike most block/net configs which are lists). This format allows merging between multiple input files than a list like

sources:
   s1: {'key': 'key1', 'source': 'source1'}

sources:
   s2: {'key': 'key2'}
   s1: {'keyserver': 'foo'}

This would be merged into
   s1: {'key': 'key1', 'source': 'source1', keyserver: 'foo'}
   s2: {'key': 'key2'}

Here is just one of the most common examples for this feature: install with curtin in an isolated environment (derived repository):

For that we need to: * insert the PGP key of the local repository to be trusted

  • since you are locked down you can’t pull from keyserver.ubuntu.com
  • if you have an internal keyserver you could pull from there, but let us assume you don’t even have that; so you have to provide the raw key
  • in the example I’ll use the key of the “Ubuntu CD Image Automatic Signing Key” which makes no sense as it is in the trusted keyring anyway, but it is a good example. (Also the key is shortened to stay readable)
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1
mQGiBEFEnz8RBAC7LstGsKD7McXZgd58oN68KquARLBl6rjA2vdhwl77KkPPOr3O
RwIbDAAKCRBAl26vQ30FtdxYAJsFjU+xbex7gevyGQ2/mhqidES4MwCggqQyo+w1
Twx6DKLF+3rF5nf1F3Q=
=PBAe
-----END PGP PUBLIC KEY BLOCK-----
  • replace the mirrors used to some mirrors available inside the isolated environment for apt to pull repository data from.
    • lets consider we have a local mirror at mymirror.local but otherwise following the usual paths
    • make an example with a partial mirror that doesn’t mirror the backports suite, so backports have to be disabled

That would be specified as

apt:
  primary:
    - arches [default]
      uri: http://mymirror.local/ubuntu/
  disable_suites: [backports]
  sources:
    localrepokey:
      key: | # full key as block
        -----BEGIN PGP PUBLIC KEY BLOCK-----
        Version: GnuPG v1

        mQGiBEFEnz8RBAC7LstGsKD7McXZgd58oN68KquARLBl6rjA2vdhwl77KkPPOr3O
        RwIbDAAKCRBAl26vQ30FtdxYAJsFjU+xbex7gevyGQ2/mhqidES4MwCggqQyo+w1
        Twx6DKLF+3rF5nf1F3Q=
        =PBAe
        -----END PGP PUBLIC KEY BLOCK-----

The file examples/apt-source.yaml holds various further examples that can be configured with this feature.

Common snippets

This is a collection of additional ideas people can use the feature for customizing their to-be-installed system.

  • enable proposed on installing
apt:
  sources:
    proposed.list:
      source: |
        deb $MIRROR $RELEASE-proposed main restricted universe multiverse
  • Make debug symbols available
apt:
  sources:
    ddebs.list:
      source: |
        deb http://ddebs.ubuntu.com $RELEASE main restricted universe multiverse
        deb http://ddebs.ubuntu.com $RELEASE-updates main restricted universe multiverse
        deb http://ddebs.ubuntu.com $RELEASE-security main restricted universe multiverse
        deb http://ddebs.ubuntu.com $RELEASE-proposed main restricted universe multiverse

Timing

The feature is implemented at the stage of curthooks_commands, which runs just after curtin has extracted the image to the target. Additionally it can be ran as standalong command “curtin -v –config <yourconfigfile> apt-config”.

This will pick up the target from the environment variable that is set by curtin, if you want to use it to a different target or outside of usual curtin handling you can add --target <path> to it to overwrite the target path. This target should have at least a minimal system with apt, apt-add-repository and dpkg being installed for the functionality to work.

Dependencies

Cloud-init might need to resolve dependencies and install packages in the ephemeral environment to run curtin. Therefore it is recommended to not only provide an apt configuration to curtin for the target, but also one to the install environment via cloud-init.

apt preserve_sources_list setting

cloud-init and curtin treat the preserve_sources_list setting slightly differently, and thus this setting deserves its own section.

Interpretation / Meaning

curtin reads preserve_sources_list to indicate whether or not it should update the target systems’ /etc/apt/sources.list. This includes replacing the mirrors used (apt/primary…).

cloud-init reads preserve_sources_list to indicate whether or not it should render /etc/apt/sources.list from its built-in template.

defaults

Just for reference, the preserve_sources_list defaults in curtin and cloud-init are:

  • curtin: true By default curtin will not modify /etc/apt/sources.list in the installed OS. It is assumed that this file is intentionally as it is.
  • cloud-init: false
  • cloud-init in ephemeral environment: false
  • cloud-init system installed by curtin: true (curtin writes this to a file /etc/cloud/cloud.cfg.d/curtin-preserve-sources.cfg in the target). It does this because we have already written the sources.list that is desired in the installer. We do not want cloud-init to overwrite it when it boots.

preserve_sources_list in MAAS

Curtin and cloud-init use the same apt configuration language. MAAS provides apt config in three different scenarios.

  1. To cloud-init in ephemeral environment (rescue, install or commissioning)
    Here MAAS should not send a value. If it wants to be explicit it should send preserve_sources_list: false.
  2. To curtin in curtin config
    MAAS should send ``preserve_sources_list: false``. curtin will correctly read and update mirrors in official Ubuntu images, so setting this to ‘false’ is correct. In some cases for custom images, the user might want to be able to have their /etc/apt/sources.list left untouched entirely. In such cases they may want to override this value.
  3. To cloud-init via curtin config in debconf_selections.
    MAAS should not send a value. Curtin will handle telling cloud-init to not update /etc/apt/sources.list. MAAS does not need to do this.
  4. To installed system via vendor-data or user-data.
    MAAS should not send a value. MAAS does not currently send a value. The user could send one in user-data, but then if they did presumably they did that for a reason.

Legacy format

Versions of cloud-init in 14.04 and older only support:

apt_preserve_sources_list: VALUE

Versions of cloud-init present 16.04+ read the “new” style apt configuration, but support the old style configuration also. The new style configuration is:

apt:
  preserve_sources_list: VALUE

Note: If versions of cloud-init that support the new style config receive conflicting values in old style and new style, cloud-init will raise exception and exit failure. It simplly doesn’t know what behavior is desired.