Troubleshooting

Molecule

Ignoring errors

Sometimes a task or handler will fail in Molecule tests which you know won’t fail in your live environment. To ignore the failure just for the Molecule environment, add a test for the value of MOLECULE_FILE in the runtime environment.

1
2
3
4
5
6

- name: Start service.
  service:
    name: special-agent-snoopy
    state: started
  listen: start-service
  ignore_errors: lookup('env', 'MOLECULE_FILE')

Debug messages

First, turn on debug mode in Molecule molecule --debug test.

When an Ansible stage in executed by Molecule against your test instance, it may fail, un-helpfully telling you that it cannot display the error because no_log: true.

1
2
3
4

    ok: [localhost] => {
        "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result",
        "changed": false
    }

In your molecule/default/molecule.yml file, edit the provisioner arguments to add log: True as so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml
--- a/molecule/default/molecule.yml
+++ b/molecule/default/molecule.yml
@@ -10,6 +10,7 @@ platforms:
     image: centos7
 provisioner:
   name: ansible
+  log: True
   lint:
     name: ansible-lint
 verifier:

assert host.socket

The container you’re testing needs to have ss or netstat installed to assert tests against sockets. You shouldn’t need to install netstat as a playbook task when installing something like apache2 on your container. In the case of molecule, you want to install netstat as a dependency of the testing platform, as part of the Dockerfile that builds your test container, otherwise, you’ll get an error like this:

1

RuntimeError: could not use the Socket module, either "ss" or "netstat" utility is required in $PATH

Update molecule/default/Dockerfile.j2 to include the net-tools package, so that it installs when your test container initializes, before the ansible playbook job runs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

diff --git a/molecule/default/Dockerfile.j2 b/molecule/default/Dockerfile.j2
index e6aa95d302050835bfe3d10b040a8a415df695d3..7b128716a9521d20d22b219ea52b858feb15b662 100644
--- a/molecule/default/Dockerfile.j2
+++ b/molecule/default/Dockerfile.j2
@@ -6,9 +6,9 @@ FROM {{ item.registry.url }}/{{ item.image }}
 FROM {{ item.image }}
 {% endif %}

-RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates && apt-get clean; \
-    elif [ $(command -v dnf) ]; then dnf makecache && dnf --assumeyes install python sudo python-devel python*-dnf bash && dnf clean all; \
-    elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl bash && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
-    elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml && zypper clean -a; \
-    elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates; \
+RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates net-tools && apt-get clean; \
+    elif [ $(command -v dnf) ]; then dnf makecache && dnf --assumeyes install python sudo python-devel python*-dnf bash net-tools && dnf clean all; \
+    elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl bash net-tools && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
+    elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml net-tools && zypper clean -a; \
+    elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates net-tools; \
     elif [ $(command -v xbps-install) ]; then xbps-install -Syu && xbps-install -y python sudo bash ca-certificates && xbps-remove -O; fi

Missing services

Ansible may fail to find services you would normally have available on a standard virtual machine.

1

fatal: [instance]: FAILED! => {"changed": false, "msg": "Could not find the requested service httpd: "}

Molecule defaults to using CentOS, which supports systemd, but disables it by default. You may need to adjust the default molecule/default/Dockerfile.j2 RUN options to support the service module in your Ansible playbook.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]

Additionally, you’ll need to mount the host filesystem for a centos:7 container to run systemd. Add a volumes section to your molecule/default/molecule.yml configuration.

1
2
3
4
5

platforms:
  - name: instance
    image: centos:7
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro

Can’t find service command

Install initscripts for CentOS to use the service ansible module to start or enable services for systemd tests.

1

ValueError: cannot find "service" command

Python errors

http.client

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

==================================== ERRORS ====================================
____________________ ERROR collecting tests/test_default.py ____________________
ImportError while importing test module '/home/travis/build/deekayen/deekayen.molecule_sandbox/molecule/default/tests/test_default.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_default.py:1: in <module>
import http.client
tests/http.py:2: in <module>
import http.client
E   ModuleNotFoundError: No module named 'http.client'; 'http' is not a package
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.10s ===============================

The offending line to cause the error was introduced when adding import http.client to the start of molecule/default/tests/test_default.py, while attempting to have Python make a connection test to port 80 on an affected test container.

You may note that the default molecule/default/Dockerfile.j2 file generated by molecule init installs python, not python3 by default to the test container.

Ansible Lint

Excluding files from linting

When invoking ansible-lint, you can allow the utility to discover files on its own.

1
2
3
4

---
ansible-lint:
   script:
     - ansible-lint .

In continuous integration, sometimes the include path configurations become broad. On a shared job runner, other Ansible Galaxy roles can be cached in the runtime user’s home folder, where ansible-lint finds and scans them. Exclude known cache paths from scanning with --exclude.

1
2
3
4

---
ansible-lint:
   script:
     - ansible-lint --exclude=/home/gitlab-runner/.ansible/roles .

When generating dynamic yaml files as jinja2 templates, ansible-lint can become confused about whether variables really need to be quoted or not.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

$ git ls-files | grep yml | xargs ansible-lint
Syntax Error while loading YAML.
  found unacceptable key (unhashable type: 'AnsibleMapping')
The error appears to be in '/home/gitlab-runner/builds/f7Qtasdf/0/ansible-example/roles/demo/templates/templatefile.yml.j2': line 5, column 26, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
  data:
    LINODE_REGION: {{ linode_region }}
                  ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
    with_items:
      - {{ foo }}
Should be written as:
    with_items:
      - "{{ foo }}"

In the above example, the grep yml sent the jinja2 template to ansible-lint for review. Stack some reverse grep statements to exclude them from the linter. This example skips any files which are likely jinja2 templates for Ansible vault files.

1
2
3
4

---
ansible-lint:
   script:
     - git ls-files | grep yml | grep -v vault | grep -v j2 | xargs ansible-lint

Ansible Review

Unknown library for tasks

When ansible-review fails to find a module name, first check that you spelled the module name in the task correctly. If spelled correctly, check the version of the underlying ansible installation on your test host with ansible --version. Compare the Ansible version with the documentation for the module to see when the module was added to the Ansible project. For example, you cannot use Ansible 2.7.10 to successfully test a role which uses the iam_password_policy role, as it was introduced in Ansible 2.8, as noted by New in version 2.8. at the top of the iam_password_policy documentation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

$ git ls-files | grep -E "\.yml$" | xargs ansible-review -q
Couldn't parse task at tasks/main.yml:4 (no action detected in task. This often indicates a misspelled module name, or incorrect module path.
The error appears to have been in '<unicode string>': line 4, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
(could not open file to display line))
{ u'iam_password_policy': { '__file__': 'tasks/main.yml',
                            '__line__': 6,
                            u'allow_pw_change': True,
                            u'min_pw_length': 16,
                            u'pw_expire': False,
                            u'pw_max_age': 60,
                            u'pw_reuse_prevent': 5,
                            u'require_lowercase': True,
                            u'require_numbers': True,
                            u'require_symbols': True,
                            u'require_uppercase': True,
                            u'state': u'present'},
  u'name': u'Set IAM account password policy.'}

When implementing your own python or shell scripts to create a custom task type, ansible-review will find the tasks which implement your custom module but show them as the same parse error as if you had an Ansible version mismatch. To have the CI engine find your custom libary files, update the ANSIBLE_LIBRARY environment variable to include the path in your testing workspace1. A common location for your custom task modules is the library file in the root of your playbook or role repository.

1
2
3
4
5
6
7

---
variables:
  ANSIBLE_LIBRARY: "./library:~/.ansible/plugins/modules:/usr/share/ansible/plugins/modules"

ansible-review:
  script:
    - git ls-files | grep -E "\.yml$" | xargs ansible-review -q

Alternatively, change the DEFAULT_MODULE_PATH to see your custom libraries in a tool independent location by adding the library path to an ansible.cfg file in the root of your project2.

1
2

[defaults]
library = ./library:~/.ansible/plugins/modules:/usr/share/ansible/plugins/modules

  1. Ansible review dependencies ↩︎

  2. ansible default config ↩︎