Posted by: terryhowe | September 29, 2016

Bash Environment Variable Search and Replace

I was trying to modify a bash environment variable today and traditionally how you did that was with command substitution and sed or awk.  Something like:

OS_TENANT_NAME=`echo $OS_TENANT_NAME | sed -e s/mesos/admin/g`


I had to throw in the accent grave there to make it super old school.

Anyway, I’m doing this and thinking that by now there has to be a better way to do this and bash does support search and replace in variables now:

$ echo ${OS_TENANT_NAME}
$ echo ${OS_TENANT_NAME}

Posted by: terryhowe | September 28, 2016

Why Working with Mesos is Like Embedded Programming

The other day I was using Mesosphere and it reminded me of when I was doing embedded programming a long time ago.

  • You have all these docker containers running the minimalistic operating systems.  You connect to one and you have no command history, lots of commands are missing and even command line options that you like are missing.  This all reminds of be working on projects where I was using Hard Hat or Busy Box on IPTV set top boxes and various modems and gateways.
  • There is an arcane procedure for uploading images into mesos just as there is  for uploading an image in an embedded environment.  You are never sure if your image is going to work or just brick the whole system.
  • Connecting to a docker container isn’t much different than tethering to some embedded device.  Instead connecting to the terminal through the serial port, you run docker exect -it and get a similar user experience.

The only problem is when you are running in the mesos world, you have to figure out where your docker container is running.

Posted by: terryhowe | September 24, 2016

Jenkins Groovy Set User Password

When you want to automate Jenkins deploys, groovy is a great way to go because you don’t have to parse xml files.  I was having trouble finding out how to set a user password in Jenkins without parsing xml.  This works:

def user = hudson.model.User.current();
Posted by: terryhowe | May 19, 2016

OpenStack Client Parsing

I’ve talked to a lot of people using the OpenStack Client that are trying to parse reports formatted with prettytable to access fields.  This kind of surprises me because it isn’t required to use awk, sed or anything else to do this, just use the command line options.  For example, if you wanted to get the id of a server and you know the name:

$ ID=$(openstack server show --format value -c id nfs-server)
$ echo $ID

You can use this technique to get any field that is available to the show command.

These switches are also available for create:

$ VOLUME_ID=$(os volume create --size 1 --format value -c id foo)

The –format and -c switches are also on the list commands, but the names of the fields may be different.  You can use these options for pipes and other scripting activities. Here is a silly example:

$ os server list --format value -c ID | while read ID
> do
> echo $ID
> done
Posted by: terryhowe | May 4, 2016

Ansible Vault Vars and Files

I’ve been talking a lot about Hashicorp Vault lately, but there is also regular old Ansible Vault.  Two of my favorite ways to use Ansible Vault are with include_vars and the file lookup.  With include_vars, you just need to encrypt your entire variables file:

ansible-vault encrypt roles/jenkins/vars/credentials.yml

The variables can be included in your task with no special decryption action:

- include_vars: "credentials.yml"

Similarly, the file lookup plugin automatically decrypts files completely. So, encrypt your certificate:

ansible-vault encrypt roles/project/files/ssl.cert

Use the file lookup plugin to decrypt it and upload it:

- copy: content="{{ lookup('file', 'ssl.cert') }}" dest=/project/config/ssl.cert mode=0400

Posted by: terryhowe | May 3, 2016

Ansible 2.0 Modules and Action Plugins

I posted yesterday about Action Plugins for Ansible 2.0 and I’d like to expand on that a little with the associated sample Ansible 2.0 module.  Modules live in the library directory and action plugins in action_plugin.  Here is my sample super simple action plugin:

from ansible.plugins.action import ActionBase

class ActionModule(ActionBase):

    def run(self, tmp=None, task_vars=None):

        if task_vars is None:
            task_vars = dict()
        result = super(ActionModule, self).run(tmp, task_vars)
        args = self._task.args.copy()
        if 'name' not in args:
            args['name'] = 'Pete'
        result.update(self._execute_module(module_args=args, task_vars=task_vars))

All this action plugin does is default the name to Pete if it is not set. Now the associated module:

#!/usr/bin/env python
module: hello
version_added: "0.1"
short_description: Hello module
    - Module to say hello.
            - Name to greet.
        default: False
- action: hello name='bob'

def main():
    argument_spec = dict(
        name = dict(required=True, type='str'),
    module = AnsibleModule(argument_spec=argument_spec)
    result = { "changed": False, "rc" : 0}
    name = module.params.get('name')
    if name:
        result['msg'] = "Hello %s!" % name
        result['failed'] = True
        result['msg'] = "Empty name is invalid"
    if result.get('failed'):

from ansible.module_utils.basic import *
from ansible.module_utils.hashivault import *

if __name__ == '__main__':

This module returns a message “Hello %(name)s”. If the name is empty, it fails. Here is a sample playbook to run the action plugin and module:

- hosts: localhost
    - hello:
    - hello: name='bob'
    - hello: name=''

When this playbook is run, the results look like:

$ ansible-playbook -v -i dev-admin-nce test.yml 
Using /etc/ansible/ansible.cfg as config file

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [hello] *******************************************************************
ok: [localhost] => {"changed": false, "msg": "Hello Pete!", "rc": 0}

TASK [hello] *******************************************************************
ok: [localhost] => {"changed": false, "msg": "Hello bob!", "rc": 0}

TASK [hello] *******************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Empty name is invalid", "rc": 0}

PLAY RECAP *********************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=1   


Posted by: terryhowe | May 2, 2016

Hashicorp Vault Ansible Module

I created a Ansible module for Hashicorp Vault.  The Module is generic and does just simple reads and writes to secrets.  It supports token, userpass, github and ldap authentication.  For the backend, it uses the hvac sdk package.

Ansible Hashicorp Vault Module

No doubt, there is something there that could be better, so throw out a pull request.  The git repository can be found here.

Posted by: terryhowe | May 2, 2016

Setting Ansible Module Defaults Using Action Plugins

In my last blog entry, I gave a simple example of an action plugin that calls a module.  To take that a bit further, here is an action plugin that sets some default value.  This can be handy if you are using a module and want to set some default values for that module that are specific to your application.

from ansible.plugins.action import ActionBase

class ActionModule(ActionBase):

    def run(self, tmp=None, task_vars=None):

        if task_vars is None:
            task_vars = dict()
        result = super(ActionModule, self).run(tmp, task_vars)
        args = self._task.args.copy()
        if 'authtype' not in args:
            args['authtype'] = 'userpass'
        if 'url' not in args:
            args['url'] = self._templar.template('https://vault.{{domain}}:8200')
        if 'username' not in args:
            args['username'] = self._templar.template('{{vault_username}}')
        if 'password' not in args:
            args['password'] = self._templar.template('{{vault_password}}')
        result.update(self._execute_module(module_args=args, task_vars=task_vars))
        self._task.register = args['register']
        return result

There are a couple things going on here. We are setting authtype to to userpass. We are also using the templar to set some arguments based on our variables.

The other thing going on here is we are changing the output after the module is called. Setting self._task.register will put the result back into the specified Ansible variable.

Posted by: terryhowe | May 2, 2016

Ansible 2.0 Action Plugins

Ansible action plugins are by far the most arcane plugins.  Finding documentation on 1.x plugins is hard enough to find and 2.0 is even more difficult.  When I tried to write my first 2.x action plugin, I got:

Unexpected Exception: No module named runner.return_data

A sample Ansible 2.0 action plugin does not use the runner return_data, it derives off ActionBase:

from ansible.plugins.action import ActionBase

class ActionModule(ActionBase):

    def run(self, tmp=None, task_vars=None):

        if task_vars is None:
            task_vars = dict()
        result = super(ActionModule, self).run(tmp, task_vars)
        result['msg'] = 'woot'
        return result

Posted by: terryhowe | March 23, 2016

Git Removing stale local brances with git prune

I’m always forgetting how to prune the local branches that have been removed from the remote.  Simple thing, but it isn’t sticking in the brain. Anyway, often when i write about something, I remember it.

git fetch; git remote prune origin

Older Posts »