Provisioning a virtual machine with Ansible

Time goes so fast, it’s hard to believe that a year and a half went by since I started my blogging series about combining Vagrant, VMware and Ansible. To think that at the time I thought I would complete the series within the same month… Anyway, you are now reading part 2, where I will dive into actually provisioning a VMware virtual machine managed by Vagrant using Ansible. If you want to know how to start using Vagrant with VMware, you should read part 1.

Please keep in mind again that technologies are interchangeable here. Combining Ansible and VMware with Vagrant is just my personal choice. This post is about provisioning a machine managed by Vagrant with Ansible. You could be not using Vagrant with a virtual machine, but a physical machine or you could be not using VMware, but VirtualBox instead.

What do you need?

Before you can start provisioning you need to install Ansible on the host machine (your computer or laptop for example) and you should add a configuration section in the Vagrantfile. The Vagrantfile is the configuration file for Vagrant that tells it where to find its base box and how to use it.

Installing Ansible

For linux distributions, Ansible comes as a package that you can install with the appropriate package manager, such as Aptitude for Ubuntu by typing in a terminal window:

For a Mac, as in my case, you can do the same using MacPorts:

Configuring Vagrant to use Ansible

In the Vagrantfile, add the following section:

Also, make sure a specific private network is configured for you machine, so that you know the ip of the machine that Ansible will be provisioning:

As you may have guessed, verbose will set the relative amount of output you will see when the Ansible playbook is running. Adding more v’s will increase the amount of output.

The playbook is the script that Ansible uses to do the actual provisioning. More about the playbook later. “playbook.yml” is the path to the playbook, which in this case is relative to the path of the Vagrantfile itself.

The inventory_path is the path to the inventory file, which is used by Ansible to configure which servers are available for provisioning. The playbook will then specify which of these servers actually will be provisioned with the given playbook. Since Ansible is a tool intended to provision entire server clusters, this may seem a bit of a faff when all you want is to provision a single virtual machine for development purposes. If you omit this directive, according to the documentation found here, Vagrant will automatically create an inventory file for all the machines it controls. I did not try this and I like to have precise control over which machines are being provisioned, so therefore I explicitly specify the inventory_path.

The inventory file

The inventory file in my case is simply called inventory and has the following contents:

Note that the ip in the inventory file corresponds with the ip of the private network specified in the Vagrantfile. In here, default is the name of the machine that will be used in the playbook.

The playbook

Since I started blogging about Ansible, I have successfully tried a few different playbooks, but they where all built up gradually, with trial and error. I do not tend to follow large, elaborate examples, but instead, I add small steps to my playbooks, going from error to error until they do what I want. While this makes me understand every single step in my playbook, it also leads to inefficient playbooks, where tasks that could easily be grouped together are scattered around the playbook, because they were added to it at will, when I needed them. Another drawback of what I used so far, is that I came from a background of using shell scripts for provisioning and in some cases I have literally translated what I had in the shell script to an Ansible task. While shell scripts are a fine and valid way to provision your VM, their approach is different from that of Ansible. If you translate a command from a shell script to a very similar task in Ansible, it is more than likely that you will not be tapping in to the full potential of Ansible modules.

From the above, you may already have guessed that Ansible playbooks are built out of tasks by using modules. A task is something you want to be done on the target machine and a module is a piece of software built into Ansible that make it easy for you to specify a certain task.

With that out of the way, let me try a very simple playbook while trying to be as proficient with modules as I possibly can.

A simple example playbook

To try the example playbook I will start from scratch, so that I do not accidentally omit any steps here. I open up a terminal window, go to my home directory and type

and then

to enter the newly created directory. In there, I will initialize a new vagrant box, based on the base box I created earlier.

This should give you the following output:

This will initialize a new vagrant environment based on the base box that I created before. ‘base’ in this case is just how I named my base box.

Now I want to make a few adjustments to the Vagrantfile to prepare it for Ansible, but first I want to have a go at vagrant up, just to verify I did not make any mistakes and my machine will be up and running just fine.

The output should look similar to this:

From the output, you can see that the machine has been assigned the local ip address 192.168.4.193. In order to make sure it gets the same ip address next time it is booted, you should specify this ip as the private network ip in the Vagrantfile.

My Vagrantfile now lives in ~/vagrant-blog, I open it and uncomment the line that specifies the private network. I also change the ip address, so that the line now reads:

Now it’s time to add the inventory file, that I will name inventory once more. And this goes in the inventory file:

There’s two more steps, as we have learned above: the section in the Vagrantfile that configures the provisioning and the playbook itself. If you name the playbook playbook.yml you can use the configuration exactly as described above, but I will add it here just for completeness’ sake.

With that in our Vagrantfile, we are now ready to write our first simple playbook. We will make the playbook install the Apache webserver and display the default website in a browser on the host machine. Please note that my approach to playbooks is really, really simple. You can read more about playbooks on docs.ansible.com, but beware it might dazzle you, so if you only need a development environment in Vagrant, you might want to stick with really simple. For a different approach and a very clear explanation, you may also want to read this post by Adam Brett.

The playbook:

If your vagrant instance is already runnig, you may now run

If it is not running, you may bring it up and it will be provisioned, because it has never been provisioned before:

If you want to force provisioning when bringing it up, you may type:

Now the machine will be provisioned using ansible and when it’s ready, you may fire up a browser on your host machine and go to http://192.168.4.193/ to see the Apache default page.

Now this is very basic of course, so in my next post, I will follow up explaining an Ansible playbook that will compile PHP 7 for you on Ubuntu 14. Stay tuned!

Using Vagrant with WMware Fusion: vagrant-vmware-fusion

This is part 1 of my series on Ansible provisioning for Vagrant running VMware.

In this post I will describe how things are supposed to work first time right when you run a VMware virtual machine using Vagrant. If you are interested in what could go wrong, or if you encounter errors yourself, you are advised to read [todo: I will follow up with a post on what went wrong].

Please note that Vagrant is most widely used with VirtualBox. If you are using Vagrant with VirtualBox and you are happy with that, there is no reason why you should try VMware.

The plugin: vagrant-vmware-fusion

While Vagrant works with VirtualBox by default, you need a plugin for it to work with VMware. There are two VMware providers available, one for VMware Fusion (used on the Mac) and one for WMware Workstation (used on Windows and Linux). The provider plugin that I will be using is called vagrant-vmware-fusion.

VMware and the vagrant-vmware-fusion plugin both are not free, unlike VirtualBox.

My software environment

If you made it till here you might be interested in actually following along, so I might as well tell you what software I am using: I’m on Mac OS X Mavericks (version 10.9.2). My Vagrant is at 1.5.4 and my vagrant-vmware-fusion plugin is version 2.4.0. My VMware Fusion installation is at version 6.0.3.

In case you haven’t got Vagrant installed, you must install it using the packaged installer, which you can find here: http://www.vagrantup.com/downloads.html

VMware Fusion can be downloaded here: http://www.vmware.com/products/fusion/

The vagrant-vmware-fusion plugin can be purchased here (there is no trial version): http://www.vagrantup.com/vmware

Installation

The software installations are straightforward, except for the plugin. It is done on the command line, after you have installed both Vagrant and VMware Fusion. Make sure you have the vagrant command by typing

In my case, this outputs:

Because there is no trial version of the plugin, you have to purchase a license before you can continue. Although there is no trial period, purchasing the license is risk-free. Should you not be satisfied with your purchase, a single e-mail to its vendors will suffice to get your money back and have the license revoked.

All you will download after purchasing the license is a license file. Next, you may proceed installing the plugin by following the installation instructions. Make sure that you keep the license file somewhere and make notes of it.

To verify if the plugin is installed, type:

In my case that yields:

Adding and starting your VM

Now that you are all set, you will need a virtual machine to bring up using the famous vagrant up command. The good news is you can download a base box from the vagrant website: I downloaded the Ubuntu precise64 VMware box base box for vmware.

Place the base box somewhere on your file system and cd into that directory on the command line. Your next step is to add the box to Vagrant:

This should give you the following output (except for the directory where you have placed the base box):

And then:

[Edit: July 12, 2014] I received feedback from Jim Preston, that as of Vagrant 1.6.8 you need to include the box name with the init command, like so:

I haven’t yet tried this myself, but I thought it would be wise to include it here, since otherwise you might get stuck on this bit.[/Edit]

[Edit: July 20, 2014]I installed Vagrant 1.6.3 and verified whether the name of the box was needed with the init command and in my case it wasn’t needed.[/Edit]

Which results in:

And finally:

Do not forget the –provider flag. You may omit it later, but the first time you bring up your box, you need it. The above command will yield the following output if everything went ok:

When you’re done:

That’s it! Now you know how to use Vagrant and VMware together. If you would like to know how to provision your new box using Ansible, then please read my next post [todo].

Ansible provisioning for Vagrant running VMware

Introduction

This is the beginning of a series of posts about using Vagrant with WMware virtualization and Ansible provisioning. That’s three technologies in one sentence. Combining technologies is always challenging, especially if you have little to no experience with some of them. This means that things will go wrong many times during my research and it will be far from a smooth experience. Although I would have loved to describe my adventure in chronological order, this would be annoying, as you would most certainly lose track of how things are supposed to work.

So instead of mixing how things went wrong with how things should have worked the first time, I will present you with posts describing my project as if everything went right the first time. For those interested in my personal struggle, the amount of effort involved and especially those encountering errors themselves, I will post about what went wrong separately.

The idea and the technologies

Let me start with a brief explanation of each technology and my project.

Vagrant

Vagrant is a technology that allows you to script the installation and configuration of a virtual machine. Because it’s scripted, it’s repeatable, distributable and it can be kept under version control. The major benefits are experienced in development, where you can keep multiple development environments on a single host machine (your development laptop or whatever you use). You can share these environments with your team and use version control to keep them in sync between you and your team mates. Of course, you will build your vagrant setup so that it resembles your production environment as close as possible. A machine is built from a base box, which is installed and configured using a provisioner. This can be repeated over and over again, but once provisioned there is no need to do it all again: Vagrant will notice if you made any changes to your provisioning script. Vagrant is most widely used with VirtualBox, which is a free (as in beer) virtualization tool. Base boxes can be obtained as downloads from the vagrant website and other sources you might decide to trust.

VMware

VMware is a virtualization brand, like VirtualBox.  Virtualization allows you to run a computer (the guest) inside another computer (the host). Virtualization allows you to have as many machines on your host machine as you have room for in terms of disk space. You run them when you need them and shut them down or suspend them when you’re done. You may run multiple virtual machines at the same time, as long as physical memory is available. While virtualization by itself allows you to have multiple different development environments on the same machine, it is not so easily shared and distributed as when combined with Vagrant. With Vagrant, you only need to share the scripts, not the machines. Also, you cannot keep virtual machines under version control, as they are mostly binary representations of disk states.

VMware is a drop-in replacement for VirtualBox and it’s not free. On windows and linux, you use WMware Workstation, while on the Mac, you use VMware Fusion. Vagrant uses a plugin, vagrant-vmware-fusion, to work with VMware on Mac OSX. The vagrant-vmware-fusion plugin is also not free.

Ansible

The most commonly know provisioners for Vagrant are shell, Chef and Puppet. Shell means just basic shell commands that you would type yourself if you are doing manual installation and configuration. The other two are sophisticated provisioning systems with quite a learning curve. Ansible is a provisioning tool like the latter two, but it is simple and powerful at the same time.

My project

There is a lot of buzz around Vagrant. The first time I saw someone use it was during MidwestPHP in March 2013, where Chris Tankersly used it in his presentation. Like everyone else mentioning Vagrant he talked about it rather casually, saying that ‘vagrant up’ gives you a complete development environment with almost no effort. Whenever I hear someone talk about a new technology in such a way, I get suspicious. In my experience, good things rarely come with ‘almost no effort’. I was quite convinced, that if at that moment I would type ‘vagrant up’ in some terminal window on my machine, it would give me absolutely nothing. Every tool needs the right context to work: More about that later in this series.

Why VMWare?

On my Mac, with OSX Mavericks installed, I use VMware Fusion for virtualization. Because I come from a Windows history, I keep a Windows 7 VM, so that I can still run applications that I purchased for Windows. At the time when I started using virtualization I was still working on Windows and used VMware Workstation. When I moved to OSX, VMware Fusion was the promoted choice and I didn’t know of the existence of VirtualBox in the first place. In addition, when I started using Vagrant about half a year ago, I wasn’t completely satisfied with how VirtualBox worked. This was mostly due to unfamiliarity with it’s UI and some usability issues I was experiencing with it, like not being able to cancel a dialog. For my daily work however, I do work with VirtualBox, because my team uses it and I want to be able to share their Vagrant configuration and base box. Also, I did not have the time to become familiar with using the vagrant-vmware-fusion plugin when the project started at a fast pace. In spite of the fact that by now, VirtualBox is doing a good job in virtualizing my two different development environments, I still want Vagrant to work with VMware fusion. I bought all the licenses, I like the UI and the overall stability of VMware, so now I want to work with it.

With Ansible, we get to the most interesting part of this series. With Ansible, I have no experience at all. It is a configuration management system that you can use to spin up machines in the cloud, or spin up your own local virtualized development environment. As such, you can use it with Vagrant. Ansible allows me to skip learning supposedly more complicated systems like Chef and Puppet, while at the same getting to understand a sophisticated tool. I fell in love with Ansible during a presentation by Jeremy Coates at PHP Benelux 2014. His enthusiasm was as obvious as contagious.

So, because Vagrant is a cool technology that everybody uses for good reasons and because I like VMware and because I am interested in learning Ansible, I decided I wanted to use them all three together. While there is excellent documentation available on each of these technologies, it’s not easy to find a step by step guide on how to use them together, so I decided to fill that gap with a series of posts. Keep an eye on this blog over the next month to read them all.

To start with using Vagrant and VMware together, read part 1 of this series: Using Vagrant with WMware Fusion: vagrant-vmware-fusion

Setup logging in application.ini for Zend Framework 1

Today I had to look it up again: how to quickly setup logging for a Zend Framework 1 application. To make it easy for myself to find it again, should I ever need it, I will post it here.

A log is a resource that ZF 1 can load using a predefined resource plugin. That means that we can be lazy: If we setup a logger as a resource called ‘log’ in application.ini, we can use it.

But then, when I started writing this, I stumbled on this blog post, that does exactly the same thing, so just read it here:

http://blog.niklasottosson.com/?p=1158

What this post doesn’t show you is how to add logging to firePHP in the same go:

Logging with FirePHP in ZF2

Yesterday I wanted to set up logging to FirePHP in CuddleFish (my pet project business framework). I thought it would be easy, and looking back, it is very easy. Before I figured out how to set it up, it was not so easy, so I’d better blog about it to remind myself of how it’s done.

First of all, you have to set up FirePHP. You can download it from the FirePHP website or install it using pear:

Next, in the Module.php file of your module, you can add a service that provides the configured logger:

Simplifying it

In fact, since we require_once the core FirePHP class, there is no need to pass it to the constructor of the writer using the bridge. I only came to realize this while writing this post. When the FirePHP class exists, it will be used automatically by the writer. So the simplified code becomes:

Now if you need the log somewhere, say in a controller, you can pass it in using the factory for the controller and call:

Alternatively you can call getServiceLocator() inside the controller to get the ServiceManager and get the log instance:

There is however a downside to this: If you have a lot of calls to the log, it is more code to type than just calling $this->log->info(‘something’);

If you have an initalizer in your controller, you can set the log there:

If you use a factory to create your controller, you can add a method setLog() to your controller:

The factory can be a closure in the controller configuration, like we did for the log:

Now the log is available as $this->log right after instantiation of the controller.

 

Unit testing Zend Framework 1.x with PHPStorm

Warning: This post is outdated. If you want something more up-to-date: The Dutch Web Alliance has an excellent series on unit testing and debugging with PHPStorm, by Joshua Thijssen

In case I ever forget, to easisly install the latest phpunit, follow the manual blindly, which means to:

At the moment of writing, the above yields an error, complaining about Strutures\Graph.php. The solution is to get the Structures_Graph tar by typing:

This will yield an error once more, but you can see where the tar file is downloaded to. You can extract it manually and then copy the Structures directory into your PEAR directory (at the same level as the ‘Console’ directory). After that, you can

Once pear is up to date, configure it to auto discover channels. This is really important! And the next step is install phpunit.

NOTE that for ZF 1.x you still need to specify the correct version of phpunit:

PHPStorm context menu detail
Run option in PHPStorm context menu

Last night I wanted to run a unit test for Zend Framework 1.11. I only recently switched from ZendStudio to PHPStorm and I didn’t yet run any PHPUnit unit tests from inside PHPStorm. I always liked running unit tests in ZendStudio, although this wasn’t always easy to setup, so I feared the new learning curve ahead of me. This wasn’t quite as hard as I imagined: like anything else in PHPStorm, it is easy to find and easy to setup. Just in case I forget how it all works, I write this down.

First of all, there are some differences in how PHPStorm and ZendStudio approach unit testing. In ZendStudio, you have to select ‘run as PHPUnit test’ from the context menu. The context menu is what you see when you click your right mouse button on the code you are editing and that is also the file that contains the unit test that you want to run. In PHPStorm, you just choose ‘Run’ from the context menu (see image).

Another option in the PHPStorm context menu is to setup a run configuration for the file. Once it is setup,  you will be able to select the run configuration from the menu bar and you get the opportunity to edit it as well. When setting up a run configuration, you can define the scope: Directory, Class,  Method or As Defined in the Configuration File, which is absolutely a great feature.

One of the things that will go wrong in ZendStudio is that you forget to include PHPUnit on the include path of your project, in which case ZendStudio cannot find the PHPUnit library. It will not tell you, it will just not run. PHPStorm will tell you it could not connect a testreporter if it can’t find PHPUnit. This is the exact error message: “Unable to attach test reporter to test framework or test framework quit unexpectedly”. In that case, you don’t have installed PHPUnit for the php version PHPStorm is using. In case you have several versions of php available on your system, you can configure which one PHPStorm should use. This is much the same as it is in ZendStudio. See the next image for how it is done in PHPStorm. In the same dialog, you can add entries to the include path for your php version. Make sure you add the path where PHPUnit lives (in my case that is “/Applications/MAMP/bin/php/php5.3.6/lib”).

Setting the php version in PHPStorm
Setting the php version in PHPStorm

One nifty little feature here is the little ‘i’ button, that brings up all ini settings if you click it. These are editable, but I did not check if that actually takes effect. To change the interpreter, you click the button with the three dots to select the path to your php. By clicking the ‘+’ button you can add entries to the include path.

Setting up PHPUnit

PHPUnit can be easily installed using PEAR. I once had this setup, but because I alwasy used ZendStudio, I had no need to keep it up and running an my installation had been broken by installing MAMP Pro over MAMP. I remembered that installing PHPUnit was a pain, because the pear installer that had shipped with MAMP was to old for the PHPUnit installation. Luckily, that problem did not re-occur.  I could easily install phpunit with pear, but I did no longer have the pear command. I broke that along with the same MAMP Pro installation.

Setup the pear command

To get the pear command back, the directory where it is located must be on your computers PATH. This is not the php include_path, but the PATH environment variable of your system. The PATH variable is set in your .profile file, that is located in your home directory. In my case, the home directory is /Users/bartmcleod. To edit the profile file, just open up terminal and type:

Inside .profile you will see something like:

I changed that to:

to include the location where pear installs it’s commands. Once you have done this, you can proceed to install PHPUnit and you will also have the phpunit command available, since that goes into the same directory.

Installing PHPUnit for Zend Framework 1.x

You will think that installing PHPUnit with PEAR is easy: something along the lines of ‘sudo pear install phpunit’. Of course I tried that first and it executes and everything looks fine, except it didn’t really work for me. Then I found I had to type ‘sudo pear install phpunit/PHPUnit’. That worked, but it didn’t work with Zend Framework 1.x unit tests. They do not support PHPUnit 3.6. I found that the most recent version they support is PHPUnit 3.5.15. So I tried to uninstall phpunit and subsequently install that specific version, using the instructions I found here: http://stackoverflow.com/questions/1569012/how-to-install-an-older-version-of-phpunit-through-pear

In the same article however, it is specifically stated that to install 3.5.15, you have to uninstall separate phpunit components one by one, before installing the specific older version. If you don’t, pear will automatically detect that these components need the latest version and install that instead of the requested older version. This is annoying and hard to believe when you see it happen on the commandline. I had to try several times before I actually believed what was going on.

Because I did not want to type all the commands or paste them one by one, I just created two batch scripts. One to uninstall the latest versions and one to install the 3.5.15 versions.

Uninstall:

Re-install 3.5.15:

The stackoverlow article from the above url gives more details and use cases that might apply to your particular situation, so if this set of commands doesn’t work for you, make sure you check the original post.

If unit testing with PHPUnit still doesn’t work from inside  PHPStorm

I upgraded to PHPStorm 4 and I tried to repeat the above procedure, and it didn’t work immediately. I kept getting the “Unable to attach test reporter to test framework or test framework quit unexpectedly” error message. I had to change the interpreter from php 5.4 (that ships with PHPStorm apparently) to php 5.3 local, like I had it set up before. I verified that the correct phpunit version is still installed, but with no luck. While trying again and again I found it seems to matter where you click inside the code. Clicking on a single test function will allow you to test only that function, while clicking on the class name will allow you to run the entire test class.

display_errors should be on (obviously)

While that is a nice feature, it doesn’t cover all the cases where unit testing doesn’t work. In one case for example, I had display_errors set to ‘off’ (accidentally) and if you do, you get the same error message  (“Unable to attach test reporter to test framework or test framework quit unexpectedly”) while you see nothing else that could help you detect the error.

Mind your slashes!

In yet another case with the same error message, I found I included the test superclass using windows (back)slashes as directory separators. Apparently, I had been developing those tests on a windows box and didn’t use forward slashes. The error message you get in that case is “Warning: require_once(PHPUnit\Framework\TestCase.php): failed to open stream: No such file or directory in …”. You might not notice immediately that the wrong  slashes are used if you run the tests on *nix or a Mac.

Zend Framework 2.0 patterns

Today I came across a presentation by Matthew Weier O’Phinney on slideshare, that I just do not want to forget about. Because bookmarks tend to disappear when I switch browsers and because I create bookmarks but rarely look at them a second time (using 10+ different browser installations and having bookmarks spread all over) I decided to embed the presentation on my blog, so that it sticks. So here it comes:

I have to remember to try this Depency Injection phenomenon I hear so much about lately.

Zend_Tool and Model generation

Zend_Tool is a command line tool that helps you setup a Zend Framework MVC application. You can also use it after you have set up your application to add controllers, actions, views, configuration and models. Since I use Zend Studio for development, I am used to the integrated version of Zend_Tool. This means that if you set up a Zend Framework MVC application as a new Zend Studio project, you will have menus available that let you add things like controllers and actions.

This is really nice of course, but the nature of the tool, that is command-line by birth, makes that I would like to be able to use it from the command-line too. Continue reading “Zend_Tool and Model generation”

Enabling syntax highlighting

Another UPDATE

This post is outdated. I am now using ‘Crayon Syntax Highligther’ instead of ‘Syntax Highlighter Evolved’. I had to update the code samples on this page accordingly, so reading this post might lead to confusion.

UPDATE

There is a newer version of the plugin that I wrote about in this post. It’s functionality is available by default to wordpress.com users. The new plugin is called SyntaxHighlighter Evolved.

SyntaxHighlighter Evolved is compatible with Syntax HighLighter for WordPress, my blog uses SyntaxHighlighter Evolved now. Everything in this post also applies to SyntaxHighlighter Evolved.

I would like to know what happens if I enter some php code. Does the syntax get highlighted by default in WordPress? Is it going to be stripped out? Will it be forbidden? Let me just type “echo ‘Hello World!'” and see what happens:

<?php echo ‘Hello World’; ?>

I expect this to show up just as I typed it, nothing special.

Now let me type it into the html editor, and see what I get then:

(You see nothing here, but on the backend, in the editor there is a hidden html comment that contains the php code. On the front-end, without the editor, the comment gets stripped out completely) Continue reading “Enabling syntax highlighting”

Do not require_once() a directory!

If you see the following error messages on the command line in windows, you know you going to be wasting an uncertain amount of time figuring out the right permissions for the directory D:\ZendFramework:

Warning: require_once(D:\ZendFramework\library\Zend\Loader\Autoloader): failed to open stream: Permission denied in D:\werk7\tiris\loadDocuments.php on line 15

Fatal error: require_once(): Failed opening required ‘Zend/Loader/Autoloader’ (include_path=’.;D:\ZendFramework\library’) in D:\werk7\tiris\loadDocuments.php on line 15

What is happening here? Let me tell you what I was trying to do. I was trying to execute a php script from the command line. Continue reading “Do not require_once() a directory!”