Virtual Micromagnetics Documentation

_images/virtual_micromagnetics_logo.png

Welcome to the Virtual Micromagnetics project, where we aim to enable accessible and reproducible micromagnetics simulation, without compromise.

Provided by Mark Vousden, Hans Fangohr, and others at the University of Southampton. Funded by EPSRC’s DTC grant EP/G03690X/1. The license for this software is available here.

The Virtual Micromagnetics project creates virtual environments that run micromagnetic (and in some cases, atomistic) simulations of magnetic behaviour. These environments produce system virtual machines which emulate a configured set of software on your computer. This means you as a user only need to manage the software to support the virtual machine, as opposed to the complicated set of dependencies most simulation packages require. As a result, these virtual environments are far simpler to maintain, meaning you have more time to solve the mysteries of the universe instead of:

  • wondering why the latest version of a package is incompatible with earlier simulations.
  • wondering how to maintain multiple versions of a package to support old simulation software.
  • persuading your high-performance computing system administrator to support your long list of software dependencies.
  • setting up user accounts and packages for new students to run simulations.

Contents

Getting Started: As a User

To start a Virtual Micromagnetics environment, you will need the following software:

After restarting your machine, command the following in an empty directory:

vagrant init virtualmicromagnetics/full
vagrant up --provider virtualbox

These commands will download the Full Virtual Micromagnetics environment from the Internet to your computer, and load the environment automatically. When complete, you should be greeted with this window:

_images/user-window-1.png

This is output from a virtual machine running on your computer! Virtual machines produced in this way run Ubuntu GNU/Linux (https://www.ubuntu.com) with the XFCE window manager. From here, you can follow instructions in the welcome file on the desktop to run simulations with the installed packages. Never worry about software dependencies again!

_images/user-window-2.png

Next, see Virtual Micromagnetics Environments and Simulation Software for the environments that are available besides the Full Virtual Micromagnetics environment, and the software on these environments.

Virtual Micromagnetics Environments and Simulation Software

In Getting Started: As a User, we created a virtual machine based on the Full Virtual Micromagnetics environment. Here, we detail the micromagnetic simulation packages and other software supported and used by Virtual Micromagnetics, and we describe the list of available Virtual Micromagnetics environments.

Software

The following micromagnetic simulation packages are supported by Virtual Micromagnetics:

Each environment that uses a simulation package also contains its examples, and a link to its documentation page which can be opened in the virtual machine. The following infrastructure software is also used in all environments:

Environments

The Full environment contains all simulation packages supported by Virtual Micromagnetics, as well as dependencies requested by our users, including:

We recognise that many of our users will not require these tools. To that end, a “Lite” environment can be used instead, which contains all of the simulation packages the Full environment does, without these optional packages. To download and use the Lite Virtual Micromagnetics environment, command:

vagrant init virtualmicromagnetics/lite
vagrant up --provider virtualbox

Note that this is syntactically similar to the command used in Getting Started: As a User, and can be adapted for all other Virtual Micromagnetics environments. Environments exist for specific simulation packages, such as “virtualmicromagnetics/oommf”. The following table shows the list of environments available under Virtual Micromagnetics, and the software they contain:

Software vs. Environment Full Lite OOMMF Magpar NMag Fidimag
Micromagnetic OOMMF      
Magpar      
NMag      
Fidimag      
Infrastructure Ubuntu
XFCE
Python 2
IPython
Other Cython          
FEniCS          
Gmsh          
Netgen          
ParaView          
Sundials          

See Getting Started: As a Poweruser if more fine-grained control over your software interests you. However, we firstly recommend reading Virtual Machines and Related Software to understand more about virtual machines, virtual environments, and the software used to create them in the Virtual Micromagnetics project.

Getting Started: As a Poweruser

In Virtual Micromagnetics Environments and Simulation Software, we learned about the different Virtual Micromagnetics environments available to users, which bundle sets of configured software. Here we outline how you can create virtual environments yourself, which you can distribute to others. We recommend reading Virtual Machines and Related Software, if you have not already.

To create a new, custom virtual environment, you will need the following software in addition to the software list in Getting Started: As a User:

With this software:

  1. Grab a copy of Virtual Micromagnetics. You can do this via Git by cloning our repository with git clone -b release https://github.com/computationalmodelling/virtualmicromagnetics.git, or by grabbing a release version at https://github.com/computationalmodelling/virtualmicromagnetics/releases
  2. Install some sub-packages:
    • Ansible role “blockinfile”; command ansible-galaxy install yaegashi.blockinfile -proles/ from the Virtual Micromagnetics software directory
    • Vagrant plugin “vagrant-vbguest”; command vagrant plugin install vagrant-vbguest
  3. Make, with make from the Virtual Micromagnetics software directory, with an Internet connection.

After time of the order of hours, you should find a box file at ./artefacts/virtualmicromagnetics-full-*.box. If not, see Troubleshooting for more help. If so, congratulations on building your first Virtual Micromagnetics environment! This file represents your virtual environment, which you can share with other users. You can use this environment yourself by commanding the following in an empty directory:

vagrant init $PATH_TO_BOX_FILE
vagrant up --provider virtualbox

where $PATH_TO_BOX_FILE is the aforementioned artefact file. Now that you can create virtual environments, see Developer Notes (valid for version 1.0.2) to learn how to customise the software you place on them.

Developer Notes (valid for version 1.0.2)

In Getting Started: As a Poweruser, we created a virtual environment from scratch that can be shared with other users. Here, we show how you can completely specify your own environment. Knowledge of Ansible is needed, which can be gleaned from their excellent documentation at http://docs.ansible.com/ansible/.

Overview: The Build and Run Processes

This graph shows the operations involved in the build and run processes.

_images/graph.png

Lets break this down:

  • Input Environment -> Developer’s Virtual Machine: The input environment is a virtual environment containing only the operating system and few convenience tools. In development, Vagrant and VirtualBox create a virtual machine from this environment in the create_vm role (see The Build Process).
  • Ansible Provisions: Vagrant commands Ansible to provision this machine using an Ansible playbook.
  • Developer’s Virtual Machine -> Output Environment: Vagrant then packages the virtual machine into a new virtual environment, which can be distributed to others. Tagged releases are uploaded by administrators to atlas.hashicorp.com, where they become available to all Vagrant users.
  • Output Environment -> User’s Virtual Machine: The previous steps are run in the build phase by a developer as the build process. This step represents the user following the instructions in Getting Started: As a User to create a virtual machine for themselves, which is referred to as the run process.

The run process performed by the user has been documented in Getting Started: As a User. Here we introduce the build process, which represents the other section of this diagram.

The Build Process

The build (make) process in step 3 in Getting Started: As a Poweruser allowed us to create a virtual environment. The Makefile in the software repository can build multiple targets. Each target runs Ansible on the master.yml playbook, which in turn runs the create_vm role in the roles directory. This creates a virtual machine and provisions it with the playbook passed as a command-line argument in Makefile, which lives in the jobs directory. It will also do some post-provisioning tasks using the hookbook, again passed as a command-line argument. The fundamental difference between the playbook and the hookbook is that the playbook is run on the guest virtual machine by vagrant, and the hookbook is run on the host machine. Different Makefile targets may place different build artefacts in the artefacts directory.

Roles add or configure software, playbooks describe the roles that must be enacted to provision the machine, hookbooks describe what to do with that machine (like creating a box file), and jobs are Makefile targets that produce certain machines.

To add a new environment, one needs to add a job that follows the pattern of existing jobs.

Where Things Are

In order to add jobs, one should edit Makefile. In order to do that, one would need to know where things are, hence the purpose of this section. The virtual micromagnetics repository is structured as follows:

  • Makefile: This is the Makefile through which all jobs are conducted.
  • ansible.cfg and inventory.txt: These files are used by Ansible when the master.yml playbook is run. They contain configuration information.
  • roles/: This directory contains roles (obviously). Each role is given a subdirectory, and should not overlap. Each role directory contains tasks, and may also contain the subdirectories:
    • vars/ (variable definitions),
    • templates/ (files to duplicate to the guest virtual machine),
    • meta/ (metadata, such as role dependencies),
    • files/ (files used by tasks that aren’t covered by the usecases of templates)
  • jobs/: This directory contains playbooks and directories that can be thought of as jobs in Makefile. They are either provisioning playbooks, or post-provisioning hookbooks.
  • machines/: This directory is created by Makefile, and houses the vagrant environment for each individual virtual machine. The provision process is recorded to a log file in the machine’s directory (for example, the provision log for the lite build job exists in machines/virtualmicromagnetics-lite/virtualmicromagnetics-lite.log)
  • artefacts/: This directory is created by Makefile, and houses build artefacts.

Examples

Create New Machine with Existing Software

Lets create a custom machine called doc-example, that contains Fidimag but no X server. Firstly, we add a target to Makefile (append the following to the Makefile):

# This target builds a virtual hard disk file containing an OOMMF and Fidimag
# installation.
doc-example:
    ansible-playbook master.yml -c local -i localhost, -v -k --extra-vars="vm_name=virtualmicromagnetics-doc-example playbook=provision_virtualmicromagnetics_doc-example.yml hookbook=hook.yml extra_resources_dir=guest_resources/"

Now we need to describe what the state of the machine should be, by writing the playbook jobs/provision_virtualmicromagnetics_doc-examples.yml:

---
# This Ansible playbook is a provision playbook designed to be used with
# vagrant. This playbook provisions a machine suitable for micromagnetic
# simulation with Fidimag. It is executed by the virtual machine.

- hosts: all

  vars:
    vm_name: virtualmicromagnetics-doc-example

  roles:
    - fidimag
    - fidimag_examples
    - add_super_user
    - { role: set_hostname, HOSTNAME: {{ vm_name }} }

Now we are ready to build the environment by commanding (again, from the repository root directory):

make doc-example

This creates another virtual environment in the artefacts directory.

Adding Software

In The Build Process, we introduce roles. Roles can add new software to a virtual environment. By way of example, we can create a role to install Emacs (https://www.gnu.org/software/emacs/) from the Ubuntu software repository. We firstly create a directory structure:

# Create a role for Emacs.
mkdir --parents roles/emacs/tasks

Now we introduce some content using information from the Ansible documentation (http://docs.ansible.com/ansible/, and http://docs.ansible.com/ansible/apt_module.html). Write the following to roles/emacs/tasks/main.yml:

---
# This Ansible playbook installs Emacs.

- name: Install Emacs.
  apt:
    pkg=emacs
    state=latest
    update_cache=yes
    cache_valid_time=86400
  sudo: yes

This role, when run, will ensure that the latest version of Emacs and its dependencies are installed on the virtual machine, and updates the apt cache. Roles can be parameterised and have dependencies, which can cause them to become complicated. By way of example, installing Emacs on the new doc-example environment requires us to append the line:

- emacs

To clarify, playbook jobs/provision_virtualmicromagnetics_doc-examples.yml now looks like:

---
# This Ansible playbook is a provision playbook designed to be used with
# vagrant. This playbook provisions a machine suitable for micromagnetic
# simulation with fidimag. It is executed by the virtual machine.

- hosts: all

  vars:
    vm_name: virtualmicromagnetics-doc-example

  roles:
    - fidimag
    - fidimag_examples
    - add_super_user
    - { role: set_hostname, HOSTNAME: {{ vm_name }} }
    - emacs
Further Tinkering

We have explored how a new virtual environment can be created, and how new software can be added. In this section, we describe how the virtual machine itself can be configured using Vagrant’s parameters. Vagrantfiles are files used by Vagrant written using Ruby syntax. These files specify parameters of the virtual machine created from a virtual environment. When running the commands in Getting Started: As a User, we create a Vagrantfile in the working directory that describes the virtual machine to Vagrant. Vagrantfiles can also be built into a virtual environment. Built-in Vagrantfiles can be found in guest_resources/vagrantfiles.

For example, if you wish to specify that 2048MB of memory must be used in the virtual machine created in Create New Machine with Existing Software [1], we can add a builtin Vagrantfile at guest_resources/vagrantfiles/Vagrantfile_virtualmicromagnetics-doc-example_builtin with the following content:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.provider :virtualbox do |vb|
    vb.memory = 2048
  end

This Vagrantfile will be detected by the hookbook and included automatically when the environment is packaged. For more information on Vagrantfiles, see the Vagrant documentation (https://www.vagrantup.com/docs/vagrantfile/).

[1]Note that this is not such a good idea if you want to distribute your environment to different users, since they may have a different amount of available memory to you.

Summary and Final Words

To summarise, virtual environments are created from an empty Ubuntu virtual machine after being provisioned and packaged. This build process allows the user to create a Virtual Micromagnetics virtual machine using Vagrant and VirtualBox. We have also presented how a new environment can be created, how the software of that environment can be controlled, and how the virtual machines can be parameterised.

Thank you for using Virtual Micromagnetics! If you create roles for your favourite software, consider sharing them with the community. You can create a pull request at our GitHub repository at https://github.com/computationalmodelling/virtualmicromagnetics, or contacting Mark at mark[dot]vousden[at]soton[dot]ac[dot]uk.

Troubleshooting

Currently there is not troubleshooting information for Virtual Micromagnetics. If you have an issue, please add it to our issue tracker at https://github.com/computationalmodelling/virtualmicromagnetics/issues, or email mark[dot]vousden[at]soton[dot]ac[dot]uk. We will endeavour to resolve the issue and add it here if it is widespread.

Glossary

Box File
A file used by Vagrant to create Vagrant environments. A box file represents a template from which Virtual machines can be created.
Host
Host Machine
A machine that uses virtualisation software to host virtual machines. Host machines allocate resources, including memory and disk space, to support the running of a virtual machine. Note that a virtual machine can host other virtual machines, making it both a host and virtual machine.
Idempotent
Idempotency
Idempotent provisioning runs only the commands required to achieve the desired state of the system. Commands that do not change the state are not run. This can save considerable build time when handling dependencies, because a dependency should not be downloaded and reinstalled if it is already in the state it should be in.
Manager
Vagrant
Manager software provides an interface for managing Virtual machines from Virtual environments. Vagrant is an example of manager software, which provides a command-line interface to virtual machine management which can be automated. See Managers.
Vagrant Environment
The directory structure (including Vagrantfile) created by Vagrant when vagrant init is commanded. A single virtual machine can be managed in a given environment.
Virtual Environment
Virtual environments produce the same virtual machine on all virtualisation-capable computers. Virtual environments are also box files, but are not Vagrant environments.
Provider
Virtualiser
VirtualBox
Provider (or virtualiser) software supports the creation of, and interaction with, virtual machines from a host machine. VirtualBox is an example of a free provider software. See Providers.
Provisioner
Provisioning
Ansible
Provisioner software runs a set of commands on a machine, virtual or otherwise, to ensure it is in a particular state. Ansible is an example of provisioner software that enables idempotent provisioning. See Provisioners.
System Virtual Machine
Virtual Machine

“An efficient, isolated duplicate of a real machine.” [1]

Software that imitates certain other software on certain hardware. In this project, this includes a complete operating system, and a combination of one or many simulation packages and dependencies. General system virtual machines are described in brief at Virtual Machines and Related Software.

More strictly, a virtual machine is a specific instance of a virtual environment. When we build box file artefacts, we are creating virtual environments, not virtual machines. Once Vagrant creates a Vagrant environment from a box file and vagrant up is commanded, a corresponding virtual machine is created. Virtual machines created on a host are managed by virtual machine providers, which typically list the machines they are maintaining.

Virtual Micromagnetics

“Enabling accessible and reproducible micromagnetic simulation.”

The name of this project, which represents the collection of virtual environments and the software written to create them.

References

[1]Smith, J., Nair, R. (2005). “The Architecture of Virtual Machines”. Computer (IEEE Computer Society) 38 (5): 32–38. doi:10.1109/MC.2005.173
_images/oommf-simulation-1.png

OOMMF running in Virtual Micromagnetics.