Welcome to nebulizer’s documentation!¶
Nebulizer is a Python utility which provides a high-level interactive command-line interface to remotely administer Galaxy servers.
Nebulizer was developed as a tool to help “part-time” Galaxy admins perform day-to-day administrative tasks across multiple Galaxy instances, enabling various admin operations to be executed via the command line as an alternative to using the Galaxy web interface.
Nebulizer is built on top of the Bioblend library and offers an interface and range of functionality that complements the lower-level interfaces offered by Ephemeris and Parsec.
Contents:
Command-line utilities to help with managing users, data libraries and tools in a Galaxy instance, using the Galaxy API via the Bioblend library.
- Free software: Academic Free License version 3.0
- Documentation: https://nebulizer.readthedocs.io
- Code: https://github.com/pjbriggs/nebulizer
Note
Nebulizer is still a work in progress.
Please exercise caution when attempting irreversible operations, especially against production Galaxy instances (for example when creating users or data libraries).
Quick Start¶
This quick start gives some examples of using nebulizer
commands
to perform remote administration tasks on a Galaxy instance from the
command line.
Getting Nebulizer¶
It is recommended to install Nebulizer via pip
in a virtualenv,
for example:
% virtualenv .venv
% source .venv/bin/activate
% pip install nebulizer
This will provide an executable called nebulizer
with a number
of subcommands for performing different tasks remotely on Galaxy
instances.
Nebulizer Basics¶
Generally Nebulizer commands take the form:
nebulizer COMMAND GALAXY [OPTIONS]
To interact remotely with a Galaxy instance using Nebulizer requires at minimum the URL of the instance and then either an API key or a user login name.
For example to list the data libraries available on Galaxy Main:
nebulizer -k 9b376af2250818d14949b3c list_libraries https://usegalaxy.org
or
nebulizer -u USER@DOMAIN list_libraries https://usegalaxy.org
In this second case Nebulizer will prompt for the Galaxy
password to authenticate the user login, unless it’s supplied via
the -P
option.
To store the Galaxy URL-API key pair against an alias main
, to
avoid needing full authentication details each time:
nebulizer add_key main https://usegalaxy.org 9b376af2250818d14949b3c
or alternatively get Nebulizer to fetch the API key itself by supplying the user login:
nebulizer -u USER@DOMAIN add_key main https://usegalaxy.org
More information on managing API keys in Nebulizer can found here.
The stored alias is then used in subsequent commands, for example to list the data libraries again it is now sufficient to do just:
nebulizer list_libraries main
The following sections contain examples of how Nebulizer might be used to perform various administrive tasks.
Nebulizer provides subcommands to perform various administrive tasks:
list_users
create_user
create_batch_users
create_users_from_file
delete_user
list_libraries
create_library
create_library_folder
add_library_datasets
list_tools
list_tool_panel
install_tool
update_tool
uninstall_tool
search_toolshed
quotas
quota_add
quota_mod
quota_del
ping
(check if a Galaxy instance is alive)config
(fetch configuration for a Galaxy instance)
See the tutorial for a walkthrough some of these commands.
Installation¶
Virtualenv and pip¶
For a traditional installation of Nebulizer, first set up a Python
virtualenv (this example creates a new one in .venv
) and then
install with pip
:
$ virtualenv .venv;
$ source .venv/bin/activate
$ pip install nebulizer
When installed this way, Nebulizer can be upgraded as follows:
$ . .venv/bin/activate
$ pip install -U nebulizer
To install or update to the latest development branch of Nebulizer
with pip
, use the following pip install
idiom instead:
$ pip install -U git+git://github.com/pjbriggs/nebulizer.git@devel
Nebulizer should work with recent Python 3 versions.
Conda¶
Nebulizer can be installed using Conda (most easily obtained via the Miniconda Python distribution):
$ conda config --add channels bioconda
$ conda install nebulizer
Note that the version available via bioconda
may lag the
most recent version available via pip
.
Configuration¶
Nebulizer doesn’t require any additional configuration after installation: it is possible to supply either a Galaxy API key or a Galaxy username/password combination on the command line (alongside the URL of the Galaxy instance) each time a command is executed.
However this is both laborious and potentially insecure.
Nebulizer can store Galaxy URL-API key pairs against aliases
locally to make this easier, using the add_key
command.
The simplest way to create a new alias is:
nebulizer -u EMAIL add_key ALIAS GALAXY_URL
Nebulizer will prompt for the password for the account and will then fetch and store the API key automatically.
Note
For example: to add an alias main
for Galaxy main at
https://usegalaxy:
nebulizer -u peter.briggs@manchester.ac.uk add_key main https://usegalaxy.org
Alternatively you can supply an API directly on the command line:
nebulizer add_key ALIAS GALAXY_URL API_KEY
Note
For example:
nebulizer add_key main https://usegalaxy.org 9b376af2250818d14949b3c
The stored alias can then be used instead of supplying the
full Galaxy URL with an email/password combination or an
API key; for example to get configuration information from
Galaxy main using the config
command:
nebulizer config main
See Storing and managing Galaxy API keys for more information on managing stored aliases.
Managing Users¶
Querying user information¶
list_users
displays user emails and names in a Galaxy
instance:
nebulizer list_users GALAXY
-l
: returns extended information for each user (status, whether they are an admin user, disk usage and quota size and usage).--status
: filter list on user status, which can be one ofactive
(the default),deleted
(only list deleted users which haven’t been purged) orall
(list all active, deleted, and purged users).--sort
: specify one or more fields to sort the output on; valid fields areemail
,disk_usage
,quota
,quota_usage
. Multiple fields should be separated by commas (e.g.--sort=quota,disk_usage
).--name
: filter list on user email (can include glob-style wildcards e.g.--name="*bloggs*"
).
Creating and deleting individual users¶
create_user
makes a new user account:
nebulizer create_user GALAXY USER@DOMAIN [ PUBLIC_NAME ]
If PUBLIC_NAME
is not supplied then one will be
generated automatically from the email address.
-c
: check if the account already exists-p
: supply password for the new account (otherwise Nebulizer will prompt for a password)
delete_user
removes a user account:
nebulizer delete_user GALAXY USER@DOMAIN
--purge
: purge the account as well as deleting (or purge an unpurged account which has been previously deleted)
Note
Purging a user account overwrites the email and username for that account with random strings; it also marks the datasets and histories associated with that account as deleted. These data can then be removed from disk by running Galaxy’s clean-up scripts.
For information on the clean-up scripts see the Galaxy documentation at https://galaxyproject.org/admin/config/performance/purge-histories-and-datasets/
Creating batches of users from a template name¶
create_batch_users
makes a set of user accounts
based on a template name:
nebulizer create_batch_users GALAXY TEMPLATE [START] END
The template email address should include a #
symbol
which acts as as a placeholder for an integer index, e.g.
user#@example.org
The integer indices are generate from the range
START...END
(if START
is not supplied then it is
set to 1).
For example:
nebulizer create_batch_users user#@example.org 1 5
creates accounts:
user1@galaxy.org
user2@galaxy.org
user3@galaxy.org
user4@galaxy.org
user5@galaxy.org
-c
: check if the accounts already exist-p
: supply password for the new accounts (otherwise Nebulizer will prompt for a password); the same password will be applied to all the new accounts
Creating user accounts from a file¶
create_users_from_file
makes a set of user accounts
using data from a file:
nebulizer create_users_from_file GALAXY USERS_FILE
USER_FILE
is a tab-delimited file with the following
fields on each line:
email|password|public_name
defining a new account.
Note
Optionally public_name
can be left out and will
then be generated automatically.
-c
: check if the accounts already exist
Managing Data Libraries¶
Querying data libraries¶
list_libraries
displays information about the data libraries
and their contents in a Galaxy instance:
nebulizer list_libraries GALAXY [ LIBRARY[/PATH...] ]
By default all the libraries in the instance are listed; if the
name of a data library (LIBRARY
) is included then the datasets
and folders in that library are listed. If a PATH
(names of
subfolders separated by /
) is supplied then the contents of
the matching folder are displayed.
For example: listing the data libraries in Galaxy main:
nebulizer list_libraries https://usegalaxy.org
List the contents of the “Tutorials” data library:
nebulizer list_libraries https://usegalaxy.org Tutorials
List the contents of the “ChIP_seq (Reb1)” folder in the “Tutorials” data library:
nebulizer list_libraries https://usegalaxy.org "Tutorial/ChIP_seq (Reb1)"
-l
: list extended information about libraries, folders and datasets
Creating and populating data libraries¶
create_library
creates a new data library:
nebulizer create_library GALAXY LIBRARY_NAME [--description DESCRIPTION] [--synopsis SYNOPSIS]
create_library_folder
creates a new folder in an existing
data library or folder:
nebulizer create_library_folder GALAXY LIBRARY_NAME/[PATH]
For example: create a data library called NGS data
and a folder
Run 21
within it:
nebulizer create_library GALAXY "NGS data 2020" \
--description="Sequencing data analysed in 2020"
nebulizer create_library_folder GALAXY "NGS data 2020/Run 21"
add_library_datasets
uploads datasets to a data library:
nebulizer add_library_datasets GALAXY LIBRARY_NAME/PATH FILE [FILE...]
--file-type
: specify the Galaxy data type to assign to datasets--dbkey
: specify the dbkey for the datasets--server
: upload files from the filesystem of the server that Galaxy is running on (default is to upload from the local filesystem)--link
: create symlinks to the files on the server (if--server
is also specified)
For example, add Fastq files to a data library folder:
nebulizer add_library_datasets galaxy "NGS data/Run 21" ~/Sample1_R*.fq \
--file-type=fastqsanger --dbkey=hg38
Managing Tools¶
Querying tool information¶
By default, list_tools
displays information on all the tool
repositories that are installed in a Galaxy instance:
nebulizer list_tools GALAXY
For each installed repository the details include: repository name, toolshed, owner, revision id and changeset, and installation status.
Repository details are also preceeded by a single-character ‘status’ indicator:
D
= deprecated;^
= newer revision is also installed;u
= update (newer revision) is available but not installed;U
= upgrade available (new revision with a change of tool version) but not installed;*
= latest revision installed
list_tools
supports a number of options to modify its
behaviour, including:
--updateable
: only list tool repositories that have uninstalled available updates or upgrades--built-ins
: include details of the “built-in” tools within the Galaxy instance (i.e. those not installed from a toolshed)
An alternative ‘tool-centric’ view of the tools in a Galaxy instance
can be obtained using the --mode=tools
option.
Note
This is a new version of the list_tools
which replaces the
old list_installed_tools
command (which is no longer
available). The --mode=tools
option replicates the output
from the old list_tools
command.
list_tool_panel
displays information on the tool panel
sections in a Galaxy instance:
nebulizer list_tool_panel GALAXY
New tool panel sections can be created when installing tool repositories.
Install, update and remove tools¶
install_tool
installs tools from a Galaxy toolshed (by default
the main Galaxy toolshed at https://toolshed.g2.bx.psu.edu):
nebulizer install_tool GALAXY TOOL_REPOSITORY
The tool repository can be specified as:
[TOOLSHED] OWNER/TOOLNAME [ REVISION ]
e.g.devteam/fastqc
(defaults to most recent revision from the main toolshed), ordevteam/fastqc e7b2202befea
(specifies revisione7b2202befea
)- toolshed URL with or without a revision (e.g.
https://toolshed.g2.bx.psu.edu/view/devteam/fastqc/e7b2202befea
) --tool-panel-section
: specify a tool panel section to install the tools from the repository into (otherwise tools appear at the “top level” of the tool panel)
For example: to install the most recent FastQC tool from the main
Galaxy toolshed under the NGS: QC and manipulation
section of
the tool panel:
nebulizer install_tool GALAXY devteam/fastqc \
--tool-panel-section="NGS: QC and manipulation"
To install a specific revision of the Trimmer tool from the test toolshed:
nebulizer install_tool GALAXY \
https://testtoolshed.g2.bx.psu.edu/view/devteam/trimmer/dec27ea206c3 \
--tool-panel-section="Test tools"
update_tool
installs the latest revision of a previously
installed tool repository, if a new version is available:
nebulizer update_tool GALAXY TOOL_REPOSITORY
The tool repository is specified as with install_tool
except that a revision cannot be included. For example:
nebulizer update_tool GALAXY devteam/fastqc
It is also possible to include glob-style wildcards in the
tool repository name and/or owner e.g. devteam/*
or
bgruening/deeptools_*
. To request update of all tools:
nebulizer update_tool GALAXY '*/*'
Note
update_tool
doesn’t uninstall the older versions of
the tools that are updated.
Warning
By default checks on the availability of updates for tools
performed by the list_tools
and update_tool
commands are done using information cached by the Galaxy
instance in question. As a result these commands may not
always indicate when updates are available.
To force these commands to check the installed revisions
against those in the toolshed, add the --check-toolshed
option. Note however that this can impose a significant
overhead which can make the commands much slower.
uninstall_tool
removes a previously installed tool:
nebulizer uninstall_tool GALAXY TOOL_REPOSITORY
The tool repository is specified as with install_tool
,
for example to uninstall and deactivate a specific revision
of a tool:
nebulizer uninstall_tool GALAXY devteam/fastqc/e7b2202befea
To uninstall all installed revisions of a tool and remove from disk:
nebulizer uninstall_tool localhost devteam/fastqc/* \
--remove_from_disk
Searching for tool repositories on a Toolshed¶
search_toolshed
searches for tools on a toolshed:
nebulizer search_toolshed QUERY
QUERY
can include glob-style wildcards. For example, to
search the main toolshed for Deeptools related tools:
nebulizer search_toolshed "deeptools_*"
--toolshed
: specify the URL of the toolshed to search.
Bulk tool repository management¶
install_tool --file
installs the tool repositories listed in
a tab-delimited file into a Galaxy instance:
nebulizer install_tool GALAXY TOOLS_FILE
TOOLS_FILE
must be a tab-delimited list of repositories,
one repository per line in the format:
TOOLSHED|OWNER|REPOSITORY|REVISON|SECTION
For example:
toolshed.g2.bx.psu.edu devteam bowtie_wrappers 9ca609a2a421 NGS: Mapping
list_tools --mode=export
can generate a list of tool repositories
already installed in a Galaxy instance in this format, e.g.:
nebulizer list_tools GALAXY --mode=export
By combining these two commands it is possible to ‘clone’ the installed tools from one Galaxy instance into another.
For example to replicate the tools installed on the ‘Palfinder’ instance into a local Galaxy:
nebulizer list_tools https://palfinder.ls.manchester.ac.uk --mode=export > palfinder.tsv
nebulizer install_tool http://127.0.0.1 --file palfinder.tsv
Warning
Bulk installation of tools in this manner should be used with caution, especially when installing into a Galaxy instance which already has installed tools.
Managing Quotas¶
Querying quotas¶
quotas
lists the quotas defined in a Galaxy instance:
nebulizer quotas GALAXY
-l
: returns extended information for each quota, including the associated users and groups.--status
: filter list on quota status, which can be one ofactive
(the default),deleted
(only list deleted quotas) orall
(list all active and deleted quotas).--name
: filter list on quota name (can include glob-style wildcards e.g.--name="*NGS*"
).
Creating and deleting quotas¶
quota_add
defines a new quota:
nebulizer quota_add GALAXY QUOTA_NAME SIZE
SIZE
can either be an amount (e.g. 10GB
, 0.2 T
) or
an amount preceeded by an operation (one of +
, -
or
=
, e.g. =300gb
, +100G
). If an operation isn’t
specified then =
is assumed.
-d
/--description
: set the description for the quota (defaults to the quota name).--default_for
: set the quota as the the default for either ‘registered’ or ‘unregistered’ users.
Users and groups can be associated with the new quota using the
-u
and -g
options:
-u
/--users
: associate one or more users with the quota, as a comma-separated list of email addresses.-g
/--groups
: associate one or more groups with the quota, as a comma-separated list of group names.
quota_del
deletes an existing quota:
nebulizer quota_del GALAXY QUOTA_NAME
Note
A deleted quota can be restored using the --undelete
option of the quota_mod
command.
Modifying quota definitions¶
quota_mod
updates a quota definition:
nebulizer quota_mod GALAXY QUOTA_NAME ...
Options allow various quota properties to be modified:
-n
/--name
: sets a new name for the quota.-d
/--description
: sets a new description.-q
/--quota-size
: updates the size of the quota, and how it is applied.--default_for
: set the quota as the the default for either ‘registered’ or ‘unregistered’ users.
Users and groups can be associated with the following options:
-a
/--add-users
: associate one or more users with the quota, as a comma-separated list of email addresses.-r
/--remove-users
: disassociate one or more users from the quota, as a comma-separated list of email addresses.-A
/--add-groups
: associate one or more groups with the quota, as a comma-separated list of group names.-R
/--remove-groups
: disassociate one or more groups from the quota, as a comma-separated list of group names.
Previously deleted quotas can be restored using the
-u
/--undelete
option.
Querying Galaxy instances¶
Checking status and configuration of a Galaxy server¶
‘Ping’ a Galaxy instance to check it’s alive and responding to requests:
nebulizer ping GALAXY
Get information about an instance’s configuration using
- ::
- nebulizer config GALAXY
Storing and managing Galaxy API keys¶
The majority of Nebulizer’s commands require valid credentials in order to interact with a Galaxy instance. These can be supplied explicitly on the command line each time:
nebulizer -u USER@DOMAIN [ -P PASSWORD ] ...command...
nebulizer -k API_KEY ...command...
Alternatively it is possible to store Galaxy URL-API key pairs in
a file called .nebulizer
located in the user’s home directory,
with each pair being associated with an alias.
The file is created automatically the first time an alias is
created using add_key
command, and consists of tab-delimited
lines with three fields:
alias|Galaxy_URL|API_key
for example:
demo http://127.0.0.1:8080 4551fbf7cd8b1bc59db....
This file can be manually edited using a text editor such as
vi
; however Nebulizer also provides a set of commands for
querying and modifying the file contents.
list_keys
shows the aliases with their associated Galaxy
URLs:
nebulizer list_keys
Note
By default the API keys are not shown by list_keys
;
use the --show-api-keys
option to include them.
Note
Use the whomai
command to find out which user is
associated with an alias:
nebulizer whoami ALIAS
add_key
will store a Galaxy-API key combination under a
new alias. If the API key is known then the general form of
the command is:
nebulizer add_key ALIAS GALAXY_URL API_KEY
However it is usually easier to get Nebulizer to fetch the key automatically, by supplying a Galaxy username (email):
nebulizer -u USER@DOMAIN add_key ALIAS GALAXY_URL
Multiple Galaxy URL-key pairs can be stored; only the associated aliases need to be unique.
update_key
will update the details stored for an existing
alias:
nebulizer update_key ALIAS --new-url GALAXY_URL
nebulizer update_key ALIAS --new-api-key API_KEY
nebulizer -u USER@DOMAIN update_key ALIAS --fetch-api-key
remove_key
deletes an existing ALIAS and associated
credentials:
nebulizer remove_key ALIAS
General options¶
Nebulizer supports a number of general options for managing how connections are made to Galaxy servers, how to authenticate, and what information to output.
Authenticating without stored credentials¶
Use --api_key
(-k
) to explicitly specify the API key to
use for authentication on the command line, e.g.
nebulizer -k API_KEY COMMAND...
Use the --username
(-u
) option to authenticate using the
normal Galaxy login credentials (i.e. email and password)
instead of the API key, e.g.
nebulizer -u USER@DOMAIN COMMAND...
You will be prompted to enter the password, but you can also use
the --galaxy_password
(-P
) option to specify it explicitly
on the command line.
Controlling warnings and debugging output¶
--suppress-warnings
(-q
) suppresses warning messages from
Nebulizer; conversely debugging output can be enabled using the
--debug
option.
Handling SSL certificate verification failures¶
Nebulizer commands will fail for Galaxy instances which are served over
https
protocol without a valid SSL certificate, reporting an error like:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590), 0 attempts left: None
In this case adding the --no-verify
(-n
) option turns off the
certificate verification and should enable a connection to be made.
Nebulizer Tutorial¶
This tutorial offers a short hands-on introduction to the main functionality offered by Nebulizer.
0. Preparation: making a temporary local Galaxy instance¶
Before starting it is recommended to set up a temporary local Galaxy instance to run the tutorial on, rather than trying Nebulizer out on an existing installation.
The following commands should fetch and start up the local instance:
# Get the Galaxy source and put into a new directory
# called 'galaxy_for_demo'
git clone -b release_21.01 https://github.com/galaxyproject/galaxy.git galaxy_for_demo
# Move into the source directory
cd galaxy_for_demo
# Make a configuration file
cp config/galaxy.yml.sample config/galaxy.yml
You will need to edit the configuration file
config/galaxy.yml
using a text editor (e.g. vi
,
nano
, gedit
etc) to specify an admin user, by
changing the line:
#admin_users: null
to e.g.
admin_users: admin@localhost.org
or whatever email address you want. You will also need to change the line:
#allow_user_deletion: false
to
allow_user_deletion: true
Then start Galaxy running:
sh run.sh
Note
This step can take some time when it’s first run; however once Galaxy has installed its dependencies and performed its initial configuration then subsequent startups of the demo Galaxy should be much quicker.
Use ctrl-C
to stop Galaxy running, and repeat this
command to start it up again.
Once Galaxy is running you can connect to it by pointing a web browser to http://127.0.0.1:8080/
Finally you will need to register an account in Galaxy
with the same email address as the one used for
admin_users
(admin@localhost.org
in this example),
using the web browser.
Now you’re ready to run through the tutorial.
Note
Full instructions for setting up a local Galaxy instance can be found at https://galaxyproject.org/admin/get-galaxy/
1. Install Nebulizer¶
Installing Nebulizer is best done in a Python virtual environment - for example:
virtualenv -p python3 venv.nebulizer
. venv.nebulizer/bin/activate
pip install nebulizer
Note
It is recommended to do this in a different directory to the local Galaxy instance from the previous section.
Once this is done you should have access to the
nebulizer
utility.
To list the available commands:
nebulizer --help
You use the ping command to check if a Galaxy instance is active. For example to check the main Galaxy server:
nebulizer ping https://usegalaxy.org
which should produce output like:
https://usegalaxy.org: status = ok time = 574.041 (ms)
https://usegalaxy.org: status = ok time = 587.883 (ms)
https://usegalaxy.org: status = ok time = 549.526 (ms)
...
Note
Do control-C
to terminate the “ping”.
You can also use the config command to query the details of a Galaxy instance’s configuration. For example to query the local Galaxy:
nebulizer config http://127.0.0.1:8080
2. Set up aliases for Galaxy API keys¶
Most Nebulizer commands require you to interact with a Galaxy instance using an account on that instance.
For these commands you can authorise access each time by specifying your registered email address and password or Galaxy API key on the command line. For example:
nebulizer -u admin@localhost.org whoami http://127.0.0.1:8080
Note
-u
will prompt you to enter the password for
the account before performing the action; -k
can be used to specify the API key.
Warning
This won’t work if you didn’t make an account
for admin@localhost.org
when preparing the
local Galaxy in the previous step!
This is quite laborious when executing several commands, so Nebulizer allows you to associate Galaxy instances and their API keys with aliases; these are used as shortcuts when running the commands.
To see the aliases and associated Galaxy servers:
nebulizer list_keys
Note
If you’ve never used Nebulizer before then nothing will be listed.
To set up a new alias called local
and associate it with
the admin account in our local Galaxy, we can do:
nebulizer -u admin@localhost.org add_key local http://127.0.0.1:8080
This will prompt you for the password for the account and
then create the alias. Once this is done you can repeat the
list_keys
command and see an entry for the local Galaxy:
local http://127.0.0.1:8080
In subsequent commands you can use local
rather than
specifying the full Galaxy URL, and won’t need to enter
your email or password. For example:
nebulizer whoami local
Now we’re ready to do some basic administration of our local Galaxy using Nebulizer.
Note
See Storing and managing Galaxy API keys for more details.
3. Listing, adding and deleting users¶
We can list the users in our local Galaxy with:
nebulizer list_users local
There will be just one account (the original admin account we made at the start).
We can add a new user using:
nebulizer create_user local ann.onymous@manchester.ac.uk
Note
This will prompt you for a password for the new account;
use the -p
option to set the password via the
command line.
Do the list_users
command again to see new user listed.
Use the -l
option to display additional information
about each user is displayed, including status and disk
usage (and quota usage, if quotas are enabled).
Batches of user accounts can be created from a “template”
name using the create_batch_users
command; this can be
useful for example when setting up Galaxy instances for
teaching:
nebulizer create_batch_users local user#@bcc2020.org 5
Note
This will prompt you for a password which will be assigned to all the new accounts.
Use the list_users
command to see the new accounts:
user1@bcc2020.org user1
user2@bcc2020.org user2
...
user5@bcc2020.org user5
Accounts can also be deleted:
nebulizer delete_user local user5@bcc2020.org
The user will no longer be listed by list_users
.
Warning
If the deletion fails then check that the Galaxy
configuration has allow_user_deletion
set
to true
.
Note
See Managing Users for more details.
4. Creating and populating data libraries¶
We can list the data libraries in our local Galaxy instance using:
nebulizer list_libraries local
Initially our local Galaxy doesn’t contain any library data; we can create a new data library using:
nebulizer create_library local "Example data"
Note
Use the -d
and -s
options to add description
and synopsis information for the new library.
Now this will be listed by the list_libraries
command.
We can list the contents of a library by specifying its
name:
nebulizer list_libraries local "Example data"
Initially the library is empty; we can create a folder within the library:
nebulizer create_library_folder local "Example data/Fastqs"
To list the contents of a library folder specify the “path” to the folder:
nebulizer list_libraries local "Example data/Fastqs"
Datasets can be added to libraries and folders from the local workstation:
nebulizer add_library_datasets local "Example data/Fastqs" Illumina_SG_R* --dbkey=hg38
When listing the contents of libraries and folders,
additional information is reported by specifying the -l
option:
nebulizer list_libraries local "Example data/Fastqs" -l
Note
See Managing Data Libraries for more details.
5. Installing and managing tools¶
We can list the tools installed in our local Galaxy using:
nebulizer list_tools local
Initially there are no tools installed; we can search the main Galaxy toolshed for the tools we want to install, for example the FastQC tool:
nebulizer search_toolshed fastqc
Warning
The time taken for searching depends on the speed of the toolshed, so sometimes this can be slow if e.g. the toolshed is experiencing issues.
This will list all the tool repositories and toolshed versions available to install:
devteam fastqc 21:e7b2202befea
devteam fastqc 19:9da02be9c6cc
devteam fastqc 16:ff9530579d1f
...
We can install the latest version of FastQC with
nebulizer install_tool local devteam/fastqc --tool-panel-section="NGS tools"
Note
Using --tool-panel-section
will create a new section
in the Galaxy tool panel and put the tools from this
repository under it; otherwise tools are not installed
under any section. You can use the list_tool_panel
command to see what tool panel sections are already
present.
Running list_tools
now shows the tool repository is
installed:
* fastqc toolshed.g2.bx.psu.edu devteam 21:e7b2202befea Installed
Note
The *
next to tool repository indicates that this
is most recent version.
Use the --mode=tools
option to list the associated
tools instead.
We can install a specific version of a tool repository, for example the Trimmomatic tool:
nebulizer install_tool local pjbriggs/trimmomatic 51b771646466 --tool-panel-section="NGS tools"
Running list_tools
now shows this tool repository is also
installed:
* fastqc toolshed.g2.bx.psu.edu devteam 21:e7b2202befea Installed
U trimmomatic toolshed.g2.bx.psu.edu pjbriggs 12:51b771646466 Installed
Here U
indicates there is a newer revision available
with a new version of the tool (u
indicates a newer
revision without a tool version update).
Rerunning the list_tools
command with the --updateable
option filters the list of tool repositories to just those with
available updates.
We can update Trimmomatic to the newest version automatically
by running the update_tool
command:
nebulizer update_tool local pjbriggs/trimmomatic
Note
This installs the most recent version but doesn’t remove the older version.
The uninstall_tool
command removes an installed
repository; for example to uninstall the older Trimmomatic
tool version:
nebulizer uninstall_tool local pjbriggs/trimmomatic 51b771646466
Running list_tools
shows that the older tool repository
is no longer present.
Note
See Managing Tools for more details.
History¶
v0.7.1 (2021-06-29)¶
v0.7.0 (2021-05-17)¶
Breaking changes:
Dropped support for Python 2.7:
nebulizer
now needs Python 3.6+ (thanks to Hugo van Kemenade @hugovk) (PR #102)Substantial refactoring and simplification of the tool managament commands (PR #113): - Now only
list_tools
,install_tool
,update_tool
and
delete_tool
commands are supported
list_installed_tools
renamed tolist_tools
- old functionality of
list_tools
replaced bylist_tools --mode=tools
list_repositories
command dropped; functionality replaced bylist_tools --mode=export
install_repositories
command dropped; functionality replaced byinstall_tool --file=...
New commands:
- New
quota
,quota_add
,quota_mod
andquota_del
commands for managing quotas (PR #66)
Updates to existing commands:
-l
option forsearch_toolshed
includes the shed URL (PR #91)update_tool
allows use of wildcards (i.e.*
) when specifying tool repository names and owners, to enable multiple tool repositories to updated at once (PR #92)- New
--status
option forlist_users
command allows deleted and purged user accounts to also be listed (PR #97)- New
--sort
option forlist_users
command allows sorting of listed accounts by disk usage, quota and quota usage (PR #104)
Bug fixes:
Other updates:
- Added support for Python 3.9 (thanks to Hugo van Kemenade @hugovk) (PR #108)
v0.6.0 (2020-07-14)¶
New commands:
Updates to existing commands:
list_keys
doesn’t report API keys unless--show-api-keys
option is specified (PR #58)- Additional fields reported by
--long-listing-format
option oflist_users
(disk and quota usage, status); doesn’t report ID by default (PR #59)- Enable flexible tool repository specification syntax for
install_tool
andupdate_tool
(PR #60)remove_key
prompts user to confirm API key deletion (PR #72)- Use spaces rather than tabs to line up fields in output from
list_users
,list_installed_tools
,list_tools
,list_tool_panel
,list_keys
,config
,list_libraries
; use--show_id
to report Galaxy IDs for users and data libraries (PR #68, PR #69, PR #70)
Documentation:
Removed functionality:
- Removed deprecated utilities
manage_users
,manage_tools
andmanage_libraries
(PR #61)
v0.4.3 (2018-10-05)¶
- Ensure that
click
dependency is version 6.7 or earlier, to avoid subcommand names changing from e.g.list_users
tolist-users
(PR #49)
v0.4.2 (2017-08-24)¶
- Commands now explicitly return appropriate exit code values indicating success (
0
) or failure (non-zero values).- New option
--check-toolshed
added tolist_installed_tools
andupdate_tool
commands, to check installed revisions directly against those available in the toolshed (PR #41)- Update
install_tool
,update_tool
andinstall_repositories
to install tool dependencies through a resolver (e.g.conda
) by default (issue #43)- New options added to
install_tool
,update_tool
andinstall_repositories
commands, to explicit control how tool and repository dependencies should be handled (PR #44):
--install-tool-dependencies [yes|no]
: install tool dependencies via the toolshed, if any are defined (default isyes
)--install-tool-dependencies [yes|no]
: install tool dependencies via the toolshed, if any are defined (default isyes
)--install-resolver-dependencies [yes|no]
: install dependencies through a resolver that supports installation (e.g.conda
) (default isyes
)
v0.4.0 (2016-11-18)¶
- New subcommand
ping
: ‘ping’ a Galaxy instance to see if it’s responsive (PR #33).- New subcommand
whoami
: reports user associated with the API key (PR #37).add_library_datasets
: refuses to perform upload if using the master API key (essentially API key must have an associated user).install_repositories
: prints a list of all tool repositories that couldn’t be installed.- New
--timeout
and--nowait
options added forinstall_tool
,update_tool
andinstall_repositories
subcommands.- Fix to treat tool repositories with status
New
as still installing when trying to install tools (PR #31).- Some improvements to logging (PR #38).
v0.3.0 (2016-10-26)¶
- New class
tools.ToolPanel
and updates to existingtools.ToolPanelSection
class.install_tool
: fix behaviour so that command does nothing if a version is not specified and at least one version of the tool is already installed.list_repositories
andinstall_repositories
: new commands to generate a list of installed tool repositories from a Galaxy instance and then reinstall tool repositories from a list with the same format (PR #19).install_tool
: fix incorrect reporting of target tool panel section (PR #20)add_key
andupdate_key
: fix automatic retrieval of API key, which only worked previously if connecting user was an admin account (PR #23)list_tool_panel
: shows tools in order they appear in Galaxy when using--list-tools
option.- Deprecated utilities (
manage_users
,manage_tools
andmanage_libraries
) issue warnings when run.- License updated to Academic Free License (AFL).
- Initial version of documentation also made available via ReadTheDocs (PR #21)
v0.2.0 (2016-10-17)¶
- Implemented new
nebulizer
utility which provides all previous functionality via subcommands, plus commands for managing API keys automatically (oldmanage_users
,manage_tools
andmanage_libraries
utilities are still available for backwards-compatibility but are deprecated).- New general options:
-q
/--suppress-warnings
: prevent warning messages fromnebulizer
commands.- Various fixes and improvements to underlying functionality:
install_tools
: now checks if tool is already installed; handles tool revisions that include the revision number; polls Galaxy until tool is installed, or operation times out; exit status reflects the success or failure of the installation.update_tool
: now works even if original tool isn’t in a tool panel sectionlist_installed_tools
: now groups tools under correct repo revision when using--list-tools
option.
v0.1.1 (2016-05-16)¶
- Add
-u
/--username
and-P
/--galaxy_password
options to all commands to allow interaction with Galaxy instance via API using normal login credentials instead of API key.
v0.1.0 (2015-11-06)¶
- Initial release of
nebulizer
utilities for administering Galaxy instances via the command line.