DMagic

DMagic is an open-sourced Python toolbox to interface with the Advanced Photon Source scheduling system.

This guide is maintained on GitHub.

About

DMagic aims to provide easy command-line interface to the APS scheduling system

Basic functionalities include the ability, for the current experiment, to retrieve the users’ email addresses from the APS scheduling as well as experiment information. This infomation can be printed (“show” option) or used to update EPICS PVs (“tag” option)

DMagic supports Globus a tool to automatically create Globus shared folders with users.

DMagic and Globus are designed to help managing the massive amount of data generated at the Advanced Photon Source by providing an automatic way to tag, share, notify and distribute data to users.

Install

This section covers the basics of how to download and install DMagic.

Pre-requisites

Install from Anaconda python3.x.

You will also need to have the APS Data Management system installed for your beamline; contact the SDM group for this installation. Once installed you can run dmagic in a terminal with:

On a BM line:

$ source /home/dm_bm/etc/dm.setup.sh

On an ID line:

$ source /home/dm_id/etc/dm.setup.sh

then:

$ dmagic show

Alternatively you can download the Data Management API via conda

conda install -c aps-anl-tag aps-dm-api

There are also several environment variables that must be set for the DM API to work properly. They can be found in the /home/dm_bm/etc/dm.conda.setup.sh script. Copy everything in this script except the change to the PATH to your account’s ~/.bashrc file.

this includes adding in your .bashrc:

DM_APS_DB_WEB_SERVICE_URL=https://xraydtn02.xray.aps.anl.gov:11236
export DM_APS_DB_WEB_SERVICE_URL
DM_BEAMLINE_NAME=20-BM-B
export DM_BEAMLINE_NAME

Please use the DM_BEAMLINE_NAME as defined in the beamline directory

To test the access to the APS scheduling system:

python -c "from dm.aps_db_web_service.api.esafApsDbApi import EsafApsDbApi; api = EsafApsDbApi();print(api.listEsafs(sector='20',     year=2021))"

Installing from source

In a prepared virtualenv or as root for system-wide installation clone the DMagic from GitHub repository

$ git clone https://github.com/xray-imaging/DMagic DMagic

To install DMagic, run:

$ cd DMagic
$ python setup.py install

Warning

Make sure your python installation is in a location set by #!/usr/bin/env python, if not please edit the first line of the bin/dmagic file to match yours.

Configuration

To run DMagic you need to set the beamline name (as defined in the APS scheduling system) and the user info PVs where to store the information retrieved from the scheduling system. If you are using tomoScan these are provided in the beamline specific section user information section.

Once you have this information you can edit the config.py file entering the name of your beamline and the IOC prefix for the user info PVs:

$ gedit DMagic/dmagic/config.py

You can also change these at runtime using the –beamline and –tomoscan-prefix options. For more info:

$ dmagic show -h
usage: dmagic show [-h] [--beamline BEAMLINE]
                   [--tomoscan-prefix TOMOSCAN_PREFIX] [--config FILE]
                   [--verbose]

optional arguments:
  -h, --help            show this help message and exit
  --beamline BEAMLINE   beam line (default: 7-BM-B)
  --tomoscan-prefix TOMOSCAN_PREFIX
                        The tomoscan prefix, i.e.'7bmb1:' or '2bma:TomoScan:'
                        (default: 7bmb1:)
  --config FILE         File name of configuration (default:
                        /home/beams/USER2BMB/dmagic.conf)
  --verbose             Verbose output (default: True)

If you are not running tomoScan look at the EPICS tools section below.

EPICS tools

If you are not running tomoScan:

dbLoadRecords("$(TOP)/experimentInfo.db", "P=32idcTXM:")
medm screen

Update

dmagic is constantly updated to include new features. To update your locally installed version:

$ cd dmagic
$ git pull
$ python setup.py install

Dependencies

Install the following package:

$ pip install validate-email
$ pip install pyinotify
$ pip install pyepics
$ pip install pytz
$ conda install decorator
$ conda install numpy

Warning

If requiere edit your .cshrc to set PYEPICS_LIBCA: Example: setenv PYEPICS_LIBCA /APSshare/epics/extensions-base/3.14.12.2-ext1/lib/linux-x86_64/libca.so

Usage

DMagic retrieves user and experiment information from the APS scheduling system. To initlialize DMagic status:

$ dmagic init

this creates the DMagic config file: ~/dmagic.conf with default values.

To show the list of users currently running at the beameline:

$ dmagic show

To configure for your beamline:

$ dmagic show --beamline 32-ID --tomoscan-prefix 32id:TomoScan

To update the EPICS PVs with data retrieved from the APS scheduling system run:

$ dmagic tag

The information associated with the current user/experiment will be updated in the medm screen:

medm screen

For help and to access all options:

dmagic -h
dmagic show -h
dmagic tag -h

If running with the APS data management:

[user2bmb@arcturus]$ bash
[user2bmb@arcturus]$ conda activate dm
[user2bmb@arcturus]$ source /home/dm_bm/etc/dm.setup.sh
[user2bmb@arcturus]$ dmagic show
[user2bmb@arcturus]$ dmagic tag

API reference

This section contains the API reference and usage information for DMagic.

DMagic Modules:

dmagic.scheduling

Module containing routines to access the APS scheduling system.

You must create in your home directory a scheduling.ini configuration file

Functions:

get_current_users(args) Get users running at beamline currently
get_current_emails(args[, exclude_pi]) Find user’s emails currently running at beamline
get_current_pi(args) Get information about the PI for the current experiment.
get_current_proposal_id(args) Get the proposal id for the current proposal.
get_current_proposal_title(args) Get the title of the current proposal.
print_current_experiment_info(args) Print the current experiment info running at beamline
dmagic.scheduling.get_current_users(args)[source]

Get users running at beamline currently

Returns:users (dictionary-like object containing user information)
dmagic.scheduling.get_current_emails(args, exclude_pi=True)[source]

Find user’s emails currently running at beamline

Parameters:users (dictionary-like object containing user information)
Returns:List of user emails (default (all but PI))
dmagic.scheduling.get_current_pi(args)[source]

Get information about the PI for the current experiment.

Returns:dictionary-like object containing information for PI
dmagic.scheduling.get_current_proposal_id(args)[source]

Get the proposal id for the current proposal.

Returns:proposal ID as an int
dmagic.scheduling.get_current_proposal_title(args)[source]

Get the title of the current proposal.

Returns:str (title of the proposal)
dmagic.scheduling.print_current_experiment_info(args)[source]

Print the current experiment info running at beamline

Returns:Print experiment information

Indices and tables