ETTLIMS¶

ETTLIMS is a Lab Information Management System developed for the Ettema Lab by BILS.

This documentation has the following sections:

Pipeline Design
Describes the workflow at the Ettema Lab
Database design
Describes how the workflow has been implemented in the database
Naming Scheme
The naming scheme for barcodes and UID for objects in the database
User guide
A user guide for users of the system
Installation
How to install the LIMS
Developer guide
A guide for developers

Other links:

Contents:

Pipeline Design¶

_images/flowchartlab.png

Database design¶

_images/lims_er_sample.png _images/storage_er.png

Naming Scheme¶

The constraints are:

  • Naming should be stable: it should never change. This implies that enough room should be given to account for many years of future samples
  • Name should be informative: one should get an idea of where the gene/SAG comes from from its name.
  • Name should be concise: ideally, a gene name (which is the end product in a way) should be kept under 20 characters

Quick guide:

sample well asbly reserved
_____  ___  _     _
O331AC_C09A_A002340
     –    –  –––––
 plate    run gene

Complete description¶

  • Sample:
    • Five characters, alphanumeric, letter in capitals
    • Should be as informative as possible (e.g. O331A for Ocean Drilling program exp. 331, sample A)
  • Plates:
    • Start with sample ID
    • One extra character (A-Z) to indicate the nth plate from this sample (e.g. O331AC for the third plate sorted from sample O331A)
    • The same extra character (a-z and 0-9) for plate dilutions (e.g. O331Ab for the second plate dilution from sample O331A)
    • For metagenomes, this should indicate different extractions from the same sample.
  • Well/Metagenome/Pure culture/Amplicons:
    • Starts with plate ID, then an underscore
    • Three characters in addition: one letter (A-L) and two digits (00-16) to describe the well.
    • Metagenomes are labeled with X, then 01. Further digits are reserved for later use.
    • Pure cultures are labeled with Z, then 01. Further digits are reserved for later use.
    • Amplicons are labeled with Y, then 01. Further digits are reserved for later use.
    • A1 should be always written A01
    • E.g. O331AC_C09
  • DNA library
    • Describes many DNA libraries from the same source: can be different dilution plates, different sequencing runs, different preparations, for example.
    • Starts with the well ID
    • One character (A-Z; 0-9), in order
    • E.g. O331AC_C09A
  • Assembly
    • Describes one assembly performed on one dataset from one run (SAG)
    • Starts with run ID, then an underscore
    • One character (A-Z; 0-9), in order
    • E.g. O331AC_C09A_B
  • Genes:
    • Starts with assembly ID
    • 6 digits, with a 10 increment to allow adding genes in the middle
    • tRNAs have t prefix, rRNAs a r instead of the first digit
    • E.g. O331AC_C09A_B002340, O331AC_C09A_Bt05610

User guide¶

Viewing area¶

_images/screenshot_home.png

The viewing area currently just gives a simple listing of the different objects in the database. The main feature of the viewing area is the ability to see a tree of all related objects with a sample of choice at the root. The listing of the objects is more limited than the admin area, so for browsing objects in the database it is still more user-friendly to use the admin area.

Admin area¶

_images/screenshot_admin.png

The admin area is where one actually inputs information into the database. Adding, editing, sorting and listing of objects is possible.

Bulk import¶

The Container and Sample tables have an option to bulk import multiple objects in one go with a csv, json or excel file. This is only recommended for advanced users, because you skip a lot of extra checks done that you would normally have when you fill in the forms. The importer’s error messages can be rather cryptic. That being said, here is how to do it.

Import a bunch of Samples in one go¶
  1. Go to the import page and download the csv template
_images/sample_import.png
  1. Edit the csv template.
_images/sample_csv_firsthalf.png

The columns that link to other tables should contain the actual id. You can look them up in their corresponding tables. For instance for the sample_type you can look at the Sample types listing:

_images/sample_types_listing.png

and for collaborator in the Collaborator listing:

_images/collaborator_listing.png

The new ids for the samples don’t have to be filled in, since those will be generated by the system itself.

After the extra_columns_json you can add as many columns as you like. These will be stored in the extra_columns_json field in JSON format on import.

_images/sample_csv_secondhalf.png

3. Import the edited csv file. Make sure you saved the csv without added quotation marks around the values. Don’t mind the crossed out date part, this is a known bug in the importer.

_images/sample_import_preview.png
Bulk import a group of wells to store samples in one go¶
  1. Create a 384 Well Plate to connect the Wells to using the admin interface
_images/container_wellplate.png
  1. Get the id for the new 384 Well Plate (51)
_images/container_add_success.png
  1. Determine the id for the container type you want to add (4)
_images/container_types.png
  1. Click on import in the Container listing
_images/container_admin_import.png
  1. Change the given csv template. It uses the id from step 2 as the parent container and the sample type from step 3. You don’t have to specify an id for the new containers themselves because those will be automatically generated by the system. The content_type is an id that refers to the type of object that is stored, in this case 15 which represents the code for a Sample. This id might be different on your server though. You can look up the ids for other objects in the ContentTypes listing in the admin area. The object_id should contain the id of the object, in this example we used the ones from the previous step.
_images/container_csv.png
  1. Import the csv file.
_images/container_import_preview.png

Presentation¶

There’s also a presentation of the system available that incorporates several movies of how to use the system.

Issues¶

Check out the Issues section if you run into errors.

Installation¶

Installation¶

Clone the repository to your computer:

git clone https://github.com/BILS/ETTLIMS

Local installation:

pip install -r lims_project/requirements/local.txt

Running ETTLIMS¶

Locally with SQLite¶

Without example data:

cd lims_project
python manage.py syncdb --settings=lims_project.settings.local && \
python manage.py runserver 127.0.0.1:8000 --settings=lims_project.settings.local

With all example data:

cd lims_project
python manage.py syncdb --settings=lims_project.settings.local && \
python manage.py loaddata example barcode_example --settings=lims_project.settings.local && \
python manage.py runserver 127.0.0.1:8000 --settings=lims_project.settings.local

Remove example from the loaddata command if you only want the barcode_example data and vice versa.

Empty the database:

cd lims_project
rm lims_project/lims_project.sqlite

Development with PostgreSQL database¶

First you need to set up PostgreSQL. You’ll have to create a database django and user django, see the development settings. Then the commands are similar to running it locally, but you change lims_project.settings.local to lims_project.settings.development e.g.:

cd lims_project
python manage.py syncdb --settings=lims_project.settings.development && \
python manage.py loaddata example barcode_example --settings=lims_project.settings.development && \
python manage.py runserver 127.0.0.1:8000 --settings=lims_project.settings.development

To empty the PostgreSQL database you can run:

python manage.py sqlclear sessions admin lims auth contenttypes \
    --settings=lims_project.settings.development | \
grep -v parent_id | \
python manage.py dbshell --settings=lims_project.settings.development

The grep -v parent_id is necessary to remove a non-existing constraint from the generated sql statements by django. This is a known django bug.

Developer guide¶

Contribute¶

Contributions are greatly appreciated, please read the contribution instructions.

Issues¶

Known issues¶

  • Date part gets crossed out in bulk import. The importer thinks the date is being edited from the default value

Reporting issues¶

An up to date list of issues can be found at: https://github.com/bils/ettlims/issues. Check if your issue is there, if not add it.