nbconvert: Convert Notebooks to other formats¶
Using nbconvert
enables:
presentation of information in familiar formats, such as PDF.
publishing of research using LaTeX and opens the door for embedding notebooks in papers.
collaboration with others who may not use the notebook in their work.
sharing contents with many people via the web using HTML.
Overall, notebook conversion and the nbconvert
tool give scientists and
researchers the flexibility to deliver information in a timely way across
different formats.
Primarily, the nbconvert
tool allows you to convert a Jupyter .ipynb
notebook document file into another static format including HTML, LaTeX, PDF,
Markdown, reStructuredText, and more. nbconvert
can also add productivity
to your workflow when used to execute notebooks programmatically.
If used as a Python library (import nbconvert
), nbconvert
adds
notebook conversion within a project. For example, nbconvert
is used to
implement the “Download as” feature within the Jupyter Notebook web
application. When used as a command line tool (invoked as
jupyter nbconvert ...
), users can conveniently convert just one or a
batch of notebook files to another format.
Contents:
Installation¶
See also
- Installing Jupyter
Nbconvert is part of the Jupyter ecosystem.
Installing nbconvert¶
Nbconvert is packaged for both pip and conda, so you can install it with:
pip install nbconvert
# OR
conda install nbconvert
If you’re new to Python, we recommend installing Anaconda, a Python distribution which includes nbconvert and the other Jupyter components.
Important
To unlock nbconvert’s full capabilities requires Pandoc and TeX (specifically, XeLaTeX). These must be installed separately.
Installing Pandoc¶
For converting markdown to formats other than HTML, nbconvert uses Pandoc (1.12.1 or later).
To install pandoc on Linux, you can generally use your package manager:
sudo apt-get install pandoc
On other platforms, you can get pandoc from their website.
Installing TeX¶
For converting to PDF, nbconvert uses the TeX document preparation
ecosystem. It produces an intermediate .tex
file which is
compiled by the XeTeX engine with the LaTeX2e format (via the
xelatex
command) to produce PDF output.
New in version 5.0: We use XeTeX as the rendering engine rather than pdfTeX (as in earlier versions). XeTeX can access fonts through native operating system libraries, it has better support for OpenType formatted fonts and Unicode characters.
To install a complete TeX environment (including XeLaTeX and the necessary supporting packages) by hand can be tricky. Fortunately, there are packages that make this much easier. These packages are specific to different operating systems:
Linux: TeX Live
E.g. on Debian or Ubuntu:
sudo apt-get install texlive-xetex
macOS (OS X): MacTeX.
Windows: MikTex
Because nbconvert depends on packages and fonts included in standard TeX distributions, if you do not have a complete installation, you may not be able to use nbconvert’s standard tooling to convert notebooks to PDF.
PDF conversion on a limited TeX environment¶
If you are only able to install a limited TeX environment, there are two main routes you could take to convert to PDF:
- Using TeX by hand
You could convert to
.tex
directly; this requires Pandoc.edit the file to accord with your local environment
run
xelatex
directly.
- Custom exporter
You could write a custom exporter that takes your system’s limitations into account.
Using as a command line tool¶
The command-line syntax to run the nbconvert
script is:
$ jupyter nbconvert --to FORMAT notebook.ipynb
This will convert the Jupyter notebook file notebook.ipynb
into the output
format given by the FORMAT
string.
Default output format - HTML¶
The default output format is HTML, for which the --to
argument may be
omitted:
$ jupyter nbconvert notebook.ipynb
Supported output formats¶
The currently supported output formats are:
Jupyter also provides a few templates for output formats. These can be
specified via an additional --template
argument and are listed in the
sections below.
HTML¶
--to html
--template full
(default)A full static HTML render of the notebook. This looks very similar to the interactive view.
--template basic
Simplified HTML, useful for embedding in webpages, blogs, etc. This excludes HTML headers.
LaTeX¶
--to latex
Latex export. This generates
NOTEBOOK_NAME.tex
file, ready for export. Images are output as .png files in a folder.--template article
(default)Latex article, derived from Sphinx’s howto template.
--template report
Latex report, providing a table of contents and chapters.
Note
nbconvert uses pandoc to convert between various markup languages, so pandoc is a dependency when converting to latex or reStructuredText.
PDF¶
--to pdf
Generates a PDF via latex. Supports the same templates as
--to latex
.
Reveal.js HTML slideshow¶
--to slides
This generates a Reveal.js HTML slideshow.
Running this slideshow requires a copy of reveal.js (version 3.x).
By default, this will include a script tag in the html that will directly load reveal.js from a public CDN.
This means that if you include your slides on a webpage, they should work as expected. However, some features (specifically, speaker notes & timers) will not work on website because they require access to a local copy of reveal.js.
Speaker notes require a local copy of reveal.js. Then, you need to tell
nbconvert
how to find that local copy.
Timers only work if you already have speaker notes, but also require a local https server. You can read more about this in ServePostProcessorExample.
To make this clearer, let’s look at an example of how to get speaker notes working with a local copy of reveal.js: SlidesWithNotesExample.
Note
In order to designate a mapping from notebook cells to Reveal.js slides, from within the Jupyter notebook, select menu item View –> Cell Toolbar –> Slideshow. That will reveal a drop-down menu on the upper-right of each cell. From it, one may choose from “Slide,” “Sub-Slide”, “Fragment”, “Skip”, and “Notes.” On conversion, cells designated as “skip” will not be included, “notes” will be included only in presenter notes, etc.
Example: creating slides w/ speaker notes¶
Let’s suppose you have a notebook your_talk.ipynb
that you want to convert
to slides. For this example, we’ll assume that you are working in the same
directory as the notebook you want to convert (i.e., when you run ls .
,
your_talk.ipynb
shows up amongst the list of files).
First, we need a copy of reveal.js in the same directory as your slides. One way to do this is to use the following commands in your terminal:
git clone https://github.com/hakimel/reveal.js.git
cd reveal.js
git checkout 3.5.0
cd ..
Then we need to tell nbconvert to point to this local copy. To do that we use
the --reveal-prefix
command line flag to point to the local copy.
jupyter nbconvert your_talk.ipynb --to slides --reveal-prefix reveal.js
This will create file your_talk.slides.html
, which you should be able to
access with open your_talk.slides.html
. To access the speaker notes, press
s
after the slides load and they should open in a new window.
Note: This does not enable slides that run completely offline. While you have a local copy of reveal.js, by default, the slides need to access mathjax, require, and jquery via a public CDN. Addressing this use case is an open issue and PRs are always encouraged.
Serving slides with an https server: --post serve
¶
Once you have speaker notes working you may notice that your timers don’t work. Timers require a bit more infrastructure; you need to serve your local copy of reveal.js from a local https server.
Fortunately, nbconvert
makes this fairly straightforward through the use of
the ServePostProcessor
. To activate this server, we append the command line
flag --post serve
to our call to nbconvert.
jupyter nbconvert your_talk.ipynb --to slides --reveal-prefix reveal.js --post serve
This will run the server, which will occupy the terminal that you ran the
command in until you stop it. You can stop the server by pressing ctrl C
twice.
Markdown¶
--to markdown
Simple markdown output. Markdown cells are unaffected, and code cells indented 4 spaces. Images are output as .png files in a folder.
Ascii¶
--to asciidoc
Ascii output. Images are output as .png files in a folder.
reStructuredText¶
--to rst
Basic reStructuredText output. Useful as a starting point for embedding notebooks in Sphinx docs. Images are output as .png files in a folder.
Note
nbconvert uses pandoc to convert between various markup languages, so pandoc is a dependency when converting to LaTeX or reStructuredText.
Executable script¶
--to script
Convert a notebook to an executable script. This is the simplest way to get a Python (or other language, depending on the kernel) script out of a notebook. If there were any magics in an Jupyter notebook, this may only be executable from a Jupyter session.
For example, to convert a Julia notebook to a Julia executable script:
jupyter nbconvert --to script my_julia_notebook.ipynb
Notebook and preprocessors¶
--to notebook
New in version 3.0.
This doesn’t convert a notebook to a different format per se, instead it allows the running of nbconvert preprocessors on a notebook, and/or conversion to other notebook formats. For example:
jupyter nbconvert --to notebook --execute mynotebook.ipynb
This will open the notebook, execute it, capture new output, and save the
result in mynotebook.nbconvert.ipynb
. Specifying --inplace
will
overwrite the input file instead of writing a new file. By default,
nbconvert
will abort conversion if any exceptions occur during
execution of a cell. If you specify --allow-errors
(in addition to the
–execute` flag) then conversion will continue and the output from any
exception will be included in the cell output.
The following command:
jupyter nbconvert --to notebook --nbformat 3 mynotebook
will create a copy of mynotebook.ipynb
in mynotebook.v3.ipynb
in version 3 of the notebook format.
If you want to convert a notebook in-place, you can specify the output file to be the same as the input file:
jupyter nbconvert --to notebook mynb --output mynb
Be careful with that, since it will replace the input file.
Note
nbconvert uses pandoc to convert between various markup languages, so pandoc is a dependency when converting to latex or reStructuredText.
The output file created by nbconvert
will have the same base name as
the notebook and will be placed in the current working directory. Any
supporting files (graphics, etc) will be placed in a new directory with the
same base name as the notebook, suffixed with _files
:
$ jupyter nbconvert notebook.ipynb
$ ls
notebook.ipynb notebook.html notebook_files/
For simple single-file output, such as html, markdown, etc., the output may be sent to standard output with:
$ jupyter nbconvert --to markdown notebook.ipynb --stdout
Converting multiple notebooks¶
Multiple notebooks can be specified from the command line:
$ jupyter nbconvert notebook*.ipynb
$ jupyter nbconvert notebook1.ipynb notebook2.ipynb
or via a list in a configuration file, say mycfg.py
, containing the text:
c = get_config()
c.NbConvertApp.notebooks = ["notebook1.ipynb", "notebook2.ipynb"]
and using the command:
$ jupyter nbconvert --config mycfg.py
Using nbconvert as a library¶
In this notebook, you will be introduced to the programmatic API of nbconvert and how it can be used in various contexts.
A great blog post by [@jakevdp](https://github.com/jakevdp) will be used to demonstrate. This notebook will not focus on using the command line tool. The attentive reader will point-out that no data is read from or written to disk during the conversion process. This is because nbconvert has been designed to work in memory so that it works well in a database or web-based environment too.
Quick overview¶
Credit: Jonathan Frederic (@jdfreder on github)
The main principle of nbconvert is to instantiate an Exporter
that controls the pipeline through which notebooks are converted.
First, download @jakevdp’s notebook (if you do not have requests
, install it by running pip install requests
, or if you don’t have pip installed, you can find it on PYPI):
[1]:
from urllib.request import urlopen
url = 'http://jakevdp.github.com/downloads/notebooks/XKCD_plots.ipynb'
response = urlopen(url).read().decode()
response[0:60] + ' ...'
[1]:
'{\n "metadata": {\n "name": "XKCD_plots"\n },\n "nbformat": 3,\n ...'
The response is a JSON string which represents a Jupyter notebook.
Next, we will read the response using nbformat. Doing this will guarantee that the notebook structure is valid. Note that the in-memory format and on disk format are slightly different. In particual, on disk, multiline strings might be split into a list of strings.
[2]:
import nbformat
jake_notebook = nbformat.reads(response, as_version=4)
jake_notebook.cells[0]
[2]:
{'cell_type': 'markdown',
'metadata': {},
'source': '# XKCD plots in Matplotlib'}
The nbformat API returns a special type of dictionary. For this example, you don’t need to worry about the details of the structure (if you are interested, please see the nbformat documentation).
The nbconvert API exposes some basic exporters for common formats and defaults. You will start by using one of them. First, you will import one of these exporters (specifically, the HTML exporter), then instantiate it using most of the defaults, and then you will use it to process the notebook we downloaded earlier.
[3]:
from traitlets.config import Config
# 1. Import the exporter
from nbconvert import HTMLExporter
# 2. Instantiate the exporter. We use the `basic` template for now; we'll get into more details
# later about how to customize the exporter further.
html_exporter = HTMLExporter()
html_exporter.template_file = 'basic'
# 3. Process the notebook we loaded earlier
(body, resources) = html_exporter.from_notebook_node(jake_notebook)
The exporter returns a tuple containing the source of the converted notebook, as well as a resources dict. In this case, the source is just raw HTML:
[4]:
print(body[:400] + '...')
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="XKCD-plots-in-Matplotlib">XKCD plots in Matplotlib<a class="anchor-link" href="#XKCD-plots-in-Matplotlib">¶</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div cl...
If you understand HTML, you’ll notice that some common tags are omitted, like the body
tag. Those tags are included in the default HtmlExporter
, which is what would have been constructed if we had not modified the template_file
.
The resource dict contains (among many things) the extracted .png
, .jpg
, etc. from the notebook when applicable. The basic HTML exporter leaves the figures as embedded base64, but you can configure it to extract the figures. So for now, the resource dict should be mostly empty, except for a key containing CSS and a few others whose content will be obvious:
[5]:
print("Resources:", resources.keys())
print("Metadata:", resources['metadata'].keys())
print("Inlining:", resources['inlining'].keys())
print("Extension:", resources['output_extension'])
Resources: dict_keys(['metadata', 'output_extension', 'raw_mimetypes', 'inlining'])
Metadata: dict_keys(['name'])
Inlining: dict_keys(['css'])
Extension: .html
Exporter
s are stateless, so you won’t be able to extract any useful information beyond their configuration. You can re-use an exporter instance to convert another notebook. In addition to the from_notebook_node
used above, each exporter exposes from_file
and from_filename
methods.
Extracting Figures using the RST Exporter¶
When exporting, you may want to extract the base64 encoded figures as files. While the HTML exporter does not do this by default, the RstExporter
does:
[6]:
# Import the RST exproter
from nbconvert import RSTExporter
# Instantiate it
rst_exporter = RSTExporter()
# Convert the notebook to RST format
(body, resources) = rst_exporter.from_notebook_node(jake_notebook)
print(body[:970] + '...')
print('[.....]')
print(body[800:1200] + '...')
XKCD plots in Matplotlib
========================
This notebook originally appeared as a blog post at `Pythonic
Perambulations <http://jakevdp.github.com/blog/2012/10/07/xkcd-style-plots-in-matplotlib/>`__
by Jake Vanderplas.
.. raw:: html
<!-- PELICAN_BEGIN_SUMMARY -->
*Update: the matplotlib pull request has been merged! See* `*This
post* <http://jakevdp.github.io/blog/2013/07/10/XKCD-plots-in-matplotlib/>`__
*for a description of the XKCD functionality now built-in to
matplotlib!*
One of the problems I've had with typical matplotlib figures is that
everything in them is so precise, so perfect. For an example of what I
mean, take a look at this figure:
.. code:: python
from IPython.display import Image
Image('http://jakevdp.github.com/figures/xkcd_version.png')
.. image:: output_3_0.png
Sometimes when showing schematic plots, this is the type of figure I
want to display. But drawing it by hand is a pain: I'd rather just use
matp...
[.....]
image:: output_3_0.png
Sometimes when showing schematic plots, this is the type of figure I
want to display. But drawing it by hand is a pain: I'd rather just use
matplotlib. The problem is, matplotlib is a bit too precise. Attempting
to duplicate this figure in matplotlib leads to something like this:
.. code:: python
Image('http://jakevdp.github.com/figures/mpl_version.png')
.. imag...
Notice that base64 images are not embedded, but instead there are filename-like strings, such as output_3_0.png
. The strings actually are (configurable) keys that map to the binary data in the resources dict.
Note, if you write an RST Plugin, you are responsible for writing all the files to the disk (or uploading, etc…) in the right location. Of course, the naming scheme is configurable.
As an exercise, this notebook will show you how to get one of those images. First, take a look at the 'outputs'
of the returned resources dictionary. This is a dictionary that contains a key for each extracted resource, with values corresponding to the actual base64 encoding:
[7]:
sorted(resources['outputs'].keys())
[7]:
['output_13_1.png',
'output_16_0.png',
'output_18_1.png',
'output_3_0.png',
'output_5_0.png']
In this case, there are 5 extracted binary figures, all png
s. We can use the Image display object to actually display one of the images:
[8]:
from IPython.display import Image
Image(data=resources['outputs']['output_3_0.png'], format='png')
[8]:
Note that this image is being rendered without ever reading or writing to the disk.
Extracting Figures using the HTML Exporter¶
As mentioned above, by default, the HTML exporter does not extract images – it just leaves them as inline base64 encodings. However, this is not always what you might want. For example, here is a use case from @jakevdp:
I write an awesome blog using Jupyter notebooks converted to HTML, and I want the images to be cached. Having one html file with all of the images base64 encoded inside it is nice when sharing with a coworker, but for a website, not so much. I need an HTML exporter, and I want it to extract the figures!
Some theory¶
Before we get into actually extracting the figures, it will be helpful to give a high-level overview of the process of converting a notebook to a another format:
Retrieve the notebook and it’s accompanying resources (you are responsible for this).
Feed the notebook into the
Exporter
, which:Sequentially feeds the notebook into an array of
Preprocessor
s. Preprocessors only act on the structure of the notebook, and have unrestricted access to it.Feeds the notebook into the Jinja templating engine, which converts it to a particular format depending on which template is selected.
The exporter returns the converted notebook and other relevant resources as a tuple.
You write the data to the disk using the built-in
FilesWriter
(which writes the notebook and any extracted files to disk), or elsewhere using a customWriter
.
Using different preprocessors¶
To extract the figures when using the HTML exporter, we will want to change which Preprocessor
s we are using. There are several preprocessors that come with nbconvert, including one called the ExtractOutputPreprocessor
.
The ExtractOutputPreprocessor
is responsible for crawling the notebook, finding all of the figures, and putting them into the resources directory, as well as choosing the key (i.e. filename_xx_y.extension
) that can replace the figure inside the template. To enable the ExtractOutputPreprocessor
, we must add it to the exporter’s list of preprocessors:
[9]:
# create a configuration object that changes the preprocessors
from traitlets.config import Config
c = Config()
c.HTMLExporter.preprocessors = ['nbconvert.preprocessors.ExtractOutputPreprocessor']
# create the new exporter using the custom config
html_exporter_with_figs = HTMLExporter(config=c)
html_exporter_with_figs.preprocessors
[9]:
['nbconvert.preprocessors.ExtractOutputPreprocessor']
We can compare the result of converting the notebook using the original HTML exporter and our new customized one:
[10]:
(_, resources) = html_exporter.from_notebook_node(jake_notebook)
(_, resources_with_fig) = html_exporter_with_figs.from_notebook_node(jake_notebook)
print("resources without figures:")
print(sorted(resources.keys()))
print("\nresources with extracted figures (notice that there's one more field called 'outputs'):")
print(sorted(resources_with_fig.keys()))
print("\nthe actual figures are:")
print(sorted(resources_with_fig['outputs'].keys()))
resources without figures:
['inlining', 'metadata', 'output_extension', 'raw_mimetypes']
resources with extracted figures (notice that there's one more field called 'outputs'):
['inlining', 'metadata', 'output_extension', 'outputs', 'raw_mimetypes']
the actual figures are:
['output_13_1.png', 'output_16_0.png', 'output_18_1.png', 'output_3_0.png', 'output_5_0.png']
Custom Preprocessors¶
There are an endless number of transformations that you may want to apply to a notebook. In particularly complicated cases, you may want to actually create your own Preprocessor
. Above, when we customized the list of preprocessors accepted by the HTMLExporter
, we passed in a string – this can be any valid module name. So, if you create your own preprocessor, you can include it in that same list and it will be used by the exporter.
To create your own preprocessor, you will need to subclass from nbconvert.preprocessors.Preprocessor
and overwrite either the preprocess
and/or preprocess_cell
methods.
Example¶
The following demonstration adds the ability to exclude a cell by index.
Note: injecting cells is similar, and won’t be covered here. If you want to inject static content at the beginning/end of a notebook, use a custom template.
[11]:
from traitlets import Integer
from nbconvert.preprocessors import Preprocessor
class PelicanSubCell(Preprocessor):
"""A Pelican specific preprocessor to remove some of the cells of a notebook"""
# I could also read the cells from nb.metadata.pelican if someone wrote a JS extension,
# but for now I'll stay with configurable value.
start = Integer(0, help="first cell of notebook to be converted")
end = Integer(-1, help="last cell of notebook to be converted")
start.tag(config='True')
end.tag(config='True')
def preprocess(self, nb, resources):
self.log.info("I'll keep only cells from %d to %d", self.start, self.end)
nb.cells = nb.cells[self.start:self.end]
return nb, resources
Here a Pelican exporter is created that takes PelicanSubCell
preprocessors and a config
object as parameters. This may seem redundant, but with the configuration system you can register an inactive preprocessor on all of the exporters and activate it from config files or the command line.
[12]:
# Create a new config object that configures both the new preprocessor, as well as the exporter
c = Config()
c.PelicanSubCell.start = 4
c.PelicanSubCell.end = 6
c.RSTExporter.preprocessors = [PelicanSubCell]
# Create our new, customized exporter that uses our custom preprocessor
pelican = RSTExporter(config=c)
# Process the notebook
print(pelican.from_notebook_node(jake_notebook)[0])
Sometimes when showing schematic plots, this is the type of figure I
want to display. But drawing it by hand is a pain: I'd rather just use
matplotlib. The problem is, matplotlib is a bit too precise. Attempting
to duplicate this figure in matplotlib leads to something like this:
.. code:: python
Image('http://jakevdp.github.com/figures/mpl_version.png')
.. image:: output_5_0.png
Programmatically creating templates¶
[13]:
from jinja2 import DictLoader
dl = DictLoader({'full.tpl':
"""
{%- extends 'basic.tpl' -%}
{% block footer %}
FOOOOOOOOTEEEEER
{% endblock footer %}
"""})
exportHTML = HTMLExporter(extra_loaders=[dl])
(body, resources) = exportHTML.from_notebook_node(jake_notebook)
for l in body.split('\n')[-4:]:
print(l)
</div>
</div>
FOOOOOOOOTEEEEER
Real World Uses¶
@jakevdp uses Pelican and Jupyter Notebook to blog. Pelican will use nbconvert programmatically to generate blog post. Have a look a Pythonic Preambulations for Jake’s blog post.
@damianavila wrote the Nikola Plugin to write blog post as Notebooks and is developping a js-extension to publish notebooks via one click from the web app.
As @Mbussonn requested… easieeeeer! Deploy your Nikola site with just a click in the IPython notebook! http://t.co/860sJunZvj cc @ralsina
— Damián Avila (@damian_avila) August 21, 2013
LaTeX citations¶
nbconvert
now has support for LaTeX citations. With this capability you
can:
Manage citations using BibTeX.
Cite those citations in Markdown cells using HTML data attributes.
Have
nbconvert
generate proper LaTeX citations and run BibTeX.
For an example of how this works, please see the citations example in the nbconvert-examples repository.
Executing notebooks¶
Jupyter notebooks are often saved with output cells that have been cleared. nbconvert provides a convenient way to execute the input cells of an .ipynb notebook file and save the results, both input and output cells, as a .ipynb file.
In this section we show how to execute a .ipynb
notebook
document saving the result in notebook format. If you need to export
notebooks to other formats, such as reStructured Text or Markdown (optionally
executing them) see section Using nbconvert as a library.
Executing notebooks can be very helpful, for example, to run all notebooks in Python library in one step, or as a way to automate the data analysis in projects involving more than one notebook.
Executing notebooks from the command line¶
The same functionality of executing notebooks is exposed through a command line interface or a Python API interface. As an example, a notebook can be executed from the command line with:
jupyter nbconvert --to notebook --execute mynotebook.ipynb
Executing notebooks using the Python API interface¶
This section will illustrate the Python API interface.
Example¶
Let’s start with a complete quick example, leaving detailed explanations to the following sections.
Import: First we import nbconvert and the ExecutePreprocessor
class:
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
Load: Assuming that notebook_filename
contains the path of a notebook,
we can load it with:
with open(notebook_filename) as f:
nb = nbformat.read(f, as_version=4)
Configure: Next, we configure the notebook execution mode:
ep = ExecutePreprocessor(timeout=600, kernel_name='python3')
We specified two (optional) arguments timeout
and kernel_name
, which
define respectively the cell execution timeout and the execution kernel.
The option to specify kernel_name is new in nbconvert 4.2. When not specified or when using nbconvert <4.2, the default Python kernel is chosen.
Execute/Run (preprocess): To actually run the notebook we call the method
preprocess
:
ep.preprocess(nb, {'metadata': {'path': 'notebooks/'}})
Hopefully, we will not get any errors during the notebook execution
(see the last section for error handling). Note that path
specifies
in which folder to execute the notebook.
Save: Finally, save the resulting notebook with:
with open('executed_notebook.ipynb', 'w', encoding='utf-8') as f:
nbformat.write(nb, f)
That’s all. Your executed notebook will be saved in the current folder
in the file executed_notebook.ipynb
.
Execution arguments (traitlets)¶
The arguments passed to ExecutePreprocessor
are configuration options
called traitlets.
There are many cool things about traitlets. For example,
they enforce the input type, and they can be accessed/modified as
class attributes. Moreover, each traitlet is automatically exposed
as command-line options. For example, we can pass the timeout from the
command-line like this:
jupyter nbconvert --ExecutePreprocessor.timeout=600 --to notebook --execute mynotebook.ipynb
Let’s now discuss in more detail the two traitlets we used.
The timeout
traitlet defines the maximum time (in seconds) each notebook
cell is allowed to run, if the execution takes longer an exception will be
raised. The default is 30 s, so in cases of long-running cells you may want to
specify an higher value. The timeout
option can also be set to None
or -1
to remove any restriction on execution time.
The second traitlet, kernel_name
, allows specifying the name of the kernel
to be used for the execution. By default, the kernel name is obtained from the
notebook metadata. The traitlet kernel_name
allows specifying a
user-defined kernel, overriding the value in the notebook metadata. A common
use case is that of a Python 2/3 library which includes documentation/testing
notebooks. These notebooks will specify either a python2 or python3 kernel in
their metadata (depending on the kernel used the last time the notebook was
saved). In reality, these notebooks will work on both Python 2 and Python 3,
and, for testing, it is important to be able to execute them programmatically
on both versions. Here the traitlet kernel_name
helps simplify and
maintain consistency: we can just run a notebook twice, specifying first
“python2” and then “python3” as the kernel name.
Handling errors and exceptions¶
In the previous sections we saw how to save an executed notebook, assuming there are no execution errors. But, what if there are errors?
Execution until first error¶
An error during the notebook execution, by default, will stop the execution
and raise a CellExecutionError
. Conveniently, the source cell causing
the error and the original error name and message are also printed.
After an error, we can still save the notebook as before:
with open('executed_notebook.ipynb', mode='w', encoding='utf-8') as f:
nbformat.write(nb, f)
The saved notebook contains the output up until the failing cell, and includes a full stack-trace and error (which can help debugging).
Handling errors¶
A useful pattern to execute notebooks while handling errors is the following:
from nbconvert.preprocessors import CellExecutionError
try:
out = ep.preprocess(nb, {'metadata': {'path': run_path}})
except CellExecutionError:
out = None
msg = 'Error executing the notebook "%s".\n\n' % notebook_filename
msg += 'See notebook "%s" for the traceback.' % notebook_filename_out
print(msg)
raise
finally:
with open(notebook_filename_out, mode='w', encoding='utf-8') as f:
nbformat.write(nb, f)
This will save the executed notebook regardless of execution errors.
In case of errors, however, an additional message is printed and the
CellExecutionError
is raised. The message directs the user to
the saved notebook for further inspection.
Execute and save all errors¶
As a last scenario, it is sometimes useful to execute notebooks which raise
exceptions, for example to show an error condition. In this case, instead of
stopping the execution on the first error, we can keep executing the notebook
using the traitlet allow_errors
(default is False). With
allow_errors=True
, the notebook is executed until the end, regardless of
any error encountered during the execution. The output notebook, will contain
the stack-traces and error messages for all the cells raising exceptions.
Widget state¶
If your notebook contains any Jupyter Widgets, the state of all the widgets can be stored in the notebook’s metadata. This allows rendering of the live widgets on for instance nbviewer, or when converting to html.
We can tell nbconvert to not store the state using the store_widget_state argument:
jupyter nbconvert --ExecutePreprocessor.store_widget_state=False --to notebook --execute mynotebook.ipynb
This widget rendering is not performed against a browser during execution, so only widget default states or states manipulated via user code will be calculated during execution. %%javascript cells will execute upon notebook rendering, enabling complex interactions to function as expected when viewed by a UI.
If you can’t view widget results after execution, you may need to select Trust Notebook under the File menu.
Configuration options¶
Configuration options may be set in a file, ~/.jupyter/jupyter_nbconvert_config.py
,
or at the command line when starting nbconvert, i.e. jupyter nbconvert --Application.log_level=10
.
- Application.log_datefmtUnicode
Default:
'%Y-%m-%d %H:%M:%S'
The date format used by logging formatters for %(asctime)s
- Application.log_formatUnicode
Default:
'[%(name)s]%(highlevel)s %(message)s'
The Logging format template
- Application.log_level0|10|20|30|40|50|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’
Default:
30
Set the log level by value or name.
- JupyterApp.answer_yesBool
Default:
False
Answer yes to any prompts.
- JupyterApp.config_fileUnicode
Default:
''
Full path of a config file.
- JupyterApp.config_file_nameUnicode
Default:
''
Specify a config file to load.
- JupyterApp.generate_configBool
Default:
False
Generate default config file.
- NbConvertApp.export_formatUnicode
Default:
'html'
The export format to be used, either one of the built-in formats [‘asciidoc’, ‘custom’, ‘html’, ‘latex’, ‘markdown’, ‘notebook’, ‘pdf’, ‘python’, ‘rst’, ‘script’, ‘slides’] or a dotted object name that represents the import path for an Exporter class
- NbConvertApp.from_stdinBool
Default:
False
read a single notebook from stdin.
- NbConvertApp.ipywidgets_base_urlUnicode
Default:
'https://unpkg.com/'
URL base for ipywidgets package
- NbConvertApp.notebooksList
Default:
[]
List of notebooks to convert. Wildcards are supported. Filenames passed positionally will be added to the list.
- NbConvertApp.output_baseUnicode
Default:
''
overwrite base name use for output files. can only be used when converting one notebook at a time.
- NbConvertApp.output_files_dirUnicode
Default:
'{notebook_name}_files'
Directory to copy extra files (figures) to. ‘{notebook_name}’ in the string will be converted to notebook basename
- NbConvertApp.postprocessor_classDottedOrNone
Default:
''
PostProcessor class used to write the results of the conversion
- NbConvertApp.use_output_suffixBool
Default:
True
Whether to apply a suffix prior to the extension (only relevant when converting to notebook format). The suffix is determined by the exporter, and is usually ‘.nbconvert’.
- NbConvertApp.writer_classDottedObjectName
Default:
'FilesWriter'
Writer class used to write the results of the conversion
- NbConvertBase.default_languageUnicode
Default:
'ipython'
Deprecated default highlight language as of 5.0, please use language_info metadata instead
- NbConvertBase.display_data_priorityList
Default:
['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.
- Exporter.default_preprocessorsList
Default:
['nbconvert.preprocessors.TagRemovePreprocessor', 'nbconvert....
List of preprocessors available by default, by name, namespace, instance, or type.
- Exporter.file_extensionFilenameExtension
Default:
'.txt'
Extension of the file that should be written to disk
- Exporter.preprocessorsList
Default:
[]
List of preprocessors, by name or namespace, to enable.
- TemplateExporter.exclude_code_cellBool
Default:
False
This allows you to exclude code cells from all templates if set to True.
- TemplateExporter.exclude_inputBool
Default:
False
This allows you to exclude code cell inputs from all templates if set to True.
- TemplateExporter.exclude_input_promptBool
Default:
False
This allows you to exclude input prompts from all templates if set to True.
- TemplateExporter.exclude_markdownBool
Default:
False
This allows you to exclude markdown cells from all templates if set to True.
- TemplateExporter.exclude_outputBool
Default:
False
This allows you to exclude code cell outputs from all templates if set to True.
- TemplateExporter.exclude_output_promptBool
Default:
False
This allows you to exclude output prompts from all templates if set to True.
- TemplateExporter.exclude_rawBool
Default:
False
This allows you to exclude raw cells from all templates if set to True.
- TemplateExporter.exclude_unknownBool
Default:
False
This allows you to exclude unknown cells from all templates if set to True.
- TemplateExporter.filtersDict
Default:
{}
Dictionary of filters, by name and namespace, to add to the Jinja environment.
- TemplateExporter.raw_mimetypesList
Default:
[]
formats of raw cells to be included in this Exporter’s output.
- TemplateExporter.template_extensionUnicode
Default:
'.tpl'
No description
- TemplateExporter.template_fileUnicode
Default:
''
Name of the template file to use
- TemplateExporter.template_pathList
Default:
['.']
No description
- HTMLExporter.anchor_link_textUnicode
Default:
'¶'
The text used as the text for anchor links.
- LatexExporter.template_extensionUnicode
Default:
'.tplx'
No description
- NotebookExporter.nbformat_version1|2|3|4
Default:
4
The nbformat version to write. Use this to downgrade notebooks.
- PDFExporter.bib_commandList
Default:
['bibtex', '{filename}']
Shell command used to run bibtex.
- PDFExporter.latex_commandList
Default:
['xelatex', '{filename}', '-quiet']
Shell command used to compile latex.
- PDFExporter.latex_countInt
Default:
3
How many times latex will be called.
- PDFExporter.verboseBool
Default:
False
Whether to display the output of latex commands.
- SlidesExporter.font_awesome_urlUnicode
Default:
'https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/cs...
URL to load font awesome from.
Defaults to loading from cdnjs.
- SlidesExporter.jquery_urlUnicode
Default:
'https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.m...
URL to load jQuery from.
Defaults to loading from cdnjs.
- SlidesExporter.require_js_urlUnicode
Default:
'https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/req...
URL to load require.js from.
Defaults to loading from cdnjs.
- SlidesExporter.reveal_scrollBool
Default:
False
If True, enable scrolling within each slide
- SlidesExporter.reveal_themeUnicode
Default:
'simple'
Name of the reveal.js theme to use.
We look for a file with this name under
reveal_url_prefix
/css/theme/reveal_theme
.css.https://github.com/hakimel/reveal.js/tree/master/css/theme has list of themes that ship by default with reveal.js.
- SlidesExporter.reveal_transitionUnicode
Default:
'slide'
Name of the reveal.js transition to use.
The list of transitions that ships by default with reveal.js are: none, fade, slide, convex, concave and zoom.
- SlidesExporter.reveal_url_prefixUnicode
Default:
''
The URL prefix for reveal.js (version 3.x). This defaults to the reveal CDN, but can be any url pointing to a copy of reveal.js.
For speaker notes to work, this must be a relative path to a local copy of reveal.js: e.g., “reveal.js”.
If a relative path is given, it must be a subdirectory of the current directory (from which the server is run).
See the usage documentation (https://nbconvert.readthedocs.io/en/latest/usage.html#reveal-js-html-slideshow) for more details.
- Preprocessor.enabledBool
Default:
False
No description
- CSSHTMLHeaderPreprocessor.highlight_classUnicode
Default:
'.highlight'
CSS highlight class identifier
- CSSHTMLHeaderPreprocessor.styleUnicode
Default:
'default'
Name of the pygments style to use
- ClearOutputPreprocessor.remove_metadata_fieldsSet
Default:
{'collapsed', 'scrolled'}
No description
- ConvertFiguresPreprocessor.from_formatUnicode
Default:
''
Format the converter accepts
- ConvertFiguresPreprocessor.to_formatUnicode
Default:
''
Format the converter writes
- ExecutePreprocessor.allow_errorsBool
Default:
False
If False (default), when a cell raises an error the execution is stopped and a CellExecutionError is raised. If True, execution errors are ignored and the execution is continued until the end of the notebook. Output from exceptions is included in the cell output in both cases.
- ExecutePreprocessor.force_raise_errorsBool
Default:
False
If False (default), errors from executing the notebook can be allowed with a raises-exception tag on a single cell, or the allow_errors configurable option for all cells. An allowed error will be recorded in notebook output, and execution will continue. If an error occurs when it is not explicitly allowed, a CellExecutionError will be raised. If True, CellExecutionError will be raised for any error that occurs while executing the notebook. This overrides both the allow_errors option and the raises-exception cell tag.
- ExecutePreprocessor.interrupt_on_timeoutBool
Default:
False
If execution of a cell times out, interrupt the kernel and continue executing other cells rather than throwing an error and stopping.
- ExecutePreprocessor.iopub_timeoutInt
Default:
4
The time to wait (in seconds) for IOPub output. This generally doesn’t need to be set, but on some slow networks (such as CI systems) the default timeout might not be long enough to get all messages.
- ExecutePreprocessor.kernel_manager_classType
Default:
'builtins.object'
The kernel manager class to use.
- ExecutePreprocessor.kernel_nameUnicode
Default:
''
Name of kernel to use to execute the cells. If not set, use the kernel_spec embedded in the notebook.
- ExecutePreprocessor.raise_on_iopub_timeoutBool
Default:
False
If False (default), then the kernel will continue waiting for iopub messages until it receives a kernel idle message, or until a timeout occurs, at which point the currently executing cell will be skipped. If True, then an error will be raised after the first timeout. This option generally does not need to be used, but may be useful in contexts where there is the possibility of executing notebooks with memory-consuming infinite loops.
- ExecutePreprocessor.shutdown_kernel‘graceful’|’immediate’
Default:
'graceful'
If graceful (default), then the kernel is given time to clean up after executing all cells, e.g., to execute its atexit hooks. If immediate, then the kernel is signaled to immediately terminate.
- ExecutePreprocessor.startup_timeoutInt
Default:
60
The time to wait (in seconds) for the kernel to start. If kernel startup takes longer, a RuntimeError is raised.
- ExecutePreprocessor.store_widget_stateBool
Default:
True
If True (default), then the state of the Jupyter widgets created at the kernel will be stored in the metadata of the notebook.
- ExecutePreprocessor.timeoutInt
Default:
30
The time to wait (in seconds) for output from executions. If a cell execution takes longer, an exception (TimeoutError on python 3+, RuntimeError on python 2) is raised.
None or -1 will disable the timeout. If timeout_func is set, it overrides timeout.
- ExecutePreprocessor.timeout_funcAny
Default:
None
A callable which, when given the cell source as input, returns the time to wait (in seconds) for output from cell executions. If a cell execution takes longer, an exception (TimeoutError on python 3+, RuntimeError on python 2) is raised.
Returning None or -1 will disable the timeout for the cell. Not setting timeout_func will cause the preprocessor to default to using the timeout trait for all cells. The timeout_func trait overrides timeout if it is not None.
- ExtractOutputPreprocessor.extract_output_typesSet
Default:
{'image/svg+xml', 'image/jpeg', 'image/png', 'application/pdf'}
No description
- ExtractOutputPreprocessor.output_filename_templateUnicode
Default:
'{unique_key}_{cell_index}_{index}{extension}'
No description
- HighlightMagicsPreprocessor.languagesDict
Default:
{}
Syntax highlighting for magic’s extension languages. Each item associates a language magic extension such as %%R, with a pygments lexer such as r.
- LatexPreprocessor.styleUnicode
Default:
'default'
Name of the pygments style to use
- RegexRemovePreprocessor.patternsList
Default:
[]
No description
- SVG2PDFPreprocessor.commandUnicode
Default:
''
The command to use for converting SVG to PDF
This string is a template, which will be formatted with the keys to_filename and from_filename.
The conversion call must read the SVG from {from_flename}, and write a PDF to {to_filename}.
- SVG2PDFPreprocessor.inkscapeUnicode
Default:
''
The path to Inkscape, if necessary
- TagRemovePreprocessor.remove_all_outputs_tagsSet
Default:
set()
Tags indicating cells for which the outputs are to be removed,matches tags in cell.metadata.tags.
- TagRemovePreprocessor.remove_cell_tagsSet
Default:
set()
Tags indicating which cells are to be removed,matches tags in cell.metadata.tags.
- TagRemovePreprocessor.remove_input_tagsSet
Default:
set()
Tags indicating cells for which input is to be removed,matches tags in cell.metadata.tags.
- TagRemovePreprocessor.remove_single_output_tagsSet
Default:
set()
Tags indicating which individual outputs are to be removed,matches output i tags in cell.outputs[i].metadata.tags.
- WriterBase.filesList
Default:
[]
List of the files that the notebook references. Files will be included with written output.
- FilesWriter.build_directoryUnicode
Default:
''
Directory to write output(s) to. Defaults to output to the directory of each notebook. To recover previous default behaviour (outputting to the current working directory) use . as the flag value.
- FilesWriter.relpathUnicode
Default:
''
When copying files that the notebook depends on, copy them in relation to this path, such that the destination filename will be os.path.relpath(filename, relpath). If FilesWriter is operating on a notebook that already exists elsewhere on disk, then the default will be the directory containing that notebook.
- ServePostProcessor.browserUnicode
Default:
''
Specify what browser should be used to open slides. See https://docs.python.org/3/library/webbrowser.html#webbrowser.register to see how keys are mapped to browser executables. If not specified, the default browser will be determined by the webbrowser standard library module, which allows setting of the BROWSER environment variable to override it.
- ServePostProcessor.ipUnicode
Default:
'127.0.0.1'
The IP address to listen on.
- ServePostProcessor.open_in_browserBool
Default:
True
Should the browser be opened automatically?
- ServePostProcessor.portInt
Default:
8000
port for the server to listen on.
- ServePostProcessor.reveal_cdnUnicode
Default:
'https://cdnjs.cloudflare.com/ajax/libs/reveal.js/3.5.0'
URL for reveal.js CDN.
- ServePostProcessor.reveal_prefixUnicode
Default:
'reveal.js'
URL prefix for reveal.js
Customizing nbconvert¶
Under the hood, nbconvert uses Jinja templates to specify how the notebooks should be formatted. These templates can be fully customized, allowing you to use nbconvert to create notebooks in different formats with different styles as well.
Converting a notebook to an (I)Python script and printing to stdout¶
Out of the box, nbconvert can be used to convert notebooks to plain Python files. For example, the following command converts the example.ipynb
notebook to Python and prints out the result:
[1]:
!jupyter nbconvert --to python 'example.ipynb' --stdout
[NbConvertApp] Converting notebook example.ipynb to python
# coding: utf-8
# # Example notebook
# ### Markdown cells
#
# This is an example notebook that can be converted with `nbconvert` to different formats. This is an example of a markdown cell.
# ### LaTeX Equations
#
# Here is an equation:
#
# $$
# y = \sin(x)
# $$
# ### Code cells
# In[1]:
print("This is a code cell that produces some output")
# ### Inline figures
# In[1]:
import matplotlib.pyplot as plt
import numpy as np
plt.ion()
x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
From the code, you can see that non-code cells are also exported. If you wanted to change that behaviour, you would first look to nbconvert configuration options page to see if there is an option available that can give you your desired behaviour.
In this case, if you wanted to remove code cells from the output, you could use the TemplateExporter.exclude_markdown
traitlet directly, as below.
[2]:
!jupyter nbconvert --to python 'example.ipynb' --stdout --TemplateExporter.exclude_markdown=True
[NbConvertApp] Converting notebook example.ipynb to python
# coding: utf-8
# In[1]:
print("This is a code cell that produces some output")
# In[1]:
import matplotlib.pyplot as plt
import numpy as np
plt.ion()
x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
Custom Templates¶
As mentioned above, if you want to change this behavior, you can use a custom template. The custom template inherits from the Python template and overwrites the markdown blocks so that they are empty.
Below is an example of a custom template, which we write to a file called simplepython.tpl
. This template removes markdown cells from the output, and also changes how the execution count numbers are formatted:
[3]:
%%writefile simplepython.tpl
{% extends 'python.tpl'%}
## remove markdown cells
{% block markdowncell %}
{% endblock markdowncell %}
## change the appearance of execution count
{% block in_prompt %}
# [{{ cell.execution_count if cell.execution_count else ' ' }}]:
{% endblock in_prompt %}
Overwriting simplepython.tpl
Using this template, we see that the resulting Python code does not contain anything that was previously in a markdown cell, and only displays execution counts (i.e., [#]:
not In[#]:
):
[4]:
!jupyter nbconvert --to python 'example.ipynb' --stdout --template=simplepython.tpl
[NbConvertApp] Converting notebook example.ipynb to python
# coding: utf-8
# [1]:
print("This is a code cell that produces some output")
# [1]:
import matplotlib.pyplot as plt
import numpy as np
plt.ion()
x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
Template structure¶
Nbconvert templates consist of a set of nested blocks. When defining a new template, you extend an existing template by overriding some of the blocks.
All the templates shipped in nbconvert have the basic structure described here, though some may define additional blocks.
[5]:
from IPython.display import HTML, display
with open('template_structure.html') as f:
display(HTML(f.read()))
Main page
Outputs
Extra HTML blocks
basic.tpl
full.tpl
<head>
</head>
Extra Latex blocks
A few gotchas¶
Jinja blocks use {% %}
by default which does not play nicely with LaTeX, so those are replaced by ((* *))
in LaTeX templates.
Templates using cell tags¶
The notebook file format supports attaching arbitrary JSON metadata to each cell. In addition, every cell has a special tags
metadata field that accepts a list of strings that indicate the cell’s tags. To apply these, go to the View → CellToolbar → Tags
option which will create a Tag editor at the top of every cell.
First choose a notebook you want to convert to html, and apply the tags: "Easy"
, "Medium"
, or "Hard"
.
With this in place, the notebook can be converted using a custom template.
Design your template in the cells provided below.
Hint: tags are located at cell.metadata.tags
, the following Python code collects the value of the tag:
cell['metadata'].get('tags', [])
Which you can then use inside a Jinja template as in the following:
[6]:
%%writefile mytemplate.tpl
{% extends 'full.tpl'%}
{% block any_cell %}
{% if 'Hard' in cell['metadata'].get('tags', []) %}
<div style="border:thin solid red">
{{ super() }}
</div>
{% elif 'Medium' in cell['metadata'].get('tags', []) %}
<div style="border:thin solid orange">
{{ super() }}
</div>
{% elif 'Easy' in cell['metadata'].get('tags', []) %}
<div style="border:thin solid green">
{{ super() }}
</div>
{% else %}
{{ super() }}
{% endif %}
{% endblock any_cell %}
Overwriting mytemplate.tpl
Now, if we collect the result of using nbconvert with this template, and display the resulting html, we see the following:
[7]:
example = !jupyter nbconvert --to html 'example.ipynb' --template=mytemplate.tpl --stdout
example = example[3:] # have to remove the first three lines which are not proper html
from IPython.display import HTML, display
display(HTML('\n'.join(example)))
Example notebook¶
Markdown cells¶
This is an example notebook that can be converted with nbconvert
to different formats. This is an example of a markdown cell.
LaTeX Equations¶
Here is an equation:
$$ y = \sin(x) $$Code cells¶
print("This is a code cell that produces some output")
Inline figures¶
import matplotlib.pyplot as plt
import numpy as np
plt.ion()
x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
Templates using custom cell metadata¶
We demonstrated above how to use cell tags in a template to apply custom styling to a notebook. But remember, the notebook file format supports attaching arbitrary JSON metadata to each cell, not only cell tags. Here, we describe an exercise for using an example.difficulty
metadata field (rather than cell tags) to do the same as before (to mark up different cells as being “Easy”, “Medium” or “Hard”).
How to edit cell metadata¶
To edit the cell metadata from within the notebook, go to the menu item: View → Cell Toolbar → Edit Metadata
. This will bring up a toolbar above each cell with a button that says “Edit Metadata”. Click this button, and a field will pop up in which you will directly edit the cell metadata JSON.
NB: Because it is JSON, you will need to ensure that what you write is valid JSON.
Template challenges: dealing with missing custom metadata fields¶
One of the challenges of dealing with custom metadata is to handle the case where the metadata is not present on every cell. This can get somewhat tricky because of JSON objects tendency to be deeply nested coupled with Python’s (and therefore Jinja’s) approach to calling into dictionaries. Specifically, the following code will error:
foo = {}
foo["bar"]
Accordingly, it is better to use the `{}.get
method <https://docs.python.org/3.6/library/stdtypes.html#dict.get>`__ which allows you to set a default value to return if no key is found as the second argument.
Hint: if your metadata items are located at cell.metadata.example.difficulty
, the following Python code would get the value defaulting to an empty string (''
) if nothing is found:
cell['metadata'].get('example', {}).get('difficulty', '')
Exercise: Write a template for handling custom metadata¶
Now, write a template that will look for Easy
, Medium
and Hard
metadata values for the cell.metadata.example.difficulty
field and wrap them in a div with a green, orange, or red thin solid border (respectively).
NB: This is the same design and logic as used in the previous cell tag example.
How to get example.ipynb
¶
We have provided an example file in example.ipynb
in the nbconvert documentation that has already been marked up with both tags and the above metadata for you to test with. You can get it from this link to the raw file or by cloning the repository from GitHub and navingating to nbconvert/docs/source/example.ipynb
.
Convert example.ipynb
using cell tags¶
First, make sure that you can reproduce the previous result using the cell tags template that we have provided above.
Easy: If you want to make it easy on yourself, create a new file my_template.tpl
in the same directory as example.ipynb
and copy the contents of the cell we use to write mytemplate.tpl
to the file system.
Then run jupyter nbconvert --to html 'example.ipynb' --template=mytemplate.tpl
and see if your
Moderate: If you want more of a challenge, try recreating the jinja template by modifying the following jinja template file:
{% extends 'full.tpl'%}
{% block any_cell %}
<div style="border:thin solid red">
{{ super() }}
</div>
{% endblock any_cell %}
Hard: If you want even more of a challenge, try recreating the jinja template from scratch.
Write your template¶
Once you’ve done at least the Easy version of the previous step, try modifying your template to use cell.metadata.example.difficulty
fields rather than cell tags.
Convert example.ipynb
with formatting from custom metadata¶
Once you’ve written your template, try converting example.ipynb
using the following command (making sure that your_template.tpl
is in your local directory where you are running the command):
jupyter nbconvert --to html 'example.ipynb' --template=your_template.tpl --stdout
The resulting display should pick out different cells to be bordered with green, orange, or red.
If you do that successfullly, the resulting html document should look like the following cell’s contents:
example
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS_HTML"></script>
<!-- MathJax configuration -->
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
processEscapes: true,
processEnvironments: true
},
// Center justify equations in code and markdown cells. Elsewhere
// we use CSS to left justify single line equations in code cells.
displayAlign: 'center',
"HTML-CSS": {
styles: {'.MathJax_Display': {"margin": 0}},
linebreaks: { automatic: true }
}
});
</script>
<!-- End of mathjax configuration --></head>
<div style="border:thin solid red">
Example notebook¶
</div>
Markdown cells¶
This is an example notebook that can be converted with nbconvert to different formats. This is an example of a markdown cell.
LaTeX Equations¶
Here is an equation:
<div style="border:thin solid green">
Code cells¶
</div>
In [1]:
print("This is a code cell that produces some output")
Inline figures¶
<div style="border:thin solid orange">
In [1]:
import matplotlib.pyplot as plt
import numpy as np
plt.ion()
x = np.linspace(0, 2 * np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
Out[1]:
</div>
In [ ]:
</div>
Customizing exporters¶
New in version 4.2: You can now use the --to
flag to use custom export formats defined
outside nbconvert.
The command-line syntax to run the nbconvert
script is:
jupyter nbconvert --to FORMAT notebook.ipynb
This will convert the Jupyter document file notebook.ipynb
into the output
format designated by the FORMAT
string as explained below.
Extending the built-in format exporters¶
A few built-in formats are available by default: html, pdf, script, latex. Each of these has its own exporter with many configuration options that can be extended. Having the option to point to a different exporter allows authors to create their own fully customized templates or export formats.
A custom exporter must be an importable Python object. We recommend that these be distributed as Python libraries.
Registering a custom exporter as an entry point¶
Additional exporters may be registered as named entry_points.
nbconvert uses the nbconvert.exporters
entry point to find exporters
from any package you may have installed.
If you are writing a Python package that provides custom exporters,
you can register the custom exporters in your package’s setup.py
. For
example, your package may contain two custom exporters, named “simple” and
“detail”, and can be registered in your package’s setup.py
as follows:
setup(
...
entry_points = {
'nbconvert.exporters': [
'simple = mymodule:SimpleExporter',
'detail = mymodule:DetailExporter',
],
}
)
Now people who have installed your Python package containing the two custom exporters can call the entry point name:
jupyter nbconvert --to detail mynotebook.ipynb
instead of having to specify the full import name of the custom exporter.
Using a custom exporter without entrypoints¶
We encourage registering custom exporters as entry points as described in the
previous section. Registering a custom exporter with an entry point simplifies
using the exporter. If a custom exporter has not been registered with an
entry point, the exporter can still be used by providing the fully qualified
name of this exporter as the argument of the --to
flag when running from
the command line:
$ jupyter nbconvert --to <full.qualified.name of custom exporter> notebook.ipynb
For example, assuming a library tcontrib has a custom exporter name TExporter, you would convert to this custom format using the following:
$ jupyter nbconvert --to tcontrib.TExporter notebook.ipynb
A library can contain multiple exporters. Creators of custom exporters should make sure that all other flags of the command line behave the same for the custom exporters as for built-in exporters.
Parameters controlled by an external exporter¶
An external exporter can control almost any parameter of the notebook conversion process, from simple parameters such as the output file extension, to more complex ones such as the execution of the notebook or a custom rendering template.
All external exporters can expose custom options using the traitlets
configurable API. Refer to the library that provides these exporters for
details on how these configuration options works.
You can use the Jupyter configuration files to configure an external exporter. As
for any nbconvert
exporters you can use either the configuration file syntax of
c.MyExporter.config_option=value
or the command line flag form
--MyExporter.config_option=value
.
Writing a custom Exporter
¶
Under the hood exporters are python classes that expose a certain interface. Any importable classes that expose this interface can be use as an exporter for nbconvert.
For simplicity we expose basic classes that implement all the relevant methods that you have to subclass and overwrite just the relevant methods to provide a custom exporter. Below we show you the step to create a custom exporter that provides a custom file extension, and a custom template that inserts before and after each markdown cell.
We will lay out files to be ready for Python packaging and distributing on PyPI, although the exact art of Python packaging is beyond the scope of this explanation.
We will use the following layout for our package to expose a custom exporter:
mypackage
├── LICENSE.md
├── setup.py
└── mypackage
├── __init__.py
└── templates
└── test_template.tpl
If you wished to create this same directory structure you could use the following commands
when you are at the directory under which you wish to build your mypackage
package:
mkdir -p mypackage/mypackage/templates
touch mypackage/LICENSE.md
touch mypackage/setup.py
touch mypackage/mypackage/__init__.py
touch mypackage/mypackage/templates/test_template.tpl
Important
You should not publish this package without adding content to your LICENSE.md
file.
For example, nbconvert
follows the Jupyter Project convention of using a Modified BSD
License (also known as New or Revised or 3-Clause BSD).
For a guide on picking the right license for your use case,
please see choose a license.
If you do not specify the license, your code may be unusable by many open source projects.
As you can see the layout is relatively simple, in the case where a template is not needed we would actually have only one file with an Exporter implementation. Of course you can change the layout of your package to have a more fine-grained structure of the subpackage. But lets see what a minimum example looks like.
We are going to write an exporter that:
exports to html, so we will reuse the built-in html exporter
changes the file extension to .test_ext
# file __init__.py
import os
import os.path
from traitlets.config import Config
from nbconvert.exporters.html import HTMLExporter
#-----------------------------------------------------------------------------
# Classes
#-----------------------------------------------------------------------------
class MyExporter(HTMLExporter):
"""
My custom exporter
"""
# If this custom exporter should add an entry to the
# "File -> Download as" menu in the notebook, give it a name here in the
# `export_from_notebook` class member
export_from_notebook = "My format"
def _file_extension_default(self):
"""
The new file extension is `.test_ext`
"""
return '.test_ext'
@property
def template_path(self):
"""
We want to inherit from HTML template, and have template under
`./templates/` so append it to the search path. (see next section)
"""
return super().template_path+[os.path.join(os.path.dirname(__file__), "templates")]
def _template_file_default(self):
"""
We want to use the new template we ship with our library.
"""
return 'test_template' # full
And the template file, that inherits from the html full template and prepend/append text to each markdown cell (see Jinja2 docs for template syntax):
{% extends "full.tpl" %}
{% block markdowncell -%}
## this is a markdown cell
{{ super() }}
## THIS IS THE END
{% endblock markdowncell %}
Assuming you install this package locally, or from PyPI, you can now use:
jupyter nbconvert --to mypackage.MyEporter notebook.ipynb
Customizing Syntax Highlighting¶
Under the hood, nbconvert uses pygments to highlight code. Both pdf and html exporting support changing the highlighting style.
Using Builtin styles¶
Pygments has a number of builtin styles available. To use them, we just need to set the style setting in the relevant preprocessor.
To change the html highlighting export with:
jupyter nbconvert --to html notebook.ipynb --CSSHTMLHeaderPreprocessor.style=<name>
To change pdf and latex highlighting export with:
jupyter nbconvert --to pdf notebook.ipynb --LatexPreprocessor.style=<name>
where <name>
is the name of the pygments style. Available styles may vary from system to system.
You can find all available styles with:
pygmentize -L styles
from a terminal or
from pygments.styles import get_all_styles
print(list(get_all_styles()))
from python.
You can preview all the styles from an environment that can display html like jupyter notebook with:
from pygments.styles import get_all_styles
from pygments.formatters import Terminal256Formatter
from pygments.lexers import PythonLexer
from pygments import highlight
code = """
import os
def function(test=1):
if test in [3,4]:
print(test)
"""
for style in get_all_styles():
highlighted_code = highlight(code, PythonLexer(), Terminal256Formatter(style=style))
print(f"{style}:\n{highlighted_code}")
Making your own styles¶
To make your own style you must subclass pygments.styles.Style
, and then you must register your new style with Pygments using
their pluggin system. This is explained in detail in the Pygments documentation.
Architecture of nbconvert¶
This is a high-level outline of the basic workflow, structures and objects in nbconvert. Specifically, this exposition has a two-fold goal:
to alert you to the affordances available for customisation or direct contributions
to provide a map of where and when different events occur, which should aid in tracking down bugs.
A detailed pipeline exploration¶
Nbconvert takes in a notebook, which is a JSON object, and operates on that object.
This can include operations that take a notebook and return a notebook.
For example, that operation could be to execute the notebook as though it were a continuous script; if it were executed --in-place
then it would overwrite the current notebook.
Or it could be that we wish to systematically alter the notebook, for example by clearing all output cells.
Format agnostic operations on cell content that do not violate the nbformat spec can be interpreted as a notebook to notebook conversion step; such operations can be performed as part of the preprocessing step.
But often we want to have the notebook’s structured content in a different format. Importantly, in many cases the structure of the notebook should be reflected in the structure of the output, adapted to the output’s format. For that purpose, the original JSON structure of the document is crucial scaffolding needed to support this kind of structured output. In order to maintain structured, it can be useful to apply our conversion programmatically on the structure itself. To do so, when converting to formats other than the notebook, we use the jinja templating engine.
The basic unit of structure in a notebook is the cell. Accordingly, since our templating engine is capable of expressing structure, the basic unit in our templates will often be specified at the cell level. Each cell has a certain type; the three most important cell types for our purposes are code, markdown, and raw NbConvert. Code cells can be split further into their input and their output. Operations can also occur separately on input and output and their respective subcomponents. Markdown cells and raw NbConvert cells do not have analogous substructure.
The template’s structure then can be seen as a mechanism for selecting content on which to operate. Because the template operates on individual cells, this has some upsides and drawbacks. One upside is that this allows the template to have access to the individual cell’s metadata, which enables intelligently transforming the appropriate content. The transformations occur as a series of replacement rules and filters. For many purposes these filters take the form of external calls to pandoc, which is a utility for converting between many different document formats. One downside is that this makes operations that require global coördination (e.g., cross referencing across cells) somewhat challenging to implement as filters inside templates.
Note that all that we’ve described is happening in memory. This is crucial in order to ensure that this functionality is available when writing files is more challenging. Nonetheless, the reason for using nbconvert almost always involves producing some kind of output file. We take the in-memory object and write a file appropriate for the output type.
The entirety of heretofore described process can be described as part of an Exporter
.
Exporter
s often involves Preprocessor
s, filters
, templates
and Writer
s.
These classes and functions are described in greater detail below.
Finally, one can apply a Postprocessor
after the writing has occurred.
For example, it is common when converting to slides to start a webserver and open a browser window with the newly created document (--to slides --post serve
).
Classes¶
Exporters¶
The primary class in nbconvert is the Exporter
.
Exporters encapsulate the operation of turning a notebook into another format.
There is one Exporter for each format supported in nbconvert.
The first thing an Exporter does is load a notebook, usually from a file via nbformat
.
Most of what a typical Exporter does is select and configure preprocessors, filters, and templates.
If you want to convert notebooks to additional formats, a new Exporter is probably what you are looking for.
See also
Once the notebook is loaded, it is preprocessed…
Preprocessors¶
A Preprocessor
is an object that transforms the content of the notebook to be exported.
The result of a preprocessor being applied to a notebook is always a notebook.
These operations include re-executing the cells, stripping output,
removing bundled outputs to separate files, etc.
If you want to add operations that modify a notebook before exporting,
a preprocessor is the place to start.
See also
Once a notebook is preprocessed, it’s time to convert the notebook into the destination format.
Templates and Filters¶
Most Exporters in nbconvert are a subclass of TemplateExporter
,
which means they use a jinja template to render a notebook into the destination format.
If you want to change how an exported notebook looks in an existing format,
a custom template is the place to start.
A jinja template is composed of blocks that look like this (taken from nbconvert’s default html template):
{% block stream_stdout -%}
<div class="output_subarea output_stream output_stdout output_text">
<pre>
{{- output.text | ansi2html -}}
</pre>
</div>
{%- endblock stream_stdout %}
This block determines how text output on stdout
is displayed in HTML.
The {{- output.text | ansi2html -}}
bit means
“Take the output text and pass it through ansi2html, then include the result here.”
In this example, ansi2html
is a filter.
Filters are a jinja concept; they are Python callables which take something (typically text) as an input, and produce a text output.
If you want to perform new or more complex transformations of particular outputs,
a filter may be what you need.
Typically, filters are pure functions.
However, if you have a filter that itself requires some configuration,
it can be an instance of a callable, configurable class.
See also
Once it has passed through the template, an Exporter is done with the notebook, and returns the file data. At this point, we have the file data as text or bytes and we can decide where it should end up. When you are using nbconvert as a library, as opposed to the command-line application, this is typically where you would stop, take your exported data, and go on your way.
Writers¶
A Writer
takes care of writing the resulting file(s) where they should end up.
There are two basic Writers in nbconvert:
stdout - writes the result to stdout (for pipe-style workflows)
Files (default) - writes the result to the filesystem
Once the output is written, nbconvert has done its job.
Postprocessors¶
A Postprocessor
is something that runs after everything is exported and written to the filesystem.
The only postprocessor in nbconvert at this point is the ServePostProcessor
,
which is used for serving reveal.js HTML slideshows.
Python API for working with nbconvert¶
Contents:
NbConvertApp¶
See also
- Configuration options
Configurable options for the nbconvert application
-
class
nbconvert.nbconvertapp.
NbConvertApp
(**kwargs)¶ Application used to convert from notebook file type (
*.ipynb
)-
init_notebooks
()¶ Construct the list of notebooks.
If notebooks are passed on the command-line, they override (rather than add) notebooks specified in config files. Glob each notebook to replace notebook patterns with filenames.
-
convert_notebooks
()¶ Convert the notebooks in the self.notebook traitlet
-
convert_single_notebook
(notebook_filename, input_buffer=None)¶ Convert a single notebook.
Performs the following steps:
Initialize notebook resources
Export the notebook to a particular format
Write the exported notebook to file
(Maybe) postprocess the written file
- Parameters
notebook_filename (str) –
input_buffer – If input_buffer is not None, conversion is done and the buffer is used as source into a file basenamed by the notebook_filename argument.
-
init_single_notebook_resources
(notebook_filename)¶ Step 1: Initialize resources
This initializes the resources dictionary for a single notebook.
- Returns
- resources dictionary for a single notebook that MUST include the following keys:
config_dir: the location of the Jupyter config directory
unique_key: the notebook name
output_files_dir: a directory where output files (not including the notebook itself) should be saved
- Return type
-
export_single_notebook
(notebook_filename, resources, input_buffer=None)¶ Step 2: Export the notebook
Exports the notebook to a particular format according to the specified exporter. This function returns the output and (possibly modified) resources from the exporter.
-
write_single_notebook
(output, resources)¶ Step 3: Write the notebook to file
This writes output from the exporter to file using the specified writer. It returns the results from the writer.
- Parameters
output –
resources (dict) – resources for a single notebook including name, config directory and directory to save output
- Returns
results from the specified writer output of exporter
- Return type
file
-
postprocess_single_notebook
(write_results)¶ Step 4: Post-process the written file
Only used if a postprocessor has been specified. After the converted notebook is written to a file in Step 3, this post-processes the notebook.
-
Exporters¶
See also
- Configuration options
Configurable options for the nbconvert application
-
nbconvert.exporters.
export
(exporter, nb, **kw)¶ Export a notebook object using specific exporter class.
- Parameters
exporter (
Exporter
class or instance) – Class or instance of the exporter that should be used. If the method initializes its own instance of the class, it is ASSUMED that the class type provided exposes a constructor (__init__
) with the same signature as the base Exporter class.nb (
NotebookNode
) – The notebook to export.config (config (optional, keyword arg)) – User configuration instance.
resources (dict (optional, keyword arg)) – Resources used in the conversion process.
- Returns
- outputstr
The resulting converted notebook.
- resourcesdictionary
Dictionary of resources used prior to and during the conversion process.
- Return type
-
nbconvert.exporters.
get_exporter
(name)¶ Given an exporter name or import path, return a class ready to be instantiated
Raises ValueError if exporter is not found
-
nbconvert.exporters.
get_export_names
()¶ Return a list of the currently supported export targets
Exporters can be found in external packages by registering them as an nbconvert.exporter entrypoint.
Exporter base classes¶
-
class
nbconvert.exporters.
Exporter
(config=None, **kw)¶ Class containing methods that sequentially run a list of preprocessors on a NotebookNode object and then return the modified NotebookNode object and accompanying resources dict.
-
__init__
(config=None, **kw)¶ Public constructor
- Parameters
config (
Config
) – User configuration instance.**kw – Additional keyword arguments passed to parent __init__
-
from_notebook_node
(nb, resources=None, **kw)¶ Convert a notebook from a notebook node instance.
- Parameters
nb (
NotebookNode
) – Notebook node (dict-like with attr-access)resources (dict) – Additional resources that can be accessed read/write by preprocessors and filters.
**kw – Ignored
-
from_filename
(filename, resources=None, **kw)¶ Convert a notebook from a notebook file.
-
from_file
(file_stream, resources=None, **kw)¶ Convert a notebook from a notebook file.
- Parameters
file_stream (file-like object) – Notebook file-like object to convert.
resources (dict) – Additional resources that can be accessed read/write by preprocessors and filters.
**kw – Ignored
-
register_preprocessor
(preprocessor, enabled=False)¶ Register a preprocessor. Preprocessors are classes that act upon the notebook before it is passed into the Jinja templating engine. preprocessors are also capable of passing additional information to the Jinja templating engine.
- Parameters
preprocessor (
Preprocessor
) – A dotted module name, a type, or an instanceenabled (bool) – Mark the preprocessor as enabled
-
-
class
nbconvert.exporters.
TemplateExporter
(config=None, **kw)¶ Exports notebooks into other file formats. Uses Jinja 2 templating engine to output new formats. Inherit from this class if you are creating a new template type along with new filters/preprocessors. If the filters/ preprocessors provided by default suffice, there is no need to inherit from this class. Instead, override the template_file and file_extension traits via a config file.
Filters available by default for templates:
add_anchor
add_prompts
ansi2html
ansi2latex
ascii_only
citation2latex
comment_lines
convert_pandoc
escape_latex
filter_data_type
get_lines
get_metadata
highlight2html
highlight2latex
html2text
indent
ipython2python
json_dumps
markdown2asciidoc
markdown2html
markdown2latex
markdown2rst
path2url
posix_path
prevent_list_blocks
strip_ansi
strip_dollars
strip_files_prefix
wrap_text
-
__init__
(config=None, **kw)¶ Public constructor
-
from_notebook_node
(nb, resources=None, **kw)¶ Convert a notebook from a notebook node instance.
- Parameters
nb (
NotebookNode
) – Notebook noderesources (dict) – Additional resources that can be accessed read/write by preprocessors and filters.
-
from_filename
(filename, resources=None, **kw)¶ Convert a notebook from a notebook file.
-
from_file
(file_stream, resources=None, **kw)¶ Convert a notebook from a notebook file.
- Parameters
file_stream (file-like object) – Notebook file-like object to convert.
resources (dict) – Additional resources that can be accessed read/write by preprocessors and filters.
**kw – Ignored
-
register_preprocessor
(preprocessor, enabled=False)¶ Register a preprocessor. Preprocessors are classes that act upon the notebook before it is passed into the Jinja templating engine. preprocessors are also capable of passing additional information to the Jinja templating engine.
- Parameters
preprocessor (
Preprocessor
) – A dotted module name, a type, or an instanceenabled (bool) – Mark the preprocessor as enabled
Specialized exporter classes¶
The NotebookExporter
inherits directly from
Exporter
, while the other exporters listed here
inherit either directly or indirectly from
TemplateExporter
.
-
class
nbconvert.exporters.
NotebookExporter
(config=None, **kw)¶ Exports to an IPython notebook.
This is useful when you want to use nbconvert’s preprocessors to operate on a notebook (e.g. to execute it) and then write it back to a notebook file.
-
class
nbconvert.exporters.
HTMLExporter
(config=None, **kw)¶ Exports a basic HTML document. This exporter assists with the export of HTML. Inherit from it if you are writing your own HTML template and need custom preprocessors/filters. If you don’t need custom preprocessors/ filters, just change the ‘template_file’ config option.
-
class
nbconvert.exporters.
SlidesExporter
(config=None, **kw)¶ Exports HTML slides with reveal.js
-
class
nbconvert.exporters.
LatexExporter
(config=None, **kw)¶ Exports to a Latex template. Inherit from this class if your template is LaTeX based and you need custom tranformers/filters. Inherit from it if you are writing your own HTML template and need custom tranformers/filters. If you don’t need custom tranformers/filters, just change the ‘template_file’ config option. Place your template in the special “/latex” subfolder of the “../templates” folder.
-
class
nbconvert.exporters.
MarkdownExporter
(config=None, **kw)¶ Exports to a markdown document (.md)
-
class
nbconvert.exporters.
PDFExporter
(config=None, **kw)¶ Writer designed to write to PDF files.
This inherits from
LatexExporter
. It creates a LaTeX file in a temporary directory using the template machinery, and then runs LaTeX to create a pdf.
-
class
nbconvert.exporters.
PythonExporter
(config=None, **kw)¶ Exports a Python code file.
-
class
nbconvert.exporters.
RSTExporter
(config=None, **kw)¶ Exports reStructuredText documents.
Preprocessors¶
See also
- Configuration options
Configurable options for the nbconvert application
-
class
nbconvert.preprocessors.
Preprocessor
(**kw)¶ A configurable preprocessor
Inherit from this class if you wish to have configurability for your preprocessor.
Any configurable traitlets this class exposed will be configurable in profiles using c.SubClassName.attribute = value
you can overwrite
preprocess_cell()
to apply a transformation independently on each cell orpreprocess()
if you prefer your own logic. See corresponding docstring for information.- Disabled by default and can be enabled via the config by
‘c.YourPreprocessorName.enabled = True’
-
__init__
(**kw)¶ Public constructor
- Parameters
config (Config) – Configuration file structure
**kw – Additional keyword arguments passed to parent
-
preprocess
(nb, resources)¶ Preprocessing to apply on each notebook.
Must return modified nb, resources.
If you wish to apply your preprocessing to each cell, you might want to override preprocess_cell method instead.
- Parameters
nb (NotebookNode) – Notebook being converted
resources (dictionary) – Additional resources used in the conversion process. Allows preprocessors to pass variables into the Jinja engine.
-
preprocess_cell
(cell, resources, index)¶ Override if you want to apply some preprocessing to each cell. Must return modified cell and resource dictionary.
- Parameters
cell (NotebookNode cell) – Notebook cell being processed
resources (dictionary) – Additional resources used in the conversion process. Allows preprocessors to pass variables into the Jinja engine.
index (int) – Index of the cell being processed
Specialized preprocessors¶
-
class
nbconvert.preprocessors.
ConvertFiguresPreprocessor
(**kw)¶ Converts all of the outputs in a notebook from one format to another.
-
class
nbconvert.preprocessors.
SVG2PDFPreprocessor
(**kw)¶ Converts all of the outputs in a notebook from SVG to PDF.
-
class
nbconvert.preprocessors.
ExtractOutputPreprocessor
(**kw)¶ Extracts all of the outputs from the notebook file. The extracted outputs are returned in the ‘resources’ dictionary.
-
class
nbconvert.preprocessors.
LatexPreprocessor
(**kw)¶ Preprocessor for latex destined documents.
Mainly populates the latex key in the resources dict, adding definitions for pygments highlight styles.
-
class
nbconvert.preprocessors.
CSSHTMLHeaderPreprocessor
(*pargs, **kwargs)¶ Preprocessor used to pre-process notebook for HTML output. Adds IPython notebook front-end CSS and Pygments CSS to HTML output.
-
class
nbconvert.preprocessors.
HighlightMagicsPreprocessor
(config=None, **kw)¶ Detects and tags code cells that use a different languages than Python.
-
class
nbconvert.preprocessors.
ClearOutputPreprocessor
(**kw)¶ Removes the output from all code cells in a notebook.
-
class
nbconvert.preprocessors.
RegexRemovePreprocessor
(**kw)¶ Removes cells from a notebook that match one or more regular expression.
For each cell, the preprocessor checks whether its contents match the regular expressions in the patterns traitlet which is a list of unicode strings. If the contents match any of the patterns, the cell is removed from the notebook.
To modify the list of matched patterns, modify the patterns traitlet. For example, execute the following command to convert a notebook to html and remove cells containing only whitespace:
jupyter nbconvert --RegexRemovePreprocessor.patterns="['\s*\Z']" mynotebook.ipynb
The command line argument sets the list of patterns to
'\s*\Z'
which matches an arbitrary number of whitespace characters followed by the end of the string.See https://regex101.com/ for an interactive guide to regular expressions (make sure to select the python flavor). See https://docs.python.org/library/re.html for the official regular expression documentation in python.
-
class
nbconvert.preprocessors.
ExecutePreprocessor
(**kw)¶ Executes all the cells in a notebook
-
preprocess
(nb, resources, km=None)¶ Preprocess notebook executing each code cell.
The input argument nb is modified in-place.
- Parameters
nb (NotebookNode) – Notebook being executed.
resources (dictionary) – Additional resources used in the conversion process. For example, passing
{'metadata': {'path': run_path}}
sets the execution path torun_path
.km (KernelManager (optional)) – Optional kernel manager. If none is provided, a kernel manager will be created.
- Returns
nb (NotebookNode) – The executed notebook.
resources (dictionary) – Additional resources used in the conversion process.
-
preprocess_cell
(cell, resources, cell_index)¶ Executes a single code cell. See base.py for details.
To execute all cells see
preprocess()
.
-
process_message
(msg, cell, cell_index)¶ Processes a kernel message, updates cell state, and returns the resulting output object that was appended to cell.outputs.
The input argument cell is modified in-place.
- Parameters
- Returns
output – The execution output payload (or None for no output).
- Return type
- Raises
CellExecutionComplete – Once a message arrives which indicates computation completeness.
-
setup_preprocessor
(nb, resources, km=None)¶ Context manager for setting up the class to execute a notebook.
The assigns nb to self.nb where it will be modified in-place. It also creates and assigns the Kernel Manager (self.km) and Kernel Client(self.kc).
It is intended to yield to a block that will execute codeself.
When control returns from the yield it stops the client’s zmq channels, shuts down the kernel, and removes the now unused attributes.
- Parameters
nb (NotebookNode) – Notebook being executed.
resources (dictionary) – Additional resources used in the conversion process. For example, passing
{'metadata': {'path': run_path}}
sets the execution path torun_path
.km (KernerlManager (optional)) – Optional kernel manaher. If none is provided, a kernel manager will be created.
- Returns
nb (NotebookNode) – The executed notebook.
resources (dictionary) – Additional resources used in the conversion process.
-
start_new_kernel
(**kwargs)¶ Creates a new kernel manager and kernel client.
- Parameters
kwargs – Any options for self.kernel_manager_class.start_kernel(). Because that defaults to KernelManager, this will likely include options accepted by KernelManager.start_kernel()`, which includes cwd.
- Returns
km (KernelManager) – A kernel manager as created by self.kernel_manager_class.
kc (KernelClient) – Kernel client as created by the kernel manager km.
-
-
nbconvert.preprocessors.
coalesce_streams
(cell, resources, index)¶ Merge consecutive sequences of stream output into single stream to prevent extra newlines inserted at flush calls
- Parameters
cell (NotebookNode cell) – Notebook cell being processed
resources (dictionary) – Additional resources used in the conversion process. Allows transformers to pass variables into the Jinja engine.
index (int) – Index of the cell being processed
Filters¶
Filters are for use with the TemplateExporter
exporter.
They provide a way for you transform notebook contents to a particular format depending
on the template you are using. For example, when converting to HTML, you would want to
use the ansi2html()
function to convert ANSI colors (from
e.g. a terminal traceback) to HTML colors.
See also
- Exporters
API documentation for the various exporter classes
-
nbconvert.filters.
add_anchor
(html, anchor_link_text='¶')¶ Add an id and an anchor-link to an html header
For use on markdown headings
-
nbconvert.filters.
add_prompts
(code, first='>>> ', cont='... ')¶ Add prompts to code snippets
-
nbconvert.filters.
ansi2html
(text)¶ Convert ANSI colors to HTML colors.
- Parameters
text (unicode) – Text containing ANSI colors to convert to HTML
-
nbconvert.filters.
ansi2latex
(text)¶ Convert ANSI colors to LaTeX colors.
- Parameters
text (unicode) – Text containing ANSI colors to convert to LaTeX
-
nbconvert.filters.
ascii_only
(s)¶ ensure a string is ascii
-
nbconvert.filters.
citation2latex
(s)¶ Parse citations in Markdown cells.
This looks for HTML tags having a data attribute names data-cite and replaces it by the call to LaTeX cite command. The tranformation looks like this:
<cite data-cite=”granger”>(Granger, 2013)</cite>
Becomes
cite{granger}
Any HTML tag can be used, which allows the citations to be formatted in HTML in any manner.
-
nbconvert.filters.
comment_lines
(text, prefix='# ')¶ Build a Python comment line from input text.
-
nbconvert.filters.
escape_latex
(text)¶ Escape characters that may conflict with latex.
- Parameters
text (str) – Text containing characters that may conflict with Latex
-
class
nbconvert.filters.
DataTypeFilter
(**kw)¶ Returns the preferred display format
-
nbconvert.filters.
get_lines
(text, start=None, end=None)¶ Split the input text into separate lines and then return the lines that the caller is interested in.
-
nbconvert.filters.
convert_pandoc
(source, from_format, to_format, extra_args=None)¶ Convert between any two formats using pandoc.
This function will raise an error if pandoc is not installed. Any error messages generated by pandoc are printed to stderr.
- Parameters
source (string) – Input string, assumed to be valid in from_format.
from_format (string) – Pandoc format of source.
to_format (string) – Pandoc format for output.
- Returns
out – Output as returned by pandoc.
- Return type
string
-
class
nbconvert.filters.
Highlight2HTML
(pygments_lexer=None, **kwargs)¶
-
class
nbconvert.filters.
Highlight2Latex
(pygments_lexer=None, **kwargs)¶
-
nbconvert.filters.
html2text
(element)¶ extract inner text from html
Analog of jQuery’s $(element).text()
-
nbconvert.filters.
indent
(instr, nspaces=4, ntabs=0, flatten=False)¶ Indent a string a given number of spaces or tabstops.
indent(str,nspaces=4,ntabs=0) -> indent str by ntabs+nspaces.
- Parameters
instr (basestring) – The string to be indented.
nspaces (int (default: 4)) – The number of spaces to be indented.
ntabs (int (default: 0)) – The number of tabs to be indented.
flatten (bool (default: False)) – Whether to scrub existing indentation. If True, all lines will be aligned to the same indentation. If False, existing indentation will be strictly increased.
- Returns
str|unicode
- Return type
string indented by ntabs and nspaces.
-
nbconvert.filters.
ipython2python
(code)¶ Transform IPython syntax to pure Python syntax
- Parameters
code (str) – IPython code, to be transformed to pure Python
-
nbconvert.filters.
markdown2html
(source)¶ Convert a markdown string to HTML using mistune
-
nbconvert.filters.
markdown2latex
(source, markup='markdown', extra_args=None)¶ Convert a markdown string to LaTeX via pandoc.
This function will raise an error if pandoc is not installed. Any error messages generated by pandoc are printed to stderr.
- Parameters
source (string) – Input string, assumed to be valid markdown.
markup (string) – Markup used by pandoc’s reader default : pandoc extended markdown (see http://pandoc.org/README.html#pandocs-markdown)
- Returns
out – Output as returned by pandoc.
- Return type
string
-
nbconvert.filters.
markdown2rst
(source, extra_args=None)¶ Convert a markdown string to ReST via pandoc.
This function will raise an error if pandoc is not installed. Any error messages generated by pandoc are printed to stderr.
- Parameters
source (string) – Input string, assumed to be valid markdown.
- Returns
out – Output as returned by pandoc.
- Return type
string
-
nbconvert.filters.
path2url
(path)¶ Turn a file path into a URL
-
nbconvert.filters.
posix_path
(path)¶ Turn a path into posix-style path/to/etc
Mainly for use in latex on Windows, where native Windows paths are not allowed.
-
nbconvert.filters.
prevent_list_blocks
(s)¶ Prevent presence of enumerate or itemize blocks in latex headings cells
-
nbconvert.filters.
strip_ansi
(source)¶ Remove ANSI escape codes from text.
- Parameters
source (str) – Source to remove the ANSI from
-
nbconvert.filters.
strip_dollars
(text)¶ Remove all dollar symbols from text
- Parameters
text (str) – Text to remove dollars from
Writers¶
See also
- Configuration options
Configurable options for the nbconvert application
-
class
nbconvert.writers.
WriterBase
(config=None, **kw)¶ Consumes output from nbconvert export…() methods and writes to a useful location.
-
__init__
(config=None, **kw)¶ Constructor
-
write
(output, resources, **kw)¶ Consume and write Jinja output.
- Parameters
output (string) – Conversion results. This string contains the file contents of the converted file.
resources (dict) – Resources created and filled by the nbconvert conversion process. Includes output from preprocessors, such as the extract figure preprocessor.
-
Specialized writers¶
-
class
nbconvert.writers.
DebugWriter
(config=None, **kw)¶ Consumes output from nbconvert export…() methods and writes useful debugging information to the stdout. The information includes a list of resources that were extracted from the notebook(s) during export.
-
class
nbconvert.writers.
FilesWriter
(**kw)¶ Consumes nbconvert output and produces files.
-
class
nbconvert.writers.
StdoutWriter
(config=None, **kw)¶ Consumes output from nbconvert export…() methods and writes to the stdout stream.
Postprocessors¶
See also
- Configuration options
Configurable options for the nbconvert application
Making an nbconvert
release¶
This document guides a contributor through creating a release of nbconvert
.
Assign all merged PRs to milestones¶
Go to GitHub and assign all PRs that have been merged to milestones. This will be helpful when you update the changelog. If you go to this GitHub page you will find all the PRs that currently have no milestones.
Manually categorize tickets¶
Group the tickets by these general categories (or others if they are relevant). This usually a manual processes to evaluate the changes in each PR.
New Features
Deprecations
Fixing Problems
Testing, Docs, and Builds
Collect major changes¶
From the tickets write up any major features / changes that deserve a paragraph to describe how they work.
Update docs/source/changelog.rst¶
Copy these changes with the new version to the top of changelog.rst. Prior release changelogs can be used to pick formatting of the message.
Check installed tools¶
Review CONTRIBUTING.md
, particularly the testing and release sections.
Clean the repository¶
You can remove all non-tracked files with:
git clean -xfdi
This would ask you for confirmation before removing all untracked files.
Make sure the dist/
folder is clean and avoid stale builds from
previous attempts.
Create the release¶
Update the changelog to account for all the PRs assigned to this milestone.
Update version number in
notebook/_version.py
and remove.dev
from dev_info.Commit and tag the release with the current version number:
git commit -am "release $VERSION" git tag $VERSION
You are now ready to build the
sdist
andwheel
:python setup.py sdist python setup.py bdist_wheel
You can now test the
wheel
and thesdist
locally before uploading to PyPI. Make sure to use twine to upload the archives over SSL.twine upload dist/*
The conda-forge bot will automatically add a PR on your behalf to the nbconvert-feedstock repo. You may want to review this PR to ensure conda-forge will be updated cleanly.
Release the new version¶
Push directly on master, including –tags separately
git push upstream git push upstream --tags
Return to development state¶
- If all went well, change the
notebook/_version.py
back adding the .dev
suffix.
Email googlegroup with update letter¶
Make sure to email jupyter@googlegroups.com with the subject line of “[ANN] NBConvert $VERSION – …” and include at least the significant changes, contributors, and individual PR notes (if not many significant changes).
Changes in nbconvert¶
5.5¶
The following 18 authors contributed 144 commits – Thank you all!
Benjamin Ragan-Kelley
Clayton A Davis
DInne Bosman
Doug Blank
Henrique Silva
Jeff Hale
Lukasz Mitusinski
M Pacer
Maarten Breddels
Madhumitha N
Matthew Seal
Paul Gowder
Philipp A
Rick Lupton
Rüdiger Busche
Thomas Kluyver
Tyler Makaro
WrRan
The full list of changes they made can be seen on GitHub
Significant Changes¶
Deprecations¶
Python 3.4 support was dropped. Many of our upstream libraries stopped supporting 3.4 and it was found that serious bugs were being caught during testing against those libraries updating past 3.4.
See PR #979 for details.
IPyWidget Support¶
Now when a notebook executing contains Jupyter Widgets, the state of all the widgets can be stored in the notebook’s metadata. This allows rendering of the live widgets on, for instance nbviewer, or when converting to html.
You can tell nbconvert to not store the state using the store_widget_state argument:
jupyter nbconvert --ExecutePreprocessor.store_widget_state=False --to notebook --execute mynotebook.ipynb
This widget rendering is not performed against a browser during execution, so only widget default states or states manipulated via user code will be calculated during execution. %%javascript cells will execute upon notebook rendering, enabling complex interactions to function as expected when viewed by a UI.
If you can’t view widget results after execution, you may need to select Trust Notebook under the File menu of the UI in question.
Execute Preprocessor Rework¶
Based on monkey patching required in papermill the run_cell code path in the ExecutePreprocessor was reworked to allow for accessing individual message parses without reimplementing the entire function. Now there is a processs_message function which take a ZeroMQ message and applies all of its side-effect updates on the cell/notebook objects before returning the output it generated, if it generated any such output.
The change required a much more extensive test suite covering cell execution as test coverage on the various, sometimes wonky, code paths made improvements and reworks impossible to prove undamaging. Now changes to kernel message processing has much better coverage, so future additions or changes with specs over time will be easier to add.
Out Of Memory Kernel Failure Catches¶
When running out of memory on a machine, if the kernel process was killed by the operating system it would result in a timeout error at best and hang indefinitely at worst. Now regardless of timeout configuration, if the underlying kernel process dies before emitting any messages to the effect an exception will be raised notifying the consumer of the lost kernel within a few seconds.
Latex / PDF Template Improvements¶
The latex template was long overdue for improvements. The default template had a rewrite which makes exports for latex and pdf look a lot better. Code cells in particular render much better with line breaks and styling the more closely matches notebook browser rendering. Thanks t-makaro for the efforts here!
See PR #992 for details
Comprehensive notes¶
New Features¶
Fixing Problems¶
Refactored execute preprocessor to have a process_message function PR #905:
Fixed latex export for svg data in python 3 PR #985:
Enabled configuration to be shared to exporters from script exporter PR #993:
Make latex errors less verbose PR #988:
Typo in template syntax PR #984:
Improved attachments +fix supporting non-unique names PR #980:
PDFExporter “output_mimetype” traitlet is not longer ‘text/latex’ PR #972:
FIX: respect wait for clear_output PR #969:
address deprecation warning in cgi.escape PR #963:
Correct inaccurate description of available LaTeX template PR #958:
Fixed kernel death detection for executions with timeouts PR #998:
Fixed export names for various templates PR #1000, PR #1001, and PR #1001:
Testing, Docs, and Builds¶
Added tests for each branch in execute’s run_cell method PR #982:
Mention formats in –to options more clearly PR #991:
Adds ascii output type to command line docs page, mention image folder output PR #956:
Simplify setup.py PR #949:
Use utf-8 encoding in execute_api example PR #921:
Upgrade pytest on Travis PR #941:
Fix LaTeX base template name in docs PR #940:
Updated release instructions based on 5.4 release walk-through PR #887:
Fixed broken link to jinja docs PR #997:
5.4.1¶
Thanks to the following 11 authors who contributed 57 commits.
Benjamin Ragan-Kelley
Carol Willing
Clayton A Davis
Daniel Rodriguez
M Pacer
Matthew Seal
Matthias Geier
Matthieu Parizy
Rüdiger Busche
Thomas Kluyver
Tyler Makaro
5.4¶
Significant Changes¶
Deprecations¶
Python 3.3 support was dropped. The version of python is no longer common and new versions have many fixes and interface improvements that warrant the change in support.
See PR #843 for implementation details.
Changes in how we handle metadata¶
There were a few new metadata fields which are now respected in nbconvert.
nb.metadata.authors
metadata attribute will be respected in latex exports. Multiple authors will be added with ,
separation against their names.
nb.metadata.title
will be respected ahead of nb.metadata.name
for title assignment. This better matches with the notebook format.
nb.metadata.filename
will override the default output_filename_template
when extracting notebook resources in the ExtractOutputPreprocessor
. The attribute is helpful for when you want to consistently fix to a particular output filename, especially when you need to set image filenames for your exports.
The raises-exception
cell tag (nb.cells[].metadata.tags[raises-exception]
) allows for cell exceptions to not halt execution. The tag is respected in the same way by nbval and other notebook interfaces. nb.metadata.allow_errors
will apply this rule for all cells. This feature is toggleable with the force_raise_errors
configuration option.
Errors from executing the notebook can be allowed with a raises-exception
tag on a single cell, or the allow_errors
configurable option for all cells. An allowed error will be recorded in notebook output, and execution will continue.
If an error occurs when it is not explicitly allowed, a CellExecutionError
will be raised.
If force_raise_errors
is True, CellExecutionError
will be raised for any error that occurs while executing the notebook. This overrides both the allow_errors
option and the raises-exception
cell tags.
See PR #867, PR #703, PR #685, PR #672, and PR #684 for implementation changes.
Configurable kernel managers when executing notebooks¶
The kernel manager can now be optionally passed into the ExecutePreprocessor.preprocess
and the executenb
functions as the keyword argument km
. This means that the kernel can be configured as desired before beginning preprocessing.
This is useful for executing in a context where the kernel has external dependencies that need to be set to non-default values. An example of this might be a Spark kernel where you wish to configure the Spark cluster location ahead of time without building a new kernel.
Overall the ExecutePreprocessor has been reworked to make it easier to use. Future releases will continue this trend to make this section of the code more inheritable and reusable by others. We encourage you read the source code for this version if you’re interested in the detailed improvements.
See PR #852 for implementation changes.
Surfacing exporters in front-ends¶
Exporters are now exposed for front-ends to consume, including classic notebook. As an example, this means that latex exporter will be made available for latex ‘text/latex’ media type from the Download As interface.
Raw Templates¶
Template exporters can now be assigned raw templates as string attributes by setting the raw_template
variable.
class AttrExporter(TemplateExporter):
# If the class has a special template and you want it defined within the class
raw_template = """{%- extends 'rst.tpl' -%}
{%- block in_prompt -%}
raw template
{%- endblock in_prompt -%}
"""
exporter_attr = AttrExporter()
output_attr, _ = exporter_attr.from_notebook_node(nb)
assert "raw template" in output_attr
See PR #675 for implementation changes.
New command line flags¶
The --no-input
will hide input cells on export. This is great for notebooks which generate “reports” where you want the code that was executed to not appear by default in the extracts.
An alias for notebook
was added to exporter commands. Now --to ipynb
will behave as --to notebook
does.
Comprehensive notes¶
New Features¶
No input flag (
--no-input
) PR #825Add alias
--to ipynb
for notebook exporter PR #873Add
export_from_notebook
PR #864If set, use
nb.metadata.authors
for LaTeX author line PR #867Populate language_info metadata when executing PR #860
Support for
\mathscr
PR #830Allow the execute preprocessor to make use of an existing kernel PR #852
Refactor ExecutePreprocessor PR #816
Update widgets CDN for ipywidgets 7 w/fallback PR #792
Add support for adding custom exporters to the “Download as” menu. PR #759
Enable ANSI underline and inverse PR #696
Update notebook css to 5.4.0 PR #748
Change default for slides to direct to the reveal cdn rather than locally PR #732
Use “title” instead of “name” for metadata to match the notebook format PR #703
Img filename metadata PR #685
Added MathJax compatibility definitions PR #687
Per cell exception PR #684
Set BIBINPUTS and BSTINPUTS environment variables when making PDF PR #676
If
nb.metadata.title
is set, default to that for notebook PR #672
Fixing Problems¶
Fix api break PR #872
Don’t remove empty cells by default PR #784
Handle attached images in html converter PR #780
No need to check for the channels already running PR #862
Update
font-awesome
version for slides PR #793Properly treat JSON data PR #847
Skip executing empty code cells PR #739
Ppdate log.warn (deprecated) to log.warning PR #804
Cleanup notebook.tex during PDF generation PR #768
Windows unicode error fixed, nosetest added to setup.py PR #757
Better content hiding; template & testing improvements PR #734
Fix Jinja syntax in custom template example. PR #738
Fix for an issue with empty math block PR #729
Add parser for Multiline math for LaTeX blocks PR #716 PR #717
Use defusedxml to parse potentially untrusted XML PR #708
Fixes for traitlets 4.1 deprecation warnings PR #695
Testing, Docs, and Builds¶
A couple of typos PR #870
Add python_requires metadata. PR #871
Document
--inplace
command line flag. PR #839Fix minor typo in
usage.rst
PR #863Add note about local
reveal_url_prefix
PR #844Move
onlyif_cmds_exist
decorator to test-specific utils PR #854Include LICENSE file in wheels PR #827
Added Ubuntu Linux Instructions PR #724
Removing more nose remnants via dependencies. PR #758
Remove offline statement and add some clarifications in slides docs PR #743
Linkify PR number PR #710
Added shebang for python PR #694
Upgrade mistune dependency PR #705
add feature to improve docs by having links to prs PR #662
Update notebook CSS from version 4.3.0 to 5.1.0 PR #682
Explicitly exclude or include all files in Manifest. PR #670
5.3¶
Major features¶
Tag Based Element Filtering¶
For removing individual elements from notebooks, we need a way to signal to nbconvert that the elements should be removed. With this release, we introduce the use of tags for that purpose.
Tags are user-defined strings attached to cells or outputs. They are stored in cell or output metadata. For more on tags see the nbformat docs on cell metadata.
Usage:
Apply tags to the elements that you want to remove.
For removing an entire cell, the cell input, or all cell outputs apply the tag to the cell.
For removing individual outputs, put the tag in the output metadata
using a call like display(your_output_element, metadata={tags=[<your_tags_here>]})
.
NB: Use different tags depending on whether you want to remove the entire cell, the input, all outputs, or individual outputs.
Add the tags for removing the different kinds of elements to the following traitlets. Which kind of element you want to remove determines which traitlet you add the tags to.
The following traitlets remove elements of different kinds:
remove_cell_tags
: removes cellsremove_input_tags
: removes inputsremove_all_outputs_tag
: removes all outputsremove_single_output_tag
: removes individual outputs
Comprehensive notes¶
new: configurable
browser
in ServePostProcessor PR #618new:
--clear-output
command line flag to clear output in-place PR #619new: remove elements based on tags with
TagRemovePreprocessor
. PR #640, PR #643new: CellExecutionError can now be imported from
nbconvert.preprocessors
PR #656new: slides now can enable scrolling and custom transitions PR #600
docs: Release instructions for nbviewer-deploy
docs: improved instructions for handling errors using the
ExecutePreprocessor
PR #656tests: better height/width metadata testing for images in rst & html PR #601 PR #602
tests: normalise base64 output data to avoid false positives PR #650
tests: normalise ipython traceback messages to handle old and new style PR #631
bug: mathjax obeys
\\(\\)
&\\[\\]
(both nbconvert & pandoc) PR #609 PR #617bug: specify default templates using extensions PR #639
bug: fix pandoc version number PR #638
bug: require recent mistune version PR #630
bug: catch errors from IPython
execute_reply
anderror
messages PR #642nose completely removed & dependency dropped PR #595 PR #660
mathjax processing in mistune now only uses inline grammar PR #611
removeRegex now enabled by default on all TemplateExporters, does not remove cells with outputs PR #616
validate notebook after applying each preprocessor (allowing additional attributes) PR #645
changed COPYING.md to LICENSE for more standard licensing that GitHub knows how to read PR #654
5.2.1¶
Major features¶
In this release (along with the usual bugfixes and documentation improvements, which are legion) we have a few new major features that have been requested for a long time:
Global Content Filtering¶
You now have the ability to remove input or output from code cells, markdown cells and the input and output prompts. The easiest way to access all of these is by using traitlets like TemplateExporter.exclude_input = True (or, for example HTMLExporter.exclude_markdown = True if you wanted to make it specific to HTML output). On the command line if you just want to not have input or output prompts just use –no-prompt.
Execute notebooks from a function¶
You can now use the executenb function to execute notebooks as though you ran the execute preprocessor on the notebooks. It returns the standard notebook and resources options.
Remove cells based on regex pattern¶
This removes cells based on their matching a regex pattern (by default, empty cells). This is the RegexRemovePreprocessor.
Script exporter entrypoints for nonpython scripts¶
Now there is an entrypoint for having an exporter specific to the type of script that is being exported. While designed for use with the IRkernel in particular (with a script exporter focused on exporting R scripts) other non-python kernels that wish to have a language specific exporter can now surface that directly.
Comprehensive notes¶
new: configurable ExecutePreprocessor.startup_timeout configurable PR #583
new: RemoveCell preprocessor based on cell content (defaults to empty cell) PR #575
new: function for executing notebooks: executenb PR #573
new: global filtering to remove inputs, outputs, markdown cells (&c.), this works on all templates PR #554
new: script exporter entrypoint PR #531
new: configurable anchor link text (previously ¶) HTMLExporter.anchor_link_text PR #522
new: configurable values for slides exporter PR #542 PR #558
improved releases (how-to documentation, version-number generation and checking) PR #593
language information from cell magics (for highlighting) is now included in more formats PR #586
better traceback behaviour when execution errs PR #521
deprecated nose test features removed PR #519
bug fixed: we now respect width and height metadata on jpeg and png mimetype outputs PR #588
bug fixed: now we respect the resolve_references filter in report.tplx PR #577
bug fixed: output metadata now is removed by ClearOutputPreprocessor PR #569
bug fixed: display id respected in execute preproessor PR #563
bug fixed: dynamic defaults for optional jupyter_client import PR #559
bug fixed: don’t self-close non-void HTML tags PR #548
buf fixed: upgrade jupyter_client dependency to 4.2 PR #539
bug fixed: LaTeX output through md→LaTeX conversion shouldn’t be touched PR #535
bug fixed: now we escape < inside math formulas when converting to html PR #514
Credits¶
This release has been larger than previous releases. In it 33 authors contributed a total of 546 commits.
Many thanks to the following individuals who contributed to this release (in alphabetical order):
Adam Chainz
Andreas Mueller
Bartosz T
Benjamin Ragan-Kelley
Carol Willing
Damián Avila
Elliot Marsden
Gao, Xiang
Jaeho Shin
Jan Schulz
Jeremy Kun
Jessica B. Hamrick
John B Nelson
juhasch
Livia Barazzetti
M Pacer
Matej Urbas
Matthias Bussonnier
Matthias Geier
Maximilian Albert
Michael Scott Cuthbert
Nicholas Bollweg
Paul Gowder
Paulo Villegas
Peter Parente
Philipp A
Scott Sanderson
Srinivas Reddy Thatiparthy
Sylvain Corlay
Thomas Kluyver
Till Hoffmann
Xiang Gao
YuviPanda
5.1¶
5.0¶
Use xelatex by default for latex export, improving unicode and font support.
Use entrypoints internally to access Exporters, allowing for packages to declare custom exporters more easily.
New ASCIIDoc Exporter.
New preprocessor for sanitised html output.
New general
convert_pandoc
filter to reduce the need to hard-code lists of filters in templates.Use pytest, nose dependency to be removed.
Refactored Exporter code to avoid ambiguity and cyclic dependencies.
Update to traitlets 4.2 API.
Fixes for Unicode errors when showing execution errors on Python 2.
Default math font matches default Palatino body text font.
General documentation improvements. For example, testing, installation, custom exporters.
Improved link handling for LaTeX output
Refactored the automatic id generation.
New kernel_manager_class configuration option for allowing systems to be set up to resolve kernels in different ways.
Kernel errors now will be logged for debugging purposes when executing notebooks.
4.2¶
Custom Exporters can be provided by external packages, and registered with nbconvert via setuptools entrypoints.
allow nbconvert reading from stdin with
--stdin
option (write intonotebook
basename)Various ANSI-escape fixes and improvements
Various LaTeX/PDF export fixes
Various fixes and improvements for executing notebooks with
--execute
.
4.1¶
setuptools fixes for entrypoints on Windows
various fixes for exporters, including slides, latex, and PDF
fixes for exceptions met during execution
include markdown outputs in markdown/html exports