You're reading an old version of this documentation. For up-to-date information, please have a look at v0.12.

aspecd.report module

General facilities for generating reports.

To do scientific research in terms of reproducibility and traceability it’s highly necessary to report all the steps done on a given dataset and never separate the dataset from its metadata. Similarly, a recipe in context of recipe-driven data analysis stores a lot of relevant information on what tasks have been performed on a series of datasets (and even more so the recipe history). However, having a dataset (or recipe history) containing all these metadata is only useful if there are easy ways to retrieve and present the information stored. This is the task of reports. This module provides functionality to create reports based on templates provided either by the user or by the package as such.

“Batteries included”: Templates contained in the package 

The “batteries included” approach of Python itself is probably responsible to a great deal for the success of Python as a language. ASpecD, similarly, tries to provide you with a sensible set of tools you need for your routine data analysis in spectroscopy. Reports are no exception to that rule.

Hence, ASpecD comes bundled with a (growing) series of templates allowing you to create reports of datasets and alike. Thus, getting access to all information stored in a single dataset is as simple as calling a single reporter, and in context of recipe-driven data analysis, it is even simpler:

- kind: report
  type: LaTeXReporter
  properties:
    template: dataset.tex
    filename: report.tex
  compile: true

This would create a report of a dataset that is then stored in the file report.tex, using the template dataset.tex bundled with the ASpecD package. As you even set compile to true, it would even compile the LaTeX report, including all figures generated during cooking the recipe and referenced from within the report. Hence, you end up in your current directory with both, a LaTeX file report.tex and a PDF file report.pdf.

But what if you don’t like the way the bundled templates look like? Don’t worry, we’ve got you covered: Simply provide a relative or absolute path to your own template, even with the same name. Hence, in the above example, if you place a file dataset.tex in the directory you serve the recipe from, it will be used instead of the bundled one. Developers of other packages can do the same and provide templates with the same name as those provided by the ASpecD package. Theirs will take precedence. For details, see below.

Output formats 

Template engines provide means of a “separation of concerns” (a term coined by Edsger W. Dijkstra and highly important not only in software development): The data source is entirely independent of the formatting of the report, and one and the same template engine can be used to create reports in a multitude of output formats.

Currently, the ASpecD framework supports two output formats (more may be added in the future): plain text (txt) and LaTeX. The respective reporter classes are:

TxtReporter

LaTeXReporter

The bundled templates used with the respective reporters are stored within the templates/report/ directory of the ASpecD package, and here within subdirectories (txt, latex) for each of the formats. This makes it easy to add additional formats and to shorten the template paths when using the reporters.

Choosing the language of a report 

While ASpecD is currently designed to predominantly use English as its native language, reports support different languages, provided the templates for the respective language are contained in the package.

To select a language (different than the default, English), set the Reporter.language attribute accordingly. By default, two-letter ISO codes are used, such as en (English) and de (German). Please note that this setting will only have an effect if templates for the requested language are provided either by the ASpecD package or by the package based on ASpecD.

In context of recipe-driven data analysis, getting a report on a dataset in German rather than in English, use the following:

- kind: report
  type: LaTeXReporter
  properties:
    language: de
    template: datensatz.tex
    filename: bericht.tex
  compile: true

Note that the name of the template changes as well, as it would not make too much sense to have English filenames for templates written in other languages.

Important

As the ASpecD framework natively uses the English language for descriptions, parameter names and alike, providing templates in different languages will likely result in a mixture of languages within the final report.

Package developers: Organisation of templates 

As mentioned above, the bundled templates used with the respective reporters are stored within the templates/report/ directory of the ASpecD package. Each output format has its own subdirectory, currently existing directories are txt and latex. Furthermore, to allow for different languages, each of these directories can (and should) contain another interlayer in terms of directories representing the languages. By default, two-letter ISO codes are used, such as en (English) and de (German).

Currently, the ASpecD template organisation looks similar to the following:

templates/
    report/
        latex/
            de/
                basis.tex
                datensatz.tex
                ...
            en/
                base.tex
                dataset.tex
                ...
        txt/
            en/
                dataset.txt
                ...

If you do not plan to support multiple languages, you can skip the language directories. In this case, the templates are supposed to be in English language.

To change the contents of a (sub)template for your package, e.g. the “colophon.tex” template containing important information on how a report has been generated, simply provide a template with this name in the corresponding template directory within your package. As long as you are using recipe-driven data analysis and provide a default package in your recipe, the reporters will be notified and look for templates in your package before defaulting to the templates provided with the ASpecD framework.

Background: Jinja2 

The report functionality relies heavily on the Jinja2 template engine . Therefore, it is very useful to be generally familiar with the concepts of Jinja2. Basically, this template engine allows users to specify rather complicated replacements and logic within the template. However, logic within templates should be used sparingly, and the template should always be rendered correctly even without processing it by the template engine. Only then templates are easy to develop and adapt.

Two concepts used by Jinja2 the user of the report facilities of the ASpecD framework should be familiar with are “environment” and “context”. The former is a list of settings determining the type of delimiters used within a certain template for the control structures that are understood by Jinja2. As Jinja2 is developed with web applications (and hence HTML) in mind, those delimiters may not be feasible for other types of languages a template may be written in, such as LaTeX.

Currently, the aspecd.report module of the ASpecD framework provides a generic environment as well as dedicated Txt and LaTeX environments, implemented as respective classes:

GenericEnvironment

TxtEnvironment

LaTeXEnvironment

These environments get automatically loaded by the respective reporter classes:

Reporter

TxtReporter

LaTeXReporter

While the TxtEnvironment and TxtReporter classes are basically identical to the GenericEnvironment and Reporter classes, respectively, the LaTeXEnvironment and LaTeXReporter provide some heavy adaptations to rendering and even compiling LaTeX templates.

The second important concept of Jinja2 is that of the “context”: Think of it as a dictionary containing all the key–value pairs you can use to replace placeholders within a template with their actual values. In the simplest of all cases within the context of the ASpecD framework, this could be the metadata of an aspecd.dataset.Dataset.

Module documentation 

class aspecd.report.Reporter(template='', filename='')

Bases: ToDictMixin

Base class for reports.

To generate a report from a template, you will need a template in the first place. The path to the template (including its filename) is set in the template attribute that is either provided as parameter at initialisation or assigned later on. Similarly, you should specify an output filename, stored in the filename attribute. The variables known to the template that should be replaced by appropriate content, termed “context” in Jinja2 language, are stored in the context attribute. This is, by default, an ordered dict to preserve the order of the keys it contains. This is sometimes very useful.

Next, you want to render the template, and most often, save it to a file afterwards. If you would like to separate those steps, you can call the render() and save() methods separately. For convenience, simply call create() to render the report and save it to the file whose name you have provided.

The whole procedure may look as follows:

template = "/path/to/my/template.tex"
filename = "/path/to/my/final/report.tex"
report_ = aspecd.report.Reporter(template=template, filename=filename)
report_.create()

template

Template file used to generate report.

Type:: str

filename

Name of the resulting template file.

Type:: str

context

Variables of a template that are replaced with the given content.

It contains a key “sysinfo” containing system-related information, i.e. the information contained in the aspecd.system.SystemInfo class.

Furthermore, the key “template_dir” contains the (relative or absolute) path to the template provided in template. This is particularly useful for including subtemplates.

A key “timestamp” contains the current timestamp when starting to render the report.

Type:: collections.OrderedDict

environment

Jinja2 environment used for rendering the template.

Defaults to a aspecd.report.GenericEnvironment object with settings for rendering generic templates.

Type:: aspecd.report.GenericEnvironment

report

Actual report, i.e. rendered template

Type:: str

package

Name of the package a template loader shall be added for

Additionally to adding a template loader, the package name and its version number as well as all its dependencies get added to the sysinfo key of the context dictionary.

Type:: str

package_path

Path to the templates within the package defined by package

Type:: str

language

(Human) language of the templates

Usually a two-letter code. Only if the corresponding template directory exists within the package, the language will be set.

Type:: str

comment

User-supplied comment describing intent, purpose, reason, …

Type:: str

Parameters:

template (str) – Path to template file used to generate report.
filename (str) – Path to the output file the report should be rendered to.

Raises:

aspecd.report.FileNotFoundError – Raised if the template file provided does not exist.
aspecd.report.MissingFilenameError – Raised if no output file for the report is provided.

Changed in version 0.6.3: New attributes package, package_path, language

Changed in version 0.6.4: New attribute comment

render()

Render the template.

The actual rendering of the template should be implemented within the non-public method _render().

Before calling the non-public method _render(), the renderer checks the existence of the file provided as attribute template as well as whether an output filename has been provided. If this is not the case, a corresponding exception will be raised.

Raises:: FileNotFoundError – Raised if the template file provided does not exist.

save()

Save report to file.

Raises:: aspecd.report.MissingFilenameError – Raised if no output file for the report is provided.

create()

Create report, thus rendering template and saving result to file.

Convenience method to render the template and save the generated report to a file, thus simply calling render() and save() subsequently.

class aspecd.report.TxtReporter(template='', filename='')

Bases: Reporter

Plain text reporter.

The most basic format for a report is plain text. Its probably biggest advantage is that it is intrinsically platform-independent (except of the encoding, but UTF-8 should not pose any problems any more in 2021 and after).

The perhaps biggest disadvantage is the lack of standard formatting and the overall limited formatting options, not to speak of including figures.

As such, the plain text reporter can be used for a first overview and for maximum portability. For well-formatted reports, have a look at the LaTeXReporter.

environment

Jinja2 environment used for rendering the template.

Similar to the aspecd.report.GenericEnvironment, but with the package path for template lookup set to the path for txt templates within the ASpecD package.

Type:: aspecd.report.TxtEnvironment

New in version 0.6.

class aspecd.report.LaTeXReporter(template='', filename='')

Bases: Reporter

LaTeX Reporter.

Often, templates for reports are written in LaTeX, and the results typeset as PDF file upon a (pdf)LaTeX run. For convenience, this class offers the necessary facilities to compile the template once written.

The whole procedure may look as follows:

template = "template.tex"
filename = "report.tex"
report_ = aspecd.report.LaTeXReporter(template=template,
                                      filename=filename)
report_.create()
report_.compile()

This will result with a file “report.pdf” in the current directory. If you specify a relative or absolute path for the filename of the report, the resulting PDF file will be copied to that path accordingly.

Note that for compiling a temporary directory is used, such as not to clutter the current working directory with all the auxiliary files usually created during a (pdf)LaTeX run. Furthermore, currently, only a single (pdf)LaTeX run is performed with option “-interaction=nonstopmode” passed in order to not block further execution.

Important

For enhanced security, the temporary directory used for compiling the template will be removed after successful compilation. Therefore, no traces of your report should remain outside the current directory controlled by the user.

Note

Due to problems with LaTeX rendering text containing underscores, the keys in the context dict are recursively parsed and each key containing underscores converted to camel case (but preserving the case of the first character: “foo_bar” => “fooBar”). Thus, the template can be compiled using LaTeX without having to replace the placeholder variables beforehand.

environment

Jinja2 environment used for rendering the template.

Defaults to a aspecd.report.LaTeXEnvironment object with settings for rendering LaTeX templates.

Type:: aspecd.report.LaTeXEnvironment

includes

List of files that need to be present for compiling the template.

These files will be copied into the temporary directory used for compiling the template.

Type:: list

latex_executable

Name of/path to the LaTeX executable.

Defaults to “pdflatex”

Type:: str

Parameters:

template (str) – Path to template file used to generate report.
filename (str) – Path to the output file the report should be rendered to.

Raises:

aspecd.report.LaTeXExecutableNotFoundError – Raised if the LaTeX executable could not be found

compile()

Compile LaTeX template.

The template is copied to a temporary directory and the LaTeX executable specified in latex_executable called on the report. Afterwards, the result is copied back to the original directory.

Additionally, all files necessary to compile the report are copied to the temporary directory as well.

Raises:: aspecd.report.LaTeXExecutableNotFoundError – Raised if the LaTeX executable could not be found

class aspecd.report.GenericEnvironment(env=None, path='templates/report/', lang=None)

Bases: Environment

Jinja2 environment for rendering generic templates.

The environment does not change any of the jinja settings except of the loaders. Here, a list of loaders using jinja2.ChoiceLoader is implemented. Using this loader makes it possible to search subsequently in different places for a template. Here, the first hit is used, therefore, the sequence of loaders is crucial.

Currently, there are two loaders implemented, in exactly this sequence:

jinja2.FileSystemLoader

Looking for templates in the current directory and using an absolute path
jinja2.PackageLoader

Looking for templates in the aspecd package in the package path “templates/report/”, i.e. the base directory for all report templates of the ASpecD package.

Additional package loaders can be inserted before the package loader for the ASpecD package using the method add_package_loader(). This allows packages based on the ASpecD framework to define templates with the same name as those in ASpecD and thus to load the templates provided by the derived package rather than those from ASpecD.

path

Path to the templates within the package

Default: “templates/report”

Type:: str

Parameters:

env (dict) –
Dictionary used for creating the jinja2.Environment

Can be used by derived classes to override the environment.
path (str) – Path to the templates within the package
lang (str) –
Language of the templates

Usually a two-letter code; if a corresponding subdirectory to path exists within the template package directory, it will be added to the path of the template package loaders.

Changed in version 0.6.3: New attribute package_path, new parameters env, path, lang

set_language()

Adjust the template directory of the package loaders for the language.

Only if a language is set in the class and the corresponding template directory exists, the loaders will be updated. As it seems, there is currently no way to adjust the package_path of a jinja2.PackageLoader, hence the loaders are replaced with new loaders with adjusted package_path.

add_package_loader(package_name='', package_path='')

Add a package loader for a given package name.

The package loader will be inserted before the loader for the ASpecD package. This allows packages based on the ASpecD framework to define templates with the same name as those in ASpecD and thus to load the templates provided by the derived package rather than those from ASpecD.

Only in case of the given package path to exist the package loader will be added.

Parameters:

package_name (str) – Name of the package to add the loader for
package_path (str) –
Path to the templates within the package

Defaults to package_path

New in version 0.6.3.

class aspecd.report.TxtEnvironment(lang=None)

Bases: GenericEnvironment

Jinja2 environment for rendering generic text templates.

The environment does not change any of the jinja settings except of the loaders. Here, a list of loaders using jinja2.ChoiceLoader is implemented. Using this loader makes it possible to search subsequently in different places for a template. Here, the first hit is used, therefore, the sequence of loaders is crucial.

Currently, there are two loaders implemented, in exactly this sequence:

jinja2.FileSystemLoader

Looking for templates in the current directory and using an absolute path
jinja2.PackageLoader

Looking for templates in the aspecd package in the package path “templates/report/txt/”, i.e. the directory for all bare text report templates of the ASpecD package.

Changed in version 0.6.3: Now based on GenericEnvironment

class aspecd.report.LaTeXEnvironment(lang=None)

Bases: GenericEnvironment

Jinja2 environment for rendering LaTeX-based templates.

This environment is designed for using templates written in LaTeX that can be rendered by LaTeX without having their control code replaced by Jinja2. While variables are usually output in LaTeX, control structures are prefixed by a LaTeX comment character (%). For convenience, the following table lists all the variables currently set within this environment.

Variable	Value
block_start_string	%{
block_end_string	}%
variable_start_string	{@
variable_end_string	}
comment_start_string	%#{
comment_end_string	}
line_statement_prefix	%%
line_comment_prefix	%#
trim_blocks	True
autoescape	False

While every measure is taken to keep the above information as accurate as possible, for authoritative information the reader is referred to the actual source code.

Besides extensively modifying the control codes used within the template, the environment implements a list of loaders using jinja2.ChoiceLoader. Using this loader makes it possible to search subsequently in different places for a template. Here, the first hit is used, therefore, the sequence of loaders is crucial.

Currently, there are two loaders implemented, in exactly this sequence:

jinja2.FileSystemLoader

Looking for templates in the current directory and using an absolute path
jinja2.PackageLoader

Looking for templates in the aspecd package in the package path “templates/report/latex/”, i.e. the directory for all bare text report templates of the ASpecD package.

Changed in version 0.6.3: Now based on GenericEnvironment

aspecd.report module

“Batteries included”: Templates contained in the package

Output formats

Choosing the language of a report

Package developers: Organisation of templates

Background: Jinja2

Module documentation

“Batteries included”: Templates contained in the package 

Output formats 

Choosing the language of a report 

Package developers: Organisation of templates 

Background: Jinja2 

Module documentation 