Advanced topics

This file covers some more advanced use cases of nbgrader.

Advanced “Assignment List” installation

See also

Installation

General installation instructions.

Managing assignment files
Details on fetching and submitting assignments using the “Assignment List”

plugin.

Warning

The “Assignment List” extension is not currently compatible with multiple courses on the same server: it will only work if there is a single course on the server. This is a known issue (see #544). PRs welcome!

This section covers some further and configuration scenarios that often occur with the assignment list extension.

In previous versions of nbgrader, a special process had to be used to enable this extension for all users on a multi-user system. As described in the main Installation documentation this is no longer required.

If you know you have released an assignment but still don’t see it in the list of assignments, check the output of the notebook server to see if there are any errors. If you do in fact see an error, try running the command manually on the command line from the directory where the notebook server is running. For example:

$ nbgrader list
[ListApp | ERROR] Unwritable directory, please contact your instructor: /srv/nbgrader/exchange

This error that the exchange directory isn’t writable is an easy mistake to make, but also relatively easy to fix. If the exchange directory is at /srv/nbgrader/exchange, then make sure you have run:

chmod +rw /srv/nbgrader/exchange

Getting information from the database

nbgrader offers a fairly rich API for interfacing with the database. The API should allow you to access pretty much anything you want, though if you find something that can’t be accessed through the API please open an issue!

In this example, we’ll go through how to create a CSV file of grades for each student and assignment using nbgrader and pandas.

New in version 0.4.0: nbgrader now comes with CSV export functionality out-of-the box using the nbgrader export command. However, this example is still kept for reference as it may be useful for defining your own exporter.

import pandas as pd
from nbgrader.api import Gradebook, MissingEntry

# Create the connection to the database
with Gradebook('sqlite:///gradebook.db') as gb:

    grades = []

    # Loop over each assignment in the database
    for assignment in gb.assignments:

        # Loop over each student in the database
        for student in gb.students:

            # Create a dictionary that will store information about this student's
            # submitted assignment
            score = {}
            score['max_score'] = assignment.max_score
            score['student'] = student.id
            score['assignment'] = assignment.name

            # Try to find the submission in the database. If it doesn't exist, the
            # `MissingEntry` exception will be raised, which means the student
            # didn't submit anything, so we assign them a score of zero.
            try:
                submission = gb.find_submission(assignment.name, student.id)
            except MissingEntry:
                score['score'] = 0.0
            else:
                score['score'] = submission.score

            grades.append(score)

    # Create a pandas dataframe with our grade information, and save it to disk
    grades = pd.DataFrame(grades).set_index(['student', 'assignment']).sortlevel()
    grades.to_csv('grades.csv')

    # Print out what the grades look like
    with open('grades.csv', 'r') as fh:
        print(fh.read())

After running the above code, you should see that grades.csv contains something that looks like:

student,assignment,max_score,score
bitdiddle,ps1,9.0,1.5
hacker,ps1,9.0,3.0

Using nbgrader preprocessors

Several of the nbgrader preprocessors can be used with nbconvert without actually relying on the rest of the nbgrader machinery. In particular, the following preprocessors can be applied to other nbconvert workflows:

  • ClearOutput – clears outputs of all cells

  • ClearSolutions – removes solutions between the solution delimeters (see “Autograded answer” cells).

  • HeaderFooter – concatenates notebooks together, prepending a “header” notebook and/or appending a “footer” notebook to another notebook.

  • LimitOutput – limits the amount of output any given cell can have. If a cell has too many lines of outputs, they will be truncated.

Using these preprocessors in your own nbconvert workflow is relatively straightforward. In your nbconvert_config.py file, you would add, for example:

c.Exporter.preprocessors = ['nbgrader.preprocessors.ClearSolutions']

See also the nbconvert docs on custom preprocessors.

Calling nbgrader apps from Python

New in version 0.5.0: Much of nbgrader’s high level functionality can now be accessed through an official Python API.

Grading in a docker container

For security reasons, it may be advantageous to do the grading with a kernel running in isolation, e.g. in a docker container. We will assume that docker is already installed and an appropriate image has been downloaded. Otherwise, refer to the docker documentation for information on how to install and run docker.

A convenient way to switch to a kernel running in a docker container is provided by envkernel which serves a double purpose. In a first step, it is writing a new kernelspec file. Later it ensures that the docker container is run and the kernel started.

Presently, envkernel is only available from its Github repository and can be installed directly from there into a virtual environment

pip install https://github.com/NordicHPC/envkernel/archive/master.zip

As an alternative, the script envkernel.py can be put in a different location, e.g. /opt/envkernel, as long as it is accessible there also later during grading.

Now, a new kernel can be installed by means of

./envkernel.py docker --name=NAME --display-name=DNAME DOCKER-IMAGE

Here, NAME should be replaced by the name to be given to the kernel. After installation of the kernel, it will be displayed in the list of kernels when executing jupyter kernelspec list. DNAME should be replaced by the name under which the kernel shall be known in the Jupyter notebook GUI. After installation of the kernel, this name will be listed as a possible kernel when starting a new notebook. Finally, DOCKER-IMAGE should be replaced by the name of the docker image in which the kernel is to be run, e.g. python:3, continuumio/anaconda3, or some other suitable image.

The command given above will install the kernel in the system-wide location for Jupyter data files. If installation in the corresponding user directory is desired, the option --user should be added before the name of the docker image. By default, envkernel will install a Python kernel. For the installation of other kernels, see the README of envkernel.

In order to run the grading process with the new kernel, one can specify its name in nbgrader_config.py

c.ExecutePreprocessor.kernel_name = NAME

where NAME should be replaced by the name chosen when running the envkernel script. Alternatively, the name can be specified when running nbgrader from the command line

nbgrader autograde --ExecutePreprocessor.kernel_name=NAME ASSIGNMENT_NAME

In addition to docker, envkernel also supports singularity as a containerization system. For details on using envkernel with singularity, see the README of envkernel.