Managing assignment files manually

Distributing assignments to students and collecting them can be a logistical nightmare. If you are relying on distributing the release version of the assignment to the students using your institution’s existing learning management system the process of downloading the students submissions from the learning management system and then getting them back into nbgrader can be simplified by relying on some of nbgrader’s built-in functionality.

Releasing and fetching assignments

After an assignment has been created using nbgrader assign, the instructor must actually release that assignment to students. This section of the documentation assumes you are using your institution’s existing learning management system for distributing the release version of the assignment. It is also assumed that the students will fetch the assignments from - and submit their assignments to - the learning management system.

If this is not the case and you are using nbgrader in a shared server environment, you can do this with the nbgrader release command (see Managing assignment files)

Collecting assignments

See also

nbgrader zip collect
Command line options for nbgrader zip_collect
ZipCollect plugins
Plugins for nbgrader zip_collect
The philosophy and the approach
More details on how the nbgrader hierarchy is structured.
Configuration options
Details on nbgrader_config.py

Once the students have submitted their assignments and you have downloaded these assignment files from your institution’s learning management system, you can get theses files back into nbgrader by using the nbgrader zip_collect sub-command.

Directory Structure:

nbgrader zip_collect makes a few assumptions about how the downloaded assignment files are organized. By default, nbgrader zip_collect assumes that your downloaded assignment files will be organized with the following directory structure:

{downloaded}/{assignment_id}/{collect_step}/

where:

  • The downloaded directory is the main directory where your downloaded assignment files are placed. This location can also be customized (see the configuration options) so that you can run the nbgrader commands from anywhere on your system, but still have them operate on the same directory.
  • assignment_id corresponds to the unique name of an assignment.
  • The collect_step sub-directory corresponds to a step in the zip_collect workflow.

Workflow

The workflow for using nbgrader zip_collect is

  1. You, as an instructor, place submitted files/archives in

    {downloaded}/{assignment_id}/{archive_directory}
    
  2. You run nbgrader zip_collect {assignment_id}, which will:

  1. Extract these archive files - or copy non-archive files - to

    {downloaded}/{assignment_id}/{extracted_directory}
    
  2. Copy these extracted files to

    {submitted_directory}/{student_id}/{assignment_id}/{notebook_id}.ipynb
    
  1. At this point you can use nbgrader autograde to grade the files in the submitted directory (See Autograde assignments).

There are a few subtleties about how nbgrader zip_collect determines the correct student and notebook ids, which we’ll go through in the sections below.

Step 1: Download submission files or archives

The first step in the collect process is to extract files from any archive (zip) files - and copy any non-archive files - found in the following directory:

{downloaded}/{assignment_id}/{archive_directory}/

where:

  • The archive_directory contains the actual submission files or archives downloaded from your institution’s learning management system. nbgrader zip_collect assumes you have already created this directory and placed all downloaded submission files or archives in this directory.

For demo purposes we have already created the directories needed by the nbgrader zip_collect sub-command and placed the downloaded assignment submission files and archive (zip) files in there. For example we have one .zip file and one .ipynb file:

%%bash

ls -l downloaded/ps1/archive
total 64
-rw-r--r--  1 jhamrick  staff  18022 May 31 11:30 notebooks.zip
-rw-r--r--  1 jhamrick  staff   8628 Jul  8 11:51 ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb

But before we can run the nbgrader zip_collect sub-command we first need to specify a few config options:

%%file nbgrader_config.py

c = get_config()

# Only set for demo purposes so as to not mess up the other documentation
c.CourseDirectory.submitted_directory = 'submitted_zip'

# Only collect submitted notebooks with valid names
c.ZipCollectApp.strict = True

# Apply this regular expression to the extracted file filename (absolute path)
c.FileNameCollectorPlugin.named_regexp = (
    '.*_(?P<student_id>\w+)_attempt_'
    '(?P<timestamp>[0-9\-]+)_'
    '(?P<file_id>.*)'
)

c.CourseDirectory.db_assignments = [dict(name="ps1", duedate="2015-02-02 17:00:00 UTC")]
c.CourseDirectory.db_students = [
    dict(id="bitdiddle", first_name="Ben", last_name="Bitdiddle"),
    dict(id="hacker", first_name="Alyssa", last_name="Hacker")
]
Overwriting nbgrader_config.py

Setting the strict flag True skips any submitted notebooks with invalid names.

By default the nbgrader zip_collect sub-command uses the FileNameCollectorPlugin to collect files from the extracted_directory. This is done by sending each filename (absolute path) through to the FileNameCollectorPlugin, which in turn applies a named group regular expression (named_regexp) to the filename.

The FileNameCollectorPlugin returns None if the given file should be skipped or it returns an object that must contain the student_id and file_id data, and can optionally contain the timestamp, first_name, last_name, and email data.

Thus if using the default FileNameCollectorPlugin you must at least supply the student_id and file_id named groups. This plugin assumes all extracted files have the same filename or path structure similar to the downloaded notebook:

%%bash

ls -l downloaded/ps1/archive
total 64
-rw-r--r--  1 jhamrick  staff  18022 May 31 11:30 notebooks.zip
-rw-r--r--  1 jhamrick  staff   8628 Jul  8 11:51 ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb

Note

When collecting files in assignment sub-folders the file_id data must include the relative path to {assignment_id} and the filename in order to preserve the assignment directory structure.

If you wish to use a custom plugin for the ZipCollectApp see ZipCollect plugins for more information.

Before we extract the files, we also need to have run nbgrader assign:

%%bash

nbgrader assign "ps1" --IncludeHeaderFooter.header=source/header.ipynb --force
[AssignApp | WARNING] Removing existing assignment: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/release/ps1
[AssignApp | INFO] Copying /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/source/./ps1/jupyter.png -> /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/release/./ps1/jupyter.png
[AssignApp | INFO] Updating/creating assignment 'ps1': {'duedate': '2015-02-02 17:00:00 UTC'}
[AssignApp | INFO] Converting notebook /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/source/./ps1/problem1.ipynb
[AssignApp | INFO] Writing 7884 bytes to /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/release/./ps1/problem1.ipynb
[AssignApp | INFO] Converting notebook /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/source/./ps1/problem2.ipynb
[AssignApp | INFO] Writing 2258 bytes to /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/release/./ps1/problem2.ipynb
[AssignApp | INFO] Setting destination file permissions to 644

Step 2: Extract, collect, and copy files

With the nbgrader_config.py file created we can now run the nbgrader zip_collect sub-command. This will:

  1. Extract archive - or copy non-archive - files from the {archive_directory} into the following directory:

    {downloaded}/{assignment_id}/{extracted_directory}/
    
  2. Then collect and copy files from the extracted_directory above to the students submitted_directory:

    {submitted_directory}/{student_id}/{assignment_id}/{notebook_id}.ipynb
    

For example:

%%bash

nbgrader zip_collect ps1
[ZipCollectApp | INFO] Using file extractor: ExtractorPlugin
[ZipCollectApp | INFO] Using file collector: FileNameCollectorPlugin
[ZipCollectApp | WARNING] Directory not found. Creating: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted
[ZipCollectApp | INFO] Extracting from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/archive/notebooks.zip
[ZipCollectApp | INFO]   Extracting to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/archive/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO] Start collecting files...
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
[ZipCollectApp | WARNING] Skipped notebook with invalid name 'myproblem1.ipynb'
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | WARNING] 4 files collected, 3 files skipped
[ZipCollectApp | INFO] Start transfering files...
[ZipCollectApp | WARNING] Directory not found. Creating: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem2.ipynb
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem1.ipynb
[ZipCollectApp | INFO] Creating timestamp: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/timestamp.txt
[ZipCollectApp | WARNING] Directory not found. Creating: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem1.ipynb
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem2.ipynb
[ZipCollectApp | INFO] Creating timestamp: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/timestamp.txt

After running the nbgrader zip_collect sub-command, the archive (zip) files were extracted - and the non-archive files were copied - to the extracted_directory:

%%bash

ls -l downloaded/ps1/extracted/
ls -l downloaded/ps1/extracted/notebooks/
total 24
drwxr-xr-x  8 jhamrick  staff   272 Jul  8 11:52 notebooks
-rw-r--r--  1 jhamrick  staff  8628 Jul  8 11:52 ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
total 88
-rw-rw-r--  1 jhamrick  staff  5733 Jul  8 11:52 ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
-rw-rw-r--  1 jhamrick  staff  7712 Jul  8 11:52 ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
-rw-rw-r--  1 jhamrick  staff  2228 Jul  8 11:52 ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
-rw-rw-r--  1 jhamrick  staff  5733 Jul  8 11:52 ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
-rw-rw-r--  1 jhamrick  staff  8830 Jul  8 11:52 ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
-rw-rw-r--  1 jhamrick  staff  2358 Jul  8 11:52 ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb

By default archive files will be extracted into their own sub-directory in the extracted_directory and any archive files, within archive files, will also be extracted into their own sub-directory along the path. To change this default behavior you can write your own extractor plugin for zip_collect (see ZipCollect plugins).

These extracted files were then collected and copied into the students submitted_directory:

%%bash

ls -l submitted_zip
total 0
drwxr-xr-x  3 jhamrick  staff  102 Jul  8 11:52 bitdiddle
drwxr-xr-x  3 jhamrick  staff  102 Jul  8 11:52 hacker
%%bash

ls -l submitted_zip/hacker/ps1/
total 40
-rw-r--r--  1 jhamrick  staff  8628 Jul  8 11:52 problem1.ipynb
-rw-rw-r--  1 jhamrick  staff  2358 Jul  8 11:52 problem2.ipynb
-rw-r--r--  1 jhamrick  staff    19 Jul  8 11:52 timestamp.txt

Custom plugins

See also

ZipCollect plugins
Plugins for nbgrader zip_collect

Unfortunately, for the demo above, the timestamp strings from the filenames did not parse correctly:

%%bash

cat submitted_zip/hacker/ps1/timestamp.txt
2016-01-31 06:00:00

This is an issue with the underlying dateutils package used by nbgrader. But not to worry, we can easily create a custom collector plugin to correct the timestamp strings when the files are collected, for example:

%%file plugin.py

from nbgrader.plugins import FileNameCollectorPlugin


class CustomPlugin(FileNameCollectorPlugin):
    def collect(self, submission_file):
        info = super(CustomPlugin, self).collect(submission_file)
        if info is not None:
            info['timestamp'] = '{}-{}-{} {}:{}:{}'.format(
                *tuple(info['timestamp'].split('-'))
            )
        return info
Writing plugin.py
%%bash

# Use force flag to overwrite existing files
nbgrader zip_collect --force --collector=plugin.CustomPlugin ps1
[ZipCollectApp | INFO] Using file extractor: ExtractorPlugin
[ZipCollectApp | INFO] Using file collector: CustomPlugin
[ZipCollectApp | WARNING] Clearing existing files in /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted
[ZipCollectApp | INFO] Extracting from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/archive/notebooks.zip
[ZipCollectApp | INFO]   Extracting to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/archive/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO] Start collecting files...
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
[ZipCollectApp | WARNING] Skipped notebook with invalid name 'myproblem1.ipynb'
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | WARNING] 4 files collected, 3 files skipped
[ZipCollectApp | INFO] Start transfering files...
[ZipCollectApp | WARNING] Clearing existing files in /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem1.ipynb
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem2.ipynb
[ZipCollectApp | INFO] Creating timestamp: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/timestamp.txt
[ZipCollectApp | WARNING] Clearing existing files in /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem2.ipynb
[ZipCollectApp | INFO] Copying from: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem1.ipynb
[ZipCollectApp | INFO] Creating timestamp: /Users/jhamrick/project/tools/nbgrader/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/timestamp.txt

The --force flag is used this time to overwrite existing extracted and submitted files. Now if we check the timestamp we see it parsed correctly:

%%bash

cat submitted_zip/hacker/ps1/timestamp.txt
2016-01-30 20:30:10

Note that there should only ever be one instructor who runs the nbgrader zip_collect command (and there should probably only be one instructor – the same instructor – who runs nbgrader assign, nbgrader autograde and nbgrader formgrade as well). However this does not mean that only one instructor can do the grading, it just means that only one instructor manages the assignment files. Other instructors can still perform grading by accessing the formgrader URL.