How nbgrader server_extenstions call the exchange¶

Defined directories¶

CourseDirectory defines the following directories (and their defaults):

source_directory - Where new assignments that are created by instructors are put (defaults to source)
release_directory - Where assignments that have been processed for release are copied to (defaults to release)
submitted_directory - Where student submissions are copied to, when an instructor collects (defaults to submitted)
autograded_directory - Where student submissions are copied to, having been autograded (defaults to autograded)
feedback_directory - Where feedback is copied to, when Instructors generate feedback (defaults to feedback)
solution_directory - Where solution is copied to, when Istructors generate solution (defaults to solution)

Also, taken from the nbgrader help:

The nbgrader application is a system for assigning and grading notebooks.
Each subcommand of this program corresponds to a different step in the
grading process. In order to facilitate the grading pipeline, nbgrader
places some constraints on how the assignments must be structured. By
default, the directory structure for the assignments must look like this:

    {nbgrader_step}/{student_id}/{assignment_id}/{notebook_id}.ipynb

where 'nbgrader_step' is the step in the nbgrader pipeline, 'student_id'
is the ID of the student, 'assignment_id' is the name of the assignment,
and 'notebook_id' is the name of the notebook (excluding the extension).

`Exchange`¶

Base class. Contains some required configuration parameters and elements - the prominant ones include path_includes_course and coursedir.

This class defines the following methods which are expeceted to be overridden in subclasses:

init_src(): Define the location files are copied from
init_dest(): Define the location files are copied to
copy_files(): Actually copy the files.

The class also defines a convenience method, which may be overridden in subclasses:

def start(self):
    self.set_timestamp()
    self.init_src()
    self.init_dest()
    self.copy_files()

This method is used to perform the relevant action (fetch/release, list, submit etc.)

`ExchangeError`¶

This is the error that should be raised if any error occurs while performing any of the actions.

`ExchangeCollect`¶

Fetches [all] submissions for a specified assignment from the exchange and puts them in the [instructors] home space.

The exchange is called thus:

self.coursedir.assignment_id = assignment_id
exchange = ExchangeFactory(config=config)
collect = exchange.Collect(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    collect.start()
except ExchangeError:
    self.fail("nbgrader collect failed")

The config object passed to the ExchangeFactory needs to contain the configuration specifying which concrete class to use for the ExchangeCollect abstract class.

Expected behaviours¶

The expected destination for collected files is {self.coursedir.submitted_directory}/{student_id}/{self.coursedir.assignment_id}
collect.update is a flag to indicate whether collected files should be replaced if a later submission is available. There is an assumption this defaults to True

`ExchangeFetch`¶

(Depreciated, use ExchangeFetchAssignment)

`ExchangeFetchAssignment`¶

Gets the named assignment & puts the files in the users home space.

The nbgrader server_extension calls it thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        fetch = exchange.FetchAssignment(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        fetch.start()
    .....

Returns…. nothing

Expected behaviours¶

The expected destination for files is {self.assignment_dir}/{self.coursedir.assignment_id} however if self.path_includes_course is True, then the location should be {self.assignment_dir}/{self.coursedir.course_id}/{self.coursedir.assignment_id}

self.coursedir.ignore is described as a:

List of file names or file globs.
Upon copying directories recursively, matching files and
directories will be ignored with a debug message.

This should be honoured.

In the default exchange, existing files are not replaced.

`ExchangeFetchFeedback`¶

This copies feedback from the exchange into the students home space.

The nbgrader server_extension calls it thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        fetch = exchange.FetchFeedback(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        fetch.start()
    .....

returns…. nothing

Expected behaviours¶

Files should be copied into a feedback directory in whichever directory ExchangeFetchAssignment deposited files.
Each submission should be copied into a feedback/{timestamp} directory, where timestamp is the timestamp from the timestamp.txt file generated during the submission.

When writing your own Exchange¶

You to need to consider stopping students from seeing each others submissions

`ExchangeList`¶

This class is responsible for determining what assignments are available to the user.

It has three flags to define various modes of operation:

self.remove=True: If this flag is set, the assignment files (as defined below) are removed from the exchange.
self.inbound=True or self.cached=True: These both refer to submitted assignments. The assignment_list plugin sets config.ExchangeList.cached = True when it queries for submitted notebooks.
neither: This is released (and thus fetched) assignments.

Note that CourseDirectory and Authenticator are defined when the server_sextension assignment_list calls the lister:

with self.get_assignment_dir_config() as config:
    try:
        if course_id:
            config.CourseDirectory.course_id = course_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        lister = exchange.List(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        assignments = lister.start()
    ....

returns a List of Dicts - eg:

[
    {'course_id': 'course_2', 'assignment_id': 'car c2', 'status': 'released', 'path': '/tmp/exchange/course_2/outbound/car c2', 'notebooks': [{'notebook_id': 'Assignment', 'path': '/tmp/exchange/course_2/outbound/car c2/Assignment.ipynb'}]},
    {'course_id': 'course_2', 'assignment_id': 'tree c2', 'status': 'released', 'path': '/tmp/exchange/course_2/outbound/tree c2', 'notebooks': [{'notebook_id': 'Assignment', 'path': '/tmp/exchange/course_2/outbound/tree c2/Assignment.ipynb'}]}
]

The format and structure of this data is discussed in ExchangeList Date Return structure below.

Note¶

This gets called TWICE by the assignment_list server_extension - once for released assignments, and again for submitted assignments.

`ExchangeRelease`¶

(Depreciated, use ExchangeReleaseAssignment)

`ExchangeReleaseAssignment`¶

This should copy the assignment from the release location (normally {self.coursedir.release_directory}/{self.coursedir.assignment_id}) and copies it into the exchange service.

The class should check for the assignment existing (look in {self.coursedir.release_directory}/{self.coursedir.assignment_id}) before actually copying

The exchange is called thus:

exchange = ExchangeFactory(config=config)
release = exchange.ReleaseAssignment(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    release.start()
except ExchangeError:
    self.fail(``nbgrader release_assignment failed``)

returns…. nothing

`ExchangeReleaseFeedback`¶

This should copy all the feedback for the current assignment to the exchange.

Feedback is generated by the Instructor. From GenerateFeedbackApp:

Create HTML feedback for students after all the grading is finished.
This takes a single parameter, which is the assignment ID, and then (by
default) looks at the following directory structure:

    autograded/*/{assignment_id}/*.ipynb

from which it generates feedback the the corresponding directories
according to:

    feedback/{student_id}/{assignment_id}/{notebook_id}.html

The exchange is called thus:

exchange = ExchangeFactory(config=config)
release_feedback = exchange.ReleaseFeedback(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    release_feedback.start()
except ExchangeError:
    self.fail("nbgrader release_feedback failed")

returns….. nothing

`ExchangeSubmit`¶

This should copy the assignment from the user’s work space, and make it available for instructors to collect.

The exchange is called thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id
        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        submit = exchange.Submit(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        submit.start()
    .....

The source for files to be submitted needs to match that in ExchangeFetchAssignment.

returns…. nothing

When writing your own Exchange¶

You to need to consider stopping students from seeing each others submissions
nbgrader functionality requires a file called timestamp.txt to be in the submission, containing the timestamp of that submission. The creation of this file is the responsibility of this class.
Whilst nothing is done as yet, the default exchange checks the names of submitted notebooks, and logs differences.
Submissions need to record student_id, as well as course_id & assignment_id
The default exchange copies files to both an inbound and cache store.

ExchangeList Date Return structure¶

As mentioned in the ExchangeList class documentation above, this data is returned as a List of Dicts.

The format of the Dicts vary depending on the type of assignments being listed.

Removed¶

Returns a list of assignments formatted as below (whether they are released or submitted), but with the status set to removed

Released & Submitted¶

The first step is to loop through a list of assignments (lets call each one a path) and get some basic data:

released

{course_id: xxxx, assignment_id: yyyy}

submitted

{course_id: xxxx, assignment_id: yyyy, student_id: aaaa, timestamp: ISO 8601}

We then add status and path information:

if self.inbound or self.cached:
    info['status'] = 'submitted'
    info['path'] = path  # ie, where it is in the exchange
elif os.path.exists(assignment_dir):
    info['status'] = 'fetched'
    info['path'] = os.path.abspath(assignment_dir)  # ie, where it in on the students home space.
else:
    info['status'] = 'released'
    info['path'] = path # again, where it is in the exchange

if self.remove:
    info['status'] = 'removed'
    # Note, no path - it's been deleted.

(assignment_dir is the directory in the students home space, so needs to take into account self.path_includes_course)

Next loop through all the notebooks in the path, and get some basic data:

nb_info = {'notebook_id': /name, less extension/, 'path': /path_to_file/}

If the notebook is info['status'] != 'submitted':

that’s all the data we have:

info['notebooks'].append(nb_info)

else, add feedback details for this notebook:

nb_info['has_local_feedback'] = _has_local_feedback()
nb_info['has_exchange_feedback'] = _has_exchange_feedback()
if nb_info['has_local_feedback']:
    nb_info['local_feedback_path'] = _local_feedback_path()
if nb_info['has_local_feedback'] and nb_info['has_exchange_feedback']:
    nb_info['feedback_updated'] = _exchange_feedback_checksum() != _local_feedback_checksum()
info['notebooks'].append(nb_info)

Having looped through all notebooks

If info['status'] == 'submitted', add feedback notes to the top-level assignment record:

info['has_local_feedback'] = _any_local_feedback()
info['has_exchange_feedback'] = _any_exchange_feedback()
info['feedback_updated'] = _any_feedback_updated()
if info['has_local_feedback']:
    info['local_feedback_path'] = os.path.join(
        assignment_dir, 'feedback', info['timestamp'])
else:
    info['local_feedback_path'] = None

How nbgrader server_extenstions call the exchange¶

Defined directories¶

Exchange¶

ExchangeError¶

ExchangeCollect¶

Expected behaviours¶

ExchangeFetch¶

ExchangeFetchAssignment¶

Expected behaviours¶

ExchangeFetchFeedback¶

Expected behaviours¶

When writing your own Exchange¶

ExchangeList¶

Note¶

ExchangeRelease¶

ExchangeReleaseAssignment¶

ExchangeReleaseFeedback¶

ExchangeSubmit¶

When writing your own Exchange¶

ExchangeList Date Return structure¶

Removed¶

Released & Submitted¶

`Exchange`¶

`ExchangeError`¶

`ExchangeCollect`¶

`ExchangeFetch`¶

`ExchangeFetchAssignment`¶

`ExchangeFetchFeedback`¶

`ExchangeList`¶

`ExchangeRelease`¶

`ExchangeReleaseAssignment`¶

`ExchangeReleaseFeedback`¶

`ExchangeSubmit`¶