Frequently asked questions¶
Can I use nbgrader for purely manually graded assignments (i.e., without autograding)?¶
Yes, absolutely! Mark all the cells where students write their answers as
“Manually graded answer” cells and then during grading run nbgrader autograde
and the formgrader as normal. If you don’t want to even execute the
notebooks, you can pass the --no-execute
flag to
nbgrader autograde.
Can I assign my own cell ids rather than using the autogenerated ones?¶
Yes, by default nbgrader
will assign your cells an id like
cell-8dc50d93bc80e053, but you can alter this to be something more
descriptive if you like. You must use unique names for each graded cell.
Can I hide the test cells in a nbgrader assignment?¶
Yes, as of version 0.5.0 of nbgrader
you will be able to hide tests
in “Autograder tests” cells (see “Autograder tests” cells with hidden tests).
How does nbgrader ensure that students do not change the tests?¶
Please see the documentation on “Read-only” cells.
Does nbgrader support parallel autograding of assignments?¶
Not yet, though it is on the todo list (see #174). PRs welcome!
Does nbgrader protect against infinite loops?¶
Yes. nbgrader will stop executing a cell after a certain period of time. This
timeout is customizable through the ExecutePreprocessor.timeout
configuration option. See Configuration options.
Does nbgrader protect against unsafe code?¶
Not directly, but you can use a kernel inside a Docker container to do secure grading. Please see Grading in a docker container for details.
How does nbgrader handle late assignments?¶
By default nbgrader won’t explicitly assign late penalties, but it will compute how late each submission is. If you wish to customize this default behavior see adding customization plugins.
For this to work, you must include a duedate for the assignment and then a
timestamp.txt
file in the folder for each submission with a single line
containing a timestamp (e.g. 2015-02-02 14:58:23.948203 America/Los_Angeles
). Then, when
you run nbgrader autograde
, nbgrader will record these timestamps into the
database. You can access the timestamps through the API, like so:
from nbgrader.api import Gradebook
with Gradebook("sqlite:///gradebook.db") gb:
assignment = gb.find_assignment("ps1")
for submission in assignment.submissions:
print("Submission from '{}' is {} seconds late".format(
submission.student_id, submission.total_seconds_late))
Note that if you use the release_assignment/fetch_assignment/submit/collect
commands (see
Exchanging assignment files), the timestamp.txt
files will be included
automatically.
Do I have to use sqlite for the nbgrader database?¶
No, and in fact, if you have multiple people grading accessing the formgrader at the same time we strongly encourage you not to use sqlite because it is not threadsafe. Postgres is also supported, and anything else that works with SQLAlchemy is likely to work (e.g. MySQL), though only sqlite and Postgres have been tested. If you want to use another SQL-based database and find that it doesn’t work for some reason, please open an issue!
Does nbgrader work with non-Python kernels?¶
Yes, though it hasn’t been extensively tested with other kernels and it is likely there are some edge cases where things do not work quite right. One thing in particular that you will need to do is customize how the student version is produced – for example, you will need to change the delimiters for the solution regions to use the appropriate comment marks for your language.
If you run into any issues using nbgrader with other kernels, please open an issue!
How do I get out grade information from the database?¶
nbgrader offers a fairly rich API for interfacing with the database. Please see Getting information from the database for more details.
Can I use the “Assignment List” extension with multiple classes?¶
Added in version 0.6.0.
Yes! To use the “Assignment List” extension with multiple courses, you will
want to set the following config options in your students’
nbgrader_config.py
files:
from nbgrader.auth JupyterHubAuthPlugin
c = get_config()
c.Exchange.path_includes_course = True
c.Authenticator.plugin_class = JupyterHubAuthPlugin
The first option (Exchange.path_includes_course
) will tell the transfer
apps (i.e. nbgrader fetch_assignment
, nbgrader submit
, and nbgrader
list
) to assume that the paths for assignments include the course name, such
as ./course101/ps1
rather than just ./ps1
(which is the default). Then,
when using the “Assignment List” extension, students will be able to switch
between different classes.
The second option (Authenticator.plugin_class
) tells nbgrader to ask JupyterHub which courses the student is enrolled in. It does require a bit more setup than simply editing the nbgrader_config.py
file, however. Please see JupyterHub Authentication for details.
Is nbgrader compatible with Windows/Mac/Linux?¶
Linux and Mac¶
nbgrader is fully compatible with Linux and also with Mac (with the exception of JupyterHub integration, as JupyterHub does not run on Mac).
Windows¶
The core pieces of nbgrader will also work on Windows: the “Create Assignment”
extension, nbgrader generate_assignment
, nbgrader autograde
, nbgrader
formgrade
, nbgrader generate_feedback
, nbgrader validate
, and
nbgrader export
.
However, the parts of nbgrader corresponding to file management (the
“Assignment List” extension, nbgrader release_assignment
, nbgrader
fetch_assignment
, nbgrader submit
, nbgrader collect
, nbgrader
list
, nbgrader release_feedback
, nbgrader fetch_feedback
) will not
work under Windows.
What happens if I do some manual grading, and then rerun the autograder?¶
If you rerun the autograder, nbgrader will never overwrite any manual grades or comments that you have added, and manual grades always take precedence over autogrades.
However, if you have given a manual grade, then rerun the autograder, and the autograder produces a grade as well, then it will mark that problem as “needing manual grade”. This functionality is primarily to aid you in grading in the scenarios where you want to grade a newer version of the student’s submission—for example, if you gave them a chance to revise it. In this hypothetical scenario, a student might have not completed a problem, leading you to originally assign it a low partial credit score. But then they turn in a newer version, which you run through the autograder and which attains full credit. Since the manual grade always takes precedence over the autograde, the student would still receive the low score unless you updated your grade: hence the motivation for marking it as needing to be manually graded (again).
Do students have to install anything on their own computers to use nbgrader?¶
No, nbgrader only needs to be installed for the instructor. However, students may optionally install the Validate extension to verify that their submission passes all the test cases.
What should I do if validation or grading of a notebook fails with a “Timeout waiting for execute reply” error?¶
This occurs because the validator or autograder is taking too long to validate or autograde your notebook. This
can be fixed by adding the following line to nbgrader_config.py
:
# increase timeout to 60 seconds
c.ExecutePreprocessor.timeout = 60
Can I modify the notebook behavior during autograding or validation?¶
Yes, when running the autograder or validator, nbgrader sets the NBGRADER_EXECUTION
environment variable, either to 'autograde'
or 'validate'
. You can check whether
this variable is set and then modify the logic in the notebook accordingly. For example,
you could use this to skip executing some cells during autograding only.