Autograde a notebook by running it
Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
<cmd> --help-all
--debug
set log level to DEBUG (maximize logging output)
Equivalent to: [--Application.log_level=DEBUG]
--quiet
set log level to CRITICAL (minimize logging output)
Equivalent to: [--Application.log_level=CRITICAL]
--create
Deprecated: Create an entry for the student in the database, if one does not already exist. This is now the default.
Equivalent to: [--Autograde.create_student=True]
--no-execute
Don't execute notebooks and clear output when autograding.
Equivalent to: [--Execute.enabled=False --ClearOutput.enabled=False]
--force
Overwrite an assignment/submission if it already exists.
Equivalent to: [--BaseConverter.force=True]
-f
Overwrite an assignment/submission if it already exists.
Equivalent to: [--BaseConverter.force=True]
--course=<Unicode>
A key that is unique per instructor and course. This can be specified,
either by setting the config option, or using the --course option on the
command line.
Default: ''
Equivalent to: [--CourseDirectory.course_id]
--log-level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
Equivalent to: [--Application.log_level]
--student=<Unicode>
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it. With `nbgrader
submit`, this instead forces the use of an alternative student ID for the
submission. See `nbgrader submit --help`.
If the ID is purely numeric and you are passing it as a flag on the command
line, you will need to escape the quotes in order to have it detected as a
string, for example `--student=""12345""`. See:
https://github.com/jupyter/nbgrader/issues/743
for more details.
Default: '*'
Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
Default: ''
Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
Default: '*'
Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
is another configurable variable.
Default: ''
Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
Default: ''
Equivalent to: [--CourseDirectory.root]
Class options
=============
The command-line option below sets the respective configurable class-parameter:
--Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
--C.a='range(3)'
Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--Application.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--Application.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--Application.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--JupyterApp.config_file=<Unicode>
Full path of a config file.
Default: ''
--JupyterApp.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--JupyterApp.generate_config=<Bool>
Generate default config file.
Default: False
--JupyterApp.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--JupyterApp.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--JupyterApp.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--JupyterApp.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--NbGrader.config_file=<Unicode>
Full path of a config file.
Default: ''
--NbGrader.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--NbGrader.generate_config=<Bool>
Generate default config file.
Default: False
--NbGrader.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--NbGrader.logfile=<Unicode>
Name of the logfile to log to. By default, log output is not written to any
file.
Default: ''
--NbGrader.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--NbGrader.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--NbGrader.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
AutogradeApp(NbGrader) options
------------------------------
--AutogradeApp.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--AutogradeApp.config_file=<Unicode>
Full path of a config file.
Default: ''
--AutogradeApp.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--AutogradeApp.generate_config=<Bool>
Generate default config file.
Default: False
--AutogradeApp.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--AutogradeApp.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--AutogradeApp.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--AutogradeApp.logfile=<Unicode>
Name of the logfile to log to. By default, log output is not written to any
file.
Default: ''
--AutogradeApp.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--AutogradeApp.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--AutogradeApp.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
A plugin for collecting assignments.
Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
A plugin for exchange.
Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
A plugin for fetching assignments.
Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
A plugin for fetching feedback.
Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
A plugin for listing exchange files.
Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
A plugin for releasing assignments.
Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
A plugin for releasing feedback.
Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
A plugin for submitting assignments.
Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'
CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
Default: ''
--CourseDirectory.autograded_directory=<Unicode>
The name of the directory that contains assignment submissions after they
have been autograded. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
A key that is unique per instructor and course. This can be specified,
either by setting the config option, or using the --course option on the
command line.
Default: ''
--CourseDirectory.db_url=<Unicode>
URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
is another configurable variable.
Default: ''
--CourseDirectory.directory_structure=<Unicode>
Format string for the directory structure that nbgrader works over during
the grading process. This MUST contain named keys for 'nbgrader_step',
'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
'notebook_id', as this will be automatically joined with the rest of the
path.
Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
The name of the directory that contains assignment feedback after grading
has been completed. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
Make all instructor files group writeable (g+ws, default g+r only) and
exchange directories group readable/writeable (g+rws, default g=nothing only
) by default. This should only be used if you carefully set the primary
groups of your notebook servers and fully understand the unix permission
model. This changes the default permissions from 444 (unwriteable) to 664
(writeable), so that other instructors are able to delete/overwrite files.
Default: False
--CourseDirectory.ignore=<list-item-1>...
List of file names or file globs. Upon copying directories recursively,
matching files and directories will be ignored with a debug message.
Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
List of file names or file globs. Upon copying directories recursively, non
matching files will be ignored with a debug message.
Default: ['*']
--CourseDirectory.max_dir_size=<Int>
Maximum size of directories (in kilobytes; default: 100Mb). Upon copying
directories recursively, larger files will be ignored with a warning.
Default: 100000
--CourseDirectory.max_file_size=<Int>
Maximum size of files (in kilobytes; default: 100Mb). Upon copying
directories recursively, larger files will be ignored with a warning.
Default: 100000
--CourseDirectory.notebook_id=<Unicode>
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
Default: '*'
--CourseDirectory.release_directory=<Unicode>
The name of the directory that contains the version of the assignment that
will be released to students. This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
Default: 'release'
--CourseDirectory.root=<Unicode>
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
Default: ''
--CourseDirectory.solution_directory=<Unicode>
The name of the directory that contains the assignment solution after
grading has been completed. This corresponds to the `nbgrader_step` variable
in the `directory_structure` config option.
Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
The name of the directory that contains the master/instructor version of
assignments. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
Default: 'source'
--CourseDirectory.source_with_tests_directory=<Unicode>
The name of the directory that contains notebooks with both solutions and
instantiated test code (i.e., all AUTOTEST directives are removed and
replaced by actual test code). This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
Default: 'source_with_tests'
--CourseDirectory.student_id=<Unicode>
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it. With `nbgrader
submit`, this instead forces the use of an alternative student ID for the
submission. See `nbgrader submit --help`.
If the ID is purely numeric and you are passing it as a flag on the command
line, you will need to escape the quotes in order to have it detected as a
string, for example `--student=""12345""`. See:
https://github.com/jupyter/nbgrader/issues/743
for more details.
Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
Comma-separated list of student IDs to exclude. Counterpart of student_id.
This is useful when running commands on all students, but certain students
cause errors or otherwise must be left out. Works at least for autograde,
generate_feedback, and release_feedback.
Default: ''
--CourseDirectory.submitted_directory=<Unicode>
The name of the directory that contains assignments that have been submitted
by students for grading. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
Default: 'submitted'
BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
Whether to overwrite existing assignments/submissions
Default: False
--BaseConverter.permissions=<Int>
Permissions to set on files output by nbgrader. The default is generally
read-only (444), with the exception of nbgrader generate_assignment and
nbgrader generate_feedback, in which case the user also has write
permission.
Default: 0
--BaseConverter.post_convert_hook=<Any>
An optional hook function that you can implement to do some work after
converting. This function is called after the notebooks are converted and
should be used for specific converters such as Autograde, GenerateAssignment
or GenerateFeedback.
It will be called as (all arguments are passed as keywords)::
hook(assignment=assignment, student=student, notebooks=notebooks)
Default: None
--BaseConverter.pre_convert_hook=<Any>
An optional hook function that you can implement to do some bootstrapping
work before converting. This function is called before the notebooks are
converted and should be used for specific converters such as Autograde,
GenerateAssignment or GenerateFeedback.
It will be called as (all arguments are passed as keywords)::
hook(assignment=assignment, student=student, notebooks=notebooks)
Default: None
Autograde(BaseConverter) options
--------------------------------
--Autograde.autograde_preprocessors=<list-item-1>...
Default: [<class 'nbgrader.preprocessors.execute.Execute'>, <class 'nb...
--Autograde.create_student=<Bool>
Whether to create the student at runtime if it does not already exist.
Default: True
--Autograde.exclude_overwriting=<key-1>=<value-1>...
A dictionary with keys corresponding to assignment names and values being a
list of filenames (relative to the assignment's source directory) that
should NOT be overwritten with the source version. This is to allow students
to e.g. edit a python file and submit it alongside the notebooks in their
assignment.
Default: {}
--Autograde.exporter_class=<Type>
Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Autograde.force=<Bool>
Whether to overwrite existing assignments/submissions
Default: False
--Autograde.permissions=<Int>
Permissions to set on files output by nbgrader. The default is generally
read-only (444), with the exception of nbgrader generate_assignment and
nbgrader generate_feedback, in which case the user also has write
permission.
Default: 0
--Autograde.post_convert_hook=<Any>
An optional hook function that you can implement to do some work after
converting. This function is called after the notebooks are converted and
should be used for specific converters such as Autograde, GenerateAssignment
or GenerateFeedback.
It will be called as (all arguments are passed as keywords)::
hook(assignment=assignment, student=student, notebooks=notebooks)
Default: None
--Autograde.pre_convert_hook=<Any>
An optional hook function that you can implement to do some bootstrapping
work before converting. This function is called before the notebooks are
converted and should be used for specific converters such as Autograde,
GenerateAssignment or GenerateFeedback.
It will be called as (all arguments are passed as keywords)::
hook(assignment=assignment, student=student, notebooks=notebooks)
Default: None
--Autograde.sanitize_preprocessors=<list-item-1>...
Default: [<class 'nbgrader.preprocessors.clearoutput.ClearOutput'>, <c...
Examples
--------
Autograde submitted assignments. This takes one argument for the
assignment id, and then (by default) autogrades assignments from the
following directory structure:
submitted/*/{assignment_id}/*.ipynb
and saves the autograded files to the corresponding directory in:
autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb
The student IDs will be created in the database if they don't already
exist.
Note that the assignment must also be present in the database. If it is
not, you should first create it using `nbgrader generate_assignment`. Then, during
autograding, the cells that contain tests for the students' answers will
be overwritten with the master version of the tests that is saved in the
database (this prevents students from modifying the tests in order to
improve their score).
To grade all submissions for "Problem Set 1":
nbgrader autograde "Problem Set 1"
To grade only the submission by student with ID 'Hacker':
nbgrader autograde "Problem Set 1" --student Hacker
To grade only the notebooks that start with '1':
nbgrader autograde "Problem Set 1" --notebook "1*"
By default, student submissions are re-executed and their output cleared.
For long running notebooks, it can be useful to disable this with the
'--no-execute' flag:
nbgrader autograde "Problem Set 1" --no-execute
Note, however, that doing so will not guarantee that students' solutions
are correct. If you use this flag, you should make sure you manually
check all solutions. For example, if a student saved their notebook with
all outputs cleared, then using --no-execute would result in them
receiving full credit on all autograded problems.