Validate a notebook by running it
Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
<cmd> --help-all
--invert
Complain when cells pass, rather than vice versa.
Equivalent to: [--Validator.invert=True]
Class options
=============
The command-line option below sets the respective configurable class-parameter:
--Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
--C.a='range(3)'
Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--Application.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--Application.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--Application.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--JupyterApp.config_file=<Unicode>
Full path of a config file.
Default: ''
--JupyterApp.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--JupyterApp.generate_config=<Bool>
Generate default config file.
Default: False
--JupyterApp.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--JupyterApp.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--JupyterApp.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--JupyterApp.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--NbGrader.config_file=<Unicode>
Full path of a config file.
Default: ''
--NbGrader.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--NbGrader.generate_config=<Bool>
Generate default config file.
Default: False
--NbGrader.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--NbGrader.logfile=<Unicode>
Name of the logfile to log to. By default, log output is not written to any
file.
Default: ''
--NbGrader.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--NbGrader.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--NbGrader.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
ValidateApp(NbGrader) options
-----------------------------
--ValidateApp.answer_yes=<Bool>
Answer yes to any prompts.
Default: False
--ValidateApp.config_file=<Unicode>
Full path of a config file.
Default: ''
--ValidateApp.config_file_name=<Unicode>
Specify a config file to load.
Default: ''
--ValidateApp.generate_config=<Bool>
Generate default config file.
Default: False
--ValidateApp.log_datefmt=<Unicode>
The date format used by logging formatters for %(asctime)s
Default: '%Y-%m-%d %H:%M:%S'
--ValidateApp.log_format=<Unicode>
The Logging format template
Default: '[%(name)s]%(highlevel)s %(message)s'
--ValidateApp.log_level=<Enum>
Set the log level by value or name.
Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
Default: 30
--ValidateApp.logfile=<Unicode>
Name of the logfile to log to. By default, log output is not written to any
file.
Default: ''
--ValidateApp.logging_config=<key-1>=<value-1>...
Configure additional log handlers.
The default stderr logs handler is configured by the log_level, log_datefmt
and log_format settings.
This configuration can be used to configure additional handlers (e.g. to
output the log to a file) or for finer control over the default handlers.
If provided this should be a logging configuration dictionary, for more
information see:
https://docs.python.org/3/library/logging.config.html#logging-config-
dictschema
This dictionary is merged with the base logging configuration which defines
the following:
* A logging formatter intended for interactive use called
``console``.
* A logging handler that writes to stderr called
``console`` which uses the formatter ``console``.
* A logger with the name of this application set to ``DEBUG``
level.
This example adds a new handler that writes to a file:
.. code-block:: python
c.Application.logging_config = {
"handlers": {
"file": {
"class": "logging.FileHandler",
"level": "DEBUG",
"filename": "<path/to/file>",
}
},
"loggers": {
"<application-name>": {
"level": "DEBUG",
# NOTE: if you don't list the default "console"
# handler here then it will be disabled
"handlers": ["console", "file"],
},
},
}
Default: {}
--ValidateApp.show_config=<Bool>
Instead of starting the Application, dump configuration to stdout
Default: False
--ValidateApp.show_config_json=<Bool>
Instead of starting the Application, dump configuration to stdout (as JSON)
Default: False
ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
A plugin for collecting assignments.
Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
A plugin for exchange.
Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
A plugin for fetching assignments.
Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
A plugin for fetching feedback.
Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
A plugin for listing exchange files.
Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
A plugin for releasing assignments.
Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
A plugin for releasing feedback.
Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
A plugin for submitting assignments.
Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'
CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
Default: ''
--CourseDirectory.autograded_directory=<Unicode>
The name of the directory that contains assignment submissions after they
have been autograded. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
A key that is unique per instructor and course. This can be specified,
either by setting the config option, or using the --course option on the
command line.
Default: ''
--CourseDirectory.db_url=<Unicode>
URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
is another configurable variable.
Default: ''
--CourseDirectory.directory_structure=<Unicode>
Format string for the directory structure that nbgrader works over during
the grading process. This MUST contain named keys for 'nbgrader_step',
'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
'notebook_id', as this will be automatically joined with the rest of the
path.
Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
The name of the directory that contains assignment feedback after grading
has been completed. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
Make all instructor files group writeable (g+ws, default g+r only) and
exchange directories group readable/writeable (g+rws, default g=nothing only
) by default. This should only be used if you carefully set the primary
groups of your notebook servers and fully understand the unix permission
model. This changes the default permissions from 444 (unwriteable) to 664
(writeable), so that other instructors are able to delete/overwrite files.
Default: False
--CourseDirectory.ignore=<list-item-1>...
List of file names or file globs. Upon copying directories recursively,
matching files and directories will be ignored with a debug message.
Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
List of file names or file globs. Upon copying directories recursively, non
matching files will be ignored with a debug message.
Default: ['*']
--CourseDirectory.max_dir_size=<Int>
Maximum size of directories (in kilobytes; default: 100Mb). Upon copying
directories recursively, larger files will be ignored with a warning.
Default: 100000
--CourseDirectory.max_file_size=<Int>
Maximum size of files (in kilobytes; default: 100Mb). Upon copying
directories recursively, larger files will be ignored with a warning.
Default: 100000
--CourseDirectory.notebook_id=<Unicode>
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
Default: '*'
--CourseDirectory.release_directory=<Unicode>
The name of the directory that contains the version of the assignment that
will be released to students. This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
Default: 'release'
--CourseDirectory.root=<Unicode>
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
Default: ''
--CourseDirectory.solution_directory=<Unicode>
The name of the directory that contains the assignment solution after
grading has been completed. This corresponds to the `nbgrader_step` variable
in the `directory_structure` config option.
Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
The name of the directory that contains the master/instructor version of
assignments. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
Default: 'source'
--CourseDirectory.source_with_tests_directory=<Unicode>
The name of the directory that contains notebooks with both solutions and
instantiated test code (i.e., all AUTOTEST directives are removed and
replaced by actual test code). This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
Default: 'source_with_tests'
--CourseDirectory.student_id=<Unicode>
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it. With `nbgrader
submit`, this instead forces the use of an alternative student ID for the
submission. See `nbgrader submit --help`.
If the ID is purely numeric and you are passing it as a flag on the command
line, you will need to escape the quotes in order to have it detected as a
string, for example `--student=""12345""`. See:
https://github.com/jupyter/nbgrader/issues/743
for more details.
Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
Comma-separated list of student IDs to exclude. Counterpart of student_id.
This is useful when running commands on all students, but certain students
cause errors or otherwise must be left out. Works at least for autograde,
generate_feedback, and release_feedback.
Default: ''
--CourseDirectory.submitted_directory=<Unicode>
The name of the directory that contains assignments that have been submitted
by students for grading. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
Default: 'submitted'
Validator(LoggingConfigurable) options
--------------------------------------
--Validator.changed_warning=<Unicode>
Warning to display when a cell has changed.
Default: "THE CONTENTS OF {num_changed} TEST CELL(S) HAVE CHANGED!\nTh...
--Validator.failed_warning=<Unicode>
Warning to display when a cell fails.
Default: 'VALIDATION FAILED ON {num_failed} CELL(S)! If you submit\nyo...
--Validator.ignore_checksums=<Bool>
Don't complain if cell checksums have changed (if they are locked cells) or
haven't changed (if they are solution cells). Note that this will NOT ignore
changes to cell types.
Default: False
--Validator.indent=<Unicode>
A string containing whitespace that will be used to indent code and errors
Default: ' '
--Validator.invert=<Bool>
Complain when cells pass, rather than fail.
Default: False
--Validator.passed_warning=<Unicode>
Warning to display when a cell passes (when invert=True)
Default: 'NOTEBOOK PASSED ON {num_passed} CELL(S)!\n'
--Validator.type_changed_warning=<Unicode>
Warning to display when a cell's type has changed.
Default: "THE TYPES OF {num_changed} CELL(S) HAVE CHANGED!\nThis might...
--Validator.validate_all=<Bool>
Validate all cells, not just the graded tests cells.
Default: False
--Validator.width=<Int>
Maximum line width for displaying code/errors
Default: 90
Examples
--------
You can run `nbgrader validate` on just a single file, e.g.:
nbgrader validate "Problem 1.ipynb"
Or, you can run it on multiple files using shell globs:
nbgrader validate "Problem Set 1/*.ipynb"
If you want to test instead that none of the tests pass (rather than that
all of the tests pass, which is the default), you can use --invert:
nbgrader validate --invert "Problem 1.ipynb"