nbgrader autogradeΒΆ

Autograde a notebook by running it

Options
-------

Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.

--debug
    set log level to DEBUG (maximize logging output)
--quiet
    set log level to CRITICAL (minimize logging output)
--create
    Deprecated: Create an entry for the student in the database, if one does not already exist. This is now the default.
--no-execute
    Don't execute notebooks and clear output when autograding.
--force
    Overwrite an assignment/submission if it already exists.
-f
    Overwrite an assignment/submission if it already exists.
--course=<Unicode> (CourseDirectory.course_id)
    Default: ''
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
--log-level=<Enum> (Application.log_level)
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--student=<Unicode> (CourseDirectory.student_id)
    Default: '*'
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
--assignment=<Unicode> (CourseDirectory.assignment_id)
    Default: ''
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
--notebook=<Unicode> (CourseDirectory.notebook_id)
    Default: '*'
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
--db=<Unicode> (CourseDirectory.db_url)
    Default: ''
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
--course-dir=<Unicode> (CourseDirectory.root)
    Default: ''
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.

Class parameters
----------------

Parameters are set from command-line arguments of the form:
`--Class.trait=value`. This line is evaluated in Python, so simple expressions
are allowed, e.g.:: `--C.a='range(3)'` For setting C.a=[0,1,2].

AutogradeApp options
--------------------
--AutogradeApp.answer_yes=<Bool>
    Default: False
    Answer yes to any prompts.
--AutogradeApp.config_file=<Unicode>
    Default: ''
    Full path of a config file.
--AutogradeApp.config_file_name=<Unicode>
    Default: ''
    Specify a config file to load.
--AutogradeApp.generate_config=<Bool>
    Default: False
    Generate default config file.
--AutogradeApp.log_datefmt=<Unicode>
    Default: '%Y-%m-%d %H:%M:%S'
    The date format used by logging formatters for %(asctime)s
--AutogradeApp.log_format=<Unicode>
    Default: '[%(name)s]%(highlevel)s %(message)s'
    The Logging format template
--AutogradeApp.log_level=<Enum>
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--AutogradeApp.logfile=<Unicode>
    Default: ''
    Name of the logfile to log to. By default, log output is not written to any
    file.

NbGrader options
----------------
--NbGrader.answer_yes=<Bool>
    Default: False
    Answer yes to any prompts.
--NbGrader.config_file=<Unicode>
    Default: ''
    Full path of a config file.
--NbGrader.config_file_name=<Unicode>
    Default: ''
    Specify a config file to load.
--NbGrader.generate_config=<Bool>
    Default: False
    Generate default config file.
--NbGrader.log_datefmt=<Unicode>
    Default: '%Y-%m-%d %H:%M:%S'
    The date format used by logging formatters for %(asctime)s
--NbGrader.log_format=<Unicode>
    Default: '[%(name)s]%(highlevel)s %(message)s'
    The Logging format template
--NbGrader.log_level=<Enum>
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--NbGrader.logfile=<Unicode>
    Default: ''
    Name of the logfile to log to. By default, log output is not written to any
    file.

CourseDirectory options
-----------------------
--CourseDirectory.assignment_id=<Unicode>
    Default: ''
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
--CourseDirectory.autograded_directory=<Unicode>
    Default: 'autograded'
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
--CourseDirectory.course_id=<Unicode>
    Default: ''
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
--CourseDirectory.db_assignments=<List>
    Default: []
    A list of assignments that will be created in the database. Each item in the
    list should be a dictionary with the following keys:
        - name
        - duedate (optional)
    The values will be stored in the database. Please see the API documentation
    on the `Assignment` database model for details on these fields.
--CourseDirectory.db_students=<List>
    Default: []
    A list of student that will be created in the database. Each item in the
    list should be a dictionary with the following keys:
        - id
        - first_name (optional)
        - last_name (optional)
        - email (optional)
    The values will be stored in the database. Please see the API documentation
    on the `Student` database model for details on these fields.
--CourseDirectory.db_url=<Unicode>
    Default: ''
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
--CourseDirectory.directory_structure=<Unicode>
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
--CourseDirectory.feedback_directory=<Unicode>
    Default: 'feedback'
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
--CourseDirectory.groupshared=<Bool>
    Default: False
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
--CourseDirectory.ignore=<List>
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
--CourseDirectory.include=<List>
    Default: ['*']
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
--CourseDirectory.max_file_size=<Int>
    Default: 100000
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
--CourseDirectory.notebook_id=<Unicode>
    Default: '*'
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
--CourseDirectory.release_directory=<Unicode>
    Default: 'release'
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
--CourseDirectory.root=<Unicode>
    Default: ''
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
--CourseDirectory.source_directory=<Unicode>
    Default: 'source'
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
--CourseDirectory.student_id=<Unicode>
    Default: '*'
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
--CourseDirectory.student_id_exclude=<Unicode>
    Default: ''
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
--CourseDirectory.submitted_directory=<Unicode>
    Default: 'submitted'
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.

BaseConverter options
---------------------
--BaseConverter.force=<Bool>
    Default: False
    Whether to overwrite existing assignments/submissions
--BaseConverter.permissions=<Int>
    Default: 0
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.

Autograde options
-----------------
--Autograde.create_student=<Bool>
    Default: True
    Whether to create the student at runtime if it does not already exist.
--Autograde.exclude_overwriting=<Dict>
    Default: {}
    A dictionary with keys corresponding to assignment names and values being a
    list of filenames (relative to the assignment's source directory) that
    should NOT be overwritten with the source version. This is to allow students
    to e.g. edit a python file and submit it alongside the notebooks in their
    assignment.
--Autograde.force=<Bool>
    Default: False
    Whether to overwrite existing assignments/submissions
--Autograde.permissions=<Int>
    Default: 0
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.

Examples
--------

    Autograde submitted assignments. This takes one argument for the
    assignment id, and then (by default) autogrades assignments from the
    following directory structure:

        submitted/*/{assignment_id}/*.ipynb

    and saves the autograded files to the corresponding directory in:

        autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb

    The student IDs will be created in the database if they don't already
    exist.

    Note that the assignment must also be present in the database. If it is
    not, you should first create it using `nbgrader generate_assignment`. Then, during
    autograding, the cells that contain tests for the students' answers will
    be overwritten with the master version of the tests that is saved in the
    database (this prevents students from modifying the tests in order to
    improve their score).

    To grade all submissions for "Problem Set 1":

        nbgrader autograde "Problem Set 1"

    To grade only the submission by student with ID 'Hacker':

        nbgrader autograde "Problem Set 1" --student Hacker

    To grade only the notebooks that start with '1':

        nbgrader autograde "Problem Set 1" --notebook "1*"

    By default, student submissions are re-executed and their output cleared.
    For long running notebooks, it can be useful to disable this with the
    '--no-execute' flag:

        nbgrader autograde "Problem Set 1" --no-execute

    Note, however, that doing so will not guarantee that students' solutions
    are correct. If you use this flag, you should make sure you manually
    check all solutions. For example, if a student saved their notebook with
    all outputs cleared, then using --no-execute would result in them
    receiving full credit on all autograded problems.