lancet.lancet Package

lancet Package

Lancet consists of three fundamental class types: Argument Specifiers, Command Templates and Launchers. The review_and_launch decorator is a helper utility that helps coordinate the use of these objects.

Argument Specifiers

Argument specifiers are intended to offer a succinct, declarative and composable way of specifying large parameter sets. High-dimensional parameter sets are typical of large-scale scientific models and can make specifying such models and simulations difficult. Using argument specifiers, you can document how the parameters of your model vary across runs without extended lists of arguments or requiring deeply nested loops.

Argument specifiers can be freely intermixed with Python code, simplifying the use of scientific software with a Python interface. They also invoke commandline programs using CommandTemplate objects to build commands and Launcher objects to execute them. Argument specifiers can compose together using Cartesian Products or Concatenation to express huge numbers of argument sets concisely. In this way they can help simplify the management of simulation, analysis and visualisation tools with Python.

Command Templates

When working with external tools, a command template is needed to turn an argument specifier into an executable command. These objects are designed to be customisable with sensible defaults to reduce boilerplate. They may require argument specifiers with a certain structure (certain argument names and value types) to operate correctly.

Launchers

A Launcher is designed to execute commands on a given computational platform, making use of as much concurrency where possible. The default Launcher class executes commands locally while the QLauncher class is designed to launch commands on Sun Grid Engine clusters.

The review_and_launch decorator

This decorator helps codify a pattern of Lancet use that checks for consistency and offers an in-depth review of all settings before launch. The goal is to help users identify mistakes early before consuming computational time and resources.

lancet.__init__.load_ipython_extension(ip)

IPython pretty printing support (optional). To load the extension you may execute the following in IPython:

%load_ext lancet

class lancet.__init__.vcs_metadata(**params)

Bases: param.parameterized.ParameterizedFunction

params(commands=Dict, paths=List, name=String)

Simple utility to capture basic version control information for Git, SVN and Mercurial. Returns a dictionary with the version, latest commit message and the diffs relative to the current working directories. Can be customized by setting the commands dictionary at the class level.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

commands {‘.hg’: ([‘hg’, ‘parents’, ‘–templat... Dict V RW paths [] List (0, None) V RW

Parameter docstrings: =====================

commands: The subprocess command lists to get the version, commit  message and diffs for different version control systems. The  commands are executed if a subdirectory matching the dictionary  key exists paths: List of repositories to generate version control information from.

param List paths (allow_None=False, bounds=(0, None), constant=False, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

List of repositories to generate version control information from.

param Dict commands (allow_None=False, constant=False, default={‘.hg’: ([‘hg’, ‘parents’, ‘–template’, ‘”{rev}:{node}”’], [‘hg’, ‘log’, ‘-l’, ‘1’], [‘hg’, ‘diff’]), ‘.git’: ([‘git’, ‘rev-parse’, ‘HEAD’], [‘git’, ‘log’, ‘–oneline’, ‘-n’, ‘1’], [‘git’, ‘diff’]), ‘.svn’: ([‘svnversion’], [‘svn’, ‘log’, ‘-l’, ‘1’, ‘-q’], [‘svn’, ‘diff’])}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The subprocess command lists to get the version, commit message and diffs for different version control systems. The commands are executed if a subdirectory matching the dictionary key exists

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf51a4c8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf51a3c0>

inspect_value = <functools.partial object at 0x2b06cf51a418>

instance = <functools.partial object at 0x2b06cf51a470>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix='n ', unknown_value='<?>', qualify=False, separator='')

Same as Parameterized.pprint, except that X.classname(Y is replaced with X.classname.instance(Y

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Same as Parameterized.script_repr, except that X.classname(Y is replaced with X.classname.instance(Y

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf51a520>

set_param = <functools.partial object at 0x2b06cf51a628>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

core Module

class lancet.core.Args(specs=None, fp_precision=None, **params)

Bases: lancet.core.Arguments

params(specs=List, fp_precision=Integer, name=String)

An Arguments class that supports statically specified or precomputed argument sets. It may be used directly to specify argument values but also forms the base class for a family of more specific static Argument classes. Each subclass is less flexible and general but allows arguments to be easily and succinctly specified. For instance, the Range subclass allows parameter ranges to be easily declared.

The constructor of Args accepts argument definitions in two different formats. The keyword format allows constant arguments to be specified directly and easily. For instance:

>>> v1 = Args(a=2, b=3)
>>> v1
Args(fp_precision=4,a=2,b=3)

The alternative input format takes an explicit list of the argument specifications:

>>> v2 = Args([{'a':2, 'b':3}]) # Equivalent behaviour to above
>>> v1.specs  == v2.specs
True

This latter format is completely flexible and general, allowing any arbitrary list of arguments to be specified as desired. This is not generally recommended however as the structure of a parameter space is often expressed more clearly by composing together simpler, more succinct Args objects with the CartesianProduct (*) or Concatenation (+) operators.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

fp_precision 4 Integer C RW specs [] List (0, None) C RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8ba680>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba998>

inspect_value = <functools.partial object at 0x2b06cf8ba9f0>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8baaa0>

set_param = <functools.partial object at 0x2b06cf8bab50>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.Arguments(**params)

Bases: lancet.core.PrettyPrinted, param.parameterized.Parameterized

params(fp_precision=Integer, name=String)

The abstract, base class that defines the core interface and methods for all members of the Arguments family of classes, including either the simple, static members of Args below, or the sophisticated parameter exploration algorithms subclassing from DynamicArgs defined in dynamic.py.

The Args subclass may be used directly and forms the root of one family of classes that have statically defined or precomputed argument sets (defined below). The second subfamily are the DynamicArgs, designed to allow more sophisticated, online parameter space exploration techniques such as hill climbing, bisection search, genetic algorithms and so on.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

constant_items

Returns the set of constant items as a list of tuples. This allows easy conversion to dictionary format. Note, the items should be supplied in the same key ordering as for constant_keys for consistency.

constant_keys

Returns the list of parameter names whose values are constant as the argument specifier is iterated. Note that the union of constant and varying_keys should partition the entire set of keys in the case where there are no unsortable keys.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8bab50>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba838>

inspect_value = <functools.partial object at 0x2b06cf8ba730>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

next()

Called to get a list of specifications: dictionaries with parameter name keys and string values.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8ba7e0>

set_param = <functools.partial object at 0x2b06cf8baba8>

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

varying_keys

Returns the list of parameters whose values vary as the argument specifier is iterated. Whenever it is possible, keys should be sorted from those slowest to faster varying and sorted alphanumerically within groups that vary at the same rate.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.CartesianProduct(first, second)

Bases: lancet.core.Args

params(first=ClassSelector, second=ClassSelector, specs=List, fp_precision=Integer, name=String)

CartesianProduct is the Cartesian product of two specifiers. The specifier created by the compositon (firsts * second) generates the cartesian produce of the arguments in first followed by the arguments in second. Note that len(first * second) = len(first)*len(second)

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

first None ClassSelector C RW AN fp_precision 4 Integer C RW second None ClassSelector C RW AN specs [] List (0, None) C RW

Parameter docstrings: =====================

first: The first specifier in the Cartesian product. fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. second: The second specifier in the Cartesian product. specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision.

param ClassSelector second (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The second specifier in the Cartesian product.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param ClassSelector first (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The first specifier in the Cartesian product.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8baba8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba940>

inspect_value = <functools.partial object at 0x2b06cf8baa48>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8baaa0>

set_param = <functools.partial object at 0x2b06cf8bac00>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.Concatenate(first, second)

Bases: lancet.core.Args

params(first=ClassSelector, second=ClassSelector, specs=List, fp_precision=Integer, name=String)

Concatenate is the sequential composition of two specifiers. The specifier created by the compositon (firsts + second) generates the arguments in first followed by the arguments in second.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

first None ClassSelector C RW AN fp_precision 4 Integer C RW second None ClassSelector C RW AN specs [] List (0, None) C RW

Parameter docstrings: =====================

first: The first specifier in the concatenation. fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. second: The second specifier in the concatenation. specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision.

param ClassSelector second (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The second specifier in the concatenation.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param ClassSelector first (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The first specifier in the concatenation.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8bac00>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba788>

inspect_value = <functools.partial object at 0x2b06cf8ba628>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8ba9f0>

set_param = <functools.partial object at 0x2b06cf8bac58>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.FileInfo(source, key, filetype, ignore=, []**params)

Bases: lancet.core.Args

params(filetype=ClassSelector, ignore=List, key=String, source=ClassSelector, specs=List, fp_precision=Integer, name=String)

Loads metadata from a set of filenames. For instance, you can load metadata associated with a series of image files given by a FilePattern. Unlike other explicit instances of Args, this object extends the values of an existing Args object. Once you have loaded the metadata, FileInfo allows you to load the file data into a pandas DataFrame or a HoloViews Table.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

filetype None ClassSelector C RW AN fp_precision 4 Integer C RW ignore [] List (0, None) C RW key ‘’ String C RW source None ClassSelector V RW AN specs [] List (0, None) C RW

Parameter docstrings: =====================

filetype: A FileType object to be applied to each file path. fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. ignore: Metadata keys that are to be explicitly ignored.  key: The key used to find the file paths for inspection. source: The argument specifier that supplies the file paths. specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision.

param ClassSelector filetype (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

A FileType object to be applied to each file path.

param List ignore (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

Metadata keys that are to be explicitly ignored.

param ClassSelector source (allow_None=True, constant=False, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The argument specifier that supplies the file paths.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param String key (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The key used to find the file paths for inspection.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8baaa0>

classmethod from_pattern(pattern, filetype=None, key='filename', root=None, ignore=[])

Convenience method to directly chain a pattern processed by FilePattern into a FileInfo instance.

Note that if a default filetype has been set on FileInfo, the filetype argument may be omitted.

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba940>

inspect_value = <functools.partial object at 0x2b06cf8bab50>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

load(val, **kwargs)

Load the file contents into the supplied pandas dataframe or HoloViews Table. This allows a selection to be made over the metadata before loading the file contents (may be slow).

load_dframe(dframe)

Load the file contents into the supplied dataframe using the specified key and filetype.

load_table(table)

Load the file contents into the supplied Table using the specified key and filetype. The input table should have the filenames as values which will be replaced by the loaded data. If data_key is specified, this key will be used to index the loaded data to retrive the specified item.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8bad08>

set_param = <functools.partial object at 0x2b06cf8bad60>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.FilePattern(key, pattern, root=None, **params)

Bases: lancet.core.Args

params(key=String, pattern=String, root=String, specs=List, fp_precision=Integer, name=String)

A FilePattern specifier allows files to be matched and information encoded in filenames to be extracted via an extended form of globbing. This object may be used to specify filename arguments to CommandTemplates when launching jobs but it also very useful for collating files for analysis.

For instance, you can find the absolute filenames of all npz files in a ‘data’ subdirectory (relative to the root) that start with ‘timeseries’ using the pattern ‘data/timeseries*.npz’.

In addition to globbing supported by the glob module, patterns can extract metadata encoded in filenames using a subset of the Python format specification syntax. To illustrate, you can use ‘data/timeseries-{date}.npz’ to record the date strings associated with matched files. Note that a particular named fields can only be used in a particular pattern once.

By default metadata is extracted as strings but format types are supported in the usual manner eg. ‘data/timeseries-{day:d}-{month:d}.npz’ will extract the day and month from the filename as integer values. Only field names and types are recognised with other format specification syntax ignored. Type codes supported: ‘d’, ‘b’, ‘o’, ‘x’, ‘e’,’E’,’f’, ‘F’,’g’, ‘G’, ‘n’ (if ommited, result is a string by default).

Note that ordering is determined via ascending alphanumeric sort and that actual filenames should not include any globbing characters, namely: ‘?’,’*’,’[‘ and ‘]’ (general good practice for filenames anyway).

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

fp_precision 4 Integer C RW key None String C RW AN pattern None String C RW AN root None String C RW AN specs [] List (0, None) C RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. key: The key name given to the matched file path strings. pattern: The pattern files are to be searched against. root: The root directory from which patterns are to be loaded.  The root is set relative to os.getcwd(). specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision.

param String pattern (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The pattern files are to be searched against.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param String key (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The key name given to the matched file path strings.

param String root (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The root directory from which patterns are to be loaded. The root is set relative to os.getcwd().

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod directory(directory, root=None, extension=None, **kwargs)

Load all the files in a given directory selecting only files with the given extension if specified. The given kwargs are passed through to the normal constructor.

fields()

Return the fields specified in the pattern using Python’s formatting mini-language.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8bad08>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8bad60>

inspect_value = <functools.partial object at 0x2b06cf8bac58>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8baba8>

set_param = <functools.partial object at 0x2b06cf8bacb0>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.Identity(**params)

Bases: lancet.core.Arguments

params(fp_precision=Integer, name=String)

The identity element for any Arguments object ‘args’ under the * operator (CartesianProduct) and + operator (Concatenate). The following identities hold:

args is (Identity() * args) args is (args * Identity())

args is (Identity() + args) args is (args + Identity())

Note that the empty Args() object can also fulfill the role of Identity under the addition operator.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW AN

Parameter docstrings: =====================

fp_precision: fp_precision is disabled as Identity() never contains any  arguments.

param Integer fp_precision (allow_None=True, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=-1, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

fp_precision is disabled as Identity() never contains any arguments.

constant_items

Returns the set of constant items as a list of tuples. This allows easy conversion to dictionary format. Note, the items should be supplied in the same key ordering as for constant_keys for consistency.

constant_keys

Returns the list of parameter names whose values are constant as the argument specifier is iterated. Note that the union of constant and varying_keys should partition the entire set of keys in the case where there are no unsortable keys.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8bab50>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8baba8>

inspect_value = <functools.partial object at 0x2b06cf8ba998>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

next()

Called to get a list of specifications: dictionaries with parameter name keys and string values.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8badb8>

set_param = <functools.partial object at 0x2b06cf8baec0>

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

varying_keys

Returns the list of parameters whose values vary as the argument specifier is iterated. Whenever it is possible, keys should be sorted from those slowest to faster varying and sorted alphanumerically within groups that vary at the same rate.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.List(key, values, **params)

Bases: lancet.core.Args

params(key=String, values=List, specs=List, fp_precision=Integer, name=String)

An argument specifier that takes its values from a given list.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

fp_precision 4 Integer C RW key ‘default’ String C RW specs [] List (0, None) C RW values [] List (0, None) C RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. key: The key assigned to the elements of the supplied list. specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision. values: The list values that are to be returned by the specifier

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param List values (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The list values that are to be returned by the specifier

param String key (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=default, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The key assigned to the elements of the supplied list.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8baba8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8bacb0>

inspect_value = <functools.partial object at 0x2b06cf8ba628>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8baf18>

set_param = <functools.partial object at 0x2b06cf8baf70>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.core.Log(log_path, tid_key='tid', **params)

Bases: lancet.core.Args

params(log_path=String, tid_key=String, specs=List, fp_precision=Integer, name=String)

Specifier that loads arguments from a log file in task id (tid) order. This wrapper class allows a concise representation of file logs with the option of adding the task id to the loaded specifications.

For full control over the arguments, you can use this class to create a fully specified Args object as follows:

Args(Log.extract_log(<log_file>).values()),

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

fp_precision 4 Integer C RW log_path None String C RW AN specs [] List (0, None) C RW tid_key ‘tid’ String C RW AN

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. log_path: The relative or absolute path to the log file. If a  relative path is given, the absolute path is computed  relative to os.getcwd(). specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision. tid_key: If not None, the key given to the tid values included  in the loaded specifications. If None, the tid number  is ignored.

param String log_path (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The relative or absolute path to the log file. If a relative path is given, the absolute path is computed relative to os.getcwd().

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param String tid_key (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=tid, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

If not None, the key given to the tid values included in the loaded specifications. If None, the tid number is ignored.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

static extract_log(log_path, dict_type=<type 'dict'>)

Parses the log file generated by a launcher and returns dictionary with tid keys and specification values.

Ordering can be maintained by setting dict_type to the appropriate constructor (i.e. OrderedDict). Keys are converted from unicode to strings for kwarg use.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8bae10>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8bab50>

inspect_value = <functools.partial object at 0x2b06cf8bafc8>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cfb3d050>

set_param = <functools.partial object at 0x2b06cfb3d100>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

static write_log(log_path, data, allow_append=True)

Writes the supplied specifications to the log path. The data may be supplied as either as a an Args or as a list of dictionaries.

By default, specifications will be appropriately appended to an existing log file. This can be disabled by setting allow_append to False.

class lancet.core.PrettyPrinted

Bases: object

A mixin class for generating pretty-printed representations.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

class lancet.core.Range(key, start_value, end_value, steps=2, mapfn=<function identityfn at 0x2b06c9b508c0>, **params)

Bases: lancet.core.Args

params(end_value=Number, key=String, mapfn=Callable, start_value=Number, steps=Integer, specs=List, fp_precision=Integer, name=String)

Range generates an argument from a numerically interpolated range which is linear by default. An optional function can be specified to sample a numeric range with regular intervals.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

end_value None Number C RW AN fp_precision 4 Integer C RW key ‘’ String C RW mapfn <function identityfn at 0x2b06c9b508c... Callable C RW specs [] List (0, None) C RW start_value None Number C RW AN steps 2 Integer (1, None) C RW

Parameter docstrings: =====================

end_value: The ending numeric value of the range (inclusive). fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. key: The key assigned to the values computed over the numeric range. mapfn: The function to be mapped across the linear range. The  identity function is used by by default specs: The static list of specifications (ie. dictionaries) to be  returned by the specifier. Float values are rounded  according to fp_precision. start_value: The starting numeric value of the range. steps: The number of steps to interpolate over. Default is 2  which returns the start and end values without interpolation.

param Callable mapfn (allow_None=False, constant=True, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The function to be mapped across the linear range. The identity function is used by by default

param Number end_value (allow_None=True, bounds=None, constant=True, default=None, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The ending numeric value of the range (inclusive).

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param Integer steps (allow_None=False, bounds=(1, None), constant=True, default=2, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The number of steps to interpolate over. Default is 2 which returns the start and end values without interpolation.

param String key (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The key assigned to the values computed over the numeric range.

param Number start_value (allow_None=True, bounds=None, constant=True, default=None, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The starting numeric value of the range.

param List specs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The static list of specifications (ie. dictionaries) to be returned by the specifier. Float values are rounded according to fp_precision.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf8badb8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf8ba940>

inspect_value = <functools.partial object at 0x2b06cf8baaf8>

lexsort(*order)

The lexical sort order is specified by a list of string arguments. Each string is a key name prefixed by ‘+’ or ‘-‘ for ascending and descending sort respectively. If the key is not found in the operand’s set of varying keys, it is ignored.

linspace(start, stop, n)

Simple replacement for numpy linspace

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf8ba628>

set_param = <functools.partial object at 0x2b06cf8baaa0>

show(exclude=[])

Convenience method to inspect the available argument values in human-readable format. The ordering of keys is determined by how quickly they vary.

The exclude list allows specific keys to be excluded for readability (e.g. to hide long, absolute filenames).

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

lancet.core.set_fp_precision(value)

Function to set the floating precision across lancet.

lancet.core.to_table(args, vdims=[])

Helper function to convet an Args object to a HoloViews Table

dynamic Module

class lancet.dynamic.DynamicArgs(**params)

Bases: lancet.core.Arguments

params(output_extractor=Callable, fp_precision=Integer, name=String)

DynamicArgs are declarative specifications that specify a parameter space via a dynamic algorithmic process instead of using precomputed arguments. Unlike the static Args objects, new arguments can only be generated in response to some feedback from the processes that are executed. This type of dynamic feedback is a common feature for many algorithms such as hill climbing optimization, genetic algorithms, bisection search and other sophisticated optimization and search procedures.

Like the Args objects, a DynamicArgs object is an iterator. On each iteration, one or more argument sets defining a collection of independent jobs are returned (these jobs should be possible to execute concurrently). Between iterations, the output_extractor function is used to extract the necessary information from the standard output streams of the previously executed jobs. This information is used to update the internal state of the DynamicArgs object can then generate more arguments to explore or terminate. All DynamicArgs classes need to declare the expected return format of the output_extractor function.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW output_extractor <function loads at 0x2b06c9b50848> Callable V RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. output_extractor: The function that returns the relevant data from the standard  output stream dumped to file in the streams subdirectory. This  information must be retyrned in a format suitable for updating  the specifier.. By default uses json.loads but pickle.loads  could also be a valid option. The callable must take a string  and return a Python object suitable for updating the specifier.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param Callable output_extractor (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The function that returns the relevant data from the standard output stream dumped to file in the streams subdirectory. This information must be retyrned in a format suitable for updating the specifier.. By default uses json.loads but pickle.loads could also be a valid option. The callable must take a string and return a Python object suitable for updating the specifier.

constant_items

Returns the set of constant items as a list of tuples. This allows easy conversion to dictionary format. Note, the items should be supplied in the same key ordering as for constant_keys for consistency.

constant_keys

Returns the list of parameter names whose values are constant as the argument specifier is iterated. Note that the union of constant and varying_keys should partition the entire set of keys in the case where there are no unsortable keys.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf7a9998>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf7a9cb0>

inspect_value = <functools.partial object at 0x2b06cf7a9c00>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

static output_extractor(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

Deserialize s (a str or unicode instance containing a JSON document) to a Python object.

If s is a str instance and is encoded with an ASCII based encoding other than utf-8 (e.g. latin-1) then an appropriate encoding name must be specified. Encodings that are not ASCII based (such as UCS-2) are not allowed and should be decoded to unicode first.

object_hook is an optional function that will be called with the result of any object literal decode (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN, null, true, false. This can be used to raise an exception if invalid JSON numbers are encountered.

To use a custom JSONDecoder subclass, specify it with the cls kwarg; otherwise JSONDecoder is used.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf7a9d60>

set_param = <functools.partial object at 0x2b06cf7a9e10>

show()

When dynamic, not all argument values may be available.

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

update(tids, info)

Called to update the state of the iterator. This methods receives the set of task ids from the previous set of tasks together with the launch information to allow the output values to be parsed using the output_extractor. This data is then used to determine the next desired point in the parameter space by calling the _update_state method.

varying_keys

Returns the list of parameters whose values vary as the argument specifier is iterated. Whenever it is possible, keys should be sorted from those slowest to faster varying and sorted alphanumerically within groups that vary at the same rate.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.dynamic.DynamicCartesianProduct(first, second)

Bases: lancet.dynamic.DynamicArgs

params(output_extractor=Callable, fp_precision=Integer, name=String) Parameters of ‘DynamicCartesianProduct’ =======================================  Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW output_extractor <function loads at 0x2b06c9b50848> Callable V RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. output_extractor: The function that returns the relevant data from the standard  output stream dumped to file in the streams subdirectory. This  information must be retyrned in a format suitable for updating  the specifier.. By default uses json.loads but pickle.loads  could also be a valid option. The callable must take a string  and return a Python object suitable for updating the specifier.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param Callable output_extractor (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The function that returns the relevant data from the standard output stream dumped to file in the streams subdirectory. This information must be retyrned in a format suitable for updating the specifier.. By default uses json.loads but pickle.loads could also be a valid option. The callable must take a string and return a Python object suitable for updating the specifier.

constant_items

Returns the set of constant items as a list of tuples. This allows easy conversion to dictionary format. Note, the items should be supplied in the same key ordering as for constant_keys for consistency.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf7a99f0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf7a9a48>

inspect_value = <functools.partial object at 0x2b06cf7a9af8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

static output_extractor(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

Deserialize s (a str or unicode instance containing a JSON document) to a Python object.

If s is a str instance and is encoded with an ASCII based encoding other than utf-8 (e.g. latin-1) then an appropriate encoding name must be specified. Encodings that are not ASCII based (such as UCS-2) are not allowed and should be decoded to unicode first.

object_hook is an optional function that will be called with the result of any object literal decode (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN, null, true, false. This can be used to raise an exception if invalid JSON numbers are encountered.

To use a custom JSONDecoder subclass, specify it with the cls kwarg; otherwise JSONDecoder is used.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf7a9b50>

set_param = <functools.partial object at 0x2b06cf7a9ec0>

show()

When dynamic, not all argument values may be available.

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.dynamic.DynamicConcatenate(first, second)

Bases: lancet.dynamic.DynamicArgs

params(output_extractor=Callable, fp_precision=Integer, name=String) Parameters of ‘DynamicConcatenate’ ==================================  Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW output_extractor <function loads at 0x2b06c9b50848> Callable V RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. output_extractor: The function that returns the relevant data from the standard  output stream dumped to file in the streams subdirectory. This  information must be retyrned in a format suitable for updating  the specifier.. By default uses json.loads but pickle.loads  could also be a valid option. The callable must take a string  and return a Python object suitable for updating the specifier.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param Callable output_extractor (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The function that returns the relevant data from the standard output stream dumped to file in the streams subdirectory. This information must be retyrned in a format suitable for updating the specifier.. By default uses json.loads but pickle.loads could also be a valid option. The callable must take a string and return a Python object suitable for updating the specifier.

constant_items

Returns the set of constant items as a list of tuples. This allows easy conversion to dictionary format. Note, the items should be supplied in the same key ordering as for constant_keys for consistency.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf7a9cb0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf7a9c00>

inspect_value = <functools.partial object at 0x2b06cf7a9d08>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

static output_extractor(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

Deserialize s (a str or unicode instance containing a JSON document) to a Python object.

If s is a str instance and is encoded with an ASCII based encoding other than utf-8 (e.g. latin-1) then an appropriate encoding name must be specified. Encodings that are not ASCII based (such as UCS-2) are not allowed and should be decoded to unicode first.

object_hook is an optional function that will be called with the result of any object literal decode (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN, null, true, false. This can be used to raise an exception if invalid JSON numbers are encountered.

To use a custom JSONDecoder subclass, specify it with the cls kwarg; otherwise JSONDecoder is used.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cf7a9d60>

set_param = <functools.partial object at 0x2b06cf7a9e68>

show()

When dynamic, not all argument values may be available.

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the argument specifier. Unlike the repr, a summary does not have to be complete but must supply the most relevant information about the object to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.dynamic.SimpleGradientDescent(key, **params)

Bases: lancet.dynamic.DynamicArgs

params(key=String, max_steps=Integer, start=Number, stepsize=Number, output_extractor=Callable, fp_precision=Integer, name=String)

Very simple gradient descent optimizer designed to illustrate how Dynamic Args may be implemented. This class has been deliberately kept simple to clearly illustrate how Dynamic Args may be implemented. A more practical example would likely probably make use of mature, third party optimization libraries (such as the routines offered in scipy.optimize).

This particular algorithm greedily minimizes an output value via greedy gradient descent. The local parameter space is explored by examining the change in output value when an increment or decrement of ‘stepsize’ is made in the parameter space, centered around the current position. The initial parameter is initialized with the ‘start’ value and the optimization process terminates when either a local minima/maxima has been found or when ‘max_steps’ is reached.

The ‘output_extractor’ function is expected to return a single scalar number to drive the gradient descent algorithm forwards.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

fp_precision 4 Integer C RW key ‘’ String C RW max_steps 100 Integer C RW output_extractor <function loads at 0x2b06c9b50848> Callable V RW start 0.0 Number C RW stepsize 1.0 Number C RW

Parameter docstrings: =====================

fp_precision: The floating point precision to use for floating point  values. Unlike other basic Python types, floats need care  with their representation as you only want to display up to  the precision actually specified. A floating point precision  of 0 casts number to integers before representing them. key: The name of the argument that will be optimized in a greedy fashion. max_steps: Once max_steps is reached, the optimization terminates. output_extractor: The function that returns the relevant data from the standard  output stream dumped to file in the streams subdirectory. This  information must be retyrned in a format suitable for updating  the specifier.. By default uses json.loads but pickle.loads  could also be a valid option. The callable must take a string  and return a Python object suitable for updating the specifier. start: The starting argument value for the gradient ascent or descent stepsize: The size of the steps taken in parameter space.

param Number start (allow_None=False, bounds=None, constant=True, default=0.0, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The starting argument value for the gradient ascent or descent

param Number stepsize (allow_None=False, bounds=None, constant=True, default=1.0, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The size of the steps taken in parameter space.

param Integer fp_precision (allow_None=False, bounds=None, constant=True, default=4, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

The floating point precision to use for floating point values. Unlike other basic Python types, floats need care with their representation as you only want to display up to the precision actually specified. A floating point precision of 0 casts number to integers before representing them.

param String key (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The name of the argument that will be optimized in a greedy fashion.

param Integer max_steps (allow_None=False, bounds=None, constant=True, default=100, inclusive_bounds=(True, True), instantiate=True, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

Once max_steps is reached, the optimization terminates.

param Callable output_extractor (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The function that returns the relevant data from the standard output stream dumped to file in the streams subdirectory. This information must be retyrned in a format suitable for updating the specifier.. By default uses json.loads but pickle.loads could also be a valid option. The callable must take a string and return a Python object suitable for updating the specifier.

copy()

Convenience method to avoid using the specifier without exhausting it.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06cf7a99f0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06cf7a9f70>

inspect_value = <functools.partial object at 0x2b06cfa390a8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

static output_extractor(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

Deserialize s (a str or unicode instance containing a JSON document) to a Python object.

If s is a str instance and is encoded with an ASCII based encoding other than utf-8 (e.g. latin-1) then an appropriate encoding name must be specified. Encodings that are not ASCII based (such as UCS-2) are not allowed and should be decoded to unicode first.

object_hook is an optional function that will be called with the result of any object literal decode (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN, null, true, false. This can be used to raise an exception if invalid JSON numbers are encountered.

To use a custom JSONDecoder subclass, specify it with the cls kwarg; otherwise JSONDecoder is used.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06cfa39520>

set_param = <functools.partial object at 0x2b06cfa39578>

show()

When dynamic, not all argument values may be available.

classmethod spec_formatter(spec)

Formats the elements of an argument set appropriately

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

update(tids, info)

Called to update the state of the iterator. This methods receives the set of task ids from the previous set of tasks together with the launch information to allow the output values to be parsed using the output_extractor. This data is then used to determine the next desired point in the parameter space by calling the _update_state method.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

filetypes Module

class lancet.filetypes.CustomFile(data_fn=None, metadata_fn=None, **params)

Bases: lancet.filetypes.FileType

params(data_fn=Callable, metadata_fn=Callable, data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

A customizable FileType that takes two functions as input and maps them to the loading interface for all FileTypes.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_fn None Callable V RW AN data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW metadata_fn None Callable V RW AN

Parameter docstrings: =====================

data_fn: A callable that takes a filename and returns a dictionary of  data values data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only. metadata_fn: A callable that takes a filename and returns a dictionary of  metadata values

param Callable data_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A callable that takes a filename and returns a dictionary of data values

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param Callable metadata_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A callable that takes a filename and returns a dictionary of metadata values

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020a9f0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020ab50>

inspect_value = <functools.partial object at 0x2b06d020ac00>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020ac58>

set_param = <functools.partial object at 0x2b06d020aba8>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.FileOption(first, second, **params)

Bases: lancet.filetypes.FileType

params(first=ClassSelector, second=ClassSelector, data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

Allows a FileType out of a pair of FileTypes to handle a given file. For instance, given a mixed list of .png image filenames and .npz Numpy filenames, the ImageFile() | NumpyFile() object an handle either type of filename appropriately.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [] List (0, None) C RW first None ClassSelector V RW AN hash_suffix True Boolean (0, 1) V RW second None ClassSelector V RW AN

Parameter docstrings: =====================

data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. first: The first potential FileType to handle a given filename. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only. second: The second potential FileType to handle a given filename.

param ClassSelector second (allow_None=True, constant=False, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The second potential FileType to handle a given filename.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param ClassSelector first (allow_None=True, constant=False, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The first potential FileType to handle a given filename.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020aba8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020a998>

inspect_value = <functools.partial object at 0x2b06d020a9f0>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020a940>

set_param = <functools.partial object at 0x2b06d020aa48>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.FileType(**params)

Bases: lancet.core.PrettyPrinted, param.parameterized.Parameterized

params(data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

The base class for all supported file types in Lancet. This class is designed to be simple and easily extensible to support new files and has only three essential methods: ‘save’, ‘data’ and ‘metadata’).

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW

Parameter docstrings: =====================

data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

data(filename)

Data returned as a dictionary.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020ac00>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020ab50>

inspect_value = <functools.partial object at 0x2b06d020aba8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

metadata(filename)

The metadata returned as a dictionary.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

save(filename, metadata={}, **data)

The implementation in the base class simply checks there is no clash between the metadata and data keys.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020ac58>

set_param = <functools.partial object at 0x2b06d020adb8>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.HVZFile(data_fn=None, metadata_fn=None, **params)

Bases: lancet.filetypes.CustomFile

params(data_fn=Callable, metadata_fn=Callable, data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

FileType supporting the .hvz file format of the the HoloViews library (http://ioam.github.io/holoviews).

Equivalent to the following CustomFile:

CustomFile(metadata_fn=lambda f: Unpickler.key(f),

data_fn = lambda f: {e: Unpickler.load(f, [e])

for e in Unpickler.entries(f)})

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_fn <function hvz_data_fn at 0x2b06ccb182... Callable V RW data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW metadata_fn <function hvz_metadata_fn at 0x2b06cc... Callable V RW

Parameter docstrings: =====================

data_fn: By default loads all the entries in the .hvz file using  Unpickler.load and returns them as a dictionary. data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only. metadata_fn: Returns the key stored in the .hvz file as metadata using the  Unpickler.key method.

param Callable data_fn (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

By default loads all the entries in the .hvz file using Unpickler.load and returns them as a dictionary.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param Callable metadata_fn (allow_None=False, constant=False, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Returns the key stored in the .hvz file as metadata using the Unpickler.key method.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020ac00>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020a998>

inspect_value = <functools.partial object at 0x2b06d020adb8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020a940>

set_param = <functools.partial object at 0x2b06d020ad08>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.ImageFile(**params)

Bases: lancet.filetypes.FileType

params(data_mode=ObjectSelector, image_info=Dict, data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

Image support - requires PIL or Pillow.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_key ‘images’ String V RW data_mode ‘RGBA’ ObjectSelector V RW directory ‘.’ String V RW AN extensions [‘.png’, ‘.jpg’] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW image_info {‘format’: ‘format’, ‘mode’: ‘mode’, ... Dict V RW

Parameter docstrings: =====================

data_key: The name (key) given to the loaded image data. data_mode: Sets the mode of the mode of the Image object. Palette  mode’P is not supported directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only. image_info: Dictionary of the metadata to load. Each key is the  name given to the metadata item and each value is the PIL  Image attribute to return.

param ObjectSelector data_mode (allow_None=None, check_on_set=True, compute_default_fn=None, constant=False, default=RGBA, instantiate=False, objects=[‘L’, ‘RGB’, ‘RGBA’, ‘I’, ‘F’], pickle_default_value=True, precedence=None, readonly=False)

Sets the mode of the mode of the Image object. Palette mode’P is not supported

param Dict image_info (allow_None=False, constant=False, default={‘format’: ‘format’, ‘mode’: ‘mode’, ‘size’: ‘size’}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Dictionary of the metadata to load. Each key is the name given to the metadata item and each value is the PIL Image attribute to return.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[‘.png’, ‘.jpg’], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=images, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the loaded image data.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020ad08>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020ab50>

inspect_value = <functools.partial object at 0x2b06d020ac00>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

save(filename, imdata, **data)

Data may be either a PIL Image object or a Numpy array.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020ac58>

set_param = <functools.partial object at 0x2b06d020a9f0>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.JSONFile(**params)

Bases: lancet.filetypes.FileType

params(data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

It is assumed you won’t store very large volumes of data as JSON. For this reason, the contents of JSON files are loaded as metadata.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [‘.json’] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW

Parameter docstrings: =====================

data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[‘.json’], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020a9f0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020a998>

inspect_value = <functools.partial object at 0x2b06d020adb8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020a940>

set_param = <functools.partial object at 0x2b06d020ad60>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.MatplotlibFile(**params)

Bases: lancet.filetypes.FileType

params(data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

Since version 1.0, Matplotlib figures support pickling. An mpkl file is simply a pickled matplotlib figure.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [‘.mpkl’] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW

Parameter docstrings: =====================

data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[‘.mpkl’], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020ab50>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020ad60>

inspect_value = <functools.partial object at 0x2b06d020acb0>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020aba8>

set_param = <functools.partial object at 0x2b06d020aa48>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.filetypes.NumpyFile(**params)

Bases: lancet.filetypes.FileType

params(compress=Boolean, data_key=String, directory=String, extensions=List, hash_suffix=Boolean, name=String)

An npz file is the standard way to save Numpy arrays. This is a highly flexible FileType that supports most native Python objects including Numpy arrays.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

compress True Boolean (0, 1) V RW data_key ‘data’ String V RW directory ‘.’ String V RW AN extensions [‘.npz’] List (0, None) C RW hash_suffix True Boolean (0, 1) V RW

Parameter docstrings: =====================

compress: Whether or not the compressed npz format should be used. data_key: The name (key) given to the file contents if the key cannot be  determined from the file itself. directory: Directory in which to load or save the file. Note that this  is a class level parameter only. extensions: The set of supported file extensions. hash_suffix: Whether to ensure the saved filename is unique by adding a  short hash suffix. Note that this is a class level parameter  only.

param Boolean compress (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether or not the compressed npz format should be used.

param List extensions (allow_None=False, bounds=(0, None), constant=True, default=[‘.npz’], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The set of supported file extensions.

param Boolean hash_suffix (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether to ensure the saved filename is unique by adding a short hash suffix. Note that this is a class level parameter only.

param String directory (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Directory in which to load or save the file. Note that this is a class level parameter only.

param String data_key (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=data, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The name (key) given to the file contents if the key cannot be determined from the file itself.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

classmethod file_supported(filename)

Returns a boolean indicating whether the filename has an appropriate extension for this class.

force_new_dynamic_value = <functools.partial object at 0x2b06d020adb8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d020a998>

inspect_value = <functools.partial object at 0x2b06d020aa48>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d020a940>

set_param = <functools.partial object at 0x2b06d020a9f0>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

launch Module

class lancet.launch.Command(executable=None, **params)

Bases: lancet.core.PrettyPrinted, param.parameterized.Parameterized

params(do_format=Boolean, executable=String, name=String)

A command is a way of converting the dictionaries returned by argument specifiers into a particular command. When called with an argument specifier, a command template returns a list of strings corresponding to a subprocess Popen argument list.

__call__(self, spec, tid=None, info={}):

All Commands must be callable. The tid argument is the task id and info is a dictionary of run-time information supplied by the launcher. See the _setup_launch method of Launcher to see details about the launch information supplied.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

do_format True Boolean (0, 1) V RW executable ‘python’ String C RW

Parameter docstrings: =====================

do_format: Set to True to receive input arguments as formatted strings,  False for the raw unformatted objects. executable: The executable that is to be run by this Command. Unless the  executable is a standard command expected on the system path,  this should be an absolute path. By default this invokes  python or the python environment used to invoke the Command  (Topographica for instance).

param String executable (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=python, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The executable that is to be run by this Command. Unless the executable is a standard command expected on the system path, this should be an absolute path. By default this invokes python or the python environment used to invoke the Command (Topographica for instance).

param Boolean do_format (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Set to True to receive input arguments as formatted strings, False for the raw unformatted objects.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

finalize(info)

Optional method that allows a Command to save state before launch. The info argument is supplied by the Launcher.

force_new_dynamic_value = <functools.partial object at 0x2b06d05ecba8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d05ec8e8>

inspect_value = <functools.partial object at 0x2b06d05ecc00>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05eccb0>

set_param = <functools.partial object at 0x2b06d05ecd08>

show(args, file_handle=None, **kwargs)

Write to file_handle if supplied, othewise print output

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the Command configuration. Unlike the repr, a summary does not have to be complete but must supply key information relevant to the user. Must begin by stating the executable.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

verify(args)

Optional, final check that ensures valid arguments have been passed before launch. Allows the constant and varying_keys to be be checked and can inspect the specs attribute if an instance of Args. If invalid, raise an Exception with the appropriate error message, otherwise return None.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.Launcher(batch_name, args, command, **params)

Bases: lancet.core.PrettyPrinted, param.parameterized.Parameterized

params(args=ClassSelector, batch_name=String, command=ClassSelector, description=String, max_concurrency=Integer, metadata=Dict, output_directory=String, reduction_fn=Callable, subdir=List, tag=String, timestamp=NumericTuple, timestamp_format=String, name=String)

A Launcher is constructed using a name, an argument specifier and a command template. It can then launch the corresponding tasks appropriately when invoked.

This default Launcher uses subprocess to launch tasks. It is intended to illustrate the basic design and should be used as a base class for more complex Launchers. In particular all Launchers should retain the same behaviour of writing stdout/stderr to the streams directory, writing a log file and recording launch information.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

args None ClassSelector C RW AN batch_name None String C RW AN command None ClassSelector C RW AN description ‘’ String V RW max_concurrency 2 Integer V RW AN metadata {} Dict V RW output_directory ‘.’ String V RW reduction_fn None Callable V RW AN subdir [] List (0, None) V RW tag ‘’ String V RW timestamp (0, 0, 0, 0, 0, 0, 0, 0, 0) NumericTuple V RW timestamp_format ‘%Y-%m-%d_%H%M’ String V RW AN

Parameter docstrings: =====================

args: The specifier used to generate the varying parameters for the tasks. batch_name: A unique identifier for the current batch command: The command template used to generate the commands for the current tasks. description: A short description of the purpose of the current set of tasks. max_concurrency: Concurrency limit to impose on the launch. As the current class  uses subprocess locally, multiple processes are possible at  once. Set to None for no limit (eg. for clusters) metadata: Metadata information to add to the info file. output_directory: The output directory - the directory that will contain all  the root directories for the individual launches. reduction_fn: A callable that will be invoked when the Launcher has completed  all tasks. For example, this could inform the user of completion  (eg. send an e-mail) among other possibilities. subdir: A list of subdirectory names that allows custom organization  within the output directory before the root directory. tag: A very short, identifiable human-readable string that  meaningfully describes the batch to be executed. Should not  include spaces as it may be used in filenames. timestamp: Optional override of timestamp (default timestamp set on launch  call) in Python struct_time 9-tuple format. Useful when you  need to have a known root_directory path (see root_directory  documentation) before launch. For example, you should store  state related to analysis (eg. pickles) in the same location as  everything else. timestamp_format: The timestamp format for the root directories in python datetime  format. If None, the timestamp is omitted from root directory  name.

param Callable reduction_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A callable that will be invoked when the Launcher has completed all tasks. For example, this could inform the user of completion (eg. send an e-mail) among other possibilities.

param NumericTuple timestamp (allow_None=False, constant=False, default=(0, 0, 0, 0, 0, 0, 0, 0, 0), instantiate=False, length=9, pickle_default_value=True, precedence=None, readonly=False)

Optional override of timestamp (default timestamp set on launch call) in Python struct_time 9-tuple format. Useful when you need to have a known root_directory path (see root_directory documentation) before launch. For example, you should store state related to analysis (eg. pickles) in the same location as everything else.

param ClassSelector args (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The specifier used to generate the varying parameters for the tasks.

param String output_directory (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The output directory - the directory that will contain all the root directories for the individual launches.

param String tag (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A very short, identifiable human-readable string that meaningfully describes the batch to be executed. Should not include spaces as it may be used in filenames.

param ClassSelector command (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The command template used to generate the commands for the current tasks.

param List subdir (allow_None=False, bounds=(0, None), constant=False, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A list of subdirectory names that allows custom organization within the output directory before the root directory.

param String batch_name (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A unique identifier for the current batch

param Integer max_concurrency (allow_None=True, bounds=None, constant=False, default=2, inclusive_bounds=(True, True), instantiate=False, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

Concurrency limit to impose on the launch. As the current class uses subprocess locally, multiple processes are possible at once. Set to None for no limit (eg. for clusters)

param Dict metadata (allow_None=False, constant=False, default={}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Metadata information to add to the info file.

param String timestamp_format (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=%Y-%m-%d_%H%M, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The timestamp format for the root directories in python datetime format. If None, the timestamp is omitted from root directory name.

param String description (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A short description of the purpose of the current set of tasks.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06d05ec998>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_root_directory(timestamp=None)

A helper method that supplies the root directory name given a timestamp.

get_value_generator = <functools.partial object at 0x2b06d05ecc00>

inspect_value = <functools.partial object at 0x2b06d05eca48>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05ecaa0>

set_param = <functools.partial object at 0x2b06d05ecd60>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the Launcher configuration. Unlike the repr, a summary does not have to be complete but must supply key information relevant to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.Output(output_dir, **params)

Bases: param.parameterized.Parameterized

params(expansions=Dict, output_dir=String, name=String) A convenience class to help collect the generated outputs from an

invocation of a Launcher (which is called a “launch”).

Given an output directory, the object reads the log file, info file and provides easy access to the output streams for each of the launches.

The Launch information for each launch is stored in a namedtuple called LaunchInfo which has the following fields:

'timestamp', 'path', 'tids', 'specs', 'stdout', 'stderr', 'log', 'info'

If there are any expansions specified (see the ShellCommand’s expansions parameter), those are also added to the named tuple.

Here is an example of using the class:

>> output = Output('output')
>> len(output)
2
>> output[-1]._fields # the fields of the namedtuple.
('timestamp', 'path', 'tids', 'specs', 'stdout', 'stderr', 'log', 'info')

>> output[-1].path # the path of the last run.
u'/tmp/output/2015-06-21_0325-prime_quintuplet'

>> len(output[-1].specs) # spec the arguments for each case.
16
>> output[-1].specs[-1]
{'integer': 115}
>> len(output[1].stdout) # the path to the stdout for each case.
16
>> open(output[1].stdout[-1]).read()
'109: 109

‘

One can iterate over the LaunchInfo for the launches like so:

>> for li in output:
..     print(li.path)
..
/tmp/output/2015-06-21_0315-prime_quintuplet
/tmp/output/2015-06-21_0325-prime_quintuplet

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Mode 

expansions {} Dict C RW output_dir ‘.’ String V RW

Parameter docstrings: =====================

expansions: Perform expansions (analogous to a ShellCommand) given a callable.  Allows extension of the specification that supports callables that  expand to valid argument values. The callable must have the signature  (spec, info, tid). A typical usage for a function value is to build a  valid output filename given the context.    See the ShellCommand.RootDirectory, ShellCommand.LongFilename and  ShellCommand.Expand. output_dir: The output directory - the directory that will contain all  the root directories for the individual launches.

param Dict expansions (allow_None=False, constant=True, default={}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Perform expansions (analogous to a ShellCommand) given a callable. Allows extension of the specification that supports callables that expand to valid argument values. The callable must have the signature (spec, info, tid). A typical usage for a function value is to build a valid output filename given the context. See the ShellCommand.RootDirectory, ShellCommand.LongFilename and ShellCommand.Expand.

param String output_dir (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The output directory - the directory that will contain all the root directories for the individual launches.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06d05ecaf8>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d05ecc58>

inspect_value = <functools.partial object at 0x2b06d05ec8e8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05eca48>

set_param = <functools.partial object at 0x2b06d05ece10>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

update()

Update the launch information – use if additional launches were made.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.QLauncher(batch_name, args, command, **params)

Bases: lancet.launch.Launcher

params(qsub_flag_options=Dict, qsub_switches=List, args=ClassSelector, batch_name=String, command=ClassSelector, description=String, max_concurrency=Integer, metadata=Dict, output_directory=String, reduction_fn=Callable, subdir=List, tag=String, timestamp=NumericTuple, timestamp_format=String, name=String)

Launcher that operates with Grid Engine using default arguments chosen to be suitable for a typical cluster (tested on the Edinburgh University Eddie cluster).

One of the main features of this class is that it is non-blocking - it alway exits shortly after invoking qsub. This means that the script is not left running or waiting for long periods of time.

By convention the standard output and error streams go to the corresponding folders in the ‘streams’ subfolder of the root directory - any -o or -e qsub options will be overridden. The job name (the -N flag) is specified automatically and any user value will be ignored.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

args None ClassSelector C RW AN batch_name None String C RW AN command None ClassSelector C RW AN description ‘’ String V RW max_concurrency 2 Integer V RW AN metadata {} Dict V RW output_directory ‘.’ String V RW qsub_flag_options {‘-b’: ‘y’} Dict V RW qsub_switches [‘-V’, ‘-cwd’] List (0, None) V RW reduction_fn None Callable V RW AN subdir [] List (0, None) V RW tag ‘’ String V RW timestamp (0, 0, 0, 0, 0, 0, 0, 0, 0) NumericTuple V RW timestamp_format ‘%Y-%m-%d_%H%M’ String V RW AN

Parameter docstrings: =====================

args: The specifier used to generate the varying parameters for the tasks. batch_name: A unique identifier for the current batch command: The command template used to generate the commands for the current tasks. description: A short description of the purpose of the current set of tasks. max_concurrency: Concurrency limit to impose on the launch. As the current class  uses subprocess locally, multiple processes are possible at  once. Set to None for no limit (eg. for clusters) metadata: Metadata information to add to the info file. output_directory: The output directory - the directory that will contain all  the root directories for the individual launches. qsub_flag_options: Specifies qsub flags and their corresponding options as a  dictionary. Valid values may be strings or lists of string. If  a plain Python dictionary is used, the keys arealphanumerically  sorted, otherwise the dictionary is assumed to be an  OrderedDict (Python 2.7+ or param.external.OrderedDict) and the  key ordering will be preserved.    By default the -b (binary) flag is set to ‘y’ to allow binaries  to be directly invoked. Note that the ‘-‘ is added to the key  if missing (to make into a valid flag) so you can specify using  keywords in the dict constructor: ie. using  qsub_flag_options=dict(key1=value1, key2=value2, ....) qsub_switches: Specifies the qsub switches (flags without arguments) as a list  of strings. By default the -V switch is used to exports all  environment variables in the host environment to the batch job. reduction_fn: A callable that will be invoked when the Launcher has completed  all tasks. For example, this could inform the user of completion  (eg. send an e-mail) among other possibilities. subdir: A list of subdirectory names that allows custom organization  within the output directory before the root directory. tag: A very short, identifiable human-readable string that  meaningfully describes the batch to be executed. Should not  include spaces as it may be used in filenames. timestamp: Optional override of timestamp (default timestamp set on launch  call) in Python struct_time 9-tuple format. Useful when you  need to have a known root_directory path (see root_directory  documentation) before launch. For example, you should store  state related to analysis (eg. pickles) in the same location as  everything else. timestamp_format: The timestamp format for the root directories in python datetime  format. If None, the timestamp is omitted from root directory  name.

param Callable reduction_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A callable that will be invoked when the Launcher has completed all tasks. For example, this could inform the user of completion (eg. send an e-mail) among other possibilities.

param NumericTuple timestamp (allow_None=False, constant=False, default=(0, 0, 0, 0, 0, 0, 0, 0, 0), instantiate=False, length=9, pickle_default_value=True, precedence=None, readonly=False)

Optional override of timestamp (default timestamp set on launch call) in Python struct_time 9-tuple format. Useful when you need to have a known root_directory path (see root_directory documentation) before launch. For example, you should store state related to analysis (eg. pickles) in the same location as everything else.

param ClassSelector args (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The specifier used to generate the varying parameters for the tasks.

param List qsub_switches (allow_None=False, bounds=(0, None), constant=False, default=[‘-V’, ‘-cwd’], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

Specifies the qsub switches (flags without arguments) as a list of strings. By default the -V switch is used to exports all environment variables in the host environment to the batch job.

param String output_directory (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The output directory - the directory that will contain all the root directories for the individual launches.

param Dict qsub_flag_options (allow_None=False, constant=False, default={‘-b’: ‘y’}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Specifies qsub flags and their corresponding options as a dictionary. Valid values may be strings or lists of string. If a plain Python dictionary is used, the keys arealphanumerically sorted, otherwise the dictionary is assumed to be an OrderedDict (Python 2.7+ or param.external.OrderedDict) and the key ordering will be preserved. By default the -b (binary) flag is set to ‘y’ to allow binaries to be directly invoked. Note that the ‘-‘ is added to the key if missing (to make into a valid flag) so you can specify using keywords in the dict constructor: ie. using qsub_flag_options=dict(key1=value1, key2=value2, ....)

param String tag (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A very short, identifiable human-readable string that meaningfully describes the batch to be executed. Should not include spaces as it may be used in filenames.

param ClassSelector command (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The command template used to generate the commands for the current tasks.

param List subdir (allow_None=False, bounds=(0, None), constant=False, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A list of subdirectory names that allows custom organization within the output directory before the root directory.

param String batch_name (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A unique identifier for the current batch

param Integer max_concurrency (allow_None=True, bounds=None, constant=False, default=2, inclusive_bounds=(True, True), instantiate=False, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

Concurrency limit to impose on the launch. As the current class uses subprocess locally, multiple processes are possible at once. Set to None for no limit (eg. for clusters)

param Dict metadata (allow_None=False, constant=False, default={}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Metadata information to add to the info file.

param String timestamp_format (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=%Y-%m-%d_%H%M, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The timestamp format for the root directories in python datetime format. If None, the timestamp is omitted from root directory name.

param String description (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A short description of the purpose of the current set of tasks.

collate_and_launch()

Method that collates the previous jobs and launches the next block of concurrent jobs when using DynamicArgs. This method is invoked on initial launch and then subsequently via a commandline call (to Python via qsub) to collate the previously run jobs and launch the next block of jobs.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06d05ecc58>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_root_directory(timestamp=None)

A helper method that supplies the root directory name given a timestamp.

get_value_generator = <functools.partial object at 0x2b06d05ecd08>

inspect_value = <functools.partial object at 0x2b06d05ecd60>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

qdel_batch()

Runs qdel command to remove all remaining queued jobs using the <batch_name>* pattern . Necessary when StopIteration is raised with scheduled jobs left on the queue. Returns exit-code of qdel.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05ecaa0>

set_param = <functools.partial object at 0x2b06d05ece68>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the Launcher configuration. Unlike the repr, a summary does not have to be complete but must supply key information relevant to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.ScriptLauncher(batch_name, args, command, **params)

Bases: lancet.launch.Launcher

params(json_name=String, script_path=String, args=ClassSelector, batch_name=String, command=ClassSelector, description=String, max_concurrency=Integer, metadata=Dict, output_directory=String, reduction_fn=Callable, subdir=List, tag=String, timestamp=NumericTuple, timestamp_format=String, name=String)

Script-based launcher. Calls a script with a path to a JSON file containing process group job options. This easily supports more environment-specific job-submission schemes without having to create a new Launcher every time.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

args None ClassSelector C RW AN batch_name None String C RW AN command None ClassSelector C RW AN description ‘’ String V RW json_name ‘processes_%s.json’ String V RW max_concurrency 2 Integer V RW AN metadata {} Dict V RW output_directory ‘.’ String V RW reduction_fn None Callable V RW AN script_path ‘/var/lib/buildbot/slaves/lancet_docs... String V RW subdir [] List (0, None) V RW tag ‘’ String V RW timestamp (0, 0, 0, 0, 0, 0, 0, 0, 0) NumericTuple V RW timestamp_format ‘%Y-%m-%d_%H%M’ String V RW AN

Parameter docstrings: =====================

args: The specifier used to generate the varying parameters for the tasks. batch_name: A unique identifier for the current batch command: The command template used to generate the commands for the current tasks. description: A short description of the purpose of the current set of tasks. json_name: Name of the JSON file output per process group. max_concurrency: Concurrency limit to impose on the launch. As the current class  uses subprocess locally, multiple processes are possible at  once. Set to None for no limit (eg. for clusters) metadata: Metadata information to add to the info file. output_directory: The output directory - the directory that will contain all  the root directories for the individual launches. reduction_fn: A callable that will be invoked when the Launcher has completed  all tasks. For example, this could inform the user of completion  (eg. send an e-mail) among other possibilities. script_path: Path to script which is called for every group, with JSON file,  batch_name, number of commands for this group and max_concurrency as  arguments. subdir: A list of subdirectory names that allows custom organization  within the output directory before the root directory. tag: A very short, identifiable human-readable string that  meaningfully describes the batch to be executed. Should not  include spaces as it may be used in filenames. timestamp: Optional override of timestamp (default timestamp set on launch  call) in Python struct_time 9-tuple format. Useful when you  need to have a known root_directory path (see root_directory  documentation) before launch. For example, you should store  state related to analysis (eg. pickles) in the same location as  everything else. timestamp_format: The timestamp format for the root directories in python datetime  format. If None, the timestamp is omitted from root directory  name.

param String json_name (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=processes_%s.json, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Name of the JSON file output per process group.

param Callable reduction_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A callable that will be invoked when the Launcher has completed all tasks. For example, this could inform the user of completion (eg. send an e-mail) among other possibilities.

param NumericTuple timestamp (allow_None=False, constant=False, default=(0, 0, 0, 0, 0, 0, 0, 0, 0), instantiate=False, length=9, pickle_default_value=True, precedence=None, readonly=False)

Optional override of timestamp (default timestamp set on launch call) in Python struct_time 9-tuple format. Useful when you need to have a known root_directory path (see root_directory documentation) before launch. For example, you should store state related to analysis (eg. pickles) in the same location as everything else.

param ClassSelector args (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The specifier used to generate the varying parameters for the tasks.

param String output_directory (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The output directory - the directory that will contain all the root directories for the individual launches.

param String tag (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A very short, identifiable human-readable string that meaningfully describes the batch to be executed. Should not include spaces as it may be used in filenames.

param ClassSelector command (allow_None=True, constant=True, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

The command template used to generate the commands for the current tasks.

param List subdir (allow_None=False, bounds=(0, None), constant=False, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A list of subdirectory names that allows custom organization within the output directory before the root directory.

param String batch_name (allow_None=True, basestring=<type ‘basestring’>, constant=True, default=None, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

A unique identifier for the current batch

param Integer max_concurrency (allow_None=True, bounds=None, constant=False, default=2, inclusive_bounds=(True, True), instantiate=False, pickle_default_value=True, precedence=None, readonly=False, time_dependent=False, time_fn=<Time Time00001>)

Concurrency limit to impose on the launch. As the current class uses subprocess locally, multiple processes are possible at once. Set to None for no limit (eg. for clusters)

param String script_path (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=/var/lib/buildbot/slaves/lancet_docs/build/doc/launch_process_group.py, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Path to script which is called for every group, with JSON file, batch_name, number of commands for this group and max_concurrency as arguments.

param Dict metadata (allow_None=False, constant=False, default={}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Metadata information to add to the info file.

param String timestamp_format (allow_None=True, basestring=<type ‘basestring’>, constant=False, default=%Y-%m-%d_%H%M, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The timestamp format for the root directories in python datetime format. If None, the timestamp is omitted from root directory name.

param String description (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

A short description of the purpose of the current set of tasks.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06d05eccb0>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_root_directory(timestamp=None)

A helper method that supplies the root directory name given a timestamp.

get_value_generator = <functools.partial object at 0x2b06d05ece68>

inspect_value = <functools.partial object at 0x2b06d05ecaf8>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05eca48>

set_param = <functools.partial object at 0x2b06d05ec940>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

summary()

A succinct summary of the Launcher configuration. Unlike the repr, a summary does not have to be complete but must supply key information relevant to the user.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.ShellCommand(executable, **params)

Bases: lancet.launch.Command

params(expansions=Dict, long_prefix=String, posargs=List, short_prefix=String, do_format=Boolean, executable=String, name=String)

A generic Command that can be used to invoke shell commands on most operating systems where Python can be run. By default, follows the GNU coding convention for commandline arguments.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

do_format True Boolean (0, 1) V RW executable ‘python’ String C RW expansions {} Dict C RW long_prefix ‘–’ String C RW posargs [] List (0, None) C RW short_prefix ‘-‘ String C RW

Parameter docstrings: =====================

do_format: Set to True to receive input arguments as formatted strings,  False for the raw unformatted objects. executable: The executable that is to be run by this Command. Unless the  executable is a standard command expected on the system path,  this should be an absolute path. By default this invokes  python or the python environment used to invoke the Command  (Topographica for instance). expansions: Allows extension of the specification that supports functions  that expand to valid argument values. If a function is used,  it must have the signature (spec, info, tid). A typical usage  for a function value is to build a valid output filename given  the context.    Three such subclasses are provided:  ‘RootDirectory’, ‘LongFilename’ and ‘Expand’. long_prefix: Although the double dash is a GNU coding convention, some  applications use single dashes for long options. posargs: The list of positional argument keys. Positional arguments are  always supplied at the end of a command in the order given. short_prefix: Although the single dash is a GNU coding convention, the  argument prefix may depend on the applications and/or platform.

param String executable (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=python, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The executable that is to be run by this Command. Unless the executable is a standard command expected on the system path, this should be an absolute path. By default this invokes python or the python environment used to invoke the Command (Topographica for instance).

param Dict expansions (allow_None=False, constant=True, default={}, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

Allows extension of the specification that supports functions that expand to valid argument values. If a function is used, it must have the signature (spec, info, tid). A typical usage for a function value is to build a valid output filename given the context. Three such subclasses are provided: ‘RootDirectory’, ‘LongFilename’ and ‘Expand’.

param Boolean do_format (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Set to True to receive input arguments as formatted strings, False for the raw unformatted objects.

param List posargs (allow_None=False, bounds=(0, None), constant=True, default=[], instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

The list of positional argument keys. Positional arguments are always supplied at the end of a command in the order given.

param String long_prefix (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=–, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

Although the double dash is a GNU coding convention, some applications use single dashes for long options.

param String short_prefix (allow_None=False, basestring=<type ‘basestring’>, constant=True, default=-, instantiate=True, pickle_default_value=True, precedence=None, readonly=False)

Although the single dash is a GNU coding convention, the argument prefix may depend on the applications and/or platform.

class Expand(template)

Bases: object

Takes a new-style format string template and expands it out using the keys in the spec, info and tid.

class ShellCommand.LongFilename(extension, excluding=[])

Bases: object

Generates a long filename based on the input arguments in the root directory with the given extension. Ignores constant items.

class ShellCommand.RootDirectory

Bases: object

Supplies the root_directory to a command.

ShellCommand.debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

ShellCommand.defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

ShellCommand.finalize(info)

Optional method that allows a Command to save state before launch. The info argument is supplied by the Launcher.

ShellCommand.force_new_dynamic_value = <functools.partial object at 0x2b06d05ecd08>

ShellCommand.get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

ShellCommand.get_value_generator = <functools.partial object at 0x2b06d05eccb0>

ShellCommand.inspect_value = <functools.partial object at 0x2b06d05ec940>

ShellCommand.message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod ShellCommand.params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

ShellCommand.pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

ShellCommand.pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod ShellCommand.print_param_defaults()

Print the default values of all cls’s Parameters.

ShellCommand.print_param_values()

Print the values of all this object’s Parameters.

ShellCommand.script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod ShellCommand.set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

ShellCommand.set_dynamic_time_fn = <functools.partial object at 0x2b06d05ecaa0>

ShellCommand.set_param = <functools.partial object at 0x2b06d05ecd60>

ShellCommand.show(args, file_handle=None, **kwargs)

Write to file_handle if supplied, othewise print output

ShellCommand.state_pop()

Restore the most recently saved state.

See state_push() for more details.

ShellCommand.state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

ShellCommand.verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

ShellCommand.verify(args)

Optional, final check that ensures valid arguments have been passed before launch. Allows the constant and varying_keys to be be checked and can inspect the specs attribute if an instance of Args. If invalid, raise an Exception with the appropriate error message, otherwise return None.

ShellCommand.warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.

class lancet.launch.review_and_launch(**params)

Bases: lancet.core.PrettyPrinted, param.parameterized.Parameterized

params(launch_args=ClassSelector, launch_fn=Callable, output_directory=String, review=Boolean, name=String)

A helper decorator that always checks for consistency and can prompt the user for a full review of the launch configuration.

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

launch_args None ClassSelector V RW AN launch_fn None Callable V RW AN output_directory ‘.’ String V RW review True Boolean (0, 1) V RW

Parameter docstrings: =====================

launch_args: An optional argument specifier to  parameterise lancet, allowing multi-launch scripts. For  instance, this may be useful for collecting statistics over  runs that are not deterministic or are affected by a random  input seed. launch_fn: The function that is to be decorated. output_directory: The output directory - the directory that will contain all  the root directories for the individual launches. review: Whether or not to perform a detailed review of the launch.

param ClassSelector launch_args (allow_None=True, constant=False, default=None, instantiate=True, is_instance=True, pickle_default_value=True, precedence=None, readonly=False)

An optional argument specifier to parameterise lancet, allowing multi-launch scripts. For instance, this may be useful for collecting statistics over runs that are not deterministic or are affected by a random input seed.

param Callable launch_fn (allow_None=True, constant=False, default=None, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The function that is to be decorated.

param Boolean review (allow_None=False, bounds=(0, 1), constant=False, default=True, instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

Whether or not to perform a detailed review of the launch.

param String output_directory (allow_None=False, basestring=<type ‘basestring’>, constant=False, default=., instantiate=False, pickle_default_value=True, precedence=None, readonly=False)

The output directory - the directory that will contain all the root directories for the individual launches.

cross_check_launchers(launchers)

Performs consistency checks across all the launchers.

debug(msg, *args, **kw)

Print msg merged with args as a debugging statement.

See Python’s logging module for details of message formatting.

defaults()

Return {parameter_name:parameter.default} for all non-constant Parameters.

Note that a Parameter for which instantiate==True has its default instantiated.

force_new_dynamic_value = <functools.partial object at 0x2b06d05ecd60>

get_param_values(onlychanged=False)

Return a list of name,value pairs for all Parameters of this object.

If onlychanged is True, will only return values that are not equal to the default value.

get_value_generator = <functools.partial object at 0x2b06d05ece10>

input_options(options, prompt='Select option', default=None)

Helper to prompt the user for input on the commandline.

inspect_value = <functools.partial object at 0x2b06d05ece68>

message(msg, *args, **kw)

Print msg merged with args as a message.

See Python’s logging module for details of message formatting.

classmethod params(parameter_name=None)

Return the Parameters of this class as the dictionary {name: parameter_object}

Includes Parameters from this class and its superclasses.

pprint(imports=None, prefix=' ', unknown_value='<?>', qualify=False, separator='')

(Experimental) Pretty printed representation that may be evaluated with eval. See pprint() function for more details.

pprint_args(pos_args, keyword_args, infix_operator=None, extra_params={})

Method to define the positional arguments and keyword order for pretty printing.

classmethod print_param_defaults()

Print the default values of all cls’s Parameters.

print_param_values()

Print the values of all this object’s Parameters.

review_args(obj, show_repr=False, heading='Arguments')

Reviews the given argument specification. Can review the meta-arguments (launch_args) or the arguments themselves.

script_repr(imports=, []prefix=' ')

Variant of __repr__ designed for generating a runnable script.

classmethod set_default(param_name, value)

Set the default value of param_name.

Equivalent to setting param_name on the class.

set_dynamic_time_fn = <functools.partial object at 0x2b06d05eca48>

set_param = <functools.partial object at 0x2b06d05ecdb8>

state_pop()

Restore the most recently saved state.

See state_push() for more details.

state_push()

Save this instance’s state.

For Parameterized instances, this includes the state of dynamically generated values.

Subclasses that maintain short-term state should additionally save and restore that state using state_push() and state_pop().

Generally, this method is used by operations that need to test something without permanently altering the objects’ state.

verbose(msg, *args, **kw)

Print msg merged with args as a verbose message.

See Python’s logging module for details of message formatting.

warning(msg, *args, **kw)

Print msg merged with args as a warning, unless module variable warnings_as_exceptions is True, then raise an Exception containing the arguments.

See Python’s logging module for details of message formatting.