ProSuite package

Data Model

class prosuite.data_model.BaseDataset(name: str, filter_expression: str = '')

Bases: object

The base class for datasets representing tabular data. It is either a Dataset or, in the future, a TransformedDataset

name: str

The name of the dataset. Must be unique with respect to all other datasets of the model.

class prosuite.data_model.Dataset(name: str, model: Model, filter_expression: str = '')

Bases: BaseDataset

A dataset represents data from a table or feature class in a workspace, optionally filtered by an SQL expression.

Parameters:

  • name (str) – table or featureclass name

  • model (class:prosuite.data_model.Model) – The prosuite.data_model.Model this dataset belongs to.

  • filter_expression (str, optional) – A where clause that filters the table. The syntax of the where clause is defined in the document SQLSyntax_en.pdf

property alias_name: str

The alias name of the dataset in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

property id: int

The unique identifier of the dataset in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

model: Model

The dataset’s data model.

class prosuite.data_model.Model(name, catalog_path)

Bases: object

The Model represents the data model in a workspace (file-gdb or enterprise-gdb)

Parameters:

  • name (str) – name of the model

  • catalog_path – The catalog path of the associated workspace.

catalog_path examples:

c:/data.gdb c:/enterprise_gdb.sde

add_dataset(dataset)

Add a dataset to the model.

catalog_path: str

The catalog path of the associated workspace.

datasets

List of datasets in the model.

property default_database_name: str

Gets the default database name for this model as defined in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

property default_database_schema_owner: str

Gets the default database schema owner for this model as defined in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

property element_names_are_qualified: bool

Gets whether Whether model element names (datasets, associations) are qualified in this model. For models not loaded from the data dictionary always fully qualify dataset/table names from RDBMS. This property is read-only.

get_dataset_by_name(name)

Get a dataset from this model by its name.

property id: str

The unique identifier of the model in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

name: str

The unique name of the model.

property wkid: int

The spatial reference well-known ID (WKID) for the model as defined in the data dictionary, or None it has not been loaded from the data dictionary. This property is read-only.

class prosuite.data_model.TransformedDataset(transformer_descriptor: str, name: str = '', filter_expression: str = '')

Bases: BaseDataset

A transformed dataset represents tabular data that has been transformed from other input tables or feature classes, optionally filtered by an SQL expression.

Parameters:

  • transformer_descriptor (str) – The transformer descriptor, i.e. the algorithm used to generate this dataset.

  • name (str) – name of the transformed table or featureclass

  • filter_expression (str, optional) – A where clause that filters the table. The syntax of the where clause is defined in the document SQLSyntax_en.pdf

generate_name()

Generates a technical name using the dataset name(s) and the transformer descriptor. This is the default name of a transformed dataset if it was created by the standard factory method from prosuite.factories.transformers.

generate_name_with_timestamp()

Generates a technical name using the dataset name(s) and the transformer descriptor, and appends a timestamp to ensure uniqueness.

name: str

The unique name of the transformed dataset.

parameters: List[Parameter]

The list of parameters. Typically the parameters are specified in the factory method used to create the transformed dataset (see prosuite.factories.transformers.Transformers) and do not need to be changed through this list.

transformer_descriptor

The transformer descriptor, i.e. the algorithm used to generate this dataset.

Quality

class prosuite.quality.Condition(test_descriptor: str, name: str = '')

Bases: object

A quality condition. Conditions are the primary building blocks of quality specifications. Each Condition is a configuration of a test algorithm for one or more datasets. Conditions must be created with the factory methods from prosuite.factories.quality_conditions

category: str

The name of the category, if this issue is assigned to a category.

description: str

Freely definable description of the Quality Condition. This description can be displayed when viewing issues in the issue navigator, and may contain explanations to the Quality Condition and instructions for correcting the issues.

generate_name()

Generates a technical name using the dataset name(s) and the test descriptor. This is the default name of a condition if it was created by the standard factory method from prosuite.factories.quality_conditions.

property id: int

The unique identifier of the condition in the data dictionary, or None if it has not been loaded from the data dictionary. This property is read-only.

issue_filter_expression: str

The filter expression to be used for multiple issue_filters.

issue_filters: List[IssueFilter]

The list of issue filters for this condition.

issue_type: IssueType

Defines if a failing test returns a warning or an error issue. Quality conditions with Issue Type = Warning are considered “soft” conditions, for which exceptions (“Allowed Issues”) may be defined.

name: str

The unique name of the quality condition.

parameters: List[Parameter]

The list of parameters. Typically the parameters are specified in the factory method used to create the quality condition (see prosuite.factories.quality_conditions) and do not need to be changed through this list.

stop_on_error: bool

Indicates if the occurrence of an error for an object should stop any further testing of the same object. This can be used to prevent further tests on a feature after a serious geometry error (e.g. incorrectly oriented rings) was detected for the feature. The used Test Descriptor provides a standard value for this property. It can optionally be overridden here.

test_descriptor: str

The test descriptor, i.e. the algorithm used to verify this condition.

url: str

Optional URL to a website providing additional information for this Quality Condition. Certain Quality Conditions require more detailed information about the test logic and/or the correction guidelines than the field “Description” can provide. This information can for example be assembled in a wiki, and the URL may be provided here. When viewing issues in the issue navigator, the corresponding web page can be opened. In the HTML verification reports these URLs are used to render the names of the Quality Conditions as links.

class prosuite.quality.DdxSpecification(ddx_id: int, project_short_name: str)

Bases: object

Represents a specification defined in the Data Dictionary (DDX). This requires a server that is configured with a data dictionary connection.

ddx_id: int

ID of the specification in the data dictionary. Find out by using the properties window of the specification in the data dictionary editor.

project_short_name: str

Short name of the project in the data dictionary. This is used to determine the main main workspace of the verification. In case the issues are written to the error datasets in the data model the project short name is used to determine the correct error datasets. See update_issues_in_verified_model

class prosuite.quality.DeleteIssuesInVerifiedModel(value)

Bases: IntEnum

delete_all_issues_in_perimeter = 1
delete_issues_in_perimeter_with_condition = 0
delete_no_issues = 2
class prosuite.quality.IssueFilter(issue_filter_descriptor: str, name: str = '')

Bases: object

An issue filter optionally filters issues found by a quality condition by configurable criteria. Its purpose is the reduction of false positives. Each quality condition can have none, one or multiple issue filters.

Parameters:

  • issue_filter_descriptor (str) – The issue filter descriptor, i.e. the algorithm used to filter issues found by the condition.

  • name (str) – name of the issue filter

description: str

The description of the issue filter.

generate_name()

Generates a technical name using the dataset name(s) and the test descriptor. This is the default name of a condition if it was created by the standard factory method from prosuite.factories.quality_conditions.

property id: int

The unique identifier of the issue filter in the data dictionary, or None if it has not been loaded from the data dictionary. This property is read-only.

issue_filter_descriptor: str

The issue filter descriptor, i.e. the algorithm used to filter the issues.

name: str

The (unique) name of the issue filter.

parameters: List[Parameter]

The list of parameters. Typically the parameters are specified in the factory method used to create the issue filter (see prosuite.factories.issue_filters) and do not need to be changed through this list.

class prosuite.quality.IssueType(value)

Bases: Enum

Error = 1
Warning = 0
class prosuite.quality.Parameter(name: str, value)

Bases: object

A parameter configures a quality condition. Parameters can represent Datasets (dataset parameter) or scalar values (scalar parameters). Parameters have a name and a value.

Dataset parameters: value is of type dataset. The parameter can retrieve the workspace id (model name or model id if it has been retrieved from the data dictionary) and the where clause (filter expression) of the dataset.

Scalar parameters: value is a simple type (number, string, bool).

property dataset

The dataset associated with this parameter (if any).

get_string_value() → str
get_where_clause()
get_workspace_id()
is_dataset_parameter() → bool

Check if the parameter is a dataset parameter. A dataset parameter is a parameter that has a Dataset, a TransformedDataset or a list of datasets as its value. :return: True if the parameter is a dataset parameter, False otherwise.

is_dataset_value() → bool

Check if the parameter value is a Dataset (as opposed to a TransformedDataset or a list). :return: True if the parameter is a dataset parameter, False otherwise.

is_transformed_dataset_value() → bool

Check if the parameter value is a TransformedDataset (as opposed to a Dataset or a list).”

name: str

The parameter name.

property value

The parameter value. Can be a dataset, transformed dataset, list of datasets, or a scalar value.

static value_is_list_of_datasets(value)
class prosuite.quality.Specification(name: str = 'Custom Specification', description: str = '')

Bases: object

Represents a quality specification defined in code. Quality specifications are collections of quality conditions.

Parameters:

  • name (str) – specification name

  • description (str) – specification description

add_condition(condition: Condition)

Adds conditions to the specification

get_conditions() → List[Condition]

Returns the List of conditions

project_short_name: str

Short name of the project in the data dictionary. If the project short name is provided, the datasets registered in the data dictionary are used. Otherwise, the datasets are retrieved using the provided catalog path of the datasets’ model. The project short name is also used to determine the main workspace of the verification. The project short name is also used to determine the correct error datasets in the data model in case the issues are stored in the error datasets of the data model. See update_issues_in_verified_model In case the issues are written to the error datasets in the data model the project short name is used to determine the correct error datasets. See update_issues_in_verified_model

class prosuite.quality.XmlSpecification(specification_file: str, specification_name: str = None, data_source_replacements: List[List[str]] = None)

Bases: object

Represents a specification defined in the xml specification schema. Quality specifications are collections of quality conditions.

Parameters:

  • specification_file (str) – path to the xml specification file

  • specification_name (str) – name of the specification (in the xml file) that should be executed. If not defined in the constructor, it needs to be defined before it is used.

  • data_source_replacements ([[str]]) – a list containing a list with two string elements. These represent a workspace id together with the path to the workspace. Example: [[“TLM_Production”, “C:/temp/user@topgist.sde”]]. More details are provided below the XML verification sample.

static get_specification_names(specification_file: str) → List[str]

Returns a list of the specification names of all specifications in all categories of the xml-specifications doc.

Parameters:

specification_file (str) – path of xml specification file.

Returns:

List of specification names

Return type:

[str]

Verification

Utils

prosuite.utils.append_timestamp_to_basepath(base_path)
prosuite.utils.get_ssl_channel_credentials() → grpc.ChannelCredentials

Creates an ssl channel credentials object for use with an SSL-enabled channel for the authentication with a server that requires TLS/SSL. The appropriate root certificates for server authentication are loaded from the windows certificate store.

Returns:

A ChannelCredentials object

prosuite.utils.get_value_or_default(value: Any, default: Any) → Any

Returns the value if it is not None. Else returns the default value.

prosuite.utils.load_file_as_string(file_name: str) → str
prosuite.utils.load_json_file(json_file_path: str)
prosuite.utils.objectify_xml(path_file) → lxml.objectify.ObjectifiedElement

Try to objectify the xml file and return the objectified element. Return None if it fails.

prosuite.utils.to_bool(value: str | bool | int | None = None, default_value: bool = False) → bool

” treats “true” or “True” or ‘yes’ or ‘Yes’ or 1 or ‘1’ as True (bool). Anything else returns False (bool)

prosuite.utils.to_float(value: str | int | float | None, default_value: float = 0) → float | None
Parameters:

  • value – value that gets converted to float type

  • default_value – value that is returned in case the float conversion fails

Returns:

prosuite.utils.to_int(value: str | int | None, default_value: int = 0) → int | None

Tries to convert the input value to int. If conversion is not possible, returns the default value.

Parameters:

  • value – value that gets converted to int type

  • default_value – value that is returned in case the int conversion fails

Returns:

int representation of the input value or 0

prosuite.utils.to_spatial_reference_xml(sr_id: str)
prosuite.utils.try_get_from_oe(prop: str, object: lxml.objectify.ObjectifiedElement) → str | None
prosuite.utils.try_get_from_str_dict(key: str, dictionary: dict, default_value: int | float | str | None = '') → int | float | str | None

returns the value of the lookup element if it is available in the dict. if the lookup element is not in the dict, the default value is returned.

prosuite.utils.try_get_lxml_attrib(node: lxml.objectify.ObjectifiedElement, attribute_name: str) → str | None

If the attribute exists on the objectified node then return it, else return None

Organisational modules - Subpackages