The ibm-watson-machine-learning (V4) library ¶

The ibm-watson-machine-learning Python library allows you to work with IBM Watson Machine Learning services. You can train, store, and deploy your models, score them using APIs, and finally integrate them with your application development. The ibm-watson-machine-learning library can work with IBM Cloud Pak® for Data as a Service (later referred to as IBM Cloud) V2 instance plans (created after Sept 1 2020) as well as IBM Cloud Pak® for Data version 3.5 and higher. For older versions of IBM Cloud Pak® for Data, use the beta version of the library provided under link.

Note: For IBM Cloud, if you are using Watson Machine Learning Python client V3 or Watson Machine Learning Python client V4 Beta, refer to the documentation for the migration process, to be able to migrate your assets to new instance plans and access new features.

Important: Python 3.6 framework is no longer supported. Python 3.6 is no longer supported. Create new assets or update existing assets with a supported framework. For details, see Managing outdated software specifications or frameworks.

Important: The Watson Machine Learning Python client V3 and Watson Machine Learning Python client V4 Beta are deprecated since Sep 1st, 2020 on cloud and will be discontinued at the end of the migration period. Migrate to IBM Watson Machine Learning Client V4 GA packaged in the name of ibm-watson-machine-learning on cloud.

Contents

The ibm-watson-machine-learning (V4) library
Indices and tables

Installation ¶

The ibm-watson-machine-learning Python library is integrated into the Watson Studio Jupyter notebook. Also, the package is available from pypi. To install it for use with IBM Cloud services, type:

$pip install ibm-watson-machine-learning

API for IBM Cloud ¶

To use Watson Machine learning APIs, create an instance of APIClient with authentication details.

Requirements ¶

To create a Watson Machine Learning service instance, refer to the documentation.

Authentication ¶

Authentication for IBM Cloud

To use Watson Machine learning APIs, create an instance of APIClient with authentication details.

Example of creating the client using apikey:

Note: There is no need to provide instance_id. For the new(V2) instance plans, the instance_id is picked up from the associated space/project. To access your assets from old(V1) instance plans, provide your old V1 instance_id in the wml_credentials and create a client object.

Note: Depending on the region of your provisioned service instance, use one of the following as your url:

Dallas: https://us-south.ml.cloud.ibm.com

London: https://eu-gb.ml.cloud.ibm.com

Frankfurt: https://eu-de.ml.cloud.ibm.com

Tokyo: https://jp-tok.ml.cloud.ibm.com

Note: To determine apikey go to https://cloud.ibm.com/iam/apikeys and generate your apikey.

Example of creating the client using apikey:

from ibm_watson_machine_learning import APIClient

wml_credentials = {
                   "url": "https://us-south.ml.cloud.ibm.com",
                   "apikey":"***********"
                  }

client = APIClient(wml_credentials)

Example of creating the client using token:

from ibm_watson_machine_learning import APIClient

wml_credentials = {
                   "url": "https://us-south.ml.cloud.ibm.com",
                   "token":"***********",
                  }

client = APIClient(wml_credentials)

Example of creating the client using V1 Lite instance plan:

from ibm_watson_machine_learning import APIClient

wml_credentials = {
                   "url": "https://us-south.ml.cloud.ibm.com",
                   "apikey":"**********",
                   "instance_id": "v1 lite instance_id"
                  }

client = APIClient(wml_credentials)

Note: Setting default space/project id is mandatory. For details, refer to client.set.default_space() API in this document.

Supported machine learning frameworks ¶

For the list of supported machine learning frameworks (models) on IBM Cloud, refer to Watson Machine Learning Documentation.

APIs to Migrate V4 Beta and V3 assets to V4 ¶

Migration APIs for v4 GA Cloud to migrate assets from V3 or V4 beta. For details and examples of the migration process, refer to the document.

class metanames.Migrationv4GACloudMetaNames[source]¶

Set of MetaNames for v4ga Cloud migration.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
DESCRIPTION	str	N	`Testing migration`	`string`
OLD_INSTANCE_ID	str	Y	`df40cf1-252f-424b-b52d-5cdd98143aec`	`string`
SPACE_ID	str	N	`3fc54cf1-252f-424b-b52d-5cdd9814987f`	`string`
PROJECT_ID	str	N	`4fc54cf1-252f-424b-b52d-5cdd9814987f`	`string`
MODEL_IDS	list	N	`['afaecb4-254f-689f-4548-9b4298243291']`	`['string']`
FUNCTION_IDS	list	N	`['all']`	`['string']`
EXPERIMENT_IDS	list	N	`['ba2ecb4-4542-689a-2548-ab4232b43291']`	`['string']`
PIPELINE_IDS	list	N	`['4fabcb4-654f-489b-9548-9b4298243292']`	`['string']`
SKIP_MIGRATED_ASSETS	bool	N	`False`
MAPPING	dict	N	`{'dfaecf1-252f-424b-b52d-5cdd98143481': '4fbc211-252f-424b-b52d-5cdd98df310a'}`

Connections ¶

class client.Connections(client)[source]¶

Store and manage your Connections.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.ConnectionMetaNames object>¶: MetaNames for Connection creation.

create(meta_props)[source]¶

Creates a connection. Input to PROPERTIES field examples

MySQL

>>> client.connections.ConfigurationMetaNames.PROPERTIES: {
>>>   "database": "database",
>>>   "password": "password",
>>>   "port": "3306",
>>>   "host": "host url",
>>>   "ssl": "false",
>>>   "username": "username"
>>> }

Google Big query

Method1: Use service account json. The service account json generated can be provided as: input as-is. Provide actual values in json. Example is only indicative to show the fields. Refer to Google big query documents how to generate the service account json

>>> client.connections.ConfigurationMetaNames.PROPERTIES: {
>>>     "type": "service_account",
>>>     "project_id": "project_id",
>>>     "private_key_id": "private_key_id",
>>>     "private_key": "private key contents",
>>>     "client_email": "client_email",
>>>     "client_id": "client_id",
>>>     "auth_uri": "https://accounts.google.com/o/oauth2/auth",
>>>     "token_uri": "https://oauth2.googleapis.com/token",
>>>     "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
>>>     "client_x509_cert_url": "client_x509_cert_url"
>>> }

Method2: Using OAuth Method. Refer to Google big query documents how to generate OAuth token

>>> client.connections.ConfigurationMetaNames.PROPERTIES: {
>>>    "access_token": "access token generated for big query",
>>>    "refresh_token": "refresh token",
>>>    "project_id": "project_id",
>>>    "client_secret": "This is your gmail account password",
>>>   "client_id": "client_id"
>>> }

MS SQL

>>> client.connections.ConfigurationMetaNames.PROPERTIES: {
>>>      "database": "database",
>>>      "password": "password",
>>>      "port": "1433",
>>>      "host": "host",
>>>      "username": "username"
>>> }

Tera data

>>> client.connections.ConfigurationMetaNames.PROPERTIES: {
>>>      "database": "database",
>>>      "password": "password",
>>>      "port": "1433",
>>>      "host": "host",
>>>      "username": "username"
>>> }

Parameters

Important

meta_props: meta data of the connection configuration. To see available meta names use:
```
>>> client.connections.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the stored connection

return type: dict

Example

>>> sqlserver_data_source_type_id = client.connections.get_datasource_type_uid_by_name('sqlserver')
>>> connections_details = client.connections.create({
>>> client.connections.ConfigurationMetaNames.NAME: "sqlserver connection",
>>> client.connections.ConfigurationMetaNames.DESCRIPTION: "connection description",
>>> client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: sqlserver_data_source_type_id,
>>> client.connections.ConfigurationMetaNames.PROPERTIES: { "database": "database",
>>>                                                         "password": "password",
>>>                                                         "port": "1433",
>>>                                                         "host": "host",
>>>                                                         "username": "username"}
>>> })

delete(connection_id)[source]¶

Delete a stored Connection.

Parameters

Important

connection_id: Unique id of the connection to be deleted.

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.connections.delete(connection_id)

get_datasource_type_uid_by_name(name)[source]¶

Get stored datasource types id for the given datasource type name.

Parameters

Important

name: name of datasource type

type: int

Output

Important

This method only prints the id of given datasource type name.

return type: Str

Example

>>> client.connections.get_datasource_type_uid_by_name('cloudobjectstorage')

get_details(connection_id=None)[source]¶

Get connection details for the given unique Connection id. If no connection_id is passed, details for all connections will be returned.

Parameters

Important

connection_id: Unique id of Connection

type: str

Output

Important

returns: Metadata of the stored Connection

return type: dict

Example

>>> connection_details = client.connections.get_details(connection_id)
>>> connection_details = client.connections.get_details()

static get_uid(connection_details)[source]¶

Get Unique Id of stored connection.

Parameters

Important

connection_details: Metadata of the stored connection

type: dict

Output

Important

returns: Unique Id of stored connection

return type: str

Example

>>> connection_uid = client.connection.get_uid(connection_details)

list()[source]¶

List all stored connections.

Output

Important

This method only prints the list of all connections in a table format.

return type: None

Example

>>> client.connections.list()

list_datasource_types()[source]¶

List stored datasource types assets.

Output

Important

This method only prints the list of datasources type in a table format.

return type: None

Example

>>> client.connections.list_datasource_types()

list_uploaded_db_drivers()[source]¶

List uploaded db driver jars. Supported for IBM Cloud Pak for Data only.

Output

Important

This method only prints the uploaded db driver jar names.

return type: Str

Example

>>> client.connections.list_uploaded_db_drivers()

sign_db_driver_url(jar_name: str)[source]¶

Get signed db driver jar url to be used during creating of JDBC generic connection. The jar name passed as argument needs to be uploaded into system first. Supported for IBM Cloud Pak for Data only, version 4.0.4 and above..

Parameters

Important

jar_name: db driver jar name

type: str

Example

>>> client.connections.sign_db_driver_url('db2jcc4.jar')

upload_db_driver(path: str)[source]¶

Upload db driver jar. Supported for IBM Cloud Pak for Data only, version 4.0.4 and above.

Parameters

Important

path: path to db driver jar

type: str

Example

>>> client.connections.upload_db_driver('example/path/db2jcc4.jar')

class metanames.ConnectionMetaNames[source]¶

Set of MetaNames for Spaces Specs.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`my_space`
DESCRIPTION	str	N	`my_description`
DATASOURCE_TYPE	str	Y	`1e3363a5-7ccf-4fff-8022-4850a8024b68`
PROPERTIES	dict	Y	`{'database': 'BLUDB', 'host': 'dashdb-txn-sbox-yp-dal09-04.services.dal.bluemix.net', 'password': 'a1b2c3d4#', 'username': 'usr21370'}`
FLAGS	list	N	`['personal_credentials']`

Data assets ¶

class client.Assets(client)[source]¶

Store and manage your data assets.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.AssetsMetaNames object>¶: MetaNames for Data Assets creation.

create(name, file_path)[source]¶

Creates a data asset and uploads content to it.

Parameters

Important

name: Name to be given to the data asset

type: str
file_path: Path to the content file to be uploaded

type: str

Output

Important

returns: metadata of the stored data asset

return type: dict

Example

>>> asset_details = client.data_assets.create(name="sample_asset",file_path="/path/to/file")

delete(asset_uid)[source]¶

Delete a stored data asset.

Parameters

Important

asset_uid: Unique Id of data asset

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.data_assets.delete(asset_uid)

download(asset_uid, filename)[source]¶

Download and store the content of a data asset.

Parameters

Important

asset_uid: The Unique Id of the data asset to be downloaded

type: str
filename: filename to be used for the downloaded file

type: str

Output

Important

returns: Normalized path to the downloaded asset content

return type: str

Example

>>> client.data_assets.download(asset_uid,"sample_asset.csv")

get_content(asset_uid)[source]¶

Download the content of a data asset.

Parameters

Important

asset_uid: The Unique Id of the data asset to be downloaded

type: str

Output

Important

returns: the asset content

return type: binary

Example

>>> content = client.data_assets.get_content(asset_uid).decode('ascii')

get_details(asset_uid=None)[source]¶

Get data asset details. If no asset_uid is passed, details for all assets will be returned.

Parameters

Important

returns: Unique id of asset

return type: str

Output

Important

asset_details: Metadata of the stored data asset

type: dict

Example

>>> asset_details = client.data_assets.get_details(asset_uid)

static get_href(asset_details)[source]¶

Get url of stored data asset.

Parameters

Important

asset_details: stored data asset details

type: dict

Output

Important

returns: href of stored data asset

return type: str

Example

>>> asset_details = client.data_assets.get_details(asset_uid)
>>> asset_href = client.data_assets.get_href(asset_details)

static get_id(asset_details)[source]¶

Get Unique Id of stored script asset.

Parameters

Important

asset_details: Metadata of the stored script asset

type: dict

Output

Important

returns: Unique Id of stored data asset

return type: str

Example

>>> asset_id = client.data_assets.get_id(asset_details)

static get_uid(asset_details)[source]¶

Get Unique Id of stored data asset. Deprecated!! Use get_id(details) instead

Parameters

Important

asset_details: Metadata of the stored data asset

type: dict

type: dict

Output

Important

returns: Unique Id of stored asset

return type: str

Example

>>> asset_uid = client.data_assets.get_uid(asset_details)

list(limit=None)[source]¶

List stored data assets. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all data assets in a table format.

return type: None

Example

>>> client.data_assets.list()

store(meta_props)[source]¶

Creates a data asset and uploads content to it.

Parameters

Important

meta_props: meta data of the space configuration. To see available meta names use:
```
>>> client.data_assets.ConfigurationMetaNames.get()
```
type: dict

Example

Example for data asset creation for files :

>>> metadata = {
>>>  client.data_assets.ConfigurationMetaNames.NAME: 'my data assets',
>>>  client.data_assets.ConfigurationMetaNames.DESCRIPTION: 'sample description',
>>>  client.data_assets.ConfigurationMetaNames.DATA_CONTENT_NAME: 'sample.csv'
>>> }
>>> asset_details = client.data_assets.store(meta_props=metadata)

Example of data asset creation using connection:

>>> metadata = {
>>>  client.data_assets.ConfigurationMetaNames.NAME: 'my data assets',
>>>  client.data_assets.ConfigurationMetaNames.DESCRIPTION: 'sample description',
>>>  client.data_assets.ConfigurationMetaNames.CONNECTION_ID: '39eaa1ee-9aa4-4651-b8fe-95d3ddae',
>>>  client.data_assets.ConfigurationMetaNames.DATA_CONTENT_NAME: 't1/sample.csv'
>>> }
>>> asset_details = client.data_assets.store(meta_props=metadata)

Example for data asset creation with database sources type connection:

>>> metadata = {
>>>  client.data_assets.ConfigurationMetaNames.NAME: 'my data assets',
>>>  client.data_assets.ConfigurationMetaNames.DESCRIPTION: 'sample description',
>>>  client.data_assets.ConfigurationMetaNames.CONNECTION_ID: '23eaf1ee-96a4-4651-b8fe-95d3dadfe',
>>>  client.data_assets.ConfigurationMetaNames.DATA_CONTENT_NAME: 't1'
>>> }
>>> asset_details = client.data_assets.store(meta_props=metadata)

class metanames.AssetsMetaNames[source]¶

Set of MetaNames for Data Asset Specs.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`my_data_asset`
DATA_CONTENT_NAME	str	Y	`/test/sample.csv`
CONNECTION_ID	str	N	`39eaa1ee-9aa4-4651-b8fe-95d3ddae`
DESCRIPTION	str	N	`my_description`

Deployments ¶

class client.Deployments(client)[source]¶

Deploy and score published artifacts (models and functions).

create(artifact_uid=None, meta_props=None, rev_id=None, **kwargs)[source]¶

Create a deployment from an artifact. As artifact we understand model or function which may be deployed.

Parameters

Important

artifact_uid: Published artifact UID (model or function uid)

type: str
meta_props: metaprops. To see the available list of metanames use:
```
>>> client.deployments.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the created deployment

return type: dict

Example

>>> meta_props = {
>>> wml_client.deployments.ConfigurationMetaNames.NAME: "SAMPLE DEPLOYMENT NAME",
>>> wml_client.deployments.ConfigurationMetaNames.ONLINE: {},
>>> wml_client.deployments.ConfigurationMetaNames.HARDWARE_SPEC : { "id":  "e7ed1d6c-2e89-42d7-aed5-8sb972c1d2b"},
>>> wml_client.deployments.ConfigurationMetaNames.SERVING_NAME : 'sample_deployment'
>>> }
>>> deployment_details = client.deployments.create(artifact_uid, meta_props)

create_job(deployment_id, meta_props, retention=None, transaction_id=None, _asset_id=None)[source]¶

Create an asynchronous deployment job.

Parameters

Important

deployment_id: Unique Id of Deployment

type: str

meta_props: metaprops. To see the available list of metanames use:

>>> client.deployments.ScoringMetaNames.get() or client.deployments.DecisionOptimizationmetaNames.get()

type: dict

retention: How many job days job meta should be retained (optional). Takes integer values >= -1.
Supported only on Cloud.

type:: int

Output

Important

returns: metadata of the created async deployment job

return type: dict

Note

The valid payloads for scoring input are either list of values, pandas or numpy dataframes.

Example

>>> scoring_payload = {wml_client.deployments.ScoringMetaNames.INPUT_DATA: [{'fields': ['GENDER','AGE','MARITAL_STATUS','PROFESSION'], 'values': [['M',23,'Single','Student'],['M',55,'Single','Executive']]}]}
>>> async_job = client.deployments.create_job(deployment_id, scoring_payload)

delete(deployment_uid)[source]¶

Delete deployment.

Parameters

Important

deployment uid: Unique Id of Deployment

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.deployments.delete(deployment_uid)

delete_job(job_uid, hard_delete=False)[source]¶

Cancels a deployment job that is currenlty running. This method is also be used to delete metadata details of the completed or canceled jobs when hard_delete parameter is set to True.

Parameters

Important

job_uid: Unique Id of deployment job which should be canceled

type: str
hard_delete: specify True or False.
True - To delete the completed or canceled job. False - To cancel the currently running deployment job. Default value is False.

type: Boolean

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.deployments.delete_job(job_uid)

download(virtual_deployment_uid, filename=None)[source]¶

Downloads file deployment of specified deployment Id. Currently supported format is Core ML. Supported only for IBM Cloud Pak® for Data 3.0 and below.

Parameters

Important

virtual_deployment_uid: Unique Id of virtual deployment.

type: str
filename: filename of downloaded archive (optional).

type: str

Output

Important

returns: Path to downloaded file.

return type: str

Example

>>> client.deployments.download(virtual_deployment_uid)

get_details(deployment_uid=None, serving_name=None, limit=None, asynchronous=False, get_all=False, _silent=False)[source]¶

Get information about your deployment(s). If deployment_uid is not passed, all deployment details are fetched.

Parameters

Important

deployment_uid: Unique Id of Deployment (optional)

type: str
serving_name: Serving name to filter deployments (optional)

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: metadata of deployment(s)

return type: dict

dict (if deployment_uid is not None) or {“resources”: [dict]} (if deployment_uid is None)

Note

If deployment_uid is not specified, all deployments metadata is fetched (can be filtered with serving_name)

Example

>>> deployment_details = client.deployments.get_details(deployment_uid)
>>> deployment_details = client.deployments.get_details(deployment_uid=deployment_uid)
>>> deployments_details = client.deployments.get_details()
>>> deployments_details = client.deployments.get_details(limit=100)
>>> deployments_details = client.deployments.get_details(limit=100, get_all=True)
>>> deployments_details = []
>>> for entry in client.deployments.get_details(limit=100, asynchronous=True, get_all=True):
>>>    deployments_details.extend(entry)

get_download_url(deployment_details)[source]¶

Get deployment_download_url from deployment details.

Parameters

Important

deployment_details: Created deployment details.

type: dict

Output

Important

returns: Deployment download URL that is used to get file deployment (for example: Core ML).

return type: str

Example

>>> deployment_url = client.deployments.get_download_url(deployment)

static get_href(deployment_details)[source]¶

Get deployment_href from deployment details.

Parameters

Important

deployment_details: Metadata of the deployment.

type: dict

Output

Important

returns: Deployment href that is used to manage the deployment.

return type: str

Example

>>> deployment_href = client.deployments.get_href(deployment)

static get_id(deployment_details)[source]¶

Get deployment id from deployment details.

Parameters

Important

deployment_details: Metadata of the deployment

type: dict

Output

Important

returns: deployment ID that is used to manage the deployment

return type: str

Example

>>> deployment_id = client.deployments.get_id(deployment)

get_job_details(job_uid=None, include=None, limit=None)[source]¶

Get information about your deployment job(s). If deployment job_uid is not passed, all deployment jobs details are fetched.

Parameters

Important

job_uid: Unique Job ID (optional)

type: str
include: fields to be retrieved from ‘decision_optimization’ and ‘scoring’ section mentioned as value(s) (comma separated) as output response fields (optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: metadata of deployment job(s)

return type: dict

dict (if job_uid is not None) or {“resources”: [dict]} (if job_uid is None)

Note

If job_uid is not specified, all deployment jobs metadata associated with the deployment Id is fetched

Example

>>> deployment_details = client.deployments.get_job_details()
>>> deployments_details = client.deployments.get_job_details(job_uid=job_uid)

get_job_href(job_details)[source]¶

Get the href of the deployment job.

Parameters

Important

job_details: metadata of the deployment job

type: dict

Output

Important

returns: href of the deployment job

return type: str

Example

>>> job_details = client.deployments.get_job_details(job_uid=job_uid)
>>> job_status = client.deployments.get_job_href(job_details)

get_job_status(job_id)[source]¶

Get the status of the deployment job.

Parameters

Important

job_id: Unique Id of the deployment job

type: str

Output

Important

returns: status of the deployment job

return type: dict

Example

>>> job_status = client.deployments.get_job_status(job_uid)

get_job_uid(job_details)[source]¶

Get the Unique Id of the deployment job.

Parameters

Important

job_details: metadata of the deployment job

type: dict

Output

Important

returns: Unique Id of the deployment job

return type: str

Example

>>> job_details = client.deployments.get_job_details(job_uid=job_uid)
>>> job_status = client.deployments.get_job_uid(job_details)

static get_scoring_href(deployment_details)[source]¶

Get scoring url from deployment details.

Parameters

Important

deployment_details: Metadata of the deployment

type: dict

Output

Important

returns: scoring endpoint url that is used for making scoring requests

return type: str

Example

>>> scoring_href = client.deployments.get_scoring_href(deployment)

static get_serving_href(deployment_details)[source]¶

Get serving url from deployment details.

Parameters

Important

deployment_details: Metadata of the deployment

type: dict

Output

Important

returns: serving endpoint url that is used for making scoring requests

return type: str

Example

>>> scoring_href = client.deployments.get_serving_href(deployment)

static get_uid(deployment_details)[source]¶

Get deployment_uid from deployment details. Deprecated!! Use get_id(deployment_details) instead

Parameters

Important

deployment_details: Metadata of the deployment

type: dict

Output

Important

returns: deployment UID that is used to manage the deployment

return type: str

Example

>>> deployment_uid = client.deployments.get_uid(deployment)

is_serving_name_available(serving_name: str) → bool[source]¶

Check if serving name is available for usage.

Parameters

Important

serving_name: Serving name to filter deployments (optional)

type: str

Output

Important

returns: information if serving name is available

return type: bool

Example

>>> is_available = client.deployments.is_serving_name_available('test')

list(limit=None)[source]¶

List deployments. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all deployments in a table format.

return type: None

Example

>>> client.deployments.list()

list_jobs(limit=None)[source]¶

List the async deployment jobs. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all async jobs in a table format.

return type: None

Note

This method list only async deployment jobs created for WML deployment.

Example

>>> client.deployments.list_jobs()

score(deployment_id, meta_props, transaction_id=None)[source]¶

Make scoring requests against deployed artifact.

Parameters

Important

deployment_id: Unique Id of the deployment to be scored

type: str

meta_props: Meta props for scoring

>>> Use client.deployments.ScoringMetaNames.show() to view the list of ScoringMetaNames.

type: dict

transaction_id: transaction id to be passed with records during payload logging (optional)

type: str

Output

Important

returns: scoring result containing prediction and probability

return type: dict

Note

client.deployments.ScoringMetaNames.INPUT_DATA is the only metaname valid for sync scoring.
The valid payloads for scoring input are either list of values, pandas or numpy dataframes.

Example

>>> scoring_payload = {wml_client.deployments.ScoringMetaNames.INPUT_DATA:
>>>        [{'fields':
>>>            ['GENDER','AGE','MARITAL_STATUS','PROFESSION'],
>>>            'values': [
>>>                ['M',23,'Single','Student'],
>>>                ['M',55,'Single','Executive']
>>>                ]
>>>         }
>>>       ]}
>>> predictions = client.deployments.score(deployment_id, scoring_payload)
>>> predictions = client.deployments.score(deployment_id, scoring_payload,async=True)

update(deployment_uid, changes)[source]¶

Updates existing deployment metadata. If ASSET is patched, then ‘id’ field is mandatory and it starts a deployment with the provided asset id/rev. Deployment id remains same

Parameters

Important

deployment_uid: Unique Id of deployment which should be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated deployment

return type: dict

Examples

>>> metadata = {client.deployments.ConfigurationMetaNames.NAME:"updated_Deployment"}
>>> updated_deployment_details = client.deployments.update(deployment_uid, changes=metadata)

>>> metadata = {client.deployments.ConfigurationMetaNames.ASSET: { "id": "ca0cd864-4582-4732-b365-3165598dc945",
>>>                                                                "rev":"2" }}
>>> deployment_details = client.deployments.update(deployment_uid, changes=metadata)

class metanames.DeploymentNewMetaNames[source]¶

Set of MetaNames for Deployments Specs.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
TAGS	list	N	`['string1', 'string2']`	`['string']`
NAME	str	N	`my_deployment`
DESCRIPTION	str	N	`my_deployment`
CUSTOM	dict	N	`{}`
ASSET	dict	N	`{'id': '4cedab6d-e8e4-4214-b81a-2ddb122db2ab', 'rev': '1'}`
HARDWARE_SPEC	dict	N	`{'id': '3342-1ce536-20dc-4444-aac7-7284cf3befc'}`
HYBRID_PIPELINE_HARDWARE_SPECS	list	N	`[{'node_runtime_id': 'auto_ai.kb', 'hardware_spec': {'id': '3342-1ce536-20dc-4444-aac7-7284cf3befc', 'num_nodes': '2'}}]`
ONLINE	dict	N	`{}`
BATCH	dict	N	`{}`
R_SHINY	dict	N	`{'authentication': 'anyone_with_url'}`
VIRTUAL	dict	N	`{}`
OWNER	str	N	`<owner_id>`

class metanames.ScoringMetaNames[source]¶

Set of MetaNames for Scoring.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	N	`jobs test`
INPUT_DATA	list	N	`[{'fields': ['name', 'age', 'occupation'], 'values': [['john', 23, 'student']]}]`	`[{'name(optional)': 'string', 'id(optional)': 'string', 'fields(optional)': 'array[string]', 'values': 'array[array[string]]'}]`
INPUT_DATA_REFERENCES	list	N		`[{'id(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'href(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}]`
OUTPUT_DATA_REFERENCE	dict	N		`{'type(required)': 'string', 'connection(required)': {'href(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}`
EVALUATIONS_SPEC	list	N	`[{'id': 'string', 'input_target': 'string', 'metrics_names': ['auroc', 'accuracy']}]`	`[{'id(optional)': 'string', 'input_target(optional)': 'string', 'metrics_names(optional)': 'array[string]'}]`
ENVIRONMENT_VARIABLES	dict	N	`{'my_env_var1': 'env_var_value1', 'my_env_var2': 'env_var_value2'}`

class metanames.DecisionOptimizationMetaNames[source]¶

Set of MetaNames for Decision Optimization.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
INPUT_DATA	list	N	`[{'fields': ['name', 'age', 'occupation'], 'values': [['john', 23, 'student']]}]`	`[{'name(optional)': 'string', 'id(optional)': 'string', 'fields(optional)': 'array[string]', 'values': 'array[array[string]]'}]`
INPUT_DATA_REFERENCES	list	N	`[{'fields': ['name', 'age', 'occupation'], 'values': [['john', 23, 'student']]}]`	`[{'name(optional)': 'string', 'id(optional)': 'string', 'fields(optional)': 'array[string]', 'values': 'array[array[string]]'}]`
OUTPUT_DATA	list	N		`[{'name(optional)': 'string'}]`
OUTPUT_DATA_REFERENCES	list	N		`{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'endpoint_url(required)': 'string', 'access_key_id(required)': 'string', 'secret_access_key(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}`
SOLVE_PARAMETERS	dict	N

Export/Import ¶

class client.Export(client)[source]¶

cancel(export_id, space_id=None, project_id=None)[source]¶

Cancel an export job. ‘export_id’ and ‘space_id’( or ‘project_id’ has to be provided ) Note: To delete a export_id job, use delete() api

Parameters

Important

export_id: Export job identifier

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.export_assets.cancel(export_id='6213cf1-252f-424b-b52d-5cdd9814956c',
>>>                      space_id='3421cf1-252f-424b-b52d-5cdd981495fe')

delete(export_id, space_id=None, project_id=None)[source]¶

Deletes the given export_id job. ‘space_id’ or ‘project_id’ has to be provided

Parameters

Important

export_id: Export job identifier

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.export_assets.delete(export_id='6213cf1-252f-424b-b52d-5cdd9814956c',
>>>                      space_id= '98a53931-a8c0-4c2f-8319-c793155e4598')

get_details(export_id=None, space_id=None, project_id=None, limit=None, asynchronous=False, get_all=False)[source]¶

Get metadata of the given export job. if no export_id is specified all exports metadata is returned.

Parameters

Important

export_id: export job identifier (optional)

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: export(s) metadata

return type: dict

dict (if export_id is not None) or {“resources”: [dict]} (if export_id is None)

Note

If export_id is not specified, all export(s) metadata is fetched

Example

>>> details = client.export_assets.get_details(export_id, space_id= '98a53931-a8c0-4c2f-8319-c793155e4598')
>>> details = client.export_assets.get_details()
>>> details = client.export_assets.get_details(limit=100)
>>> details = client.export_assets.get_details(limit=100, get_all=True)
>>> details = []
>>> for entry in client.export_assets.get_details(limit=100, asynchronous=True, get_all=True):
>>>    details.extend(entry)

get_exported_content(export_id, space_id=None, project_id=None, file_path=None)[source]¶

Get the exported content as a zip file

Parameters

Important

export_id: export job identifier

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str
file_path: name of local file to create. This should be absolute path of the file and
the file shouldn’t exist

type: str

Output

Important

returns: Path to the downloaded function content

return type: str

Example

>>> client.exports.get_exported_content(export_id,
>>>                                     space_id= '98a53931-a8c0-4c2f-8319-c793155e4598'
>>>                                     file_path='/home/user/my_exported_content.zip')

static get_id(export_details)[source]¶

Get ID of export job from export details

Parameters

Important

export_details: Metadata of the export job

type: dict

Output

Important

returns: ID of the export job

return type: str

Example

>>> id = client.export_assets.get_id(export_details)

list(space_id=None, project_id=None, limit=None)[source]¶

List export jobs. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

This method only prints the list of all export jobs in a table format.

return type: None

Example

>>> client.export_assets.list()

start(meta_props, space_id=None, project_id=None)[source]¶

Start the export. Either space_id or project_id has to be provided and is mandatory. ALL_ASSETS is by default False. No need to provide explicitly unless it has to be set to True Either ALL_ASSETS or ASSET_TYPES or ASSET_IDS has to be given in the meta_props. Only one of these can be provided

In the meta_props:

ALL_ASSETS is a boolean. When set to True, it exports all assets in the given space ASSET_IDS is an array containing the list of assets ids to be exported ASSET_TYPES is for providing the asset types to be exported. All assets of that asset type will be exported

Eg: wml_model, wml_model_definition, wml_pipeline, wml_function, wml_experiment, software_specification, hardware_specification, package_extension, script

Parameters

Important

meta_props: meta data. To see available meta names use **client.export_assets.ConfigurationMetaNames.get()

type: dict
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: Response json

return type: dict

Example

>>> metadata = {
>>>    client.export_assets.ConfigurationMetaNames.NAME: "export_model",
>>>    client.export_assets.ConfigurationMetaNames.ASSET_IDS: ["13a53931-a8c0-4c2f-8319-c793155e7517", "13a53931-a8c0-4c2f-8319-c793155e7518"]
>>> }
>>> details = client.export_assets.start(meta_props=metadata, space_id="98a53931-a8c0-4c2f-8319-c793155e4598")

>>> metadata = {
>>>    client.export_assets.ConfigurationMetaNames.NAME: "export_model",
>>>    client.export_assets.ConfigurationMetaNames.ASSET_TYPES: ["wml_model"],
>>> }
>>> details = client.export_assets.start(meta_props=metadata, space_id="98a53931-a8c0-4c2f-8319-c793155e4598")

>>> metadata = {
>>>    client.export_assets.ConfigurationMetaNames.NAME: "export_model",
>>>    client.export_assets.ConfigurationMetaNames.ALL_ASSETS: True,
>>> }

class client.Import(client)[source]¶

cancel(import_id, space_id=None, project_id=None)[source]¶

Cancel an import job. ‘import_id’ and ‘space_id’( or ‘project_id’ has to be provided ) Note: To delete an import_id job, use delete() api

Parameters

Important

import_id: Import job identifier

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.import_assets.cancel(import_id='6213cf1-252f-424b-b52d-5cdd9814956c',
>>>                      space_id='3421cf1-252f-424b-b52d-5cdd981495fe')

delete(import_id, space_id=None, project_id=None)[source]¶

Deletes the given import_id job. ‘space_id’ or ‘project_id’ has to be provided

Parameters

Important

import_id: Import job identifier

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.import_assets.delete(import_id='6213cf1-252f-424b-b52d-5cdd9814956c',
>>>                      space_id= '98a53931-a8c0-4c2f-8319-c793155e4598')

get_details(import_id=None, space_id=None, project_id=None, limit=None, asynchronous=False, get_all=False)[source]¶

Get metadata of the given import job. if no import_id is specified, all imports metadata is returned.

Parameters

Important

import_id: import job identifier (optional)

type: str
space_id: Space identifier

type: str
project_id: Project identifier

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: import(s) metadata

return type: dict

dict (if import_id is not None) or {“resources”: [dict]} (if import_id is None)

Note

If import_id is not specified, all import(s) metadata is fetched

Example

>>> details = client.import_assets.get_details(import_id)
>>> details = client.import_assets.get_details()
>>> details = client.import_assets.get_details(limit=100)
>>> details = client.import_assets.get_details(limit=100, get_all=True)
>>> details = []
>>> for entry in client.import_assets.get_details(limit=100, asynchronous=True, get_all=True):
>>>    details.extend(entry)

static get_id(import_details)[source]¶

Get ID of import job from import details

Parameters

Important

import_details: Metadata of the import job

type: dict

Output

Important

returns: ID of the import job

return type: str

Example

>>> id = client.import_assets.get_id(import_details)

list(space_id=None, project_id=None, limit=None)[source]¶

List import jobs. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

This method only prints the list of all import jobs in a table format.

return type: None

Example

>>> client.import_assets.list()

start(file_path=None, space_id=None, project_id=None)[source]¶

Start the import. Either space_id or project_id has to be provided and is mandatory. Note that on IBM Cloud Pak® for Data 3.5, import into non-empty space/project is not supported

Parameters

Important

file_path: file path to zip file with exported assets

type: dict
space_id: Space identifier

type: str
project_id: Project identifier

type: str

Output

Important

returns: Response json

return type: dict

Example

>>> details = client.import_assets.start(space_id="98a53931-a8c0-4c2f-8319-c793155e4598",
>>>                                      file_path="/home/user/data_to_be_imported.zip")

Federated Learning ¶

Federated Learning provides the tools for training a model collaboratively, by coordinating local training runs and fusing the results. Even though data sources are never moved, combined, or shared among parties or the aggregator, all of them contribute to training and improving the quality of the global model.

Tutorial and Samples

Aggregation ¶

The aggregator process, which fuses the parties’ training results, runs as a Watson Machine Learning training job. For more information on creating and querying a training job, see the API documentation for the client.training class. The parameters available to configure a Federated Learning training are described in the IBM Cloud API Docs.

Configure and start aggregation ¶

from ibm_watson_machine_learning import APIClient

client = APIClient( credentials )

PROJECT_ID = "8ae1a720-83ed-4c57-b719-8dd086bd7ce0"
client.set.default_project( PROJECT_ID )

aggregator_metadata = {
    client.training.ConfigurationMetaNames.NAME: 'Federated Tensorflow MNIST',
    client.training.ConfigurationMetaNames.DESCRIPTION: 'MNIST digit recognition with Federated Learning using Tensorflow',
    client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCES: [],
    client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: {
        'type': 'container',
        'name': 'outputData',
        'connection': {},
        'location': {
           'path': '/projects/' + PROJECT_ID + '/assets/trainings/'
        }

    },
    client.training.ConfigurationMetaNames.FEDERATED_LEARNING: {
        'model': {
            'type': 'tensorflow',
            'spec': {
               'id': untrained_model_id
            },
            'model_file': untrained_model_name
        },
        'fusion_type': 'iter_avg',
        'metrics': 'accuracy',
        'epochs': 3,
        'rounds': 99,
        'remote_training' : {
            'quorum': 1.0,
            'max_timeout': 3600,
            'remote_training_systems': [ { 'id': rts_1_id }, { 'id': rts_2_id} ]
        },
        'hardware_spec': {
            'name': 'S'
        },
        'software_spec': {
            'name': 'runtime-22.1-py3.9'
        }
    }
}


aggregator = client.training.run(aggregator_metadata, asynchronous=True)
aggregator_id = client.training.get_id(aggregator)

Local training ¶

Training is performed locally by parties which connect to the aggregator. The parties must be members of the project or space in which the aggregator is running and are identified to the Federated Learning aggregator as Remote Training Systems.

Configure and start local training ¶

from ibm_watson_machine_learning import APIClient

client = APIClient( party_1_credentials )

PROJECT_ID = "8ae1a720-83ed-4c57-b719-8dd086bd7ce0"
client.set.default_project( PROJECT_ID )

# The party needs, at mimimum, to specify how the data are loaded for training.  The data
# handler class and any input to the class is provided.  In this case, the info block
# contains a key to locate the training data from the current working directory.
party_metadata = {
                    client.remote_training_systems.ConfigurationMetaNames.DATA_HANDLER: {
                       "class": MNISTDataHandler,
                       "info": {
                          "npz_file": "./training_data.npz"
                       }
                 }
# The party object is created
party = client.remote_training_systems.create_party(remote_training_system_id = "d516d42c-6c59-41f2-b7ca-c63d11ea79a1", party_metadata)
# Send training logging to standard output
party.monitor_logs()
# Start training.  Training will run in the Python process that is executing this code.
# The supplied aggregator_id refers to the Watson Machine Learning training job that will perform aggregation.
party.run(aggregator_id = "564fb126-9bfd-409b-beb3-5d401e4c50ec", asynchronous = False)

class remote_training_system.RemoteTrainingSystem(client)[source]¶

The RemoteTrainingSystem class represents a Federated Learning party and provides a list of identities that are permitted to join training as the RemoteTrainingSystem.

create_party(remote_training_system_id, party_metadata)[source]¶

Creates a party object using the specified remote training system id and the party metadata.

Parameters

Important

remote_training_system_id: remote training system identifier

type: str
party_metadata: the party configuration

type: dict

Output

Important

returns: A party object with the specified rts_id and configuration

return type: Party

Examples

>>> party_metadata = {
>>>     wml_client.remote_training_systems.ConfigurationMetaNames.DATA_HANDLER: {
>>>        "info": {
>>>            "npz_file": "./data_party0.npz"
>>>        },
>>>        "name": "MnistTFDataHandler",
>>>        "path": "./mnist_keras_data_handler.py"
>>>     },
>>>     wml_client.remote_training_systems.ConfigurationMetaNames.LOCAL_TRAINING: {
>>>        "name": "LocalTrainingHandler",
>>>        "path": "ibmfl.party.training.local_training_handler"
>>>     },
>>>     wml_client.remote_training_systems.ConfigurationMetaNames.HYPERPARAMS: {
>>>        "epochs": 3
>>>     },
>>> }
>>> party = client.remote_training_systems.create_party(remote_training_system_id, party_metadata)
>>> 
>>> party_metadata = {
>>>     wml_client.remote_training_systems.ConfigurationMetaNames.DATA_HANDLER: {
>>>        "info": {
>>>            "npz_file": "./data_party0.npz"
>>>        },
>>>        "class": MnistTFDataHandler
>>>     }
>>> }
>>> party = client.remote_training_systems.create_party(remote_training_system_id, party_metadata)

create_revision(remote_training_system_id)[source]¶

Create a new remote training system revision.

Parameters: remote_training_system_id ({str_type}) – Unique remote training system ID

Example:

>>> client.remote_training_systems.create_revision(remote_training_system_id)

delete(remote_training_systems_id)[source]¶

Deletes the given remote_training_systems_id definition. ‘space_id’ or ‘project_id’ has to be provided

Parameters

Important

remote_training_systems_id: Remote Training System identifier

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.remote_training_systems.delete(remote_training_systems_id='6213cf1-252f-424b-b52d-5cdd9814956c')

get_details(remote_training_system_id=None, limit=None, asynchronous=False, get_all=False)[source]¶

Get metadata of the given Remote Training System. If remote_training_system_id is not specified, metadata is returned for all Remote Training Systems.

Parameters

Important

remote_training_system_id: remote training system identifier (optional)

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: remote training system(s) metadata

return type: dict

dict (if remote_training_systems_id is not None) or {“resources”: [dict]} (if remote_training_systems_id is None)

Example

>>> details = client.remote_training_systems.get_details(remote_training_systems_id)
>>> details = client.remote_training_systems.get_details()
>>> details = client.remote_training_systems.get_details(limit=100)
>>> details = client.remote_training_systems.get_details(limit=100, get_all=True)
>>> details = []
>>> for entry in client.remote_training_systems.get_details(limit=100, asynchronous=True, get_all=True):
>>>    details.extend(entry)

static get_id(remote_training_system_details)[source]¶

Get ID of remote_training_system

Parameters

Important

remote_training_system_details: Metadata of the stored remote training system

type: dict

Output

Important

returns: ID of stored remote training system

return type: str

Example

>>> details = client.remote_training_systems.get_details(remote_training_system_id)
>>> id = client.remote_training_systems.get_id(details)

get_revision_details(remote_training_system_id, rev_id)[source]¶

Get metadata from the specific revision of a stored remote system

Parameters

remote_training_system_id ({str_type}) – stored functions, definition
rev_id – Unique id of the remote system revision.

:type rev_id : int

Returns: stored remote system revision metadata
Return type: dict

Example:

>>> details = client.remote_training_systems.get_details(remote_training_system_id, rev_id)

list(limit=None)[source]¶

List stored remote training systems. If limit is set to None, only the first 50 records are shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all remote training systems in a table format.

return type: None

Example

>>> client.remote_training_systems.list()

list_revisions(remote_training_system_id, limit=None)[source]¶

List all revisions for the given remote_training_system_id

Parameters

remote_training_system_id ({str_type}) – Unique id of stored remote system
limit (int) – limit number of fetched records (optional)

Returns

Summary list of all remote systems revisions

Return type

table

>>> details = client.remote_training_systems.list_revisions(remote_training_system_id)

store(meta_props)[source]¶

Create a remote training system. Either space_id or project_id has to be provided.

Parameters

Important

meta_props: meta data. To see available meta names use **client.remote_training_systems.ConfigurationMetaNames.get()

type: str or dict

Output

Important

returns: Response json

return type: dict

Example

>>> metadata = {
>>>    client.remote_training_systems.ConfigurationMetaNames.NAME: "my-resource",
>>>    client.remote_training_systems.ConfigurationMetaNames.TAGS: ["tag1", "tag2"],
>>>    client.remote_training_systems.ConfigurationMetaNames.ORGANIZATION: {"name": "name", "region": "EU"}
>>>    client.remote_training_systems.ConfigurationMetaNames.ALLOWED_IDENTITIES: [{"id": "43689024", "type": "user"}],
>>>    client.remote_training_systems.ConfigurationMetaNames.REMOTE_ADMIN: {"id": "43689020", "type": "user"}
>>> }
>>> client.set.default_space('3fc54cf1-252f-424b-b52d-5cdd9814987f')
>>> details = client.remote_training_systems.store(meta_props=metadata)

update(remote_training_system_id, changes)[source]¶

Updates existing remote training system metadata.

Parameters

Important

remote_training_system_id: remote training system identifier

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Example

>>> metadata = {
>>> client.remote_training_systems.ConfigurationMetaNames.NAME:"updated_remote_training_system"
>>> }
>>> details = client.remote_training_systems.update(remote_training_system_id, changes=metadata)

class party_wrapper.Party(client=None, **kwargs)[source]¶

The Party class embodies a Federated Learning party, with methods to run, cancel, and query local training. Refer to the client.remote_training_system.create_party() API for more information about creating an instance of the Party class.

cancel()[source]¶

Stop the local connection to the training on the party side

Example

>>> party.cancel()

get_round()[source]¶

Get the current round number

Output

Important

returns: The current round number

return type: int

Example

>>> party.get_round()

is_running()[source]¶

Check if the training job is running

Output

Important

returns: If the job is running

return type: bool

Example

>>> party.is_running()

monitor_logs(log_level='INFO')[source]¶

Enable logging of the training job to standard output. This method should be called before calling the run() method.

Parameters

Important

log_level: log level specified by user

type: str

Output

Important

returns: This method only outputs the logs to stdout

return type: None

Example

>>> party.monitor_logs()

monitor_metrics(metrics_file='-')[source]¶

Enable output of training metrics.

Parameters

Important

metrics_file: a filename specified by user to which the metrics should be written

type: str

Output

Important

returns: This method outputs the metrics to stdout if a filename is not specified

return type: None

Example

>>> party.monitor_metrics()

run(aggregator_id=None, experiment_id=None, asynchronous=True, verify=True, timeout: int = 600)[source]¶

Connect to a Federated Learning aggregator and run local training. Exactly one of aggregator_id and experiment_id must be supplied.

Parameters

Important

aggregator_id: aggregator identifier
- If aggregator_id is supplied, the party will connect to the given aggregator.
type: str
experiment_id: experiment identifier
- If experiment_id is supplied, the party will connect to the most recently created aggregator for the experiment.
type: str
asynchronous:
- True - party starts to run the job in the background and progress can be checked later.
- False - method will wait until training is complete and then print the job status.
type: bool
verify: verify certificate

type: bool
timeout: timeout in seconds.
- If the aggregator is not ready within timeout seconds, exit.
type: int, or None for no timeout

Examples

>>> party.run( aggregator_id = "69500105-9fd2-4326-ad27-7231aeb37ac8", asynchronous = True, verify = True )
>>> party.run( experiment_id = "2466fa06-1110-4169-a166-01959adec995", asynchronous = False )

Factsheets ¶

class client.Factsheets(client)[source]¶

Link WML Model to Model Entry

list_model_entries(catalog_id=None)[source]¶

Returns all WKC Model Entry assets for a catalog

Parameters

Important

catalog_id: Catalog ID where you want to register model, if None list from all catalogs.

type: str

Output

Important

returns: all WKC Model Entry assets for a catalog

return type: dict

Example

>>> model_entries = client.factsheets.list_model_entries(catalog_id)

register_model_entry(model_id, meta_props, catalog_id=None)[source]¶

Link WML Model to Model Entry

Parameters

Important

model_id: Published model/asset ID

type: str
catalog_id: Catalog ID where you want to register model

type: str
meta_props: metaprops. To see the available list of metanames use:
```
>>> client.factsheets.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the registration

return type: dict

Example

>>> meta_props = {
>>> wml_client.factsheets.ConfigurationMetaNames.ASSET_ID: '83a53931-a8c0-4c2f-8319-c793155e7517'
>>> }
>>> registration_details = client.factsheets.register_model_entry(model_id, catalog_id, meta_props)

>>> meta_props = {
>>> wml_client.factsheets.ConfigurationMetaNames.NAME: "New model entry",
>>> wml_client.factsheets.ConfigurationMetaNames.DESCRIPTION: "New model entry"
>>> }
>>> registration_details = client.factsheets.register_model_entry(model_id, meta_props)

unregister_model_entry(asset_id, catalog_id: Optional[str] = None)[source]¶

Unregister WKC Model Entry

Parameters

Important

asset_id: WKC model entry id

type: str
catalog_id: Catalog ID where asset is stored, optional, when not provided,
default client space or project will be taken.

type: str

Example

>>> model_entries = client.factsheets.unregister_model_entry(asset_id='83a53931-a8c0-4c2f-8319-c793155e7517',
>>>    catalog_id='34553931-a8c0-4c2f-8319-c793155e7517')
or

>>> client.set.default_space('98f53931-a8c0-4c2f-8319-c793155e7517')
>>> model_entries = client.factsheets.unregister_model_entry(asset_id='83a53931-a8c0-4c2f-8319-c793155e7517')

class metanames.FactsheetsMetaNames[source]¶

Set of MetaNames for Factsheets metanames.

Available MetaNames:

MetaName	Type	Required	Schema
ASSET_ID	str	N	`13a53931-a8c0-4c2f-8319-c793155e7517`
NAME	str	N	`New model entry`
DESCRIPTION	str	N	`New model entry`
MODEL_ENTRY_CATALOG_ID	str	Y	`13a53931-a8c0-4c2f-8319-c793155e7517`

Hardware specifications ¶

class client.HwSpec(client)[source]¶

Store and manage your hardware specs.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.HwSpecMetaNames object>¶: MetaNames for Hardware Specification.

get_details(hw_spec_uid)[source]¶

Get hardware specification details.

Parameters

Important

hw_spec_uid : Unique id of the hardware spec ** type**: str

Output

Important

returns: Metadata of the hardware specifications

return type: dict

Example

>>> hw_spec_details = client.hardware_specifications.get_details(hw_spec_uid)

static get_href(hw_spec_details)[source]¶

Get url of hardware specifications.

Parameters

Important

hw_spec_details: hardware specifications details

type: dict

Output

Important

returns: href of hardware specifications

return type: str

Example

>>> hw_spec_details = client.hw_spec.get_details(hw_spec_uid)
>>> hw_spec_href = client.hw_spec.get_href(hw_spec_details)

static get_id(hw_spec_details)[source]¶

Get ID of hardware specifications asset.

Parameters

Important

asset_details: Metadata of the hardware specifications

type: dict

Output

Important

returns: Unique Id of hardware specifications

return type: str

Example

>>> asset_uid = client.hardware_specifications.get_id(hw_spec_details)

get_id_by_name(hw_spec_name)[source]¶

Get Unique Id of hardware specification for the given name.

Parameters

Important

hw_spec_name: name of the hardware spec

type: str

Output

Important

returns: Unique Id of hardware specification

return type: str

Example

>>> asset_uid = client.hardware_specifications.get_id_by_name(hw_spec_name)

static get_uid(hw_spec_details)[source]¶

Get UID of hardware specifications asset. Deprecated!! Use get_id(hw_spec_details) instead

Parameters

Important

asset_details: Metadata of the hardware specifications

type: dict

Output

Important

returns: Unique Id of hardware specifications

return type: str

Example

>>> asset_uid = client.hardware_specifications.get_uid(hw_spec_details)

get_uid_by_name(hw_spec_name)[source]¶

Get Unique Id of hardware specification for the given name. Deprecated!! Use get_id_by_name(self, hw_spec_name) instead

Parameters

Important

hw_spec_name: name of the hardware spec

type: str

Output

Important

returns: Unique Id of hardware specification

return type: str

Example

>>> asset_uid = client.hardware_specifications.get_uid_by_name(hw_spec_name)

list(name=None)[source]¶

List hardware specifications.

Parameters

Important

name : Unique id of the hardware spec ** type**: str

Output

Important

This method only prints the list of all assets in a table format.

return type: None

Example

>>> client.hardware_specifications.list()

class metanames.HwSpecMetaNames[source]¶

Set of MetaNames for Software Specifications Specs.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`Python 3.6 with pre-installed ML package`
DESCRIPTION	str	N	`my_description`
HARDWARE_CONFIGURATION	dict	N	`{}`

Model definitions ¶

class client.ModelDefinition(client)[source]¶

Store and manage your model_definitions.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.ModelDefinitionMetaNames object>¶: MetaNames for model_definition creation.

create_revision(model_definition_uid)[source]¶

Creates revision for the given model_definition. Revisions are immutable once created. The metadata and attachment at model_definition is taken and a revision is created out of it.

Parameters

Important

model_definition: model_definition ID.

type: str

Output

Important

returns: Stored model_definition revisions metadata.

return type: dict

Example

>>> model_definition_revision = client.model_definitions.create_revision(model_definition_id)

delete(model_definition_uid)[source]¶

Delete a stored model_definition.

Parameters

Important

model_definition_uid: Unique Id of stored Model definition

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.model_definitions.delete(model_definition_uid)

download(model_definition_uid, filename, rev_id=None)[source]¶

Download the content of a script asset.

Parameters

Important

model_definition_uid: The Unique Id of the model_definition asset to be downloaded

type: str
filename: filename to be used for the downloaded file

type: str
rev_id: Revision id

type: str

Output

Important

returns: Path to the downloaded asset content

return type: str

Example

>>> client.model_definitions.download(model_definition_uid, "model_definition_file")

get_details(model_definition_uid=None)[source]¶

Get metadata of stored model_definition. If no model_definition_uid is passed, details for all model_definitions will be returned.

Parameters

Important

model_definition_uid: Unique Id of model_definition

type: str

Output

Important

returns: metadata of model definition

return type: dict (if model_definition_uid is not None)

Example

>>> model_definition_details = client.model_definitions.get_details(model_definition_uid)

get_href(model_definition_details)[source]¶

Get href of stored model_definition.

Parameters

Important

model_definition_details: Stored model_definition details.

type: dict

Output

Important

returns: href of stored model_definition.

return type: str

Example

>>> model_definition_uid = client.model_definitions.get_href(model_definition_details)

get_id(model_definition_details)[source]¶

Get Unique Id of stored model_definition asset.

Parameters

Important

model_definition_details: Metadata of the stored model_definition asset

type: dict

Output

Important

returns: Unique Id of stored model_definition asset

return type: str

Example

>>> asset_uid = client.model_definition.get_id(asset_details)

get_revision_details(model_definition_uid, rev_uid=None)[source]¶

Get metadata of model_definit

Parameters

Important

model_definition_uid: model_definition ID.

type: str
rev_uid: Revision ID. If this parameter is not provided, returns latest revision if existing else error.

type: int

Output

Important

returns: Stored model definitions metadata.

return type: dict

Example

>>> script_details = client.model_definitions.get_revision_details(model_definition_uid, rev_uid)

get_uid(model_definition_details)[source]¶

Get uid of stored model. Deprecated!! Use get_id(model_definition_details) instead

Parameters

Important

param model_definition_details: stored model_definition details

type: dict

Output

Important

returns: uid of stored model_definition

return type: str

Example

>>> model_definition_uid = client.model_definitions.get_uid(model_definition_details)

list(limit=None)[source]¶

List stored model_definition assets. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all model_definition assets in a table format.

return type: None

Example

>>> client.model_definitions.list()

list_revisions(model_definition_uid, limit=None)[source]¶

List stored model_definition assets. If limit is set to None there will be only first 50 records shown.

Parameters

Important

model_definition_uid: Unique id of model_definition

type: str
limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all model_definition revision in a table format.

return type: None

Example

>>> client.model_definitions.list_revisions()

store(model_definition, meta_props)[source]¶

Create a model_definitions.

Parameters

Important

meta_props: meta data of the model_definition configuration. To see available meta names use:
```
>>> client.model_definitions.ConfigurationMetaNames.get()
```
type: dict
model_definition: Path to the content file to be uploaded

type: str

Output

Important

returns: Metadata of the model_defintion created

return type: dict

Example

>>>  client.model_definitions.store(model_definition, meta_props)

update(model_definition_id, meta_props=None, file_path=None)[source]¶

Update model_definition with either metadata or attachment or both.

Parameters

Important

model_definition_id: model_definition ID.

type: str
file_path: Path to the content file to be uploaded.

type: str

** Output**

Important

returns: Updated metadata of model_definition.

return type: dict

Example

>>> model_definition_details = client.model_definition.update(model_definition_id, meta_props, file_path)

class metanames.ModelDefinitionMetaNames[source]¶

Set of MetaNames for Model Definition.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`my_model_definition`
DESCRIPTION	str	N	`my model_definition`
PLATFORM	dict	Y	`{'name': 'python', 'versions': ['3.5']}`	`{'name(required)': 'string', 'versions(required)': ['versions']}`
VERSION	str	Y	`1.0`
COMMAND	str	N	`python3 convolutional_network.py`
CUSTOM	dict	N	`{'field1': 'value1'}`
SPACE_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`

Package extensions ¶

class client.PkgExtn(client)[source]¶

Store and manage your software Packages Extension specs.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.PkgExtnMetaNames object>¶: MetaNames for Package Extensions creation.

delete(pkg_extn_uid)[source]¶

Delete a package extension.

Parameters

Important

pkg_extn_uid: Unique Id of package extension

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.package_extensions.delete(pkg_extn_uid)

download(pkg_extn_id, filename)[source]¶

Download a package extension.

Parameters

Important

pkg_extn_uid: The Unique Id of the package extension to be downloaded

type: str
filename: filename to be used for the downloaded file

type: str

Output

Important

returns: Path to the downloaded package extension content

return type: str

Example

>>> client.assets.download(asset_uid,"sample_conda.yml/custom_library.zip")

get_details(pkg_extn_uid)[source]¶

Get package extensions details.

Parameters

Important

pkg_extn_details: Metadata of the package extensions

type: dict

Output

Important

returns: pkg_extn UID

return type: str

Example

>>> pkg_extn_details = client.pkg_extn.get_details(pkg_extn_uid)

static get_href(pkg_extn_details)[source]¶

Get url of stored package extensions.

Parameters

Important

asset_details: package extensions details

type: dict

Output

Important

returns: href of package extensions details

return type: str

Example

>>> pkg_extn_details = client.package_extensions.get_details(pkg_extn_uid)
>>> pkg_extn_href = client.package_extensions.get_href(pkg_extn_details)

static get_id(pkg_extn_details)[source]¶

Get Unique Id of package extensions.

Parameters

Important

asset_details: Metadata of the package extensions

type: dict

Output

Important

returns: Unique Id of package extension

return type: str

Example

>>> asset_id = client.package_extensions.get_id(pkg_extn_details)

get_id_by_name(pkg_extn_name)[source]¶

Get ID of package extensions.

Parameters

Important

asset_details: Metadata of the package extension

type: dict

Output

Important

returns: Unique Id of package extension

return type: str

Example

>>> asset_id = client.package_extensions.get_id_by_name(pkg_extn_name)

static get_uid(pkg_extn_details)[source]¶

Get Unique Id of package extensions. Deprecated!! use get_id(pkg_extn_details) instead

Parameters

Important

asset_details: Metadata of the package extensions

type: dict

Output

Important

returns: Unique Id of package extension

return type: str

Example

>>> asset_uid = client.package_extensions.get_uid(pkg_extn_details)

get_uid_by_name(pkg_extn_name)[source]¶

Get UID of package extensions. Deprecated!! Use get_id_by_name(pkg_extn_name) instead

Parameters

Important

asset_details: Metadata of the package extension

type: dict

Output

Important

returns: Unique Id of package extension

return type: str

Example

>>> asset_uid = client.package_extensions.get_uid_by_name(pkg_extn_name)

list()[source]¶

List package extensions.

Output

Important

This method only prints the list of all package extensionss in a table format.

return type: None

Example

>>> client.package_extensions.list()

store(meta_props, file_path)[source]¶

Create a package extensions.

Parameters

meta_props (dict) –

meta data of the pacakge extension. To see available meta names use:

client.package_extensions.ConfigurationMetaNames.get()

Returns

metadata of the package extensions

Return type

dict

Example

meta_props = {
    client.package_extensions.ConfigurationMetaNames.NAME: "skl_pipeline_heart_problem_prediction",
    client.package_extensions.ConfigurationMetaNames.DESCRIPTION: "description scikit-learn_0.20",
    client.package_extensions.ConfigurationMetaNames.TYPE: "conda_yml"
}

Example

pkg_extn_details = client.package_extensions.store(meta_props=meta_props,file_path="/path/to/file")

class metanames.PkgExtnMetaNames[source]¶

Set of MetaNames for Package Extensions Specs.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`Python 3.6 with pre-installed ML package`
DESCRIPTION	str	N	`my_description`
TYPE	str	Y	`conda_yml/custom_library`

Repository ¶

class client.Repository(client)[source]¶

Store and manage your models, functions, spaces, pipelines and experiments using Watson Machine Learning Repository.

Important

To view ModelMetaNames, use:

>>> client.repository.ModelMetaNames.show()

To view ExperimentMetaNames, use:

>>> client.repository.ExperimentMetaNames.show()

To view FunctionMetaNames, use:

>>> client.repository.FunctionMetaNames.show()

To view PipelineMetaNames, use:

>>> client.repository.PipelineMetaNames.show()

create_experiment_revision(experiment_uid)[source]¶

Create a new version for a experiment.

Parameters

Important

experiment_uid: Unique ID of the experiment.

type: str

Output

Important

returns: experiment version details.

return type: dict

Example

>>> stored_experiment_revision_details = client.repository.create_experiment_revision(experiment_uid)

create_function_revision(function_uid)[source]¶

Create a new version for a function.

Parameters

Important

function_uid: Unique ID of the function.

type: str

Output

Important

returns: Function version details.

return type: dict

Example

>>> stored_function_revision_details = client.repository.create_function_revision( function_uid)

create_member(space_uid, meta_props)[source]¶

Create a member within a space.

Parameters

Important

meta_props: meta data of the member configuration. To see available meta names use:
```
>>> client.spaces.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the stored member

return type: dict

Note

client.spaces.MemberMetaNames.ROLE can be any one of the following “viewer, editor, admin”
client.spaces.MemberMetaNames.IDENTITY_TYPE can be any one of the following “user,service”
client.spaces.MemberMetaNames.IDENTITY can be either service-ID or IAM-userID

Example

>>> metadata = {
>>>  client.spaces.MemberMetaNames.ROLE:"Admin",
>>>  client.spaces.MemberMetaNames.IDENTITY:"iam-ServiceId-5a216e59-6592-43b9-8669-625d341aca71",
>>>  client.spaces.MemberMetaNames.IDENTITY_TYPE:"service"
>>> }
>>> members_details = client.repository.create_member(space_uid=space_id, meta_props=metadata)

create_model_revision(model_uid)[source]¶

Create a new version for a model.

Parameters

Important

model_uid: Model ID.

type: str

Output

Important

returns: Model version details.

return type: dict

Example

>>> stored_model_revision_details = client.repository.create_model_revision( model_uid="MODELID")

create_pipeline_revision(pipeline_uid)[source]¶

Create a new version for a model.

Parameters

Important

pipeline_uid: Unique ID of the Pipeline.

type: str

Output

Important

returns: Pipeline version details.

return type: dict

Example

>>> stored_pipeline_revision_details = client.repository.create_pipeline_revision( pipeline_uid)

create_revision(artifact_uid)[source]¶

Create revision for passed artifact_uid.

Parameters

Important

artifact_uid: Unique id of stored model, experiment, function or pipelines.

type: str

Output

Important

returns: Artifact new revision metadata.

return type: dict

Example

>>> details = client.repository.create_revision(artifact_uid)

delete(artifact_uid)[source]¶

Delete model, experiment, pipeline, space, runtime, library or function from repository.

Parameters

Important

artifact_uid: Unique id of stored model, experiment, function, pipeline, space, library or runtime

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.repository.delete(artifact_uid)

download(artifact_uid, filename='downloaded_artifact.tar.gz', rev_uid=None, format=None)[source]¶

Downloads configuration file for artifact with specified uid.

Parameters

Important

artifact_uid: Unique Id of model, function, runtime or library

type: str
filename: Name of the file to which the artifact content has to be downloaded

default value: downloaded_artifact.tar.gz

type: str

Output

Important

returns: Path to the downloaded artifact content

return type: str

Note

If filename is not specified, the default filename is “downloaded_artifact.tar.gz”.

Example

>>> client.repository.download(model_uid, 'my_model.tar.gz')
>>> client.repository.download(model_uid, 'my_model.json') # if original model was saved as json, works only for xgboost 1.3

get_details(artifact_uid=None)[source]¶

Get metadata of stored artifacts. If artifact_uid is not specified returns all models, experiments, functions, pipelines, spaces, libraries and runtimes metadata.

Parameters

Important

artifact_uid: Unique Id of stored model, experiment, function, pipeline, space, library or runtime (optional)

type: str

Output

Important

returns: stored artifact(s) metadata

return type: dict

dict (if artifact_uid is not None) or {“resources”: [dict]} (if artifact_uid is None)

Note

If artifact_uid is not specified, all models, experiments, functions, pipelines, spaces, libraries and runtimes metadata is fetched

Example

>>> details = client.repository.get_details(artifact_uid)
>>> details = client.repository.get_details()

get_experiment_details(experiment_uid=None, limit=None)[source]¶

Get metadata of experiment. If no experiment_uid is specified all experiments metadata is returned.

Parameters

Important

experiment_uid: Unique Id of experiment (optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: experiment(s) metadata

return type: dict

dict (if experiment_uid is not None) or {“resources”: [dict]} (if experiment_uid is None)

Note

If experiment_uid is not specified, all experiments metadata is fetched

Example

>>> experiment_details = client.respository.get_experiment_details(experiment_uid)

static get_experiment_href(experiment_details)[source]¶

Get href of stored experiment.

Parameters

Important

experiment_details: Metadata of the stored experiment

type: dict

Output

Important

returns: href of stored experiment

return type: str

Example

>>> experiment_details = client.repository.get_experiment_detailsf(experiment_uid)
>>> experiment_href = client.repository.get_experiment_href(experiment_details)

static get_experiment_id(experiment_details)[source]¶

Get Unique Id of stored experiment.

Parameters

Important

experiment_details: Metadata of the stored experiment

type: dict

Output

Important

returns: Unique Id of stored experiment

return type: str

Example

>>> experiment_details = client.repository.get_experiment_details(experiment_uid)
>>> experiment_uid = client.repository.get_experiment_id(experiment_details)

get_experiment_revision_details(experiment_uid, rev_id)[source]¶

Get metadata of experiment revision.

Parameters

Important

experiment_uid: Unique Id of experiment

type: str
rev_id: Unique id of experiment revision

type: str

Output

Important

returns: experiment revision metadata

return type: dict

Example

>>> experiment_rev_details = client.respository.get_experiment__revision_details(experiment_uid, rev_uid)

static get_experiment_uid(experiment_details)[source]¶

Get Unique Id of stored experiment.

Parameters

Important

experiment_details: Metadata of the stored experiment

type: dict

Output

Important

returns: Unique Id of stored experiment

return type: str

Example

>>> experiment_details = client.repository.get_experiment_detailsf(experiment_uid)
>>> experiment_uid = client.repository.get_experiment_uid(experiment_details)

get_function_details(function_uid=None, limit=None)[source]¶

Get metadata of function. If no function_uid is specified all functions metadata is returned.

Parameters

Important

function_uid: Unique Id of function (optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: function(s) metadata

return type: dict (if function_uid is not None) or {“resources”: [dict]} (if function_uid is None)

Note

If function_uid is not specified, all functions metadata is fetched

Example

>>> function_details = client.respository.get_function_details(function_uid)
>>> function_details = client.respository.get_function_details()

static get_function_href(function_details)[source]¶

Get href of stored function.

Parameters

Important

function_details: Metadata of the stored function

type: dict

Output

Important

returns: href of stored function

return type: str

Example

>>> function_details = client.repository.get_function_detailsf(function_uid)
>>> function_url = client.repository.get_function_href(function_details)

static get_function_id(function_details)[source]¶

Get Id of stored function.

Parameters

Important

function_details: Metadata of the stored function

type: dict

Output

Important

returns: Id of stored function

return type: str

Example

>>> function_details = client.repository.get_function_details(function_uid)
>>> function_id = client.repository.get_function_id(function_details)

get_function_revision_details(function_uid, rev_id)[source]¶

Get metadata of function revision.

Parameters

Important

function_uid: Unique Id of function

type: str
rev_id: Unique Id of function revision

type: str

Output

Important

returns: function revision metadata

return type: dict

Example

>>> function_rev_details = client.respository.get_function_revision_details(function_uid, rev_id)

static get_function_uid(function_details)[source]¶

Get Unique Id of stored function. Deprecated!! Use get_function_id(function_details) instead

Parameters

Important

function_details: Metadata of the stored function

type: dict

Output

Important

returns: Unique Id of stored function

return type: str

Example

>>> function_details = client.repository.get_function_detailsf(function_uid)
>>> function_uid = client.repository.get_function_uid(function_details)

static get_member_href(member_details)[source]¶

Get member_href from member details.

Parameters

Important

space_details: Metadata of the stored member

type: dict

Output

Important

returns: member href

return type: str

Example

>>> member_details = client.repository.get_member_details(member_id)
>>> member_href = client.repository.get_member_href(member_details)

static get_member_uid(member_details)[source]¶

Get member_uid from member details.

Parameters

Important

member_details: Metadata of the created member

type: dict

Output

Important

returns: unique id of member

return type: str

Example

>>> member_details = client.repository.get_member_details(member_id)
>>> member_id = client.repository.get_member_uid(member_details)

get_members_details(space_uid, member_id=None, limit=None)[source]¶

Get metadata of members associated with a space. If member_uid is not specified, it returns all the members metadata.

Parameters

Important

space_uid: Unique id of member (optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: metadata of member(s) of a space

return type: dict (if member_id is not None) or {“resources”: [dict]} (if member_id is None)

Note

If member id is not specified, all members metadata is fetched

Example

>>> member_details = client.repository.get_member_details(space_uid,member_id)

get_model_details(model_uid=None, limit=None, asynchronous=False, get_all=False)[source]¶

Get metadata of stored model. If model_uid is not specified returns all models metadata.

Parameters

Important

model_uid: Unique Id of Model (optional)

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: metadata of model(s)

return type: dict (if model_uid is not None) or {“resources”: [dict]} (if model_uid is None)

Note

If model_uid is not specified, all models metadata is fetched

Example

>>> model_details = client.repository.get_model_details(model_uid)
>>> models_details = client.repository.get_model_details()
>>> models_details = client.repository.get_model_details(limit=100)
>>> models_details = client.repository.get_model_details(limit=100, get_all=True)
>>> models_details = []
>>> for details in client.repository.get_model_details(limit=100, asynchronous=True, get_all=True):
>>>     models_details.append(details)

static get_model_href(model_details)[source]¶

Get href of stored model.

Parameters

Important

model_details: Metadata of the stored model

type: dict

Output

Important

returns: href of stored model

return type: str

Example

>>> model_details = client.repository.get_model_detailsf(model_uid)
>>> model_uid = client.repository.get_model_href(model_details)

static get_model_id(model_details)[source]¶

Get Unique Id of stored model.

Parameters

Important

model_details: Metadata of the stored model

type: dict

Output

Important

returns: Unique Id of stored model

return type: str

Example

>>> model_id = client.repository.get_model_id(model_details)

get_model_revision_details(model_uid, rev_uid)[source]¶

Get metadata of model revision.

Parameters

Important

experiment_uid: Unique Id of model

type: str
limit: Unique id of model revision

type: str

Output

Important

returns: model revision metadata

return type: dict

Example

>>> model_rev_details = client.respository.get_model_revision_details(model_uid, rev_uid)

static get_model_uid(model_details)[source]¶: This method is deprecated, please use get_id() instead.”

get_pipeline_details(pipeline_uid=None, limit=None)[source]¶

Get metadata of stored pipelines. If pipeline_uid is not specified returns all pipelines metadata.

Parameters

Important

pipeline_uid: Unique id of Pipeline(optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: metadata of pipeline(s)

return type: dict (if pipeline_uid is not None) or {“resources”: [dict]} (if pipeline_uid is None)

Note

If pipeline_uid is not specified, all pipelines metadata is fetched

Example

>>> pipeline_details = client.repository.get_pipeline_details(pipeline_uid)
>>> pipeline_details = client.repository.get_pipeline_details()

static get_pipeline_href(pipeline_details)[source]¶

Get pipeline_hef from pipeline details.

Parameters

Important

pipeline_details: Metadata of the stored pipeline

type: dict

Output

Important

returns: pipeline href

return type: str

Example

>>> pipeline_details = client.repository.get_pipeline_details(pipeline_uid)
>>> pipeline_href = client.repository.get_pipeline_href(pipeline_details)

static get_pipeline_id(pipeline_details)[source]¶

Get pipeline_uid from pipeline details.

Parameters

Important

pipeline_details: Metadata of the stored pipeline

type: dict

Output

Important

returns: Unique Id of pipeline

return type: str

Example

>>> pipeline_details = client.repository.get_pipeline_details(pipeline_uid)
>>> pipeline_uid = client.repository.get_pipeline_id(pipeline_details)

get_pipeline_revision_details(pipeline_uid, rev_id)[source]¶

Get metadata of stored pipeline revision.

Parameters

Important

pipeline_uid: Unique id of Pipeline

type: str
rev_id: Unique id Pipeline revision

type: str

Output

Important

returns: metadata of revision pipeline(s)

return type: dict

Example

>>> pipeline_rev_details = client.repository.get_pipeline_revision_details(pipeline_uid, rev_id)

static get_pipeline_uid(pipeline_details)[source]¶

Get pipeline_uid from pipeline details.

Parameters

Important

pipeline_details: Metadata of the stored pipeline

type: dict

Output

Important

returns: Unique Id of pipeline

return type: str

Example

>>> pipeline_details = client.repository.get_pipeline_details(pipeline_uid)
>>> pipeline_uid = client.repository.get_pipeline_uid(pipeline_details)

get_space_details(space_uid=None, limit=None)[source]¶

Get metadata of stored space. If space_uid is not specified returns all model spaces metadata.

Parameters

Important

space_uid: Unique id of Space (optional)

type: str
limit: limit number of fetched records (optional)

type: int

Output

Important

returns: metadata of space(s)

return type: dict (if space_uid is not None) or {“resources”: [dict]} (if space_uid is None)

Note

If space_uid is not specified, all spaces metadata is fetched

Example

>>> space_details = client.repository.get_space_details(space_uid)
>>> space_details = client.repository.get_space_details()

static get_space_href(space_details)[source]¶

Get space_href from space details.

Parameters

Important

space_details: Metadata of the stored space

type: dict

Output

Important

returns: space href

return type: str

Example

>>> space_details = client.repository.get_space_details(space_uid)
>>> space_href = client.repository.get_space_href(space_details)

static get_space_uid(space_details)[source]¶

Get space_uid from space details.

Parameters

Important

space_details: Metadata of the stored space

type: dict

Output

Important

returns: Unique Id of space

return type: str

Example

>>> space_details = client.repository.get_space_details(space_uid)
>>> space_uid = client.repository.get_space_uid(space_details)

list()[source]¶

List stored models, pipelines, runtimes, libraries, functions, spaces and experiments. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all models, pipelines, runtimes, libraries, functions, spaces and experiments in a table format.

return type: None

Example

>>> client.repository.list()

list_experiments(limit=None)[source]¶

List stored experiments. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all experiments in a table format.

return type: None

Example

>>> client.repository.list_experiments()

list_experiments_revisions(experiment_uid, limit=None)[source]¶

List stored experiment revisions. If limit is set to None there will be only first 50 records shown.

Parameters

Important

experiment_uid: Uniquie Id of the experiment

type: str

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all revisions of given experiment ID in a table format.

return type: None

Example

>>> client.repository.list_experiments_revisions(experiment_uid)

list_functions(limit=None)[source]¶

List stored functions. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all functions in a table format.

return type: None

Example

>>> client.respository.list_functions()

list_functions_revisions(function_uid, limit=None)[source]¶

List stored function revisions. If limit is set to None there will be only first 50 records shown.

Parameters

Important

function_uid: Uniquie Id of the function

type: str

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all revisions of given function ID in a table format.

return type: None

Example

>>> client.repository.list_functions_revisions(function_uid)

list_members(space_uid, limit=None)[source]¶

List stored members of a space. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all members associated with a space in a table format.

return type: None

Example

>>> client.spaces.list_members()

list_models(limit=None, asynchronous=False, get_all=False)[source]¶

List stored models. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

This method only prints the list of all models in a table format.

return type: None

Example

>>> client.repository.list_models()
>>> client.repository.list_models(limit=100)
>>> client.repository.list_models(limit=100, get_all=True)
>>> [entry for entry in client.repository.list_models(limit=100, asynchronous=True, get_all=True)]

list_models_revisions(model_uid, limit=None)[source]¶

List stored model revisions. If limit is set to None there will be only first 50 records shown.

Parameters

Important

model_uid: Uniquie Id of the model

type: str
limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all revisions of given model ID in a table format.

return type: None

Example

>>> client.repository.list_models_revisions(model_uid)

list_pipelines(limit=None)[source]¶

List stored pipelines. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all pipelines in a table format.

return type: None

Example

>>> client.repository.list_pipelines()

list_pipelines_revisions(pipeline_uid, limit=None)[source]¶

List stored pipeline revisions. If limit is set to None there will be only first 50 records shown.

Parameters

Important

model_uid: Uniquie Id of the pipeline

type: str

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all revisions of given pipeline ID in a table format.

return type: None

Example

>>> client.repository.list_pipelines_revisions(pipeline_uid)

list_spaces(limit=None)[source]¶

List stored spaces. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all spaces in a table format.

return type: None

Example

>>> client.repository.list_spaces()

load(artifact_uid)[source]¶

Load model from repository to object in local environment.

Parameters

Important

artifact_uid: Unique Id of model

type: str

Output

Important

returns: model object

return type: object

Example

>>> model_obj = client.repository.load(model_uid)

promote_model(model_id: str, source_project_id: str, target_space_id: str)[source]¶

Promote model from project to space. Supported only for IBM Cloud Pak® for Data.

Parameters

model_id ({str_type}) – stored model
source_project_id ({str_type}) – stored model
target_space_id ({str_type}) – stored model

Returns

promoted model id

Return type

str

A way you might use me is:

>>> promoted_model_id = client.models.promote(model_id, source_project_id, target_space_id)

store_experiment(meta_props)[source]¶

Create an experiment.

Parameters

Important

meta_props: meta data of the experiment configuration. To see available meta names use:
```
>>> client.experiments.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: Metadata of the experiment created

return type: dict

Example

>>> metadata = {
>>>  client.experiments.ConfigurationMetaNames.NAME: 'my_experiment',
>>>  client.experiments.ConfigurationMetaNames.EVALUATION_METRICS: ['accuracy'],
>>>  client.experiments.ConfigurationMetaNames.TRAINING_REFERENCES: [
>>>      {
>>>        'pipeline': {'href': pipeline_href_1}
>>>      },
>>>      {
>>>        'pipeline': {'href':pipeline_href_2}
>>>      },
>>>   ]
>>> }
>>> experiment_details = client.repository.store_experiment(meta_props=metadata)
>>> experiment_href = client.repository.get_experiment_href(experiment_details)

store_function(function, meta_props)[source]¶

Create a function.

Parameters

Important

meta_props: meta data or name of the function. To see available meta names use:
>>> client.repository.FunctionMetaNames.show()
type: dict
function: path to file with archived function content or function (as described above)
- As a ‘function’ may be used one of the following:
- filepath to gz file
- ‘score’ function reference, where the function is the function which will be deployed
- generator function, which takes no argument or arguments which all have primitive python default values and as result return ‘score’ function
type: str or function

Output

Important

returns: Metadata of the function created.

return type: dict

Example

The most simple use is (using score function):

>>> meta_props = {
>>>    client.repository.FunctionMetaNames.NAME: "function",
>>>    client.repository.FunctionMetaNames.DESCRIPTION: "This is ai function",
>>>    client.repository.FunctionMetaNames.SOFTWARE_SPEC_UID: "53dc4cf1-252f-424b-b52d-5cdd9814987f"}

>>> def score(payload):
>>>      values = [[row[0]*row[1]] for row in payload['values']]
>>>      return {'fields': ['multiplication'], 'values': values}
>>> stored_function_details = client.repository.store_function(score, meta_props)

Other, more interesting example is using generator function. In this situation it is possible to pass some variables:

>>> wml_creds = {...}
>>> def gen_function(wml_credentials=wml_creds, x=2):
>>>        def f(payload):
>>>            values = [[row[0]*row[1]*x] for row in payload['values']]
>>>            return {'fields': ['multiplication'], 'values': values}
>>>        return f
>>> stored_function_details = client.repository.store_function(gen_function, meta_props)

store_model(model, meta_props=None, training_data=None, training_target=None, pipeline=None, feature_names=None, label_column_names=None, subtrainingId=None, round_number=None, experiment_metadata=None)[source]¶

Create a model.

Parameters

Important

model:
Can be one of following:
- The train model object:
  scikit-learn
  
  xgboost
  
  spark (PipelineModel)
- path to saved model in format:
  keras (.tgz)
  
  pmml (.xml)
  
  scikit-learn (.tar.gz)
  
  tensorflow (.tar.gz)
  
  spss (.str)
  
  spark (.tar.gz)
- directory containing model file(s):
  scikit-learn
  
  xgboost
  
  tensorflow
- unique id of trained model

#. Example: >>> published_model_details = wml_client.repository.store_model(model=”rf_model_0.4.tar.gz”, meta_props=metadata) #. training_data: Spark DataFrame supported for spark models. Pandas dataframe, numpy.ndarray or array supported for scikit-learn models

type: spark dataframe, pandas dataframe, numpy.ndarray or array

meta_props: meta data of the models configuration. To see available meta names use:
```
>>> client.repository.ModelMetaNames.get()
```
type: dict
training_target: array with labels required for scikit-learn models

type: array
pipeline: pipeline required for spark mllib models

type: object
feature_names: Feature names for the training data in case of Scikit-Learn/XGBoost models. This is applicable only in the case where the training data is not of type - pandas.DataFrame.

type: numpy.ndarray or list
label_column_names: Label column names of the trained Scikit-Learn/XGBoost models.

type: numpy.ndarray and list
round_number: round number of a Federated Learning experiment that has been configured to save intermediate models. This applies when model is a training id. type: int
experiment_metadata: metadata retrieved from the experiment that created the model
type: dict

Output

Important

returns: Metadata of the model created

return type: dict

Note

For a keras model, model content is expected to contain a .h5 file and an archived version of it.
feature_names is an optional argument containing the feature names for the training data in case of Scikit-Learn/XGBoost models. Valid types are numpy.ndarray and list. This is applicable only in the case where the training data is not of type - pandas.DataFrame.
If the training data is of type pandas.DataFrame and feature_names are provided, feature_names are ignored.
The value can be a single dictionary(being deprecated, use list even for single schema) or a list if you are using single input data schema. you can provide multiple schemas as dictionaries inside a list.

Example

>>> stored_model_details = client.repository.store_model(model, name)

In more complicated cases you should create proper metadata, similar to this one:

>>> sw_spec_id = client.software_specifications.get_id_by_name('scikit-learn_0.23-py3.7')
>>> sw_spec_id
>>> metadata = {
>>>        client.repository.ModelMetaNames.NAME: 'customer satisfaction prediction model',
>>>        client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: sw_spec_id,
>>>        client.repository.ModelMetaNames.TYPE: 'scikit-learn_0.23'
>>>}

In case when you want to provide input data schema of the model, you can provide it as part of meta

>>> sw_spec_id = client.software_specifications.get_id_by_name('spss-modeler_18.1')
>>> sw_spec_id
>>> metadata = {
>>>        client.repository.ModelMetaNames.NAME: 'customer satisfaction prediction model',
>>>        client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: sw_spec_id,
>>>        client.repository.ModelMetaNames.TYPE: 'spss-modeler_18.1',
>>>        client.repository.ModelMetaNames.INPUT_DATA_SCHEMA: [{'id': 'test',
>>>                                                             'type': 'list',
>>>                                                             'fields': [{'name': 'age', 'type': 'float'},
>>>                                                                        {'name': 'sex', 'type': 'float'},
>>>                                                                         {'name': 'fbs', 'type': 'float'},
>>>                                                                         {'name': 'restbp', 'type': 'float'}]
>>>                                                               },
>>>                                                               {'id': 'test2',
>>>                                                                'type': 'list',
>>>                                                                'fields': [{'name': 'age', 'type': 'float'},
>>>                                                                           {'name': 'sex', 'type': 'float'},
>>>                                                                           {'name': 'fbs', 'type': 'float'},
>>>                                                                           {'name': 'restbp', 'type': 'float'}]
>>>                                                               }]
>>>             }

store_model() method used with a local tar.gz file that contains a model:

>>> stored_model_details = client.repository.store_model(path_to_tar_gz, meta_props=metadata, training_data=None)

store_model() method used with a local directory that contains model files:

>>> stored_model_details = client.repository.store_model(path_to_model_directory, meta_props=metadata, training_data=None)

store_model() method used with the GUID of a trained model

>>> stored_model_details = client.repository.store_model(trained_model_guid, meta_props=metadata, training_data=None)

store_model() method used with a pipeline that was generated by an experiment

>>> metadata = {
>>>        client.repository.ModelMetaNames.NAME: 'customer satisfaction prediction model'
>>>}
>>> stored_model_details = client.repository.store_model(pipeline_model, meta_props=metadata, experiment_metadata=experiment_metadata)

store_pipeline(meta_props)[source]¶

Create a pipeline.

Parameters

Important

meta_props: meta data of the pipeline configuration. To see available meta names use:
```
>>> client.pipelines.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: Metadata of the pipeline createdn return type: dict

Example

>>> metadata = {
>>>  client.pipelines.ConfigurationMetaNames.NAME: 'my_training_definition',
>>>  client.pipelines.ConfigurationMetaNames.DOCUMENT: {"doc_type":"pipeline","version": "2.0","primary_pipeline": "dlaas_only","pipelines": [{"id": "dlaas_only","runtime_ref": "hybrid","nodes": [{"id": "training","type": "model_node","op": "dl_train","runtime_ref": "DL","inputs": [],"outputs": [],"parameters": {"name": "tf-mnist","description": "Simple MNIST model implemented in TF","command": "python3 convolutional_network.py --trainImagesFile ${DATA_DIR}/train-images-idx3-ubyte.gz --trainLabelsFile ${DATA_DIR}/train-labels-idx1-ubyte.gz --testImagesFile ${DATA_DIR}/t10k-images-idx3-ubyte.gz --testLabelsFile ${DATA_DIR}/t10k-labels-idx1-ubyte.gz --learningRate 0.001 --trainingIters 6000","compute": {"name": "k80","nodes": 1},"training_lib_href":"/v4/libraries/64758251-bt01-4aa5-a7ay-72639e2ff4d2/content"},"target_bucket": "wml-dev-results"}]}]}}
>>> pipeline_details = client.repository.store_pipeline(pipeline_filepath, meta_props=metadata)
>>> pipeline_href = client.repository.get_pipeline_href(pipeline_details)

store_space(meta_props)[source]¶

Create a space.

Parameters

Important

meta_props: meta data of the space configuration. To see available meta names use:
```
>>> client.spaces.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: Metadata of the space created

return type: dict

Example

>>> metadata = {
>>>  client.spaces.ConfigurationMetaNames.NAME: 'my_space'
>>> }
>>> space_details = client.repository.store_space(meta_props=metadata)
>>> space_href = client.repository.get_space_href(experiment_details)

update_experiment(experiment_uid, changes)[source]¶

Updates existing experiment metadata.

Parameters

Important

experiment_uid: Unique of Id experiment which definition should be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated experiment

return type: dict

Example

>>> metadata = {
>>> client.repository.ExperimentMetaNames.NAME:"updated_exp"
>>> }
>>> exp_details = client.repository.update_experiment(experiment_uid, changes=metadata)

update_function(function_uid, changes, update_function=None)[source]¶

Updates existing function metadata.

Parameters

Important

function_uid: Unique Id of function which define what should be updated

type: str
changes: Elements which should be changed, where keys are ConfigurationMetaNames.

type: dict
update_function: Path to file with archived function content or function which should be changed for specific function_uid. This parameters is valid only for CP4D 3.0.0.

type: str or function

Output

Important

returns: metadata of updated function

return type: dict

Example

>>> metadata = {
>>> client.repository.FunctionMetaNames.NAME:"updated_function"
>>> }
>>>
>>> function_details = client.repository.update_function(function_uid, changes=metadata)

update_model(model_uid, updated_meta_props=None, update_model=None)[source]¶

Updates existing model metadata.

Parameters

Important

model_uid: Unique id of model which definition should be updated

type: str
updated_meta_props: elements which should be changed, where keys are ConfigurationMetaNames

type: dict
update_model: archived model content file or path to directory containing archived model file which should be changed for specific model_uid. This parameters is valid only for CP4D 3.0.0.

type: object or archived model content file

Output

Important

returns: metadata of updated model

return type: dict

Example 1

>>> metadata = {
>>> client.repository.ModelMetaNames.NAME:"updated_model"
>>> }
>>> model_details = client.repository.update_model(model_uid, updated_meta_props=metadata)

Example 2

>>> metadata = {
>>> client.repository.ModelMetaNames.NAME:"updated_model"
>>> }
>>> model_details = client.repository.update_model(model_uid, updated_meta_props=metadata, update_model="newmodel_content.tar.gz")

update_pipeline(pipeline_uid, changes)[source]¶

Updates existing pipeline metadata.

Parameters

Important

pipeline_uid: Unique Id of pipeline which definition should be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated pipeline

return type: dict

Example

>>> metadata = {
>>> client.repository.PipelineMetanames.NAME:"updated_pipeline"
>>> }
>>> pipeline_details = client.repository.update_pipeline(pipeline_uid, changes=metadata)

update_space(space_uid, changes)[source]¶

Updates existing space metadata.

Parameters

Important

space_uid: Unique Id of space which definition should be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated space

return type: dict

Example

>>> metadata = {
>>> client.repository.SpacesMetaNames.NAME:"updated_space"
>>> }
>>> space_details = client.repository.update_space(space_uid, changes=metadata)

class metanames.ModelMetaNames[source]¶

Set of MetaNames for models.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`my_model`
DESCRIPTION	str	N	`my_description`
INPUT_DATA_SCHEMA	list	N	`{'id': '1', 'type': 'struct', 'fields': [{'name': 'x', 'type': 'double', 'nullable': False, 'metadata': {}}, {'name': 'y', 'type': 'double', 'nullable': False, 'metadata': {}}]}`	`{'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}`
TRAINING_DATA_REFERENCES	list	N	`[]`	`[{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'endpoint_url(required)': 'string', 'access_key_id(required)': 'string', 'secret_access_key(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}]`
TEST_DATA_REFERENCES	list	N	`[]`	`[{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'endpoint_url(required)': 'string', 'access_key_id(required)': 'string', 'secret_access_key(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}]`
OUTPUT_DATA_SCHEMA	dict	N	`{'id': '1', 'type': 'struct', 'fields': [{'name': 'x', 'type': 'double', 'nullable': False, 'metadata': {}}, {'name': 'y', 'type': 'double', 'nullable': False, 'metadata': {}}]}`	`{'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}`
LABEL_FIELD	str	N	`PRODUCT_LINE`
TRANSFORMED_LABEL_FIELD	str	N	`PRODUCT_LINE_IX`
TAGS	list	N	`['string', 'string']`	`['string', 'string']`
SIZE	dict	N	`{'in_memory': 0, 'content': 0}`	`{'in_memory(optional)': 'string', 'content(optional)': 'string'}`
PIPELINE_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
RUNTIME_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
TYPE	str	Y	`mllib_2.1`
CUSTOM	dict	N	`{}`
DOMAIN	str	N	`Watson Machine Learning`
HYPER_PARAMETERS	dict	N
METRICS	list	N
IMPORT	dict	N	`{'connection': {'endpoint_url': 'https://s3-api.us-geo.objectstorage.softlayer.net', 'access_key_id': '*', 'secret_access_key': '*'}, 'location': {'bucket': 'train-data', 'path': 'training_path'}, 'type': 's3'}`	`{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'endpoint_url(required)': 'string', 'access_key_id(required)': 'string', 'secret_access_key(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}}`
TRAINING_LIB_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
MODEL_DEFINITION_UID	str	N	`53628d6_cdee13-35d3-s8989343`
SOFTWARE_SPEC_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
TF_MODEL_PARAMS	dict	N	`{'save_format': 'None', 'signatures': 'struct', 'options': 'None', 'custom_objects': 'string'}`
FAIRNESS_INFO	dict	N	`{'favorable_labels': ['X']}`

Note: project (MetaNames.PROJECT_UID) and space (MetaNames.SPACE_UID) meta names are not supported and considered as invalid. Instead use client.set.default_space(<SPACE_GUID>) to set the space or client.set.default_project(<PROJECT_GUID>).

class metanames.ExperimentMetaNames[source]¶

Set of MetaNames for experiments.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`Hand-written Digit Recognition`
DESCRIPTION	str	N	`Hand-written Digit Recognition training`
TAGS	list	N	`[{'value': 'dsx-project.<project-guid>', 'description': 'DSX project guid'}]`	`[{'value(required)': 'string', 'description(optional)': 'string'}]`
EVALUATION_METHOD	str	N	`multiclass`
EVALUATION_METRICS	list	N	`[{'name': 'accuracy', 'maximize': False}]`	`[{'name(required)': 'string', 'maximize(optional)': 'boolean'}]`
TRAINING_REFERENCES	list	Y	`[{'pipeline': {'href': '/v4/pipelines/6d758251-bb01-4aa5-a7a3-72339e2ff4d8'}}]`	[{'pipeline(optional)': {'href(required)': 'string', 'data_bindings(optional)': [{'data_reference(required)': 'string', 'node_id(required)': 'string'}], 'nodes_parameters(optional)': [{'node_id(required)': 'string', 'parameters(required)': 'dict'}]}, 'training_lib(optional)': {'href(required)': 'string', 'compute(optional)': {'name(required)': 'string', 'nodes(optional)': 'number'}, 'runtime(optional)': {'href(required)': 'string'}, 'command(optional)': 'string', 'parameters(optional)': 'dict'}}]
SPACE_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
LABEL_COLUMN	str	N	`label`
CUSTOM	dict	N	`{'field1': 'value1'}`

class metanames.FunctionMetaNames[source]¶

Set of MetaNames for AI functions.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`ai_function`
DESCRIPTION	str	N	`This is ai function`
RUNTIME_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
SOFTWARE_SPEC_UID	str	N	`53628d69-ced9-4f43-a8cd-9954344039a8`
INPUT_DATA_SCHEMAS	list	N	`[{'id': '1', 'type': 'struct', 'fields': [{'name': 'x', 'type': 'double', 'nullable': False, 'metadata': {}}, {'name': 'y', 'type': 'double', 'nullable': False, 'metadata': {}}]}]`	`[{'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}]`
OUTPUT_DATA_SCHEMAS	list	N	`[{'id': '1', 'type': 'struct', 'fields': [{'name': 'multiplication', 'type': 'double', 'nullable': False, 'metadata': {}}]}]`	`[{'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}]`
TAGS	list	N	`[{'value': 'ProjectA', 'description': 'Functions created for ProjectA'}]`	`[{'value(required)': 'string', 'description(optional)': 'string'}]`
TYPE	str	N	`python`
CUSTOM	dict	N	`{}`
SAMPLE_SCORING_INPUT	list	N	`{'input_data': [{'fields': ['name', 'age', 'occupation'], 'values': [['john', 23, 'student'], ['paul', 33, 'engineer']]}]}`	`{'id(optional)': 'string', 'fields(optional)': 'array', 'values(optional)': 'array'}`
SPACE_UID	str	N	`3628d69-ced9-4f43-a8cd-9954344039a8`

class metanames.PipelineMetanames[source]¶

Set of MetaNames for pipelines.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`Hand-written Digit Recognitionu`
DESCRIPTION	str	N	`Hand-written Digit Recognition training`
SPACE_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
TAGS	list	N	`[{'value': 'dsx-project.<project-guid>', 'description': 'DSX project guid'}]`	`[{'value(required)': 'string', 'description(optional)': 'string'}]`
DOCUMENT	dict	N	{'doc_type': 'pipeline', 'version': '2.0', 'primary_pipeline': 'dlaas_only', 'pipelines': [{'id': 'dlaas_only', 'runtime_ref': 'hybrid', 'nodes': [{'id': 'training', 'type': 'model_node', 'op': 'dl_train', 'runtime_ref': 'DL', 'inputs': [], 'outputs': [], 'parameters': {'name': 'tf-mnist', 'description': 'Simple MNIST model implemented in TF', 'command': 'python3 convolutional_network.py --trainImagesFile ${DATA_DIR}/train-images-idx3-ubyte.gz --trainLabelsFile ${DATA_DIR}/train-labels-idx1-ubyte.gz --testImagesFile ${DATA_DIR}/t10k-images-idx3-ubyte.gz --testLabelsFile ${DATA_DIR}/t10k-labels-idx1-ubyte.gz --learningRate 0.001 --trainingIters 6000', 'compute': {'name': 'k80', 'nodes': 1}, 'training_lib_href': '/v4/libraries/64758251-bt01-4aa5-a7ay-72639e2ff4d2/content'}, 'target_bucket': 'wml-dev-results'}]}]}	`{'doc_type(required)': 'string', 'version(required)': 'string', 'primary_pipeline(required)': 'string', 'pipelines(required)': [{'id(required)': 'string', 'runtime_ref(required)': 'string', 'nodes(required)': [{'id': 'string', 'type': 'string', 'inputs': 'list', 'outputs': 'list', 'parameters': {'training_lib_href': 'string'}}]}]}`
CUSTOM	dict	N	`{'field1': 'value1'}`
IMPORT	dict	N	`{'connection': {'endpoint_url': 'https://s3-api.us-geo.objectstorage.softlayer.net', 'access_key_id': '*', 'secret_access_key': '*'}, 'location': {'bucket': 'train-data', 'path': 'training_path'}, 'type': 's3'}`	`{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'endpoint_url(required)': 'string', 'access_key_id(required)': 'string', 'secret_access_key(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}}`
RUNTIMES	list	N	`[{'id': 'id', 'name': 'tensorflow', 'version': '1.13-py3'}]`
COMMAND	str	N	`convolutional_network.py --trainImagesFile train-images-idx3-ubyte.gz --trainLabelsFile train-labels-idx1-ubyte.gz --testImagesFile t10k-images-idx3-ubyte.gz --testLabelsFile t10k-labels-idx1-ubyte.gz --learningRate 0.001 --trainingIters 6000`
LIBRARY_UID	str	N	`fb9752c9-301a-415d-814f-cf658d7b856a`
COMPUTE	dict	N	`{'name': 'k80', 'nodes': 1}`

Script ¶

class client.Script(client)[source]¶

Store and manage your scripts assets.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.ScriptMetaNames object>¶: MetaNames for script Assets creation.

create_revision(script_uid)[source]¶

Creates revision for the given script. Revisions are immutable once created. The metadata and attachment at script_uid is taken and a revision is created out of it

Parameters

Important

script_uid: Script ID. Mandatory.

type: str

** Output**

Important

returns: Stored script revisions metadata.

return type: dict

Example

>>> script_revision = client.scripts.create_revision(script_uid)

delete(asset_uid)[source]¶

Delete a stored script asset.

Parameters

Important

asset_uid: Unique Id of script asset

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.script.delete(asset_uid)

download(asset_uid, filename, rev_uid=None)[source]¶

Download the content of a script asset.

Parameters

Important

asset_uid: The Unique Id of the script asset to be downloaded

type: str
filename: filename to be used for the downloaded file

type: str
rev_uid: Revision id

type: str

Output

Important

returns: Path to the downloaded asset content

return type: str

Example

>>> client.script.download(asset_uid,"script_file")

get_details(script_uid=None)[source]¶

Get script asset details. If no script_uid is passed, details for all script assets will be returned.

Parameters

Important

script_uid: Unique id of script

type: str

Output

Important

returns: Metadata of the stored script asset

return type: dict

Example

>>> script_details = client.scripts.get_details(script_uid)

static get_href(asset_details)[source]¶

Get url of stored scripts asset.

Parameters

Important

asset_details: stored script details

type: dict

Output

Important

returns: href of stored script asset

return type: str

Example

>>> asset_details = client.script.get_details(asset_uid)
>>> asset_href = client.script.get_href(asset_details)

static get_id(asset_details)[source]¶

Get Unique Id of stored script asset.

Parameters

Important

asset_details: Metadata of the stored script asset

type: dict

Output

Important

returns: Unique Id of stored script asset

return type: str

Example

>>> asset_uid = client.script.get_id(asset_details)

get_revision_details(script_uid=None, rev_uid=None)[source]¶

Get metadata of script_uid revision.

Paramaters

Important

script_uid: Script ID. Mandatory.

type: str
rev_uid: Revision ID. If this parameter is not provided, returns latest revision if existing else error.

type: int

Output

Important

returns: Stored script(s) metadata

return type: dict

Example

>>> script_details = client.scripts.get_revision_details(script_uid, rev_uid)

static get_uid(asset_details)[source]¶

Get Unique Id of stored script asset. This method is deprecated. Use ‘get_id(asset_details)’ instead

Parameters

Important

asset_details: Metadata of the stored script asset

type: dict

Output

Important

returns: Unique Id of stored script asset

return type: str

Example

>>> asset_uid = client.script.get_uid(asset_details)

list(limit=None)[source]¶

List stored scripts. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all script in a table format.

return type: None

Example

>>> client.script.list()

list_revisions(script_uid, limit=None)[source]¶

List all revisions for the given script uid.

Parameters

Important

script_uid: Stored script ID.

type: str
limit: Limit number of fetched records (optional).

type: int

Output

Important

This method only prints the list of all script in a table format.

return type: None

Example

>>> client.scripts.list_revisions(script_uid)

store(meta_props, file_path)[source]¶

Creates a Scripts asset and uploads content to it.

Parameters

Important

meta_props: Name to be given to the Scripts asset

type: str
file_path: Path to the content file to be uploaded

type: str

Output

Important

returns: metadata of the stored Scripts asset

return type: dict

Example

>>> metadata = {
>>>        client.script.ConfigurationMetaNamess.NAME: 'my first script',
>>>        client.script.ConfigurationMetaNames.DESCRIPTION: 'description of the script',
>>>        client.script.ConfigurationMetaNames.SOFTWARE_SPEC_UID: '0cdb0f1e-5376-4f4d-92dd-da3b69aa9bda'
>>>    }
>>>
>>> asset_details = client.scripts.store(meta_props=metadata,file_path="/path/to/file")

update(script_uid, meta_props=None, file_path=None)[source]¶

Update script with either metadata or attachment or both.

Parameters

Important

param script_uid: Script UID.

type: str

Output

Important

returns: Updated metadata of script.

return type: dict

Example

>>> script_details = client.script.update(model_uid, meta, content_path)

class metanames.ScriptMetaNames[source]¶

Set of MetaNames for Script Specifications.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`Python script`
DESCRIPTION	str	N	`my_description`
SOFTWARE_SPEC_UID	str	Y	`53628d69-ced9-4f43-a8cd-9954344039a8`

Service instance ¶

Set ¶

class client.Set(client)[source]¶

Set a space_id/project_id to be used in the subsequent actions.

default_project(project_id)[source]¶

Set a project ID.

Parameters

Important

project_id: GUID of the project

type: str

Output

Important

returns: “SUCCESS”

return type: str

Example

>>>  client.set.default_project(project_id)

default_space(space_uid)[source]¶

Set a space ID.

Parameters

Important

space_uid: GUID of the space to be used:

type: str

Output

Important

returns: The space that is set here is used for subsequent requests.

return type: str(“SUCCESS”/”FAILURE”)

Example

>>>  client.set.default_space(space_uid)

Spaces ¶

class client.PlatformSpaces(client)[source]¶

Store and manage your spaces

MemberMetaNames = <ibm_watson_machine_learning.metanames.SpacesPlatformMemberMetaNames object>¶: MetaNames for spaces creation.

create_member(space_id, meta_props)[source]¶

Create a member within a space.

Parameters

Important

meta_props: meta data of the member configuration. To see available meta names use:
```
>>> client.spaces.MemberMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the stored member

return type: dict

Note

‘role’ can be any one of the following “viewer, editor, admin”
‘type’ can be any one of the following “user,service”
‘id’ can be either service-ID or IAM-userID

Example

>>> metadata = {
>>>  client.spaces.MemberMetaNames.MEMBERS: [{"id":"IBMid-100000DK0B", "type": "user", "role": "admin" }]
>>> }
>>> members_details = client.spaces.create_member(space_id=space_id, meta_props=metadata)

>>> metadata = {
>>>  client.spaces.MemberMetaNames.MEMBERS: [{"id":"iam-ServiceId-5a216e59-6592-43b9-8669-625d341aca71", "type": "service", "role": "admin" }]
>>> }
>>> members_details = client.spaces.create_member(space_id=space_id, meta_props=metadata)

delete(space_id)[source]¶

Delete a stored space.

Parameters

Important

space_uid: space ID

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.spaces.delete(space_id)

delete_member(space_id, member_id)[source]¶

Delete a member associated with a space.

Parameters

Important

space_id: space UID

type: str
member_id: member UID

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.spaces.delete_member(space_id,member_id)

get_details(space_id=None, limit=None, asynchronous=False, get_all=False)[source]¶

Get metadata of stored space(s)

Parameters

Important

space_id: Space ID

type: str
limit: Applicable when space_id is not provided. If space_id is provided, this will be ignored

type: str
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

returns: metadata of stored space(s)

return type: dict

Example

>>> space_details = client.spaces.get_details(space_uid)
>>> space_details = client.spaces.get_details(limit=100)
>>> space_details = client.spaces.get_details(limit=100, get_all=True)
>>> space_details = []
>>> for entry in client.spaces.get_details(limit=100, asynchronous=True, get_all=True):
>>>    space_details.extend(entry)

static get_id(space_details)[source]¶

Get space_id from space details.

Parameters

Important

space_details: Metadata of the stored space

type: dict

Output

Important

returns: space ID

return type: str

Example

>>> space_details = client.spaces.store(meta_props)
>>> space_id = client.spaces.get_id(space_details)

get_member_details(space_id, member_id)[source]¶

Get metadata of member associated with a space

Parameters

Important

space_id: member ID

type: str

Output

Important

returns: metadata of member of a space

return type: dict

Example

>>> member_details = client.spaces.get_member_details(space_uid,member_id)

static get_uid(space_details)[source]¶

Get Unique Id of the space. This method is deprecated. Use ‘get_id(space_details)’ instead

Parameters

Important

asset_details: Metadata of the space

type: dict

type: dict

Output

Important

returns: Unique Id of space

return type: str

Example

>>> space_details = client.spaces.store(meta_props)
>>> space_uid = client.spaces.get_uid(space_details)

list(limit=None, member=None, roles=None)[source]¶

List stored spaces. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int
member: Filters the result list to only include spaces where the user with a matching user id
is a member

type: string
roles: limit number of fetched records

type: string

Output

Important

This method only prints the list of all spaces in a table format.

return type: None

Example

>>> client.spaces.list()

list_members(space_id, limit=None, identity_type=None, role=None, state=None)[source]¶

List stored members of a space. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records

type: int
identity_type: Find the member by type

type: string
role: Find the member by role

type: string
state: Find the member by state

type: string

Output

Important

This method only prints the list of all members associated with a space in a table format.

return type: None

Example

>>> client.spaces.list_members(space_id)

store(meta_props, background_mode=True)[source]¶

Create a space. The instance associated with the space via COMPUTE will be used for billing purposes on cloud. Note that STORAGE and COMPUTE are applicable only for cloud

Parameters

Important

meta_props: meta data of the space configuration. To see available meta names use:
```
>>> client.spaces.ConfigurationMetaNames.get()
```

type: dict

background_mode: Indicator if store() method will run in background (async) or (sync). Default: True

type: bool

Output

Important

returns: metadata of the stored space

return type: dict

Example

>>> metadata = {
>>>  client.spaces.ConfigurationMetaNames.NAME: 'my_space',
>>>  client.spaces.ConfigurationMetaNames.DESCRIPTION: 'spaces',
>>>  client.spaces.ConfigurationMetaNames.STORAGE: {"resource_crn": "provide crn of the COS storage"},
>>>  client.spaces.ConfigurationMetaNames.COMPUTE: {"name": "test_instance",
>>>                                                 "crn": "provide crn of the instance"}
>>> }
>>> spaces_details = client.spaces.store(meta_props=metadata)

update(space_id, changes)[source]¶

Updates existing space metadata. ‘STORAGE’ cannot be updated. STORAGE and COMPUTE are applicable only for cloud.

Parameters

Important

space_uid: ID of space which definition should be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated space

return type: dict

Example

>>> metadata = {
>>> client.spaces.ConfigurationMetaNames.NAME:"updated_space",
>>> client.spaces.ConfigurationMetaNames.COMPUTE: {"name": "test_instance",
>>>                                                "crn": "v1:staging:public:pm-20-dev:us-south:a/09796a1b4cddfcc9f7fe17824a68a0f8:f1026e4b-77cf-4703-843d-c9984eac7272::"
>>>                                               }
>>> }
>>> space_details = client.spaces.update(space_id, changes=metadata)

update_member(space_id, member_id, changes)[source]¶

Updates existing member metadata.

Parameters

Important

space_id: ID of space

type: str
member_id: ID of member that needs to be updated

type: str
changes: elements which should be changed, where keys are ConfigurationMetaNames

type: dict

Output

Important

returns: metadata of updated member

return type: dict

Example

>>> metadata = {
>>>  client.spaces.MemberMetaNames.MEMBER: {"role": "editor"}
>>> }
>>> member_details = client.spaces.update_member(space_id, member_id, changes=metadata)

class metanames.SpacesPlatformMetaNames[source]¶

Set of MetaNames for Platform Spaces Specs.

Available MetaNames:

MetaName	Type	Required	Schema
NAME	str	Y	`my_space`
DESCRIPTION	str	N	`my_description`
STORAGE	dict	N	`{'type': 'bmcos_object_storage', 'resource_crn': '', 'delegated(optional)': 'false'}`
COMPUTE	dict	N	`{'name': 'name', 'crn': 'crn of the instance'}`

class metanames.SpacesPlatformMemberMetaNames[source]¶

Set of MetaNames for Platform Spaces Member Specs.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
MEMBERS	list	N	`[{'id': 'iam-id1', 'role': 'editor', 'type': 'user', 'state': 'active'}, {'id': 'iam-id2', 'role': 'viewer', 'type': 'user', 'state': 'active'}]`	`[{'id(required)': 'string', 'role(required)': 'string', 'type(required)': 'string', 'state(optional)': 'string'}]`
MEMBER	dict	N	`{'id': 'iam-id1', 'role': 'editor', 'type': 'user', 'state': 'active'}`

Software specifications ¶

class client.SwSpec(client)[source]¶

Store and manage your software specs.

ConfigurationMetaNames = <ibm_watson_machine_learning.metanames.SwSpecMetaNames object>¶: MetaNames for Software Specification creation.

add_package_extension(sw_spec_uid, pkg_extn_id)[source]¶

Add a package extension to software specifications existing metadata.

Parameters

Important

sw_spec_uid: Unique Id of software specification which should be updated

type: str
pkg_extn_id: Unique Id of package extension which should needs to added to software specification

type: str

Example

>>> client.software_specifications.add_package_extension(sw_spec_uid, pkg_extn_id)

delete(sw_spec_uid)[source]¶

Delete a software specification.

Parameters

Important

sw_spec_uid: Unique Id of software specification

type: str

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.software_specifications.delete(sw_spec_uid)

delete_package_extension(sw_spec_uid, pkg_extn_id)[source]¶

Delete a package extension from software specifications existing metadata.

Parameters

Important

sw_spec_uid: Unique Id of software specification which should be updated

type: str
pkg_extn_id: Unique Id of package extension which should needs to deleted from software specification

type: str

Example

>>> client.software_specifications.delete_package_extension(sw_spec_uid, pkg_extn_id)

get_details(sw_spec_uid=None)[source]¶

Get software specification details. If no sw_spec_id is passed, details for all software specifications will be returned.

Parameters

Important

returns: sw_spec UID

return type: str

Output

Important

sw_spec_details: Metadata of the stored sw_spec

type: dict

Example

>>> sw_spec_details = client.software_specifications.get_details(sw_spec_uid)

static get_href(sw_spec_details)[source]¶

Get url of software specification.

Parameters

Important

sw_spec_details: software specification details

type: dict

Output

Important

returns: href of software specification

return type: str

Example

>>> sw_spec_details = client.software_specifications.get_details(sw_spec_uid)
>>> sw_spec_href = client.software_specifications.get_href(sw_spec_details)

static get_id(sw_spec_details)[source]¶

Get Unique Id of software specification.

Parameters

Important

sw_spec_details: Metadata of the software specification

type: dict

Output

Important

returns: Unique Id of software specification

return type: str

Example

>>> asset_uid = client.software_specifications.get_id(sw_spec_details)

get_id_by_name(sw_spec_name)[source]¶

Get Unique Id of software specification.

Parameters

Important

sw_spec_name: Name of the software specification

type: str

Output

Important

returns: Unique Id of software specification

return type: str

Example

>>> asset_uid = client.software_specifications.get_id_by_name(sw_spec_name)

static get_uid(sw_spec_details)[source]¶

Get Unique Id of software specification. Deprecated!! Use get_id(sw_spec_details) instead

Parameters

Important

sw_spec_details: Metadata of the software specification

type: dict

Output

Important

returns: Unique Id of software specification

return type: str

Example

>>> asset_uid = client.software_specifications.get_uid(sw_spec_details)

get_uid_by_name(sw_spec_name)[source]¶

Get Unique Id of software specification. Deprecated!! Use get_id_by_name(self, sw_spec_name) instead

Parameters

Important

sw_spec_name: Name of the software specification

type: str

Output

Important

returns: Unique Id of software specification

return type: str

Example

>>> asset_uid = client.software_specifications.get_uid_by_name(sw_spec_name)

list(limit=None)[source]¶

List software specifications.

Parameters

Important

limit: limit number of fetched records

type: int

Output

Important

This method only prints the list of all software specifications in a table format.

return type: None

Example

>>> client.software_specifications.list()

store(meta_props)[source]¶

Create a software specification.

Parameters

Important

meta_props: meta data of the space configuration. To see available meta names use:
```
>>> client.software_specifications.ConfigurationMetaNames.get()
```
type: dict

Output

Important

returns: metadata of the stored space

return type: dict

Example

>>> meta_props = {
>>>    client.software_specifications.ConfigurationMetaNames.NAME: "skl_pipeline_heart_problem_prediction",
>>>    client.software_specifications.ConfigurationMetaNames.DESCRIPTION: "description scikit-learn_0.20",
>>>    client.software_specifications.ConfigurationMetaNames.PACKAGE_EXTENSIONS_UID: [],
>>>    client.software_specifications.ConfigurationMetaNames.SOFTWARE_CONFIGURATIONS: {},
>>>    client.software_specifications.ConfigurationMetaNames.BASE_SOFTWARE_SPECIFICATION_ID: "guid"
>>> }

class metanames.SwSpecMetaNames[source]¶

Set of MetaNames for Software Specifications Specs.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
NAME	str	Y	`Python 3.6 with pre-installed ML package`
DESCRIPTION	str	N	`my_description`
PACKAGE_EXTENSIONS	list	N	`[{'guid': 'value'}]`
SOFTWARE_CONFIGURATION	dict	N	`{'platform': {'name': 'python', 'version': '3.6'}}`	`{'platform(required)': 'string'}`
BASE_SOFTWARE_SPECIFICATION	dict	Y	`{'guid': 'BASE_SOFTWARE_SPECIFICATION_ID'}`

Training ¶

class client.Training(client)[source]¶

Train new models.

cancel(training_uid, hard_delete=False)[source]¶

Cancel a training which is currently running and remove it. This method is also be used to delete metadata details of the completed or canceled training run when hard_delete parameter is set to True.

Parameters

Important

training_uid: Training UID

type: str
hard_delete: specify True or False. True - To delete the completed or canceled training runs. False - To cancel the currently running training run. Default value is False. type: Boolean

Output

Important

returns: status (“SUCCESS” or “FAILED”)

return type: str

Example

>>> client.training.cancel(training_uid)

get_details(training_uid=None, limit=None, asynchronous=False, get_all=False, training_type=None, state=None, tag_value=None, training_definition_id=None, _internal=False)[source]¶

Get metadata of training(s). If training_uid is not specified returns all model spaces metadata.

Parameters

Important

training_uid: Unique Id of Training (optional)

type: str
limit: limit number of fetched records (optional)

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool
training_type: Filter the fetched list of trainings based on training type
[pipeline or experiment] (optional)

type: str
state: Filter the fetched list of training based on their state:
[queued, running, completed, failed] (optional)

type: str
tag_value: Filter the fetched list of training based on ther tag value (optional)

type: str
training_definition_id: Filter the fetched trainings which are using the given training_definition
(optional)

type: str

Output

Important

returns: metadata of training(s)

return type: dict

The output can be {“resources”: [dict]} or a dict

Note

If training_uid is not specified, all trainings metadata is fetched

Example

>>> training_run_details = client.training.get_details(training_uid)
>>> training_runs_details = client.training.get_details()
>>> training_runs_details = client.training.get_details(limit=100)
>>> training_runs_details = client.training.get_details(limit=100, get_all=True)
>>> training_runs_details = []
>>> for entry in client.training.get_details(limit=100, asynchronous=True, get_all=True):
>>>    training_runs_details.extend(entry)

static get_href(training_details)[source]¶

Get training_href from training details.

Parameters

Important

training_details: Metadata of the training created

type: dict

Output

Important

returns: training href

return type: str

Example

>>> training_details = client.training.get_details(training_uid)
>>> run_url = client.training.get_href(training_details)

static get_id(training_details)[source]¶

Get training_id from training details.

Parameters

Important

training_details: Metadata of the training created

type: dict

Output

Important

returns: Unique id of training

return type: str

Example

>>> training_details = client.training.get_details(training_id)
>>> training_id = client.training.get_id(training_details)

get_metrics(training_uid)[source]¶

Get metrics.

Parameters

Important

training_uid: training UID

type: str

Output

Important

returns: Metrics of a training run

return type: list of dict

Example

>>> training_status = client.training.get_metrics(training_uid)

get_status(training_uid)[source]¶

Get the status of a training created.

Parameters

Important

training_uid: training UID

type: str

Output

Important

returns: training_status

return type: dict

Example

>>> training_status = client.training.get_status(training_uid)

static get_uid(training_details)[source]¶: This method is deprecated, please use get_id() instead.”

list(limit=None, asynchronous=False, get_all=False)[source]¶

List stored trainings. If limit is set to None there will be only first 50 records shown.

Parameters

Important

limit: limit number of fetched records at once

type: int
asynchronous: if True, it will work as a generator (optional)

type: bool
get_all: if True, it will get all entries in ‘limited’ chunks (optional)

type: bool

Output

Important

This method only prints the list of all trainings in a table format.

return type: None

Example

>>> client.training.list()
>>> training_runs_df = client.training.list(limit=100)
>>> training_runs_df = client.training.list(limit=100, get_all=True)
>>> training_runs_df = []
>>> for entry in client.training.list(limit=100, asynchronous=True, get_all=True):
>>>    training_runs_df.extend(entry)

list_intermediate_models(training_uid)[source]¶

List the intermediate_models.

Parameters

Important

training_uid: Training ID

type: str

Output

Important

This method only prints the list of all intermediate_models associated with an AUTOAI training in a table format.

return type: None

Note

This method prints the training logs. This method is not supported for IBM Cloud Pak® for Data.

Example

>>> client.training.list_intermediate_models()

list_subtrainings(training_uid)[source]¶

List the sub-trainings.

Parameters

Important

training_uid: Training ID

type: str

Output

Important

This method only prints the list of all sub-trainings associated with a training in a table format.

return type: None

Example

>>> client.training.list_subtrainings()

monitor_logs(training_uid)[source]¶

Monitor the logs of a training created.

Parameters

Important

training_uid: Training UID

type: str

Output

Important

returns: None

return type: None

Note

This method prints the training logs. This method is not supported for IBM Cloud Pak® for Data.

Example

>>> client.training.monitor_logs(training_uid)

monitor_metrics(training_uid)[source]¶

Monitor the metrics of a training created.

Parameters

Important

training_uid: Training UID

type: str

Output

Important

returns: None

return type: None

Note

This method prints the training metrics. This method is not supported for IBM Cloud Pak® for Data.

Example

>>> client.training.monitor_metrics(training_uid)

run(meta_props, asynchronous=True)[source]¶

Create a new Machine Learning training.

Parameters

Important

meta_props: meta data of the training configuration. To see available meta names use:
```
>>> client.training.ConfigurationMetaNames.show()
```
type: str
asynchronous:
- True - training job is submitted and progress can be checked later.
- False - method will wait till job completion and print training stats.
type: bool

Output

Important

returns: Metadata of the training created

return type: dict

Examples

Example meta_props for Training run creation in IBM Cloud Pak® for Data version 3.0.1 or above:

>>> metadata = {
>>>  client.training.ConfigurationMetaNames.NAME: 'Hand-written Digit Recognition',
>>>  client.training.ConfigurationMetaNames.DESCRIPTION: 'Hand-written Digit Recognition Training',
>>>  client.training.ConfigurationMetaNames.PIPELINE: {
>>>                "id": "4cedab6d-e8e4-4214-b81a-2ddb122db2ab",
>>>                "rev": "12",
>>>                "model_type": "string",
>>>                "data_bindings": [
>>>                  {
>>>                    "data_reference_name": "string",
>>>                    "node_id": "string"
>>>                  }
>>>                ],
>>>                "nodes_parameters": [
>>>                  {
>>>                    "node_id": "string",
>>>                    "parameters": {}
>>>                  }
>>>                ],
>>>                "hardware_spec": {
>>>                  "id": "4cedab6d-e8e4-4214-b81a-2ddb122db2ab",
>>>                  "rev": "12",
>>>                  "name": "string",
>>>                  "num_nodes": "2"
>>>                }
>>>      },
>>>  client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCES: [{
>>>          'type': 's3',
>>>          'connection': {},
>>>          'location': {
>>>              'href': 'v2/assets/asset1233456',
>>>          }
>>>          "schema": "{ "id": "t1", "name": "Tasks", "fields": [ { "name": "duration", "type": "number" } ]}"
>>>      }],
>>> client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: {
>>>          'id' : 'string',
>>>          'connection': {
>>>              'endpoint_url': 'https://s3-api.us-geo.objectstorage.service.networklayer.com',
>>>              'access_key_id': '***',
>>>              'secret_access_key': '***'
>>>          },
>>>          'location': {
>>>              'bucket': 'wml-dev-results',
>>>               'path' : "path"
>>>          }
>>>          'type': 's3'
>>>      }
>>>   }

NOTE: You can provide either one of the below values can be provided for training:

client.training.ConfigurationMetaNames.EXPERIMENT
client.training.ConfigurationMetaNames.PIPELINE
client.training.ConfigurationMetaNames.MODEL_DEFINITION

Example meta_prop values for training run creation in other versions:

>>> metadata = {
>>>  client.training.ConfigurationMetaNames.NAME: 'Hand-written Digit Recognition',
>>>  client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCES: [{
>>>          'connection': {
>>>              'endpoint_url': 'https://s3-api.us-geo.objectstorage.service.networklayer.com',
>>>              'access_key_id': '***',
>>>              'secret_access_key': '***'
>>>          },
>>>          'source': {
>>>              'bucket': 'wml-dev',
>>>          }
>>>          'type': 's3'
>>>      }],
>>> client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: {
>>>          'connection': {
>>>              'endpoint_url': 'https://s3-api.us-geo.objectstorage.service.networklayer.com',
>>>              'access_key_id': '***',
>>>              'secret_access_key': '***'
>>>          },
>>>          'target': {
>>>              'bucket': 'wml-dev-results',
>>>          }
>>>          'type': 's3'
>>>      },
>>> client.training.ConfigurationMetaNames.PIPELINE_UID : "/v4/pipelines/<PIPELINE-ID>"
>>> }
>>> training_details = client.training.run(definition_uid, meta_props=metadata)
>>> training_uid = client.training.get_id(training_details)

Example of a Federated Learning training job >>> aggregator_metadata = { >>> wml_client.training.ConfigurationMetaNames.NAME: ‘Federated_Learning_Tensorflow_MNIST’, >>> wml_client.training.ConfigurationMetaNames.DESCRIPTION: ‘MNIST digit recognition with Federated Learning using Tensorflow’, >>> wml_client.training.ConfigurationMetaNames.TRAINING_DATA_REFERENCES: [], >>> wml_client.training.ConfigurationMetaNames.TRAINING_RESULTS_REFERENCE: { >>> ‘type’: results_type, >>> ‘name’: ‘outputData’, >>> ‘connection’: {}, >>> ‘location’: { >>> ‘path’: ‘/projects/’ + PROJECT_ID + ‘/assets/trainings/’ >>> } >>> >>> }, >>> wml_client.training.ConfigurationMetaNames.FEDERATED_LEARNING: { >>> ‘model’: { >>> ‘type’: ‘tensorflow’, >>> ‘spec’: { >>> ‘id’: untrained_model_id >>> }, >>> ‘model_file’: untrained_model_name >>> }, >>> ‘fusion_type’: ‘iter_avg’, >>> ‘metrics’: ‘accuracy’, >>> ‘epochs’: 3, >>> ‘rounds’: 10, >>> ‘remote_training’ : { >>> ‘quorum’: 1.0, >>> ‘max_timeout’: 3600, >>> ‘remote_training_systems’: [ { ‘id’: prime_rts_id }, { ‘id’: nonprime_rts_id} ] >>> }, >>> ‘hardware_spec’: { >>> ‘name’: ‘S’ >>> }, >>> ‘software_spec’: { >>> ‘name’: ‘runtime-22.1-py3.9’ >>> } >>> } >>> } >>> >>> >>> aggregator = wml_client.training.run(aggregator_metadata, asynchronous=True) >>> aggregator_id = wml_client.training.get_id(aggregator)

class metanames.TrainingConfigurationMetaNames[source]¶

Set of MetaNames for trainings.

Available MetaNames:

MetaName	Type	Required	Example value	Schema
TRAINING_DATA_REFERENCES	list	Y	`[{'connection': {'href': '/v2/connections/2d07a6b4-8fa9-43ab-91c8-befcd9dab8d2?space_id=440ada9b-af87-4da8-a9fa-a5450825e260'}, 'location': {'bucket': 'train-data', 'path': 'training_path'}, 'type': 'connection_asset', 'schema': {'id': '1', 'fields': [{'name': 'x', 'type': 'double', 'nullable': 'False'}]}}]`	`[{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'href(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}]`
TRAINING_RESULTS_REFERENCE	dict	Y	`{'connection': {'href': '/v2/connections/2d07a6b4-8fa9-43ab-91c8-befcd9dab8d2?space_id=440ada9b-af87-4da8-a9fa-a5450825e260'}, 'location': {'bucket': 'test-results', 'path': 'training_path'}, 'type': 'connection_asset'}`	`{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'href(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}}`
TEST_DATA_REFERENCES	list	N	`[{'connection': {'href': '/v2/connections/2d07a6b4-8fa9-43ab-91c8-befcd9dab8d2?space_id=440ada9b-af87-4da8-a9fa-a5450825e260'}, 'location': {'bucket': 'train-data', 'path': 'training_path'}, 'type': 'connection_asset', 'schema': {'id': '1', 'fields': [{'name': 'x', 'type': 'double', 'nullable': 'False'}]}}]`	`[{'name(optional)': 'string', 'type(required)': 'string', 'connection(required)': {'href(required)': 'string'}, 'location(required)': {'bucket': 'string', 'path': 'string'}, 'schema(optional)': {'id(required)': 'string', 'fields(required)': [{'name(required)': 'string', 'type(required)': 'string', 'nullable(optional)': 'string'}]}}]`
TAGS	list	N	`[{'value': 'string', 'description': 'string'}]`	`[{'value(required)': 'string', 'description(optional)': 'string'}]`
PIPELINE_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
EXPERIMENT_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
PIPELINE_DATA_BINDINGS	str	N	`[{'data_reference_name': 'string', 'node_id': 'string'}]`	`[{'data_reference_name(required)': 'string', 'node_id(required)': 'string'}]`
PIPELINE_NODE_PARAMETERS	dict	N	`[{'node_id': 'string', 'parameters': {}}]`	`[{'node_id(required)': 'string', 'parameters(required)': 'dict'}]`
SPACE_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
TRAINING_LIB	dict	N	`{'href': '/v4/libraries/3c1ce536-20dc-426e-aac7-7284cf3befc6', 'compute': {'name': 'k80', 'nodes': 0}, 'runtime': {'href': '/v4/runtimes/3c1ce536-20dc-426e-aac7-7284cf3befc6'}, 'command': 'python3 convolutional_network.py', 'parameters': {}}`	`{'href(required)': 'string', 'type(required)': 'string', 'runtime(optional)': {'href': 'string'}, 'command(optional)': 'string', 'parameters(optional)': 'dict'}`
TRAINING_LIB_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
TRAINING_LIB_MODEL_TYPE	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
TRAINING_LIB_RUNTIME_UID	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
TRAINING_LIB_PARAMETERS	dict	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
COMMAND	str	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
COMPUTE	dict	N	`3c1ce536-20dc-426e-aac7-7284cf3befc6`
PIPELINE_MODEL_TYPE	str	N	`tensorflow_1.1.3-py3`

AutoAI ¶

This version of ibm-watson-machine-learning client introduces support for AutoAI Experiments.

AutoAI experiment ¶

AutoAI ¶

class ibm_watson_machine_learning.experiment.autoai.autoai.AutoAI(wml_credentials: Optional[Union[dict, ibm_watson_machine_learning.workspace.workspace.WorkSpace]] = None, project_id: Optional[str] = None, space_id: Optional[str] = None, verify=None)[source]¶

Bases: ibm_watson_machine_learning.experiment.base_experiment.base_experiment.BaseExperiment

AutoAI class for pipeline models optimization automation.

wml_credentials: dictionary, required: Credentials to Watson Machine Learning instance.
project_id: str, optional: ID of the Watson Studio project.
space_id: str, optional: ID of the Watson Studio Space.
verify: Union[bool, str], optional: User can pass verify the path to a CA_BUNDLE file or directory with certificates of trusted CAs. If set to True, default path to truststore will be taken. If set to False, no verification will be made.

>>> from ibm_watson_machine_learning.experiment import AutoAI
>>>
>>> experiment = AutoAI(
>>>        wml_credentials={
>>>              "apikey": "...",
>>>              "iam_apikey_description": "...",
>>>              "iam_apikey_name": "...",
>>>              "iam_role_crn": "...",
>>>              "iam_serviceid_crn": "...",
>>>              "instance_id": "...",
>>>              "url": "https://us-south.ml.cloud.ibm.com"
>>>            },
>>>         project_id="...",
>>>         space_id="...")

class ClassificationAlgorithms(value)¶

Bases: enum.Enum

Classification algorithms that AutoAI could use for IBM Cloud.

DT = 'DecisionTreeClassifier'¶

EX_TREES = 'ExtraTreesClassifier'¶

GB = 'GradientBoostingClassifier'¶

LGBM = 'LGBMClassifier'¶

LR = 'LogisticRegression'¶

RF = 'RandomForestClassifier'¶

SnapBM = 'SnapBoostingMachineClassifier'¶

SnapDT = 'SnapDecisionTreeClassifier'¶

SnapLR = 'SnapLogisticRegression'¶

SnapRF = 'SnapRandomForestClassifier'¶

SnapSVM = 'SnapSVMClassifier'¶

XGB = 'XGBClassifier'¶

class DataConnectionTypes¶

Bases: object

Supported types of DataConnection. OneOf: [s3, FS]

CA = 'connection_asset'¶

CN = 'container'¶

DS = 'data_asset'¶

FS = 'fs'¶

S3 = 's3'¶

class ForecastingAlgorithms(value)¶

Bases: enum.Enum

Forecasting algorithms that AutoAI could use for IBM Cloud Pak® for Data(CP4D).

ARIMA = 'ARIMA'¶

BATS = 'BATS'¶

ENSEMBLER = 'Ensembler'¶

HW = 'HoltWinters'¶

LR = 'LinearRegression'¶

RF = 'RandomForest'¶

SVM = 'SVM'¶

class Metrics¶

Bases: object

Supported types of classification and regression metrics in autoai.

ACCURACY_AND_DISPARATE_IMPACT_SCORE = 'accuracy_and_disparate_impact'¶

ACCURACY_SCORE = 'accuracy'¶

AVERAGE_PRECISION_SCORE = 'average_precision'¶

EXPLAINED_VARIANCE_SCORE = 'explained_variance'¶

F1_SCORE = 'f1'¶

F1_SCORE_MACRO = 'f1_macro'¶

F1_SCORE_MICRO = 'f1_micro'¶

F1_SCORE_WEIGHTED = 'f1_weighted'¶

LOG_LOSS = 'neg_log_loss'¶

MEAN_ABSOLUTE_ERROR = 'neg_mean_absolute_error'¶

MEAN_SQUARED_ERROR = 'neg_mean_squared_error'¶

MEAN_SQUARED_LOG_ERROR = 'neg_mean_squared_log_error'¶

MEDIAN_ABSOLUTE_ERROR = 'neg_median_absolute_error'¶

PRECISION_SCORE = 'precision'¶

PRECISION_SCORE_MACRO = 'precision_macro'¶

PRECISION_SCORE_MICRO = 'precision_micro'¶

PRECISION_SCORE_WEIGHTED = 'precision_weighted'¶

R2_AND_DISPARATE_IMPACT_SCORE = 'r2_and_disparate_impact'¶

R2_SCORE = 'r2'¶

RECALL_SCORE = 'recall'¶

RECALL_SCORE_MACRO = 'recall_macro'¶

RECALL_SCORE_MICRO = 'recall_micro'¶

RECALL_SCORE_WEIGHTED = 'recall_weighted'¶

ROC_AUC_SCORE = 'roc_auc'¶

ROOT_MEAN_SQUARED_ERROR = 'neg_root_mean_squared_error'¶

ROOT_MEAN_SQUARED_LOG_ERROR = 'neg_root_mean_squared_log_error'¶

class PipelineTypes¶

Bases: object

Supported types of Pipelines.

LALE = 'lale'¶

SKLEARN = 'sklearn'¶

class PredictionType¶

Bases: object

Supported types of learning. OneOf: [BINARY, MULTICLASS, REGRESSION]

BINARY = 'binary'¶

CLASSIFICATION = 'classification'¶

FORECASTING = 'forecasting'¶

MULTICLASS = 'multiclass'¶

REGRESSION = 'regression'¶

class RegressionAlgorithms(value)¶

Bases: enum.Enum

Regression algorithms that AutoAI could use for IBM Cloud.

DT = 'DecisionTreeRegressor'¶

EX_TREES = 'ExtraTreesRegressor'¶

GB = 'GradientBoostingRegressor'¶

LGBM = 'LGBMRegressor'¶

LR = 'LinearRegression'¶

RF = 'RandomForestRegressor'¶

RIDGE = 'Ridge'¶

SnapBM = 'SnapBoostingMachineRegressor'¶

SnapDT = 'SnapDecisionTreeRegressor'¶

SnapRF = 'SnapRandomForestRegressor'¶

XGB = 'XGBRegressor'¶

class SamplingTypes¶

Bases: object

Types of training data sampling

FIRST_VALUES = 'first_n_records'¶

LAST_VALUES = 'truncate'¶

RANDOM = 'random'¶

STRATIFIED = 'stratified'¶

class TShirtSize¶

Bases: object

Possible sizes of the AutoAI POD Depends on the POD size, AutoAI could support different data sets sizes.

S - small (2vCPUs and 8GB of RAM) M - Medium (4vCPUs and 16GB of RAM) L - Large (8vCPUs and 32GB of RAM)) XL - Extra Large (16vCPUs and 64GB of RAM)

L = 'l'¶

M = 'm'¶

ML = 'ml'¶

S = 's'¶

XL = 'xl'¶

class Transformers¶

Bases: object

Supported types of congito transformers names in autoai.

ABS = 'abs'¶

CBRT = 'cbrt'¶

COS = 'cos'¶

CUBE = 'cube'¶

DIFF = 'diff'¶

DIVIDE = 'divide'¶

FEATUREAGGLOMERATION = 'featureagglomeration'¶

ISOFORESTANOMALY = 'isoforestanomaly'¶

LOG = 'log'¶

MAX = 'max'¶

MINMAXSCALER = 'minmaxscaler'¶

NXOR = 'nxor'¶

PCA = 'pca'¶

PRODUCT = 'product'¶

ROUND = 'round'¶

SIGMOID = 'sigmoid'¶

SIN = 'sin'¶

SQRT = 'sqrt'¶

SQUARE = 'square'¶

STDSCALER = 'stdscaler'¶

SUM = 'sum'¶

TAN = 'tan'¶

optimizer(name: str, *, prediction_type: ibm_watson_machine_learning.utils.autoai.enums.PredictionType, prediction_column: Optional[str] = None, prediction_columns: Optional[List[str]] = None, timestamp_column_name: Optional[str] = None, scoring: Optional[ibm_watson_machine_learning.utils.autoai.enums.Metrics] = None, desc: Optional[str] = None, test_size: Optional[float] = None, holdout_size: float = 0.1, max_number_of_estimators: Optional[int] = None, train_sample_rows_test_size: Optional[float] = None, include_only_estimators: Optional[List[Union[ibm_watson_machine_learning.utils.autoai.enums.ClassificationAlgorithms, ibm_watson_machine_learning.utils.autoai.enums.RegressionAlgorithms, ibm_watson_machine_learning.utils.autoai.enums.ForecastingAlgorithms]]] = None, daub_include_only_estimators: Optional[List[Union[ibm_watson_machine_learning.utils.autoai.enums.ClassificationAlgorithms, ibm_watson_machine_learning.utils.autoai.enums.RegressionAlgorithms]]] = None, backtest_num: Optional[int] = None, lookback_window: Optional[int] = None, forecast_window: Optional[int] = None, backtest_gap_length: Optional[int] = None, feature_columns: Optional[List[str]] = None, pipeline_types: Optional[List[ibm_watson_machine_learning.utils.autoai.enums.ForecastingPipelineTypes]] = None, supporting_features_at_forecast: Optional[bool] = None, cognito_transform_names: Optional[List[ibm_watson_machine_learning.utils.autoai.enums.Transformers]] = None, data_join_graph: Optional[ibm_watson_machine_learning.preprocessing.multiple_files_preprocessor.DataJoinGraph] = None, csv_separator: Union[List[str], str] = ',', excel_sheet: Optional[Union[str, int]] = None, encoding: str = 'utf-8', positive_label: Optional[str] = None, data_join_only: bool = False, drop_duplicates: bool = True, text_processing: Optional[bool] = None, word2vec_feature_number: Optional[int] = None, daub_give_priority_to_runtime: Optional[float] = None, fairness_info: Optional[dict] = None, sampling_type: Optional[ibm_watson_machine_learning.utils.autoai.enums.SamplingTypes] = None, sample_size_limit: Optional[int] = None, sample_rows_limit: Optional[int] = None, sample_percentage_limit: Optional[float] = None, n_parallel_data_connections: Optional[int] = None, number_of_batch_rows: Optional[int] = None, categorical_imputation_strategy: Optional[ibm_watson_machine_learning.utils.autoai.enums.ImputationStrategy] = None, numerical_imputation_strategy: Optional[ibm_watson_machine_learning.utils.autoai.enums.ImputationStrategy] = None, numerical_imputation_value: Optional[float] = None, imputation_threshold: Optional[float] = None, retrain_on_holdout: Optional[bool] = None, categorical_columns: Optional[list] = None, numerical_columns: Optional[list] = None, test_data_csv_separator: Union[List[str], str] = ',', test_data_excel_sheet: Optional[str] = None, test_data_encoding: str = 'utf-8', **kwargs) → Union[ibm_watson_machine_learning.experiment.autoai.optimizers.remote_auto_pipelines.RemoteAutoPipelines, ibm_watson_machine_learning.experiment.autoai.optimizers.local_auto_pipelines.LocalAutoPipelines][source]¶

Initialize an AutoAi optimizer.

name: str, required

Name for the AutoPipelines

prediction_type: PredictionType, required

Type of the prediction.

prediction_column: str, required for multiclass, binary and regression prediction types

Name of the target/label column

prediction_columns: List[str], required for forecasting prediction type

Names of the target/label columns.

timestamp_column_name: str, optional, used only for forecasting prediction type

Name of timestamp column for time series forecasting.

scoring: Metrics, optional

Type of the metric to optimize with. Not used for forecasting.

desc: str, optional

Description

test_size: deprecated

Use holdout_size instead.

holdout_size: float, optional

Percentage of the entire dataset to leave as a holdout. Default 0.1

max_number_of_estimators: int, optional

Maximum number (top-K ranked by DAUB model selection) of the selected algorithm, or estimator types, for example LGBMClassifierEstimator, XGBoostClassifierEstimator, or LogisticRegressionEstimator to use in pipeline composition. The default is None that means the true default value will be determined by the internal different algorithms, where only the highest ranked by model selection algorithm type is used.

train_sample_rows_test_size: float, optional

Training data sampling percentage

daub_include_only_estimators: deprecated

Use include_only_estimators instead.

include_only_estimators: List[Union[‘ClassificationAlgorithms’, ‘RegressionAlgorithms’, ‘ForecastingAlgorithms’]], optional

List of estimators to include in computation process. See: AutoAI.ClassificationAlgorithms, AutoAI.RegressionAlgorithms or AutoAI.ForecastingAlgorithms

backtest_num: int, optional

Used for forecasting prediction type. Configure number of backtests. Default value: 4 Value from range [0, 20]

lookback_window: int, optional

Used for forecasting prediction type. Configure length of lookback window. Default value: 10 If set to -1 lookback window will be auto-detected

forecast_window: int, optional

Used for forecasting prediction type. Configure length of forecast window. Default value: 1 Value from range [1, 60]

backtest_gap_length: int, optional

Used for forecasting prediction type. Configure gap between backtests. Default value: 0 Value from range [0, data length / 4]

feature_columns: List[str], optional

List of feature columns used for forecasting prediction type. May contain target column and/or supporting feature columns.

pipeline_types: List[ForecastingPipelineTypes], optional

List of pipeline types to be used for forecasting prediction type.

supporting_features_at_forecast: bool, optional

Enables usage of future supporting feature values during forecast.

cognito_transform_names: List[‘Transformers’], optional

List of transformers to include in the feature enginnering computation process. See: AutoAI.Transformers

csv_separator: Union[List[str], str], optional

The separator, or list of separators to try for separating columns in a CSV file. Not used if the file_name is not a CSV file. Default is ‘,’.

excel_sheet: Union[str, int], optional

Name of the excel sheet to use. Only use when xlsx file is an input. Support for number of the sheet is deprecated. By default first sheet is used.

encoding: str, optional

Encoding type for CSV training file.

positive_label: str, optional

The positive class to report when binary classification. When multiclass or regression, this will be ignored.

t_shirt_size: TShirtSize, optional

The size of the remote AutoAI POD instance (computing resources). Only applicable to a remote scenario. See: AutoAI.TShirtSize

data_join_graph: DataJoinGraph, optional

A graph object with definition of join structure for multiple input data sources. Data preprocess step for multiple files.

data_join_only: bool, optional

If True only preprocessing will be executed.

drop_duplicates: bool, optional

If True duplicated rows in data will be removed before further processing. Default is True.

text_processing: bool, optional

If True text processing will be enabled. Default is True. Applicable only on Cloud.

word2vec_feature_number: int, optional

Number of features which will be generated from text column. Will be applied only if text_processing is True. If None the default value will be taken.

daub_give_priority_to_runtime: float, optional

The importance of run time over score for pipelines ranking. Can take values between 0 and 5. If set to 0.0 only score is used. If set to 1 equally score and runtime are used. If set to value higher than 1 the runtime gets higher importance over score.

fairness_info: dict, optional

Dictionary that specifies metadata needed for measuring fairness. It contains three key values: favorable_labels, unfavorable_labels and protected_attributes. The favorable_labels attribute indicates that when the class column contains one of the value from list, that is considered a positive outcome. The unfavorable_labels is oposite to the favorable_labels and is obigatory for regression learning type. A protected attribute is a list of features that partition the population into groups whose outcome should have parity, if protected attribute is empty list then automatic detection of protected attributes will be run. If passed fairness metric will be calculated.

n_parallel_data_connections: int, optional

Number of maximum parallel connection to data source. Supported only for IBM Cloud Pak® for Data 4.0.1 and above.

categorical_imputation_strategy: ImputationStrategy, optional

Missing values imputation strategy for categorical columns.

Possible values (only non-forecasting scenario): - ImputationStrategy.MEAN - ImputationStrategy.MEDIAN - ImputationStrategy.MOST_FREQUENT (default)

numerical_imputation_strategy: ImputationStrategy, optional

Missing values imputation strategy for numerical columns.

Possible values (non-forecasting scenario): - ImputationStrategy.MEAN - ImputationStrategy.MEDIAN (default) - ImputationStrategy.MOST_FREQUENT

Possible values (forecasting scenario): - ImputationStrategy.MEAN - ImputationStrategy.MEDIAN - ImputationStrategy.BEST_OF_DEFAULT_IMPUTERS (default) - ImputationStrategy.VALUE - ImputationStrategy.FLATTEN_ITERATIVE - ImputationStrategy.LINEAR - ImputationStrategy.CUBIC - ImputationStrategy.PREVIOUS - ImputationStrategy.NEXT - ImputationStrategy.NO_IMPUTATION

numerical_imputation_value: float, optional

Value for filling missing values if numerical_imputation_strategy is set to ImputationStrategy.VALUE. For forecasting only.

imputation_threshold: float, optional

Maximum threshold of missing values imputation. For forecasting only.

retrain_on_holdout: bool, optional

If True final pipelines will be train also on holdout data.

categorical_columns: list, optional

List of columns names that must be treated as categorical.

numerical_columns: list, optional

List of columns names that must be treated as numerical.

sampling_type: str, optional

Type of sampling data for training. One of SamplingTypes enum values. Default is SamplingTypes.FIRST_N_RECORDS. Supported only for IBM Cloud Pak® for Data 4.0.1 and above.

sample_size_limit: int, optional

The size of sample upper bound (in bytes). The default value is 1GB. Supported only for IBM Cloud Pak® for Data 4.5 and above.

sample_rows_limit: int, optional

The size of sample upper bound (in rows). Supported only for IBM Cloud Pak® for Data 4.6 and above.

sample_percentage_limit: float, optional

The size of sample upper bound (as fraction of dataset size). Supported only for IBM Cloud Pak® for Data 4.6 and above.

number_of_batch_rows: int, optional

Number of rows to read in each batch when reading from flight connection.

test_data_csv_separator: Union[List[str], str], optional

The separator, or list of separators to try for separating columns in a CSV user-defined holdout/test file. Not used if the file_name is not a CSV file. Default is ‘,’.

test_data_excel_sheet: Union[str, int], optional

Name of the excel sheet to use for user-defined holdout/test data. Only use when xlsx file is an test dataset file. By default first sheet is used.

test_data_encoding: str, optional

Encoding type for CSV user-defined holdout/test file.

RemoteAutoPipelines or LocalAutoPipelines, depends on how you initialize the AutoAI object.

>>> from ibm_watson_machine_learning.experiment import AutoAI
>>> experiment = AutoAI(...)
>>>
>>> fairness_info = {
>>>            "protected_attributes": [
>>>                {"feature": "Sex", "reference_group": ['male'], "monitored_group": ['female']},
>>>                {"feature": "Age", "reference_group": [[50,60]], "monitored_group": [[18, 49]]}
>>>            ],
>>>            "favorable_labels": ["No Risk"],
>>>            "unfavorable_labels": ["Risk"],
>>>            }
>>>
>>> optimizer = experiment.optimizer(
>>>        name="name of the optimizer.",
>>>        prediction_type=AutoAI.PredictionType.BINARY,
>>>        prediction_column="y",
>>>        scoring=AutoAI.Metrics.ROC_AUC_SCORE,
>>>        desc="Some description.",
>>>        holdout_size=0.1,
>>>        max_num_daub_ensembles=1,
>>>        fairness_info= fairness_info,
>>>        cognito_transform_names=[AutoAI.Transformers.SUM,AutoAI.Transformers.MAX],
>>>        train_sample_rows_test_size=1,
>>>        include_only_estimators=[AutoAI.ClassificationAlgorithms.LGBM, AutoAI.ClassificationAlgorithms.XGB],
>>>        t_shirt_size=AutoAI.TShirtSize.L
>>>    )
>>>
>>> optimizer = experiment.optimizer(
>>>        name="name of the optimizer.",
>>>        prediction_type=AutoAI.PredictionType.MULTICLASS,
>>>        prediction_column="y",
>>>        scoring=AutoAI.Metrics.ROC_AUC_SCORE,
>>>        desc="Some description.",
>>>    )

runs(*, filter: str) → Union[ibm_watson_machine_learning.experiment.autoai.runs.auto_pipelines_runs.AutoPipelinesRuns, ibm_watson_machine_learning.experiment.autoai.runs.local_auto_pipelines_runs.LocalAutoPipelinesRuns][source]¶

Get the historical runs but with WML Pipeline name filter (for remote scenario). Get the historical runs but with experiment name filter (for local scenario).

filter: str, required: WML Pipeline name to filter the historical runs. or experiment name to filter the local historical runs.

AutoPipelinesRuns or LocalAutoPipelinesRuns

>>> from ibm_watson_machine_learning.experiment import AutoAI
>>> experiment = AutoAI(...)
>>>
>>> experiment.runs(filter='Test').list()

Working with AutoAI class and optimizer ¶

The AutoAI experiment class is responsible for creating experiments and scheduling training. All experiment results are stored automatically in the user-specified Cloud Object Storage (COS). Then, the AutoAI feature can fetch the results and provide them directly to the user for further usage.

Configure optimizer ¶

For an AutoAI object initialization WML credentials (with apikey and url) and one of project_id or space_id.

from ibm_watson_machine_learning.experiment import AutoAI

experiment = AutoAI(wml_credentials,
    space_id='76g53e0-0b32-4a0e-9152-3d50324855ddb')
)

pipeline_optimizer = experiment.optimizer(
            name='test name',
            desc='test description',
            prediction_type=AutoAI.PredictionType.BINARY,
            prediction_column='y',
            scoring=AutoAI.Metrics.ACCURACY_SCORE,
            test_size=0.1,
            max_num_daub_ensembles=1,
            train_sample_rows_test_size=1.,
            daub_include_only_estimators = [
                 AutoAI.ClassificationAlgorithms.XGB,
                 AutoAI.ClassificationAlgorithms.LGBM
                 ],
            cognito_transform_names = [
                 AutoAI.Transformers.SUM,
                 AutoAI.Transformers.MAX
                 ]
        )

Configure optimizer for time series forecasting ¶

Time series forecasting is a special AutoAI prediction scenario, with specific parameters used to configure forecasting: prediction_columns, timestamp_column_name, backtest_num, lookback_window, forecast_window, backtest_gap_length

from ibm_watson_machine_learning.experiment import AutoAI

experiment = AutoAI(wml_credentials,
    space_id='76g53e0-0b32-4a0e-9152-3d50324855ddb')
)

pipeline_optimizer = experiment.optimizer(
    name='forecasting optimiser',
    desc='description',
    prediction_type=experiment.PredictionType.FORECASTING,
    prediction_columns=['value'],
    timestamp_column_name='timestamp',
    backtest_num=4,
    lookback_window=5,
    forecast_window=2,
    holdout_size=0.05,
    max_number_of_estimators=1,
    include_only_estimators=[AutoAI.ForecastingAlgorithms.ENSEMBLER],
    t_shirt_size=TShirtSize.L
)

Optimizer and deployment fitting procedures are the same as in the basic scenario.

Configure optimizer for time series forecasting with supporting features ¶

Additional parameters can be passed to run time series forecasting scenarios with supporting features: feature_columns, pipeline_types, supporting_features_at_forecast

For more information about supporting features refer to time series documentation .

from ibm_watson_machine_learning.experiment import AutoAI
from ibm_watson_machine_learning.utils.autoai.enums import ForecastingPipelineTypes

experiment = AutoAI(wml_credentials,
    space_id='76g53e0-0b32-4a0e-9152-3d50324855ddb')
)

pipeline_optimizer = experiment.optimizer(
    name='forecasting optimizer',
    desc='description',
    prediction_type=experiment.PredictionType.FORECASTING,
    prediction_columns=['value'],
    timestamp_column_name='week',
    feature_columns=['a', 'b', 'value'],
    pipeline_types=[ForecastingPipelineTypes.FlattenEnsembler] + ForecastingPipelineTypes.get_exogenous(),
    supporting_features_at_forecast=True
)

Predicting for time series forecasting scenario with supporting features:

# Example data:
#   new_observations:
#       week       a   b  value
#       14.0       0   0  134
#       15.0       1   4  96
#       ...
#
#   supporting_features:
#       week       a   b
#       16.0       1   3
#       ...

# with DataFrame or np.array:
pipeline_optimizer.predict(new_observations, supporting_features=supporting_features)

Online scoring for time series forecasting scenario with supporting features:

# with DataFrame:
web_service.score(payload={'observations': new_observations_df, 'supporting_features': supporting_features_df})

Batch scoring for time series forecasting scenario with supporting features:

# with DataFrame:
batch_service.run_job(payload={'observations': new_observations_df, 'supporting_features': supporting_features_df})

# with DataConnection:
batch_service.run_job(payload={'observations': new_observations_data_connection, 'supporting_features': supporting_features_data_connection})

Get configuration parameters ¶

To see current configuration parameters, call the get_params() method.

config_parameters = pipeline_optimizer.get_params()
print(config_parameters)
{
    'name': 'test name',
    'desc': 'test description',
    'prediction_type': 'classification',
    'prediction_column': 'y',
    'scoring': 'roc_auc',
    'test_size': 0.1,
    'max_num_daub_ensembles': 1
}

Fit optimizer ¶

To schedule an AutoAI experiment, call the fit() method. This will trigger a training and an optimization process on WML. The fit() method can be synchronous (background_mode=False), or asynchronous (background_mode=True). If you don’t want to wait for fit to end, invoke the async version. It immediately returns only fit/run details. If you invoke the async version, you see a progress bar with information about the learning/optimization process.

fit_details = pipeline_optimizer.fit(
        training_data_reference=[training_data_connection],
        training_results_reference=results_connection,
        background_mode=True)

# OR

fit_details = pipeline_optimizer.fit(
        training_data_reference=[training_data_connection],
        training_results_reference=results_connection,
        background_mode=False)

Get run status, get run details ¶

If you use the fit() method asynchronously, you can monitor the run/fit details and status, using the following two methods:

status = pipeline_optimizer.get_run_status()
print(status)
'running'

# OR

'completed'

run_details = pipeline_optimizer.get_run_details()
print(run_details)
{'entity': {'pipeline': {'href': '/v4/pipelines/5bfeb4c5-90df-48b8-9e03-ba232d8c0838'},
        'results_reference': {'connection': {'access_key_id': '...',
                                             'endpoint_url': '...',
                                             'secret_access_key': '...'},
                              'location': {'bucket': '...',
                                           'logs': '53c8cb7b-c8b5-44aa-8b52-6fde3c588462',
                                           'model': '53c8cb7b-c8b5-44aa-8b52-6fde3c588462/model',
                                           'path': '.',
                                           'pipeline': './33825fa2-5fca-471a-ab1a-c84820b3e34e/pipeline.json',
                                           'training': './33825fa2-5fca-471a-ab1a-c84820b3e34e',
                                           'training_status': './33825fa2-5fca-471a-ab1a-c84820b3e34e/training-status.json'},
                              'type': 's3'},
        'space': {'href': '/v4/spaces/71ab11ea-bb77-4ae6-b98a-a77f30ade09d'},
        'status': {'completed_at': '2020-02-17T10:46:32.962Z',
                   'message': {'level': 'info',
                               'text': 'Training job '
                                       '33825fa2-5fca-471a-ab1a-c84820b3e34e '
                                       'completed'},
                   'state': 'completed'},
        'training_data_references': [{'connection': {'access_key_id': '...',
                                                     'endpoint_url': '...',
                                                     'secret_access_key': '...'},
                                      'location': {'bucket': '...',
                                                   'path': '...'},
                                      'type': 's3'}]},
 'metadata': {'created_at': '2020-02-17T10:44:22.532Z',
              'guid': '33825fa2-5fca-471a-ab1a-c84820b3e34e',
              'href': '/v4/trainings/33825fa2-5fca-471a-ab1a-c84820b3e34e',
              'id': '33825fa2-5fca-471a-ab1a-c84820b3e34e',
              'modified_at': '2020-02-17T10:46:32.987Z'}}

Get data connections ¶

The data_connections list contains all training connections that you referenced while calling the fit() method.

data_connections = pipeline_optimizer.get_data_connections()

Summary ¶

It is possible to get a ranking of all the computed pipeline models, sorted based on a scoring metric supplied when configuring the optimizer (scoring parameter). The output type is a pandas.DataFrame with pipeline names, computation timestamps, machine learning metrics, and the number of enhancements implemented in each of the pipelines.

results = pipeline_optimizer.summary()
print(results)
               Number of enhancements  ...  training_f1
Pipeline Name                          ...
Pipeline_4                          3  ...     0.555556
Pipeline_3                          2  ...     0.554978
Pipeline_2                          1  ...     0.503175
Pipeline_1                          0  ...     0.529928

Get pipeline details ¶

To see pipeline composition steps and nodes, use the get_pipeline_details() method. If you leave pipeline_name empty, the method returns the details of the best computed pipeline.

pipeline_params = pipeline_optimizer.get_pipeline_details(pipeline_name='Pipeline_1')
print(pipeline_params)
{
    'composition_steps': [
        'TrainingDataset_full_199_16', 'Split_TrainingHoldout',
        'TrainingDataset_full_179_16', 'Preprocessor_default', 'DAUB'
        ],
    'pipeline_nodes': [
        'PreprocessingTransformer', 'LogisticRegressionEstimator'
        ]
}

Get pipeline ¶

Use the get_pipeline() method to load a specific pipeline. By default, get_pipeline() returns a lale pipeline. For information on lale pipelines, refer to (https://github.com/ibm/lale).

pipeline = pipeline_optimizer.get_pipeline(pipeline_name='Pipeline_4')
print(type(pipeline))
'lale.operators.TrainablePipeline'

You can also load a pipeline as a scikit learn (sklearn) pipeline model type.

pipeline = pipeline_optimizer.get_pipeline(pipeline_name='Pipeline_4', astype=AutoAI.PipelineTypes.SKLEARN)
print(type(pipeline))
# <class 'sklearn.pipeline.Pipeline'>

Working with deployments ¶

This section describes classes that enable you to work with Watson Machine Learning deployments.

Web Service ¶

Web Service is an online type of deployment. It allows you to upload and deploy your model, to score it through an online web service. You must pass the location where the training was performed (source_space_id or source_project_id). The model can be deployed to any space or project (target_space_id or target_project_id).

Note: WebService supports only the AutoAI deployment type.

from ibm_watson_machine_learning.deployment import WebService

# note: only AutoAI deployment is possible now
service = WebService(wml_credentials,
     source_space_id='76g53e0-0b32-4a0e-9152-3d50324855ddb',
     target_space_id='1234abc1234abc1234abc1234abc1234abcd')
 )

service.create(
       experiment_run_id="...",
       model=model,
       deployment_name='My new deployment'
   )

Batch ¶

Batch manages the batch type of deployment. It allows you to upload and deploy a model and run a batch deployment job. As in Web Service, you must pass the location where the training was performed (source_space_id or source_project_id). The model can be deployed to any space or project (target_space_id or target_project_id).

The input data can be provided as a pandas.DataFrame, a data-asset, or a Cloud Object Storage (COS) file.

Note: WebService supports only the AutoAI deployment type.

Example of a batch deployment creation:

from ibm_watson_machine_learning.deployment import Batch

service_batch = Batch(wml_credentials, source_space_id='76g53e0-0b32-4a0e-9152-3d50324855ddb')
service_batch.create(
        experiment_run_id="6ce62a02-3e41-4d11-89d1-484c2deaed75",
        model="Pipeline_4",
        deployment_name='Batch deployment')

Example of a batch job creation with inline data as pandas.DataFrame type:

scoring_params = service_batch.run_job(
            payload=test_X_df,
            background_mode=False)

Example of batch job creation with a COS object:

from ibm_watson_machine_learning.helpers.connections import S3Location, DataConnection

connection_details = client.connections.create({
    client.connections.ConfigurationMetaNames.NAME: "Connection to COS",
    client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: client.connections.get_datasource_type_uid_by_name('bluemixcloudobjectstorage'),
    client.connections.ConfigurationMetaNames.PROPERTIES: {
        'bucket': 'bucket_name',
        'access_key': 'COS access key id',
        'secret_key': 'COS secret access key'
        'iam_url': 'COS iam url',
        'url': 'COS endpoint url'
    }
})

connection_id = client.connections.get_uid(connection_details)

payload_reference = DataConnection(
        connection_asset_id=connection_id,
        location=S3Location(bucket='bucket_name',   # note: COS bucket name where deployment payload dataset is located
                            path='my_path'  # note: path within bucket where your deployment payload dataset is located
                            )
    )

results_reference = DataConnection(
        connection_asset_id=connection_id,
        location=S3Location(bucket='bucket_name',   # note: COS bucket name where deployment output should be located
                            path='my_path_where_output_will_be_saved'  # note: path within bucket where your deployment output should be located
                            )
    )
payload_reference.write("local_path_to_the_batch_payload_csv_file", remote_name="batch_payload_location.csv"])

scoring_params = service_batch.run_job(
    payload=[payload_reference],
    output_data_reference=results_reference,
    background_mode=False)   # If background_mode is False, then synchronous mode is started. Otherwise job status need to be monitored.

Example of a batch job creation with a data-asset object.

from ibm_watson_machine_learning.helpers.connections import DataConnection, CloudAssetLocation, DeploymentOutputAssetLocation

payload_reference = DataConnection(location=CloudAssetLocation(asset_id=asset_id))
results_reference = DataConnection(
        location=DeploymentOutputAssetLocation(name="batch_output_file_name.csv"))

    scoring_params = service_batch.run_job(
        payload=[payload_reference],
        output_data_reference=results_reference,
        background_mode=False)

Working with DataConnection ¶

DataConnection is the base class to start working with your data storage needed for AutoAI backend. You can use it to fetch training data and store all of the results. There are several ways you can use the DataConnection object. This is a basic scenario.

To start an AutoAI experiment, first specify where your training dataset is located. Currently, AutoAI supports Cloud Object Storage (COS) and data assets on Cloud.

Cloud DataConnection Initialization ¶

To upload your experiment dataset, you must initialize DataConnection with your COS credentials.

from ibm_watson_machine_learning.autoai.helpers.connections import S3Location, DataConnection

connection_details = client.connections.create({
    client.connections.ConfigurationMetaNames.NAME: "Connection to COS",
    client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: client.connections.get_datasource_type_uid_by_name('bluemixcloudobjectstorage'),
    client.connections.ConfigurationMetaNames.PROPERTIES: {
        'bucket': 'bucket_name',
        'access_key': 'COS access key id',
        'secret_key': 'COS secret access key'
        'iam_url': 'COS iam url',
        'url': 'COS endpoint url'
    }
})

connection_id = client.connections.get_uid(connection_details)

# note: this DataConnection will be used as a reference where to find your training dataset
training_data_connection = DataConnection(
        connection_asset_id=connection_id,
        location=S3Location(bucket='bucket_name',   # note: COS bucket name where training dataset is located
                            path='my_path'  # note: path within bucket where your training dataset is located
                            )
    )

# note: this DataConnection will be used as a reference where to save all of the AutoAI experiment results
results_connection = DataConnection(
        connection_asset_id=connection_id,
        # note: bucket name and path could be different or the same as specified in the training_data_connection
        location=S3Location(bucket='bucket_name',
                            path='my_path'
                            )
    )

Upload your training dataset ¶

An AutoAI experiment should be able to access your training data. If you do not have a training dataset stored already, you can do it by invoking the write() method of the DataConnection object.

training_data_connection.write(data='local_path_to_the_dataset', remote_name='training_dataset.csv')

Download your training dataset ¶

To download a stored dataset, use the read() method of the DataConnection object.

dataset = training_data_connection.read()   # note: returns a pandas DataFrame

AutoAI Helpers ¶

DataConnection ¶

class ibm_watson_machine_learning.helpers.connections.connections.DataConnection(location: Optional[Union[ibm_watson_machine_learning.helpers.connections.connections.S3Location, ibm_watson_machine_learning.helpers.connections.connections.FSLocation, ibm_watson_machine_learning.helpers.connections.connections.AssetLocation, ibm_watson_machine_learning.helpers.connections.connections.CP4DAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.WMLSAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.WSDAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.CloudAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.NFSLocation, ibm_watson_machine_learning.helpers.connections.connections.DeploymentOutputAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.ConnectionAssetLocation, ibm_watson_machine_learning.helpers.connections.connections.DatabaseLocation, ibm_watson_machine_learning.helpers.connections.connections.ContainerLocation]] = None, connection: Optional[Union[ibm_watson_machine_learning.helpers.connections.connections.S3Connection, ibm_watson_machine_learning.helpers.connections.connections.NFSConnection, ibm_watson_machine_learning.helpers.connections.connections.ConnectionAsset]] = None, data_join_node_name: Optional[Union[List[str], str]] = None, data_asset_id: Optional[str] = None, connection_asset_id: Optional[str] = None, **kwargs)[source]¶

Bases: ibm_watson_machine_learning.helpers.connections.base_data_connection.BaseDataConnection

Data Storage Connection class needed for WML training metadata (input data).

connection: Union[‘NFSConnection’, ‘ConnectionAsset’], optional: connection parameters of specific type
location: Union[S3Location, FSLocation, AssetLocation],: required location parameters of specific type
data_join_node_name: Union[str, List[str]], optional: Names for node(s). If no value provided, data file name will be used as node name. If str will be passed, it will became node name. If multiple names will be passed, several nodes will have the same data connection (used for excel files with multiple sheets).
data_asset_id: str, optional,: Data asset ID if DataConnection should be pointing out to Data Asset.

classmethod from_studio(path: str) → List[ibm_watson_machine_learning.helpers.connections.connections.DataConnection][source]¶

Create DataConnections from the credentials stored (connected) in Watson Studio. Only for COS.

path: str, required: Path in COS bucket to the training dataset.

List with DataConnection objects.

>>> data_connections = DataConnection.from_studio(path='iris_dataset.csv')

read(with_holdout_split: bool = False, csv_separator: str = ',', excel_sheet: Optional[Union[str, int]] = None, encoding: Optional[str] = 'utf-8', raw: Optional[bool] = False, binary: Optional[bool] = False, read_to_file: Optional[str] = None, number_of_batch_rows: Optional[int] = None, sampling_type: Optional[str] = None, sample_size_limit: Optional[int] = None, sample_rows_limit: Optional[int] = None, sample_percentage_limit: Optional[float] = None, **kwargs) → Union[pandas.core.frame.DataFrame, Tuple[pandas.core.frame.DataFrame, pandas.core.frame.DataFrame], bytes][source]¶

Download dataset stored in remote data storage. Returns batch up to 1GB.

with_holdout_split: bool, optional: If True, data will be split to train and holdout dataset as it was by AutoAI.
csv_separator: str, optional: Separator / delimiter for CSV file, default is ‘,’
excel_sheet: Union[str, int], optional: Excel file sheet name to use. Only use when xlsx file is an input. Support for number of the sheet is deprecated.
encoding: str, optional: Encoding type of the CSV
raw: bool, optional: Default False. If False there wil be applied simple data preprocessing (the same as in the backend). If True, data will be not preprocessed.
binary: bool, optional: Indicates to retrieve data in binary mode. The result will be a python binary type variable. Default False.
read_to_file: str, optional: Stream read data to file under path specified as value of this parameter. Use this parameter to prevent keeping data in-memory.
number_of_batch_rows: int, optional: Number of rows to read in each batch when reading from flight connection.
sampling_type: str, optional: A sampling strategy how to read the data.
sample_size_limit: int, optional: Upper limit for overall data that should be downloaded in bytes, Default: 1 GB. If more than one of: sample_size_limit, sample_rows_limit, sample_percentage_limit are set, then downloaded data is limited to the lowest threshold.
sample_rows_limit: int, optional: Upper limit for overall data that should be downloaded in number of rows. If more than one of: sample_size_limit, sample_rows_limit, sample_percentage_limit are set, then downloaded data is limited to the lowest threshold.
sample_percentage_limit: float, optional: Upper limit for overall data that should be downloaded in percent of all dataset. This parameter is ignored, when sampling_type parameter is set to first_n_records. Must be a float number between 0 and 1. If more than one of: sample_size_limit, sample_rows_limit, sample_percentage_limit are set, then downloaded data is limited to the lowest threshold.

pandas.DataFrame contains dataset from remote data storage : Xy_train

Tuple[pandas.DataFrame, pandas.DataFrame, pandas.DataFrame, pandas.DataFrame] : X_train, X_holdout, y_train, y_holdout

Tuple[pandas.DataFrame, pandas.DataFrame] : X_test, y_test containing training data and holdout data from remote storage.

bytes object

Auto holdout split from backend (only train data provided)

>>> train_data_connections = optimizer.get_data_connections()
>>>
>>> data = train_data_connections[0].read() # all train data
>>> # or
>>> X_train, X_holdout, y_train, y_holdout = train_data_connections[0].read(with_holdout_split=True) # train and holdout data

User provided train and test data

>>> optimizer.fit(training_data_reference=[DataConnection],
>>>               training_results_reference=DataConnection,
>>>               test_data_reference=DataConnection)
>>>
>>> test_data_connection = optimizer.get_test_data_connections()
>>> X_test, y_test = test_data_connection.read() # only holdout data
>>>
>>> # and
>>>
>>> train_data_connections = optimizer.get_data_connections()
>>> data = train_connections[0].read() # only train data

set_client(wml_client)[source]¶

Set initialized wml client in connection to enable write/read operations with connection to service.

wml_client: APIClient, required: Wml client to connect to service.

>>> DataConnection.set_client(wml_client)

write(data: Union[str, pandas.core.frame.DataFrame], remote_name: Optional[str] = None, **kwargs) → None[source]¶

Upload file to a remote data storage.

data: str, required: Local path to the dataset or pandas.DataFrame with data.
remote_name: str, required: Name that dataset should be stored with in remote data storage.

S3Location ¶

class ibm_watson_machine_learning.helpers.connections.connections.S3Location(bucket: str, path: str, **kwargs)[source]¶

Bases: ibm_watson_machine_learning.helpers.connections.base_location.BaseLocation

Connection class to COS data storage in S3 format.

bucket: str, required: COS bucket name
path: str, required: COS data path in the bucket.
excel_sheet: str, optional: Name of excel sheet if pointed dataset is excel file used for Batched Deployment scoring.
model_location: str, optional: Path to the pipeline model in the COS.
training_status: str, optional: Path t the training status json in COS.

get_location() → str[source]¶

CloudAssetLocation ¶

class ibm_watson_machine_learning.helpers.connections.connections.CloudAssetLocation(asset_id: str)[source]¶

Bases: ibm_watson_machine_learning.helpers.connections.connections.AssetLocation

Connection class to data assets as input data references to batch deployment job on Cloud.

asset_id: str, required: Asset ID of the file loaded on space on Cloud.

DeploymentOutputAssetLocation ¶

class ibm_watson_machine_learning.helpers.connections.connections.DeploymentOutputAssetLocation(name: str, description: str = '')[source]¶

Bases: ibm_watson_machine_learning.helpers.connections.base_location.BaseLocation

Connection class to data assets where output of batch deployment will be stored.

name: str, required: name of .csv file which will be saved as data asset.
description: str, optional: description of the data asset

get_credentials_from_config ¶

class ibm_watson_machine_learning.helpers.helpers.get_credentials_from_config(env_name, credentials_name, config_path='./config.ini')[source]¶

Bases:

Load credentials from config file.

[DEV_LC]

wml_credentials = { } cos_credentials = { }

Parameters

env_name (str) – the name of [ENV] defined in config file
credentials_name (str) – name of credentials
config_path (str) – path to the config file

Returns

dict

>>> get_credentials_from_config(env_name='DEV_LC', credentials_name='wml_credentials')

Enums ¶

class ibm_watson_machine_learning.utils.autoai.enums.ClassificationAlgorithms(value)[source]¶

Bases: enum.Enum

Classification algorithms that AutoAI could use for IBM Cloud.

DT = 'DecisionTreeClassifier'¶

EX_TREES = 'ExtraTreesClassifier'¶

GB = 'GradientBoostingClassifier'¶

LGBM = 'LGBMClassifier'¶

LR = 'LogisticRegression'¶

RF = 'RandomForestClassifier'¶

SnapBM = 'SnapBoostingMachineClassifier'¶

SnapDT = 'SnapDecisionTreeClassifier'¶

SnapLR = 'SnapLogisticRegression'¶

SnapRF = 'SnapRandomForestClassifier'¶

SnapSVM = 'SnapSVMClassifier'¶

XGB = 'XGBClassifier'¶

class ibm_watson_machine_learning.utils.autoai.enums.ClassificationAlgorithmsCP4D(value)[source]¶

Bases: enum.Enum

Classification algorithms that AutoAI could use for IBM Cloud Pak® for Data(CP4D). The SnapML estimators (SnapDT, SnapRF, SnapSVM, SnapLR) are supported on IBM Cloud Pak® for Data version 4.0.2 and above.

DT = 'DecisionTreeClassifierEstimator'¶

EX_TREES = 'ExtraTreesClassifierEstimator'¶

GB = 'GradientBoostingClassifierEstimator'¶

LGBM = 'LGBMClassifierEstimator'¶

LR = 'LogisticRegressionEstimator'¶

RF = 'RandomForestClassifierEstimator'¶

SnapBM = 'SnapBoostingMachineClassifier'¶

SnapDT = 'SnapDecisionTreeClassifier'¶

SnapLR = 'SnapLogisticRegression'¶

SnapRF = 'SnapRandomForestClassifier'¶

SnapSVM = 'SnapSVMClassifier'¶

XGB = 'XGBClassifierEstimator'¶

class ibm_watson_machine_learning.utils.autoai.enums.DataConnectionTypes[source]¶

Bases: object

Supported types of DataConnection. OneOf: [s3, FS]

CA = 'connection_asset'¶

CN = 'container'¶

DS = 'data_asset'¶

FS = 'fs'¶

S3 = 's3'¶

class ibm_watson_machine_learning.utils.autoai.enums.Directions[source]¶

Bases: object

Possible metrics directions

ASCENDING = 'ascending'¶

DESCENDING = 'descending'¶

class ibm_watson_machine_learning.utils.autoai.enums.ForecastingAlgorithms(value)[source]¶

Bases: enum.Enum

Forecasting algorithms that AutoAI could use for IBM Cloud Pak® for Data(CP4D).

ARIMA = 'ARIMA'¶

BATS = 'BATS'¶

ENSEMBLER = 'Ensembler'¶

HW = 'HoltWinters'¶

LR = 'LinearRegression'¶

RF = 'RandomForest'¶

SVM = 'SVM'¶

class ibm_watson_machine_learning.utils.autoai.enums.ForecastingAlgorithmsCP4D(value)[source]¶

Bases: enum.Enum

Forecasting algorithms that AutoAI could use for IBM Cloud.

ARIMA = 'ARIMA'¶

BATS = 'BATS'¶

ENSEMBLER = 'Ensembler'¶

HW = 'HoltWinters'¶

LR = 'LinearRegression'¶

RF = 'RandomForest'¶

SVM = 'SVM'¶

class ibm_watson_machine_learning.utils.autoai.enums.ForecastingPipelineTypes(value)[source]¶

Bases: enum.Enum

Forecasting pipeline types that AutoAI could use for IBM Cloud Pak® for Data(CP4D).

ARIMA = 'ARIMA'¶

ARIMAX = 'ARIMAX'¶

ARIMAX_DMLR = 'ARIMAX_DMLR'¶

ARIMAX_PALR = 'ARIMAX_PALR'¶

ARIMAX_RAR = 'ARIMAX_RAR'¶

ARIMAX_RSAR = 'ARIMAX_RSAR'¶

Bats = 'Bats'¶

DifferenceFlattenEnsembler = 'DifferenceFlattenEnsembler'¶

ExogenousDifferenceFlattenEnsembler = 'ExogenousDifferenceFlattenEnsembler'¶

ExogenousFlattenEnsembler = 'ExogenousFlattenEnsembler'¶

ExogenousLocalizedFlattenEnsembler = 'ExogenousLocalizedFlattenEnsembler'¶

ExogenousMT2RForecaster = 'ExogenousMT2RForecaster'¶

ExogenousRandomForestRegressor = 'ExogenousRandomForestRegressor'¶

ExogenousSVM = 'ExogenousSVM'¶

FlattenEnsembler = 'FlattenEnsembler'¶

HoltWinterAdditive = 'HoltWinterAdditive'¶

HoltWinterMultiplicative = 'HoltWinterMultiplicative'¶

LocalizedFlattenEnsembler = 'LocalizedFlattenEnsembler'¶

MT2RForecaster = 'MT2RForecaster'¶

RandomForestRegressor = 'RandomForestRegressor'¶

SVM = 'SVM'¶

static get_exogenous()[source]¶

Get list of pipelines that use supporting features (exogenous pipelines).

List of pipelines using supporting features.

static get_non_exogenous()[source]¶

Get list of pipelines not using supporting features (non-exogenous pipelines).

List of pipelines that do not use supporting features.

class ibm_watson_machine_learning.utils.autoai.enums.ImputationStrategy(value)[source]¶

Bases: enum.Enum

Missing values imputation strategies.

BEST_OF_DEFAULT_IMPUTERS = 'best_of_default_imputers'¶

CUBIC = 'cubic'¶

FLATTEN_ITERATIVE = 'flatten_iterative'¶

LINEAR = 'linear'¶

MEAN = 'mean'¶

MEDIAN = 'median'¶

MOST_FREQUENT = 'most_frequent'¶

NEXT = 'next'¶

NO_IMPUTATION = 'no_imputation'¶

PREVIOUS = 'previous'¶

VALUE = 'value'¶

class ibm_watson_machine_learning.utils.autoai.enums.Metrics[source]¶

Bases: object

Supported types of classification and regression metrics in autoai.

ACCURACY_AND_DISPARATE_IMPACT_SCORE = 'accuracy_and_disparate_impact'¶

ACCURACY_SCORE = 'accuracy'¶

AVERAGE_PRECISION_SCORE = 'average_precision'¶

EXPLAINED_VARIANCE_SCORE = 'explained_variance'¶

F1_SCORE = 'f1'¶

F1_SCORE_MACRO = 'f1_macro'¶

F1_SCORE_MICRO = 'f1_micro'¶

F1_SCORE_WEIGHTED = 'f1_weighted'¶

LOG_LOSS = 'neg_log_loss'¶

MEAN_ABSOLUTE_ERROR = 'neg_mean_absolute_error'¶

MEAN_SQUARED_ERROR = 'neg_mean_squared_error'¶

MEAN_SQUARED_LOG_ERROR = 'neg_mean_squared_log_error'¶

MEDIAN_ABSOLUTE_ERROR = 'neg_median_absolute_error'¶

PRECISION_SCORE = 'precision'¶

PRECISION_SCORE_MACRO = 'precision_macro'¶

PRECISION_SCORE_MICRO = 'precision_micro'¶

PRECISION_SCORE_WEIGHTED = 'precision_weighted'¶

R2_AND_DISPARATE_IMPACT_SCORE = 'r2_and_disparate_impact'¶

R2_SCORE = 'r2'¶

RECALL_SCORE = 'recall'¶

RECALL_SCORE_MACRO = 'recall_macro'¶

RECALL_SCORE_MICRO = 'recall_micro'¶

RECALL_SCORE_WEIGHTED = 'recall_weighted'¶

ROC_AUC_SCORE = 'roc_auc'¶

ROOT_MEAN_SQUARED_ERROR = 'neg_root_mean_squared_error'¶

ROOT_MEAN_SQUARED_LOG_ERROR = 'neg_root_mean_squared_log_error'¶

class ibm_watson_machine_learning.utils.autoai.enums.MetricsToDirections(value)[source]¶

Bases: enum.Enum

Map of metrics directions.

ACCURACY = 'ascending'¶

AVERAGE_PRECISION = 'ascending'¶

EXPLAINED_VARIANCE = 'ascending'¶

F1 = 'ascending'¶

F1_MACRO = 'ascending'¶

F1_MICRO = 'ascending'¶

F1_WEIGHTED = 'ascending'¶

NEG_LOG_LOSS = 'descending'¶

NEG_MEAN_ABSOLUTE_ERROR = 'descending'¶

NEG_MEAN_SQUARED_ERROR = 'descending'¶

NEG_MEAN_SQUARED_LOG_ERROR = 'descending'¶

NEG_MEDIAN_ABSOLUTE_ERROR = 'descending'¶

NEG_ROOT_MEAN_SQUARED_ERROR = 'descending'¶

NEG_ROOT_MEAN_SQUARED_LOG_ERROR = 'descending'¶

NORMALIZED_GINI_COEFFICIENT = 'ascending'¶

PRECISION = 'ascending'¶

PRECISION_MACRO = 'ascending'¶

PRECISION_MICRO = 'ascending'¶

PRECISION_WEIGHTED = 'ascending'¶

R2 = 'ascending'¶

RECALL = 'ascending'¶

RECALL_MACRO = 'ascending'¶

RECALL_MICRO = 'ascending'¶

RECALL_WEIGHTED = 'ascending'¶

ROC_AUC = 'ascending'¶

class ibm_watson_machine_learning.utils.autoai.enums.PipelineTypes[source]¶

Bases: object

Supported types of Pipelines.

LALE = 'lale'¶

SKLEARN = 'sklearn'¶

class ibm_watson_machine_learning.utils.autoai.enums.PositiveLabelClass[source]¶

Bases: object

Metrics that need positive label definition for binary classification.

AVERAGE_PRECISION_SCORE = 'average_precision'¶

F1_SCORE = 'f1'¶

F1_SCORE_MACRO = 'f1_macro'¶

F1_SCORE_MICRO = 'f1_micro'¶

F1_SCORE_WEIGHTED = 'f1_weighted'¶

PRECISION_SCORE = 'precision'¶

PRECISION_SCORE_MACRO = 'precision_macro'¶

PRECISION_SCORE_MICRO = 'precision_micro'¶

PRECISION_SCORE_WEIGHTED = 'precision_weighted'¶

RECALL_SCORE = 'recall'¶

RECALL_SCORE_MACRO = 'recall_macro'¶

RECALL_SCORE_MICRO = 'recall_micro'¶

RECALL_SCORE_WEIGHTED = 'recall_weighted'¶

class ibm_watson_machine_learning.utils.autoai.enums.PredictionType[source]¶

Bases: object

Supported types of learning. OneOf: [BINARY, MULTICLASS, REGRESSION]

BINARY = 'binary'¶

CLASSIFICATION = 'classification'¶

FORECASTING = 'forecasting'¶

MULTICLASS = 'multiclass'¶

REGRESSION = 'regression'¶

class ibm_watson_machine_learning.utils.autoai.enums.RegressionAlgorithms(value)[source]¶

Bases: enum.Enum

Regression algorithms that AutoAI could use for IBM Cloud.

DT = 'DecisionTreeRegressor'¶

EX_TREES = 'ExtraTreesRegressor'¶

GB = 'GradientBoostingRegressor'¶

LGBM = 'LGBMRegressor'¶

LR = 'LinearRegression'¶

RF = 'RandomForestRegressor'¶

RIDGE = 'Ridge'¶

SnapBM = 'SnapBoostingMachineRegressor'¶

SnapDT = 'SnapDecisionTreeRegressor'¶

SnapRF = 'SnapRandomForestRegressor'¶

XGB = 'XGBRegressor'¶

class ibm_watson_machine_learning.utils.autoai.enums.RegressionAlgorithmsCP4D(value)[source]¶

Bases: enum.Enum

Regression algorithms that AutoAI could use for IBM Cloud Pak® for Data(CP4D). The SnapML estimators (SnapDT, SnapRF, SnapBM) are supported on IBM Cloud Pak® for Data version 4.0.2 and above.

DT = 'DecisionTreeRegressorEstimator'¶

EX_TREES = 'ExtraTreesRegressorEstimator'¶

GB = 'GradientBoostingRegressorEstimator'¶

LGBM = 'LGBMRegressorEstimator'¶

LR = 'LinearRegressionEstimator'¶

RF = 'RandomForestRegressorEstimator'¶

RIDGE = 'RidgeEstimator'¶

SnapBM = 'SnapBoostingMachineRegressor'¶

SnapDT = 'SnapDecisionTreeRegressor'¶

SnapRF = 'SnapRandomForestRegressor'¶

XGB = 'XGBRegressorEstimator'¶

class ibm_watson_machine_learning.utils.autoai.enums.RunStateTypes[source]¶

Bases: object

Supported types of AutoAI fit/run.

COMPLETED = 'completed'¶

FAILED = 'failed'¶

class ibm_watson_machine_learning.utils.autoai.enums.SamplingTypes[source]¶

Bases: object

Types of training data sampling

FIRST_VALUES = 'first_n_records'¶

LAST_VALUES = 'truncate'¶

RANDOM = 'random'¶

STRATIFIED = 'stratified'¶

class ibm_watson_machine_learning.utils.autoai.enums.TShirtSize[source]¶

Bases: object

Possible sizes of the AutoAI POD Depends on the POD size, AutoAI could support different data sets sizes.

S - small (2vCPUs and 8GB of RAM) M - Medium (4vCPUs and 16GB of RAM) L - Large (8vCPUs and 32GB of RAM)) XL - Extra Large (16vCPUs and 64GB of RAM)

L = 'l'¶

M = 'm'¶

ML = 'ml'¶

S = 's'¶

XL = 'xl'¶

class ibm_watson_machine_learning.utils.autoai.enums.Transformers[source]¶

Bases: object

Supported types of congito transformers names in autoai.

ABS = 'abs'¶

CBRT = 'cbrt'¶

COS = 'cos'¶

CUBE = 'cube'¶

DIFF = 'diff'¶

DIVIDE = 'divide'¶

FEATUREAGGLOMERATION = 'featureagglomeration'¶

ISOFORESTANOMALY = 'isoforestanomaly'¶

LOG = 'log'¶

MAX = 'max'¶

MINMAXSCALER = 'minmaxscaler'¶

NXOR = 'nxor'¶

PCA = 'pca'¶

PRODUCT = 'product'¶

ROUND = 'round'¶

SIGMOID = 'sigmoid'¶

SIN = 'sin'¶

SQRT = 'sqrt'¶

SQUARE = 'square'¶

STDSCALER = 'stdscaler'¶

SUM = 'sum'¶

TAN = 'tan'¶

class ibm_watson_machine_learning.utils.autoai.enums.VisualizationTypes[source]¶

Bases: object

Types of visualization options.

INPLACE = 'inplace'¶

PDF = 'pdf'¶

Deployment Modules for AutoAI models ¶

Batch ¶

class ibm_watson_machine_learning.deployment.Batch(source_wml_credentials: Union[dict, WorkSpace] = None, source_project_id: str = None, source_space_id: str = None, target_wml_credentials: Union[dict, WorkSpace] = None, target_project_id: str = None, target_space_id: str = None, wml_credentials: Union[dict, WorkSpace] = None, project_id: str = None, space_id: str = None)[source]¶

Bases: ibm_watson_machine_learning.deployment.base_deployment.BaseDeployment

The Batch Deployment class. With this class object you can manage any batch deployment.

source_wml_credentials: dictionary, required: Credentials to Watson Machine Learning instance where training was performed.
source_project_id: str, optional: ID of the Watson Studio project where training was performed.
source_space_id: str, optional: ID of the Watson Studio Space where training was performed.
target_wml_credentials: dictionary, required: Credentials to Watson Machine Learning instance where you want to deploy.
target_project_id: str, optional: ID of the Watson Studio project where you want to deploy.
target_space_id: str, optional: ID of the Watson Studio Space where you want to deploy.

create(model: str, deployment_name: str, metadata: Optional[Dict] = None, training_data: Optional[Union[DataFrame, ndarray]] = None, training_target: Optional[Union[DataFrame, ndarray]] = None, experiment_run_id: Optional[str] = None) → None[source]¶

Create deployment from a model.

model: str, required: AutoAI model name.
deployment_name: str, required: Name of the deployment
training_data: Union[‘pandas.DataFrame’, ‘numpy.ndarray’], optional: Training data for the model
training_target: Union[‘pandas.DataFrame’, ‘numpy.ndarray’], optional: Target/label data for the model
metadata: dictionary, optional: Model meta properties.
experiment_run_id: str, optional: ID of a training/experiment (only applicable for AutoAI deployments)

>>> from ibm_watson_machine_learning.deployment import Batch
>>>
>>> deployment = Batch(
>>>        wml_credentials={
>>>              "apikey": "...",
>>>              "iam_apikey_description": "...",
>>>              "iam_apikey_name": "...",
>>>              "iam_role_crn": "...",
>>>              "iam_serviceid_crn": "...",
>>>              "instance_id": "...",
>>>              "url": "https://us-south.ml.cloud.ibm.com"
>>>            },
>>>         project_id="...",
>>>         space_id="...")
>>>
>>> deployment.create(
>>>        experiment_run_id="...",
>>>        model=model,
>>>        deployment_name='My new deployment'
>>>    )

delete(deployment_id: str = None) → None[source]¶

Delete deployment on WML.

deployment_id: str, optional: ID of the deployment to delete. If empty, current deployment will be deleted.

>>> deployment = Batch(workspace=...)
>>> # Delete current deployment
>>> deployment.delete()
>>> # Or delete a specific deployment
>>> deployment.delete(deployment_id='...')

get(deployment_id: str) → None[source]¶

Get WML deployment.

deployment_id: str, required: ID of the deployment to work with.

WebService deployment object

>>> deployment = Batch(workspace=...)
>>> deployment.get(deployment_id="...")

get_job_id(batch_scoring_details)[source]¶: Get id from batch scoring details.

get_job_params(scoring_job_id: str = None) → Dict[source]¶

Get batch deployment job parameters.

scoring_job_id: str: Id of scoring job.

Dictionary with parameters of the scoring job.

get_job_result(scoring_job_id: str) → pandas.core.frame.DataFrame[source]¶

Get batch deployment results of job with id scoring_job_id.

scoring_job_id: str: Id of scoring job which results will be returned.

DataFrame with result.: In case of incompleted or failed MissingScoringResults scoring exception is raised.

get_job_status(scoring_job_id: str) → Dict[source]¶

Get status of scoring job.

scoring_job_id: str: Id of scoring job.

Dictionary with state of scoring job (one of: [completed, failed, starting, queued]): and additional details if they exist.

get_params() → Dict