delete(name, requestId=None, x__xgafv=None)

Deletes a single migration execution.

Args:
name: string, Required. The relative resource name of the migrationExecution to delete, in the following form:projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}. (required)
requestId: string, Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.The request ID must be a valid UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier#Format) A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format

Returns:
An object of the form:

{ # This resource represents a long-running operation that is the result of a network API call.
"done": True or False, # If the value is false, it means the operation is still in progress. If true, the operation is completed, and either error or response is available.
"error": { # The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC (https://github.com/grpc). Each Status message contains three pieces of data: error code, error message, and error details.You can find out more about this error model and how to work with it in the API Design Guide (https://cloud.google.com/apis/design/errors). # The error result of the operation in case of failure or cancellation.
"code": 42, # The status code, which should be an enum value of google.rpc.Code.
"details": [ # A list of messages that carry the error details. There is a common set of message types for APIs to use.
{
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
],
"message": "A String", # A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.
},
"metadata": { # Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any.
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
"name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the name should be a resource name ending with operations/{unique_id}.
"response": { # The normal, successful response of the operation. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is standard Get/Create/Update, the response should be the resource. For other methods, the response should have the type XxxResponse, where Xxx is the original method name. For example, if the original method name is TakeSnapshot(), the inferred response type is TakeSnapshotResponse.
"a_key": "", # Properties of the object. Contains field @type with type URL.
},
}

get(name, x__xgafv=None)

Gets details of a single migration execution.

Args:
name: string, Required. The relative resource name of the migration execution to retrieve, in the following form:projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}. (required)
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format

Returns:
An object of the form:

{ # The details of a migration execution resource.
"biglakeMetastoreMigrationConfig": { # Defines the configuration required to migrate metadata from a Dataproc Metastore service to BigLake Metastore. # Configuration information specific to migrating from Dataproc Metastore to BigLake Metastore.
"backfillStatus": { # Backfill status for the migration execution. # Output only.
"migrationSummary": { # Summary of the migration results. # Output only. Summary of the migration results. This is populated after the backfill or dry run is finished.
"catalogSummaries": [ # Output only. Summary of results for each catalog involved in the migration.
{ # Summary of results for a specific destination catalog.
"catalog": "A String", # Output only. The catalog resource name (format: projects/*/catalogs/*).
"catalogType": "A String", # Output only. The type of the catalog.
"databaseSummaries": [ # Output only. Summary of results for each database in the catalog.
{ # Summary of results for a specific database in a catalog.
"database": "A String", # Output only. The name of the database.
"planAction": "A String", # Output only. The migration plan action for the database.
"resultStatus": "A String", # Output only. The migration result status for the database. This is only set if the migration is not a dry run.
"tableSummary": { # Aggregated summary of results for all tables in a database. # Output only. Aggregated summary of results for all tables in the database.
"partitionDiscoveredCount": "A String", # Output only. Partition migration summary across all Hive tables in the database.The total number of partitions discovered at the source.
"partitionFailedCount": "A String", # Output only. The total number of partitions that failed to migrate at the target.
"partitionSuccessCount": "A String", # Output only. The total number of partitions successfully migrated at the target.
"planCounts": { # Output only. Number of tables with a specific migration plan action. The key is the action name (e.g. CREATE, UPDATE, SKIP, etc.).
"a_key": "A String",
},
"resultCounts": { # Output only. Number of tables with a specific migration result status. The key is the status name (e.g. SUCCEEDED, FAILED, SKIPPED, etc.). This is only set if the migration is not a dry run.
"a_key": "A String",
},
},
},
],
},
],
"createTime": "A String", # Output only. The UTC time when this report was finalized.
"dryRun": True or False, # Output only. Whether the migration was a dry run.
"service": "A String", # Output only. The Dataproc Metastore service name (format: projects/*/locations/*/services/*) on which the migration was executed.
},
"reportPath": "A String", # Output only. The Cloud Storage path where the backfill or dry run report is written. Format: "gs://path-to-report".
"state": "A String", # Output only. The current state of the backfill (or dry run).
},
"conflictPolicy": "A String", # Optional. The policy to handle conflicts when migrating resources, defaults to SKIP if not specified.
"dryRun": True or False, # Optional. If true, performs discovery of requested resources and analysis against the target catalog to come up with a plan for each resource (e.g. Create, Update, Skip, etc.). No metadata is actually migrated.
"hiveConfig": { # Configuration for migrating Hive metadata. # Optional. At least one of hive_config or iceberg_config must be provided, otherwise, a validation error will be thrown. If only one is provided, the service only migrates tables of that specific type. If both are provided, both Hive and Iceberg tables will be migrated.Configuration for migrating Hive tables to a BigLake Hive catalog.
"catalog": "A String", # Required. The target catalog for migrated databases and tables. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
"databases": [ # Required. The list of databases to migrate to the Hive catalog. Use "*" to migrate all databases. Note: If Iceberg tables exist in these databases, they will only be migrated if iceberg_config is also specified.
"A String",
],
},
"icebergConfig": { # Configuration for migrating Iceberg metadata. # Optional. Configuration for migrating Iceberg tables to a BigLake Iceberg REST catalog.
"catalog": "A String", # Required. The target catalog for migrated Iceberg metadata. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
"namespaces": [ # Required. The list of namespaces to migrate to the Iceberg REST catalog. Use "*" to migrate all namespaces. Note: If Hive tables exist in these namespaces, they will only be migrated if hive_config is also specified.
"A String",
],
},
"mode": "A String", # Required. Defines the behavior of the migration execution.
"reportPath": "A String", # Optional. The Cloud Storage path where the backfill / dry run report should be written. If not provided, the report will be generated in the service's artifacts bucket. Format: "gs://path/to/folder"
},
"cloudSqlMigrationConfig": { # Deprecated: Migrations to Dataproc Metastore are no longer supported. Use BigLake Metastore migration instead. Configuration information for migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore. # Deprecated: Migrations to Dataproc Metastore are no longer supported. Use BigLake Metastore migration instead. Configuration information specific to migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.
"cdcConfig": { # Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. # Required. Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. Dataproc Metastore switches to using its backend database after the cutover phase of migration.
"bucket": "A String", # Optional. The bucket to write the intermediate stream event data in. The bucket name must be without any prefix like "gs://". See the bucket naming requirements (https://cloud.google.com/storage/docs/buckets#naming). This field is optional. If not set, the Artifacts Cloud Storage bucket will be used.
"password": "A String", # Required. Input only. The password for the user that Datastream service should use for the MySQL connection. This field is not returned on request.
"reverseProxySubnet": "A String", # Required. The URL of the subnetwork resource to create the VM instance hosting the reverse proxy in. More context in https://cloud.google.com/datastream/docs/private-connectivity#reverse-csql-proxy The subnetwork should reside in the network provided in the request that Datastream will peer to and should be in the same region as Datastream, in the following format. projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"rootPath": "A String", # Optional. The root path inside the Cloud Storage bucket. The stream event data will be written to this path. The default value is /migration.
"subnetIpRange": "A String", # Required. A /29 CIDR IP range for peering with datastream.
"username": "A String", # Required. The username that the Datastream service should use for the MySQL connection.
"vpcNetwork": "A String", # Required. Fully qualified name of the Cloud SQL instance's VPC network or the shared VPC network that Datastream will peer to, in the following format: projects/{project_id}/locations/global/networks/{network_id}. More context in https://cloud.google.com/datastream/docs/network-connectivity-options#privateconnectivity
},
"cloudSqlConnectionConfig": { # Configuration information to establish customer database connection before the cutover phase of migration # Required. Configuration information to establish customer database connection before the cutover phase of migration
"hiveDatabaseName": "A String", # Required. The hive database name.
"instanceConnectionName": "A String", # Required. Cloud SQL database connection name (project_id:region:instance_name)
"ipAddress": "A String", # Required. The private IP address of the Cloud SQL instance.
"natSubnet": "A String", # Required. The relative resource name of the subnetwork to be used for Private Service Connect. Note that this cannot be a regular subnet and is used only for NAT. (https://cloud.google.com/vpc/docs/about-vpc-hosted-services#psc-subnets) This subnet is used to publish the SOCKS5 proxy service. The subnet size must be at least /29 and it should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"password": "A String", # Required. Input only. The password for the user that Dataproc Metastore service will be using to connect to the database. This field is not returned on request.
"port": 42, # Required. The network port of the database.
"proxySubnet": "A String", # Required. The relative resource name of the subnetwork to deploy the SOCKS5 proxy service in. The subnetwork should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"username": "A String", # Required. The username that Dataproc Metastore service will use to connect to the database.
},
},
"createTime": "A String", # Output only. The time when the migration execution was started.
"endTime": "A String", # Output only. The time when the migration execution finished.
"name": "A String", # Output only. The relative resource name of the migration execution, in the following form: projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
"phase": "A String", # Output only. Deprecated: Phase was designed for incoming migrations to Dataproc Metastore, not applicable when migrating away from it. The current phase of the migration execution.
"state": "A String", # Output only. The current state of the migration execution.
"stateMessage": "A String", # Output only. Additional information about the current state of the migration execution.
}

list(parent, filter=None, orderBy=None, pageSize=None, pageToken=None, x__xgafv=None)

Lists migration executions on a service.

Args:
parent: string, Required. The relative resource name of the service whose migration executions to list, in the following form:projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions. (required)
filter: string, Optional. The filter to apply to list results.
orderBy: string, Optional. Specify the ordering of results as described in Sorting Order (https://cloud.google.com/apis/design/design_patterns#sorting_order). If not specified, the results will be sorted in the default order.
pageSize: integer, Optional. The maximum number of migration executions to return. The response may contain less than the maximum number. If unspecified, no more than 500 migration executions are returned. The maximum value is 1000; values above 1000 are changed to 1000.
pageToken: string, Optional. A page token, received from a previous DataprocMetastore.ListMigrationExecutions call. Provide this token to retrieve the subsequent page.To retrieve the first page, supply an empty page token.When paginating, other parameters provided to DataprocMetastore.ListMigrationExecutions must match the call that provided the page token.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format

Returns:
An object of the form:

{ # Response message for DataprocMetastore.ListMigrationExecutions.
"migrationExecutions": [ # The migration executions on the specified service.
{ # The details of a migration execution resource.
"biglakeMetastoreMigrationConfig": { # Defines the configuration required to migrate metadata from a Dataproc Metastore service to BigLake Metastore. # Configuration information specific to migrating from Dataproc Metastore to BigLake Metastore.
"backfillStatus": { # Backfill status for the migration execution. # Output only.
"migrationSummary": { # Summary of the migration results. # Output only. Summary of the migration results. This is populated after the backfill or dry run is finished.
"catalogSummaries": [ # Output only. Summary of results for each catalog involved in the migration.
{ # Summary of results for a specific destination catalog.
"catalog": "A String", # Output only. The catalog resource name (format: projects/*/catalogs/*).
"catalogType": "A String", # Output only. The type of the catalog.
"databaseSummaries": [ # Output only. Summary of results for each database in the catalog.
{ # Summary of results for a specific database in a catalog.
"database": "A String", # Output only. The name of the database.
"planAction": "A String", # Output only. The migration plan action for the database.
"resultStatus": "A String", # Output only. The migration result status for the database. This is only set if the migration is not a dry run.
"tableSummary": { # Aggregated summary of results for all tables in a database. # Output only. Aggregated summary of results for all tables in the database.
"partitionDiscoveredCount": "A String", # Output only. Partition migration summary across all Hive tables in the database.The total number of partitions discovered at the source.
"partitionFailedCount": "A String", # Output only. The total number of partitions that failed to migrate at the target.
"partitionSuccessCount": "A String", # Output only. The total number of partitions successfully migrated at the target.
"planCounts": { # Output only. Number of tables with a specific migration plan action. The key is the action name (e.g. CREATE, UPDATE, SKIP, etc.).
"a_key": "A String",
},
"resultCounts": { # Output only. Number of tables with a specific migration result status. The key is the status name (e.g. SUCCEEDED, FAILED, SKIPPED, etc.). This is only set if the migration is not a dry run.
"a_key": "A String",
},
},
},
],
},
],
"createTime": "A String", # Output only. The UTC time when this report was finalized.
"dryRun": True or False, # Output only. Whether the migration was a dry run.
"service": "A String", # Output only. The Dataproc Metastore service name (format: projects/*/locations/*/services/*) on which the migration was executed.
},
"reportPath": "A String", # Output only. The Cloud Storage path where the backfill or dry run report is written. Format: "gs://path-to-report".
"state": "A String", # Output only. The current state of the backfill (or dry run).
},
"conflictPolicy": "A String", # Optional. The policy to handle conflicts when migrating resources, defaults to SKIP if not specified.
"dryRun": True or False, # Optional. If true, performs discovery of requested resources and analysis against the target catalog to come up with a plan for each resource (e.g. Create, Update, Skip, etc.). No metadata is actually migrated.
"hiveConfig": { # Configuration for migrating Hive metadata. # Optional. At least one of hive_config or iceberg_config must be provided, otherwise, a validation error will be thrown. If only one is provided, the service only migrates tables of that specific type. If both are provided, both Hive and Iceberg tables will be migrated.Configuration for migrating Hive tables to a BigLake Hive catalog.
"catalog": "A String", # Required. The target catalog for migrated databases and tables. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
"databases": [ # Required. The list of databases to migrate to the Hive catalog. Use "*" to migrate all databases. Note: If Iceberg tables exist in these databases, they will only be migrated if iceberg_config is also specified.
"A String",
],
},
"icebergConfig": { # Configuration for migrating Iceberg metadata. # Optional. Configuration for migrating Iceberg tables to a BigLake Iceberg REST catalog.
"catalog": "A String", # Required. The target catalog for migrated Iceberg metadata. Format: "projects/{project_id_or_number}/catalogs/{catalog_id}"
"namespaces": [ # Required. The list of namespaces to migrate to the Iceberg REST catalog. Use "*" to migrate all namespaces. Note: If Hive tables exist in these namespaces, they will only be migrated if hive_config is also specified.
"A String",
],
},
"mode": "A String", # Required. Defines the behavior of the migration execution.
"reportPath": "A String", # Optional. The Cloud Storage path where the backfill / dry run report should be written. If not provided, the report will be generated in the service's artifacts bucket. Format: "gs://path/to/folder"
},
"cloudSqlMigrationConfig": { # Deprecated: Migrations to Dataproc Metastore are no longer supported. Use BigLake Metastore migration instead. Configuration information for migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore. # Deprecated: Migrations to Dataproc Metastore are no longer supported. Use BigLake Metastore migration instead. Configuration information specific to migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.
"cdcConfig": { # Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. # Required. Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. Dataproc Metastore switches to using its backend database after the cutover phase of migration.
"bucket": "A String", # Optional. The bucket to write the intermediate stream event data in. The bucket name must be without any prefix like "gs://". See the bucket naming requirements (https://cloud.google.com/storage/docs/buckets#naming). This field is optional. If not set, the Artifacts Cloud Storage bucket will be used.
"password": "A String", # Required. Input only. The password for the user that Datastream service should use for the MySQL connection. This field is not returned on request.
"reverseProxySubnet": "A String", # Required. The URL of the subnetwork resource to create the VM instance hosting the reverse proxy in. More context in https://cloud.google.com/datastream/docs/private-connectivity#reverse-csql-proxy The subnetwork should reside in the network provided in the request that Datastream will peer to and should be in the same region as Datastream, in the following format. projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"rootPath": "A String", # Optional. The root path inside the Cloud Storage bucket. The stream event data will be written to this path. The default value is /migration.
"subnetIpRange": "A String", # Required. A /29 CIDR IP range for peering with datastream.
"username": "A String", # Required. The username that the Datastream service should use for the MySQL connection.
"vpcNetwork": "A String", # Required. Fully qualified name of the Cloud SQL instance's VPC network or the shared VPC network that Datastream will peer to, in the following format: projects/{project_id}/locations/global/networks/{network_id}. More context in https://cloud.google.com/datastream/docs/network-connectivity-options#privateconnectivity
},
"cloudSqlConnectionConfig": { # Configuration information to establish customer database connection before the cutover phase of migration # Required. Configuration information to establish customer database connection before the cutover phase of migration
"hiveDatabaseName": "A String", # Required. The hive database name.
"instanceConnectionName": "A String", # Required. Cloud SQL database connection name (project_id:region:instance_name)
"ipAddress": "A String", # Required. The private IP address of the Cloud SQL instance.
"natSubnet": "A String", # Required. The relative resource name of the subnetwork to be used for Private Service Connect. Note that this cannot be a regular subnet and is used only for NAT. (https://cloud.google.com/vpc/docs/about-vpc-hosted-services#psc-subnets) This subnet is used to publish the SOCKS5 proxy service. The subnet size must be at least /29 and it should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"password": "A String", # Required. Input only. The password for the user that Dataproc Metastore service will be using to connect to the database. This field is not returned on request.
"port": 42, # Required. The network port of the database.
"proxySubnet": "A String", # Required. The relative resource name of the subnetwork to deploy the SOCKS5 proxy service in. The subnetwork should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
"username": "A String", # Required. The username that Dataproc Metastore service will use to connect to the database.
},
},
"createTime": "A String", # Output only. The time when the migration execution was started.
"endTime": "A String", # Output only. The time when the migration execution finished.
"name": "A String", # Output only. The relative resource name of the migration execution, in the following form: projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
"phase": "A String", # Output only. Deprecated: Phase was designed for incoming migrations to Dataproc Metastore, not applicable when migrating away from it. The current phase of the migration execution.
"state": "A String", # Output only. The current state of the migration execution.
"stateMessage": "A String", # Output only. Additional information about the current state of the migration execution.
},
],
"nextPageToken": "A String", # A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
"unreachable": [ # Locations that could not be reached.
"A String",
],
}

Dataproc Metastore API . projects . locations . services . migrationExecutions

Instance Methods

Method Details