Bedrock / Client / list_evaluation_jobs

list_evaluation_jobs

Bedrock.Client.list_evaluation_jobs(**kwargs)

Lists all existing evaluation jobs.

See also: AWS API Documentation

Request Syntax

response = client.list_evaluation_jobs(
    creationTimeAfter=datetime(2015, 1, 1),
    creationTimeBefore=datetime(2015, 1, 1),
    statusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
    applicationTypeEquals='ModelEvaluation'|'RagEvaluation',
    nameContains='string',
    maxResults=123,
    nextToken='string',
    sortBy='CreationTime',
    sortOrder='Ascending'|'Descending'
)

Parameters:

• creationTimeAfter (datetime) – A filter to only list evaluation jobs created after a specified time.

• creationTimeBefore (datetime) – A filter to only list evaluation jobs created before a specified time.

• statusEquals (string) – A filter to only list evaluation jobs that are of a certain status.

• applicationTypeEquals (string) – A filter to only list evaluation jobs that are either model evaluations or knowledge base evaluations.

• nameContains (string) – A filter to only list evaluation jobs that contain a specified string in the job name.

• maxResults (integer) – The maximum number of results to return.

• nextToken (string) – Continuation token from the previous response, for Amazon Bedrock to list the next set of results.

• sortBy (string) – Specifies a creation time to sort the list of evaluation jobs by when they were created.

• sortOrder (string) – Specifies whether to sort the list of evaluation jobs by either ascending or descending order.

Return type:

dict

Returns:

Response Syntax

{
    'nextToken': 'string',
    'jobSummaries': [
        {
            'jobArn': 'string',
            'jobName': 'string',
            'status': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
            'creationTime': datetime(2015, 1, 1),
            'jobType': 'Human'|'Automated',
            'evaluationTaskTypes': [
                'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
            ],
            'modelIdentifiers': [
                'string',
            ],
            'ragIdentifiers': [
                'string',
            ],
            'evaluatorModelIdentifiers': [
                'string',
            ],
            'inferenceConfigSummary': {
                'modelConfigSummary': {
                    'bedrockModelIdentifiers': [
                        'string',
                    ],
                    'precomputedInferenceSourceIdentifiers': [
                        'string',
                    ]
                },
                'ragConfigSummary': {
                    'bedrockKnowledgeBaseIdentifiers': [
                        'string',
                    ],
                    'precomputedRagSourceIdentifiers': [
                        'string',
                    ]
                }
            },
            'applicationType': 'ModelEvaluation'|'RagEvaluation'
        },
    ]
}

Response Structure

• (dict) –

  • nextToken (string) –

    Continuation token from the previous response, for Amazon Bedrock to list the next set of results.

  • jobSummaries (list) –

    A list of summaries of the evaluation jobs.

    • (dict) –

      Summary information of an evaluation job.

      • jobArn (string) –

        The Amazon Resource Name (ARN) of the evaluation job.

      • jobName (string) –

        The name for the evaluation job.

      • status (string) –

        The current status of the evaluation job.

      • creationTime (datetime) –

        The time the evaluation job was created.

      • jobType (string) –

        Specifies whether the evaluation job is automated or human-based.

      • evaluationTaskTypes (list) –

        The type of task for model evaluation.

        • (string) –

      • modelIdentifiers (list) –

        The Amazon Resource Names (ARNs) of the model(s) used for the evaluation job.

        • (string) –

      • ragIdentifiers (list) –

        The Amazon Resource Names (ARNs) of the knowledge base resources used for a knowledge base evaluation job.

        • (string) –

      • evaluatorModelIdentifiers (list) –

        The Amazon Resource Names (ARNs) of the models used to compute the metrics for a knowledge base evaluation job.

        • (string) –

      • inferenceConfigSummary (dict) –

        Identifies the models, Knowledge Bases, or other RAG sources evaluated in a model or Knowledge Base evaluation job.

        • modelConfigSummary (dict) –

          A summary of the models used in an Amazon Bedrock model evaluation job. These resources can be models in Amazon Bedrock or models outside of Amazon Bedrock that you use to generate your own inference response data.

          • bedrockModelIdentifiers (list) –

            The Amazon Resource Names (ARNs) of the models used for the evaluation job.

            • (string) –

          • precomputedInferenceSourceIdentifiers (list) –

            A label that identifies the models used for a model evaluation job where you provide your own inference response data.

            • (string) –

        • ragConfigSummary (dict) –

          A summary of the RAG resources used in an Amazon Bedrock Knowledge Base evaluation job. These resources can be Knowledge Bases in Amazon Bedrock or RAG sources outside of Amazon Bedrock that you use to generate your own inference response data.

          • bedrockKnowledgeBaseIdentifiers (list) –

            The Amazon Resource Names (ARNs) of the Knowledge Base resources used for a Knowledge Base evaluation job where Amazon Bedrock invokes the Knowledge Base for you.

            • (string) –

          • precomputedRagSourceIdentifiers (list) –

            A label that identifies the RAG sources used for a Knowledge Base evaluation job where you provide your own inference response data.

            • (string) –

      • applicationType (string) –

        Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).

Exceptions

• Bedrock.Client.exceptions.AccessDeniedException

• Bedrock.Client.exceptions.ValidationException

• Bedrock.Client.exceptions.InternalServerException

• Bedrock.Client.exceptions.ThrottlingException