PaLM API: Tuning Quickstart with Python

View on Generative AI Run in Google Colab View source on GitHub Download notebook

In this notebook, you'll learn how to get started with the tuning service using the Python client library for the PaLM API. Here, you'll learn how to tune the text model behind the PaLM API's text generation service.

Setup

Authenticate

The PaLM API lets you tune models on your own data. Since it's your data and your tuned models this needs stricter access controls than API-Keys can provide.

Before you can run this tutorial, you'll need to setup OAuth for your project.

If you want to run this notebook in Colab start by uploading your client_secret*.json file using the "File > Upload" option.

cp client_secret*.json client_secret.json
ls client_secret.json
client_secret.json

This gcloud command turns the client_secret.json file into credentials that can be used to authenticate with the service.

import os
if 'COLAB_RELEASE_TAG' in os.environ:
  # Use `--no-browser` in colab
  !gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'
else:
  !gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'

Install the client library

pip install -q google-generativeai

Import libraries

import google.generativeai as palm

You can check you existing tuned models with the palm.list_tuned_model method.

for i, m in zip(range(5), palm.list_tuned_models()):
  print(m.name)
tunedModels/my-model-8527
tunedModels/my-model-7092
tunedModels/my-model-2778
tunedModels/my-model-1298
tunedModels/my-model-3883

Create tuned model

To create a tuned model, you need to pass your dataset to the model in the palm.create_tuned_model method. You can do this be directly defining the input and output values in the call or importing from a file into a dataframe to pass to the method.

For this example, you will tune a model to generate the next number in the sequence. For example, if the input is 1, the model should output 2. If the input is one hundred, the output should be one hundred one.

base_model = [
    m for m in palm.list_models()
    if "createTunedTextModel" in m.supported_generation_methods][0]
base_model.name
'models/text-bison-001'
import random

name = f'generate-num-{random.randint(0,10000)}'
operation = palm.create_tuned_model(
    # You can use a tuned model here too. Set `source_model="tunedModels/..."` 
    source_model=base_model.name,
    training_data=[
        {
             'text_input': '1',
             'output': '2',
        },{
             'text_input': '3',
             'output': '4',
        },{
             'text_input': '-3',
             'output': '-2',
        },{
             'text_input': 'twenty two',
             'output': 'twenty three',
        },{
             'text_input': 'two hundred',
             'output': 'two hundred one',
        },{
             'text_input': 'ninety nine',
             'output': 'one hundred',
        },{
             'text_input': '8',
             'output': '9',
        },{
             'text_input': '-98',
             'output': '-97',
        },{
             'text_input': '1,000',
             'output': '1,001',
        },{
             'text_input': '10,100,000',
             'output': '10,100,001',
        },{
             'text_input': 'thirteen',
             'output': 'fourteen',
        },{
             'text_input': 'eighty',
             'output': 'eighty one',
        },{
             'text_input': 'one',
             'output': 'two',
        },{
             'text_input': 'three',
             'output': 'four',
        },{
             'text_input': 'seven',
             'output': 'eight',
        }
    ],
    id = name,
    epoch_count = 100,
    batch_size=4,
    learning_rate=0.001,
)

Your tuned model is immediately added to the list of tuned models, but its status is set to "creating" while the model is tuned.

model = palm.get_tuned_model(f'tunedModels/{name}')

model
TunedModel(name='tunedModels/generate-num-9028',
           source_model='tunedModels/generate-num-4110',
           base_model='models/text-bison-001',
           display_name='',
           description='',
           temperature=0.7,
           top_p=0.95,
           top_k=40,
           state=<State.CREATING: 1>,
           create_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 188028, tzinfo=datetime.timezone.utc),
           update_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 188028, tzinfo=datetime.timezone.utc),
           tuning_task=TuningTask(start_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 734118, tzinfo=datetime.timezone.utc),
                                  complete_time=None,
                                  snapshots=[],
                                  hyperparameters=Hyperparameters(epoch_count=100,
                                                                  batch_size=4,
                                                                  learning_rate=0.001)))
model.state
<State.CREATING: 1>

Check tuning progress

Use metadata to check the state:

operation.metadata
tuned_model: "tunedModels/generate-num-9028"
total_steps: 375

Wait for the training to finish using operation.result(), or operation.wait_bar()

import time

for status in operation.wait_bar():
  time.sleep(30)
0%|          | 0/375 [00:00<?, ?it/s]

You can cancel your tuning job any time using the cancel() method. Uncomment the line below and run the code cell to cancel your job before it finishes.

# operation.cancel()

Once the tuning is complete, you can view the loss curve from the tuning results. The loss curve shows how much the model's predictions deviate from the ideal outputs.

import pandas as pd
import seaborn as sns

model = operation.result()

snapshots = pd.DataFrame(model.tuning_task.snapshots)

sns.lineplot(data=snapshots, x = 'epoch', y='mean_loss')
<Axes: xlabel='epoch', ylabel='mean_loss'>

png

Evaluate your model

You can use the palm.generate_text method and specify the name of your model to test your model performance.

completion = palm.generate_text(model=f'tunedModels/{name}',
                                prompt='5')
completion.result
'6'
completion = palm.generate_text(model=f'tunedModels/{name}',
                                prompt='-9')
completion.result
'-8'
completion = palm.generate_text(model=f'tunedModels/{name}',
                                prompt='four')
completion.result
'four'

As you can see, the last prompt didn't produce the ideal result, five. To produce better results you can try a few different things such as adjusting the temperature closer to zero to get more consistent results, adding more quality examples to your dataset that the model can learn from or adding a prompt or preamble to the examples.

See the tuning guide for more guidance on improving performance.

Update the description

You can update the description of your tuned model any time using the palm.update_tuned_model method.

palm.update_tuned_model(f'tunedModels/{name}', {"description":"This is my model."})
TunedModel(name='', source_model=None, base_model=None, display_name='', description='This is my model.', temperature=None, top_p=None, top_k=None, state=<State.STATE_UNSPECIFIED: 0>, create_time=None, update_time=None, tuning_task=None)
model = palm.get_tuned_model(f'tunedModels/{name}')

model
TunedModel(name='tunedModels/generate-num-4668',
           source_model=None,
           base_model='models/text-bison-001',
           display_name='',
           description='This is my model.',
           temperature=0.7,
           top_p=0.95,
           top_k=40,
           state=<State.ACTIVE: 2>,
           create_time=datetime.datetime(2023, 9, 19, 19, 3, 38, 22249, tzinfo=<UTC>),
           update_time=datetime.datetime(2023, 9, 19, 19, 11, 48, 101024, tzinfo=<UTC>),
           tuning_task=TuningTask(start_time=datetime.datetime(2023, 9, 19, 19, 3, 38, 562798, tzinfo=<UTC>),
                                  complete_time=datetime.datetime(2023, 9, 19, 19, 11, 48, 101024, tzinfo=<UTC>),
                                  snapshots=[{'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 41, 221503, tzinfo=<UTC>),
                                              'epoch': 0,
                                              'mean_loss': 7.2774773,
                                              'step': 1},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 42, 611142, tzinfo=<UTC>),
                                              'epoch': 0,
                                              'mean_loss': 6.178241,
                                              'step': 2},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 43, 886844, tzinfo=<UTC>),
                                              'epoch': 0,
                                              'mean_loss': 5.505934,
                                              'step': 3},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 45, 213316, tzinfo=<UTC>),
                                              'epoch': 1,
                                              'mean_loss': 7.9365344,
                                              'step': 4},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 46, 719674, tzinfo=<UTC>),
                                              'epoch': 1,
                                              'mean_loss': 7.656596,
                                              'step': 5},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 48, 51559, tzinfo=<UTC>),
                                              'epoch': 1,
                                              'mean_loss': 7.3750257,
                                              'step': 6},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 49, 419247, tzinfo=<UTC>),
                                              'epoch': 1,
                                              'mean_loss': 4.579882,
                                              'step': 7},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 50, 902477, tzinfo=<UTC>),
                                              'epoch': 2,
                                              'mean_loss': 6.776862,
                                              'step': 8},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 52, 213448, tzinfo=<UTC>),
                                              'epoch': 2,
                                              'mean_loss': 6.3564157,
                                              'step': 9},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 53, 679693, tzinfo=<UTC>),
                                              'epoch': 2,
                                              'mean_loss': 8.558726,
                                              'step': 10},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 55, 2348, tzinfo=<UTC>),
                                              'epoch': 2,
                                              'mean_loss': 4.783774,
                                              'step': 11},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 56, 322485, tzinfo=<UTC>),
                                              'epoch': 3,
                                              'mean_loss': 7.0234137,
                                              'step': 12},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 58, 145081, tzinfo=<UTC>),
                                              'epoch': 3,
                                              'mean_loss': 7.317513,
                                              'step': 13},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 59, 399317, tzinfo=<UTC>),
                                              'epoch': 3,
                                              'mean_loss': 5.85363,
                                              'step': 14},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 0, 646995, tzinfo=<UTC>),
                                              'epoch': 4,
                                              'mean_loss': 4.21408,
                                              'step': 15},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 1, 899798, tzinfo=<UTC>),
                                              'epoch': 4,
                                              'mean_loss': 6.6232214,
                                              'step': 16},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 3, 167955, tzinfo=<UTC>),
                                              'epoch': 4,
                                              'mean_loss': 5.61497,
                                              'step': 17},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 4, 407849, tzinfo=<UTC>),
                                              'epoch': 4,
                                              'mean_loss': 6.821261,
                                              'step': 18},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 5, 649503, tzinfo=<UTC>),
                                              'epoch': 5,
                                              'mean_loss': 3.8338904,
                                              'step': 19},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 7, 80497, tzinfo=<UTC>),
                                              'epoch': 5,
                                              'mean_loss': 5.0643735,
                                              'step': 20},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 8, 401424, tzinfo=<UTC>),
                                              'epoch': 5,
                                              'mean_loss': 6.976447,
                                              'step': 21},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 9, 688226, tzinfo=<UTC>),
                                              'epoch': 5,
                                              'mean_loss': 5.045044,
                                              'step': 22},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 10, 942147, tzinfo=<UTC>),
                                              'epoch': 6,
                                              'mean_loss': 5.1944356,
                                              'step': 23},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 12, 169501, tzinfo=<UTC>),
                                              'epoch': 6,
                                              'mean_loss': 5.342552,
                                              'step': 24},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 13, 532023, tzinfo=<UTC>),
                                              'epoch': 6,
                                              'mean_loss': 7.360283,
                                              'step': 25},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 14, 773265, tzinfo=<UTC>),
                                              'epoch': 6,
                                              'mean_loss': 2.874686,
                                              'step': 26},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 16, 68826, tzinfo=<UTC>),
                                              'epoch': 7,
                                              'mean_loss': 5.0835795,
                                              'step': 27},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 17, 328292, tzinfo=<UTC>),
                                              'epoch': 7,
                                              'mean_loss': 4.059507,
                                              'step': 28},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 18, 683769, tzinfo=<UTC>),
                                              'epoch': 7,
                                              'mean_loss': 4.668791,
                                              'step': 29},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 19, 917365, tzinfo=<UTC>),
                                              'epoch': 8,
                                              'mean_loss': 3.2776065,
                                              'step': 30},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 21, 175338, tzinfo=<UTC>),
                                              'epoch': 8,
                                              'mean_loss': 4.1344976,
                                              'step': 31},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 22, 510908, tzinfo=<UTC>),
                                              'epoch': 8,
                                              'mean_loss': 4.47365,
                                              'step': 32},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 23, 972490, tzinfo=<UTC>),
                                              'epoch': 8,
                                              'mean_loss': 2.8087254,
                                              'step': 33},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 25, 341109, tzinfo=<UTC>),
                                              'epoch': 9,
                                              'mean_loss': 3.581566,
                                              'step': 34},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 26, 594799, tzinfo=<UTC>),
                                              'epoch': 9,
                                              'mean_loss': 3.3534799,
                                              'step': 35},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 27, 857511, tzinfo=<UTC>),
                                              'epoch': 9,
                                              'mean_loss': 2.5248497,
                                              'step': 36},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 29, 100872, tzinfo=<UTC>),
                                              'epoch': 9,
                                              'mean_loss': 1.8420736,
                                              'step': 37},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 30, 356383, tzinfo=<UTC>),
                                              'epoch': 10,
                                              'mean_loss': 3.4610085,
                                              'step': 38},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 31, 819918, tzinfo=<UTC>),
                                              'epoch': 10,
                                              'mean_loss': 3.2506752,
                                              'step': 39},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 33, 77814, tzinfo=<UTC>),
                                              'epoch': 10,
                                              'mean_loss': 2.4844272,
                                              'step': 40},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 34, 314311, tzinfo=<UTC>),
                                              'epoch': 10,
                                              'mean_loss': 2.3858242,
                                              'step': 41},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 35, 572181, tzinfo=<UTC>),
                                              'epoch': 11,
                                              'mean_loss': 1.1961311,
                                              'step': 42},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 36, 845239, tzinfo=<UTC>),
                                              'epoch': 11,
                                              'mean_loss': 3.5777583,
                                              'step': 43},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 38, 120182, tzinfo=<UTC>),
                                              'epoch': 11,
                                              'mean_loss': 1.3613169,
                                              'step': 44},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 39, 611773, tzinfo=<UTC>),
                                              'epoch': 12,
                                              'mean_loss': 1.7414228,
                                              'step': 45},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 41, 835960, tzinfo=<UTC>),
                                              'epoch': 12,
                                              'mean_loss': 1.3301177,
                                              'step': 46},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 43, 118015, tzinfo=<UTC>),
                                              'epoch': 12,
                                              'mean_loss': 1.3805578,
                                              'step': 47},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 44, 383045, tzinfo=<UTC>),
                                              'epoch': 12,
                                              'mean_loss': 2.3191347,
                                              'step': 48},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 45, 617675, tzinfo=<UTC>),
                                              'epoch': 13,
                                              'mean_loss': 1.7018254,
                                              'step': 49},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 46, 856463, tzinfo=<UTC>),
                                              'epoch': 13,
                                              'mean_loss': 1.5530272,
                                              'step': 50},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 48, 159606, tzinfo=<UTC>),
                                              'epoch': 13,
                                              'mean_loss': 2.1536818,
                                              'step': 51},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 49, 388434, tzinfo=<UTC>),
                                              'epoch': 13,
                                              'mean_loss': 0.87225634,
                                              'step': 52},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 50, 649576, tzinfo=<UTC>),
                                              'epoch': 14,
                                              'mean_loss': 1.6638466,
                                              'step': 53},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 52, 113467, tzinfo=<UTC>),
                                              'epoch': 14,
                                              'mean_loss': 1.4672767,
                                              'step': 54},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 53, 491995, tzinfo=<UTC>),
                                              'epoch': 14,
                                              'mean_loss': 0.66232294,
                                              'step': 55},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 54, 849227, tzinfo=<UTC>),
                                              'epoch': 14,
                                              'mean_loss': 1.2151186,
                                              'step': 56},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 56, 117613, tzinfo=<UTC>),
                                              'epoch': 15,
                                              'mean_loss': 0.75382125,
                                              'step': 57},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 58, 244537, tzinfo=<UTC>),
                                              'epoch': 15,
                                              'mean_loss': 0.909588,
                                              'step': 58},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 59, 495142, tzinfo=<UTC>),
                                              'epoch': 15,
                                              'mean_loss': 0.85212016,
                                              'step': 59},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 0, 748073, tzinfo=<UTC>),
                                              'epoch': 16,
                                              'mean_loss': 1.0999682,
                                              'step': 60},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 2, 9621, tzinfo=<UTC>),
                                              'epoch': 16,
                                              'mean_loss': 0.49189907,
                                              'step': 61},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 3, 289800, tzinfo=<UTC>),
                                              'epoch': 16,
                                              'mean_loss': 1.2313881,
                                              'step': 62},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 4, 542260, tzinfo=<UTC>),
                                              'epoch': 16,
                                              'mean_loss': 0.4186042,
                                              'step': 63},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 5, 789722, tzinfo=<UTC>),
                                              'epoch': 17,
                                              'mean_loss': 0.5968985,
                                              'step': 64},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 7, 21547, tzinfo=<UTC>),
                                              'epoch': 17,
                                              'mean_loss': 0.32776576,
                                              'step': 65},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 8, 253903, tzinfo=<UTC>),
                                              'epoch': 17,
                                              'mean_loss': 0.085846476,
                                              'step': 66},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 9, 503217, tzinfo=<UTC>),
                                              'epoch': 17,
                                              'mean_loss': 0.87150824,
                                              'step': 67},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 10, 755627, tzinfo=<UTC>),
                                              'epoch': 18,
                                              'mean_loss': 0.50882834,
                                              'step': 68},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 11, 981698, tzinfo=<UTC>),
                                              'epoch': 18,
                                              'mean_loss': 0.05643571,
                                              'step': 69},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 13, 238454, tzinfo=<UTC>),
                                              'epoch': 18,
                                              'mean_loss': 0.11667071,
                                              'step': 70},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 14, 474345, tzinfo=<UTC>),
                                              'epoch': 18,
                                              'mean_loss': 0.05200408,
                                              'step': 71},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 15, 692710, tzinfo=<UTC>),
                                              'epoch': 19,
                                              'mean_loss': 0.21968448,
                                              'step': 72},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 16, 930777, tzinfo=<UTC>),
                                              'epoch': 19,
                                              'mean_loss': 0.071391255,
                                              'step': 73},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 18, 180590, tzinfo=<UTC>),
                                              'epoch': 19,
                                              'mean_loss': 0.39031163,
                                              'step': 74},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 19, 433064, tzinfo=<UTC>),
                                              'epoch': 20,
                                              'mean_loss': 0.05084487,
                                              'step': 75},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 20, 677200, tzinfo=<UTC>),
                                              'epoch': 20,
                                              'mean_loss': 0.04713744,
                                              'step': 76},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 21, 901118, tzinfo=<UTC>),
                                              'epoch': 20,
                                              'mean_loss': 0.196708,
                                              'step': 77},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 23, 166260, tzinfo=<UTC>),
                                              'epoch': 20,
                                              'mean_loss': 0.15159458,
                                              'step': 78},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 24, 400680, tzinfo=<UTC>),
                                              'epoch': 21,
                                              'mean_loss': 0.0280451,
                                              'step': 79},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 25, 644378, tzinfo=<UTC>),
                                              'epoch': 21,
                                              'mean_loss': 0.06759574,
                                              'step': 80},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 27, 195128, tzinfo=<UTC>),
                                              'epoch': 21,
                                              'mean_loss': 0.03170073,
                                              'step': 81},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 28, 546850, tzinfo=<UTC>),
                                              'epoch': 21,
                                              'mean_loss': 0.15327619,
                                              'step': 82},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 29, 953511, tzinfo=<UTC>),
                                              'epoch': 22,
                                              'mean_loss': 0.14349619,
                                              'step': 83},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 31, 334082, tzinfo=<UTC>),
                                              'epoch': 22,
                                              'mean_loss': 0.02684513,
                                              'step': 84},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 32, 832994, tzinfo=<UTC>),
                                              'epoch': 22,
                                              'mean_loss': 0.019874452,
                                              'step': 85},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 34, 88577, tzinfo=<UTC>),
                                              'epoch': 22,
                                              'mean_loss': 0.041133285,
                                              'step': 86},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 35, 346650, tzinfo=<UTC>),
                                              'epoch': 23,
                                              'mean_loss': 0.06348712,
                                              'step': 87},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 36, 585951, tzinfo=<UTC>),
                                              'epoch': 23,
                                              'mean_loss': 0.025213383,
                                              'step': 88},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 37, 818161, tzinfo=<UTC>),
                                              'epoch': 23,
                                              'mean_loss': 0.018140253,
                                              'step': 89},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 39, 72239, tzinfo=<UTC>),
                                              'epoch': 24,
                                              'mean_loss': 0.023763947,
                                              'step': 90},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 40, 278252, tzinfo=<UTC>),
                                              'epoch': 24,
                                              'mean_loss': 0.008751405,
                                              'step': 91},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 42, 747639, tzinfo=<UTC>),
                                              'epoch': 24,
                                              'mean_loss': 0.0082112085,
                                              'step': 92},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 44, 8609, tzinfo=<UTC>),
                                              'epoch': 24,
                                              'mean_loss': 0.037568945,
                                              'step': 93},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 45, 264108, tzinfo=<UTC>),
                                              'epoch': 25,
                                              'mean_loss': 0.027123686,
                                              'step': 94},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 46, 526159, tzinfo=<UTC>),
                                              'epoch': 25,
                                              'mean_loss': 0.0142503055,
                                              'step': 95},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 47, 768520, tzinfo=<UTC>),
                                              'epoch': 25,
                                              'mean_loss': 0.027518341,
                                              'step': 96},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 49, 89441, tzinfo=<UTC>),
                                              'epoch': 25,
                                              'mean_loss': 0.013976067,
                                              'step': 97},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 50, 342449, tzinfo=<UTC>),
                                              'epoch': 26,
                                              'mean_loss': 0.0036393465,
                                              'step': 98},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 51, 613018, tzinfo=<UTC>),
                                              'epoch': 26,
                                              'mean_loss': 0.0058721625,
                                              'step': 99},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 52, 847043, tzinfo=<UTC>),
                                              'epoch': 26,
                                              'mean_loss': 0.0008192812,
                                              'step': 100},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 54, 81155, tzinfo=<UTC>),
                                              'epoch': 26,
                                              'mean_loss': 0.025449298,
                                              'step': 101},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 55, 304198, tzinfo=<UTC>),
                                              'epoch': 27,
                                              'mean_loss': 0.0066863927,
                                              'step': 102},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 56, 557883, tzinfo=<UTC>),
                                              'epoch': 27,
                                              'mean_loss': 0.002721126,
                                              'step': 103},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 57, 787163, tzinfo=<UTC>),
                                              'epoch': 27,
                                              'mean_loss': 0.015793594,
                                              'step': 104},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 59, 81532, tzinfo=<UTC>),
                                              'epoch': 28,
                                              'mean_loss': 0.005504051,
                                              'step': 105},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 0, 328216, tzinfo=<UTC>),
                                              'epoch': 28,
                                              'mean_loss': 0.003016818,
                                              'step': 106},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 1, 561541, tzinfo=<UTC>),
                                              'epoch': 28,
                                              'mean_loss': 0.014285667,
                                              'step': 107},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 2, 782923, tzinfo=<UTC>),
                                              'epoch': 28,
                                              'mean_loss': 0.004257772,
                                              'step': 108},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 4, 10208, tzinfo=<UTC>),
                                              'epoch': 29,
                                              'mean_loss': 0.010257841,
                                              'step': 109},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 5, 268123, tzinfo=<UTC>),
                                              'epoch': 29,
                                              'mean_loss': 0.0075931884,
                                              'step': 110},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 6, 485682, tzinfo=<UTC>),
                                              'epoch': 29,
                                              'mean_loss': 0.024589492,
                                              'step': 111},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 7, 731568, tzinfo=<UTC>),
                                              'epoch': 29,
                                              'mean_loss': 0.0012908785,
                                              'step': 112},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 8, 978691, tzinfo=<UTC>),
                                              'epoch': 30,
                                              'mean_loss': 0.011656972,
                                              'step': 113},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 10, 254824, tzinfo=<UTC>),
                                              'epoch': 30,
                                              'mean_loss': 0.006077944,
                                              'step': 114},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 11, 489143, tzinfo=<UTC>),
                                              'epoch': 30,
                                              'mean_loss': 0.004817399,
                                              'step': 115},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 12, 772297, tzinfo=<UTC>),
                                              'epoch': 30,
                                              'mean_loss': 0.007809804,
                                              'step': 116},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 14, 19305, tzinfo=<UTC>),
                                              'epoch': 31,
                                              'mean_loss': 0.006533157,
                                              'step': 117},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 15, 257150, tzinfo=<UTC>),
                                              'epoch': 31,
                                              'mean_loss': 0.005006207,
                                              'step': 118},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 16, 536830, tzinfo=<UTC>),
                                              'epoch': 31,
                                              'mean_loss': 0.0004677312,
                                              'step': 119},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 17, 790581, tzinfo=<UTC>),
                                              'epoch': 32,
                                              'mean_loss': 0.007230793,
                                              'step': 120},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 19, 41049, tzinfo=<UTC>),
                                              'epoch': 32,
                                              'mean_loss': 0.0020942457,
                                              'step': 121},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 20, 306408, tzinfo=<UTC>),
                                              'epoch': 32,
                                              'mean_loss': 0.006205416,
                                              'step': 122},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 21, 545345, tzinfo=<UTC>),
                                              'epoch': 32,
                                              'mean_loss': 0.0063284263,
                                              'step': 123},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 22, 759006, tzinfo=<UTC>),
                                              'epoch': 33,
                                              'mean_loss': 0.0063140467,
                                              'step': 124},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 23, 979661, tzinfo=<UTC>),
                                              'epoch': 33,
                                              'mean_loss': 0.00040962745,
                                              'step': 125},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 25, 215222, tzinfo=<UTC>),
                                              'epoch': 33,
                                              'mean_loss': 0.0020570084,
                                              'step': 126},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 26, 457055, tzinfo=<UTC>),
                                              'epoch': 33,
                                              'mean_loss': 0.011655594,
                                              'step': 127},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 27, 705186, tzinfo=<UTC>),
                                              'epoch': 34,
                                              'mean_loss': 0.0038693137,
                                              'step': 128},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 29, 602189, tzinfo=<UTC>),
                                              'epoch': 34,
                                              'mean_loss': 0.0092602,
                                              'step': 129},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 30, 874690, tzinfo=<UTC>),
                                              'epoch': 34,
                                              'mean_loss': 0.0047834637,
                                              'step': 130},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 32, 120016, tzinfo=<UTC>),
                                              'epoch': 34,
                                              'mean_loss': 0.00579008,
                                              'step': 131},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 33, 432962, tzinfo=<UTC>),
                                              'epoch': 35,
                                              'mean_loss': 0.004663332,
                                              'step': 132},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 34, 709379, tzinfo=<UTC>),
                                              'epoch': 35,
                                              'mean_loss': 0.004609281,
                                              'step': 133},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 35, 948660, tzinfo=<UTC>),
                                              'epoch': 35,
                                              'mean_loss': 0.0044703777,
                                              'step': 134},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 37, 173785, tzinfo=<UTC>),
                                              'epoch': 36,
                                              'mean_loss': 0.0016537609,
                                              'step': 135},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 38, 412757, tzinfo=<UTC>),
                                              'epoch': 36,
                                              'mean_loss': 0.005695714,
                                              'step': 136},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 39, 648122, tzinfo=<UTC>),
                                              'epoch': 36,
                                              'mean_loss': 0.004103207,
                                              'step': 137},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 40, 885574, tzinfo=<UTC>),
                                              'epoch': 36,
                                              'mean_loss': 0.0018846963,
                                              'step': 138},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 42, 993050, tzinfo=<UTC>),
                                              'epoch': 37,
                                              'mean_loss': 0.0023978357,
                                              'step': 139},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 44, 304394, tzinfo=<UTC>),
                                              'epoch': 37,
                                              'mean_loss': 0.0033069355,
                                              'step': 140},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 45, 565453, tzinfo=<UTC>),
                                              'epoch': 37,
                                              'mean_loss': 6.576523e-05,
                                              'step': 141},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 47, 38618, tzinfo=<UTC>),
                                              'epoch': 37,
                                              'mean_loss': 0.0043391315,
                                              'step': 142},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 48, 351795, tzinfo=<UTC>),
                                              'epoch': 38,
                                              'mean_loss': 0.0046651517,
                                              'step': 143},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 49, 602102, tzinfo=<UTC>),
                                              'epoch': 38,
                                              'mean_loss': 0.0052327495,
                                              'step': 144},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 50, 976143, tzinfo=<UTC>),
                                              'epoch': 38,
                                              'mean_loss': 0.013687609,
                                              'step': 145},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 52, 227875, tzinfo=<UTC>),
                                              'epoch': 38,
                                              'mean_loss': 9.1750175e-05,
                                              'step': 146},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 53, 495933, tzinfo=<UTC>),
                                              'epoch': 39,
                                              'mean_loss': 0.0025392435,
                                              'step': 147},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 54, 736253, tzinfo=<UTC>),
                                              'epoch': 39,
                                              'mean_loss': 0.0011976548,
                                              'step': 148},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 56, 2776, tzinfo=<UTC>),
                                              'epoch': 39,
                                              'mean_loss': 0.0025211202,
                                              'step': 149},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 57, 245352, tzinfo=<UTC>),
                                              'epoch': 40,
                                              'mean_loss': 0.0018665651,
                                              'step': 150},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 58, 506403, tzinfo=<UTC>),
                                              'epoch': 40,
                                              'mean_loss': 0.0033759787,
                                              'step': 151},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 59, 775888, tzinfo=<UTC>),
                                              'epoch': 40,
                                              'mean_loss': 0.00062831433,
                                              'step': 152},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 1, 18295, tzinfo=<UTC>),
                                              'epoch': 40,
                                              'mean_loss': 0.0020364965,
                                              'step': 153},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 2, 268012, tzinfo=<UTC>),
                                              'epoch': 41,
                                              'mean_loss': 0.0006586923,
                                              'step': 154},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 3, 484102, tzinfo=<UTC>),
                                              'epoch': 41,
                                              'mean_loss': 0.0031121888,
                                              'step': 155},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 4, 776770, tzinfo=<UTC>),
                                              'epoch': 41,
                                              'mean_loss': 0.0027823262,
                                              'step': 156},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 6, 19324, tzinfo=<UTC>),
                                              'epoch': 41,
                                              'mean_loss': 0.0006065498,
                                              'step': 157},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 7, 268435, tzinfo=<UTC>),
                                              'epoch': 42,
                                              'mean_loss': 0.0023070213,
                                              'step': 158},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 8, 512407, tzinfo=<UTC>),
                                              'epoch': 42,
                                              'mean_loss': 0.0016650247,
                                              'step': 159},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 9, 745756, tzinfo=<UTC>),
                                              'epoch': 42,
                                              'mean_loss': 7.3560514e-06,
                                              'step': 160},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 10, 987670, tzinfo=<UTC>),
                                              'epoch': 42,
                                              'mean_loss': 0.012237127,
                                              'step': 161},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 12, 219737, tzinfo=<UTC>),
                                              'epoch': 43,
                                              'mean_loss': 0.0013661574,
                                              'step': 162},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 13, 484756, tzinfo=<UTC>),
                                              'epoch': 43,
                                              'mean_loss': 0.0064119953,
                                              'step': 163},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 14, 730503, tzinfo=<UTC>),
                                              'epoch': 43,
                                              'mean_loss': 0.0035314618,
                                              'step': 164},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 15, 947765, tzinfo=<UTC>),
                                              'epoch': 44,
                                              'mean_loss': 3.298011e-05,
                                              'step': 165},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 17, 190398, tzinfo=<UTC>),
                                              'epoch': 44,
                                              'mean_loss': 0.00053371524,
                                              'step': 166},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 18, 433536, tzinfo=<UTC>),
                                              'epoch': 44,
                                              'mean_loss': 0.00032033096,
                                              'step': 167},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 19, 714514, tzinfo=<UTC>),
                                              'epoch': 44,
                                              'mean_loss': 0.0019390727,
                                              'step': 168},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 20, 935080, tzinfo=<UTC>),
                                              'epoch': 45,
                                              'mean_loss': 2.4262117e-05,
                                              'step': 169},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 22, 167896, tzinfo=<UTC>),
                                              'epoch': 45,
                                              'mean_loss': 0.00012971938,
                                              'step': 170},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 23, 410928, tzinfo=<UTC>),
                                              'epoch': 45,
                                              'mean_loss': 0.0009971312,
                                              'step': 171},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 24, 649509, tzinfo=<UTC>),
                                              'epoch': 45,
                                              'mean_loss': 0.0011956232,
                                              'step': 172},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 25, 860363, tzinfo=<UTC>),
                                              'epoch': 46,
                                              'mean_loss': 8.339065e-05,
                                              'step': 173},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 27, 129342, tzinfo=<UTC>),
                                              'epoch': 46,
                                              'mean_loss': 0.0013557925,
                                              'step': 174},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 28, 356351, tzinfo=<UTC>),
                                              'epoch': 46,
                                              'mean_loss': 0.00077356555,
                                              'step': 175},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 29, 568726, tzinfo=<UTC>),
                                              'epoch': 46,
                                              'mean_loss': 0.0017021206,
                                              'step': 176},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 30, 835703, tzinfo=<UTC>),
                                              'epoch': 47,
                                              'mean_loss': 0.0010255948,
                                              'step': 177},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 32, 82326, tzinfo=<UTC>),
                                              'epoch': 47,
                                              'mean_loss': 0.003208516,
                                              'step': 178},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 33, 301035, tzinfo=<UTC>),
                                              'epoch': 47,
                                              'mean_loss': 0.0021095886,
                                              'step': 179},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 34, 589684, tzinfo=<UTC>),
                                              'epoch': 48,
                                              'mean_loss': 0.0018085723,
                                              'step': 180},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 35, 811121, tzinfo=<UTC>),
                                              'epoch': 48,
                                              'mean_loss': 0.003906385,
                                              'step': 181},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 37, 51086, tzinfo=<UTC>),
                                              'epoch': 48,
                                              'mean_loss': 0.0081788115,
                                              'step': 182},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 38, 296512, tzinfo=<UTC>),
                                              'epoch': 48,
                                              'mean_loss': 0.0021692181,
                                              'step': 183},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 39, 576450, tzinfo=<UTC>),
                                              'epoch': 49,
                                              'mean_loss': 0.0007408549,
                                              'step': 184},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 40, 818968, tzinfo=<UTC>),
                                              'epoch': 49,
                                              'mean_loss': 0.0005012541,
                                              'step': 185},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 42, 115736, tzinfo=<UTC>),
                                              'epoch': 49,
                                              'mean_loss': 0.0015736766,
                                              'step': 186},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 44, 177425, tzinfo=<UTC>),
                                              'epoch': 49,
                                              'mean_loss': 0.0010295139,
                                              'step': 187},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 45, 663911, tzinfo=<UTC>),
                                              'epoch': 50,
                                              'mean_loss': 0.0003010271,
                                              'step': 188},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 46, 907392, tzinfo=<UTC>),
                                              'epoch': 50,
                                              'mean_loss': 0.000984581,
                                              'step': 189},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 48, 197363, tzinfo=<UTC>),
                                              'epoch': 50,
                                              'mean_loss': 0.0001981319,
                                              'step': 190},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 49, 428991, tzinfo=<UTC>),
                                              'epoch': 50,
                                              'mean_loss': 0.00017777813,
                                              'step': 191},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 50, 664645, tzinfo=<UTC>),
                                              'epoch': 51,
                                              'mean_loss': 0.00032808085,
                                              'step': 192},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 51, 868913, tzinfo=<UTC>),
                                              'epoch': 51,
                                              'mean_loss': 0.00059463154,
                                              'step': 193},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 53, 130216, tzinfo=<UTC>),
                                              'epoch': 51,
                                              'mean_loss': 0.0016690929,
                                              'step': 194},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 54, 333529, tzinfo=<UTC>),
                                              'epoch': 52,
                                              'mean_loss': 0.00044837967,
                                              'step': 195},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 55, 572466, tzinfo=<UTC>),
                                              'epoch': 52,
                                              'mean_loss': 0.0005696884,
                                              'step': 196},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 56, 837266, tzinfo=<UTC>),
                                              'epoch': 52,
                                              'mean_loss': 0.0020454866,
                                              'step': 197},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 58, 84490, tzinfo=<UTC>),
                                              'epoch': 52,
                                              'mean_loss': 0.008037148,
                                              'step': 198},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 59, 322229, tzinfo=<UTC>),
                                              'epoch': 53,
                                              'mean_loss': 0.00042736152,
                                              'step': 199},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 0, 564415, tzinfo=<UTC>),
                                              'epoch': 53,
                                              'mean_loss': 0.0023432998,
                                              'step': 200},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 1, 788247, tzinfo=<UTC>),
                                              'epoch': 53,
                                              'mean_loss': 0.0002715867,
                                              'step': 201},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 3, 24660, tzinfo=<UTC>),
                                              'epoch': 53,
                                              'mean_loss': 0.00084764615,
                                              'step': 202},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 4, 292818, tzinfo=<UTC>),
                                              'epoch': 54,
                                              'mean_loss': 0.00030835773,
                                              'step': 203},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 5, 506320, tzinfo=<UTC>),
                                              'epoch': 54,
                                              'mean_loss': 0.00062151754,
                                              'step': 204},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 6, 782668, tzinfo=<UTC>),
                                              'epoch': 54,
                                              'mean_loss': 3.3617685e-05,
                                              'step': 205},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 8, 36202, tzinfo=<UTC>),
                                              'epoch': 54,
                                              'mean_loss': 0.00043850893,
                                              'step': 206},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 9, 280914, tzinfo=<UTC>),
                                              'epoch': 55,
                                              'mean_loss': 0.0014309261,
                                              'step': 207},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 10, 507550, tzinfo=<UTC>),
                                              'epoch': 55,
                                              'mean_loss': 0.00029372238,
                                              'step': 208},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 11, 790404, tzinfo=<UTC>),
                                              'epoch': 55,
                                              'mean_loss': 0.000352273,
                                              'step': 209},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 13, 28160, tzinfo=<UTC>),
                                              'epoch': 56,
                                              'mean_loss': 0.0034054378,
                                              'step': 210},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 14, 285902, tzinfo=<UTC>),
                                              'epoch': 56,
                                              'mean_loss': 0.00038256973,
                                              'step': 211},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 15, 554993, tzinfo=<UTC>),
                                              'epoch': 56,
                                              'mean_loss': 0.0029042722,
                                              'step': 212},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 16, 782794, tzinfo=<UTC>),
                                              'epoch': 56,
                                              'mean_loss': 0.0013111946,
                                              'step': 213},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 18, 18302, tzinfo=<UTC>),
                                              'epoch': 57,
                                              'mean_loss': 0.0005805618,
                                              'step': 214},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 19, 256860, tzinfo=<UTC>),
                                              'epoch': 57,
                                              'mean_loss': 0.001555414,
                                              'step': 215},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 20, 531737, tzinfo=<UTC>),
                                              'epoch': 57,
                                              'mean_loss': 0.0035115764,
                                              'step': 216},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 21, 760323, tzinfo=<UTC>),
                                              'epoch': 57,
                                              'mean_loss': 0.00041363586,
                                              'step': 217},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 23, 38254, tzinfo=<UTC>),
                                              'epoch': 58,
                                              'mean_loss': 8.1257895e-08,
                                              'step': 218},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 24, 459123, tzinfo=<UTC>),
                                              'epoch': 58,
                                              'mean_loss': 0.0016267635,
                                              'step': 219},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 25, 876654, tzinfo=<UTC>),
                                              'epoch': 58,
                                              'mean_loss': 0.0012277836,
                                              'step': 220},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 27, 142763, tzinfo=<UTC>),
                                              'epoch': 58,
                                              'mean_loss': 0.0013576464,
                                              'step': 221},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 28, 423880, tzinfo=<UTC>),
                                              'epoch': 59,
                                              'mean_loss': 0.0014309958,
                                              'step': 222},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 29, 651501, tzinfo=<UTC>),
                                              'epoch': 59,
                                              'mean_loss': 0.00035945722,
                                              'step': 223},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 30, 933277, tzinfo=<UTC>),
                                              'epoch': 59,
                                              'mean_loss': 0.0002977748,
                                              'step': 224},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 32, 192499, tzinfo=<UTC>),
                                              'epoch': 60,
                                              'mean_loss': 0.0016817113,
                                              'step': 225},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 33, 457414, tzinfo=<UTC>),
                                              'epoch': 60,
                                              'mean_loss': 0.0006552929,
                                              'step': 226},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 34, 826380, tzinfo=<UTC>),
                                              'epoch': 60,
                                              'mean_loss': 0.00012618338,
                                              'step': 227},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 36, 79976, tzinfo=<UTC>),
                                              'epoch': 60,
                                              'mean_loss': 0.00076779944,
                                              'step': 228},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 37, 468400, tzinfo=<UTC>),
                                              'epoch': 61,
                                              'mean_loss': 0.0010934594,
                                              'step': 229},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 38, 702382, tzinfo=<UTC>),
                                              'epoch': 61,
                                              'mean_loss': 0.0011660276,
                                              'step': 230},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 39, 959625, tzinfo=<UTC>),
                                              'epoch': 61,
                                              'mean_loss': 8.885516e-06,
                                              'step': 231},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 41, 195827, tzinfo=<UTC>),
                                              'epoch': 61,
                                              'mean_loss': 0.00091533817,
                                              'step': 232},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 42, 428366, tzinfo=<UTC>),
                                              'epoch': 62,
                                              'mean_loss': 0.00090357044,
                                              'step': 233},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 44, 734547, tzinfo=<UTC>),
                                              'epoch': 62,
                                              'mean_loss': 0.00091215235,
                                              'step': 234},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 45, 975299, tzinfo=<UTC>),
                                              'epoch': 62,
                                              'mean_loss': 0.0008807101,
                                              'step': 235},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 47, 381337, tzinfo=<UTC>),
                                              'epoch': 62,
                                              'mean_loss': 0.00038245833,
                                              'step': 236},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 48, 654093, tzinfo=<UTC>),
                                              'epoch': 63,
                                              'mean_loss': 0.00080911745,
                                              'step': 237},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 49, 882751, tzinfo=<UTC>),
                                              'epoch': 63,
                                              'mean_loss': 0.0003725673,
                                              'step': 238},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 51, 110358, tzinfo=<UTC>),
                                              'epoch': 63,
                                              'mean_loss': 0.0012469762,
                                              'step': 239},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 52, 679155, tzinfo=<UTC>),
                                              'epoch': 64,
                                              'mean_loss': 0.0005009441,
                                              'step': 240},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 53, 911603, tzinfo=<UTC>),
                                              'epoch': 64,
                                              'mean_loss': 0.0005480327,
                                              'step': 241},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 55, 184955, tzinfo=<UTC>),
                                              'epoch': 64,
                                              'mean_loss': -1.8597348e-06,
                                              'step': 242},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 56, 578096, tzinfo=<UTC>),
                                              'epoch': 64,
                                              'mean_loss': 0.002287712,
                                              'step': 243},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 57, 828452, tzinfo=<UTC>),
                                              'epoch': 65,
                                              'mean_loss': 6.471912e-05,
                                              'step': 244},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 59, 35379, tzinfo=<UTC>),
                                              'epoch': 65,
                                              'mean_loss': 0.00031904934,
                                              'step': 245},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 0, 293344, tzinfo=<UTC>),
                                              'epoch': 65,
                                              'mean_loss': 1.0820688e-05,
                                              'step': 246},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 1, 560935, tzinfo=<UTC>),
                                              'epoch': 65,
                                              'mean_loss': 0.0012223909,
                                              'step': 247},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 2, 793184, tzinfo=<UTC>),
                                              'epoch': 66,
                                              'mean_loss': 0.0008803075,
                                              'step': 248},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 4, 43681, tzinfo=<UTC>),
                                              'epoch': 66,
                                              'mean_loss': 0.0010418366,
                                              'step': 249},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 5, 296877, tzinfo=<UTC>),
                                              'epoch': 66,
                                              'mean_loss': 0.0011515429,
                                              'step': 250},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 6, 531123, tzinfo=<UTC>),
                                              'epoch': 66,
                                              'mean_loss': 9.024725e-05,
                                              'step': 251},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 7, 793511, tzinfo=<UTC>),
                                              'epoch': 67,
                                              'mean_loss': 1.4540274e-06,
                                              'step': 252},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 9, 25627, tzinfo=<UTC>),
                                              'epoch': 67,
                                              'mean_loss': 0.00012591947,
                                              'step': 253},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 10, 275288, tzinfo=<UTC>),
                                              'epoch': 67,
                                              'mean_loss': 0.0008015272,
                                              'step': 254},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 11, 522475, tzinfo=<UTC>),
                                              'epoch': 68,
                                              'mean_loss': 0.0008423489,
                                              'step': 255},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 12, 764784, tzinfo=<UTC>),
                                              'epoch': 68,
                                              'mean_loss': 0.0011362592,
                                              'step': 256},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 14, 58405, tzinfo=<UTC>),
                                              'epoch': 68,
                                              'mean_loss': 0.001567196,
                                              'step': 257},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 15, 317351, tzinfo=<UTC>),
                                              'epoch': 68,
                                              'mean_loss': 0.0004869902,
                                              'step': 258},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 16, 557683, tzinfo=<UTC>),
                                              'epoch': 69,
                                              'mean_loss': 0.00044071442,
                                              'step': 259},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 17, 885247, tzinfo=<UTC>),
                                              'epoch': 69,
                                              'mean_loss': 0.0005702487,
                                              'step': 260},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 19, 119570, tzinfo=<UTC>),
                                              'epoch': 69,
                                              'mean_loss': 0.0006433289,
                                              'step': 261},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 20, 364831, tzinfo=<UTC>),
                                              'epoch': 69,
                                              'mean_loss': 0.00045972248,
                                              'step': 262},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 21, 605030, tzinfo=<UTC>),
                                              'epoch': 70,
                                              'mean_loss': 0.0005696737,
                                              'step': 263},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 22, 866124, tzinfo=<UTC>),
                                              'epoch': 70,
                                              'mean_loss': 0.0011296477,
                                              'step': 264},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 24, 175216, tzinfo=<UTC>),
                                              'epoch': 70,
                                              'mean_loss': 0.00031165092,
                                              'step': 265},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 25, 651260, tzinfo=<UTC>),
                                              'epoch': 70,
                                              'mean_loss': 0.0004441709,
                                              'step': 266},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 27, 399823, tzinfo=<UTC>),
                                              'epoch': 71,
                                              'mean_loss': 0.0011137151,
                                              'step': 267},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 28, 659278, tzinfo=<UTC>),
                                              'epoch': 71,
                                              'mean_loss': 0.0009634383,
                                              'step': 268},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 29, 919579, tzinfo=<UTC>),
                                              'epoch': 71,
                                              'mean_loss': 9.269314e-05,
                                              'step': 269},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 31, 271483, tzinfo=<UTC>),
                                              'epoch': 72,
                                              'mean_loss': 0.0005366412,
                                              'step': 270},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 32, 557850, tzinfo=<UTC>),
                                              'epoch': 72,
                                              'mean_loss': 7.907781e-05,
                                              'step': 271},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 33, 808536, tzinfo=<UTC>),
                                              'epoch': 72,
                                              'mean_loss': 0.00013625692,
                                              'step': 272},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 35, 93333, tzinfo=<UTC>),
                                              'epoch': 72,
                                              'mean_loss': 0.00040144066,
                                              'step': 273},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 36, 349643, tzinfo=<UTC>),
                                              'epoch': 73,
                                              'mean_loss': 0.000531957,
                                              'step': 274},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 37, 601945, tzinfo=<UTC>),
                                              'epoch': 73,
                                              'mean_loss': 0.00064232247,
                                              'step': 275},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 38, 855989, tzinfo=<UTC>),
                                              'epoch': 73,
                                              'mean_loss': 5.250331e-08,
                                              'step': 276},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 40, 101868, tzinfo=<UTC>),
                                              'epoch': 73,
                                              'mean_loss': 0.00054669485,
                                              'step': 277},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 41, 311587, tzinfo=<UTC>),
                                              'epoch': 74,
                                              'mean_loss': 0.0001149911,
                                              'step': 278},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 42, 504293, tzinfo=<UTC>),
                                              'epoch': 74,
                                              'mean_loss': 0.0006582851,
                                              'step': 279},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 44, 606424, tzinfo=<UTC>),
                                              'epoch': 74,
                                              'mean_loss': 0.00040794525,
                                              'step': 280},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 45, 885918, tzinfo=<UTC>),
                                              'epoch': 74,
                                              'mean_loss': 0.00040962768,
                                              'step': 281},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 47, 88903, tzinfo=<UTC>),
                                              'epoch': 75,
                                              'mean_loss': 0.00083329267,
                                              'step': 282},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 48, 353963, tzinfo=<UTC>),
                                              'epoch': 75,
                                              'mean_loss': 0.0011541949,
                                              'step': 283},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 49, 634966, tzinfo=<UTC>),
                                              'epoch': 75,
                                              'mean_loss': 0.00031403676,
                                              'step': 284},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 50, 872769, tzinfo=<UTC>),
                                              'epoch': 76,
                                              'mean_loss': 0.0001571991,
                                              'step': 285},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 52, 132997, tzinfo=<UTC>),
                                              'epoch': 76,
                                              'mean_loss': 3.2733406e-05,
                                              'step': 286},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 53, 373001, tzinfo=<UTC>),
                                              'epoch': 76,
                                              'mean_loss': 9.974141e-06,
                                              'step': 287},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 54, 731122, tzinfo=<UTC>),
                                              'epoch': 76,
                                              'mean_loss': 0.00092323206,
                                              'step': 288},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 56, 5106, tzinfo=<UTC>),
                                              'epoch': 77,
                                              'mean_loss': 0.00027380337,
                                              'step': 289},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 57, 958533, tzinfo=<UTC>),
                                              'epoch': 77,
                                              'mean_loss': 0.000730188,
                                              'step': 290},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 59, 218392, tzinfo=<UTC>),
                                              'epoch': 77,
                                              'mean_loss': 8.784898e-05,
                                              'step': 291},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 0, 458718, tzinfo=<UTC>),
                                              'epoch': 77,
                                              'mean_loss': 0.00035266066,
                                              'step': 292},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 1, 678532, tzinfo=<UTC>),
                                              'epoch': 78,
                                              'mean_loss': 0.00035272574,
                                              'step': 293},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 2, 925032, tzinfo=<UTC>),
                                              'epoch': 78,
                                              'mean_loss': 0.00036251757,
                                              'step': 294},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 4, 185515, tzinfo=<UTC>),
                                              'epoch': 78,
                                              'mean_loss': 0.00063127023,
                                              'step': 295},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 5, 450029, tzinfo=<UTC>),
                                              'epoch': 78,
                                              'mean_loss': 0.00056570326,
                                              'step': 296},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 6, 669796, tzinfo=<UTC>),
                                              'epoch': 79,
                                              'mean_loss': 0.00019644247,
                                              'step': 297},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 7, 912769, tzinfo=<UTC>),
                                              'epoch': 79,
                                              'mean_loss': 1.2713252e-05,
                                              'step': 298},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 9, 283472, tzinfo=<UTC>),
                                              'epoch': 79,
                                              'mean_loss': 0.00024443713,
                                              'step': 299},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 10, 628408, tzinfo=<UTC>),
                                              'epoch': 80,
                                              'mean_loss': 0.001333565,
                                              'step': 300},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 11, 890583, tzinfo=<UTC>),
                                              'epoch': 80,
                                              'mean_loss': 0.0007013555,
                                              'step': 301},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 13, 592038, tzinfo=<UTC>),
                                              'epoch': 80,
                                              'mean_loss': 2.6616384e-05,
                                              'step': 302},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 14, 822838, tzinfo=<UTC>),
                                              'epoch': 80,
                                              'mean_loss': 0.0005623731,
                                              'step': 303},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 16, 205657, tzinfo=<UTC>),
                                              'epoch': 81,
                                              'mean_loss': 0.00032505486,
                                              'step': 304},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 18, 71680, tzinfo=<UTC>),
                                              'epoch': 81,
                                              'mean_loss': 0.0005101152,
                                              'step': 305},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 19, 294720, tzinfo=<UTC>),
                                              'epoch': 81,
                                              'mean_loss': 0.0010035196,
                                              'step': 306},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 20, 524938, tzinfo=<UTC>),
                                              'epoch': 81,
                                              'mean_loss': 0.00033056142,
                                              'step': 307},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 21, 762389, tzinfo=<UTC>),
                                              'epoch': 82,
                                              'mean_loss': 6.966456e-05,
                                              'step': 308},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 23, 25594, tzinfo=<UTC>),
                                              'epoch': 82,
                                              'mean_loss': 0.00037358585,
                                              'step': 309},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 24, 257330, tzinfo=<UTC>),
                                              'epoch': 82,
                                              'mean_loss': 0.0002650607,
                                              'step': 310},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 25, 497375, tzinfo=<UTC>),
                                              'epoch': 82,
                                              'mean_loss': 0.0009153653,
                                              'step': 311},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 26, 713095, tzinfo=<UTC>),
                                              'epoch': 83,
                                              'mean_loss': 0.0009658756,
                                              'step': 312},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 27, 973220, tzinfo=<UTC>),
                                              'epoch': 83,
                                              'mean_loss': 0.00014872942,
                                              'step': 313},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 29, 226304, tzinfo=<UTC>),
                                              'epoch': 83,
                                              'mean_loss': 9.968644e-06,
                                              'step': 314},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 30, 493632, tzinfo=<UTC>),
                                              'epoch': 84,
                                              'mean_loss': 0.00027738986,
                                              'step': 315},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 31, 735723, tzinfo=<UTC>),
                                              'epoch': 84,
                                              'mean_loss': 0.0004675896,
                                              'step': 316},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 32, 954139, tzinfo=<UTC>),
                                              'epoch': 84,
                                              'mean_loss': 0.00014443416,
                                              'step': 317},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 34, 196120, tzinfo=<UTC>),
                                              'epoch': 84,
                                              'mean_loss': 0.0006946635,
                                              'step': 318},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 35, 443514, tzinfo=<UTC>),
                                              'epoch': 85,
                                              'mean_loss': 0.0007360133,
                                              'step': 319},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 36, 687667, tzinfo=<UTC>),
                                              'epoch': 85,
                                              'mean_loss': 1.326669e-06,
                                              'step': 320},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 37, 903099, tzinfo=<UTC>),
                                              'epoch': 85,
                                              'mean_loss': 0.0005314335,
                                              'step': 321},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 39, 138629, tzinfo=<UTC>),
                                              'epoch': 85,
                                              'mean_loss': 6.947189e-05,
                                              'step': 322},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 40, 371666, tzinfo=<UTC>),
                                              'epoch': 86,
                                              'mean_loss': 0.00053617253,
                                              'step': 323},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 41, 602304, tzinfo=<UTC>),
                                              'epoch': 86,
                                              'mean_loss': 9.1956696e-05,
                                              'step': 324},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 42, 836714, tzinfo=<UTC>),
                                              'epoch': 86,
                                              'mean_loss': 0.00018627953,
                                              'step': 325},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 44, 874288, tzinfo=<UTC>),
                                              'epoch': 86,
                                              'mean_loss': 0.0002088271,
                                              'step': 326},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 46, 162027, tzinfo=<UTC>),
                                              'epoch': 87,
                                              'mean_loss': 0.00075449655,
                                              'step': 327},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 47, 431864, tzinfo=<UTC>),
                                              'epoch': 87,
                                              'mean_loss': 7.8588026e-05,
                                              'step': 328},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 48, 680277, tzinfo=<UTC>),
                                              'epoch': 87,
                                              'mean_loss': -1.3336539e-06,
                                              'step': 329},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 49, 968707, tzinfo=<UTC>),
                                              'epoch': 88,
                                              'mean_loss': 0.00012271712,
                                              'step': 330},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 51, 259023, tzinfo=<UTC>),
                                              'epoch': 88,
                                              'mean_loss': 0.0017514592,
                                              'step': 331},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 52, 511807, tzinfo=<UTC>),
                                              'epoch': 88,
                                              'mean_loss': 4.1678373e-05,
                                              'step': 332},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 53, 808914, tzinfo=<UTC>),
                                              'epoch': 88,
                                              'mean_loss': 0.0006764167,
                                              'step': 333},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 55, 179417, tzinfo=<UTC>),
                                              'epoch': 89,
                                              'mean_loss': 0.00013730745,
                                              'step': 334},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 56, 395678, tzinfo=<UTC>),
                                              'epoch': 89,
                                              'mean_loss': 0.00032095844,
                                              'step': 335},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 57, 654428, tzinfo=<UTC>),
                                              'epoch': 89,
                                              'mean_loss': 0.00015303271,
                                              'step': 336},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 58, 891717, tzinfo=<UTC>),
                                              'epoch': 89,
                                              'mean_loss': 0.00012956047,
                                              'step': 337},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 0, 151925, tzinfo=<UTC>),
                                              'epoch': 90,
                                              'mean_loss': 7.675003e-05,
                                              'step': 338},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 1, 398761, tzinfo=<UTC>),
                                              'epoch': 90,
                                              'mean_loss': 0.00044489285,
                                              'step': 339},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 2, 641274, tzinfo=<UTC>),
                                              'epoch': 90,
                                              'mean_loss': 1.4036312e-05,
                                              'step': 340},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 3, 883546, tzinfo=<UTC>),
                                              'epoch': 90,
                                              'mean_loss': 0.00015219976,
                                              'step': 341},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 5, 141535, tzinfo=<UTC>),
                                              'epoch': 91,
                                              'mean_loss': 0.00018677826,
                                              'step': 342},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 6, 373784, tzinfo=<UTC>),
                                              'epoch': 91,
                                              'mean_loss': 0.000115128816,
                                              'step': 343},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 7, 645535, tzinfo=<UTC>),
                                              'epoch': 91,
                                              'mean_loss': 0.00039966288,
                                              'step': 344},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 8, 884670, tzinfo=<UTC>),
                                              'epoch': 92,
                                              'mean_loss': 0.0001351597,
                                              'step': 345},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 10, 149429, tzinfo=<UTC>),
                                              'epoch': 92,
                                              'mean_loss': 6.1459374e-05,
                                              'step': 346},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 11, 448963, tzinfo=<UTC>),
                                              'epoch': 92,
                                              'mean_loss': 0.00023051281,
                                              'step': 347},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 12, 693703, tzinfo=<UTC>),
                                              'epoch': 92,
                                              'mean_loss': 0.00078510307,
                                              'step': 348},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 13, 938911, tzinfo=<UTC>),
                                              'epoch': 93,
                                              'mean_loss': 8.103554e-06,
                                              'step': 349},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 15, 460723, tzinfo=<UTC>),
                                              'epoch': 93,
                                              'mean_loss': 0.0019005266,
                                              'step': 350},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 16, 766511, tzinfo=<UTC>),
                                              'epoch': 93,
                                              'mean_loss': 6.863149e-06,
                                              'step': 351},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 18, 27508, tzinfo=<UTC>),
                                              'epoch': 93,
                                              'mean_loss': 0.0002926389,
                                              'step': 352},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 19, 286497, tzinfo=<UTC>),
                                              'epoch': 94,
                                              'mean_loss': 0.00013998023,
                                              'step': 353},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 20, 556785, tzinfo=<UTC>),
                                              'epoch': 94,
                                              'mean_loss': 2.2997847e-05,
                                              'step': 354},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 21, 804735, tzinfo=<UTC>),
                                              'epoch': 94,
                                              'mean_loss': 0.0005936278,
                                              'step': 355},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 23, 22367, tzinfo=<UTC>),
                                              'epoch': 94,
                                              'mean_loss': 3.43258e-05,
                                              'step': 356},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 24, 284890, tzinfo=<UTC>),
                                              'epoch': 95,
                                              'mean_loss': 0.00010312116,
                                              'step': 357},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 25, 525734, tzinfo=<UTC>),
                                              'epoch': 95,
                                              'mean_loss': 0.00015714776,
                                              'step': 358},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 26, 780375, tzinfo=<UTC>),
                                              'epoch': 95,
                                              'mean_loss': 5.73016e-05,
                                              'step': 359},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 28, 23466, tzinfo=<UTC>),
                                              'epoch': 96,
                                              'mean_loss': 0.00012817327,
                                              'step': 360},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 29, 268204, tzinfo=<UTC>),
                                              'epoch': 96,
                                              'mean_loss': 3.9030332e-05,
                                              'step': 361},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 30, 507390, tzinfo=<UTC>),
                                              'epoch': 96,
                                              'mean_loss': 0.0005360425,
                                              'step': 362},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 31, 727121, tzinfo=<UTC>),
                                              'epoch': 96,
                                              'mean_loss': 0.00017444952,
                                              'step': 363},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 33, 34563, tzinfo=<UTC>),
                                              'epoch': 97,
                                              'mean_loss': 0.0010171408,
                                              'step': 364},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 34, 447584, tzinfo=<UTC>),
                                              'epoch': 97,
                                              'mean_loss': 0.0004899306,
                                              'step': 365},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 35, 699821, tzinfo=<UTC>),
                                              'epoch': 97,
                                              'mean_loss': 0.00017226115,
                                              'step': 366},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 36, 936549, tzinfo=<UTC>),
                                              'epoch': 97,
                                              'mean_loss': 4.2724423e-07,
                                              'step': 367},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 38, 203783, tzinfo=<UTC>),
                                              'epoch': 98,
                                              'mean_loss': 1.9560219e-05,
                                              'step': 368},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 39, 464518, tzinfo=<UTC>),
                                              'epoch': 98,
                                              'mean_loss': 0.00011098804,
                                              'step': 369},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 40, 721560, tzinfo=<UTC>),
                                              'epoch': 98,
                                              'mean_loss': 0.0005288075,
                                              'step': 370},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 41, 968386, tzinfo=<UTC>),
                                              'epoch': 98,
                                              'mean_loss': 4.2606727e-05,
                                              'step': 371},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 43, 207703, tzinfo=<UTC>),
                                              'epoch': 99,
                                              'mean_loss': 1.1964934e-05,
                                              'step': 372},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 45, 297458, tzinfo=<UTC>),
                                              'epoch': 99,
                                              'mean_loss': 0.00035788305,
                                              'step': 373},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 46, 714760, tzinfo=<UTC>),
                                              'epoch': 99,
                                              'mean_loss': 7.525133e-05,
                                              'step': 374},
                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 48, 17671, tzinfo=<UTC>),
                                              'epoch': 100,
                                              'mean_loss': 5.5355486e-06,
                                              'step': 375}],
                                  hyperparameters=Hyperparameters(epoch_count=100,
                                                                  batch_size=4,
                                                                  learning_rate=0.001)))
model.description
'This is my model.'

Delete the model

You can clean up your tuned model list by deleting models you no longer need. Use the palm.delete_tuned_model method to delete a model. If you canceled any tuning jobs, you may want to delete those as their performance may be unpredictable.

palm.delete_tuned_model(f'tunedModels/{name}')

try:
  m = palm.get_tuned_model(f'tunedModels/{name}')
  print(m)
except Exception as e:
  print(f"{type(e)}: {e}")
<class 'google.api_core.exceptions.NotFound'>: 404 Tuned model tunedModels/generate-num-4668 does not exist.