 View on Generative AI
|
 Run in Google Colab
|
 View source on GitHub
|
Overview
This example demonstrates how to use the PaLM API to create embeddings so that you can perform document search. You will use the Python client library to build a word embedding that allows you to compare search strings, or questions, to document contents.
In this tutorial, you'll use embeddings to perform document search over a set of documents to ask questions related to the Google Car.
Setup
First, download and install the PaLM API Python library.
pip install -U -q google-generativeai
import google.generativeai as palm
import textwrap
import numpy as np
import pandas as pd
Grab an API Key
To get started, you'll need to create an API key.
palm.configure(api_key='PALM_KEY')
models = [m for m in palm.list_models() if 'embedText' in m.supported_generation_methods]
model = models[0]
Embedding generation
In this section, you will see how to generate embeddings for a piece of text using the embeddings from the PaLM API.
sample_text = ("Title: The next generation of AI for developers and Google Workspace"
"\n"
"Full article:\n"
"\n"
"PaLM API & MakerSuite: An approachable way to explore and prototype with generative AI applications")
# Create an embedding
embedding = palm.generate_embeddings(model=model, text=sample_text)
print(embedding)
{'embedding': [0.012246046, -0.023558903, 0.032459036, 0.06484912, 0.026284628, -0.052756585, 0.0055233696, 0.011005492, -0.03862501, -0.018264746, 0.06678695, -0.015016806, 0.0035746037, -0.009914331, -0.022514464, 0.030050583, -0.078709245, -0.0015311453, -0.02805761, 0.0036338808, -0.076459445, 0.009172192, 0.01225061, -0.016513903, 0.008183921, -0.08033063, 0.028195586, 0.029587045, -0.031249639, -0.019803159, 0.0025109726, 0.018474173, -0.006070546, 0.0042981566, 0.010934953, 0.03646359, -0.027267052, 0.006511828, 0.017602839, 0.015774623, 0.042609964, -0.04978823, 0.021985881, -0.0018003813, 0.015031357, 0.03226512, -0.049656868, 0.0056817518, 0.037447836, -0.082058676, 0.0070665455, -0.009903009, -0.0012817691, -0.009555456, 0.013598595, 0.07107551, -0.10985609, 0.00044024497, -0.027354741, -0.021102894, -0.0077857957, 0.030045867, 0.0065566953, -0.02866328, -0.024084672, 0.027182486, 0.010249044, 0.028895397, -0.048748404, 0.0144549105, 0.035357818, 0.026979432, -0.011489553, -0.04381989, 0.062055543, 0.053935688, 0.018935075, 7.997995e-05, 0.032406107, -0.042411212, -0.018950237, -0.03686655, -0.02786128, -0.14247943, -0.031954747, 0.06135447, 0.007755804, 0.007340383, -0.049610108, 0.06055633, -0.0061997687, 0.015536909, 0.017663272, 0.046989314, -0.02829787, 0.007028086, 0.009768779, -0.017920492, -0.0004482094, 0.01760272, -0.03491943, 0.005043931, 0.072463214, -0.05273243, 0.08608823, -0.02313639, 0.02748735, 0.035566464, -0.046005856, -0.07012403, -0.011197247, 0.005118026, -0.05882537, 0.012176309, -0.045815013, -0.011174939, 0.04357285, -0.018380292, 0.028166372, 0.033733334, -0.010419084, 0.040377222, -0.006756512, 0.029616103, 0.020419275, 0.017293947, 0.038005445, 0.020060254, 0.014358492, 0.006015242, 0.030263908, 0.014460127, 0.05033836, 0.00423167, 0.02703248, 0.010239786, -0.008927503, 0.013181109, 0.023492351, 0.034311254, -0.03821471, 0.004627892, -0.0040852013, -0.0020964656, -0.05259364, -0.0705314, -0.01479818, -0.0124032665, -0.09655567, 0.00198135, 0.025488432, -0.019924233, -0.03710428, 0.007010777, 0.011313427, 0.066283226, 0.04507062, 0.0016292608, 0.04087332, -0.05021339, -0.0024507563, -0.046069298, -0.054124065, 0.014468171, 0.013357496, 0.006985751, 0.005976544, -0.030506134, -0.02365677, 0.015135481, 0.044584073, -0.10413109, -0.0107979365, -0.008295349, 0.051585224, -0.03829224, 0.011085167, 0.0050877626, 0.08231769, -0.03219612, -0.03536319, -0.09238423, 0.023758749, 0.013713774, -0.06647304, 0.04775781, -0.033355825, -0.030094955, -0.008177198, 0.051271528, -0.020077743, 0.01580692, 0.07417194, -0.05540835, -0.05794891, -0.013655137, 0.06279186, -0.11484751, -0.021054761, 0.04684413, 0.008794997, -0.021544361, 0.031731065, 0.042848878, 0.009124984, 0.006002671, -0.060807146, -0.0039937478, 0.044372104, 0.02276778, -0.023919228, -0.015265087, -0.0015277741, -0.027227052, -0.008951991, -0.014200425, 0.012097831, -0.041769046, -0.09691623, -0.024216626, -0.023596313, 0.00037882378, -0.0041446723, 0.007590011, 0.021700665, 0.028424272, 0.02182814, 0.020193378, 0.028626561, -0.0016748251, -0.03279016, 0.0044993553, -0.030803349, 0.00095924473, -0.020228835, 0.016107002, 0.014520303, 0.0023317838, -0.05539688, 0.000895851, 0.01886939, 0.023983113, -0.01504259, 0.026205119, 0.009914357, 0.052187297, 0.030023996, 0.04388972, 0.022008175, 0.022883205, 0.042004555, -0.07240339, 0.018784735, 0.003868624, 0.058717627, 0.030771254, -0.014063785, 0.0363103, -0.01794209, -0.0074301963, -0.04202981, 0.049500227, -0.0021138182, 0.12777524, -0.034146316, -0.015136565, -0.027827494, -0.016238643, 0.06866837, -0.009253228, -0.0016339661, -0.027306853, 0.0029207866, 0.039376575, -0.029730743, -0.00484304, -0.033259068, 0.04545208, 0.009262109, 0.04384297, 0.017075498, -0.028545981, 0.02307001, 0.047156688, 0.013491146, 0.031000527, 0.015057356, -0.022589264, -0.0846759, 0.005845248, 0.011864375, -0.0070025027, 0.013764861, 0.02718182, 0.012324712, -0.024820248, 0.04852867, 0.0024367159, 0.005850492, 0.016530823, -0.011834324, -0.011131373, 0.001191735, 0.044042632, -0.045152735, -0.008464704, -0.02432455, 0.022879586, 0.057115532, 0.0051847813, 0.007935389, -0.037327744, 0.017658837, 0.04210197, -0.013657841, 0.007334552, -0.016862206, -0.031454947, -0.03708944, 0.040686775, -0.04624996, 0.01877762, 0.08135753, -0.007218369, 0.0004070545, -0.055425953, 0.055876233, 0.016689738, 0.07638828, 0.008354422, -0.01910822, 0.018938914, -0.054518394, -0.003591044, 0.017713007, -0.0022877606, -0.0094331335, 0.033181757, -0.021113275, -0.04498197, 0.05601438, -0.04239881, -0.024155997, -0.020966347, -0.03797506, 0.022574421, -0.076318376, -0.004959584, -0.021404805, -0.04240269, 0.0107064145, -0.015867218, -0.026940335, 0.07569258, -0.027117623, -0.021980954, 0.030429304, 0.06054544, 0.049562912, 0.00095839944, -0.04588092, 0.0029076242, -0.088651165, 0.03488081, 0.020981148, 0.011397698, -0.04327915, 0.028569348, 0.043313224, -0.009539706, -0.017482065, 0.0020923335, 0.03751124, -0.053476032, -0.079561576, -0.017803138, -0.0421531, -0.0378791, -0.0039890567, -0.0008835484, -0.053429686, 0.011735356, -0.02350256, -0.00059799175, -0.01724343, 0.07864523, 0.034462206, 0.0507343, 0.022246856, -0.035423458, -0.011222293, 0.041953508, 0.007671431, 0.031695075, -0.030306417, 0.02958094, -0.040412143, -0.04330924, -0.04039218, -0.033534866, 0.035888318, 0.051540695, -0.021477232, -0.083954886, 0.042218216, 0.020746768, 0.02682532, 0.04498249, 0.04313308, -0.040424753, 0.0018861584, -0.017179515, -0.011048184, 0.04160573, 0.018856611, -0.047430437, 0.024466624, -0.0228378, 0.031078562, 0.0007776898, 0.051385712, 0.01981563, -0.056765486, 0.03364401, 0.014105605, 0.030105526, -0.035521813, 0.034239095, -0.05366703, -0.003175909, 0.04061052, -0.007840006, -0.011056109, -0.0031887041, -0.0773631, -0.06250093, 0.13228256, 0.0448807, 0.0452502, 0.039861113, -0.009379959, 0.0061017787, -0.054143652, 0.030229399, -0.059683457, -0.03556136, 0.007690892, -0.042795043, -0.06671517, 0.004328955, -0.044326086, -0.031826798, 0.04102504, 0.11098777, 0.059205733, -0.031162312, 0.009748784, -0.0031859796, 0.00034297028, 0.015214179, -0.00037445556, -0.024338417, -0.02923963, 0.010895459, -0.030704288, -0.02304379, 0.05466228, 0.04812725, 0.013858184, 0.0071799406, -0.013480506, 0.048238866, -0.047373805, 0.0015964687, 0.06232653, 0.043619704, 0.014640049, 0.017748961, -0.049789716, 0.01905874, -0.03484224, -0.029324956, -0.02938803, -0.0127894115, 0.008088268, 0.05033771, -0.009779625, 0.020661239, -0.004912575, -0.03859561, 0.049923155, -0.043472834, 0.017737135, 0.0048368694, 0.034132574, -0.019803194, -0.0062308377, 0.022995766, 0.024034595, 0.034497425, 0.028333474, -0.013199994, 0.009532892, 0.014956127, -0.024343139, -0.023101693, -0.019819845, -0.038274676, -0.067926295, 0.020405637, 0.026949758, 0.047760095, -0.013742078, -0.027537456, -0.027341628, -0.052108474, -0.022617042, 0.021988103, -0.004878778, -0.055188403, 0.03838512, -0.02852371, -0.029049108, -0.030390456, 0.061514128, 0.061222956, 0.051862024, 0.003029712, -0.053684346, 4.938375e-05, 0.00057140755, 0.053631667, 0.03287124, 0.0070602377, -0.0019494261, -0.043916594, 0.022534015, -0.0061359294, 0.030514536, 0.115870886, -0.009872318, -0.07409435, 0.037494868, 0.0085815005, -0.01520489, 0.027977582, 0.023814408, 0.029372396, 0.013129667, 0.0011279223, -0.008254216, -0.006493126, 0.016296634, 0.0380462, 0.0129206255, -0.04741698, 0.037081294, -0.01708468, -0.011231078, -0.011991382, -0.02769527, -0.022195553, -0.006528756, -0.03245275, 0.05531176, -0.0325935, 0.030573608, 0.06419135, 0.01796485, 0.05386303, 0.022312209, -0.027932238, -0.021684878, 0.013180571, 0.026342593, 0.031921875, -0.027714772, 0.04125191, -0.0067000175, 0.036882173, -0.032648146, -0.014226238, -0.014399368, -0.022615127, -0.034392234, -0.03426428, -0.012184155, -0.057999205, 0.0009545769, -0.0083389, 0.023374686, -0.10456068, -0.013828168, -0.010597269, -0.025884187, -0.026183352, 0.028116345, -0.0062918467, -0.031959485, -0.00195724, 0.00551872, 0.047163066, 0.050132312, -0.011089595, 0.03454736, -0.0065446403, 0.027497908, -0.011359338, 0.031209284, 0.0123054935, 0.0067299386, 0.03320252, 0.0104132155, 0.012565796, -0.0054723895, -0.0012788378, -0.01601304, 0.06827864, 0.022071837, 0.019106403, 0.04867051, 0.024571512, -0.005845881, -0.050935183, 0.03698963, -0.017698955, -0.006054161, 0.012251457, -0.0031763925, -0.009850868, 0.022571698, -0.016523926, 0.015339761, -0.04153422, 0.031590454, -0.047403164, -0.019668864, 0.013377942, 0.037525933, 0.016130688, -0.0014420815, 0.03395241, 0.006446724, 0.0067957826, -0.030642867, 0.016237482, -0.059248183, -0.017643742, -0.011814861, 0.036445998, -0.012023078, -0.03969171, -0.034770712, -0.024164954, -0.004940893, 0.01273272, -0.029959105, 0.0075648203, -0.0346612, 0.040213585, -0.011875309, 0.036303695, 0.03612044, 0.051215306, -0.06879151, 0.05865379, -0.06129543, 0.028531928, 0.027353931, -0.028882181, -0.052622266, -0.0054572835, 0.038268622, -0.01889903, 0.001147878, 0.011961551, 0.055814732, 0.054686487, 0.057777297, 0.0061383895, -0.031106692, -0.0034993412, 0.014843713, -0.020202357, -0.027266696, -0.025075577, -0.024285411, 0.00020614524, 0.013779444, -0.022222523, 0.0013498501, -0.021858962, -0.084415734, 0.022417184, -0.00689182, -0.03741896, -0.08071215, -0.017459916, 0.005807038, 0.029116781, -0.0018873442, 0.028786417, 0.049730763, 0.045285672, 0.018252771, -0.010492358, -0.021893298, 0.008960559, 0.0019279895, -0.065256804, 0.018064518, -0.039222594, 0.009063778, 0.037082877, 0.016562615, 0.052926384, -0.04298042, 0.055858735, 0.05707242, 0.03907505, 0.0015263337, 0.009082476, 0.0134143485, -0.029168077, -0.00030230818, -0.010265555, 0.019662535, -0.042000905, -0.0027351528, 0.01557767, 0.021629393, -0.037543625, 0.029485308, 0.057547256, -0.012264158, 0.010961239, 0.07455477, -0.04760432, 0.020114874, -0.043387685, 0.026112124, 0.028907014, -0.0088930875, 0.025489105, 0.029058266, -0.004499017, 0.0378109, -0.01393321, -0.044656288, -0.03684158, -0.032738246, 0.03379276, 0.026568653, 0.020096838, 0.0012306226, 0.08085042, 0.034304578, 0.040584367, -0.031480588, 0.030303054, -0.029881144, -0.04158148, -0.050945546, 0.04790348, -0.003912531, -0.027478285, -0.01310397, 0.01636849]}
Building an embeddings database
Here are three sample texts to use to build the embeddings database. You will use the PaLM API to create embeddings of each of the documents. Turn them into a dataframe for better visualization.
DOCUMENT1 = "Operating the Climate Control System Your Googlecar has a climate control system that allows you to adjust the temperature and airflow in the car. To operate the climate control system, use the buttons and knobs located on the center console. Temperature: The temperature knob controls the temperature inside the car. Turn the knob clockwise to increase the temperature or counterclockwise to decrease the temperature. Airflow: The airflow knob controls the amount of airflow inside the car. Turn the knob clockwise to increase the airflow or counterclockwise to decrease the airflow. Fan speed: The fan speed knob controls the speed of the fan. Turn the knob clockwise to increase the fan speed or counterclockwise to decrease the fan speed. Mode: The mode button allows you to select the desired mode. The available modes are: Auto: The car will automatically adjust the temperature and airflow to maintain a comfortable level. Cool: The car will blow cool air into the car. Heat: The car will blow warm air into the car. Defrost: The car will blow warm air onto the windshield to defrost it."
DOCUMENT2 = "Your Googlecar has a large touchscreen display that provides access to a variety of features, including navigation, entertainment, and climate control. To use the touchscreen display, simply touch the desired icon. For example, you can touch the \"Navigation\" icon to get directions to your destination or touch the \"Music\" icon to play your favorite songs."
DOCUMENT3 = "Shifting Gears Your Googlecar has an automatic transmission. To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions."
texts = [DOCUMENT1, DOCUMENT2, DOCUMENT3]
Organize the contents of the dictionary into a dataframe for better visualization.
df = pd.DataFrame(texts)
df.columns = ['Text']
df
Get the embeddings for each of these bodies of text. Add this information to the dataframe.
# Get the embeddings of each text and add to an embeddings column in the dataframe
def embed_fn(text):
return palm.generate_embeddings(model=model, text=text)['embedding']
df['Embeddings'] = df['Text'].apply(embed_fn)
df
Document search with Q & A
Now that the embeddings are generated, let's create a Q & A system to search these documents. A user will ask a question about hyperparameter tuning, create an embedding of the question, and compare it against the collection of embeddings in the dataframe.
The embedding of the question will be a vector (list of float values), which will be compared against the vector of the documents using the dot product. This vector returned from the API is already normalized. The dot product represents the similarity in direction between two vectors.
The values of the dot product can range between -1 and 1, inclusive. If the dot product between two vectors is 1, then the vectors are in the same direction. If the dot product value is 0, then these vectors are orthogonal, or unrelated, to each other. Lastly, if the dot product is -1, then the vectors point in the opposite direction and are not similar to each other.
query = "How do you shift gears in the Google car?"
Use the find_best_passage function to calculate the dot products, and then sort the dataframe from the largest to smallest dot product value to retrieve the relevant passage out of the database.
def find_best_passage(query, dataframe):
"""
Compute the distances between the query and each document in the dataframe
using the dot product.
"""
query_embedding = palm.generate_embeddings(model=model, text=query)
dot_products = np.dot(np.stack(dataframe['Embeddings']), query_embedding['embedding'])
idx = np.argmax(dot_products)
return dataframe.iloc[idx]['Text'] # Return text from index with max value
View the most relevant document from the database:
passage = find_best_passage(query, df)
passage
'Shifting Gears Your Googlecar has an automatic transmission. To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions.'
Question and Answering Application
Let's try to use the text generation API to create a Q & A system. Input your own custom data below to create a simple question and answering example. You will still use the dot product as a metric of similarity.
def make_prompt(query, relevant_passage):
escaped = relevant_passage.replace("'", "").replace('"', "").replace("\n", " ")
prompt = textwrap.dedent("""You are a helpful and informative bot that answers questions using text from the reference passage included below. \
Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. \
However, you are talking to a non-technical audience, so be sure to break down complicated concepts and \
strike a friendly and converstional tone. \
If the passage is irrelevant to the answer, you may ignore it.
QUESTION: '{query}'
PASSAGE: '{relevant_passage}'
ANSWER:
""").format(query=query, relevant_passage=escaped)
return prompt
prompt = make_prompt(query, passage)
print(prompt)
You are a helpful and informative bot that answers questions using text from the reference passage included below. Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. However, you are talking to a non-technical audience, so be sure to break down complicated concepts and strike a friendly and converstional tone. If the passage is irrelevant to the answer, you may ignore it.
QUESTION: 'How do you shift gears in the Google car?'
PASSAGE: 'Shifting Gears Your Googlecar has an automatic transmission. To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions.'
ANSWER:
Choose one of the PaLM text generation models in order to find the answer to your query. The temperature controls the randomness of the output. The larger the value, the more random the generated text will be. The answer is a text completion object based on the prompt passed in.
text_models = [m for m in palm.list_models() if 'generateText' in m.supported_generation_methods]
text_model = text_models[0]
temperature = 0.5
answer = palm.generate_text(prompt=prompt,
model=text_model,
candidate_count=3,
temperature=temperature,
max_output_tokens=1000)
for i, candidate in enumerate(answer.candidates):
print(f"Candidate {i}: {candidate['output']}\n")
Candidate 0: To shift gears in the Google car, simply move the shift lever to the desired position. Park, reverse, neutral, drive, and low. Candidate 1: To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions. Candidate 2: To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions.
Next steps
You've now created your own document search application using the embeddings from the PaLM API! To learn more about how you can use the embeddings, check out the examples available. To learn how to use other services in the PaLM API, visit the various quickstart guides: