Get started with the Gemini API in Android apps (client SDK)

This tutorial demonstrates how to access the Gemini API directly from your Android app using the Google AI client SDK for Android. You can use this client SDK if you don't want to work directly with REST APIs or server-side code (like Python) for accessing Gemini models in your Android app.

In this tutorial, you'll learn how to do the following:

In addition, this tutorial contains sections about advanced use cases (like counting tokens) as well as options for controlling content generation.

Consider accessing Gemini on-device

The client SDK for Android described in this tutorial lets you access the Gemini Pro models which run on Google's servers. For use cases that involve processing sensitive data, offline availability, or for cost savings for frequently used user flows, you may want to consider accessing Gemini Nano which runs on-device. For more details, refer to the Android (on-device) tutorial.

Prerequisites

This tutorial assumes that you're familiar with using Android Studio to develop Android apps.

To complete this tutorial, make sure that your development environment and Android app meet the following requirements:

  • Android Studio (latest version)
  • Your Android app must target API level 21 or higher.

Set up your project

Before calling the Gemini API, you need to set up your Android project, which includes setting up your API key, adding the SDK dependencies to your Android project, and initializing the model.

Set up your API key

To use the Gemini API, you'll need an API key. If you don't already have one, create a key in Google AI Studio.

Get an API key

Secure your API key

It's strongly recommended that you do not check an API key into your version control system. Instead, you should store it in a local.properties file (which is located in your project's root directory, but excluded from version control), and then use the Secrets Gradle plugin for Android to read your API key as a Build Configuration variable.

Kotlin 

// Access your API key as a Build Configuration variable
val apiKey = BuildConfig.apiKey

Java 

// Access your API key as a Build Configuration variable
String apiKey = BuildConfig.apiKey;

All the snippets in this tutorial utilize this best practice. Also, if you want to see the implementation of the Secrets Gradle plugin, you can review the sample app for this SDK or use the latest preview of Android Studio Iguana which has a Gemini API Starter template (which includes the local.properties file to get you started).

Add the SDK dependency to your project

  1. In your module (app-level) Gradle configuration file (like <project>/<app-module>/build.gradle.kts), add the dependency for the Google AI SDK for Android:

    Kotlin 

    dependencies {
  // ... other androidx dependencies

  // add the dependency for the Google AI client SDK for Android
  implementation("com.google.ai.client.generativeai:generativeai:0.3.0")
}

    Java

    For Java, you need to add two additional libraries.

    dependencies {
    // ... other androidx dependencies

    // add the dependency for the Google AI client SDK for Android
    implementation("com.google.ai.client.generativeai:generativeai:0.3.0")

    // Required for one-shot operations (to use `ListenableFuture` from Guava Android)
    implementation("com.google.guava:guava:31.0.1-android")

    // Required for streaming operations (to use `Publisher` from Reactive Streams)
    implementation("org.reactivestreams:reactive-streams:1.0.4")
}

  2. Sync your Android project with Gradle files.

Initialize the Generative Model

Before you can make any API calls, you need to initialize the GenerativeModel object:

Kotlin 

val generativeModel = GenerativeModel(
    // Use a model that's applicable for your use case (see "Implement basic use cases" below)
    modelName = "MODEL_NAME",
    // Access your API key as a Build Configuration variable (see "Set up your API key" above)
    apiKey = BuildConfig.apiKey
)

Java

For Java, you also need to initialize the GenerativeModelFutures object.

// Use a model that's applicable for your use case (see "Implement basic use cases" below)
GenerativeModel gm = new GenerativeModel(/* modelName */ "MODEL_NAME",
// Access your API key as a Build Configuration variable (see "Set up your API key" above)
    /* apiKey */ BuildConfig.apiKey);

// Use the GenerativeModelFutures Java compatibility layer which offers
// support for ListenableFuture and Publisher APIs
GenerativeModelFutures model = GenerativeModelFutures.from(gm);

When specifying a model, note the following:

  • Use a model that's specific to your use case (for example, gemini-pro-vision is for multimodal input). Within this guide, the instructions for each implementation list the recommended model for each use case.

Implement common use cases

Now that your project is set up, you can explore using the Gemini API to implement different use cases:

Generate text from text-only input

When the prompt input includes only text, use the gemini-pro model with generateContent to generate text output:

Kotlin

Note that generateContent() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

val generativeModel = GenerativeModel(
    // For text-only input, use the gemini-pro model
    modelName = "gemini-pro",
    // Access your API key as a Build Configuration variable (see "Set up your API key" above)
    apiKey = BuildConfig.apiKey
)

val prompt = "Write a story about a magic backpack."
val response = generativeModel.generateContent(prompt)
print(response.text)

Java

Note that generateContent() returns a ListenableFuture. If you're unfamiliar with this API, see the Android documentation about Using a ListenableFuture.

// For text-only input, use the gemini-pro model
GenerativeModel gm = new GenerativeModel(/* modelName */ "gemini-pro",
// Access your API key as a Build Configuration variable (see "Set up your API key" above)
    /* apiKey */ BuildConfig.apiKey);
GenerativeModelFutures model = GenerativeModelFutures.from(gm);

Content content = new Content.Builder()
    .addText("Write a story about a magic backpack.")
    .build();

Executor executor = // ...

ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
    @Override
    public void onSuccess(GenerateContentResponse result) {
        String resultText = result.getText();
        System.out.println(resultText);
    }

    @Override
    public void onFailure(Throwable t) {
        t.printStackTrace();
    }
}, executor);

Generate text from text-and-image input (multimodal)

Gemini provides a multimodal model (gemini-pro-vision), so you can input both text and images. Make sure to review the image requirements for prompts.

When the prompt input includes both text and images, use the gemini-pro-vision model with generateContent to generate text output:

Kotlin

Note that generateContent() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

val generativeModel = GenerativeModel(
    // For text-and-images input (multimodal), use the gemini-pro-vision model
    modelName = "gemini-pro-vision",
    // Access your API key as a Build Configuration variable (see "Set up your API key" above)
    apiKey = BuildConfig.apiKey
)

val image1: Bitmap = // ...
val image2: Bitmap = // ...

val inputContent = content {
    image(image1)
    image(image2)
    text("What's different between these pictures?")
}

val response = generativeModel.generateContent(inputContent)
print(response.text)

Java

Note that generateContent() returns a ListenableFuture. If you're unfamiliar with this API, see the Android documentation about Using a ListenableFuture.

// For text-and-images input (multimodal), use the gemini-pro-vision model
GenerativeModel gm = new GenerativeModel(/* modelName */ "gemini-pro-vision",
// Access your API key as a Build Configuration variable (see "Set up your API key" above)
    /* apiKey */ BuildConfig.apiKey);
GenerativeModelFutures model = GenerativeModelFutures.from(gm);

Bitmap image1 = // ...
Bitmap image2 = // ...

Content content = new Content.Builder()
    .addText("What's different between these pictures?")
    .addImage(image1)
    .addImage(image2)
    .build();

Executor executor = // ...

ListenableFuture<GenerateContentResponse> response = model.generateContent(content);
Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
    @Override
    public void onSuccess(GenerateContentResponse result) {
        String resultText = result.getText();
        System.out.println(resultText);
    }

    @Override
    public void onFailure(Throwable t) {
        t.printStackTrace();
    }
}, executor);

Build multi-turn conversations (chat)

Using Gemini, you can build freeform conversations across multiple turns. The SDK simplifies the process by managing the state of the conversation, so unlike with generateContent, you don't have to store the conversation history yourself.

To build a multi-turn conversation (like chat), use the gemini-pro model, and initialize the chat by calling startChat(). Then use sendMessage() to send a new user message, which will also append the message and the response to the chat history.

There are two possible options for role associated with the content in a conversation:

  • user: the role which provides the prompts. This value is the default for sendMessage calls.

  • model: the role which provides the responses. This role can be used when calling startChat() with existing history.

Kotlin

Note that generateContent() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

val generativeModel = GenerativeModel(
    // For text-only input, use the gemini-pro model
    modelName = "gemini-pro",
    // Access your API key as a Build Configuration variable (see "Set up your API key" above)
    apiKey = BuildConfig.apiKey
)

val chat = generativeModel.startChat(
    history = listOf(
        content(role = "user") { text("Hello, I have 2 dogs in my house.") },
        content(role = "model") { text("Great to meet you. What would you like to know?") }
    )
)

chat.sendMessage("How many paws are in my house?")

Java

Note that generateContent() returns a ListenableFuture. If you're unfamiliar with this API, see the Android documentation about Using a ListenableFuture.

// For text-only input, use the gemini-pro model
GenerativeModel gm = new GenerativeModel(/* modelName */ "gemini-pro",
// Access your API key as a Build Configuration variable (see "Set up your API key" above)
    /* apiKey */ BuildConfig.apiKey);
GenerativeModelFutures model = GenerativeModelFutures.from(gm);

// (optional) Create previous chat history for context
Content.Builder userContentBuilder = new Content.Builder();
userContentBuilder.setRole("user");
userContentBuilder.addText("Hello, I have 2 dogs in my house.");
Content userContent = userContentBuilder.build();

Content.Builder modelContentBuilder = new Content.Builder();
modelContentBuilder.setRole("model");
modelContentBuilder.addText("Great to meet you. What would you like to know?");
Content modelContent = userContentBuilder.build();

List<Content> history = Arrays.asList(userContent, modelContent);

// Initialize the chat
ChatFutures chat = model.startChat(history);

// Create a new user message
Content userMessage = new Content.Builder()
    .setRole("user")
    .addText("How many paws are in my house?")
    .build();

Executor executor = // ...

// Send the message
ListenableFuture<GenerateContentResponse> response = chat.sendMessage(userMessage);

Futures.addCallback(response, new FutureCallback<GenerateContentResponse>() {
    @Override
    public void onSuccess(GenerateContentResponse result) {
        String resultText = result.getText();
        System.out.println(resultText);
    }

    @Override
    public void onFailure(Throwable t) {
        t.printStackTrace();
    }
}, executor);

Use streaming for faster interactions

By default, the model returns a response after completing the entire generation process. You can achieve faster interactions by not waiting for the entire result, and instead use streaming to handle partial results.

The following example shows how to implement streaming with generateContentStream to generate text from a text-and-image input prompt.

Kotlin

Note that generateContentStream() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

val generativeModel = GenerativeModel(
    // For text-and-image input (multimodal), use the gemini-pro-vision model
    modelName = "gemini-pro-vision",
    // Access your API key as a Build Configuration variable (see "Set up your API key" above)
    apiKey = BuildConfig.apiKey
)

val image1: Bitmap = // ...
val image2: Bitmap = // ...

val inputContent = content {
    image(image1)
    image(image2)
    text("What's the difference between these pictures?")
}

var fullResponse = ""
generativeModel.generateContentStream(inputContent).collect { chunk ->
    print(chunk.text)
    fullResponse += chunk.text
}

Java

The Java streaming methods in this SDK return a Publisher type from the Reactive Streams library.

// For text-and-images input (multimodal), use the gemini-pro-vision model
GenerativeModel gm = new GenerativeModel(/* modelName */ "gemini-pro-vision",
// Access your API key as a Build Configuration variable (see "Set up your API key" above)
    /* apiKey */ BuildConfig.apiKey);
GenerativeModelFutures model = GenerativeModelFutures.from(gm);

Bitmap image1 = // ...
Bitmap image2 = // ...

Content content = new Content.Builder()
    .addText("What's different between these pictures?")
    .addImage(image1)
    .addImage(image2)
    .build();

Publisher<GenerateContentResponse> streamingResponse =
    model.generateContentStream(content);

final String[] fullResponse = {""};

streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
    @Override
    public void onNext(GenerateContentResponse generateContentResponse) {
        String chunk = generateContentResponse.getText();
        fullResponse[0] += chunk;
    }

    @Override
    public void onComplete() {
        System.out.println(fullResponse[0]);
    }

    @Override
    public void onError(Throwable t) {
        t.printStackTrace();
    }

    @Override
    public void onSubscribe(Subscription s) { }
});

You can use a similar approach for text-only input and chat use cases:

Kotlin

Note that generateContentStream() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

// Use streaming with text-only input
generativeModel.generateContentStream(inputContent).collect { chunk ->
    print(chunk.text)
}

// Use streaming with multi-turn conversations (like chat)
val chat = generativeModel.startChat()
chat.sendMessageStream(inputContent).collect { chunk ->
    print(chunk.text)
}

Java

The Java streaming methods in this SDK return a Publisher type from the Reactive Streams library.

// Use streaming with text-only input
Publisher<GenerateContentResponse> streamingResponse =
    model.generateContentStream(inputContent);

final String[] fullResponse = {""};

streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
    @Override
    public void onNext(GenerateContentResponse generateContentResponse) {
        String chunk = generateContentResponse.getText();
        fullResponse[0] += chunk;
    }

    @Override
    public void onComplete() {
        System.out.println(fullResponse[0]);
    }

    // ... other methods omitted for brevity
});

// Use streaming with multi-turn conversations (like chat)
ChatFutures chat = model.startChat(history);

Publisher<GenerateContentResponse> streamingResponse =
    chat.sendMessageStream(inputContent);

final String[] fullResponse = {""};

streamingResponse.subscribe(new Subscriber<GenerateContentResponse>() {
    @Override
    public void onNext(GenerateContentResponse generateContentResponse) {
        String chunk = generateContentResponse.getText();
        fullResponse[0] += chunk;
    }

    @Override
    public void onComplete() {
        System.out.println(fullResponse[0]);
    }

    // ... other methods omitted for brevity
});

Implement advanced use cases

The common use cases described in the previous section of this tutorial help you become comfortable with using the Gemini API. This section describes some use cases that might be considered more advanced.

Count tokens

When using long prompts, it might be useful to count tokens before sending any content to the model. The following examples show how to use countTokens() for various use cases:

Kotlin

Note that countTokens() is a suspend function and needs to be called from a Coroutine scope. If you're unfamiliar with Coroutines, read Kotlin Coroutines on Android.

// For text-only input
val (totalTokens) = generativeModel.countTokens("Write a story about a magic backpack.")

// For text-and-image input (multi-modal)
val multiModalContent = content {
    image(image1)
    image(image2)
    text("What's the difference between these pictures?")
}

val (totalTokens) = generativeModel.countTokens(multiModalContent)

// For multi-turn conversations (like chat)
val history = chat.history
val messageContent = content { text("This is the message I intend to send")}
val (totalTokens) = generativeModel.countTokens(*history.toTypedArray(), messageContent)

Java

Note that countTokens() returns a ListenableFuture. If you're unfamiliar with this API, see the Android documentation about Using a ListenableFuture.

Content text = new Content.Builder()
    .addText("Write a story about a magic backpack.")
    .build();

Executor executor = // ...

// For text-only input
ListenableFuture<CountTokensResponse> countTokensResponse = model.countTokens(text);

Futures.addCallback(countTokensResponse, new FutureCallback<CountTokensResponse>() {
    @Override
    public void onSuccess(CountTokensResponse result) {
        int totalTokens = result.getTotalTokens();
        System.out.println("TotalTokens = " + totalTokens);
    }

    @Override
    public void onFailure(Throwable t) {
        t.printStackTrace();
    }
}, executor);

// For text-and-image input
Bitmap image1 = // ...
Bitmap image2 = // ...

Content multiModalContent = new Content.Builder()
    .addImage(image1)
    .addImage(image2)
    .addText("What's different between these pictures?")
    .build();

ListenableFuture<CountTokensResponse> countTokensResponse = model.countTokens(multiModalContent);

// For multi-turn conversations (like chat)
List<Content> history = chat.getChat().getHistory();

Content messageContent = new Content.Builder()
    .addText("This is the message I intend to send")
    .build();

Collections.addAll(history, messageContent);

ListenableFuture<CountTokensResponse> countTokensResponse = model.countTokens(history.toArray(new Content[0]));

Options to control content generation

You can control content generation by configuring model parameters and by using safety settings.

Configure model parameters

Every prompt you send to the model includes parameter values that control how the model generates a response. The model can generate different results for different parameter values. Learn more about Model parameters.

Kotlin 

val config = generationConfig {
    temperature = 0.9f
    topK = 16
    topP = 0.1f
    maxOutputTokens = 200
    stopSequences = listOf("red")
}

val generativeModel = GenerativeModel(
    modelName = "MODEL_NAME",
    apiKey = BuildConfig.apiKey,
    generationConfig = config
)

Java 

GenerationConfig.Builder configBuilder = new GenerationConfig.Builder();
configBuilder.temperature = 0.9f;
configBuilder.topK = 16;
configBuilder.topP = 0.1f;
configBuilder.maxOutputTokens = 200;
configBuilder.stopSequences = Arrays.asList("red");

GenerationConfig generationConfig = configBuilder.build();

GenerativeModel gm = new GenerativeModel(
    "MODEL_NAME",
    BuildConfig.apiKey,
    generationConfig
);

GenerativeModelFutures model = GenerativeModelFutures.from(gm);

Use safety settings

You can use safety settings to adjust the likelihood of getting responses that may be considered harmful. By default, safety settings block content with medium and/or high probability of being unsafe content across all dimensions. Learn more about Safety settings.

Here's how to set one safety setting:

Kotlin 

val generativeModel = GenerativeModel(
    modelName = "MODEL_NAME",
    apiKey = BuildConfig.apiKey,
    safetySettings = listOf(
        SafetySetting(HarmCategory.HARASSMENT, BlockThreshold.ONLY_HIGH)
    )
)

Java 

SafetySetting harassmentSafety = new SafetySetting(HarmCategory.HARASSMENT,
    BlockThreshold.ONLY_HIGH);

GenerativeModel gm = new GenerativeModel(
    "MODEL_NAME",
    BuildConfig.apiKey,
    null, // generation config is optional
    Collections.singletonList(harassmentSafety)
);

GenerativeModelFutures model = GenerativeModelFutures.from(gm);

You can also set more than one safety setting:

Kotlin 

val harassmentSafety = SafetySetting(HarmCategory.HARASSMENT, BlockThreshold.ONLY_HIGH)

val hateSpeechSafety = SafetySetting(HarmCategory.HATE_SPEECH, BlockThreshold.MEDIUM_AND_ABOVE)

val generativeModel = GenerativeModel(
    modelName = "MODEL_NAME",
    apiKey = BuildConfig.apiKey,
    safetySettings = listOf(harassmentSafety, hateSpeechSafety)
)

Java 

SafetySetting harassmentSafety = new SafetySetting(HarmCategory.HARASSMENT,
    BlockThreshold.ONLY_HIGH);

SafetySetting hateSpeechSafety = new SafetySetting(HarmCategory.HATE_SPEECH,
    BlockThreshold.MEDIUM_AND_ABOVE);

GenerativeModel gm = new GenerativeModel(
    "MODEL_NAME",
    BuildConfig.apiKey,
    null, // generation config is optional
    Arrays.asList(harassmentSafety, hateSpeechSafety)
);

GenerativeModelFutures model = GenerativeModelFutures.from(gm);

What's next

  • Prompt design is the process of creating prompts that elicit the desired response from language models. Writing well structured prompts is an essential part of ensuring accurate, high quality responses from a language model. Learn about best practices for prompt writing.

  • Gemini offers several model variations to meet the needs of different use cases, such as input types and complexity, implementations for chat or other dialog language tasks, and size constraints. Learn about the available Gemini models.

  • Gemini offers options for requesting rate limit increases. The rate limit for Gemini Pro models is 60 requests per minute (RPM).

  • The client SDK for Android described in this tutorial lets you access the Gemini Pro models which run on Google's servers. For use cases that involve processing sensitive data, offline availability, or for cost savings for frequently used user flows, you may want to consider accessing Gemini Nano which runs on-device. For more details, refer to the Android (on-device) tutorial.