Skip to content

Migrate Magic Selfie to Nano Banana2#157

Open
lethargicpanda wants to merge 1 commit intomainfrom
te/selfie-nanobanana
Open

Migrate Magic Selfie to Nano Banana2#157
lethargicpanda wants to merge 1 commit intomainfrom
te/selfie-nanobanana

Conversation

@lethargicpanda
Copy link
Collaborator

Migrate Magic Selfie to Nano Banana2 and remove ML Kit depency for background selection

Migrate Magic Selfie to Nano Banana2 and remove ML Kit depency for background selection
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the "Magic Selfie" sample application by upgrading its core AI technology. It transitions from a two-component system (ML Kit for segmentation and Imagen for generation) to a unified approach leveraging the Nano Banana 2 (Gemini 3.1 Flash Image Preview) model. This change simplifies the application's architecture, reduces dependencies, and enhances the image manipulation process by utilizing a single multimodal model capable of direct semantic image editing.

Highlights

  • Model Migration: The Magic Selfie sample has been migrated from using Imagen and ML Kit for background segmentation and generation to the Nano Banana 2 (Gemini 3.1 Flash Image Preview) model.
  • Simplified Workflow: The previous multi-step process involving separate foreground segmentation and background generation has been replaced by a single, more efficient multimodal model call to Gemini Flash.
  • Dependency Removal: The dependency on ML Kit for subject segmentation has been entirely removed, streamlining the project's dependencies.
  • UI and Documentation Updates: User interface states, string resources, and the sample's README documentation have been updated to reflect the new underlying AI model and simplified process.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • app/src/main/java/com/android/ai/catalog/domain/SampleCatalog.kt
    • Updated the sample tags for Magic Selfie to reflect the use of Gemini Flash.
  • app/src/main/res/values/strings.xml
    • Updated the title and description strings for the Magic Selfie sample to mention Gemini Flash and remove references to Imagen and ML Kit.
  • samples/magic-selfie/README.md
    • Updated the description to explain the use of Nano Banana 2 for semantic image editing.
    • Revised the "How it works" section to detail the multimodal prompting capabilities of Nano Banana 2.
    • Replaced the code snippet to show the new generateMagicSelfie function call.
  • samples/magic-selfie/src/main/java/com/android/ai/samples/magicselfie/data/MagicSelfieRepository.kt
    • Removed the imagenModel and subjectSegmenter instances.
    • Deleted the generateForegroundBitmap, generateBackground, and combineBitmaps functions.
    • Introduced a generativeModel initialized with gemini-3.1-flash-image-preview.
    • Added a new generateMagicSelfie function that uses multimodal input to generate the final image.
  • samples/magic-selfie/src/main/java/com/android/ai/samples/magicselfie/ui/MagicSelfieScreen.kt
    • Removed the MagicSelfieUiState.RemovingBackground check from the enabled state of the generate and take picture buttons.
  • samples/magic-selfie/src/main/java/com/android/ai/samples/magicselfie/ui/MagicSelfieUiState.kt
    • Removed the RemovingBackground data object from the sealed interface.
  • samples/magic-selfie/src/main/java/com/android/ai/samples/magicselfie/ui/MagicSelfieViewModel.kt
    • Simplified the createMagicSelfie function to directly call the new magicSelfieRepository.generateMagicSelfie method.
    • Removed the _uiState.value = MagicSelfieUiState.RemovingBackground state update.
  • samples/magic-selfie/src/main/res/values/strings.xml
    • Updated the magic_selfie_subtitle string to reflect the use of the Gemini Flash model.
Activity
  • No specific activity has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the Magic Selfie sample to use the gemini-3.1-flash-image-preview model, removing the dependency on ML Kit and simplifying the implementation. However, it introduces a potential prompt injection vulnerability in the MagicSelfieRepository where user input is directly concatenated into the LLM prompt without sanitization. Besides the security concern, the changes are well-executed and consistent across the codebase, including updates to the UI, data layer, and documentation. Consider improving the error handling and ensuring documentation consistency.

suspend fun generateMagicSelfie(bitmap: Bitmap, prompt: String): Bitmap {
val multimodalPrompt = content {
image(bitmap)
text("Change the background of this image to $prompt")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The user-provided prompt is directly concatenated into the multimodal prompt sent to the Gemini model. This makes the application vulnerable to prompt injection, where an attacker can provide a crafted prompt to manipulate the model's behavior, potentially bypassing intended constraints or generating inappropriate content. Consider sanitizing the input or using system instructions to define the model's behavior more securely.

}
val response = generativeModel.generateContent(multimodalPrompt)
return response.candidates.firstOrNull()?.content?.parts?.firstNotNullOfOrNull { it.asImageOrNull() }
?: throw Exception("No image generated")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To keep the documentation consistent with the suggested code change in MagicSelfieRepository.kt, please update the exception type in this code snippet as well. This ensures the documentation accurately reflects the implementation.

Suggested change
?: throw Exception("No image generated")
?: throw IllegalStateException("No image generated")

Comment on lines +46 to +47
return response.candidates.firstOrNull()?.content?.parts?.firstNotNullOfOrNull { it.asImageOrNull() }
?: throw Exception("No image generated")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For better error handling and clarity, it's recommended to throw a more specific exception than the generic Exception. Using IllegalStateException would be more appropriate here, as it signals that the response from the generative model was not in the expected state (i.e., it did not contain an image). This makes debugging easier and the code more self-documenting.

Suggested change
return response.candidates.firstOrNull()?.content?.parts?.firstNotNullOfOrNull { it.asImageOrNull() }
?: throw Exception("No image generated")
return response.candidates.firstOrNull()?.content?.parts?.firstNotNullOfOrNull { it.asImageOrNull() }
?: throw IllegalStateException("No image generated")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant