feat: implement Kotlin sample parity, visual test suite, and autonomous QA automation by dkhawk · Pull Request #2424 · googlemaps-samples/android-samples

dkhawk · 2026-09-15T00:09:29Z

Summary

Implement verified Kotlin sample parity fixes across Camera, VisibleRegion, Marker, Boundaries, DatasetStyling, CloudStyling, GroundOverlay, and TileOverlay
Simulate Fowler / Rattlesnake GPX track and add modern runtime permission launcher in LocationSourceDemoActivity
Add :visual-testing library module with GeminiVisualTestHelper
Add on-device visual verification test suite (VerifiedSamplesVisualTest, VisualVerificationTestSuite)
Add host-side autonomous QA evaluation engine in scripts/eval/ and run_visual_tests.py dispatcher

Stacked Base

Stacked on #2423 (feat/apidemos-java-parity-and-tests).

Reviewers

@kikoso @LoyalAbbas

snippet-bot · 2026-09-15T00:38:24Z

Here is the summary of changes.

You are about to add 8 region tags.

ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/DataDrivenBoundariesActivity.kt:67, tag maps_android_data_driven_styling_boundaries
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/DataDrivenDatasetStylingActivity.kt:72, tag maps_android_data_driven_styling_datasets
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/LiteDemoActivity.kt:59, tag maps_android_sample_lite
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/LocationSourceDemoActivity.kt:77, tag maps_android_sample_location_source
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/MultiMapDemoActivity.kt:47, tag maps_android_sample_multimap
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/StyledMapDemoActivity.kt:58, tag maps_android_sample_styled_map
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/UiSettingsDemoActivity.kt:87, tag maps_android_sample_ui_settings
ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/VisibleRegionDemoActivity.kt:63, tag maps_android_sample_visible_region

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

LoyalAbbas · 2026-09-15T14:55:22Z

+
+    protected val geminiApiKey: String by lazy {
+        try {
+            BuildConfig.MAPS_API_KEY // Falls back if GEMINI_API_KEY is not separately generated


Do we really wants to access MAPS_API_KEY here or we need to use GEMINI_API_KEY ??. I could not find GEMINI_API_KEY in the code.

LoyalAbbas · 2026-09-15T15:01:12Z

+        ActivityScenario.launch(VisibleRegionDemoActivity::class.java).use {
+            waitForMap()
+            // Tap "Actions ▾" popup menu button
+            uiDevice.click(539, 529)


In VerifiedSamplesVisualTest.kt (and lines 77, 103, 132, 158, 183, 211, 239, 265):

Absolute pixel coordinates (x=967, y=2324) are calibrated to a single 1080×2400 display density/resolution. On any CI emulator, Gradle Managed Device, or phone with different dimensions (e.g. 1080×1920, Pixel Tablet, or landscape orientation), these clicks miss the target views or tap the system navigation bar.

Fix: Replace raw pixel coordinates with view selectors (uiDevice.findObject(By.res(context.packageName, "styling_terrain_mode")).click() or Espresso onView(withId(R.id.styling_terrain_mode)).perform(click())).

What did you think?? we need to consider this or not ?

…us QA automation - Implement verified Kotlin sample parity fixes across Camera, VisibleRegion, Marker, Boundaries, DatasetStyling, CloudStyling, GroundOverlay, and TileOverlay - Simulate Fowler / Rattlesnake GPX track and add modern runtime permission launcher in LocationSourceDemoActivity - Add :visual-testing library module with GeminiVisualTestHelper - Add on-device visual verification test suite (VerifiedSamplesVisualTest, VisualVerificationTestSuite) - Add host-side autonomous QA evaluation engine in scripts/eval/ and run_visual_tests.py dispatcher

+            this.bearing = bearing
+            speed = 4.5f
+            time = System.currentTimeMillis()
+            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {


LoyalAbbas · 2026-09-16T03:05:52Z

+                    // Then we add it using a FragmentTransaction into the standard sample content container.
                    val fragmentTransaction = supportFragmentManager.beginTransaction()
-                    fragmentTransaction.add(android.R.id.content, it, MAP_FRAGMENT_TAG)
+                    fragmentTransaction.add(com.example.common_ui.R.id.sample_content_container, it, MAP_FRAGMENT_TAG)


Immediate Runtime Crash : neither ProgrammaticDemoActivity nor SamplesBaseActivity.kt calls setContentView(R.layout.activity_sample_base). As a result, R.id.sample_content_container does not exist in the Activity's view hierarchy.
Impact: Launching ProgrammaticDemoActivity crashes immediately with: java.lang.IllegalArgumentException: No view found for id ... for fragment SupportMapFragment.

Fix: call setContentView(com.example.common_ui.R.layout.activity_sample_base); in onCreate() before running the FragmentTransaction

LoyalAbbas · 2026-09-16T03:09:17Z

+            if (!key.isNullOrBlank() && key != "DEFAULT_API_KEY") {
+                key
+            } else {
+                BuildConfig.MAPS_API_KEY


When GEMINI_API_KEY is not set (the default for developers and CI environments that only configure MAPS_API_KEY), geminiApiKey resolves to BuildConfig.MAPS_API_KEY.

In verifyScreenshotWithGemini, geminiApiKey.isNotBlank() && geminiApiKey != "DEFAULT_API_KEY" evaluates to true, sending the Google Maps SDK key to generativelanguage.googleapis.com, which fails with HTTP 400/403 and throws an exception instead of using the offline assertion fallback

LoyalAbbas · 2026-09-16T03:14:54Z

+        this.listener = listener
+        if (isRunning) {
+            emitCurrentPoint()
+            handler.postDelayed(stepRunnable, intervalMs)


Can we call handler.removeCallbacks(stepRunnable) before calling handler.postDelayed(stepRunnable, intervalMs) ?

LoyalAbbas · 2026-09-16T03:15:38Z

+    fun togglePlayback(): Boolean {
+        isRunning = !isRunning
+        if (isRunning) {
+            handler.post(stepRunnable)


Can we call handler.removeCallbacks(stepRunnable) before calling handler.postDelayed(stepRunnable, intervalMs) ?

kikoso · 2026-09-16T10:19:27Z

+    protected val geminiApiKey: String by lazy {
+        try {
+            val geminiKeyField = try {
+                BuildConfig::class.java.getField("GEMINI_API_KEY")
+            } catch (e: NoSuchFieldException) {
+                null
+            }
+            val key = geminiKeyField?.get(null) as? String
+            if (!key.isNullOrBlank() && key != "DEFAULT_API_KEY") {
+                key
+            } else {
+                BuildConfig.MAPS_API_KEY


GEMINI_API_KEY isn't defined in any gradle file on this branch, so this fallback is the path that always runs — meaning we'd be sending the Maps key to generativelanguage.googleapis.com on every call.

Should we read it via buildConfigField (the way gemini_eval_engine.py already reads it from env / secrets.properties) and assumeTrue the test away when it's missing, instead of falling back?

kikoso · 2026-09-16T10:32:38Z

+    protected suspend fun verifyScreenshotWithGemini(bitmap: Bitmap, prompt: String) {
+        if (geminiApiKey.isNotBlank() && geminiApiKey != "DEFAULT_API_KEY") {
+            val response = helper.analyzeImage(bitmap, prompt, geminiApiKey)
+            Log.i(TAG, "Gemini Visual Evaluation Response:\n$response")
+            assertTrue(
+                "Gemini visual verification failed. Response: $response",
+                response?.contains("PASSED", ignoreCase = true) == true
+            )
+        } else {
+            // Offline/CI assertion fallback: verify screenshot has valid dimensions and non-empty buffer
+            assertTrue("Screenshot width must be > 0", bitmap.width > 0)
+            assertTrue("Screenshot height must be > 0", bitmap.height > 0)
+        }


I think this can't fail either way. Without a key we only assert width > 0 / height > 0, which captureScreenshot already guarantees — so on CI all 9 tests pass unconditionally. With a key, contains("PASSED") also matches "the criteria were not met, so this is not PASSED".

Should we ask for responseMimeType: "application/json" and assert on a parsed verdict field, and assumeTrue when there's no key so it skips instead of silently passing?

kikoso · 2026-09-16T10:47:25Z

+// Visual Testing
+include(":visual-testing")
+project(":visual-testing").projectDir = file("visual-testing")


Should we split this PR? A new root-level module landing here is really four unrelated changes in one commit — Kotlin sample parity across 25 activities, :visual-testing, the instrumented suite, and ~3k lines of python in scripts/eval/.

It's also feat:, so release-please will cut a minor bump for the whole repo off what's mostly internal tooling. Maybe three PRs and a chore: for the tooling one?

kikoso · 2026-09-16T10:55:04Z

+"""Forwarding wrapper for scripts/eval/run_autonomous_qa_suite.py."""
+
+import sys
+from pathlib import Path
+
+EVAL_DIR = Path(__file__).resolve().parent / "eval"
+sys.path.insert(0, str(EVAL_DIR))
+
+from run_autonomous_qa_suite import *
+import run_autonomous_qa_suite


Can we just drop these? There are five of these forwarding shims in scripts/ (plus scripts/eval/generate_html_report.py, which calls itself a "backward compatibility wrapper") — but all the targets are new in this same commit, so there's nothing to stay compatible with yet.

Also the import * on line 24 is redundant with line 25, and AGENTS.md asks for no wildcard imports.

kikoso · 2026-09-16T10:57:29Z

+@Sample(
+    id = "camera_demo",


These ids don't line up with the registry — SampleCatalogRegistry says it keys on FQCN and has com.example.kotlindemos.CameraDemoActivity for this one. 18 of the 23 annotations here are snake_case, the other 5 (Lite, LocationSource, MultiMap, StyledMap, VisibleRegion) are FQCN, so any lookup joining the two silently misses on most samples. Can we settle on one scheme?

Broader question: I couldn't find anything that actually reads @Sample — CatalogScreen goes through the registry only. Should we generate the registry from these annotations rather than hand-syncing two copies of the same metadata?

kikoso · 2026-09-16T11:02:18Z

+        // Log available models first for easier debugging.
+        listAvailableModels(apiKey)


This fires an extra round trip on every single analyzeImage call, just to Log.i the model list — so a 31-sample run doubles its request count. Should we put it behind a debug flag, or only call it once from the test setup?

kikoso · 2026-09-16T11:23:56Z

+plugins {
+    alias(libs.plugins.android.library)
+    alias(libs.plugins.kotlin.serialization)
+}


The module goes into settings.gradle.kts but not into MODULES[] in scripts/verify_all.sh, so our own verification script never assembles, tests or lints it. Should we add it there too?

kikoso · 2026-09-16T11:23:58Z

+dependencies {
+    implementation(libs.appcompat)
+    implementation(libs.core.ktx)
+    testImplementation(libs.junit)
+    testImplementation(libs.robolectric)
+    testImplementation(libs.truth)
+
+    // Dependencies for GeminiVisualTestHelper
+    implementation(libs.ktor.client.core)
+    implementation(libs.ktor.client.cio)
+    implementation(libs.ktor.client.content.negotiation)
+    implementation(libs.ktor.serialization.kotlinx.json)
+    implementation(libs.kotlinx.serialization.json)
+    implementation(libs.uiautomator)
+}


Can we trim these? The header of GeminiVisualTestHelper says it deliberately uses org.json to dodge kotlinx.serialization binary-compat issues — but we still apply the serialization plugin and pull in ktor-client-content-negotiation, ktor-serialization-kotlinx-json and kotlinx-serialization-json. appcompat and core-ktx look unused too.

Also, should minSdk on line 30 come from libs.versions.minSdk rather than being hardcoded to 23?

kikoso · 2026-09-16T11:23:59Z

+    private val client = HttpClient(CIO) {
+        install(HttpTimeout) {
+            requestTimeoutMillis = 60_000
+            connectTimeoutMillis = 60_000
+            socketTimeoutMillis = 60_000
+        }
+    }
+
+    /**


This client is never closed and the class isn't Closeable — CIO allocates a thread pool per instance, and BaseVisualVerificationTest creates one per test class. Should we make the helper Closeable and close it in an @After?

kikoso · 2026-09-16T11:24:00Z

+            })
+        }
+
+        val response: HttpResponse = client.post("https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent?key=$apiKey") {


We end up with three different endpoints hardcoded inline across this file — v1/models/gemini-2.5-flash (line 96), v1/models (line 156) and v1beta/models/gemini-3-flash-preview here. Should we pull the base URL, API version and model names out into constants so they can't drift apart?

Separately: the key goes in the query string on all three. Any reason not to use the x-goog-api-key header instead, so it doesn't end up in proxy/request logs?

dkhawk mentioned this pull request Sep 15, 2026

feat: introduce snippets common library module and documentation tools #2425

Open

dkhawk force-pushed the feat/apidemos-kotlin-parity-and-visual-qa branch from bfe7f4c to 1c5e1e9 Compare September 15, 2026 00:24

dkhawk force-pushed the feat/apidemos-kotlin-parity-and-visual-qa branch from 1c5e1e9 to de40cf2 Compare September 15, 2026 00:34

dkhawk marked this pull request as ready for review September 15, 2026 00:38

dkhawk requested review from LoyalAbbas and kikoso September 15, 2026 00:38

dkhawk added this pull request to stack #2429 September 15, 2026 00:42

LoyalAbbas reviewed Sep 15, 2026

View reviewed changes

dkhawk force-pushed the feat/apidemos-kotlin-parity-and-visual-qa branch from de40cf2 to ed84854 Compare September 15, 2026 17:55

github-advanced-security Bot found potential problems Sep 15, 2026

View reviewed changes

Comment thread ApiDemos/project/kotlin-app/src/main/java/com/example/kotlindemos/LocationSourceDemoActivity.kt

this.bearing = bearing

speed = 4.5f

time = System.currentTimeMillis()

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {

LoyalAbbas reviewed Sep 16, 2026

View reviewed changes

kikoso reviewed Sep 16, 2026

View reviewed changes

kikoso mentioned this pull request Sep 16, 2026

feat: implement Java sample parity, animations, and interaction handlers #2423

Open

		// Log available models first for easier debugging.
		listAvailableModels(apiKey)

		@Sample(
		id = "camera_demo",

Uh oh!

Conversation

dkhawk commented Sep 15, 2026

Summary

Stacked Base

Reviewers

Uh oh!

snippet-bot Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

snippet-bot Bot commented Sep 15, 2026 •

edited

Loading