diff --git a/jetpacker/.gitignore b/jetpacker/.gitignore index e6bbe5e3..8ec8c528 100644 --- a/jetpacker/.gitignore +++ b/jetpacker/.gitignore @@ -26,3 +26,8 @@ export/ # Generated third-party notices THIRD_PARTY_NOTICES + +# Python files +**/__pycache__/ +*.pyc + diff --git a/jetpacker/CONTRIBUTING.md b/jetpacker/CONTRIBUTING.md index f6abfb3c..f90da93e 100644 --- a/jetpacker/CONTRIBUTING.md +++ b/jetpacker/CONTRIBUTING.md @@ -42,6 +42,7 @@ If you wish to run, explore, or modify the code locally: 1. Clone the repository: ```bash git clone https://github.com/android/ai-samples.git + cd ai-samples/jetpacker ``` 2. Open the project in Android Studio. 3. Gradle will automatically sync and resolve dependencies. diff --git a/jetpacker/README.md b/jetpacker/README.md index 802591b8..76eb4c58 100644 --- a/jetpacker/README.md +++ b/jetpacker/README.md @@ -14,7 +14,7 @@ ## Overview -JetPacker provides users with powerful tools to manage their upcoming trips, build out rich itineraries, record voice notes, manage travel expenses, generate on-device "Trip Summaries and Tips", generate AI reviews, chat with hotel staff via automatic translation, get real-time museum assistant guidance, and contribute to Android's intelligence system with trip management functions through [AppFunctions](https://d.android.com/ai/appfunctions). +JetPacker provides users with powerful tools to manage their upcoming trips, build out rich itineraries, record voice notes, manage travel expenses, generate on-device "Trip Summaries and Tips", generate AI reviews, chat with hotel staff via automatic translation, and get real-time museum assistant guidance. ## Architecture This project is built using modern Android architecture components: @@ -25,7 +25,6 @@ This project is built using modern Android architecture components: - **On-Device AI**: ML Kit GenAI (Prompt, Speech Recognition, Translation) - **Cloud & Hybrid AI**: Firebase AI Logic (Gemini grounded with URL/Maps/Search, and hybrid models with on-device fallback) - **App Security**: Firebase App Check (with Play Integrity and Debug Provider) -- **Assistant Integration**: Android [AppFunctions](https://d.android.com/ai/appfunctions) (`androidx.appfunctions`) ## Module Overview JetPacker follows a clean, multi-module Android structure organized by responsibility and domain: @@ -44,7 +43,6 @@ JetPacker follows a clean, multi-module Android structure organized by responsib - **`:feature:home`**: Dashboard screen displaying the user's upcoming trips list. - **`:feature:create_trip`**: Unified trip form module handling both new trip creation and existing trip editing (`EditTripScreen`). - **`:feature:detail`**: Detailed view screens for specific itinerary events (Flights, Hotels, Restaurants, Museums, Tours). -- **`:feature:appfunctions`**: Android AppFunctions framework integration (`JetPackerAppFunctionService`) contributing structured functions to Android's intelligence system for trips, expenses, itineraries, and voice notes. - **`:feature:trip`**: Top-level trip container shell (`TripScreen`) holding bottom navigation and orchestrating trip sub-tabs. - **`:feature:trip:itinerary`**: Pure itinerary timeline screen displaying daily scheduled events. - **`:feature:trip:itinerary:enrichment`**: On-device AI summaries and tips (`TripSummaryAndTipsCard`) and dynamic daily theme generators. @@ -92,6 +90,25 @@ cd android ./gradlew test ``` +### Running the Booking Assistant Server +JetPacker uses a local Python-based Server-Sent Events (SSE) server to simulate the booking agents. To run the booking assistant backend: + +```bash +# Navigate to the booking-server directory +cd android/booking-server + +# Set your Gemini API Key +export GEMINI_API_KEY="your-api-key-here" + +# Run the server using its virtual environment +./.venv/bin/uvicorn booking_server:app --host 0.0.0.0 --port 8000 +``` + +> [!IMPORTANT] +> A valid `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) environment variable is required. If the key is missing at startup, the server will print a clear error banner and exit immediately. + +This runs the FastAPI booking server on port 8000. When running the app on the Android Emulator, localhost is remapped to `10.0.2.2:8000` to automatically route requests to this server. + ## On-Device AI Features JetPacker integrates local on-device AI capabilities using ML Kit. These features run entirely on-device and can be toggled or customized in `android/core/flags/src/main/kotlin/com/example/jetpacker/core/flags/FeatureFlags.kt`: - **ENABLE_TRIP_SUMMARY_AND_TIPS**: Generates an on-device card summary of the current trip using ML Kit GenAI Prompt. @@ -105,14 +122,6 @@ JetPacker also integrates online hybrid features using Firebase AI Logic (Gemini - **ENABLE_REVIEW_GENERATION**: Topic-selected review generator (uses Gemini 2.5 flash-lite on-device with cloud fallback). - **Hotel Support Chat**: Receives hotel receptionist assistance with real-time ML Kit + Gemini translation. -## AppFunctions Integration -JetPacker uses Android's [**AppFunctions**](https://d.android.com/ai/appfunctions) library (`androidx.appfunctions`) in `:feature:appfunctions` to contribute to Android's intelligence system with structured travel actions and data queries via `JetPackerAppFunctionService`: -- **Trip Management**: `searchTrip` (filter by ID, name, location, and dates) and `createTrip`. -- **Itinerary Events**: `getItinerary` and `addItineraryEvent` (activity, dining, accommodation, etc.). -- **Expense Tracking**: `getExpenses` and `addExpense` (amount, currency, category). -- **Voice Notes**: `getVoiceNotes` and `addVoiceNote` (recorded transcriptions linked to trips). -- **Day Themes**: `getDayThemes` and `setDayTheme` for daily trip customization. - ## IDE Setup & Development diff --git a/jetpacker/android/app/build.gradle.kts b/jetpacker/android/app/build.gradle.kts index a677752f..38205030 100644 --- a/jetpacker/android/app/build.gradle.kts +++ b/jetpacker/android/app/build.gradle.kts @@ -103,6 +103,7 @@ dependencies { implementation(libs.accompanist.permissions) implementation(project(":feature:trip:voice_notes")) implementation(project(":feature:trip:expenses")) + implementation(project(":feature:trip:booking_assistant")) implementation(project(":feature:trip:itinerary:enrichment")) implementation(libs.androidx.activity.compose) implementation(libs.androidx.camera.camera2) @@ -149,6 +150,8 @@ dependencies { implementation(libs.firebase.appcheck.playintegrity) implementation(libs.firebase.appcheck.debug) + + debugImplementation(libs.androidx.compose.ui.test.manifest) debugImplementation(libs.androidx.compose.ui.tooling) diff --git a/jetpacker/android/app/src/main/AndroidManifest.xml b/jetpacker/android/app/src/main/AndroidManifest.xml index 8da10625..bf3fa4ad 100644 --- a/jetpacker/android/app/src/main/AndroidManifest.xml +++ b/jetpacker/android/app/src/main/AndroidManifest.xml @@ -32,7 +32,7 @@ android:label="@string/app_name" android:supportsRtl="true" android:theme="@style/Theme.JetPacker" - android:usesCleartextTraffic="false"> + android:usesCleartextTraffic="true"> str: + sanitized = re.sub(r'[^a-zA-Z0-9_]', '_', title) + if not re.match(r'^[a-zA-Z_]', sanitized): + sanitized = '_' + sanitized + return sanitized + +app = FastAPI() + +# A2UI Catalog Specification System Prompt +schema_manager = A2uiSchemaManager( + version=VERSION_0_9, + catalogs=[ + BasicCatalog.get_config(version=VERSION_0_9), + CatalogConfig.from_path( + name="https://example.com/catalogs/booking_assistant/v1/catalog.json", + catalog_path="booking_catalog.json" + ) + ] +) + +class SessionState: + def __init__(self): + self.pause_events: Dict[str, asyncio.Future] = {} + self.queue: asyncio.Queue = asyncio.Queue() + self.bg_tasks: List[asyncio.Task] = [] + +sessions: Dict[str, SessionState] = {} + +# Custom A2UI Function Tools for ADK Agents + +from pydantic import BaseModel, Field + +A2UI_SYSTEM_INSTRUCTION = schema_manager.generate_system_prompt( + role_description="You are a helpful travel booking assistant.", + ui_description="Use InteractiveOptionPicker for flight/hotel/museum/restaurant booking choices, SeatSelectionPicker for flight seats, and BookingStatus for confirmed status.", + include_schema=True, + include_examples=True, + allowed_components=["InteractiveOptionPicker", "SeatSelectionPicker", "BookingStatus"] +).replace("${expression}", "$(expression)") + +class ComponentProperties(BaseModel): + prompt: Optional[str] = Field(default=None, description="Header prompt/choice text for picker widgets.") + options: Optional[List[str]] = Field(default=None, description="Custom option labels for InteractiveOptionPicker.") + selectedIdx: Optional[int] = Field(default=None, description="Currently selected option index (0-indexed).") + confirmBtnText: Optional[str] = Field(default=None, description="Label for picker confirmation button.") + seats: Optional[List[str]] = Field(default=None, description="Seat labels list for SeatSelectionPicker.") + selectedSeat: Optional[str] = Field(default=None, description="Currently selected seat label.") + text: Optional[str] = Field(default=None, description="Booking success status description text.") + +class A2uiComponent(BaseModel): + id: str = Field(description="Must be 'root' for the main element in the surface card.") + component: str = Field(description="The component name (e.g. 'InteractiveOptionPicker', 'SeatSelectionPicker', 'BookingStatus').") + properties: ComponentProperties = Field(description="The properties for the component.") + +def make_update_ui(surface_id: str, session: SessionState): + async def update_ui( + components: List[A2uiComponent], + tool_context: ToolContext = None + ) -> str: + """Updates the UI surface with the specified components and waits for user interaction if interactive. + + Returns the user's choice (e.g. selected option or seat label), or empty string if not interactive. + """ + # Convert Pydantic A2uiComponent objects to standard dictionaries to send, excluding None + components_dict = [c.model_dump(exclude_none=True) for c in components] + + option_picker = next((c for c in components if c.component == "InteractiveOptionPicker"), None) + seat_picker = next((c for c in components if c.component == "SeatSelectionPicker"), None) + + if option_picker: + comp_dict = next(c for c in components_dict if c["component"] == "InteractiveOptionPicker") + options = option_picker.properties.options or [] + selected_idx = option_picker.properties.selectedIdx + confirmed = False + + while not confirmed: + comp_dict["properties"]["selectedIdx"] = selected_idx + await session.queue.put({ + "updateComponents": { + "surfaceId": surface_id, + "components": components_dict + } + }) + + fut = asyncio.get_event_loop().create_future() + session.pause_events[surface_id] = fut + response = await fut + + if response.startswith("SelectOption_"): + selected_idx = int(response.removeprefix("SelectOption_")) + elif response.startswith("ConfirmSelection_"): + confirmed = True + + return options[selected_idx] if selected_idx is not None and selected_idx < len(options) else "" + + elif seat_picker: + comp_dict = next(c for c in components_dict if c["component"] == "SeatSelectionPicker") + selected_seat = seat_picker.properties.selectedSeat + seat_confirmed = False + + while not seat_confirmed: + comp_dict["properties"]["selectedSeat"] = selected_seat + await session.queue.put({ + "updateComponents": { + "surfaceId": surface_id, + "components": components_dict + } + }) + + fut = asyncio.get_event_loop().create_future() + session.pause_events[surface_id] = fut + response = await fut + + selected_seat = response + seat_confirmed = True + + return selected_seat + + else: + await session.queue.put({ + "updateComponents": { + "surfaceId": surface_id, + "components": components_dict + } + }) + return "" + + return update_ui + +# Flight Agent Coroutine +async def run_flight_agent(title: str, session: SessionState, thread_id: str): + try: + print(f"AGENT [Flight - '{title}']: started") + await session.queue.put({"createSurface": {"surfaceId": title, "catalogId": "https://example.com/catalogs/booking_assistant/v1/catalog.json"}}) + + agent = Agent( + name=sanitize_agent_name(title), + model="gemini-3.1-flash-lite", + instruction=f"Book a flight for: '{title}'. Use the update_ui tool to ask the user to select flight time and seats. Finally use update_ui to show booking confirmation status.\n\n{A2UI_SYSTEM_INSTRUCTION}", + tools=[ + FunctionTool(make_update_ui(title, session)) + ] + ) + + agent_session_id = f"{thread_id}_{sanitize_agent_name(title)}" + runner = InMemoryRunner(agent, app_name="BookingServer") + await runner.session_service.create_session( + app_name="BookingServer", + user_id=thread_id, + session_id=agent_session_id + ) + + async for _ in runner.run_async( + new_message=genai_types.Content(parts=[genai_types.Part.from_text(text=f"Please book: '{title}'")]), + user_id=thread_id, + session_id=agent_session_id + ): + pass + + except Exception as e: + print(f"AGENT [Flight - '{title}'] Error: {e}") + traceback.print_exc() + +# Hotel Agent Coroutine +async def run_hotel_agent(title: str, session: SessionState, thread_id: str): + try: + print(f"AGENT [Hotel - '{title}']: started") + await session.queue.put({"createSurface": {"surfaceId": title, "catalogId": "https://example.com/catalogs/booking_assistant/v1/catalog.json"}}) + + agent = Agent( + name=sanitize_agent_name(title), + model="gemini-3.1-flash-lite", + instruction=f"Book a hotel for: '{title}'. Use the update_ui tool to ask the user to choose a lodging room. Finally use update_ui to show booking confirmation status.\n\n{A2UI_SYSTEM_INSTRUCTION}", + tools=[ + FunctionTool(make_update_ui(title, session)) + ] + ) + + agent_session_id = f"{thread_id}_{sanitize_agent_name(title)}" + runner = InMemoryRunner(agent, app_name="BookingServer") + await runner.session_service.create_session( + app_name="BookingServer", + user_id=thread_id, + session_id=agent_session_id + ) + + async for _ in runner.run_async( + new_message=genai_types.Content(parts=[genai_types.Part.from_text(text=f"Please book: '{title}'")]), + user_id=thread_id, + session_id=agent_session_id + ): + pass + + except Exception as e: + print(f"AGENT [Hotel - '{title}'] Error: {e}") + traceback.print_exc() + +# Museum Agent Coroutine +async def run_museum_agent(title: str, session: SessionState, thread_id: str): + try: + print(f"AGENT [Museum - '{title}']: started") + await session.queue.put({"createSurface": {"surfaceId": title, "catalogId": "https://example.com/catalogs/booking_assistant/v1/catalog.json"}}) + + agent = Agent( + name=sanitize_agent_name(title), + model="gemini-3.1-flash-lite", + instruction=f"Book museum tickets for: '{title}'. Use the update_ui tool to ask the user to choose entry ticket options. Finally use update_ui to show booking confirmation status.\n\n{A2UI_SYSTEM_INSTRUCTION}", + tools=[ + FunctionTool(make_update_ui(title, session)) + ] + ) + + agent_session_id = f"{thread_id}_{sanitize_agent_name(title)}" + runner = InMemoryRunner(agent, app_name="BookingServer") + await runner.session_service.create_session( + app_name="BookingServer", + user_id=thread_id, + session_id=agent_session_id + ) + + async for _ in runner.run_async( + new_message=genai_types.Content(parts=[genai_types.Part.from_text(text=f"Please book: '{title}'")]), + user_id=thread_id, + session_id=agent_session_id + ): + pass + + except Exception as e: + print(f"AGENT [Museum - '{title}'] Error: {e}") + traceback.print_exc() + +# Restaurant Agent Coroutine +async def run_restaurant_agent(title: str, session: SessionState, thread_id: str): + try: + print(f"AGENT [Restaurant - '{title}']: started") + await session.queue.put({"createSurface": {"surfaceId": title, "catalogId": "https://example.com/catalogs/booking_assistant/v1/catalog.json"}}) + + agent = Agent( + name=sanitize_agent_name(title), + model="gemini-3.1-flash-lite", + instruction=f"Book a table reservation for restaurant: '{title}'. Use the update_ui tool to ask the user to choose a table size option. Finally use update_ui to show booking confirmation status.\n\n{A2UI_SYSTEM_INSTRUCTION}", + tools=[ + FunctionTool(make_update_ui(title, session)) + ] + ) + + agent_session_id = f"{thread_id}_{sanitize_agent_name(title)}" + runner = InMemoryRunner(agent, app_name="BookingServer") + await runner.session_service.create_session( + app_name="BookingServer", + user_id=thread_id, + session_id=agent_session_id + ) + + async for _ in runner.run_async( + new_message=genai_types.Content(parts=[genai_types.Part.from_text(text=f"Please book: '{title}'")]), + user_id=thread_id, + session_id=agent_session_id + ): + pass + + except Exception as e: + print(f"AGENT [Restaurant - '{title}'] Error: {e}") + traceback.print_exc() + + +@app.post("/") +async def root(input_data: dict): + thread_id = input_data.get("threadId", "") + run_id = input_data.get("runId", "") + + if thread_id in sessions: + old_session = sessions[thread_id] + print(f"SERVER: Cleaning up old background tasks for sessionId={thread_id}") + for task in old_session.bg_tasks: + if not task.done(): + task.cancel() + + session = SessionState() + sessions[thread_id] = session + + # Extract itinerary events + last_user_msg = None + for msg in reversed(input_data.get("messages", [])): + role = msg.get("role") or msg.get("messageRole") + if role == "user": + last_user_msg = msg + break + + user_text = "" + if last_user_msg: + content = last_user_msg.get("content") + content_parts = last_user_msg.get("contentParts") + if content: + if isinstance(content, list): + for part in content: + if isinstance(part, dict) and part.get("type") == "text": + user_text += part.get("text", "") + elif isinstance(part, str): + user_text += part + elif isinstance(content, str): + user_text = content + elif content_parts: + user_text = "".join((part.get("text", "") if isinstance(part, dict) else part.text) for part in content_parts) + + itinerary_items = [] + try: + data = json.loads(user_text) + itinerary_items = data.get("events", []) + except Exception as e: + print("Failed to parse itinerary JSON:", e) + + async def event_generator(): + print(f"SERVER [threadId={thread_id}]: Starting event generator stream") + # 1. Run Started + yield f"event: RUN_STARTED\ndata: {json.dumps({'type': 'RUN_STARTED', 'threadId': thread_id, 'runId': run_id})}\n\n" + + # 2. Start sub-agents concurrently + tasks = [] + for item in itinerary_items: + title = item.get("title", "Booking") + type_val = item.get("type") + if type_val == "TRANSPORTATION": + tasks.append(run_flight_agent(title, session, thread_id)) + elif type_val == "LODGING" or type_val == "ACCOMMODATION": + tasks.append(run_hotel_agent(title, session, thread_id)) + elif type_val in ["CULTURE", "ACTIVITY"]: + tasks.append(run_museum_agent(title, session, thread_id)) + elif type_val == "FOOD_AND_DRINK": + tasks.append(run_restaurant_agent(title, session, thread_id)) + + # Run sub-agents in background + async def run_all_and_signal(): + try: + if tasks: + await asyncio.gather(*tasks) + print(f"SERVER [threadId={thread_id}]: All sub-agent tasks completed successfully") + except Exception as gather_err: + print(f"SERVER [threadId={thread_id}]: Exception in gather:") + traceback.print_exc() + finally: + await session.queue.put("__DONE__") + + bg_task = asyncio.create_task(run_all_and_signal()) + session.bg_tasks.append(bg_task) + + # 3. Read from session queue and yield to client + while True: + try: + payload = await asyncio.wait_for(session.queue.get(), timeout=15.0) + print(f"SERVER [threadId={thread_id}]: Collected queue payload: {payload}") + if payload == "__DONE__": + print(f"SERVER [threadId={thread_id}]: Collected __DONE__ signal, ending generator stream") + break + + message_id = str(uuid.uuid4()) + + # Emit text message start + yield f"event: TEXT_MESSAGE_START\ndata: {json.dumps({'type': 'TEXT_MESSAGE_START', 'messageId': message_id, 'role': 'assistant'})}\n\n" + + # Emit JSON payload as content + payload_str = json.dumps(payload) + yield f"event: TEXT_MESSAGE_CONTENT\ndata: {json.dumps({'type': 'TEXT_MESSAGE_CONTENT', 'messageId': message_id, 'delta': payload_str})}\n\n" + + # Emit text message end + yield f"event: TEXT_MESSAGE_END\ndata: {json.dumps({'type': 'TEXT_MESSAGE_END', 'messageId': message_id})}\n\n" + except asyncio.TimeoutError: + # Send keep-alive ping + print(f"SERVER [threadId={thread_id}]: Sending keep-alive ping") + yield ": keep-alive\n\n" + + # 4. Run Finished + yield f"event: RUN_FINISHED\ndata: {json.dumps({'type': 'RUN_FINISHED', 'threadId': thread_id, 'runId': run_id, 'outcome': {'type': 'success'}})}\n\n" + print(f"SERVER [threadId={thread_id}]: Generator stream finished") + + return StreamingResponse(event_generator(), media_type="text/event-stream") + + +@app.post("/respond") +async def respond(sessionId: str = Query(...), agentId: str = Query(...), value: str = Query(...)): + print(f"SERVER: Received /respond - sessionId={sessionId}, agentId={agentId}, value={value}") + session = sessions.get(sessionId) + if not session: + raise HTTPException(status_code=404, detail="Session not found") + + fut = session.pause_events.get(agentId) + if fut and not fut.done(): + fut.set_result(value) + return {"status": "success"} + + raise HTTPException(status_code=404, detail="Pause event not found") diff --git a/jetpacker/android/core/flags/src/main/kotlin/com/example/jetpacker/core/flags/FeatureFlags.kt b/jetpacker/android/core/flags/src/main/kotlin/com/example/jetpacker/core/flags/FeatureFlags.kt index fb66dffc..e663db9d 100644 --- a/jetpacker/android/core/flags/src/main/kotlin/com/example/jetpacker/core/flags/FeatureFlags.kt +++ b/jetpacker/android/core/flags/src/main/kotlin/com/example/jetpacker/core/flags/FeatureFlags.kt @@ -31,6 +31,7 @@ object FeatureFlags { const val KEY_ENABLE_MUSEUM_ASSISTANT = "enable_museum_assistant" const val KEY_ENABLE_URL_GROUNDING = "enable_url_grounding" const val KEY_ENABLE_SEARCH_GROUNDING = "enable_search_grounding" + const val KEY_ENABLE_BOOKING_ASSISTANT = "enable_booking_assistant" private var appContext: Context? = null @@ -87,6 +88,11 @@ object FeatureFlags { val ENABLE_SEARCH_GROUNDING: Boolean get() = readFlag(KEY_ENABLE_SEARCH_GROUNDING, true) + + // Booking Assistant (server - ADK) + val ENABLE_BOOKING_ASSISTANT: Boolean + get() = readFlag(KEY_ENABLE_BOOKING_ASSISTANT, true) + private fun readLongFlag(keyName: String, defaultValue: Long): Long { val context = appContext ?: return defaultValue return context diff --git a/jetpacker/android/feature/trip/booking_assistant/build.gradle.kts b/jetpacker/android/feature/trip/booking_assistant/build.gradle.kts new file mode 100644 index 00000000..8faac7b2 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/build.gradle.kts @@ -0,0 +1,84 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +plugins { + alias(libs.plugins.android.library) + alias(libs.plugins.kotlin.compose) + alias(libs.plugins.hilt.android) + alias(libs.plugins.google.devtools.ksp) +} + +android { + namespace = "com.example.jetpacker.feature.booking_assistant" + compileSdk = libs.versions.compileSdk.get().toInt() + defaultConfig { + minSdk = libs.versions.minSdk.get().toInt() + } + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } + buildFeatures { + compose = true + buildConfig = true + } +} + +dependencies { + implementation(platform(libs.androidx.compose.bom)) + + implementation(project(":core:flags")) + implementation(project(":core:ui")) + implementation(project(":data:itinerary")) + implementation(project(":data:trips")) + + implementation(libs.androidx.compose.material3) + implementation(libs.androidx.compose.ui) + implementation(libs.androidx.compose.ui.tooling.preview) + implementation(libs.androidx.core.ktx) + implementation(libs.androidx.hilt.navigation.compose) + implementation(libs.androidx.lifecycle.runtime.compose) + implementation(libs.androidx.lifecycle.viewmodel.compose) + implementation(libs.hilt.android) + "ksp"(libs.hilt.compiler) + + debugImplementation(libs.androidx.compose.ui.tooling) + + implementation(libs.google.adk) + implementation(libs.google.adk.firebase) + "ksp"(libs.google.adk.processor) + implementation(libs.agui.client) + implementation(libs.ktor.client.core) + implementation(libs.ktor.client.okhttp) + implementation(libs.ktor.client.content.negotiation) + implementation(libs.ktor.serialization.kotlinx.json) + implementation(libs.kotlinx.serialization.json) + + implementation("androidx.a2ui:a2ui-engine:1.0.0-SNAPSHOT") + implementation("androidx.a2ui:a2ui-model:1.0.0-SNAPSHOT") + implementation("androidx.a2ui.compose:compose-runtime:1.0.0-SNAPSHOT") + implementation("androidx.a2ui.compose:compose-ui:1.0.0-SNAPSHOT") + + + testImplementation(libs.androidx.core) + testImplementation(libs.androidx.junit) + testImplementation(libs.google.truth) + testImplementation(libs.junit) + testImplementation(libs.kotlinx.coroutines.test) + testImplementation(libs.robolectric) +} + +kotlin { jvmToolchain(21) } diff --git a/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/AndroidA2uiJsonReader.kt b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/AndroidA2uiJsonReader.kt new file mode 100644 index 00000000..4c8c8788 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/AndroidA2uiJsonReader.kt @@ -0,0 +1,58 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.example.jetpacker.feature.booking_assistant + +import android.util.JsonReader +import android.util.JsonToken +import androidx.a2ui.model.processor.A2uiJsonReader +import androidx.a2ui.model.processor.A2uiJsonToken +import java.io.StringReader + +class AndroidA2uiJsonReader(json: String) : A2uiJsonReader { + private val reader = JsonReader(StringReader(json)) + + override fun peek(): A2uiJsonToken { + return when (reader.peek()) { + JsonToken.BEGIN_ARRAY -> A2uiJsonToken.BEGIN_ARRAY + JsonToken.END_ARRAY -> A2uiJsonToken.END_ARRAY + JsonToken.BEGIN_OBJECT -> A2uiJsonToken.BEGIN_OBJECT + JsonToken.END_OBJECT -> A2uiJsonToken.END_OBJECT + JsonToken.NAME -> A2uiJsonToken.NAME + JsonToken.STRING -> A2uiJsonToken.STRING + JsonToken.NUMBER -> A2uiJsonToken.NUMBER + JsonToken.BOOLEAN -> A2uiJsonToken.BOOLEAN + JsonToken.NULL -> A2uiJsonToken.NULL + JsonToken.END_DOCUMENT -> A2uiJsonToken.END_DOCUMENT + null -> A2uiJsonToken.END_DOCUMENT + } + } + + override fun beginObject() = reader.beginObject() + override fun endObject() = reader.endObject() + override fun beginArray() = reader.beginArray() + override fun endArray() = reader.endArray() + override fun hasNext(): Boolean = reader.hasNext() + override fun nextName(): String = reader.nextName() + override fun nextString(): String = reader.nextString() + override fun nextBoolean(): Boolean = reader.nextBoolean() + override fun nextDouble(): Double = reader.nextDouble() + override fun nextInt(): Int = reader.nextInt() + override fun nextLong(): Long = reader.nextLong() + override fun nextNull() = reader.nextNull() + override fun skipValue() = reader.skipValue() + override fun close() = reader.close() +} diff --git a/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantScreen.kt b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantScreen.kt new file mode 100644 index 00000000..02fcf502 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantScreen.kt @@ -0,0 +1,254 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.example.jetpacker.feature.booking_assistant + +import androidx.compose.foundation.background +import androidx.compose.foundation.clickable +import androidx.compose.foundation.layout.* +import androidx.compose.foundation.lazy.LazyColumn +import androidx.compose.foundation.lazy.items +import androidx.compose.foundation.lazy.rememberLazyListState +import androidx.compose.foundation.shape.CircleShape +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.filled.AirplanemodeActive +import androidx.compose.material.icons.filled.ArrowBack +import androidx.compose.material.icons.filled.KeyboardArrowDown +import androidx.compose.material.icons.filled.KeyboardArrowUp +import androidx.compose.material3.* +import androidx.compose.runtime.Composable +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import com.example.jetpacker.core.ui.SekuyaFontFamily + +// A2UI imports +import androidx.a2ui.compose.runtime.observeA2uiComponentState +import androidx.a2ui.compose.runtime.A2uiComponentState +import androidx.a2ui.model.processor.A2uiSurfaceModel +import androidx.a2ui.engine.model.A2uiCoreSurfaceModel +import androidx.a2ui.compose.ui.A2uiCatalog +import androidx.a2ui.compose.ui.A2uiComponent + +import androidx.hilt.navigation.compose.hiltViewModel + +@Composable +fun BookingAssistantScreen( + tripId: String, + modifier: Modifier = Modifier, + contentPadding: PaddingValues = PaddingValues(0.dp), + onBack: () -> Unit = {}, + viewModel: BookingAssistantViewModel = hiltViewModel() +) { + val events by viewModel.events.collectAsState() + val activeSurfaces by viewModel.activeSurfaces.collectAsState() + + LaunchedEffect(tripId) { + viewModel.startBookingChat(tripId) + } + + + Column( + modifier = modifier + .fillMaxSize() + .background(color = MaterialTheme.colorScheme.primary) + .padding(contentPadding) + .padding(16.dp), + horizontalAlignment = Alignment.CenterHorizontally + ) { + // Custom Header Bar + Row( + modifier = Modifier.fillMaxWidth().padding(bottom = 24.dp), + verticalAlignment = Alignment.CenterVertically + ) { + IconButton(onClick = onBack) { + Icon( + imageVector = Icons.Filled.ArrowBack, + contentDescription = "Back", + tint = MaterialTheme.colorScheme.onPrimary + ) + } + Spacer(modifier = Modifier.width(8.dp)) + Text( + text = stringResource(R.string.manage_bookings_title), + fontSize = 20.sp, + fontFamily = SekuyaFontFamily, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onPrimary + ) + } + + if (activeSurfaces.isEmpty()) { + Column( + modifier = Modifier.fillMaxWidth().padding(start = 8.dp, bottom = 24.dp) + ) { + Text( + text = stringResource(R.string.current_itinerary_label), + fontSize = 14.sp, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onPrimary.copy(alpha = 0.8f) + ) + Text( + text = stringResource(R.string.your_next_journey_title), + fontSize = 22.sp, + fontFamily = SekuyaFontFamily, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onPrimary + ) + Spacer(modifier = Modifier.height(8.dp)) + Text( + text = stringResource(R.string.booking_assistant_description), + fontSize = 14.sp, + color = MaterialTheme.colorScheme.onPrimary.copy(alpha = 0.9f) + ) + } + Box( + modifier = Modifier + .fillMaxWidth() + .weight(1f), + contentAlignment = Alignment.Center + ) { + Text( + text = "Connecting to booking server...", + fontSize = 16.sp, + fontWeight = FontWeight.Medium, + color = MaterialTheme.colorScheme.onPrimary.copy(alpha = 0.7f) + ) + } + } + + // Render active booking requirement cards in a unified list + if (activeSurfaces.isNotEmpty()) { + LazyColumn( + modifier = Modifier.fillMaxWidth().weight(1f), + verticalArrangement = Arrangement.spacedBy(16.dp) + ) { + item { + Column( + modifier = Modifier.fillMaxWidth().padding(start = 8.dp, bottom = 8.dp) + ) { + Text( + text = stringResource(R.string.current_itinerary_label), + fontSize = 14.sp, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onPrimary.copy(alpha = 0.8f) + ) + Text( + text = stringResource(R.string.your_next_journey_title), + fontSize = 22.sp, + fontFamily = SekuyaFontFamily, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onPrimary + ) + Spacer(modifier = Modifier.height(8.dp)) + Text( + text = stringResource(R.string.booking_assistant_description), + fontSize = 14.sp, + color = MaterialTheme.colorScheme.onPrimary.copy(alpha = 0.9f) + ) + } + } + items(activeSurfaces) { surfaceModel -> + Card( + modifier = Modifier.fillMaxWidth(), + shape = MaterialTheme.shapes.large, + colors = CardDefaults.cardColors(containerColor = MaterialTheme.colorScheme.surface) + ) { + Column(modifier = Modifier.padding(20.dp)) { + val coreSurface = surfaceModel as? A2uiCoreSurfaceModel + val id = coreSurface?.id ?: "" + + Row( + modifier = Modifier.fillMaxWidth().padding(bottom = 12.dp), + verticalAlignment = Alignment.CenterVertically + ) { + Text( + text = id, + fontSize = 16.sp, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onSurface + ) + } + + BookingAssistantA2uiSurface( + surface = surfaceModel, + modifier = Modifier.fillMaxWidth().wrapContentHeight() + ) + } + } + } + } + } + } +} + +@Composable +fun BookingAssistantA2uiSurface( + surface: A2uiSurfaceModel, + modifier: Modifier = Modifier +) { + val coreSurface = surface as? A2uiCoreSurfaceModel ?: return + val state = observeA2uiComponentState(coreSurface) + + when (state) { + is A2uiComponentState.Loading -> { + Box( + modifier = modifier.fillMaxWidth().height(100.dp), + contentAlignment = Alignment.Center + ) { + CircularProgressIndicator() + } + } + is A2uiComponentState.Error -> { + Text( + text = "Error: ${state.exception.message ?: "Unknown error"}", + color = MaterialTheme.colorScheme.error, + modifier = modifier + ) + } + is A2uiComponentState.Success -> { + val componentModel = state.component + val catalog = bookingAssistantCatalog() + val component = catalog.getComponent(componentModel.type) + + if (component != null) { + with(component) { + componentModel.scope.Content( + properties = componentModel.properties, + modifier = modifier + ) + } + } else { + Text( + text = "Unsupported component: ${componentModel.type}", + modifier = modifier + ) + } + } + } +} diff --git a/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantViewModel.kt b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantViewModel.kt new file mode 100644 index 00000000..0c1213d9 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/BookingAssistantViewModel.kt @@ -0,0 +1,277 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.example.jetpacker.feature.booking_assistant + +import androidx.lifecycle.ViewModel +import androidx.lifecycle.viewModelScope +import java.time.Duration +import com.agui.client.agent.HttpAgent +import com.agui.client.agent.HttpAgentConfig +import com.agui.core.types.BaseEvent +import com.agui.core.types.RunAgentInput +import com.agui.core.types.UserMessage +import com.agui.core.types.TextInputContent +import com.agui.core.types.TextMessageStartEvent +import com.agui.core.types.TextMessageContentEvent +import com.agui.core.types.TextMessageEndEvent +import com.example.jetpacker.data.itinerary.EventDao +import dagger.hilt.android.lifecycle.HiltViewModel +import io.ktor.client.HttpClient +import io.ktor.client.engine.okhttp.OkHttp +import io.ktor.client.plugins.contentnegotiation.ContentNegotiation +import io.ktor.client.plugins.sse.SSE +import io.ktor.client.request.post +import io.ktor.client.request.parameter +import io.ktor.serialization.kotlinx.json.json +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.catch +import kotlinx.coroutines.flow.combine +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.launch +import kotlinx.serialization.encodeToString +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.jsonPrimitive +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.put +import java.util.UUID +import javax.inject.Inject + +// A2UI imports +import androidx.a2ui.model.processor.A2uiSurfaceModel +import androidx.a2ui.compose.ui.A2uiMessageProcessor +import androidx.a2ui.engine.model.A2uiCoreSurfaceModel +import androidx.a2ui.model.protocol.A2uiUserAction +import androidx.a2ui.model.protocol.A2uiCreateSurfaceMessage +import androidx.a2ui.model.protocol.A2uiUpdateComponentsMessage +import androidx.a2ui.model.processor.A2uiJsonMessageParser +import androidx.a2ui.model.processor.A2uiActionInterceptor +import androidx.a2ui.model.protocol.A2uiEventAction +import androidx.a2ui.model.protocol.A2uiFunctionCallAction + +@HiltViewModel +class BookingAssistantViewModel @Inject constructor( + private val eventDao: EventDao +) : ViewModel() { + private val _events = MutableStateFlow>(emptyList()) + val events: StateFlow> = _events.asStateFlow() + + private val _activeSurfaces = MutableStateFlow>(emptyList()) + val activeSurfaces: StateFlow> = _activeSurfaces.asStateFlow() + + private val lastUpdatedSurfaces = MutableStateFlow>(emptyList()) + + private val catalog = bookingAssistantCatalog() + + private val actionInterceptor = object : A2uiActionInterceptor { + override suspend fun onInterceptAction(request: A2uiUserAction): A2uiUserAction? { + handleAction(request) + return request + } + } + + private val messageProcessor = A2uiMessageProcessor( + catalogs = listOf(catalog), + interceptors = listOf(actionInterceptor) + ) + private val parser = A2uiJsonMessageParser { json -> AndroidA2uiJsonReader(json) } + private var currentThreadId: String? = null + + init { + viewModelScope.launch { + messageProcessor.collectMessages() + } + } + + private val agUiJson = Json { + ignoreUnknownKeys = true + } + + private val httpClient = HttpClient(OkHttp) { + engine { + config { + readTimeout(Duration.ZERO) + } + } + install(SSE) + install(ContentNegotiation) { + json(agUiJson) + } + } + + fun startBookingChat(tripId: String) { + viewModelScope.launch { + try { + _events.value = listOf("Loading itinerary events from database...") + val eventsList = eventDao.getEventsForTrip(tripId).first() + + val itineraryJson = buildJsonObject { + put("events", JsonArray(eventsList.map { event -> + buildJsonObject { + put("id", event.id) + put("type", event.type.name) + put("timestamp", event.timestamp) + put("title", event.title) + put("location", event.location) + put("description", event.description ?: "") + put("extraInfo", event.extraInfo ?: "") + } + })) + }.toString() + + _events.value = _events.value + "Connecting to local booking server at http://10.0.2.2:8000..." + + val emptyJsonObject = JsonObject(emptyMap()) + val threadId = UUID.randomUUID().toString() + currentThreadId = threadId + + val config = HttpAgentConfig( + "booking-assistant", // 1: agentId + "Booking Assistant Agent", // 2: description + threadId, // 3: threadId + emptyList(), // 4: initialMessages + emptyJsonObject, // 5: initialState + false, // 6: debug + "http://localhost:8000", // 7: url + emptyMap(), // 8: headers + 30000L, // 9: requestTimeout + 10000L // 10: connectTimeout + ) + val agent = HttpAgent(config, httpClient) + + // Collect active surfaces combined with sorting order + viewModelScope.launch { + messageProcessor.activeSurfaces + .combine(lastUpdatedSurfaces) { surfaces, order -> + surfaces.sortedBy { surface -> + val id = (surface as? A2uiCoreSurfaceModel)?.id ?: "" + val index = order.indexOf(id) + if (index == -1) Int.MAX_VALUE else index + } + } + .collect { sortedSurfaces -> + android.util.Log.d("BookingAssistant", "Active surfaces updated. Count=${sortedSurfaces.size}, Surfaces=${sortedSurfaces.map { (it as? A2uiCoreSurfaceModel)?.id ?: "" }}") + _events.value = _events.value + "Active surfaces updated. Count=${sortedSurfaces.size}" + _activeSurfaces.value = sortedSurfaces + } + } + + + + val runId = UUID.randomUUID().toString() + val userMessage = UserMessage( + id = UUID.randomUUID().toString(), + content = itineraryJson, + name = "user", + contentParts = listOf(TextInputContent(itineraryJson)) + ) + + val input = RunAgentInput( + threadId, + runId, + "", + emptyJsonObject, + listOf(userMessage) + ) + + var currentMessageId: String? = null + val messageBuffer = StringBuilder() + + agent.runAgentObservable(input) + .catch { e -> + _events.value = _events.value + "Connection error: ${e.message}. Make sure the host booking server is running." + } + .collect { event -> + when (event) { + is TextMessageStartEvent -> { + currentMessageId = event.messageId + messageBuffer.setLength(0) + } + is TextMessageContentEvent -> { + if (event.messageId == currentMessageId) { + messageBuffer.append(event.delta) + } + } + is TextMessageEndEvent -> { + if (event.messageId == currentMessageId) { + val fullMessageText = messageBuffer.toString().trim() + if (fullMessageText.startsWith("{") && fullMessageText.endsWith("}")) { + try { + android.util.Log.d("BookingAssistant", "Parsing JSON message: $fullMessageText") + val parsedMsg = parser.parse(fullMessageText) + android.util.Log.d("BookingAssistant", "Successfully parsed JSON message: $parsedMsg") + val createdSurfaceId = when (parsedMsg) { + is A2uiCreateSurfaceMessage -> parsedMsg.surfaceId + else -> null + } + if (createdSurfaceId != null && !lastUpdatedSurfaces.value.contains(createdSurfaceId)) { + lastUpdatedSurfaces.value = lastUpdatedSurfaces.value + createdSurfaceId + } + messageProcessor.processMessage(parsedMsg) + } catch (e: Throwable) { + android.util.Log.e("BookingAssistant", "A2UI parsing/processing error", e) + _events.value = _events.value + "A2UI parsing error: ${e.message}" + } + } else { + android.util.Log.d("BookingAssistant", "Plain text message received: $fullMessageText") + _events.value = _events.value + "[Assistant] $fullMessageText" + } + } + currentMessageId = null + } + else -> {} + } + } + } catch (e: Throwable) { + _events.value = _events.value + "Error: ${e.message}" + } + } + } + + fun handleAction(action: A2uiUserAction) { + android.util.Log.d("BookingAssistant", "handleAction called: action=$action, threadId=$currentThreadId") + val threadId = currentThreadId ?: return + viewModelScope.launch { + try { + val value = when (action) { + is A2uiEventAction -> (action.context["name"] as? String) ?: action.eventName + is A2uiFunctionCallAction -> (action.args["name"] as? String) ?: action.functionName + else -> "" + } + val surfaceId = action.surfaceId + android.util.Log.d("BookingAssistant", "Coroutine started: surfaceId=$surfaceId, value=$value") + _events.value = _events.value + "Sending action response for $surfaceId: $value" + android.util.Log.d("BookingAssistant", "Making HTTP POST request to /respond...") + val response = httpClient.post("http://localhost:8000/respond") { + parameter("sessionId", threadId) + parameter("agentId", surfaceId) + parameter("value", value) + } + android.util.Log.d("BookingAssistant", "HTTP POST request finished: status=${response.status}") + _events.value = _events.value + "Response status: ${response.status}" + } catch (e: Exception) { + _events.value = _events.value + "Error sending response: ${e.message}" + android.util.Log.e("BookingAssistant", "Error inside coroutine: ${e.message}", e) + } + } + } +} diff --git a/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/CustomBookingAssistantCatalog.kt b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/CustomBookingAssistantCatalog.kt new file mode 100644 index 00000000..17cf8ae4 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/main/kotlin/com/example/jetpacker/feature/booking_assistant/CustomBookingAssistantCatalog.kt @@ -0,0 +1,262 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.example.jetpacker.feature.booking_assistant + +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.background +import androidx.compose.foundation.layout.* +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material3.Button +import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import androidx.a2ui.compose.ui.A2uiCatalog +import androidx.a2ui.compose.ui.A2uiComponent +import androidx.a2ui.compose.runtime.A2uiProperty +import androidx.a2ui.compose.runtime.A2uiComponentScope +import androidx.a2ui.compose.runtime.A2uiComponentProperties + +/** + * High-level Interactive Option Picker component. + * Displays a list of custom options as outlined rounded buttons and a confirm button at the bottom. + */ +class InteractiveOptionPickerComponent : A2uiComponent { + companion object { + val promptProp = A2uiProperty.string("prompt") + val optionsProp = A2uiProperty.stringList("options") + val selectedIdxProp = A2uiProperty.number("selectedIdx") + val confirmBtnTextProp = A2uiProperty.string("confirmBtnText") + } + + override val name: String = "InteractiveOptionPicker" + override val description: String = "Renders custom options and confirm action" + override val properties: List> = listOf( + promptProp, + optionsProp, + selectedIdxProp, + confirmBtnTextProp + ) + + @Composable + override fun A2uiComponentScope.Content( + properties: A2uiComponentProperties, + modifier: Modifier + ) { + val prompt = properties[promptProp] ?: "" + val options = properties[optionsProp] ?: emptyList() + val selectedIdx = properties[selectedIdxProp]?.toInt() + val confirmBtnText = properties[confirmBtnTextProp]?.let { + if (it.isEmpty()) "Confirm Selection" else it + } ?: "Confirm Selection" + + Column(modifier = modifier.fillMaxWidth()) { + if (prompt.isNotEmpty()) { + Text( + text = prompt, + fontSize = 14.sp, + fontWeight = FontWeight.SemiBold, + color = MaterialTheme.colorScheme.onSurface.copy(alpha = 0.6f), + modifier = Modifier.padding(bottom = 8.dp) + ) + } + options.forEachIndexed { idx, option -> + Button( + onClick = { + dispatchAction(mapOf("event" to mapOf("name" to "SelectOption_$idx"))) + }, + colors = ButtonDefaults.buttonColors( + containerColor = if (selectedIdx == idx) MaterialTheme.colorScheme.primaryContainer else MaterialTheme.colorScheme.surface, + contentColor = if (selectedIdx == idx) MaterialTheme.colorScheme.onPrimaryContainer else MaterialTheme.colorScheme.onSurface + ), + border = BorderStroke( + width = 1.dp, + color = if (selectedIdx == idx) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.outline + ), + shape = MaterialTheme.shapes.medium, + modifier = Modifier + .fillMaxWidth() + .padding(vertical = 4.dp) + ) { + Text( + text = option, + fontSize = 15.sp, + fontWeight = FontWeight.Medium + ) + } + } + Spacer(modifier = Modifier.height(8.dp)) + Button( + onClick = { + if (selectedIdx != null) { + dispatchAction(mapOf("event" to mapOf("name" to "ConfirmSelection_$selectedIdx"))) + } + }, + enabled = selectedIdx != null, + colors = ButtonDefaults.buttonColors( + containerColor = MaterialTheme.colorScheme.primary, + contentColor = MaterialTheme.colorScheme.onPrimary, + disabledContainerColor = MaterialTheme.colorScheme.onSurface.copy(alpha = 0.12f), + disabledContentColor = MaterialTheme.colorScheme.onSurface.copy(alpha = 0.38f) + ), + shape = MaterialTheme.shapes.extraLarge, + modifier = Modifier + .fillMaxWidth() + .padding(vertical = 8.dp) + ) { + Text( + text = confirmBtnText, + fontSize = 16.sp, + fontWeight = FontWeight.Bold + ) + } + } + } +} + +/** + * High-level Seat Selection grid component. + */ +class SeatSelectionPickerComponent : A2uiComponent { + companion object { + val promptProp = A2uiProperty.string("prompt") + val seatsProp = A2uiProperty.stringList("seats") + val selectedSeatProp = A2uiProperty.string("selectedSeat") + } + + override val name: String = "SeatSelectionPicker" + override val description: String = "Renders seat choices as a clean selection grid" + override val properties: List> = listOf( + promptProp, + seatsProp, + selectedSeatProp + ) + + @Composable + override fun A2uiComponentScope.Content( + properties: A2uiComponentProperties, + modifier: Modifier + ) { + val prompt = properties[promptProp] ?: "" + val seats = properties[seatsProp] ?: emptyList() + val selectedSeat = properties[selectedSeatProp]?.let { + if (it.isEmpty()) null else it + } + + Column(modifier = modifier.fillMaxWidth()) { + if (prompt.isNotEmpty()) { + Text( + text = prompt, + fontSize = 14.sp, + fontWeight = FontWeight.SemiBold, + color = MaterialTheme.colorScheme.onSurface.copy(alpha = 0.6f), + modifier = Modifier.padding(bottom = 8.dp) + ) + } + Column(verticalArrangement = Arrangement.spacedBy(8.dp)) { + seats.chunked(2).forEach { rowSeats -> + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = Arrangement.spacedBy(8.dp) + ) { + rowSeats.forEach { seat -> + Button( + onClick = { + dispatchAction(mapOf("event" to mapOf("name" to seat))) + }, + colors = ButtonDefaults.buttonColors( + containerColor = if (selectedSeat == seat) MaterialTheme.colorScheme.primaryContainer else MaterialTheme.colorScheme.surface, + contentColor = if (selectedSeat == seat) MaterialTheme.colorScheme.onPrimaryContainer else MaterialTheme.colorScheme.onSurface + ), + border = BorderStroke( + width = 1.dp, + color = if (selectedSeat == seat) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.outline + ), + shape = MaterialTheme.shapes.medium, + modifier = Modifier.weight(1f) + ) { + Text( + text = seat, + fontSize = 14.sp, + fontWeight = FontWeight.Bold + ) + } + } + } + } + } + } + } +} + +/** + * High-level Booking Status component. + */ +class BookingStatusComponent : A2uiComponent { + companion object { + val textProp = A2uiProperty.string("text") + } + + override val name: String = "BookingStatus" + override val description: String = "Renders final booking success confirmation" + override val properties: List> = listOf(textProp) + + @Composable + override fun A2uiComponentScope.Content( + properties: A2uiComponentProperties, + modifier: Modifier + ) { + val text = properties[textProp] ?: "" + + Row( + modifier = modifier + .fillMaxWidth() + .background(MaterialTheme.colorScheme.secondaryContainer, shape = MaterialTheme.shapes.medium) + .padding(16.dp), + verticalAlignment = Alignment.CenterVertically + ) { + Text( + text = text, + fontSize = 15.sp, + fontWeight = FontWeight.Bold, + color = MaterialTheme.colorScheme.onSecondaryContainer + ) + } + } +} + +/** + * Builds the Custom Catalog for Booking Assistant. + */ +fun bookingAssistantCatalog(): A2uiCatalog { + return A2uiCatalog( + catalogId = "https://example.com/catalogs/booking_assistant/v1/catalog.json", + components = listOf( + InteractiveOptionPickerComponent(), + SeatSelectionPickerComponent(), + BookingStatusComponent() + ), + functions = emptyList(), + themeSchema = null + ) +} diff --git a/jetpacker/android/feature/trip/booking_assistant/src/main/res/values/strings.xml b/jetpacker/android/feature/trip/booking_assistant/src/main/res/values/strings.xml new file mode 100644 index 00000000..599c5e18 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/main/res/values/strings.xml @@ -0,0 +1,35 @@ + + + + MANAGE BOOKINGS + Current Itinerary • + YOUR NEXT JOURNEY + Review and finalize your travel segments. We\'ve organized your selections for a seamless booking. + + Flights + Hotels + Museums + Restaurants + + Action required + Interactive Booking Widgets + Agent Stream Logs + Press Initialize to start the agent stream… + Initialize Booking Assistant Session + ADK local Booking Assistant + Remaps localhost to 10.0.2.2 for Android emulator. Ensure Ktor server is running on the host machine at port 8000. + diff --git a/jetpacker/android/feature/trip/booking_assistant/src/test/kotlin/com/example/jetpacker/feature/booking_assistant/A2uiCatalogTest.kt b/jetpacker/android/feature/trip/booking_assistant/src/test/kotlin/com/example/jetpacker/feature/booking_assistant/A2uiCatalogTest.kt new file mode 100644 index 00000000..1f4de062 --- /dev/null +++ b/jetpacker/android/feature/trip/booking_assistant/src/test/kotlin/com/example/jetpacker/feature/booking_assistant/A2uiCatalogTest.kt @@ -0,0 +1,77 @@ +package com.example.jetpacker.feature.booking_assistant + +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import com.agui.core.types.RunAgentInput +import com.agui.core.types.UserMessage +import com.agui.core.types.TextInputContent +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import java.util.UUID + +import com.agui.core.types.BaseEvent as AgUiEvent +import com.agui.core.types.RunStartedEvent +import com.agui.core.types.RunFinishedEvent +import com.agui.core.types.RunFinishedSuccessOutcome +import com.agui.core.types.TextMessageStartEvent +import com.agui.core.types.TextMessageContentEvent +import com.agui.core.types.TextMessageEndEvent +import com.agui.core.types.Role + +@RunWith(RobolectricTestRunner::class) +class A2uiCatalogTest { + @Test + fun testPrintJson() { + val agUiJson: Json = try { + val clazz = Class.forName("com.agui.core.types.AgUiJsonKt") + val method = clazz.getMethod("getAgUiJson") + method.invoke(null) as Json + } catch (e: Exception) { + Json { ignoreUnknownKeys = true } + } + + val itineraryContent = "{\"events\":[{\"id\":\"event-1\",\"type\":\"CULTURE\",\"title\":\"Versailles Palace Tour\",\"location\":\"Versailles\"}]}" + + val userMessage = UserMessage( + id = UUID.randomUUID().toString(), + content = itineraryContent, + name = "user", + contentParts = listOf(TextInputContent(itineraryContent)) + ) + + val input = RunAgentInput( + "test-thread-id", + "test-run-id", + "", + JsonObject(emptyMap()), + listOf(userMessage) + ) + + val jsonStr = agUiJson.encodeToString(RunAgentInput.serializer(), input) + println("SERIALIZED_INPUT_JSON: $jsonStr") + } + + @Test + fun testPrintEvents() { + val agUiJson: Json = try { + val clazz = Class.forName("com.agui.core.types.AgUiJsonKt") + val method = clazz.getMethod("getAgUiJson") + method.invoke(null) as Json + } catch (e: Exception) { + Json { ignoreUnknownKeys = true } + } + + val e1 = RunStartedEvent("threadId", "runId", null, null) + val e2 = TextMessageStartEvent("messageId", Role.ASSISTANT, null, null) + val e3 = TextMessageContentEvent("messageId", "delta", null, null) + val e4 = TextMessageEndEvent("messageId", null, null) + val e5 = RunFinishedEvent("threadId", "runId", null, RunFinishedSuccessOutcome, null, null) + + println("RunStartedEvent_JSON: " + agUiJson.encodeToString(AgUiEvent.serializer(), e1)) + println("TextMessageStartEvent_JSON: " + agUiJson.encodeToString(AgUiEvent.serializer(), e2)) + println("TextMessageContentEvent_JSON: " + agUiJson.encodeToString(AgUiEvent.serializer(), e3)) + println("TextMessageEndEvent_JSON: " + agUiJson.encodeToString(AgUiEvent.serializer(), e4)) + println("RunFinishedEvent_JSON: " + agUiJson.encodeToString(AgUiEvent.serializer(), e5)) + } +} diff --git a/jetpacker/android/feature/trip/build.gradle.kts b/jetpacker/android/feature/trip/build.gradle.kts index 6c8b2dfb..02b7b0d6 100644 --- a/jetpacker/android/feature/trip/build.gradle.kts +++ b/jetpacker/android/feature/trip/build.gradle.kts @@ -48,6 +48,7 @@ dependencies { implementation(project(":feature:trip:itinerary")) implementation(project(":feature:trip:expenses")) implementation(project(":feature:trip:voice_notes")) + implementation(project(":feature:trip:booking_assistant")) implementation(libs.androidx.compose.material.icons.core) implementation(libs.androidx.compose.material.icons.extended) diff --git a/jetpacker/android/feature/trip/itinerary/build.gradle.kts b/jetpacker/android/feature/trip/itinerary/build.gradle.kts index d22acd2f..ae810e75 100644 --- a/jetpacker/android/feature/trip/itinerary/build.gradle.kts +++ b/jetpacker/android/feature/trip/itinerary/build.gradle.kts @@ -72,6 +72,8 @@ dependencies { implementation(libs.firebase.auth.ktx) implementation(libs.firebase.firestore) + + screenshotTestImplementation(libs.androidx.compose.ui.tooling) screenshotTestImplementation(libs.screenshot.validation.api) @@ -83,7 +85,7 @@ dependencies { testImplementation(libs.robolectric) } -kotlin { jvmToolchain(17) } +kotlin { jvmToolchain(21) } screenshotTests { imageDifferenceThreshold = 0.05f diff --git a/jetpacker/android/feature/trip/itinerary/src/test/kotlin/com/example/jetpacker/feature/itinerary/ItineraryViewModelTest.kt b/jetpacker/android/feature/trip/itinerary/src/test/kotlin/com/example/jetpacker/feature/itinerary/ItineraryViewModelTest.kt index 2f41c996..79af2641 100644 --- a/jetpacker/android/feature/trip/itinerary/src/test/kotlin/com/example/jetpacker/feature/itinerary/ItineraryViewModelTest.kt +++ b/jetpacker/android/feature/trip/itinerary/src/test/kotlin/com/example/jetpacker/feature/itinerary/ItineraryViewModelTest.kt @@ -54,11 +54,7 @@ import org.junit.Test import org.junit.runner.RunWith import org.robolectric.annotation.Config -@RunWith(AndroidJUnit4::class) -@Config(sdk = [33]) -@OptIn(ExperimentalCoroutinesApi::class) class ItineraryViewModelTest { - private val testDispatcher = StandardTestDispatcher() @Before diff --git a/jetpacker/android/feature/trip/src/main/kotlin/com/example/jetpacker/feature/trip/TripScreen.kt b/jetpacker/android/feature/trip/src/main/kotlin/com/example/jetpacker/feature/trip/TripScreen.kt index d6aadd18..3e367902 100644 --- a/jetpacker/android/feature/trip/src/main/kotlin/com/example/jetpacker/feature/trip/TripScreen.kt +++ b/jetpacker/android/feature/trip/src/main/kotlin/com/example/jetpacker/feature/trip/TripScreen.kt @@ -42,6 +42,7 @@ import androidx.compose.material.icons.Icons import androidx.compose.material.icons.rounded.Add import androidx.compose.material.icons.rounded.Event import androidx.compose.material.icons.rounded.Wallet +import androidx.compose.material.icons.rounded.SmartToy import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Scaffold import androidx.compose.runtime.Composable @@ -63,12 +64,14 @@ import com.example.jetpacker.core.ui.components.JetPackerToolbarAction import com.example.jetpacker.data.itinerary.EventType import com.example.jetpacker.feature.expenses.ManageExpensesScreen import com.example.jetpacker.feature.itinerary.ItineraryScreen +import com.example.jetpacker.feature.booking_assistant.BookingAssistantScreen import com.example.jetpacker.feature.voice_notes.VoiceNotesScreen enum class TripTab { ITINERARY, EXPENSES, VOICE_NOTES, + BOOKING_ASSISTANT, } /** @@ -92,10 +95,12 @@ fun TripScreen( bottomBar = { val showExpenses = FeatureFlags.ENABLE_EXPENSE_MANAGEMENT val showVoiceNotes = FeatureFlags.ENABLE_VOICE_NOTES + val showBookingAssistant = FeatureFlags.ENABLE_BOOKING_ASSISTANT - var visibleCount = 1 + var visibleCount = 1 // itinerary is always visible if (showExpenses) visibleCount++ if (showVoiceNotes) visibleCount++ + if (showBookingAssistant) visibleCount++ if (visibleCount > 1) { AnimatedVisibility( @@ -143,6 +148,14 @@ fun TripScreen( onFabConfigChange = { fabConfig = it }, ) } + + TripTab.BOOKING_ASSISTANT -> { + BookingAssistantScreen( + tripId = tripId, + contentPadding = innerPadding, + onBack = { selectedTab = TripTab.ITINERARY }, + ) + } } } } @@ -157,6 +170,7 @@ fun JetPackerBottomBar( ) { val showExpenses = FeatureFlags.ENABLE_EXPENSE_MANAGEMENT val showVoiceNotes = FeatureFlags.ENABLE_VOICE_NOTES + val showBookingAssistant = FeatureFlags.ENABLE_BOOKING_ASSISTANT LookaheadScope { Row( @@ -170,7 +184,7 @@ fun JetPackerBottomBar( horizontalArrangement = Arrangement.Center, ) { JetPackerToolbar( - modifier = Modifier.widthIn(max = 272.dp).animateBounds(this@LookaheadScope) + modifier = Modifier.widthIn(max = 340.dp).animateBounds(this@LookaheadScope) ) { JetPackerToolbarAction( icon = Icons.Rounded.Event, @@ -195,6 +209,15 @@ fun JetPackerBottomBar( contentDescription = "Voice Notes", ) } + + if (showBookingAssistant) { + JetPackerToolbarAction( + icon = Icons.Rounded.SmartToy, + onClick = { onTabSelected(TripTab.BOOKING_ASSISTANT) }, + selected = selectedTab == TripTab.BOOKING_ASSISTANT, + contentDescription = "Booking Assistant", + ) + } } Spacer(Modifier.width(4.dp)) diff --git a/jetpacker/android/gradle/libs.versions.toml b/jetpacker/android/gradle/libs.versions.toml index f117b439..f95cfcec 100644 --- a/jetpacker/android/gradle/libs.versions.toml +++ b/jetpacker/android/gradle/libs.versions.toml @@ -1,4 +1,7 @@ [versions] +adk = "0.5.0" +agui = "0.4.1" +ktor = "3.0.3" accompanistPermissions = "0.37.3" activityCompose = "1.13.0" agp = "9.2.1" @@ -47,7 +50,7 @@ roomCompiler = "2.7.0" roomKtx = "2.7.0" roomRuntime = "2.7.0" runner = "1.6.2" -targetSdk = "36" +targetSdk = "37" truth = "1.4.2" versionCode = "1" versionName = "1.0" @@ -119,6 +122,18 @@ firebase-appcheck-playintegrity = { group = "com.google.firebase", name = "fireb firebase-appcheck-debug = { group = "com.google.firebase", name = "firebase-appcheck-debug" } androidx-appfunctions = { group = "androidx.appfunctions", name = "appfunctions", version.ref = "appfunctions" } androidx-appfunctions-compiler = { group = "androidx.appfunctions", name = "appfunctions-compiler", version.ref = "appfunctions" } +google-adk = { group = "com.google.adk", name = "google-adk-kotlin-core-android", version.ref = "adk" } +google-adk-processor = { group = "com.google.adk", name = "google-adk-kotlin-processor", version.ref = "adk" } +google-adk-firebase = { group = "com.google.adk", name = "google-adk-kotlin-firebase-android", version.ref = "adk" } +agui-client = { group = "com.ag-ui.community", name = "kotlin-client", version.ref = "agui" } +ktor-server-core = { group = "io.ktor", name = "ktor-server-core-jvm", version.ref = "ktor" } +ktor-server-netty = { group = "io.ktor", name = "ktor-server-netty-jvm", version.ref = "ktor" } +ktor-server-content-negotiation = { group = "io.ktor", name = "ktor-server-content-negotiation-jvm", version.ref = "ktor" } + +ktor-client-core = { group = "io.ktor", name = "ktor-client-core", version.ref = "ktor" } +ktor-client-okhttp = { group = "io.ktor", name = "ktor-client-okhttp", version.ref = "ktor" } +ktor-client-content-negotiation = { group = "io.ktor", name = "ktor-client-content-negotiation", version.ref = "ktor" } +ktor-serialization-kotlinx-json = { group = "io.ktor", name = "ktor-serialization-kotlinx-json", version.ref = "ktor" } [plugins] android-application = { id = "com.android.application", version.ref = "agp" } diff --git a/jetpacker/android/settings.gradle.kts b/jetpacker/android/settings.gradle.kts index b19b5638..134c1318 100644 --- a/jetpacker/android/settings.gradle.kts +++ b/jetpacker/android/settings.gradle.kts @@ -35,6 +35,7 @@ dependencyResolutionManagement { repositories { google() mavenCentral() + maven { url = uri("https://androidx.dev/snapshots/builds/15935338/artifacts/repository") } } } @@ -73,3 +74,4 @@ include(":feature:trip:voice_notes") include(":feature:trip:itinerary:enrichment") include(":feature:appfunctions") +include(":feature:trip:booking_assistant")