Extend the Capabilities of an Android App by Enabling Google Assistant with App Actions

Android users everywhere love using Google Assistant, Google’s AI-powered virtual voice assistant, to help with daily tasks such as scheduling reminders, ordering food, playing music, and even controlling smart home devices. Google Assistant makes it possible for users to take care of simple and even more complex, multi-step tasks, by simply speaking what they want Google Assistant to do. You can add this functionality to an Android app by using App Actions. In this post, we’ll learn step-by-step how to extend an Android app to Google Assistant using App Actions.

What are App Actions?

App Actions are shortcuts that users can trigger with their voice to launch different features of an app. When App Actions are registered in the code, Google Assistant understands that whenever a user says an invocation phrase, it should deep link to a specific feature within the app. For example, if Google Assistant hears the phrase, “Hey Google, tell [App Name] to record that I ran for 30 minutes,” it would open the specified app and record the data as requested. 

To learn how App Actions work under the hood, refer to How App Actions work.

Note: App actions are supported on Android 5 and higher.

What are built-in intents?

Built-in intents (BIIs) trigger tasks that users want to do or information they are looking for, like ordering food (ORDER_MENU_ITEM), starting an exercise (START_EXERCISE), or searching for businesses (GET_LOCAL_BUSINESS).

There are many BIIs ready to use. The good news is that all of the language models are built and maintained by Google, so you don’t need to worry about that part. You can refer to the Built-in intents index to choose the ones that best describe your app’s capabilities.

How to use App Actions to enable Google Assistant on an Android app

For this exercise, I created a simple App called GorillaTaxi that allows users to create and delete a taxi reservation.

 

Step 1: Choose a built-in intent

You must choose the BIIs appropriate for your app features because the BIIs describe which actions users want to perform. In this case, we’ll use:

actions.intent.CREATE_TAXI_RESERVATION: To create a taxi reservation; for example, “Hey Google, Call me a GorillaTaxi to take me to Mountain View.”

actions.intent.CANCEL_TAXI_RESERVATION: To cancel the current scheduled ride; for example, “Hey Google, Cancel my GorillaTaxi ride.”

Step 2: Implement deeplinks

Google Assistant uses deeplinks to fulfill the intent, pass parameters, and redirect users to the app. If you aren’t familiar with deeplinks, refer to the official documentation

Note: For the GorillaTaxi app, I implemented deeplinks by taking advantage of the Navigation Component because this is how navigation across the app is handled. You can also implement deeplinks manually if you prefer.

1. In the navigation graph, define the deeplink tags for both actions (create & reserve).

nav_graph.xml

 <fragment
   android:id="@+id/ReservationsFragment"...>

   <deepLink app:uri="https://actions.gorilla.com/cancel" />

</fragment>

<fragment
   android:id="@+id/ReserveTaxiFragment"...>

   <argument
       android:name="location"
       android:defaultValue="null"
       app:argType="string"
       app:nullable="true" />

   <deepLink app:uri="https://actions.gorilla.com/create?dropoffLocationName={location}" />

</fragment>

2. In the AndroidManifest.xml, add the nav-graph tag to generate the appropriate intent filter.

AndroidManifest.xml

   <activity android:name=".MainActivity"...>
   …

   <nav-graph android:value="@navigation/nav_graph" />
  
</activity>
   
 

3. Test whether the deeplinks are being handled as expected by using the Android Debug Bridge.

./adb shell am start -a android.intent.action.VIEW -d "myLink"
Step 3: Define App Actions

1. Create an actions.xml file in app/src/main/res/xml to define how to resolve each BII, as well as which parameters are needed.

actions.xml


<?xml version="1.0" encoding="utf-8"?>
<actions>
   <action intentName="actions.intent.CREATE_TAXI_RESERVATION">
       <fulfillment urlTemplate="https://actions.gorilla.com/create{?dropoffLocationName}">
           <parameter-mapping urlParameter="dropoffLocationName" intentParameter="taxiReservation.dropoffLocation.name" />
       </fulfillment>
   </action>
   <action intentName="actions.intent.CANCEL_TAXI_RESERVATION">
       <fulfillment urlTemplate="https://actions.gorilla.com/cancel" />
   </action>
</actions>
   
       

2. In the AndroidManifest.xml, reference the actions.xml file.

AndroidManifest.xml

<application>
    ...
    <meta-data
        android:name="com.google.android.actions"
        android:resource="@xml/actions" />
</application>    
Step 4: Test App Actions

The App Actions test tool is an Android Studio plugin that allows developers to test App Actions during development. This tool creates a preview of the App Actions in Google Assistant, which provides information about the actions registered in the actions.xml file for Google Assistant. You want to make sure that it can recognize our App Actions before you release the app.

Install the plugin

1. In the plugins section, search for “App Actions test tool.”

2. Install the plugin and restart Android Studio.

3. Once it is installed, you can find it in Tools > App Actions.

Check setup requirements

Be sure to complete the following steps before proceeding:

1. You must sign in with the same user account in Android Studio, on your test device, and in Google Play console.

2. Generate a signed Bundle / Apk (build variant must be “release”).

3. Upload the signed Bundle to the Google Play console as an internal testing release.

Note: Uploading an APK or Android App Bundle, that contains an actions.xml file will prompt an error because the App Actions feature is currently in developer preview. Before generating the Bundle or APK, you can comment the meta-data tag added in the AndroidManifest.xml

4. On your test device, finish the Assistant setup process.

Create a preview

1. Go to Tools > App Actions Test Tool.

2. Add an Invocation Name and click Create.

Test the app
Once you’ve created a preview, you can trigger App Actions from the test tool or by using Google Assistant. You can see these options in the following examples:


Give users the voice-assistant capabilities they love on your Android app with App Actions

Users love voice assistants like Google Assistant, and trust them to perform increasingly complex and important tasks. For Android developers, the Google Assistant integration should be on the radar because it increases the capabilities of our apps and makes the life of our users easier. Happy users lead to happy business!

References:

https://developers.google.com/assistant/app/reference/built-in-intents

https://developers.google.com/assistant/app/overview

https://developers.google.com/assistant/app/test-tool

https://codelabs.developers.google.com/codelabs/appactions/#0

Enjoy this article? Check out our other blog to see how Gorilla Logic created its own virtual assistant using Amazon Lex and Slack!

 

Andres Matamoros

Andres works as an Android developer at Gorilla Logic. Throughout his career he has built apps for a variety of industries, working on apps for banks, sport fishing, as well as attendance & time tracking. Most of his experience has been with Native Android, and also cross-platform development using Xamarin. In his spare time, he enjoys going to the movies, spending time with his family, and staying active (he considers himself an amateur cyclist!)

Related Articles

Ready to be Unstoppable?

Partner with Gorilla Logic, and you can be.