In the last post we looked at:

• In-App Intent Handling (accessing data directly from our app)
• The new 'visual list' API
• How to interact with Shortcuts using the new Swiftui app model in iOS14
• Outputting custom types

In this post we’re going to continue the same iOS 14 project we started and we'll explore how we can use arrays of custom objects as rich lists in shortcut parameters and how to take data input in a shortcut action and display it in our SwiftUI app.

This project's code is available in full on GitHub.

Let’s get started!

---

Step One

In this tutorial, we'll add a new Shortcuts action to our app which will let the user choose one of the people saved in our app and open their detail page as a modal sheet.

You'll also be able to add a string in a 'note' parameter to demonstrate how you can pass data into your app from Shortcuts.

Firstly, let's add a new intent call ViewPerson to our IntentDefinition file.

We'll add two parameters. The first is person, which will let us pick from the array of people saved in our app (using the visual list API).

Make sure the type is set to Person (this is the ShortcutsPerson type we defined in our intent definition file last tutorial), and that "Options are provided dynamically" is also ticked.

Next, add a note parameter. This is a simple string that will be displayed on the person's detail page when the app opens.

Finally, add a sensible summary to display on the shortcuts action.

Step Two

Now we'll configure our SwiftUI views to react to a change of state.

When the user selects a person in the shortcut action, adds a note and then runs the shortcut, we'll open the app and display a modal containing the detail page of the person the user has selected, including the new note text.

In our PeopleViews.swift, we'll change our PersonDetailView to accept a Binding string for the note and change the layout so that we conditionally show the text if the note text isn't empty.

struct PersonDetailView: View {

    var person: Person
    @Binding var note: String

    var body: some View {
        List {
            HStack {
                Text("Job")
                    .foregroundColor(.secondary)
                Spacer()
                Text(person.job)
            }
            HStack {
                Text("Icon")
                    .foregroundColor(.secondary)
                Spacer()
                Image(systemName: person.iconName)
            }
            if note != "" {
                HStack {
                    Text("Note")
                        .foregroundColor(.secondary)
                    Spacer()
                    Text(note)
                }
            }
        }
        .navigationBarTitle(person.name)
    }
}

In our PeopleListView, we don't need to worry about about the note binding since this this won't be the entry point for showing the modal when triggered from our shortcut action. We can just feed an empty note string using a .constant("")

struct PeopleListView: View {

    var people = peopleArray

    var body: some View {
        NavigationView {
            List(people, id: \.id) { person in
                NavigationLink(
                    destination: PersonDetailView(person: person, note: .constant("")), // Adding in a constant empty string here
                    label: {
                        Label(title: {
                            VStack(alignment: .leading) {
                                Text(person.name)
                                Text(person.job)
                                    .foregroundColor(.secondary)
                                    .font(.caption)
                            }
                        }, icon: {
                            Image(systemName: person.iconName)
                        })
                    })
            }
            .navigationBarTitle("People")
        }

    }
}

---

Step Three

In the ShortcutsExampleiOS14App.swift file, we'll add an optional @State for the person we want to pass to our detail view and set it to nil. We'll also add an empty @State note string.

@State var personToShow: Person? = nil
@State var note: String = ""

We can use a .sheet modifier attached to the PeopleListView() with the @State var personToShow as the data source. Because we're accessing a new modal sheet, we need to wrap it in a NavigationView so that our navigationBarTitle displays correctly

.sheet(item: $personToShow) { person in
    NavigationView {
        PersonDetailView(person: person, note: $note)
    }
}

Once the personToShow value is no longer nil, the modal will display.

This is what the whole file looks like now:

import SwiftUI
import Intents

@main
struct ShortcutsExampleiOS14App: App {
    @UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate
    @State var personToShow: Person? = nil // The person we'll pass in to our detail view
    @State var note: String = "" // The note we'll pass into our detail view

    var body: some Scene {
        WindowGroup {
            PeopleListView()
                .sheet(item: $personToShow) { person in
                    NavigationView {
                        PersonDetailView(person: person, note: $note)
                    }
                }
        }
    }

    // This class lets us respond to intents in an iOS14 SwiftUI app
    class AppDelegate: UIResponder, UIApplicationDelegate {
        func application(_ application: UIApplication, handlerFor intent: INIntent) -> Any? {

            switch intent {
                // If the intent being responded to is GetPeople, call the GetPeople intent handler
                case is GetPeopleIntent:
                    return GetPeopleIntentHandler()
                default:
                    return nil
            }
        }
    }

}

If we try to build and run, you'll notice an error because the .sheet modifier expects the item object to conform to Identifiable.

We can go to People.swift and add conformace to our Person struct.

struct Person: Identifiable {
    var id: Int
    var name: String
    var job: String
    var iconName: String
}

We already have an id parameter so we don't need to do anything else to conform to the Identifiable protocol.

---

Step Four

Our SwiftUI views are now primed to react to a change in state. Let's flesh out our new ViewPerson intent before we explore how to link the two.

Create a new Swift file called ViewPersonIntentHandler.swift.

As before, we'll define our intent handler class conforming to NSObject and ViewPersonIntentHandling:

import Intents

class ViewPersonIntentHandler: NSObject, ViewPersonIntentHandling {

}

Build the project and you'll be prompted to add the neccessary stubs to satisfy the protocol. These are functions to resolve the parameters and handle the intent.

Firstly, we'll resolve the 'note' parameter. We'll simply pass through the user's note string or an empty string if they don't enter anything.

func resolveNote(for intent: ViewPersonIntent, with completion: @escaping (INStringResolutionResult) -> Void) {
    let noteString = intent.note ?? ""
    completion(INStringResolutionResult.success(with: noteString))
}

With the person parameter, we need to pass in a collection of ShortcutsPerson objects for the user to pick from.

We'll map our Person array into an array of ShortcutsPerson and create the expected INObjectCollection.

func providePersonOptionsCollection(for intent: ViewPersonIntent, with completion: @escaping (INObjectCollection<ShortcutsPerson>?, Error?) -> Void) {

    // We'll convert our list of Person in the peopleArray to ShortcutsPeople along with subtitles and images which will be displayed in the shortcuts action's parameter list
    let shortcutsPeople: [ShortcutsPerson] = peopleArray.map { person in
        let shortcutsPerson = ShortcutsPerson(identifier: nil,
                                              display: person.name,
                                              pronunciationHint: nil,
                                              subtitle: person.job,
                                              image: INImage.systemImageNamed(person.iconName))
        return shortcutsPerson
    }

     // Create a collection with the array of ShortcutsPeople.
     let collection = INObjectCollection(items: shortcutsPeople)

     // Call the completion handler, passing the collection.
     completion(collection, nil)
}

We're going to throw an error if the user hasn't selected any person to view in the app. To do this, let's go back to our Intent Definition file and add a validation error to our person parameter.

Back in our intent handler, we'll throw an error if the person parameter is nil, otherwise we'll resolve it.

func resolvePerson(for intent: ViewPersonIntent, with completion: @escaping (ViewPersonPersonResolutionResult) -> Void) {
    // Show an error if the user hasn't selected a person
    guard let person = intent.person else {
        completion(ViewPersonPersonResolutionResult.unsupported(forReason: .noPerson))
        return
    }
    completion(ViewPersonPersonResolutionResult.success(with: person))
}

With both of our parameters validated, let's handle our intent.

We're going to complete with a response code of .continueInApp which, as expected, will launch our app.

func handle(intent: ViewPersonIntent, completion: @escaping (ViewPersonIntentResponse) -> Void) {
    let userActivity = NSUserActivity(activityType: "com.alexhay.example.viewPerson")
    completion(ViewPersonIntentResponse.init(code: .continueInApp, userActivity: userActivity))
}

You can see we're also passing in an NSUserActivity object with a unique ID. Before opening our app, SiriKit will populate its interation property with our intent, as per the documentation.

“If SiriKit determines that it needs to launch your app, either to handle the intent or respond to errors, it launches your app and delivers your user activity object. (Prior to delivery, SiriKit fills the interaction property of the user activity object with the intent and your response.) In your app, use the information in the provided user activity object to take whatever actions are appropriate.”

— SiriKit Documentation

Here's our complete intent handler code:

import Intents
import UIKit

class ViewPersonIntentHandler: NSObject, ViewPersonIntentHandling {
    func resolvePerson(for intent: ViewPersonIntent, with completion: @escaping (ViewPersonPersonResolutionResult) -> Void) {
        // Show an error if the user hasn't selected a person
        guard let person = intent.person else {
            completion(ViewPersonPersonResolutionResult.unsupported(forReason: .noPerson))
            return
        }
        completion(ViewPersonPersonResolutionResult.success(with: person))
    }

    func handle(intent: ViewPersonIntent, completion: @escaping (ViewPersonIntentResponse) -> Void) {
        let userActivity = NSUserActivity(activityType: "com.alexhay.example.viewPerson")
        completion(ViewPersonIntentResponse.init(code: .continueInApp, userActivity: userActivity))
    }

    func resolveNote(for intent: ViewPersonIntent, with completion: @escaping (INStringResolutionResult) -> Void) {
        let noteString = intent.note ?? ""
        completion(INStringResolutionResult.success(with: noteString))
    }

    func providePersonOptionsCollection(for intent: ViewPersonIntent, with completion: @escaping (INObjectCollection<ShortcutsPerson>?, Error?) -> Void) {

        // We'll convert our list of Person in the peopleArray to ShortcutsPeople along with subtitles and images which will be displayed in the shortcuts action's parameter list
        let shortcutsPeople: [ShortcutsPerson] = peopleArray.map { person in
            let shortcutsPerson = ShortcutsPerson(identifier: nil,
                                                  display: person.name,
                                                  pronunciationHint: nil,
                                                  subtitle: person.job,
                                                  image: INImage.systemImageNamed(person.iconName))
            return shortcutsPerson
        }

         // Create a collection with the array of ShortcutsPeople.
         let collection = INObjectCollection(items: shortcutsPeople)

         // Call the completion handler, passing the collection.
         completion(collection, nil)
    }


}

---

Step Five

Before we test our shortcut action, we need to add it as a supported intent in the app target's general tab.

We also need to add a case to AppDelegate class in ShortcutsExampleiOS14App.swift to return the appropriate intent handler.

switch intent {
    // Call the appropriate intent handler
    case is GetPeopleIntent:
        return GetPeopleIntentHandler()
    case is ViewPersonIntent:
        return ViewPersonIntentHandler()

    default:
        return nil
}

---

Step Six

Build and run our app and add our View Person action to a new shortcut.

If we run it without selecting a person first, an error will be shown as expected.

When we tap on the 'Person' parameter, we now see a list of people to choose from, including a subtitle and thumbnail.

When we run the shortcut having selected a person, our app opens but... the detail page doesn't show.

Let's add the final piece of the puzzle to trigger the state change in our SwiftUI views.

Still in ShortcutsExampleiOS14App.swift, in the new App protocol in SwiftUI, we can add a .onContinueUserActivity to our PeopleListView().

.onContinueUserActivity("com.alexhay.example.viewPerson") { userActivity in
    if let intent = userActivity.interaction?.intent as? ViewPersonIntent,
       let person = peopleArray.filter({ $0.name == intent.person?.displayString ?? "" }).first {
        note = intent.note ?? ""
        personToShow = person
    }
}

We're using the same unique identifier for the user activity object as we used in the intent handler.

We check to see if the user activity's interaction property has a populated intent property which type casts to our ViewPersonIntent.

If it does, we use the note and person fields to set our states and the detail modal displays.

Summary

Hopefully this tutorial has been helpful in showing how you can use the Visual List API to pick from a rich list of custom objects (including subtitles and images) inside of shortcut action's parameter fields.

I hope it's also provided some ideas of how you can push data and state changes from a shortcut action into a SwiftUI app.

It's a very basic example but once you see how they can connect, you can be pushing data into your UserDefaults, Core Data stack or injecting state changes into your app's environment.

You could even use some of the techniques detailed in this article to deep link into your app's view hierarchy.

You can find this complete project on GitHub.

Any problems, questions or suggestions, let me know on on Twitter!

---

These are the other posts in the series:

Part 1: Creating a project and adding the first action

Part 2: Exploring parameters: arrays, enums, calculated lists and files

Part 3: In-app intent handling, custom output types, visual list API and using the SwiftUI app protocol

Part 4: Visual Lists in parameters and pushing data from Shortcuts into a SwiftUI view

The complete code for the tutorials is also on GitHub:

Parts 1 & 2 (iOS 13)

Parts 3 & 4 (iOS 14)

In the previous post we looked at adding parameters to a Shortcut action in iOS13 that:

• accepted multiple values
• accepted custom file types
• were enums
• were conditional with a child/parent relationship
• provided a calculated list at run time

Since then, iOS14 has been released along with some exciting new features in Shortcuts, which are detailed in the 'What's New In SiriKit & Shortcuts' video from WWDC 2020.

In this tutorial, we'll be exploring 4 new areas:

• In-App Intent Handling (accessing data directly from our app)
• The new 'visual list' API
• How to interact with Shortcuts using the new Swiftui app model in iOS14
• Outputting custom types

You can find the complete project up to the end of this tutorial on GitHub.

---

Step One

Firstly, let's create a new project.

We'll use the SwiftUI interface and the new SwiftUI App life cycle, which is new on iOS14 and does away with the familiar SceneDelegate & AppDelegate ways of handling the life cycle of your app.

Step Two

Let's create some dummy data. I've added a new file called People which defines a new Person struct and an array of Person populated with 3 entries.

import SwiftUI
import UIKit

struct Person {
    var id: Int
    var name: String
    var job: String
    var iconName: String
}

let peopleArray: [Person] = [
    Person(id: 1, name: "Joe Bloggs", job: "Artist", iconName: "paintpalette"),
    Person(id: 2, name: "Peter Roberts", job: "Taxi Driver", iconName: "car"),
    Person(id: 3, name: "Sarah Love", job: "Author", iconName: "text.book.closed")
]

---

Step Three

Now create a new SwiftUI file called PeopleViews and we'll create two new SwiftUI views.

import SwiftUI

struct PeopleListView: View {

    var people = peopleArray

    var body: some View {
        NavigationView {
            List(people, id: \.id) { person in
                NavigationLink(
                    destination: PersonDetailView(person: person),
                    label: {
                        Label(title: {
                            VStack(alignment: .leading) {
                                Text(person.name)
                                Text(person.job)
                                    .foregroundColor(.secondary)
                                    .font(.caption)
                            }
                        }, icon: {
                            Image(systemName: person.iconName)
                        })
                    })
            }
            .navigationBarTitle("People")
        }

    }
}

struct PersonDetailView: View {

    var person: Person

    var body: some View {
        VStack {
            Image(systemName: person.iconName)
                .font(Font.system(size: 100))

            Text(person.job)
                .font(.callout)
                .navigationBarTitle(person.name)
                .padding(.top)
        }
        .padding()
        Spacer()
    }
}

struct PeopleViews_Previews: PreviewProvider {
    static var previews: some View {
        PeopleListView()
    }
}

PeopleListView shows a list of our dummy people and PersonDetailView is displayed when we tap on one of the names in the list.

---

Step Four

Now go to the ShortcutsExampleiOS14App.swift file and change the default ContentView() to PeopleListView() as the new entry point to our app.

Now build and run and you can see our simple dummy views in action.

Step Five

Now that we have our simple app set up, let's start adding functionality for Shortcuts. First we'll add a new IntentDefinition file. Let's leave it called Intents.

Now click on the '+' and add a new custom type.

We'll call it PersonShortcuts (since we don't want this type to clash with the existing Person type, which we declared earlier).

Change the display name to "Person" and add two properties, a job string and an image file.

Step Six

Let's create our first Shortcuts action.

Hit '+' again and this time add a new Intent.

This action will pull all the people stored in our app and return them as a custom output type in the Shortcuts app.

For this example we don't need to add any parameters, just give it a title and description, deselect Intent Is Eligibile For Siri Suggestions and add a suitable summary that will be displayed on the shortcuts action.

Step Seven

Now lets configure our intent's response - what will be output from the action in the Shortcuts app.

Add a result property with a display name of "People".

Set the type to the ShortcutsPerson type we added earlier (it will just display as "Person" in the drop-down list).

Make sure supports multiple values is checked, since we will be outputting multiple people from the action.

Select this property to be output and leave the success dialog options blank, since we don't want to speak or display the results in the interface, we just want them to be passed to the next shortcuts action.

Now let's add an error string property.

We'll add dialog options since we want to throw up an error message if something goes wrong trying to retrieve the people from our app.

Step Eight

Add a new swift file called GetPeopleIntentHandler and define a new class which conforms to the NSObject and GetPeopleIntentHandling protocols.

Try and build the app and it'll prompt you to add the stub needed to conform to the protocol.

Now let's write the code to handle our intent.

We'll:

• Loop through our person array and create a new ShortcutsPerson for each one
• Create an image from the person's iconName and add it to the ShortcutsPerson. This is the Visual List API added in iOS14, which will display that image next to each item in Shortcuts lists
• Handle any errors that may occur
• Form our response and return the new array of ShortcutsPerson objects

import Intents
import UIKit

class GetPeopleIntentHandler: NSObject, GetPeopleIntentHandling {
    func handle(intent: GetPeopleIntent, completion: @escaping (GetPeopleIntentResponse) -> Void) {
        // Define an empty array of our custom ShortcutsPerson type
        var resultArray = [ShortcutsPerson]()
        // Loop through our people array and create new ShortcutsPerson objects which are added to our result array
        for person in peopleArray {
            // The new 'Visual List' API accepts an optional INImage thumbnail to display next to the display name in Shortcuts lists. Here, we're creating one from the SFSymbol names stored in our people array
            let thumbnail = INImage.systemImageNamed(person.iconName)
            let resultPerson = ShortcutsPerson(identifier: nil, display: person.name, subtitle: person.job, image: thumbnail)
            // Populate our ShortcutsPerson's properties
            resultPerson.job = person.job
            if let imageData = UIImage(systemName: person.iconName)?.jpegData(compressionQuality: 1.0) {
                resultPerson.image = INFile(data: imageData, filename: "\(person.name).jpg", typeIdentifier: "public.jpeg")
            } else {
                // We'll display an error if we can't create a UIImage from the person's iconName
                let errorResponse = GetPeopleIntentResponse.failure(error: "Couldn't create the user image for \(person.name). \"\(person.iconName)\" may not be a valid SFSymbol Name")
                completion(errorResponse)
                return
            }
            resultArray.append(resultPerson)
        }
        // We define a response marked as successful with the success code and add our result array to it
        let response = GetPeopleIntentResponse.init(code: .success, userActivity: nil)
        response.result = resultArray
        completion(response)
    }
}

---

In-App Intent Handling

Now that we have our intent and intent handler, we need to make sure it's called when our action is run in the Shortcuts app.

Notice that unlike the previous tutorials, we haven't had to add an Intents Extension target to our app.

This is because we'll be using In-App Intent Handling, which was added in iOS14.

When the shortcuts action is run, instead of firing up just the intents extension part of the app (which is highly memory-constrained), we can now access the entirety of our app.

It is launched invisibily in the background, our intent handler fires and the result is output in the Shortcuts app.

One benefit of this is that it's much easier to access data in our app. We don't have to store our CoreData stack or UserDefaults in an app group so that both the app and intents extension targets can both access the same data.

It also means that the memory limitations of handling an intent in an intents extension are negated. We can now perform memory-intensive operations like applying filters to full-size images or performing machine learning tasks on files passed through a Shortcuts action. Previously, this would've caused a crash if the memory usage exceeded something like 30mb.

One caveat to in-app intent handling is that you have exactly 10 seconds to complete out of the intent handler before Shortcuts throws a time-out error.

This 10 seconds also includes the time to fire up the app in the background so it wouldn't be appropriate a lengthy task like encoding video, for example.

You can learn more about this in the Empower Your Intents video from WWDC 2020.

---

Step Nine

There are two steps to supporting in-app intent handling.

Firstly, we need to add our new intent to the supported intents list in our app's target.

Secondly we need to respond to the GetPeople intent being called from the Shortcuts action using our intent handler.

To do this in a pure SwiftUI app, we can define a new class which conforms to the UIApplicationDelegate protocol.

You can read more about this here.

Change the code in ShortcutsExampleiOS14App.swift to look like this:

import SwiftUI
import Intents

@main
struct ShortcutsExampleiOS14App: App {
    @UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate

    var body: some Scene {
        WindowGroup {
            PeopleListView()
        }
    }

    // This class lets us respond to intents in an iOS14 SwiftUI app
    class AppDelegate: UIResponder, UIApplicationDelegate {
        func application(_ application: UIApplication, handlerFor intent: INIntent) -> Any? {

            switch intent {
                // If the intent being responded to is GetPeople, call the GetPeople intent handler
                case is GetPeopleIntent:
                    return GetPeopleIntentHandler()
                default:
                    return nil
            }
        }
    }

}

---

Step Ten

In the Shortcuts app, add the Get People action to a new shortcut and run it. You'll see the three ShortcutsPerson objects are returned (as names).

If you add a Choose From List action after the Get People action, you'll see our new visual list being displayed, including thumbnail images and subtitles.

Change one of the iconNames in our peopleArray to an invalid SFSymbol name:

let peopleArray: [Person] = [
    Person(id: 1, name: "Joe Bloggs", job: "Artist", iconName: "paintpalette"),
    Person(id: 2, name: "Peter Roberts", job: "Taxi Driver", iconName: "notASymbol"),
    Person(id: 3, name: "Sarah Love", job: "Author", iconName: "text.book.closed")
]

Run the shortcut again, we'll see the expected error thrown.

Summary

In this tutorial, we've learned about in-app intent handling, how to output custom types from our Shortcut action and how to add images and subtitles to be displayed in lists. We've also seen how it's posssible to handle intents from the new SwiftUI app protocol on iOS14.

In the next tutorial, we'll be continuing this project to show how we can return data from a parameter in our Shortcuts action, back into our app for display.

---

These are the other posts in the series:

Part 1: Creating a project and adding the first action

Part 2: Exploring parameters: arrays, enums, calculated lists and files

Part 3: In-app intent handling, custom output types, visual list API and using the SwiftUI app protocol

Part 4: Visual Lists in parameters and pushing data from Shortcuts into a SwiftUI view

The complete code for the tutorials is also on GitHub:

Parts 1 & 2 (iOS 13)

Parts 3 & 4 (iOS 14)