Google’s Duplex is rolling out to Pixel owners — here’s how it works

Google Duplex-powered calls have finally arrived. Sort of. For some people.
A spokesperson confirmed to VentureBeat that Duplex — Google’s artificially intelligent chat agent that can arrange appointments over the phone — has expanded from a “set of trusted tester users” earlier this year to a “small group” of Google Pixel phone owners, who can now use Duplex via the Google Assistant to secure restaurant reservations in “select cities.”
Presumably, “select cities” refers to the previously announced pilot metros of New York, Atlanta, Phoenix, and San Francisco. If you’re not in the “trusted tester” group, you’ll likely get an error along the lines of “Sorry, I can’t call to make reservations for you yet, but here’s their phone number: [10 digits].”
To be clear, it’s not quite the Duplex experience Google demoed at its I/O 2018 developers conference in May — Google Assistant isn’t booking haircut appointments just yet, and it can only place calls in English. But importantly, it’s no longer limited to businesses with which Google has explicitly partnered.
“We’re currently ramping up the ability to book restaurant reservations through the Google Assistant over the phone using Duplex technology,” the spokesperson told VentureBeat. “To help deliver a good experience to Pixel users and to businesses, we’re starting with a slow rollout … and will expand to more Pixel users as we continue to ramp up.”
Google isn’t provisioning Duplex access to members of the press — we asked. But as luck would have it, our Pixel 3 and a Pixel 3 XL review units made it into the initial wave. So naturally, we put Duplex to the test.

Booking a table with Duplex

The most reliable way to book a table using the Google Assistant is by saying “Hey Google, make a restaurant reservation” or “Hey Google, reserve a table,” optionally appended with “in [a neighborhood or borough].” For example: “Make a restaurant reservation in Manhattan.”
Those utterances spring up a list of options that can be refined by cuisine (e.g., Thai, Chinese, barbecue, American). Selecting a restaurant — either by saying its name or by tapping on it — brings you to the next step in the Duplex-powered reservation flow. Alternatively, you can look up a specific restaurant first, and if the option shows up, tap “Request a table there.”

Above: If you’re lucky, there will be an option to “request a table.”

Duplex doesn’t yet work with every restaurant. For some spots we tried to reserve — even those with public phone numbers — Google Assistant threw this error message: “Unfortunately, that restaurant only takes reservations online. So you’ll have to book with them directly.” In other cases, it told us that it “[couldn’t] make reservations at [the] particular restaurant.”
Google has previously said that businesses will be able to opt out from receiving calls by toggling a setting in their Google My Business account. That might have something to do with the roadblocks we encountered, but it just as easily might be early day jitters. We’ve reached out to Google for clarification.
For restaurants that haven’t been delisted, kicking off a reservation is a relatively straightforward process. Google Assistant first prompts you to specify the size of your party, suggesting sizes like two, three, or four people. The maximum is 10 — exceed it, and the Assistant curtly cuts off the exchange with a message indicating it can’t make reservations for large parties yet.

After you’ve communicated the size of your table, confirming a date comes next. Google Assistant suggests up to seven days out, but it will just as readily facilitate a request for several weeks — or even months — in advance. Years are a different story — Google Assistant foiled our attempt to book a table in 2021.
Once you’ve settled on a date, you’ll be asked to supply a preferred time within the restaurant’s hours of operation, in addition to a backup time window (for instance, 6:00 p.m to 7:00 p.m.) in the event the requested time is not available.

Above: The Google Assistant won’t attempt to make a reservation during store hours.

At this stage, Google Assistant asks for a booking phone number, which by default is your phone number. You can supply a new one, or alternatively tap the Manage Phone Number button at the bottom of the screen to pull up the Phone settings page on the Google Account dashboard. It’s populated with numbers you’ve provided and verified for Google services like Hangouts and Google Voice, which you can edit and delete at your leisure.
Finally, after you’ve given a phone number, Google Assistant asks you to confirm the details — the place, date, and time. Then it’s off to the races.

Managing Duplex reservations

It’s at this final screen where you’ll find the My Reservations button. Select it, and it will surface an inline list of reservations automatically imported from Gmail, manually added to Google Calendar, or placed with Duplex. Duplex-facilitated reservations that haven’t been confirmed bear a blue “Pending” tag underneath the name of the restaurant and a View details link.

Select that link, and the Google Assistant serves a webpage with a confirmation number and a chronological list of attempts that have been made to book your reservation; a reservation summary with the date, time, party size, restaurant name, and restaurant address; and the name and phone number associated with the reservation.

What if you have to cancel a reservation? Easy — tap the Cancel button at the bottom of the screen, and Google Assistant places a call to inform the restaurant. Perhaps anticipating abuse, Google has imposed a hard limit of one cancellation per restaurant per day. If you cancel a reservation somewhere, you won’t be able to rebook for 12 to 24 hours. And if you cancel too many reservations in a row, you’ll be temporarily prevented from using Duplex altogether.

Above: The summary screen confirming the reservation.

One interesting item of note: Duplex is definitely a Pixel-only affair for now. We tried to get the Google Assistant to place a reservation from a Google Smart Display (the LG Xboom WK9), but got this response: “I’m sorry, I can’t make reservations … from this device yet.” We might be reading too much into it, but that reply suggests that Smart Displays will eventually be able to tap Duplex for reservations and appointments. Presumably, Google’s not rushing into this.

The Duplex call experience

Booking restaurants with a robot would be fraught with peril, you’d think, considering the number of things that might go wrong. What if the human on the other end has a thick accent that throws off Duplex’s speech recognition? What if they ask an obscure question the system hasn’t been trained to handle? Or what if the restaurant’s listed number is no longer in service?
Google revealed earlier this summer that Duplex is able to complete four out of five fully automated tasks. That sounds about right — every time Duplex allowed us to make a restaurant reservation, it completed the task successfully.
One restaurant, Cafe Prague in San Francisco, even let us film the experience from their point of view (apologies for the portrait view):
Notice that Duplex begins the conversation by saying “Hi, I’m calling to make a reservation for a client. I’m calling from Google, so the call may be recorded.”
Google received a ton of criticism after its initial Duplex demo in May — many were not amused that Google Assistant mimicked a human so well. In June, the company promised that Google Assistant using Duplex would first introduce itself.
Part of the reason Duplex sounds so natural is because it taps Google’s sophisticated WaveNet audio processing neural network, and because it intelligently inserts “speech disfluencies” — the “ums” and “ahs” people make involuntarily in the course of conversation. They come from a branch of linguistics known as pragmatics, which deals with language in use and the contexts in which it is used — including such things as taking turns in conversation, text organization, and presupposition.
Vice president of engineering for the Google Assistant Scott Huffman revealed in interviews this summer that the disfluencies turned out to be the key to advancing talks in tests of Duplex. Without them, he said, people were more likely to hang up as the exchange starts to feel overly artificial.
They have another, more practical purpose: buying time. When Duplex is confronted with an uncertain response from a conversation partner after, say, requesting a table for three, it’ll reiterate with a disfluency — “ah, for three.”
Of course, restaurant owners don’t have to speak with the Google Assistant if they decide they’d rather not. At the beginning of exchanges, Duplex makes clear that the call is automated. In states with two-party consent laws (California, Connecticut, Florida, Illinois, Maryland, Massachusetts, Montana, New Hampshire, Pennsylvania, and Washington), it informs the person on the other end that they’re being recorded. If they respond with “I don’t want to be recorded” or some variation of the phrase, the call is handed off to a human operator on an unrecorded line. (Those operators also annotate the call transcripts used to train Duplex’s algorithms.)
Those precautions are unlikely to put to rest the ethical and legal concerns raised by Duplex’s debut. But as Google steadily marches toward its goal of launching Duplex in new U.S. cities in states, they’ll be a small comfort for businesses that opt to field its autonomous phone calls.
Source: VentureBeat
To Read Our Daily News Updates, Please Visit Inventiva Or Subscribe Our Newsletter & Push.