Facebook debuts Portal and Portal+ with AI-enhanced Messenger video calls and Amazon’s Alexa

Facebook today unveiled its first-ever consumer electronic device built from the ground up. Portal and Portal+ can display photos and videos posted by friends on Facebook and are made for AI-enhanced video calls with Facebook Messenger, an app with 1.3 billion monthly active users. The devices will be able to speak with Amazon’s Alexa and stream music from Spotify, Pandora, and iHeartRadio.
Portal can do group video chat with up to seven people using the Messenger app on smartphones and tablets, as well as other Portal devices. Calls will also include ways to read stories using augmented reality and a platform for third-party developers and partners to deploy AR experiences for kids and adults.
Facebook will sell the 15.6-inch Portal+ for $349 and the 10-inch Portal for $199. Preorders begin today and Portal devices are expected to hit store shelves in early November.
Both Portal devices are imbued with a series of AI-driven services, including Smart Camera for video calls that zoom to follow a person and Smart Sound to modulate the volume of call participants so each one comes through at a good volume. Spotlight mode can also be used to make the camera follow a single person in a video call.
Work on Portal began two years ago, according to Portal team lead and VP Rafa Camargo, and required building new software experiences, new AI technology investments, and the creation of a new team within the company.
The Portal line of devices by Facebook are meant to bring people closer together in a way that feels natural and without the friction of video calls on laptops and smartphones.
The Portal team now operates under Facebook’s AR/VR division. A company spokesperson declined to comment about the size of the Portal team.
Portal was created by Building 8, the internal name of the team building consumer hardware at Facebook that‘s known for undertaking experimental skunkworks projects. Building 8 has reportedly worked on projects ranging from augmented reality to drones to projects involving feeling words on your skin and typing with your mind.
The Portal team will continue to make home hardware by Facebook beyond the two devices introduced today, Camargo told VentureBeat in an interview.
“I think the fundamental value that we want to bring is really that of depth of connection. It’s really making sure that in our platforms you’re getting the best way to feel emotionally connected and remain emotionally connected to the people you care the most about. It’s less the breadth of all your friends on Facebook and more those you really care about,” he said.
People-tracking capability in Smart Camera does not include facial recognition software, and at launch Portal will not include any form of facial recognition, as had been previously reported. Smart Camera and Smart Sound AI will deploy locally on the device.
Portal will have to compete with a smattering of similar devices from Apple and Samsung, as well as Google’s Home and Amazon’s Echo line of smart speakers. Across the gambit, each offers hands-free access to music, calls, essential information like the weather and your calendar, and a growing range of services from an AI assistant.
Facebook might be considered late to the smart speaker market, but it’s right on time for the battle brewing between large tech companies eager to ship hands-free speakers with a screen.
Portal most closely resembles the second-generation Amazon Echo Show, which made its debut last month and also has a 10-inch screen and makes video calls. It also resembles similarly sized Smart Displays from JBL and Lenovo with Google Assistant inside. Google is widely expected to debut the Pixel 3 with a charging dock that makes it easy to speak with Google Assistant, as well as a smart speaker with a screen reportedly named Google Home Hub.
With Messenger, Portal launches with the largest chat app user base on the planet. As of September 2017, 1.3 billion people used Messenger at least once a month.
That’s far beyond competitors like Google Assistant, which makes phone calls with Duo, and video calls with the Alexa app. Skype calls for Echo Show are coming soon.
To enrich experiences on and off video calls, Facebook will open a Portal App Store and introduce a software development kit that lets partners and third-party developers use Portal to create augmented reality experiences for kids and adults that can be used during video calls.
As head of the Portal platform, Camargo will oversee the creation of third-party experiences for video calls and the Portal App Store.
A Portal app SDK will be introduced, while developers will use the Spark AR platform (formerly Camera Effects) to share in-call AR experiences.
Among the apps users should expect to see at launch are those from Food Network, Newsy, and previously mentioned music streaming services. Facebook Watch will also have a standalone app. Notably missing: IGTV, Instagram’s video service that launched earlier this year will not be included among available apps at launch, a company spokesperson told VentureBeat in an email.

What Alexa and Facebook working together means

Working with Amazon means Portal will launch with access to Hulu and NBC as part of a deal Amazon rolled out last month when it debuted nearly a dozen new devices with Amazon’s Alexa inside.
Portal will not support Amazon Prime Video or DVR from the recently introduced Fire TV Recast.
Alexa inside the Portal means you can schedule a timer, answer factual questions, control smart home devices, and gain access to more than 50,000 Alexa skills. It also means Portal will incorporate elements of the Alexa Processing Language (APL) for visual hands-free content. APL governs the visual experience with Portal when speaking with Alexa, so if you ask a Portal device “Alexa, what’s the weather?” the forecast you see will be made with APL, as well as shopping lists, calendar events, and other information.

Above: Portal+

The voice you hear when you speak with a Portal device sounds female and was synthetically made to avoid the impression you’re speaking with a human. No male voice will be available when Portal comes out next month, Camargo said.
At launch, Facebook does not plan to make Portal an assistant along the lines of Siri or Cortana or Samsung’s Bixby. Conversational AI is used to power Portal speech recognition and the speech-to-text conversion required to turn voice commands into actions, but those commands will be focused on hands-free video calls and launching Portal apps.
That could change in the future.
Facebook Messenger is reportedly getting a deeper integration with intelligent assistant M, which today makes suggestions based on words used in a conversation. But Android code for the app suggests that an assistant that can dictate your messages and do things like set reminders could be on the way.
In line with what Amazon and Microsoft have previously stated, Facebook says it believes in a multi-assistant world, and it adds that this could someday include a Facebook AI assistant.
“The way we envision the future for these kind of home platforms is that the assistant is not the platform, it’s really an application,” Camargo said. “We see a future where there will be multiple assistants, and you will use them for what they are good for and what they offer. And some of them may be very broadly generic, like Alexa or Google, and others might be specialized. So it is a possibility for us — that if we find it useful at some point, we do not just the basic voice control but may add some assistant functionality. But it would probably be around helping you connect with people, which is where I think we add a lot of value.”
The extent to which M and Portal will work together has not yet been made clear, but in 2015 Facebook created a trial assistant service for intelligent assistant M, though the closed beta came to an end in January. At the time, users were told learnings from the program would go toward future projects. Some of that team involved with that project played a role in the creation of Portal, Camargo said.

Privacy

With all of the devices from the biggest tech companies on the planet to choose from, consumers have to decide which ones to trust. In the case of Portal, the devices are coming to market after a rash of incidents in the past year have weakened people‘s confidence in Facebook.
Less than two weeks ago, Facebook told the world a hacker had gained access to 50 million user accounts. Portal was reportedly supposed to arrive in April but was delayed reportedly due to fallout from Facebook’s dealings with Cambridge Analytica, a now defunct firm closely tied with the Trump presidential campaign that misused the data of 87 million users.
To reassure users, Portal video calls are encrypted and recording during calls is not allowed. AI features like Smart Camera and Smart Sound are deployed locally on the device.
And Portal will not do things like Live videos, nor will it not take pictures or screenshots.
“We made a conscious choice that, at least at launch, we’re not going to have any recording, to give clear peace of mind that nothing that you are doing in a call — which is the only time the camera is on — gets recorded,” Camargo said.
Portal devices do record exchanges with users after they hear the “Hey Portal” wake word.
Recording exchanges between a user and an AI assistant is in line with practices by Microsoft’s Cortana, Amazon’s Alexa, and Google Assistant. The recordings are used to improve far-field voice recognition and speech recognition in order to better understand spoken queries and commands.
Users will be able to delete recordings using the Facebook app.
A feature unique to Portal means that if both people on a call are using Portal they can listen to music from Spotify together during calls. Both participants can turn volume up or down without changing settings for the other person on the call.
Like Echo and Home speakers, Portal will allow you to mute the microphone, and the device also comes with a plastic cap that can be placed on top of the camera.

There’s not much Facebook in this Facebook device

Exactly how the makers of devices like Portal define success may depend on the desired shift in user behavior. Despite Facebook’s immense knowledge about its social network of billions of people, Portal will not launch with much of that in mind.
“The way we‘re going to measure success is really people hanging out together, is engagement. And then as we do surveys for users, it’s not just ‘You’re hanging out more‘, which is really the great indicator, but ‘Are you feeling emotionally closer to those people?‘ Are we achieving that?,” Camargo said.
Since the initial focus will be on video calls, don’t expect Portal to surf the latest photos of your friends on Instagram.

Above: Portal

Don’t expect Stories either, the format for 24-hour content bits that barely existed among Facebook’s family of apps a year ago. Today, Facebook and Messenger Stories have a combined 300 million daily active users, Facebook announced last month, while WhatsApp Status has 450 million daily active users.
Likewise, there will be no place to peruse your News Feed on Portal, no like or comment notifications, and no alerts about activity in groups or Facebook Marketplace — not even Facebook Messenger, Instagram, or WhatsApp message notifications.
Beyond restrictions on photos or video recording, the Superframe for content that appears when you aren’t making video calls intentionally requires a user to opt in to choose, for example, which friend’s birthday they want hear about.
Mixed in among images and videos on the Portal screen, the UI will flash reminders when close friends are online. This is similar to the way the Messenger app today lists friends online at the top of iOS and Android apps or periodically sends notifications to tell you when a friend is active online.
To make the mobile to Portal call switch easy, when a Portal user picks up their phone and opens the Messenger app, a button will appear asking if they want to transfer the call to their phone.

Facebook’s future

The introduction of Portal comes as Facebook continues to advance its mission statement, redefined in 2017, to bring the world closer together. It’s also consistent with a series of changes to the Facebook News Feed algorithm made in early 2018 that put less emphasis on Pages and more emphasis on posts from people.
The launch of Portal also seems to be in keeping with statements made by CEO Mark Zuckerberg and COO Sheryl Sandberg that Facebook plans to make its messaging apps more profitable parts of the Facebook business.
For each of the company‘s messaging apps, an emphasis has been placed on connecting businesses with customers for years now, and Portal could become part of that plan.
Facebook first brought advertising to the Messenger home page in summer 2017, while the Messenger bot platform, launched in 2016, now has more than 300,000 bots. Meanwhile, Facebook Messenger surpassed 10 billion messages a month between businesses and customers last month.
The standalone WhatsApp Business app was introduced in January, and the Business API launched in August to facilitate automated messaging with customers for large businesses.
Though Portal begins with limited use cases and partners, the device could be used to increase sharing opportunities for the Facebook family of apps or to focus on delivering communications among close friends.
Messenger is one of the most used apps on Earth, but waiting in the wings for Portal are WhatsApp, an app that also claims 1.3 billion monthly active users, and Instagram, which has 800 million monthly active users.
Portal could also help Facebook shift in several directions, going beyond social media to help people take more actions in the real world — like participating in local events, local services like food delivery, or the Facebook dating service.
Source: VentureBeat
To Read Our Daily News Updates, Please visit Inventiva or Subscribe Our Newsletter & Push.