City
Epaper

FB developing AI that can see, hear, remember whatever you do

By IANS | Published: October 15, 2021 11:42 AM

New Delhi, Oct 15 Facebook is developing a new artificial intelligence (AI)-based system that can analyse your lives ...

Open in App

New Delhi, Oct 15 Facebook is developing a new artificial intelligence (AI)-based system that can analyse your lives using first-person videos recording what they see, do and hear in order to help you with daily tasks.

Imagine your AR device displaying exactly how to hold the sticks during a drum lesson, guiding you through a recipe, helping you find your lost keys, or recalling memories as holograms that come to life in front of you.

To realise this new AI potential, Facebook AI has announced 'Ego4D' a long-term project aimed at solving research challenges in 'egocentric perception' (the perception of direction or position of oneself based on visual information).

"We brought together a consortium of 13 universities and labs across nine countries, who collected more than 2,200 hours of first-person video in the wild, featuring over 700 participants going about their daily lives," the social network said in a statement.

This dramatically increases the scale of egocentric data publicly available to the research community by an order of magnitude, more than 20x greater than any other data set in terms of hours of footage.

"Next-generation AI systems will need to learn from an entirely different kind of data videos that show the world from the center of the action, rather than the sidelines," said Kristen Grauman, lead research scientist at Facebook.

In collaboration with the consortium and Facebook Reality Labs Research (FRL Research), Facebook AI has developed five benchmark challenges centred on first-person visual experience that will spur advancements toward real-world applications for future AI assistants.

Ego4D's five benchmarks are episodic memory, forecasting, hand and object manipulation, audio-visual 'diarization' and social interaction.

"These benchmarks will catalyse research on the building blocks necessary to develop smarter AI assistants that can understand and interact not just in the real world but also in the metaverse, where physical reality, AR, and VR all come together in a single space," Facebook elaborated.

The data sets will be publicly available in November this year for researchers who sign Ego4D's data use agreement.

As a supplement to this work, researchers from FRL used Vuzix Blade smart glasses to collect an additional 400 hours of first-person video data in staged environments. This data will be released as well.

While it's easy for people to relate to both first- and third-person perspectives, AI today doesn't share that level of understanding.

"For AI systems to interact with the world the way we do, the AI field needs to evolve to an entirely new paradigm of first-person perception," Grauman said. "That means teaching AI to understand daily life activities through human eyes in the context of real-time motion, interaction, and multi-sensory observations."

Disclaimer: This post has been auto-published from an agency feed without any modifications to the text and has not been reviewed by an editor

Tags: FacebookFacebookKristen GraumanFacebook connectivityAfter facebookNl salviCs - connectivityWhatsapp facebookFacebook newsFacebook twitterProduct administration
Open in App

Related Stories

TechnologyMeta Unable to Block AI-Generated Hateful Political Ads during India's Election, New Investigation Finds

TechnologyFacebook, Instagram Down: Meta-Owned Apps Not Loading for Users

InternationalHungary President Katalin Novak Resigns for Pardoning Accomplice in Child Abuse Case (Watch Video)

TechnologyFacebook Turns 20, Instagram Sends Heartfelt Message to Zuckerberg: 'Love You Dad'

BusinessMeta Soars with Record $196 Billion Gain, Declares First Dividend

Technology Realted Stories

TechnologyResearchers discover new approach against fatty liver

TechnologyIndia produced over 40 quantum technology startups in 2 years: Jitendra Singh

TechnologyProtein that boosts body's immunity against cancer discovered

TechnologyWill work hard to provide an ideal life for elderly, disabled: Rural Development Minister

TechnologyHow fasting can help immune system better fight cancer