{"id":3258,"date":"2017-02-02T17:38:57","date_gmt":"2017-02-02T17:38:57","guid":{"rendered":"https:\/\/www.rsoftsolution.com\/blog\/facebooks-ai-unlocks-the-ability-to-search-photos-by-whats-in-them.html"},"modified":"2017-02-02T17:38:57","modified_gmt":"2017-02-02T17:38:57","slug":"facebooks-ai-unlocks-the-ability-to-search-photos-by-whats-in-them","status":"publish","type":"post","link":"https:\/\/www.rbsitsoftwaresolution.in\/blog\/facebooks-ai-unlocks-the-ability-to-search-photos-by-whats-in-them.html","title":{"rendered":"Facebook\u2019s AI unlocks the ability to search photos by what\u2019s in them"},"content":{"rendered":"


\n<\/p>\n

Initially used\u00a0to improve the experience for\u00a0visually impaired members of the Facebook community, the company\u2019s\u00a0Lumos computer vision platform is now powering image content search for all users. This means you can now search for images on Facebook with key words that describe the contents of a photo, rather than being limited by tags and captions.<\/p>\n

To accomplish the task, Facebook trained an ever-fashionable deep neural network on tens of millions of photos. Facebook\u2019s fortunate in this respect because its platform is already host to billions of captioned images. The model essentially matches search descriptors to features pulled from photos with some degree of probability.<\/p>\n

\"feb-02-2017-06-00-06\"<\/p>\n

After matching terms to images, the model ranks its output using information from both the images and the original search. Facebook also added in weights to prioritize diversity in photo results so you don\u2019t end up with 50 pics of the same thing with small changes in zoom and angle. In practice, all of this should produce more satisfying and relevant results.<\/p>\n

Eventually\u00a0Facebook will apply this technology to its growing video corpus. This could be used both in the personal context of searching a friend\u2019s video to find the exact moment she blew out the candles on her birthday cake, or in a commercial context.\u00a0The later could help raise the ceiling\u00a0on\u00a0Facebook\u2019s potential ad revenue<\/a> from News Feed.<\/p>\n

Pulling content from photos and videos provides an original vector to improve targeting. Eventually it would be nice to see a fully integrated system where one could pull information, say searching a\u00a0dress you really liked in a video, and relate it back to something on Marketplace or even connect you directly with an ad-partner to improve customer experiences while keeping revenue growth afloat.<\/p>\n

\"photo-search-nn-diagram\"<\/p>\n

Basic structure of how object recognition works<\/p>\n

Applying\u00a0Lumos to help the visually impaired<\/h2>\n

Along with today\u2019s new image content search feature, Facebook is updating its original\u00a0Automatic Alternative Text<\/a> tool. When Facebook released the tool last April,\u00a0visually impaired users could leverage existing text-to-speech tools to understand the contents of photos for the first time. The\u00a0system could tell you that a photo involved a stage and lights, but it wasn\u2019t very good at\u00a0relating actions to objects.<\/p>\n

A Facebook team fixed that problem by painstakingly labeling 130,000 photos pulled from the platform. The company was able to train a computer vision model to identify actions\u00a0happening in photos. Now you might now hear \u201cpeople dancing on stage,\u201d a much better, contextualized, description.<\/p>\n

The applied computer vision race<\/h2>\n

Facebook isn\u2019t the only one racing to apply recent computer vision\u00a0advances to\u00a0existing products. Pinterest\u2019s visual search <\/a>feature has been continuously improved to let users search images by the objects within them. This makes photos interactive and more importantly it makes them commercializable.<\/p>\n

Google on the other hand open sourced its own image captioning model <\/a>last fall that can both identify objects and classify actions with\u00a0accuracy over 90 percent. The open source activity around TensorFlow has helped the framework gain prominence and become very popular with machine learning developers.<\/p>\n

Facebook\u00a0is\u00a0focused on making machine learning easy for teams across the company to integrate into their projects.\u00a0\u00a0This means improving the use of the company\u2019s general purpose FBLearner Flow.<\/p>\n

\u201cWe\u2019re currently running 1.2 million AI experiments per month on FBLearner Flow, which is six times greater than what we were running a year ago,\u201d said\u00a0Joaquin Qui\u00f1onero Candela,\u00a0Facebook\u2019s director of applied machine learning.<\/p>\n

Lumos was built on top of FBLearner Flow. It has already been used for over 200 visual models. Aside from image content search, engineers have used the tool for fighting spam.<\/p>\n

<\/p>\n

Featured Image: John Mannes<\/small><\/p>\n

Article source: http:\/\/feedproxy.google.com\/~r\/techcrunch\/social\/~3\/1JFrEDelNeE\/<\/p>","protected":false},"excerpt":{"rendered":"

Initially used\u00a0to improve the experience for\u00a0visually impaired members of the Facebook community, the company\u2019s\u00a0Lumos computer vision platform is now powering image content search for all users. This means you can <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3],"tags":[10,7,14,15,6,12,11],"class_list":["post-3258","post","type-post","status-publish","format-standard","hentry","category-latest-update","tag-computer-news","tag-downloads","tag-microsoft-download-news","tag-networking-updates","tag-technology-update","tag-wordpress-plugin-download","tag-wordpress-theme-download"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t