Google’s Web of Semantic Robots

Semantic Robots and The Internet of Things

As digital marketers, we try to follow news of companies such as Google and their changes. We also like to keep an eye on their press releases and official announcements, and sources such as their patent filings to get a sense of where they may be going in the long run and the nearer future as well. One future appears to involve semantic robots.

When we saw an interview of Google’s CEO Larry Page, in the Financial Times, it was difficult to miss a statement such as the following, especially with two pieces of news behind it that don’t appear in the article.

• The first was a string of acquisitions made by Google of companies earlier this year that focus upon building robots.
• The second was a recently announced departure of Andy Rubin, who came to Google with the acquisition of Android, and who had moved into a role overseeing Google’s Robots Division before deciding to leave recently to focus upon becoming a venture capitalist for hardware companies.

The statement that stuck out in the interview was this one, which envisions a world filled with dreams that go much beyond self-driving cars and balloons that spread the internet across the globe:

Wouldn’t the world be a happier place if 90 percent of the people with jobs put their feet up instead and left the robots to do the work?

~ FT interview with Google CEO Larry Page

There have been a lot of articles in the press the past couple of years that talk about an Internet of Things (IoT), that have told us about home security and utility systems that can be monitored and checked upon from the Web, bicycles that have their IP addresses, refrigerators that can tell us when we need to stop at the store to restock our supply of milk for our morning breakfast, and much more.

A PEW research project report from May tells us that wearables and other devices will significantly transform society by 2025. I suspect that we will see a lot of changes even before then.

If robots are to join our household staff, and our workforces, they would join our internet of things. At least, in Google’s plans that appear to be on the horizon.

Robots an Emerging Market at Google

Earlier this year, Google went on a spending spree that added significant knowledge of robots to Google’s employee base and capabilities. Here is some more information about some of the companies that Google acquired:

Schaft Inc. – Team SCHAFT Sticks It to Competition, Wins DARPA Robotics Challenge

Industrial Perception, Inc – Industrial Perception, Inc. Debuts Next-Level Robotics for the Supply Chain

Redwood Robotics – Redwood Robotics: New Silicon Valley Startup by Meka Robotics, Willow Garage, and SRI

Meka Robotics – Rise of the compliant machines

Holomini – what they seem to be known for are Omni-directional wheels for robots.

Bot & Dolly – Bot & Dolly and the Rise of Creative Robots

Boston Dynamics – Google buys Boston Dynamics, maker of spectacular and terrifying robots

Autofuss – a sister company to Bot and Dolly, Google acquired this company at the same time as Bot & Dolly. Some of their artworks appear on video here: Reprogrammed Assembly Line Robots Make Fine Art in San Francisco

A New Robotics Team Leader

We discovered on Friday that Android founder Andy Rubin was leaving Google. We also learned that Google Research Scientist James Kuffner will be taking over the lead on Google’s Robotic’s Division.

Kuffner has a long history of working on robotics, and is perhaps most well known for co-authoring a paper titled, “Cloud-Based Robot Grasping with the Google Object Recognition Engine”

Here’s a screenshot from the paper that shows off how a robot may learn to grasp objects from the cloud:

As you can see from his robot art Gallery, Kuffner likes robots a bit, and people are saying very good things about him as a future leader of the program at Google.

Google’s Intellectual Property Involving Semantic Robots

Google has more than a couple of patent filings involving semantic robots. There are at least three of them that stood out to me.

The first couple sound like the paper that James Kuffner co-authored that I mentioned above, and he is listed on the first of these patents as a co-inventor. Interesting that as a result of both of these semantic robot patents, if you teach a robot how to do something, you end up teaching all robots connected to the cloud knowledge base on how to do it as well.

Shared robot knowledge base for use with cloud computing system
Invented by Ryan Hickman, James J. Kuffner, Jr., James R. Bruce, Chaitanya Gharpure, Damon Kohler, Arshan Poursohi, Anthony G. Francis, Jr., Thor Lewis
Assigned to Google
US Patent 8,639,644
Granted January 28, 2014
Filed: May 4, 2012

Abstract

The present application discloses shared robot knowledge bases for use with cloud computing systems.

In one embodiment, the cloud computing system collects data from a robot about an object the robot has encountered in its environment, and stores the received data in the shared robot knowledge base.

In another embodiment, the cloud computing system sends instructions for interacting with an object to a robot, receives feedback from the robot based on its interaction with the object, and updates data in the shared robot knowledge base based on the feedback.

In yet another embodiment, the cloud computing system sends instructions to a robot for executing an application based on information stored in the shared robot knowledge base.

In the disclosed embodiments, the information in the shared robot knowledge bases is updated based on robot experiences so that any particular robot may benefit from prior experiences of other robots.

Systems and methods for determining semantic information associated with objects
Invented by Damon Kohler and Ryan Hickman
Assigned to Google
US Patent 8,386,079
Granted February 26, 2013
Filed: October 28, 2011

Abstract

Methods and systems for determining semantic information associated with objects are provided. An example method includes receiving information associated with an object and information associated with a contextual situation of a robotic device from the robotic device.

For example, the information associated with the contextual situation may include information associated with an environment in which the robotic device is configured to operate. The method may further include performing a search of a database for information associated with the object and receiving results of the search. The method may also include a computing system determining semantic information associated with the use of the object based on the results of the search.

The use of the object may be based on the information associated with the contextual situation of the robotic device. According to the method, the semantic information may be stored as supplemental information associated with the object.

The patent that stands out the most to me is one that I remember from science and science fiction author Isaac Asimov‘s Three Rules of Robotics, which go like this:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its existence as long as such protection does not conflict with the First or Second Law.

The patent that reminds me of Asimov’s three rules is:

Methods and systems for autonomous robotic decision making

Here’s an example of decision making from the patent:

As an example, if the robot is asked to “clean the floor” as a first command, and to “stay quiet” when guests are in the vicinity of the robot as a second command, then the robot may temporarily stop cleaning the floor in the presence of guests. However, if the robot senses that the guests are lingering too long in its vicinity, then the robot may determine that it is now more important to “clean the floor” rather than “stay quiet”, and resume cleaning the floor.

The patent doesn’t include an actual set of “Rules”, but the idea of including some based on it isn’t too hard to believe.

The Internet of Things (IoT) will likely be coming soon enough and it may include things such as semantic robots from Google. The science fiction of yesterday is coming closer to being the science of today and tomorrow.

About Bill Slawski

With more than 26 years of SEO experience and a Juris Doctor Degree, Bill Slawski is the foremost expert on Google’s patents as related to SEO. Patent Exploration is one of the quickest and most detailed ways to find new information about SEO. Bill is the Editor of SEO by the Sea, a prominent search engine optimization blog, where he is the author of over 1,300 posts. Bill’s experience includes Fortune 500 brands and some of the largest websites in the world. Bill is a contributing author for Moz, Search Engine Land, and Search Engine Journal. In 2014-2021, he spoke at industry-leading international conferences about topics including search engine algorithms, universal and blended search, personalization in search, search and social, and duplicate content problems, structured data, and schema