Revisiting Behavior-Based Robotics

by Li Yang Ku (Gooly)

The maker of the well known Baxter robot, Rethink Robotics, closed their doors last October. Baxter robot, although not perfect, plays an important role in robot history. Its low price tag ($22,000 instead of $100,000) and human safe features (won’t be able to kill grad students) made these robots one of the most common robots among the robotics research community. Unfortunately, that was not enough to survive in the market.

Many of you may heard of Rodney Brooks, the founder and CTO of Rethink Robotics, who was also the director of MIT’s Computer Science & Artificial Intelligence Laboratory (CSAIL) and one of the founders of iRobot, but to me, it would be behavior-based robotics that best describes him. In this post, I am going to revisit Rodney Brooks’ research on behavior-based robotics and explain why it was a big deal back then.

To fully understand behavior-based robotics, we have to go back in time and look at what was happening in the research world before Rodney Brooks started advocating for behavior-based robotics in the 80s. This is right around the time of the early AI winter and before the collapse of the expert system industry. An expert system stores a huge knowledge base of logics describing facts in the world entered by experts. During query the inference engines tries to find a solution based on the given logics. It is not hard to imagine that the robots designed at that time would also be based on this kind of thinking. Shakey, the famous robot build by Stanford Research Institute in the late 60s, uses logic to solve tasks based on a symbolic model of the environment. Despite its national fame, Shakey was designed for an experimental environment consists of big blocks and, as you might know, it was not a technology breakthrough that lead to household robots.

In the late 70s, Rodney Brooks was especially frustrated of these symbolic approaches that tries to model the world in detail. Computers were not fast at that time, and trying to estimate the world model with uncertainty is even more time consuming. In a trip which Rodney was stuck in Thailand, he observed that insects seems to be much more capable than his robots despite having a small nervous system. The realization was that there is no need to model the world because the world is always there, the robot can always sense the world and use it as its own model. This simple idea is basically the core concept of behavior-based robotics.

Rodney went on and proposed the subsumption robotic architecture that is composed of different layers of state machines, which the higher layers subsumes lower layers to create more complicated behaviors. Brooks claims that this approach is radically different from tradition approach that follows the sense-model-plan-act framework. The subsumption architecture is capable of reacting to the world in real-time since the lower layers can produce outputs directly. Instead of executing actions in a pre-planned sequence, the next actions can simply be activated by new observations from the world. Rodney argues that this new approach have a very different decomposition compared to the traditional sequential information flow. In the subsumption architecture, each layer itself connects from sensing to action. Higher layers may rely on lower layers, but does not call lower layers as subroutines. Several robots were built based on this architecture, including robot Allen that can move to a goal while avoiding obstacles, robot Herbert that can pick up soda cans, insect like robot Genghis, etc.

These work were quite influential and provided a very different perspective on how to approach AI. Unlike other robots at that time, robots under the subsumption architecture can react in real-time in a human environment. Rodney went on to promote this concept and published a series of papers (with some of the best titles) such as “Planning is just a way of avoiding figuring out what to do next” and “Elephants don’t play chess.” Two crucial ideas were emphasized in these papers. 1) Situatedness: The robots should not deal with abstract descriptions, but with the environment that directly influences the robot and 2) Embodiment: The robots should experience the world directly so that their actions have immediate feedback on the robots’ own sensations. These are the central ideas that led to behavior-based solutions.

Today, computers are much faster and robots now are capable of running the good old fashion sense-model-plan-act sequence close to if not yet in real-time. Model heavy approaches such as physics-based approaches were one of the most popular topics and planning algorithms are ubiquitous among robot arms and self-driving cars. So is behavior-based robotics still relevant in 2019? Some of the concept still exists in many robots, but in a more hybrid fashion, such as having a lower level loop that allows the robot to react faster under a high level AI planning layer. Although behavior-based robotics is not mentioned as often nowadays, I am pretty sure we will revisit it when the sense-model-plan-act approach fails again.

References:

• Brooks, Rodney A. “New approaches to robotics.” Science253, no. 5025 (1991): 1227-1232.
• Brooks, Rodney A. “Elephants don’t play chess.” Robotics and autonomous systems 6, no. 1-2 (1990): 3-15.
• Brooks, Rodney A. “Planning is just a way of avoiding figuring out what to do next.” (1987).
• Talking Robots Podcast with Rodney Brooks
• Wikipedia: subsumption architecture

Advertisements

Paper Picks: IROS 2018

By Li Yang Ku (Gooly)

I was at IROS in Madrid this October presenting some fan manipulation work I did earlier (see video below), which the King of Spain also attended (see figure above.) When the King is also talking about deep learning, you know what is a hype the trend in robotics. Madrid is a fabulous city, so I am only able to pick a few papers below to share.

 

a) Roberto Lampariello, Hrishik Mishra, Nassir Oumer, Phillip Schmidt, Marco De Stefano, Alin Albu-Schaffer, “Tracking Control for the Grasping of a Tumbling Satellite with a Free-Floating Robot”

This is work done by folks at DLR (the German Aerospace Center). The goal is to grasp a satellite that is tumbling with another satellite. As you can tell this is a challenging task and this work presents progress extended from a series of previous work done by different space agencies. Research on related grasping tasks can be roughly classified as feedback control methods that solves a regulation control problem and optimal control approaches that computes a feasible optimal trajectory using an open loop approach. In this work, the authors proposes a system that combines both feedback and optimal control. This is achieved by using a motion planner which is generated off-line with all relevant constraints to provide visual servoing a reference trajectory. Servoing will deviate from the original plan but the gross motion will be maintained to avoid motion constraints (such as singularity.) This approach is tested on a gravity free facility. If you haven’t seen one of these zero gravity devices, they are quite common among space agencies and are used to turn off gravity (see figure above.)

b) Josh Tobin, Lukas Biewald , Rocky Duan , Marcin Andrychowicz, Ankur Handa, Vikash Kumar, Bob McGrew, Alex Ray, Jonas Schneider, Peter Welinder, Wojciech Zaremba, Pieter Abbeel, “Domain Randomization and Generative Models for Robotic Grasping.”

This is work done at OpenAI (mostly) that tries to tackle grasping with deep learning. Previous works on grasping with deep learning are usually trained on at most thousands of unique objects, which is relatively small compared to datasets for image classification such as ImageNet. In this work, a new data generation pipeline that cuts meshes and combine them randomly in simulation is proposed. With this approach the authors generated a million unrealistic training data and show that it can be used to learn grasping on realistic objects and achieve similar to state of the art accuracy. The proposed architecture is shown above, α is a convolutional neural network, β is a autoregressive model that generates n different grasps (n=20), and γ is another neural network trained separately to evaluate the grasp using the likelihood of success of the grasp calculated by the autoregressive model plus another observation from the in-hand camera. This use of autoregressive model is an interesting choice where the authors claimed to be advantageous since it can directly compute the likelihood of samples.

c) Barrett Ames, Allison Thackston, George Konidaris, “Learning Symbolic Representations for Planning with Parameterized Skills.”

This is a planning work (by folks I know) that combines parameterized motor skills with higher level planning. At each state the robot needs to select both an action and how to parameterize it. This work introduces a discrete abstract representation for such kind of planning and demonstrated it on Angry Birds and a coffee making task (see figure above.) The authors showed that the approach is capable of generating a state representation that requires very few symbols (here symbols are used to describe preconditions and state estimates), therefore allow an off-the-shelf probabilistic planner to plan faster. Only 16 symbols are needed for the Angry Bird task (not the real Angry Bird, a simpler version) and a plan can be found in 4.5ms. One of the observation is that the only parameter settings needed to be represented by a symbol are the ones that maximizes the probability of reaching the next state on the path to the goal.

RSS 2018 Highlights

by Li Yang Ku (Gooly)

I was at RSS (Conference on Robotics Science and System) in Pittsburgh a few weeks ago. The conference was held in the Carnegie music hall and the conference badge can also be used to visit the two Carnegie museums next to it. (The Eskimo and native American exhibition on the third floor is a must see. Just in case you don’t know, an igloo can be built within 1.5 hours by just two Inuits and there is a video of it.)

RSS is a relatively small conference compared to IROS and ICRA. With only one single track, you get to see every accepted paper from many different fields ranging from robotic whiskers to surgical robots. I would however argue that the highlights of this year’s RSS are the Keynote talks by Bernardine Dias and Chad Jenkins. Unlike most keynote talks I’ve been to, these two talks were less about new technologies but about humanity and diversity. In this post, I am going to talk about both talks plus a few interesting papers in RSS.

a) Bernardine Dias, “Robotics technology for underserved communities: challenges, rewards, and lessons learned.”

Bernadine’s group focuses on changing technologies so that they can be accessible to communities that are left behind. One of the technologies developed was a tool for helping blind students learn braille and it had significant impact among blind communities across the globe. Bernadine gave an amazing talk at RSS. However, the video of her talk is not public yet (not sure if it will be) and surprisingly not many videos of her are on the internet. The closest content I can find is a really nice audio interview with Bernardine. There is also a short video describing their work below, but what this talk is really about is not the technology or design but the lessons learned through helping these underserved communities.

When roboticist talk about helping the society, many of them focus on the technology and left the actual application to the future. Bernadine’s group are different in that they actually travel to these underserved communities to understand what they need and integrate their feedbacks to the design process directly. This is easier said then done. You have to understand each community before your visit, some acts are considered good in one culture but an insult in another. Giving without understanding often results in waste. Bernardine mentioned in her talk that one of the schools in an underserved community they collaborated with received a large one-time donations for buying computers. It was a large event where important people came and was broadcasted on the news. However, to accommodate these hardwares, this two classroom school has to give up one of there classrooms and therefore reduce the number of classes they can teach. Ironically, the school does not have resources to power these computers nor people to teach students or teachers how to use them. The donation actually result in more harm then help to the community.

b) Odest Chadwicke (Chad) Jenkins, “Robotics: Making the World a Better Place through Minimal Message-oriented Transport Layers .”

While Bernardine tries to change technologies for underserved communities, Chad tries to design interfaces for helping people with disability by deploying robots to their home. Chad showed some of the work done by Charlie Kemp’s group and his lab with Henry Evans. Henry Evans was a successful financial officer at silicon valley until he had a stroke that caused him paralyzed and mute. However, Henry did not give up living fully and strived in advocating robots for people with disability. Henry’s story is inspiring and an example of how robots can help people with disability live freely. The robot for humanity project is the result of these successful collaborations. Since then, Henry gave three Ted talks through robots and the one below shows how Chad helped him fly a quadrotor.

 

However, the highlight of Chad’s talk was when he called out for more diversity in the community. Minorities, especially African Americans and Latinos, are way underrepresented in robotics communities in the U.S. The issue of diversity is usually not what roboticist or computer scientist would thought of or list as a priority. Based on Chad’s numbers, past robotics conferences including RSSs were not immune to these kind of negligence. This not hard to see, among the thousands of conference talks I’ve been to there were probably no more then three talks by African American speakers. Although there are no obvious solutions to solve this problem yet, having the community aware or agree that this is a problem is an important first step. Chad urges people to be aware of whether everyone is given equal opportunities and simply being friendly to isolated minorities in a conference may make a difference in the long run.

c) Rico Jonschkowski, Divyam Rastogi, and Oliver Brock. “Differential Particle Filters.”

This work introduces a differentiable particle filter (DPF) that can be trained end to end. The DPF is composed of a action sampler that generates action samples, an observation encoder, a particle proposer that learns to generate new particles based on observations, and an observation likelihood estimator that weights each particle. These four components are feedforward networks that can be learned through training data. What I found interesting is that the authors made comments similar to the authors of the paper Deep Image Prior; deep learning approaches work not just because of learning but also because of the engineered structure such as convolutional layers that encode priors. This motivated the authors to look for architectures that can encode prior knowledge of algorithms into the neural network.

d) Marc Toussaint, Kelsey R. Allen, Kevin A. Smith, and Joshua B. Tenenbaum. “Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning.”

Task and Motion Planning (TAMP) approaches are about combining symbolic task planners and geometric motion planners hierarchically. Symbolic task planners can be helpful in solving tasks sequences based on high level logic, while geometric planners operate in detailed specifications of the world state. This work is an extension that further considers dynamic physical interactions. The whole robot action sequence is modeled as a sequence of modes connected by switches. Modes represent durations that have constant contact or can be modeled by kinematic abstractions. The task can therefore be written in the form of a Logic-Geometric Program where the whole sequence can be jointly optimized. The video above show that such approach can solve tasks that the authors call physical puzzles. This work also won the best paper at RSS.