Differences

This shows you the differences between two versions of the page.

Link to this comparison view

tutorials:intermediate:simple_mobile_manipulation_plan [2019/02/13 13:19]
amar Added more content
tutorials:intermediate:simple_mobile_manipulation_plan [2019/04/05 11:14] (current)
amar
Line 1: Line 1:
 **//Tested with Cram v0.7.0, ROS version: Kinetic, Ubuntu 16.04//** **//Tested with Cram v0.7.0, ROS version: Kinetic, Ubuntu 16.04//**
  
-====== Simple Mobile Manipulation Plan (Under Development) ​======+====== Simple Mobile Manipulation Plan ======
  
 This tutorial demonstrates how to write a simple plan for the mobile manipulation using the robot, how to write simple error handling and recovery behavior using a lightweight simulation. This tutorial demonstrates how to write a simple plan for the mobile manipulation using the robot, how to write simple error handling and recovery behavior using a lightweight simulation.
Line 31: Line 31:
 </​code>​ </​code>​
 You'll have your bullet world launched with the PR2 in the kitchen. Now you're good to go. You'll have your bullet world launched with the PR2 in the kitchen. Now you're good to go.
 +
 +==== Some Useful Designators ====
 +Before we proceed further let's look at some of the designators that are supported by the robot in the bullet world (and this works in the real world robot as well). To refresh your memory on designators,​ visit the tutorial [[tutorials:​beginner:​motion_designators|Creating motion designators for the TurtleSim]]. ​
 +Here are some of the supported motion and action designators:​
 +  * **Motion Designators** - They describe a low-level motion that the robot would take.
 +      * <code lisp> (desig:a motion (type moving-torso) (joint-angle ?​angle-val))</​code>​ is a motion for moving the torso of the robot and expects a key ''​joint-angle''​ with an value of the required joint angle.
 +      * <code lisp> (desig:a motion (type going) (target ?​loc-desig)) </​code>​ is a motion that moves the body of the robot and expects a key ''​target'',​ whose value should be of type ''​location-designator''​.
 +      * <code lisp> (desig:a motion (type looking) (target ?​loc-desig))</​code>​ is a motion that moves the head of the robot to look at the specfied location. It expects a key ''​target''​ with value of type ''​location''​ designator. ​
 +      * <code lisp> (desig:a motion (type detecting) (object ?​obj-desig)) </​code>​ is a a motion that perceives and detects the specified object. It expects a key ''​object''​ with a value of type ''​object-designator''​
 +    * **Action Designators** - These describe the high-level actions which may consist of multiple calls to different low-level motion to carry out a small plan.
 +        * <code lisp> (desig:an action (type picking-up) (arm ?grasp-arm) (grasp ?​grasp-pose) (object ?​obj-desig)) </​code>​ is an action that picks an object with the specified arm and grasp pose. It expects an ''​object''​ key with value of type ''​object-designators'',​ an ''​arm''​ with the value specifying the keyword for the arm to choose, and a ''​grasp''​ key whose value specifies the keyword of the grasp pose to pick the object up from.
 +        * <code lisp> (desig:an action (type placing) (arm ?grasp-arm) (object ?obj-desig) (target ?​loc-desig)) </​code>​ is an action that places the object specified to the target. It expects a key ''​arm''​ with the keyword of the arm to use, an ''​object''​ specifying value of type ''​object-designator''​ and a ''​target''​ key with value of type ''​location-designator''​.
 +
 +We will use these designators throughout this tutorial.
  
 ==== Constructing plans ==== ==== Constructing plans ====
Line 50: Line 64:
    `(and (btr:​bullet-world ?world)    `(and (btr:​bullet-world ?world)
          ​(assert (btr:object ?world :mesh bottle-1          ​(assert (btr:object ?world :mesh bottle-1
-                             ((--0.9 0.860) (0 0 0 1)on the)+                             ((-1.6 -0.9 0.860) (0 0 0 1))
                              :mass 0.2 :color (1 0 0) :mesh :bottle))                              :mass 0.2 :color (1 0 0) :mesh :bottle))
          ​(btr:​simulate ?world 100))))          ​(btr:​simulate ?world 100))))
Line 56: Line 70:
 </​code>​ </​code>​
  
-You should now see a red bottle on the table. To make our lives easier, let's define some parameters. The *final-object-destination* is the place where the pr2 has to place the red bottle in the end. The *table-access-pose* is the pose, where pr2 should stand, when he approaches the bottle and *counter-access-pose* is the pose, to access the the counter to place the bottle.+You should now see a red bottle on the table. To make our lives easier, let's define some parameters. The *final-object-destination* is the place where the pr2 has to place the red bottle in the end. The ''​*base-pose-near-table*'' ​is the pose, where pr2 should stand near the table, when he approaches the bottle and ''​*base-pose-near-counter*'' ​is the pose, to access the the counter to place the bottle.
  
 <code lisp> <code lisp>
Line 66: Line 80:
    ​(cl-transforms:​make-identity-rotation)))    ​(cl-transforms:​make-identity-rotation)))
  
-(defparameter *table-access-pose*+(defparameter *base-pose-near-table*
   (cl-transforms-stamped:​make-pose-stamped   (cl-transforms-stamped:​make-pose-stamped
    "​map"​ 0.0    "​map"​ 0.0
-   ​(cl-transforms:​make-3d-vector -1.8547d0 ​-0.150d0 0.0d0)+   ​(cl-transforms:​make-3d-vector -1.447d0 -0.150d0 0.0d0)
    ​(cl-transforms:​axis-angle->​quaternion (cl-transforms:​make-3d-vector 0 0 1) (/ pi -2))))    ​(cl-transforms:​axis-angle->​quaternion (cl-transforms:​make-3d-vector 0 0 1) (/ pi -2))))
 +   
 (defparameter *downward-look-coordinate* (defparameter *downward-look-coordinate*
   (cl-transforms-stamped:​make-pose-stamped   (cl-transforms-stamped:​make-pose-stamped
Line 78: Line 92:
    ​(cl-transforms:​make-identity-rotation)))    ​(cl-transforms:​make-identity-rotation)))
  
-(defparameter *counter-access-pose*+(defparameter *base-pose-near-counter*
   (cl-transforms-stamped:​make-pose-stamped   (cl-transforms-stamped:​make-pose-stamped
    "​base_footprint"​ 0.0    "​base_footprint"​ 0.0
-   ​(cl-transforms; Evaluation aborted on #<​CRAM-COMMON-FAILURES:​PERCEPTION-OBJECT-NOT-FOUND {1013AC5B93}>​.:​make-3d-vector -0.150d0 2.0d0 0.0d0)+   ​(cl-transforms:​make-3d-vector -0.150d0 2.0d0 0.0d0)
    ​(cl-transforms:​make-quaternion 0.0d0 0.0d0 -1.0d0 0.0d0)))    ​(cl-transforms:​make-quaternion 0.0d0 0.0d0 -1.0d0 0.0d0)))
 </​code>​ </​code>​
  
-Now our whole procedure ​would include going near the table, finding the bottle, ​and pick it up. Then go to the counter ​and set it down in the specified destinationWe can define some functions to carry out small steps sepratelyto keep the top-level readable. +Let's note down the steps that would involve picking the bottle from the table and placing ​it on the counter
-<code lisp> +  * Move the robot near the table. 
-(defun navigate-to (?​navigation-goal) +  * Move the arms out of the perception rangeso that the robot can see the bottle without obstruction
-  ​(exe:​perform (desig:a motion +  ​* Look towards the object area. 
-                        (type going) +  ​* Detect the object ​that has to be picked. 
-                        (target (desig:a location (pose ?​navigation-goal)))))) +  ​* Pick up the object ​and once again, keep the arms away from the front. 
-  +  ​* Move the robot near the counter. 
-(defun look-at (?​point-of-interest) +  * Place the bottle on the destination.
-  ​(exe:​perform (desig:a motion +
-                        (type looking) +
-                        (target (desig:a location (pose ?​point-of-interest)))))) +
-                         +
-(defun pick-up-object ​(?​object-designator &​optional (?arm :right)) +
-  ​(exe:​perform (desig:an action +
-                         (type picking-up+
-                         (arm ?arm) +
-                         (object ​?​object-designator)))) +
-                          +
-(defun place-down (?pose ?object ?arm) +
-  ​(exe:​perform (desig:an action +
-                         (type placing) +
-                         (arm ?arm) +
-                         ​(object ?object) +
-                         ​(target (desig:a location (pose ?​pose)))))) +
-                          +
-(defun perceive-object (?​object-type) +
-  ​(let ((?​perceive-direction ​*downward-look-coordinate*)) +
-    (look-at ?​perceive-direction) +
-    (exe:​perform (desig:a motion +
-                          (type detecting) +
-                          (object (desig:an object (type ?​object-type))))))) +
-</​code>​ +
- +
-Now let's try using all these methods in writing a top-level plan for pick and place.+
  
 +Implementing all the above steps in code, will look something like as shown below:
 <code lisp> <code lisp>
 (defun move-bottle () (defun move-bottle ()
   (spawn-bottle)   (spawn-bottle)
-  (proj:​with-projection-environment ​pr2-proj::pr2-bullet-projection-environment +  (pr2-proj:with-simulated-robot 
-    ​(cpl:​top-level +    (let ((?​navigation-goal *base-pose-near-table*)) 
-      ​(let ((?​navigation-goal *table-access-pose*)) +      (cpl:par 
-        (cpl:par +        (exe:​perform (desig:a motion ​ 
-          (exe:​perform (desig:a motion (type moving-torso) (joint-angle 0.3))) +                              ​(type moving-torso) 
-          (pp-plans::​park-arms) +                              ​(joint-angle 0.3))) 
-          (navigate-to ​?​navigation-goal))) +        (pp-plans::​park-arms) 
-      (let ((?​grasping-arm :right) +        ;; Moving the robot near the table. 
-            (?​perceived-bottle (perceive-object :​bottle))) +        ​(exe:perform (desig:a motion 
-        (pick-up-object ?​perceived-bottle) +                              (type going) 
-        (pp-plans::​park-arms :arm ?​grasping-arm) +                              (target (desig:a location  
-        (let ((?nav-goal *counter-access-pose*)) +                                               ​(pose ​?​navigation-goal))))))) 
-          (navigate-to ​?​nav-goal)) +    ;; Looking towards the bottle before perceiving. 
-        (coe:​on-event (make-instance '​cpoe:​robot-state-changed)) +    (let ((?​looking-direction *downward-look-coordinate*)) 
-        (let ((?​drop-pose *final-object-destination*)) +      ​(exe:​perform (desig:a motion  
-            (place-down ​?drop-pose ?​perceived-bottle ?grasping-arm)) +                            (type looking) 
-        (pp-plans::​park-arms :arm ?​grasping-arm)))))+                            (target (desig:a location  
 +                                             (pose ?​looking-direction)))))) 
 +    ;; Detect the bottle on the table. 
 +    ​(let ((?​grasping-arm :right) 
 +          (?​perceived-bottle (exe:perform (desig:a motion 
 +                                                   (type detecting) 
 +                                                   (object ​(desig:an object  
 +                                                                     ​(type ​:bottle))))))) 
 +      ;; Pick up the bottle 
 +      ​(exe:perform (desig:an action 
 +                             (type picking-up
 +                             (arm ?grasping-arm) 
 +                             ​(grasp left-side) 
 +                             (object ?​perceived-bottle))
 +      (pp-plans::​park-arms :arm ?​grasping-arm) 
 +      ;; Moving the robot near the counter. 
 +      ​(let ((?nav-goal *base-pose-near-counter*)) 
 +        (exe:perform (desig:a motion 
 +                              (type going) 
 +                              (target (desig:a location  
 +                                               ​(pose ​?nav-goal)))))) 
 +      (coe:​on-event (make-instance '​cpoe:​robot-state-changed)) 
 +      ;; Setting the bottle down on the counter 
 +      ​(let ((?​drop-pose *final-object-destination*)) 
 +        (exe:perform (desig:an action 
 +                               (type placing) 
 +                               ​(arm ​?grasping-arm) 
 +                               ​(object ​?​perceived-bottle
 +                               ​(target (desig:a location  
 +                                                (pose ?drop-pose)))))) 
 +      (pp-plans::​park-arms :arm ?​grasping-arm))))
 </​code>​ </​code>​
 +
 +Note that the plan is nested under ''​pr2-proj:​with-simulated-robot''​ indicating that all the methods resolved by the designators is being called for the projection environment (You can see these low level methods being called under the ''​cram_pr2_projections''​ package under ''​low-level.lisp''​). If ''​pr2-pms:​with-real-robot''​ is used to replace this, the functions from the real robot ROS interfaces will be called. Also note that it is possible to execute actions which are independent in parallel (In this case, moving the robot torso up and moving the robot base are independent actions done in parallel)
  
 Now run ''​(move-bottle)''​ and you will see that the robot successfully picks the bottle and places it on the counter. Now run ''​(move-bottle)''​ and you will see that the robot successfully picks the bottle and places it on the counter.
-Note:- Every time you run ''​move-bottle''​ you need to reset the world with ''​(init-projection)''​. You can include the call at the top of the test function, but mabe you want to comment some parts out to see how the plan behaves. 
  
-==== Handling Failures ​==== +// 
-The previous example worked perfectly because, we as the developer, ​knew the exact coordinates to look for the bottle. ​Now let's try a situation where we approach same table but at different position. For that, let's redefine *table-access-pose*+Note:- Every time you run ''​move-bottle''​ you need to reset the world with ''​(init-projection)''​. You can include the call at the top of the test function, but maybe you want to comment some parts out to see how the plan behaves.//​ 
 +===== Increasing the Effectiveness by Improving the plan ===== 
 +The previous example worked perfectly because, we knew the exact coordinates to look for the bottle. ​This is hardly true for the real life scenario, especially since we are dealing with kitchen environment,​ far from being precise compared to a factory floorWhat would happen if the bottle was moved a little bit to the right of its previous spawn position? ​For this, let's redefine ​our ''​spawn-bottle''​ 
 + 
 +<code lisp> 
 +BTW-TUT>​ 
 +(defun spawn-bottle () 
 +  (unless (assoc :bottle btr::*mesh-files*) 
 +    (add-objects-to-mesh-list)) 
 +  (prolog:​prolog 
 +   `(and (btr:​bullet-world ?world) 
 +         ​(assert (btr:object ?world :mesh bottle-1 
 +                             ((-2 -0.9 0.860) (0 0 0 1)) 
 +                             :mass 0.2 :color (1 0 0) :mesh :bottle)) 
 +         ​(btr:​simulate ?world 100)))) 
 +</​code>​ 
 +Now try running ''​(move-bottle)''​ again. Don't forget to clean up the projection before running. 
 +And the output will look like this. 
 +<code lisp> 
 +BTW-TUT> (move-bottle) 
 +; Evaluation aborted on #<​CRAM-COMMON-FAILURES:​PERCEPTION-OBJECT-NOT-FOUND {1013AC5B93}>​. 
 +</​code>​ 
 + 
 +{{:​tutorials:​intermediate:​btw-tut-cant-see-bottle.png?​800|}} 
 + 
 +Clearly the robot cannot find the object anymore because even though the robot is near the bottle, it is out of the field of vision of the robot due to the predefined base and object ​pose in our code. Let's try fixing this issue.  
 + 
 +=== Recovering from Failures === 
 +To understand the syntax and get a refresher on failure handling, you can refer to [[tutorials:​beginner:​failure_handling|Failure Handling]]. 
 +We're going to add some code into fixing perception in our case, so that the robot would still be able to find the bottle. Let's list down a plan of action for this. 
 +  - Tilt the head of the robot downwards 
 +  - Try to detect the bottle 
 +  - If,  
 +       successful in finding the bottle - continue with the rest of the code. 
 +       * failed to find the bottle - turn the head to a different configuration (eg., left/right) and try detecting again. 
 +   - When all possible directions fail, error out.
  
 +Let's define some additional parameters to aid us:
 <code lisp> <code lisp>
 BTW-TUT> BTW-TUT>
-(defparameter *table-access-pose*+(defparameter *left-downward-look-coordinate*
   (cl-transforms-stamped:​make-pose-stamped   (cl-transforms-stamped:​make-pose-stamped
-   "​map" 0.0 +   "​base_footprint" 0.0 
-   ​(cl-transforms:​make-3d-vector ​-1.447d0 -0.150d0 0.0d0+   ​(cl-transforms:​make-3d-vector ​0.65335d0 ​0.76d0 0.758d0
-   ​(cl-transforms:​axis-angle->​quaternion ​(cl-transforms:​make-3d-vector 0 0 1) (/ pi -2))))+   ​(cl-transforms:​make-identity-rotation))) 
 + 
 +(defparameter *right-downward-look-coordinate* 
 +  (cl-transforms-stamped:​make-pose-stamped 
 +   "​base_footprint"​ 0.0 
 +   (cl-transforms:​make-3d-vector 0.65335d0 -0.76d0 0.758d0) 
 +   (cl-transforms:​make-identity-rotation))) 
 +   
 </​code>​ </​code>​
-Now try running ​''​(move-bottle)'' ​again.(Don't forget ​to clean up the projection.)+We defined two coordinates ​''​*left-downward-look-coordinate*'' ​and ''​*right-downward-look-coordinate*''​ with respective to our robot base footprint as alternative directions ​to look for when downward look fails.
  
 +Now we define a method ''​find-object''​ which encapsulates our plan with failure handling :
 +<code lisp>
 +(defun get-preferred-arm-for-direction (direction-looked)
 +  (let ((preferred-arm :RIGHT))
 +    (when (eq direction-looked *left-downward-look-coordinate*)
 +      (setf preferred-arm :LEFT))
 +    preferred-arm))
 +    ​
 +(defun find-object (?​object-type)
 +  (let* ((possible-look-directions `(,​*downward-look-coordinate*
 +                                     ,​*left-downward-look-coordinate*
 +                                     ,​*right-downward-look-coordinate*))
 +         ​(?​looking-direction (first possible-look-directions)))
 +    (setf possible-look-directions (cdr possible-look-directions))
 +    (exe:​perform (desig:a motion ​
 +                          (type looking)
 +                          (target (desig:a location ​
 +                                           (pose ?​looking-direction)))))
 + 
 +    (cpl:​with-failure-handling
 +        ((cram-common-failures:​perception-object-not-found (e)
 +           ;; Try different look directions until there is none left.
 +           (when possible-look-directions
 +             ​(roslisp:​ros-warn (perception-failure) "​~a~%Turning head." e)
 +             ​(exe:​perform (desig:a motion ​
 +                                   (type looking) ​
 +                                   ​(direction forward)))
 +             (setf ?​looking-direction (first possible-look-directions))
 +             (setf possible-look-directions (cdr possible-look-directions))
 +             ​(exe:​perform (desig:a motion ​
 +                                   (type looking)
 +                                   ​(target (desig:a location
 +                                                    (pose ?​looking-direction)))))
 +             ​(cpl:​retry))
 +           ​(cpl:​fail '​common-fail:​looking-high-level-failure)))
 + 
 +      (let ((?​perceived-bottle
 +              (exe:​perform (desig:a motion
 +                                    (type detecting)
 +                                    (object (desig:an object ​
 +                                                      (type ?​object-type)))))))
 +        (values ?​perceived-bottle (get-preferred-arm-for-direction ?​looking-direction))))))
 +        </​code>​
 +Let's see what this method does. The ''​with-failure-handling''​ clause here deals with ''​cram-common-failures:​perception-object-not-found''​ which if you remember, was the error raised when the robot couldn'​t find the bottle. So instead of only looking downwards, the code will now iterate between downwards, left and right, and only upon a failure in all these, will an error be bubbled up - thus increasing our effective field of view. 
 +Also note that the ''​find-object''​ suggests an arm to use, depending on the direction it looked.
 +Let us also update our ''​move-bottle''​ to use this method.
 +<code lisp>
 +(defun move-bottle ()
 +  (spawn-bottle)
 +  (pr2-proj:​with-simulated-robot
 +    (let ((?​navigation-goal *base-pose-near-table*))
 +      (cpl:par
 +        (exe:​perform (desig:a motion ​
 +                              (type moving-torso)
 +                              (joint-angle 0.3)))
 +        (pp-plans::​park-arms)
 +        ;; Moving the robot near the table.
 +        (exe:​perform (desig:a motion
 +                              (type going)
 +                              (target (desig:a location ​
 +                                               (pose ?​navigation-goal)))))))
 +    ;; Find and detect the bottle on the table.
 +    (multiple-value-bind (?​perceived-bottle ?​grasping-arm) ​
 +        (find-object :bottle)
 +      (exe:​perform (desig:an action
 +                             (type picking-up)
 +                             (arm ?​grasping-arm)
 +                             ​(grasp left-side)
 +                             ​(object ?​perceived-bottle)))
 +      (pp-plans::​park-arms :arm ?​grasping-arm)
 +      ;; Moving the robot near the counter.
 +      (let ((?nav-goal *base-pose-near-counter*))
 +        (exe:​perform (desig:a motion
 +                              (type going)
 +                              (target (desig:a location ​
 +                                               (pose ?​nav-goal))))))
 +
 +      (coe:​on-event (make-instance '​cpoe:​robot-state-changed))
 +      ;; Setting the object down on the counter
 +      (let ((?​drop-pose *final-object-destination*))
 +        (exe:​perform (desig:an action
 +                               (type placing)
 +                               (arm ?​grasping-arm)
 +                               ​(object ?​perceived-bottle)
 +                               ​(target (desig:a location ​
 +                                                (pose ?​drop-pose))))))
 +      (pp-plans::​park-arms :arm ?​grasping-arm))))
 +</​code>​
 +
 +Clean up and run ''​(move-bottle)''​ again, and this time, you'll find that the robot succeeds in transporting the bottle.
 +{{:​tutorials:​intermediate:​btw-tut-found-bottle-again.png?​800|}}
 +
 +=== Expanding Failure Management Capabilities ===
 +Everything is good so far, but let's call this a lucky coincidence. For the robot, knowing which arm to use to pick up the bottle is not always enough. There are many positions with which we can grasp objects - from the object'​s front, back, left, right, etc. One might think that since the bottle is a rotationally symmetric object, it doesn'​t matter which side you approach from. But consider the bottle as an object model, which has a specific front, back and sides according to its orientation with respect to the bullet world. In which case, the side with which the arm approaches the bottle greatly matters, accounting for the configuration of the joints the robot arm will make while trying to grasp - some poses may be unachievable,​ while others may result in the collision of the arm with the table.
 +
 +Let's try to visualize this issue, by spawning the bottle in yet another position:
 +<code lisp>
 +(defun spawn-bottle ()
 +  (unless (assoc :bottle btr::​*mesh-files*)
 +    (add-objects-to-mesh-list))
 +  (prolog:​prolog
 +   `(and (btr:​bullet-world ?world)
 +         ​(assert (btr:object ?world :mesh bottle-1
 +                             ​((-1.1 -0.75 0.860) (0 0 0 1))
 +                             :mass 0.2 :color (1 0 0) :mesh :bottle))
 +         ​(btr:​simulate ?world 100))))
 +</​code>​
 +Now run ''​move-bottle''​ once more, and the output should be something similar to as below:
 <code lisp> <code lisp>
 BTW-TUT> (move-bottle) BTW-TUT> (move-bottle)
-; Evaluation aborted on #<​CRAM-COMMON-FAILURES:​PERCEPTION-OBJECT-NOT-FOUND {1013AC5B93}>.+[(PICK-PLACE PICK-UP) INFO] 1550502686.470:​ Opening gripper 
 +[(PICK-PLACE PICK-UP) INFO] 1550502686.470:​ Reaching 
 +[(PICK-PLACE MOVE-ARMS-IN-SEQUENCE) WARN] 1550502686.797:​ #<​POSE-STAMPED  
 +   ​FRAME-ID:​ "​torso_lift_link",​ STAMP: 0.0 
 +   #<​3D-VECTOR (0.6819813024319948d0 0.4206671881870251d0 -0.11278482277792945d0)>​ 
 +   #<​QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>>​ is unreachable for EE. 
 +Ignoring. 
 +[(PICK-PLACE MOVE-ARMS-IN-SEQUENCE) ERROR] 1550502687.092:​ #<​POSE-STAMPED  
 +   ​FRAME-ID:​ "​torso_lift_link",​ STAMP: 0.0 
 +   #<​3D-VECTOR (0.6797228574484719d0 0.4210222500057509d0 -0.2627674055849005d0)>​ 
 +   #<​QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>>​ is unreachable for EE. 
 +Failing. 
 +[(PP-PLANS PICK-UP) WARN] 1550502687.092:​ Manipulation messed up: #<​POSE-STAMPED  
 +   ​FRAME-ID:​ "​torso_lift_link",​ STAMP: 0.0 
 +   #<​3D-VECTOR (0.6797228574484719d0 0.4210222500057509d0 -0.2627674055849005d0)>​ 
 +   #<​QUATERNION (-0.005172943672216379d0 0.0055962335340426494d0 -0.7845776913102387d0 0.6199836767360266d0)>>​ is unreachable for EE. 
 +Ignoring. 
 +; Evaluation aborted on #<​CRAM-COMMON-FAILURES:​MANIPULATION-POSE-UNREACHABLE {10100E64C3}>​. 
 +</​code>​ 
 + 
 +The robot has failed to grasp again, even though the bottle is well within perception and grasping range. 
 + 
 +So what went wrong? If you go back to definition of ''​get-preferred-arm-for-direction'':​ 
 +<code lisp> 
 +(defun get-preferred-arm-for-direction (direction-looked) 
 +  (let ((preferred-arm :RIGHT)) 
 +    ;; Always prefers right arm unless looking at the left side 
 +    (when (eq direction-looked *left-downward-look-coordinate*) 
 +      (setf preferred-arm :LEFT)) 
 +    preferred-arm)) 
 +</​code>​ 
 +And the part where we pick up the object: 
 +<code lisp> 
 +        (exe:​perform (desig:an action 
 +                               (type picking-up) 
 +                               (arm ?​grasping-arm) 
 +                               ​(grasp left-side) 
 +                               ​(object ?​perceived-bottle))) 
 +</​code>​ 
 +We see that the robot defaults the right arm when the object is in the center and will always try to grasp the left side of the bottle, even when the object is slightly favoring the left side under it (Refer the figure below). 
 + 
 +{{:​tutorials:​intermediate:​btw-tut-grasp-config-fail.png?​800|}
 + 
 +Once again, let's formulate a strategy like the previous case here. The plan we are going for will look something like this: 
 +  - Choose the favored arm. 
 +  - Get all possible grasp poses for the given type of the object and the arm. 
 +  - Choose one grasp from the possible grasp list. 
 +  - Try to grasp the object. 
 +  - If, 
 +     * Grasp succeeds - Continue with the rest of the code 
 +     * Grasp Fails - Choose a different grasp pose from the list. 
 +  - When no more possible grasp poses, try changing the arm used to grasp **Once** and try resuming from  Step (2) 
 +  - When attempted with all arms and grasps, error out. 
 + 
 +Let's encapsulate all this in a method called ''​pick-up-object'':​ 
 +<code lisp> 
 +(defun pick-up-object (?​perceived-object ?​object-type ?​grasping-arm) 
 +  (let ((?​possible-arms '​(:​right :left))) 
 +    ;;Retry by changing the arm 
 +    (cpl:​with-retry-counters ((arm-change-retry 1)) 
 +        (cpl:​with-failure-handling 
 +            ((common-fail:​object-unreachable (e) 
 +               ​(roslisp:​ros-warn (arm-failure) "​Manipulation failed: ~a~%" e) 
 +               ​(cpl:​do-retry arm-change-retry 
 +                 (setf ?​grasping-arm (car (remove ?​grasping-arm ?​possible-arms))) 
 +                 ​(cpl:​retry)) 
 +               ​(roslisp:​ros-warn (arm-failures) "No more retries left"​))) 
 +              
 +          ;; Retry by changing the grasp 
 +          (let* ((?​possible-grasp 
 +                   ​(cram-object-interfaces:​get-object-type-grasps ?​object-type nil nil nil ?​grasping-arm)) 
 +                 ​(?​grasp (cut:​lazy-car ?​possible-grasp))) 
 +            (cpl:​with-retry-counters ((grasp-retries 3)) 
 +              (cpl:​with-failure-handling 
 +                  (((or cram-common-failures:​manipulation-pose-unreachable 
 +                        cram-common-failures:​gripper-closed-completely) (e) 
 +                     ​(roslisp:​ros-warn (grasp-failure) 
 +                                       "​~a~%Failed to grasp from ~a using ~a arm " 
 +                                       e ?grasp ?​grasping-arm) 
 +                     ​(cpl:​do-retry grasp-retries 
 +                       (when (cut:​lazy-car ?​possible-grasp) 
 +                         ​(roslisp:​ros-info (trying-new-grasp) 
 +                                           "​Trying to grasp from ~a using ~a arm" 
 +                                         ?​grasp ?​grasping-arm) 
 +                         (setf ?​possible-grasp (cut:​lazy-cdr ?​possible-grasp)) 
 +                         ​(pp-plans::​park-arms) 
 +                         (setf ?grasp (cut:​lazy-car ?​possible-grasp)) 
 +                         ​(cpl:​retry))) 
 +                     ​(roslisp:​ros-warn (grasp-failures) "No more retries left"​) 
 +                     ​(cpl:​fail '​common-fail:​object-unreachable))) 
 +                ;; Perform the grasp 
 +                (exe:​perform (desig:an action 
 +                                       (type picking-up) 
 +                                       (arm ?​grasping-arm) 
 +                                       ​(grasp ?grasp) 
 +                                       ​(object ?​perceived-object))))))))) 
 +  ?​grasping-arm) 
 +</​code>​ 
 +With this, the ''​pick-up-object''​ can now iterate through all possible grasp configurations stored in ''?​possible-grasp''​The ''​cram-object-interaces:​get-object-type-grasps''​ is a useful method which will give these values as a lazy list, provided the grasping arm and the object type. 
 +  
 +There are two nested failure-handling clauses here. The inner failure-handling part will take care of ''​cram-common-failures:​manipulation-pose-unreachable''​ and ''​cram-common-failures:​gripper-closed-completely''​ - both of which are errors which can occur during a failed grasp. The recovery method will iterate through all possible grasp configurations for a given arm. Upon failure of this, the outer failure handling clause will take care of the ''​common-fail:​object-unreachable''​ error thrown by the inner part and change the arm it's trying to grasp the object with and then retry the entire procedure again. 
 + 
 +And only upon the failure of all these will the error be bubbled up. The method also returns the grasping arm it used to pick the object up so that the rest of the code is aware of the arm in which the bottle rests. Also, note that the arm we decided on earlier in ''​find-object''​ now acts as a bias to reduce the number of searches we do to find a successful configuration. We have also written appropriate warning statements to be informed about the actions the robot is taking. 
 + 
 +Also let's redefine ''​move-bottle''​ again to include these: 
 +<code lisp> 
 +(defun move-bottle () 
 +  (spawn-bottle) 
 +  (pr2-proj:​with-simulated-robot 
 +    (let ((?​navigation-goal *base-pose-near-table*)) 
 +      (cpl:par 
 +        (exe:​perform (desig:a motion  
 +                              (type moving-torso)  
 +                              (joint-angle 0.3))) 
 +        (pp-plans::​park-arms) 
 +        ;; Moving the robot near the table. 
 +        (exe:​perform (desig:a motion 
 +                              (type going) 
 +                              (target (desig:a location  
 +                                               (pose ?​navigation-goal))))))) 
 +       
 +    (multiple-value-bind (?​perceived-bottle ?​grasping-arm)  
 +        (find-object :bottle) 
 +      (setf ?​grasping-arm (pick-up-object ?​perceived-bottle :bottle ?​grasping-arm)) 
 +      (pp-plans::​park-arms :arm ?​grasping-arm) 
 +      ;; Moving the robot near the counter. 
 +      (let ((?nav-goal *base-pose-near-counter*)) 
 +        (exe:​perform (desig:a motion 
 +                              (type going) 
 +                              (target (desig:a location  
 +                                               (pose ?​nav-goal)))))) 
 +  
 +      (coe:​on-event (make-instance '​cpoe:​robot-state-changed)) 
 +      ;; Setting the object down on the counter 
 +      (let ((?​drop-pose *final-object-destination*)) 
 +        (exe:​perform (desig:an action 
 +                               (type placing) 
 +                               (arm ?​grasping-arm) 
 +                               ​(object ?​perceived-bottle) 
 +                               ​(target (desig:a location  
 +                                                (pose ?​drop-pose)))))) 
 +      (pp-plans::​park-arms :arm ?​grasping-arm)))) 
 +</​code>​ 
 +You should see a result that looks like the one below. //[Some messages that would come up are suppressed here for readability.]//​ 
 +<code lisp>  
 +BTW-TUT> (init-projection) 
 +BTW-TUT> (move-bottle) 
 +[(PICK-PLACE PICK-UP) INFO] 1550504321.279:​ Opening gripper 
 +[(PICK-PLACE PICK-UP) INFO] 1550504321.279:​ Reaching 
 +[(GRASP-FAILURE) WARN] Failed to grasp from LEFT-SIDE using RIGHT arm  
 +[(TRYING-NEW-GRASP) INFO] 1550504800.749:​ Trying to grasp from RIGHT-SIDE using RIGHT arm 
 +[(PICK-PLACE PICK-UP) INFO] 1550504800.789:​ Opening gripper 
 +[(PICK-PLACE PICK-UP) INFO] 1550504800.789:​ Reaching 
 +[(GRASP-FAILURE) WARN] Failed to grasp from RIGHT-SIDE using RIGHT arm  
 +[(TRYING-NEW-GRASP) INFO] 1550504801.577:​ Trying to grasp from BACK using RIGHT arm 
 +[(PICK-PLACE PICK-UP) INFO] 1550504801.601:​ Opening gripper 
 +[(PICK-PLACE PICK-UP) INFO] 1550504801.602:​ Reaching 
 +[(PICK-PLACE PICK-UP) INFO] 1550504801.939:​ Gripping 
 +[(PICK-PLACE PICK-UP) INFO] 1550504801.973:​ Assert grasp into knowledge base 
 +[(PICK-PLACE PICK-UP) INFO] 1550504801.974:​ Lifting 
 +[(PICK-PLACE PLACE) INFO] 1550504802.356:​ Reaching 
 +[(PICK-PLACE PLACE) INFO] 1550504802.508:​ Putting 
 +[(PICK-PLACE PLACE) INFO] 1550504802.619:​ Opening gripper 
 +[(PICK-PLACE PLACE) INFO] 1550504802.655:​ Retract grasp in knowledge base 
 +[(PICK-PLACE PLACE) INFO] 1550504802.660:​ Retracting 
 +</​code>​ 
 +{{:​tutorials:​intermediate:​btw-tut-grasp-again.png?​800|}} 
 + 
 +The robot has once again succeeded in grasping the object.  
 + 
 +You can now try spawning the bottle in different points on the table and observing how the robot resolves the arm and grasp for it. Subsequently,​ you will notice that our plans will work as long as the bottle is within reach of the robot. You could try implementing the adjustment of the robot'​s position to facilitate the fetch. 
 + 
 +Since this is a simple tutorial in formulating and understanding mobile plans using CRAM, developing advanced plans and recovery behaviors, is left up to you. 
 + 
 +=== Useful Macros === 
 +Many a time, you will find yourselves writing very similar failure handling strategies, because the solution in most cases is to iterate safely between the solutions of a designator or the elements of a list. CRAM provides a couple of useful macros to deal with this to avoid the cumbersome process of rewriting the same logic again and again. 
 +You will find these macros under ''​cram_common_failures''​. Some examples are: 
 +  * ''​common-fail:​retry-with-designator-solutions''​ - Will retry by iterating over different solutions of the specified designator upon failure. 
 +  * ''​common-fail:​retry-with-loc-designator-solutions''​ - Same as the previous case but specialized to location designators. This macro also ensures that the location used is unique (outside a specified threshold) in successive iterations. 
 +  * ''​common-fail:​retry-with-list-solutions''​ - Will retry by iterating over elements of a list upon failure. This also supports lazy lists. 
 + 
 +To demonstrate this, let's take a look at our existing ''​pick-up-object'':​ 
 +<code lisp> 
 +defun pick-up-object (?​perceived-object ?​object-type ?​grasping-arm) 
 +  (let ((?​possible-arms '​(:​right :left))) 
 +    ;;Retry by changing the arm 
 +    (cpl:​with-retry-counters ((arm-change-retry 1)) 
 +        (cpl:​with-failure-handling 
 +            ((common-fail:​object-unreachable (e) 
 +               ​(roslisp:​ros-warn (arm-failure) "​Manipulation failed: ~a~%" e) 
 +               ​(cpl:​do-retry arm-change-retry 
 +                 (setf ?​grasping-arm (car (remove ?​grasping-arm ?​possible-arms))) 
 +                 ​(cpl:​retry)) 
 +               ​(roslisp:​ros-warn (arm-failures) "No more retries left"​))) 
 +  
 +          ;; Retry by changing the grasp 
 +          (let* ((?​possible-grasp 
 +                   ​(cram-object-interfaces:​get-object-type-grasps ?​object-type nil nil nil ?​grasping-arm)) 
 +                 ​(?​grasp (cut:​lazy-car ?​possible-grasp))) 
 +            (cpl:​with-retry-counters ((grasp-retries 3)) 
 +              (cpl:​with-failure-handling 
 +                  (((or cram-common-failures:​manipulation-pose-unreachable 
 +                        cram-common-failures:​gripper-closed-completely) (e) 
 +                     ​(roslisp:​ros-warn (grasp-failure) 
 +                                       "​~a~%Failed to grasp from ~a using ~a arm " 
 +                                       e ?grasp ?​grasping-arm) 
 +                     ​(cpl:​do-retry grasp-retries 
 +                       (when (cut:​lazy-car ?​possible-grasp) 
 +                         ​(roslisp:​ros-info (trying-new-grasp) 
 +                                           "​Trying to grasp from ~a using ~a arm" 
 +                                         ?​grasp ?​grasping-arm) 
 +                         (setf ?​possible-grasp (cut:​lazy-cdr ?​possible-grasp)) 
 +                         ​(pp-plans::​park-arms) 
 +                         (setf ?grasp (cut:​lazy-car ?​possible-grasp)) 
 +                         ​(cpl:​retry))) 
 +                     ​(roslisp:​ros-warn (grasp-failures) "No more retries left"​) 
 +                     ​(cpl:​fail '​common-fail:​object-unreachable))) 
 +                ;; Perform the grasp 
 +                (exe:​perform (desig:an action 
 +                                       (type picking-up) 
 +                                       (arm ?​grasping-arm) 
 +                                       ​(grasp ?grasp) 
 +                                       ​(object ?​perceived-object))))))))) 
 +  ?​grasping-arm) 
 +</​code>​ 
 + 
 +We can see that we do iterate through different grasp poses in a list in the inner failure handling clause. Let us rewrite this using the macro: 
 + 
 +<code lisp> 
 +(defun pick-up-object (?​perceived-object ?​object-type ?​grasping-arm) 
 +  (let ((?​possible-arms '​(:​right :left))) 
 +    ;;Retry by changing the arm 
 +    (cpl:​with-retry-counters ((arm-change-retry 1)) 
 +        (cpl:​with-failure-handling 
 +            ((common-fail:​object-unreachable (e) 
 +               ​(roslisp:​ros-warn (arm-failure) "​Manipulation failed: ~a~%" e) 
 +               ​(cpl:​do-retry arm-change-retry 
 +                 (setf ?​grasping-arm (car (remove ?​grasping-arm ?​possible-arms))) 
 +                 ​(cpl:​retry)) 
 +               ​(roslisp:​ros-warn (arm-failures) "No more retries left"​))) 
 +              
 +          ;; Retry by changing the grasp 
 +          (let* ((?​possible-grasp 
 +                   ​(cram-object-interfaces:​get-object-type-grasps ?​object-type nil nil nil ?​grasping-arm)) 
 +                 ​(?​grasp (cut:​lazy-car ?​possible-grasp))) 
 +            (cpl:​with-retry-counters ((grasp-retries 3)) 
 +              (cpl:​with-failure-handling 
 +                  (((or cram-common-failures:​manipulation-pose-unreachable 
 +                        cram-common-failures:​gripper-closed-completely) (e) 
 +                     ​(common-fail:​retry-with-list-solutions 
 +                         ?​possible-grasp 
 +                         ​grasp-retries 
 +                         ​(:​error-object-or-string (format nil  
 +                                                          "​~a~%Failed to grasp from ~a using ~a arm" 
 +                                                          e ?grasp ?​grasping-arm) 
 +                          :​warning-namespace (grasp-failure) 
 +                          :​rethrow-failure '​common-fail:​object-unreachable) 
 +                        (setf ?grasp (cut:​lazy-car ?​possible-grasp))))) 
 +                 
 +                (exe:​perform (desig:an action 
 +                                       (type picking-up) 
 +                                       (arm ?​grasping-arm) 
 +                                       ​(grasp ?grasp) 
 +                                       ​(object ?​perceived-object))))))))) 
 +  ?​grasping-arm)
 </​code>​ </​code>​
-[TODO: Add Image] +This code will behave exactly like before but all the repetitive logic has been moved to this macro. The first two arguments are always ​the iterating list (in this case, the list containing possible grasp pose) and the number ​of retries. The keyword ''​error-object-or-string''​ can accept a warning string or just the plain error object and it will be used for logging. ''​warning-namespace''​ will accept ​the name in which the logging information is classified intoSince our method rethrows ​''​common-fail:​object-unreachable''​ we pass that along into ''​rethrow-failure'',​ and if none is passed, it will rethrow the same error it received.
-Clearly ​the robot cannot find the object anymore because even though ​the robot is near the bottle, it is out of the field of vision of the robotLet's try fixing this.+