02/04/08 15-494 Cognitive Robotics 1Shape Representations15-494 Cognitive RoboticsDavid S. Touretzky &Ethan Tira-ThompsonCarnegie MellonSpring 200802/04/08 15-494 Cognitive Robotics 2Types of Shapes●Basic:–PointData, LineData, EllipseData●Complex:–PolygonData, BlobData●3-D:–SphereData, BrickData●Robot shape:–AgentData02/04/08 15-494 Cognitive Robotics 3Shapes Live in a ShapeSpace●SketchSpace and ShapeSpace are duals:●We'll be using camSkS and camShS: the camera spaces.SketchSpace ShapeSpaceRenderingExtraction02/04/08 15-494 Cognitive Robotics 4SHAPEVEC and SHAPEROOTVEC●Often we want to work with collections of shapes.●A “SHAPEVEC” is a vector of shapes of a specific type: std::vector<Shape<BlobData> >●A “SHAPEROOTVEC” is a vector of generic shapes, useful when we mix shapes of different types: std::vector<ShapeRoot>●There are macros for creating and iterating over these vectors:–NEW_SHAPEVEC, NEW_SHAPEROOTVEC–SHAPEVEC_ITERATE, SHAPEROOTVEC_ITERATE02/04/08 15-494 Cognitive Robotics 5Vectors of Shapesvoid DoStart() { VisualRoutinesBehavior::DoStart(); NEW_SKETCH(camFrame, uchar, sketchFromSeg()); NEW_SHAPEVEC(blob_shapes, BlobData, BlobData::extractBlobs(camFrame,100)); if ( blob_shapes.size() > 0 ) { NEW_SKETCH(blob0, bool, blob_shapes[0]->getRendering()); } SHAPEVEC_ITERATE(blob_shapes, BlobData, blob) cout << "Id: " << blob->getId() << " Color: " << blob->getColor() << " Area: " << blob->getArea() << endl; END_ITERATE;}02/04/08 15-494 Cognitive Robotics 6Some Orange and Yellow Blobs02/04/08 15-494 Cognitive Robotics 7Extracted Blob ShapesInverted:right clickId: 10001 Color: [253,119,15] Area: 2351Id: 10002 Color: [253,119,15] Area: 1256Id: 10003 Color: [193,177,9] Area: 1378Id: 10004 Color: [193,177,9] Area: 1065Id: 10005 Color: [193,177,9] Area: 70502/04/08 15-494 Cognitive Robotics 8Line Shapes●A line has two endpoints, which can be–Valid or invalid (e.g., line runs out of the camera frame)–Active or inactiveIf both endpoints are inactive, line extends to infinity.●Lines have several derived properties that are maintained automatically:–Length–Orientation (0 to )–Normal vector )linenormalvector02/04/08 15-494 Cognitive Robotics 9Extracting the Linesvoid DoStart() { VisualRoutinesBehavior::DoStart(); NEW_SKETCH(camFrame, uchar, sketchFromSeg()); NEW_SKETCH(pink_stuff, bool, visops::colormask(camFrame,"pink")); NEW_SHAPEVEC(lines, LineData, LineData::extractLines(pink_stuff));}02/04/08 15-494 Cognitive Robotics 10Extracted Line Shapes●“Select All Shapes” displays everything.●“ID” checkbox displays shape IDs.02/04/08 15-494 Cognitive Robotics 11Line EndPoints●Lines have two endpoints: end1Pt and end2Pt●Order is arbitrary●Extracting endpoints:–end1Pt(), end2Pt() -- simple accessor functions–leftPt(), rightPt() –- compare X coords.–topPt(), bottomPt() –- compare Y coords.●Orientation predicates:–IsHorizontal –- true if slope is < 60 degrees–IsVertical –- true if slope is > 30 degrees–Thresholds are user-adjustable02/04/08 15-494 Cognitive Robotics 12Logical EndPoint Descriptions●firstPt() –- if line is horizontal, returns leftPt(), else returns topPt()●secondPt() –- similar: returns rightPt() or bottomPt()●How do we compare two lines? Example:–Two lines are “close” if their first endpoints are close,and their second endpoints are also close.–But what about lines whose orientationsstraddle the critical value of 60 degrees?●line1->firstPt(line2) –- returns first point of line2 based on line1's decision about horizontal/verticalfirst=leftfirst=top02/04/08 15-494 Cognitive Robotics 13Extracting the Leftmost Pointvoid DoStart() { VisualRoutinesBehavior::DoStart(); NEW_SKETCH(camFrame, uchar, sketchFromSeg()); NEW_SKETCH(orange_stuff, bool, visops::colormask(camFrame,"orange")); NEW_SHAPE(line, LineData, LineData::extractLine(orange_stuff)); NEW_SHAPE(leftpt, PointData, line->leftPtShape()); leftpt->setColor(rgb(0,255,0));}02/04/08 15-494 Cognitive Robotics 14Extracted Point Shape●leftpt's parent is line●line's parent is orange_stuff02/04/08 15-494 Cognitive Robotics 15Constructing New Lines●Use a LineData(camShS, ...) constructor to make new lines in camera space.●Since we want to use smart pointers for shapes, the result should be fed to a Shape<LineData> constructor.–The NEW_SHAPE macro does this for us:NEW_SHAPE(myline, LineData, new LineData(camShS, ...));●Can define a new line by specifying:–two points–a point plus an orientation (0 to )02/04/08 15-494 Cognitive Robotics 16NEW_SHAPE●NEW_SHAPE is a bit of syntactic sugar: NEW_SHAPE(myline, LineData, new LineData(camShS,pt1,pt2))●Expands into: Shape<LineData> myline(new LineData(camShS,pt1,pt2));if ( myline.isValid() ) myline->V(“myline”); // make viewable●Use NEW_SHAPE_N for shapes not to be viewable.02/04/08 15-494 Cognitive Robotics 17Parents and Viewable IDsfoo id: 11 parentId: 0bar id: 17 parentId: 11baz id: 19 parentId: 17xam id: 23 parentId: 19foo 11xam 23On the Robot SketchGUIDisplayNotviewable02/04/08 15-494 Cognitive Robotics 18Mixing Sketches and Shapes●Problem: which side of an orange line has more yellow blobs?●If all we have is a line segment, people can still interpret it as a “barrier”.●How do we make the robot do this?02/04/08 15-494 Cognitive Robotics 19Lines as Barriersvoid DoStart() { VisualRoutinesBehavior::DoStart(); NEW_SKETCH(camFrame, uchar, sketchFromSeg()); NEW_SKETCH(orange_stuff, bool, visops::colormask(camFrame,"orange")); NEW_SKETCH(yellow_stuff, bool, visops::colormask(camFrame,"yellow")); NEW_SHAPE(boundary_line, LineData, LineData::extractLine(orange_stuff)); NEW_SKETCH(topside, bool, visops::topHalfPlane(boundary_line)); NEW_SKETCH(side1, bool, yellow_stuff & topside); NEW_SKETCH(side2, bool, yellow_stuff & !topside);02/04/08 15-494 Cognitive Robotics 20Lines as Barriers (cont.) NEW_SHAPEVEC(side1blobs, BlobData, BlobData::extractBlobs(side1,50)); NEW_SHAPEVEC(side2blobs, BlobData,BlobData::extractBlobs(side2,50)); vector<Shape<BlobData> > &winners = side1blobs.size() > side2blobs.size() ? side1blobs : side2blobs; NEW_SKETCH(result, bool, visops::zeros(yellow_stuff)); SHAPEVEC_ITERATE(winners, BlobData, b) result |= b->getRendering(); END_ITERATE; boundary_line->setInfinite(); // for display