95 891 Introduction to Artificial Intelligence Session 7 Computer Vision David Steier dmsteier andrew cmu edu September 17 2024 95 891 Introduction to Artificial Intelligence 1 Agenda Knowledge check Teachable Machine exercise Introduction to computer vision Edge and motion detection Image segmentation Object classification Facial recognition Appendix SEEM Image segmentation using LLMs Object detection 95 891 Introduction to Artificial Intelligence 2 Knowledge Check What does a regularization term do in a regression model If you could choose only one metric to evaluate how well a classifier worked what would you choose and why Given a state a policy in reinforcement learning recommends A Rewards B Actions C Discount factors D An optimal path to a solution 95 891 Introduction to Artificial Intelligence 3 Google s Teachable Machine Using the Teachable Machine We will use the Teachable Machine to build a vision model that distinguishes between your face and that of a classmate Training The Teachable Machine Model Divide into groups of two Make sure at least one of you has a laptop with a camera you can use for this exercise Open a web browser and browse to https teachablemachine withgoogle com Click on Get Started Click on Image Project Click on Standard image model Click on the pencil by Class 1 and edit it so it shows your name Click on the pencil by Class 2 and edit it so it shows your classmate s name Click on the Webcam button in the top frame and focus the camera on the first team member Press Hold to Record and record about 5 seconds of video of that person looking at the camera from various angles You should have captured about 100 frames from the video Click on the Webcam button in the bottom frame and focus the camera on the second team member Press Hold to Record and record about 5 seconds of video of that person looking at the camera from various angles You should have captured about 100 frames from the video Click Training and wait for the data to be prepared and the training to complete may take about 10 seconds The camera on the right should now be live with the trained model The model will try to classify whether the image seen by the camera is you or your classmate Experimenting With the Model How well can the model distinguish between the two members of your team Try positioning your faces at different camera angles or obscuring parts of your face to test the model Experiment with the various options under the Advanced dropdown under training such as Epochs Batch size and Learning Rate and retrain the model Can you observe any effect on training time or accuracy of the model If you have time add a class representing a third team member capture video of them retrain the model and see how well it distinguishes among the three of you What is Computer Vision Perception interprets the results of sensors to obtain information about the world Vision studies relationships among Images 2D Geometry 3D shape Photometry Object appearance Builds on digital image processing image image Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book 95 891 Introduction to Artificial Intelligence 8 Applications of Computer Vision Understanding what people are doing Linking pictures and words Reconstruction from many views Geometry from a single view Facial recognition Medical applications Adapted from Russell Norvig AI A Modern Approach 2020 Chapter 25 7 p 908 95 891 Introduction to Artificial Intelligence 9 Identifying the Afghan Girl Through AI Iris patterns show that the 1984 and 2002 pictures are of the same person Sharbat Gula http www cl cam ac uk jgd1000 afghan html 95 891 Introduction to Artificial Intelligence 10 Photometric Image Formation Lighting Reflectance and shading Optics Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book 95 891 Introduction to Artificial Intelligence 11 Reflectance and Shading Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book 95 891 Introduction to Artificial Intelligence 12 The Digital Camera Sampling and aliasing Color Compression 95 891 Introduction to Artificial Intelligence 13 Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book Optics Chromatic Aberration Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book 95 891 Introduction to Artificial Intelligence 14 Lens Distortions Szeliski R Computer Vision Algorithms and Applications 2010 https szeliski org Book 95 891 Introduction to Artificial Intelligence 15 Edge Detection from Gradients Russell Norvig 2020 Chapter 25 p 892 95 891 Introduction to Artificial Intelligence 16 Derivatives of Intensity For Edge Detection Russell Norvig 2020 Chapter 25 95 891 Introduction to Artificial Intelligence 17 Optical Flow for Motion Detection Russell Norvig 2020 Chapter 25 3 p 893 95 891 Introduction to Artificial Intelligence 18 Image Segmentation Two approaches Detecting the boundaries using supervised learning Finding the regions using clustering on pixels often represented as nodes in a graph Russell Norvig 2020 Chapter 25 p 895 95 891 Introduction to Artificial Intelligence 19 Challenges in Object Classification Lighting brightness and color Foreshortening distortion from viewing angle Aspect shape from viewing angle object shape Occlusion hidden parts of Deformation object changes Russell Norvig 2020 Chapter 25 p 896 95 891 Introduction to Artificial Intelligence 20 Challenges in Object Classification Intra Class Variation Slide credit Fei Fei Fergus Torralba 95 891 Introduction to Artificial Intelligence 21 Using CNNs to Classify Images Each combination of a convolution with a ReLU activation function is a local pattern detector Multiple layers detect patterns of patterns Data set augmentation by translation stretching rotating cropping the image improves accuracy 95 891 Introduction to Artificial Intelligence 22 Object Recognition and Human Performance When introduced ImageNet contained 1 2 M images 1000 categories Top 5 accuracy measured In 2012 AlexNet had 60M parameters 95 891 Introduction to Artificial Intelligence 23 2022 Performance at 90 Top 1 Accuracy https paperswithcode com sota image classification on imagenet Now task is much harder ImageNet now contains 14 million images from 20 000 categories including 120 dog breeds Leading ImageNet classifier has 14 billion parameters 95 891 Introduction to Artificial Intelligence 24 How Do Humans