Teaching Models @ BIG:On Efficiently Assessing Modeling Concepts∗Marion Brandsteidl, Martina Seidl, and Gerti KappelInstitute of Software Technology and Interactive SystemsVienna University of TechnologyVienna, Austria{brandsteidl|seidl|kappel}@big.tuwien.ac.atABSTRACTApproximately 1000 students of computer science and busi-ness informatics attend the course Introduction to Object-Oriented Modeling (OOM) offered by the Business Infor-matics Group (BIG) of the Vienna University of Technologyeach year in order to learn the basic concepts of the Uni-fied Modeling Language (UML) and to obtain a certificate.For finishing the course successfully, the students must passthree small tests where they have to prove their theoreticalknowledge about UML concepts as well as their ability toapply this knowledge in practical exercises. In this paper wereport our experiences in assessing the modeling knowledgeof our students and we reveal how we design the tests.KeywordsTeaching Object-Oriented Modeling, Basic Modeling Course,Assessing UML1. INTRODUCTIONOne of the most challenging tasks in teaching is the prepara-tion of suitable and adequate exams. Exam questions shouldfulfill multiple (sometimes orthogonal) requirements at thesame time: the questions should cover the most importantaspects of the lecture, they should assess whether the stu-dents have reached the learning goals—and also if we havereached our teaching goals, they should have an adequatelevel of difficulty and they should allow to test if the stu-dents have not only learned the contents of the course byheart, but if they also understand the teaching material aswell as if they are able to apply the taught concepts to “realworld problems”. The questions should be answerable withinthe duration of the test and they should be capable of beingassessed with reasonable effort. In this paper we report howwe design the tests of our university course Introduction toObject-Oriented Modeling (OOM).∗This work has been partly funded by the Austrian FederalMinistry of Transport, Innovation and Technology (BMVIT)and FFG under grant FIT-IT-819584.OOM is attended by about 1000 undergraduate studentsper year who study computer science or business informat-ics at the Vienna University of Technology. In OOM weteach modeling basics by introducing syntax and semanticsof UML 2 models1[3, 5]. Profound knowledge about theUML is a prerequisite for advanced courses like Model Engi-neering or Software Engineering. Despite the huge amountof students, we try to avoid mass processing, but we try toestablish personal mentoring instead. Besides a traditionallecture where structural as well as behavioral modeling tech-niques are introduced, we organize the lab as exercise coursesin smaller groups where the theoretical contents of the lec-ture are practiced. Furthermore we provide support via on-line forums on the e-learning platform TUWEL2(a Moodleadaptation of the Vienna University of Technology). A de-tailed description of the course is given in [1].The lecture consists of eight units covering the followingUML 2 diagrams: class and object diagram, sequence di-agram, state diagram, activity diagram, and use case dia-gram. Although not mandatory, the attendance of the lec-ture is recommended as there is room for discussion and ask-ing questions. The attendance of the lecture has no directinfluence on the grade. For the practical part the studentsare divided into groups of 50 persons. Each group meets sixtimes during the semester in order to discuss the solutionof exercise sheets for practicing modeling. For each exer-cise the assistant professor chooses one student who mustpresent and explain his/her solution as well as answer ques-tions about the theoretical background. To pass the course,the students have to solve at least 24 of the 36 exercises.For further support, we provide e-learning exercises whichaim at helping the students to get some modeling practice.Completing those exercises has no influence on the grade.The participation in OOM is awarded with 3 ECTS pointsresulting in a total workload of 75 hours for an averagestudent. To obtain a positive grade, three tests have tobe passed besides the successful completion of the practi-cal part. We consider tests not only as means to obtaingrades, but also as important didactical resource. Hence weput much effort in the development of a question catalogue.Over the time we have collected a wide range of exam ques-tions for assessing modeling basics and we have gained muchexperience in the organization which allows us to deal withthe huge amount of students in an effective manner.1http://www.uml.org/2http://tuwel.tuwien.ac.at2. TEST STRUCTUREWe assess the mo deling knowledge of our students with threetests, each covering one or two different UML 2 diagramtypes—the topic of the first test is the class diagram, testtwo is about sequence diagram and state diagram, and testthree contains activity diagram and use case diagram. Eachtest has a duration of 30 minutes and one of the three testsmay be repeated at the end of the term. We design most ofthe questions new each semester, using parts of the old testquestions (or at least the problem scope) for the exercisesheets.Although Use Case Diagrams are modeled within the earlyphases of the software development life cycle, we teach it asthe last diagram type in our lecture for didactic purposes.Students tend to underestimate the importance and the levelof difficulty this diagram type has, thus not putting enougheffort into the whole lecture. We decided to teach ClassDiagrams first due to the fact that students of computerscience tend to pay more attention to “wicked cool java”than to “just drawing boxes and lines”. The Class Diagramincludes many familiar programming concepts.We decided to give three small tests instead of one largetest at the end of the term as we experienced that it bettersupports the students in learning when they do not have tostudy the complete content at once and they may preparethemselves in a more focused manner. From their feedbackwe learned that students prefer multiple small tests.When designing the questions for the three tests, we have tokeep two important issues in mind: first of all, the tasks haveto be solvable within 30 minutes, but even so they have tocover all the important teaching contents to make sure thestudents have learned enough and understand the concepts.Second, keeping in mind that three tests for 1000