20-CS-112 Winter Quarter 2009 J. Franco, Instructor --------------------------------------------------------- Class Assessment Form --------------------- Evaluation Note: ---------------- The aim of this course is to move students of differing engineering backgrounds toward proficiency in general purpose programming of solutions to engineering computational problems. By proficiency we mean ability to use a general purpose language to solve basic computational problems that will almost certainly arise in the workplace and to correctly interpret the results of executing code on a given problem input. To achieve this aim the following two points became the dominant drivers for the course: 1. Teach just a few of the constructs of Matlab. This includes control: if-then-else, for, while, fscanf, fprintf, functions and their use. 2. Code efficient solutions to several non-trivial engineering problems in labs. Important subpoints include: 1. Show how data can be represented as matrices and that different representations can have a remarkably different effect on the development of a program and its performance 2. Show how auxilliary data structures, not directly related to the specification of the given problem, for example queues, stacks, and lists, can be used to help formulate a concise, efficient solution to the given problem. Due to the size of the class, lack of an adequate number of TAs, and that we do not expect students to be able to solve the engineering problems that are posed in labs without help, solutions to all labs were posted on the course website. Students were told not to view these solutions unless they were stuck and, in case they did view the solutions, not to turn in cut-and-paste copies as their submissions. Posted solutions contain some useless segments of code which, if present, indicated that a submission was not original. Also due to the size of the class the only practical way to examine students was to use multiple choice questions. However, this form of evaluation can, at best, only approximate true assessment. In addition, the original grader was overwhelmed by the volume of submissions and did not adequately check for original solutions. This was corrected for the last two labs because the original grader was replaced near the end of the quarter. Because of this, my broad course assessments rely more on impressions resulting from personal and in-class conversations with students and on comments of the two TAs that took care of the labs (I was fortunate to have two very good students as TAs). Hence, I do not use exam or lab scores in comments assessing the course outcomes below. Outcome Summary: ---------------- Success in meeting the stated goal seemed to correlate highly with the intended major of the student as did the nature of the relationship between instructor and student. For example, the instructor had a positive relationship with most Aerospace students who seemed eager to understand what ideas the instructor was trying to convey and even invited the instructor to speak to their learning community. By contrast, the instructor experienced extraordinary resistance from Civil Engineering students who actually tried to convince the instructor that probability and other important topics have no place in Civil Engineering. Improvement Summary: -------------------- Many freshmen and sophomores do not really know why they are here except they are expecting a high paying job upon graduation. The idea that employers will expect results has only vaguely crossed the minds of many. This course indirectly exposes to the student the mission of the College of Engineering which is to train people for creative engineering work that advances the state-of-the-art, or supports high-level design or technical consulting. It is expected that some students will use the results of this course either 1. to adjust study habits, if necessary and if they intend to buy into this mission and remain in the College of Engineering, or 2. to move on to a field that their talents and interests are more suited to, for example one of the specialties of the College of Applied Science or Information Science as taught by the business school. The course could benefit from a more rigorous recruitment program that 1) provides serious scholarships and other encouragement to high school students that clearly are suited to our career targets; 2) screens students via serious interviews that determine the best match of curriculum for given student talents; 3) encourages contact with elementary and middle school students who show signs of strong talent and interest in engineering concepts. In addition, because of the differences in perceptions of various majors, as noted above, it might be advantageous to have smaller, homegeneous groups of students attending lectures. Knowledge goals: ---------------- 1. Students will know the principal constructs of Matlab including if-then-else, for, while, fscanf, fprintf, and functions, and important operators including +,-,*,/,^,.*,.^, and []. All of the lab assignments test the ability to write code that relies on these constructs and operations: the first lab is a simple test of arithmetic operations on scalars and vectors, the second advances to plotting sin and cos functions on vectors obtained from linspace, the third tests loops and if-then-else statements as well as 2D plots, the fourth calls for solving the first of four engineering plots, namely interpolation of well log data over a gridded volume, the fifth calls for reconizing amino acid patterns in DNA strings read from file using nested loops, the sixth calls for finding a minimum cost network from matrix data using a sorting function and loops, and the last homework calls for simulating neuron firing in the brain using probabilistic functions. In addition, nearly all exam questions test the ability to recognize what function a series of operations is intended to perform. Lab success rate was high but, unfortunately, near the end of the quarter it seems a high percentage of students began resorting to submission of cut-and-paste copies of posted solutions. On the other hand, exam results were spread over a wide range of grades. This is to be expected since, as was mentioned earlier, students are not uniformly prepared for an engineering curriculum, both intellectually and spiritually. 2. Students will know the difference between elementary Matlab data structures including vectors, matrices, and scalars. What has been said above for constructs and operations applies here as well. More specifically, the DNA search lab problem has input data expressed as a character string, the minimum cost network problem as a matrix with rows indexed on edges and columns indexed on vertices and cost, the interpolation problem as a matrix with rows indexed on individual well data and columns indexed on well position and sensor reading, and the 5 questions problem of lab 2 as a sequence of scalars. Alternative representations were considered in individual labs as needed. These may be found at http://gauss.ececs.uc.edu/Courses/c112/extra/ http://gauss.ececs.uc.edu/Courses/c112/labs/ 3. Students will know how to represent input data, in the form of a matrix, vector, or scalar, so as to minimize the time needed to make the data useful to a program. This point was discussed in class but not formally tested in labs and exams. I did not formally test this because I feared total disaster. If I could get a majority of students to write Matlab code for some simple problems I would be overjoyed. To expect students to grasp an analysis of all possible encodings of data is probably asking too much. 4. Students will know how to use a modeling language such as Matlab to set up solutions to some combinatorial problems that are simplified versions of actual engineering problems. Four labs present such problems including the search problem of finding amino acids in a DNA strand, the optimization problem of finding a minimum cost network given a weighted undirected graph, simulation of neuron behavior assuming some underlying probability distributions, and interpolating well log data over a gridded volume and visualizing the result. Possible solutions to these problems were discussed during lectures. Additional help was given during labs. There is no doubt that these problems, as easy as they are for a professional, present a stiff challenge to about 80% of the students in the class. I believe this challenge is necessary if these students are going to seriously use programming as a general purpose engineering tool later (which I believe nearly all will). As said earlier, this approach meets with resistance from substantially many students, perhaps as many as 30%, despite the fact that the lab problems are obviously engineering problems and the fact that I attempt to illustrate further in lecture how this tool is going to be used in some disciplines - for example by showing NASA's efforts to verify their space vehicle design and control software. From lengthy and many discussions with students I believe the problem is rooted in improper recruitment and inadequate guidance of new students. Too many students are in the College of Engineering for reasons other than what the faculty have in mind. These reasons might be the College of Engineering is prestigious, a graduate is likely to make more money, co-op defrays the cost of an education and so on. Given this, it is not surprising that I cannot get traction with a substantial number of students. I see three things that can be done to improve the percentage of students that are happy with the course given that recruitment and advising are not going to change. First, water the course down by removing non-trivial engineering problems from the labs. This will help only slightly as it is clear from both exams that significantly many students are befuddled by the simplest use of Matlab constructs. Second, reduce lecture size, make sure each lecture is populated by students of one major, and tailor the lectures to that major. Having an instructor from that major may help. FEPs could choose which lectures they will attend. Third, offer the course later in the curriculum. Students may take this course more seriously if they see themselves handicapped on a co-op job because they do not know Matlab or some other programming tool. In such a high-pressure situation they will be forced into the unpleasant reality of desperately needing to cram into their brains some rudimentary knowledge of programming. People in such a position are likely to take a formal course in programming more seriously when they return to take classes. Of these three alternatives the second seems best as it attempts to avoid punishment of students and creates a smaller, homogeneous and therefore better learning environment for the student. The only drawback is that more faculty resources are required to implement that strategy. 5. Students will know that algorithms are used to solve combinatorial problems. The last four labs ask the student to solve engineering problems in Matlab. The solutions begin with an English language description of a solution using mathematical structures such as sets and lists and operations that move objects such as cables and cities into and out of those structures. The next step is to turn the English into pseudo-code which clearly expresses the algorithm for solving the problem. Although nearly all students realize the algorithmic nature of programming solutions, few can produce an algorithm for a non-trivial problem that they have not seen before and essentially none can prove correctness. Expecting this from freshmen and sophomores is probably unrealistic. Comprehension Goals: -------------------- A midterm and final exam were given. There were 4 problems on the midterm and 6 on the final. Each problem specified code comprising a Matlab function. There were five questions associated with each problem. One of those questions asked for the result of applying the corresponding function to some all or possible inputs. Another question asked for the type of some variable or structure that was used by the function. Other questions were concerned with the process of the function. 1. Students will explain how problem data is represented and used by a program. Exam questions were intended to test this. Results were not uniform, even for the same individual. This probably reflects the flawed nature of the exam type - for some people, on a multiple choice exam even some obviously wrong answers look reasonable when seen in print. On the other hand, with 166 students it is hard to imagine using another form of written exam that would not take an impossible amount of time to make up and grade. 2. Students will explain how problem data is read from a file. It was unnatural for students to see statements like this: fid = fopen('filename','rt'); npts = fscanf(fid,'%d',1); for i=1:npts data(i,:) = fscanf(fid,'%f',3); end and it took a considerable amount of time for many students to internalize this, particularly the format specifiers of '%d' and '%f', which were the only ones we used, and the ':', which was used often. Although indexing a vector like this: v(i) was not too hard for most students, indexing a matrix like this: v(i,j) seemed to cause a lot more difficulty. Labs 4,5, and 6 required reading data from a file. Also, extra material that was covered in some labs and shown here: http://gauss.ececs.uc.edu/Courses/c112/extra/ helped with this. The file reading problem on the final exam seemed to be one of the easiest for students. 3. Students will explain how auxilliary data structures assist in solving a problem. The last two labs were intended to test this. The minimum cost network problem calls for establishing a list of edges (presented in the lab as transmission cables), initially empty, that will be output as the solution. Edges are considered one at a time; an edge is added to the solution list if it does not connect vertices (presented in the lab as cities) that are already connected by the edges in the solution list. The student is confronted with the question how can this test be made? One answer is to assign to each vertex a group number and define the invariant: two vertices have the same group number if and only if they are connected by edges in the solution list. How is the group number handled? A simple way to do this is to use a vector where the ith element is the group number of vertex i (the convention of using numbers to represent vertex identities so as to make it easy to access data in vectors and matrices is apparently easy for students to comprehend - at least they seemed to readily accept doing it). How is this vector initialized? Although it is easy for most students to see that group numbers need to be different at the outset, it is somewhat harder for many of them to come up with the simple Matlab line group = 1:ncities; that would do the job. I suspect this is because students do not yet realize that a creative engineer will generally know quite a few languages (e.g. visualization, sensory, math, logic, common sense logic, probability, and more in addition to natural language) because a given problem usually can be solved decisively in just one or two of them and they struggle to translate into the one or two languages they use for expressing thoughts. Actually, I think this assertion may explain a lot of other mysterious inabilities of some students to comprehend what seems like very simple, even obvious, concepts, even outside this course. Back to the problem at hand, we ask the question how is the group vector updated when an edge is added to the solution list? The answer is to use a 'for' loop which indexes every element of the group vector and changes it to the group number of one of the endpoints if it equals the group number of the other endpoint (which must be made constant before the loop). This had to be explained several times in lecture and labs. It was a difficult concept to comprehend. Apparently there was a lot of confusion caused by the use of two kinds of objects: edges and vertices. Several students tried to add vertices to the solution list (despite saying several times that the solution is a set of edges) and assign group numbers to edges (despite saying several times what the meaning of the group number was). However, most students were able to see the inefficiency of checking all vertices for changes in group numbers when only the group numbers of some of the vertices need to be changed. So, we moved on to present the concept of a single vertex 'representing' a group of vertices and change the notion of group to a reference to the representing vertex. The code for updating the structure was shown in lecture and did not seem to cause any more difficulty than the simpler group vector concept. However, the exam question on this code did not yield good results. The neuron simulation lab calls for saving computed data in a vector until the end of a run whereupon the data is analyzed statistically and plotted. 4. Students will explain the use of control constructs such as if, for, and while in algorithm design. All students have been able to tell me that a 'for' statement is used to repeat a sequence of executable statements. Many do not understand the use of a loop counter such as in 'for i=1:100' - for example, many students cannot successfully use 'i' to iterate through the elements of a vector - even though much effort has been spent in class discussing this topic from about the fourth lecture on. Most can recite a preference for using 'for' when the loop count is known in advance and 'while' when the condition for termination must be checked during each iteration. Most can recite the syntax of 'if'. All but about 10% can write close to working code for an elementary problem such as guessing a number that is chosen by some individual using yes/no questions. 5. Students will explain the intended output of several simple Matlab functions over all possible inputs. As stated above, all 10 exam problems tested this. Results were not uniform, even for the same individual. Many students had trouble understanding the following: function res = seq(n) res = [0 1]; for i=3:n if res(i-2) == 0 res = [res 0]; else res = [res -1*res(i-2)]; end end end Apparently the cause of the problem was inability to accept res(i) can depend on res(i-2). I cannot say why this is so difficult for students other than to suggest, as before, that many students rely on one or two languages to express thoughts and do not yet have experience in translating between languages. Perhaps the solution is to make creativity training an integral part of the intro courses that every discipline offers and requires. Application Goals: ------------------ 1. Students will be able to envision a solution to a given elementary engineering problem in terms of pseudo-code and turn that solution description into actual Matlab code. I now believe this goal is too ambitious. Perhaps 10% of the students should be expected to be able to turn pseudo-code into code, say Matlab or C++. Another 25% can be expected to do so with some help - 40% by looking at working code that solves similar problems, 60% with help from a person with significant experience. Another 25% at least understand that there are steps for writing code that include formulation of the problem, realizing an algorithmic solution with plausible chance of being efficient and correct, using pseudo-code to express the algorithm, and turning the algorithm into code. However expecting this group to succeed in any of those parts is asking too much. The remaining students appear to have no real interest in the technical component of the course except to check some box on some form. The goal should be changed to expose students to steps used to write code for solving a problem that has a solution in the less than 300 lines of Matlab category. 2. Students will be able to use arithmetic and logic operations to solve some basic engineering combinatorial problems. The goal of the last four labs was to test this. Students who took the course seriously successfully demonstrated the use of arithmetic and logic operations, particularly the difference between '.*' and '*' and '.^' and '^' operations. Students who did not take the course seriously simply submitted cut-and-pastes of posted solutions. Lab scores do not uniformly reflect this dichotomy. 3. Students will be able to code, run, and debug Matlab programs. This was successful for a majority of students due largely to help offered by two experienced teaching assistants. Even if details were not retained, at least students were exposed to the control of computers through software.