20-CS-112 Winter Quarter 2009 J. Franco, Instructor
---------------------------------------------------------
Class Assessment Form
---------------------
Evaluation Note:
----------------
The aim of this course is to move students of differing engineering
backgrounds toward proficiency in general purpose programming of
solutions to engineering computational problems. By proficiency we
mean ability to use a general purpose language to solve basic
computational problems that will almost certainly arise in the
workplace and to correctly interpret the results of executing code on
a given problem input. To achieve this aim the following two points
became the dominant drivers for the course:
1. Teach just a few of the constructs of Matlab. This includes
control: if-then-else, for, while, fscanf, fprintf, functions
and their use.
2. Code efficient solutions to several non-trivial engineering
problems in labs.
Important subpoints include:
1. Show how data can be represented as matrices and that different
representations can have a remarkably different effect on the
development of a program and its performance
2. Show how auxilliary data structures, not directly related to the
specification of the given problem, for example queues, stacks, and
lists, can be used to help formulate a concise, efficient
solution to the given problem.
Due to the size of the class, lack of an adequate number of TAs, and
that we do not expect students to be able to solve the engineering
problems that are posed in labs without help, solutions to all labs
were posted on the course website. Students were told not to view
these solutions unless they were stuck and, in case they did view the
solutions, not to turn in cut-and-paste copies as their submissions.
Posted solutions contain some useless segments of code which, if present,
indicated that a submission was not original.
Also due to the size of the class the only practical way to examine
students was to use multiple choice questions. However, this form of
evaluation can, at best, only approximate true assessment. In
addition, the original grader was overwhelmed by the volume of
submissions and did not adequately check for original solutions. This
was corrected for the last two labs because the original grader was
replaced near the end of the quarter. Because of this, my broad
course assessments rely more on impressions resulting from personal
and in-class conversations with students and on comments of the two
TAs that took care of the labs (I was fortunate to have two very good
students as TAs). Hence, I do not use exam or lab scores in comments
assessing the course outcomes below.
Outcome Summary:
----------------
Success in meeting the stated goal seemed to correlate highly with the
intended major of the student as did the nature of the relationship
between instructor and student. For example, the instructor had a
positive relationship with most Aerospace students who seemed eager to
understand what ideas the instructor was trying to convey and even
invited the instructor to speak to their learning community. By
contrast, the instructor experienced extraordinary resistance from
Civil Engineering students who actually tried to convince the
instructor that probability and other important topics have no place
in Civil Engineering.
Improvement Summary:
--------------------
Many freshmen and sophomores do not really know why they are here
except they are expecting a high paying job upon graduation. The idea
that employers will expect results has only vaguely crossed the minds
of many. This course indirectly exposes to the student the mission of
the College of Engineering which is to train people for creative
engineering work that advances the state-of-the-art, or supports
high-level design or technical consulting. It is expected that some
students will use the results of this course either
1. to adjust study habits, if necessary and if they intend to buy
into this mission and remain in the College of Engineering, or
2. to move on to a field that their talents and interests are more
suited to, for example one of the specialties of the College of
Applied Science or Information Science as taught by the business
school.
The course could benefit from a more rigorous recruitment program that
1) provides serious scholarships and other encouragement to high
school students that clearly are suited to our career targets; 2)
screens students via serious interviews that determine the best match
of curriculum for given student talents; 3) encourages contact with
elementary and middle school students who show signs of strong talent
and interest in engineering concepts. In addition, because of the
differences in perceptions of various majors, as noted above, it might
be advantageous to have smaller, homegeneous groups of students
attending lectures.
Knowledge goals:
----------------
1. Students will know the principal constructs of Matlab including
if-then-else, for, while, fscanf, fprintf, and functions, and
important operators including +,-,*,/,^,.*,.^, and [].
All of the lab assignments test the ability to write code that
relies on these constructs and operations: the first lab is a
simple test of arithmetic operations on scalars and vectors, the
second advances to plotting sin and cos functions on vectors
obtained from linspace, the third tests loops and if-then-else
statements as well as 2D plots, the fourth calls for solving the
first of four engineering plots, namely interpolation of well log
data over a gridded volume, the fifth calls for reconizing amino
acid patterns in DNA strings read from file using nested loops,
the sixth calls for finding a minimum cost network from matrix
data using a sorting function and loops, and the last homework
calls for simulating neuron firing in the brain using
probabilistic functions. In addition, nearly all exam questions
test the ability to recognize what function a series of
operations is intended to perform.
Lab success rate was high but, unfortunately, near the end of the
quarter it seems a high percentage of students began resorting to
submission of cut-and-paste copies of posted solutions. On the
other hand, exam results were spread over a wide range of grades.
This is to be expected since, as was mentioned earlier, students
are not uniformly prepared for an engineering curriculum, both
intellectually and spiritually.
2. Students will know the difference between elementary Matlab data
structures including vectors, matrices, and scalars.
What has been said above for constructs and operations applies
here as well. More specifically, the DNA search lab problem has
input data expressed as a character string, the minimum cost
network problem as a matrix with rows indexed on edges and
columns indexed on vertices and cost, the interpolation
problem as a matrix with rows indexed on individual well data and
columns indexed on well position and sensor reading, and the 5
questions problem of lab 2 as a sequence of scalars. Alternative
representations were considered in individual labs as needed.
These may be found at
http://gauss.ececs.uc.edu/Courses/c112/extra/
http://gauss.ececs.uc.edu/Courses/c112/labs/
3. Students will know how to represent input data, in the form of a
matrix, vector, or scalar, so as to minimize the time needed to
make the data useful to a program.
This point was discussed in class but not formally tested in labs
and exams. I did not formally test this because I feared total
disaster. If I could get a majority of students to write Matlab
code for some simple problems I would be overjoyed. To expect
students to grasp an analysis of all possible encodings of data
is probably asking too much.
4. Students will know how to use a modeling language such as Matlab
to set up solutions to some combinatorial problems that are
simplified versions of actual engineering problems.
Four labs present such problems including the search problem of
finding amino acids in a DNA strand, the optimization problem of
finding a minimum cost network given a weighted undirected graph,
simulation of neuron behavior assuming some underlying
probability distributions, and interpolating well log data over a
gridded volume and visualizing the result. Possible solutions to
these problems were discussed during lectures. Additional help
was given during labs. There is no doubt that these problems, as
easy as they are for a professional, present a stiff challenge to
about 80% of the students in the class. I believe this challenge
is necessary if these students are going to seriously use
programming as a general purpose engineering tool later (which I
believe nearly all will).
As said earlier, this approach meets with resistance from
substantially many students, perhaps as many as 30%, despite the
fact that the lab problems are obviously engineering problems and
the fact that I attempt to illustrate further in lecture how this
tool is going to be used in some disciplines - for example by
showing NASA's efforts to verify their space vehicle design and
control software. From lengthy and many discussions with
students I believe the problem is rooted in improper recruitment
and inadequate guidance of new students. Too many students are
in the College of Engineering for reasons other than what the
faculty have in mind. These reasons might be the College of
Engineering is prestigious, a graduate is likely to make more
money, co-op defrays the cost of an education and so on. Given
this, it is not surprising that I cannot get traction with a
substantial number of students.
I see three things that can be done to improve the percentage of
students that are happy with the course given that recruitment
and advising are not going to change. First, water the course
down by removing non-trivial engineering problems from the labs.
This will help only slightly as it is clear from both exams that
significantly many students are befuddled by the simplest use of
Matlab constructs. Second, reduce lecture size, make sure each
lecture is populated by students of one major, and tailor the
lectures to that major. Having an instructor from that major may
help. FEPs could choose which lectures they will attend. Third,
offer the course later in the curriculum. Students may take this
course more seriously if they see themselves handicapped on a
co-op job because they do not know Matlab or some other
programming tool. In such a high-pressure situation they will be
forced into the unpleasant reality of desperately needing to cram
into their brains some rudimentary knowledge of programming.
People in such a position are likely to take a formal course in
programming more seriously when they return to take classes. Of
these three alternatives the second seems best as it attempts to
avoid punishment of students and creates a smaller, homogeneous
and therefore better learning environment for the student. The
only drawback is that more faculty resources are required to
implement that strategy.
5. Students will know that algorithms are used to solve combinatorial
problems.
The last four labs ask the student to solve engineering problems
in Matlab. The solutions begin with an English language
description of a solution using mathematical structures such as
sets and lists and operations that move objects such as cables
and cities into and out of those structures. The next step is to
turn the English into pseudo-code which clearly expresses the
algorithm for solving the problem.
Although nearly all students realize the algorithmic nature of
programming solutions, few can produce an algorithm for a
non-trivial problem that they have not seen before and
essentially none can prove correctness. Expecting this from
freshmen and sophomores is probably unrealistic.
Comprehension Goals:
--------------------
A midterm and final exam were given. There were 4 problems on
the midterm and 6 on the final. Each problem specified code
comprising a Matlab function. There were five questions
associated with each problem. One of those questions asked for
the result of applying the corresponding function to some all or
possible inputs. Another question asked for the type of some
variable or structure that was used by the function. Other
questions were concerned with the process of the function.
1. Students will explain how problem data is represented and used
by a program.
Exam questions were intended to test this. Results were not
uniform, even for the same individual. This probably reflects
the flawed nature of the exam type - for some people, on a
multiple choice exam even some obviously wrong answers look
reasonable when seen in print. On the other hand, with 166
students it is hard to imagine using another form of written exam
that would not take an impossible amount of time to make up and
grade.
2. Students will explain how problem data is read from a file.
It was unnatural for students to see statements like this:
fid = fopen('filename','rt');
npts = fscanf(fid,'%d',1);
for i=1:npts
data(i,:) = fscanf(fid,'%f',3);
end
and it took a considerable amount of time for many students to
internalize this, particularly the format specifiers of '%d' and
'%f', which were the only ones we used, and the ':', which was
used often. Although indexing a vector like this: v(i) was not
too hard for most students, indexing a matrix like this: v(i,j)
seemed to cause a lot more difficulty.
Labs 4,5, and 6 required reading data from a file. Also, extra
material that was covered in some labs and shown here:
http://gauss.ececs.uc.edu/Courses/c112/extra/
helped with this.
The file reading problem on the final exam seemed to be one of
the easiest for students.
3. Students will explain how auxilliary data structures assist in
solving a problem.
The last two labs were intended to test this. The minimum cost
network problem calls for establishing a list of edges (presented
in the lab as transmission cables), initially empty, that will be
output as the solution. Edges are considered one at a time; an
edge is added to the solution list if it does not connect
vertices (presented in the lab as cities) that are already
connected by the edges in the solution list. The student is
confronted with the question how can this test be made? One
answer is to assign to each vertex a group number and define the
invariant: two vertices have the same group number if and only if
they are connected by edges in the solution list. How is the
group number handled? A simple way to do this is to use a vector
where the ith element is the group number of vertex i (the
convention of using numbers to represent vertex identities so as
to make it easy to access data in vectors and matrices is
apparently easy for students to comprehend - at least they seemed
to readily accept doing it).
How is this vector initialized? Although it is easy for most
students to see that group numbers need to be different at the
outset, it is somewhat harder for many of them to come up with
the simple Matlab line
group = 1:ncities;
that would do the job. I suspect this is because students do not
yet realize that a creative engineer will generally know quite a
few languages (e.g. visualization, sensory, math, logic, common
sense logic, probability, and more in addition to natural
language) because a given problem usually can be solved
decisively in just one or two of them and they struggle to
translate into the one or two languages they use for expressing
thoughts. Actually, I think this assertion may explain a lot of
other mysterious inabilities of some students to comprehend what
seems like very simple, even obvious, concepts, even outside this
course.
Back to the problem at hand, we ask the question how is the group
vector updated when an edge is added to the solution list? The
answer is to use a 'for' loop which indexes every element of the
group vector and changes it to the group number of one of the
endpoints if it equals the group number of the other endpoint
(which must be made constant before the loop). This had to be
explained several times in lecture and labs. It was a difficult
concept to comprehend. Apparently there was a lot of confusion
caused by the use of two kinds of objects: edges and vertices.
Several students tried to add vertices to the solution list
(despite saying several times that the solution is a set of
edges) and assign group numbers to edges (despite saying several
times what the meaning of the group number was). However, most
students were able to see the inefficiency of checking all
vertices for changes in group numbers when only the group numbers
of some of the vertices need to be changed. So, we moved on to
present the concept of a single vertex 'representing' a group of
vertices and change the notion of group to a reference to the
representing vertex. The code for updating the structure was
shown in lecture and did not seem to cause any more difficulty
than the simpler group vector concept. However, the exam
question on this code did not yield good results.
The neuron simulation lab calls for saving computed data in a
vector until the end of a run whereupon the data is analyzed
statistically and plotted.
4. Students will explain the use of control constructs such as if,
for, and while in algorithm design.
All students have been able to tell me that a 'for' statement is
used to repeat a sequence of executable statements. Many do not
understand the use of a loop counter such as in 'for i=1:100' -
for example, many students cannot successfully use 'i' to iterate
through the elements of a vector - even though much effort has
been spent in class discussing this topic from about the fourth
lecture on. Most can recite a preference for using 'for' when
the loop count is known in advance and 'while' when the condition
for termination must be checked during each iteration. Most can
recite the syntax of 'if'. All but about 10% can write close to
working code for an elementary problem such as guessing a number
that is chosen by some individual using yes/no questions.
5. Students will explain the intended output of several simple Matlab
functions over all possible inputs.
As stated above, all 10 exam problems tested this. Results were
not uniform, even for the same individual. Many students had
trouble understanding the following:
function res = seq(n)
res = [0 1];
for i=3:n
if res(i-2) == 0
res = [res 0];
else
res = [res -1*res(i-2)];
end
end
end
Apparently the cause of the problem was inability to accept
res(i) can depend on res(i-2). I cannot say why this is so
difficult for students other than to suggest, as before, that
many students rely on one or two languages to express thoughts
and do not yet have experience in translating between languages.
Perhaps the solution is to make creativity training an integral
part of the intro courses that every discipline offers and
requires.
Application Goals:
------------------
1. Students will be able to envision a solution to a given
elementary engineering problem in terms of pseudo-code and
turn that solution description into actual Matlab code.
I now believe this goal is too ambitious. Perhaps 10% of the
students should be expected to be able to turn pseudo-code into
code, say Matlab or C++. Another 25% can be expected to do so
with some help - 40% by looking at working code that solves
similar problems, 60% with help from a person with significant
experience. Another 25% at least understand that there are steps
for writing code that include formulation of the problem,
realizing an algorithmic solution with plausible chance of being
efficient and correct, using pseudo-code to express the
algorithm, and turning the algorithm into code. However
expecting this group to succeed in any of those parts is asking
too much. The remaining students appear to have no real interest
in the technical component of the course except to check some box
on some form.
The goal should be changed to expose students to steps used to
write code for solving a problem that has a solution in the less
than 300 lines of Matlab category.
2. Students will be able to use arithmetic and logic operations to
solve some basic engineering combinatorial problems.
The goal of the last four labs was to test this. Students who
took the course seriously successfully demonstrated the use of
arithmetic and logic operations, particularly the difference
between '.*' and '*' and '.^' and '^' operations. Students who
did not take the course seriously simply submitted cut-and-pastes
of posted solutions. Lab scores do not uniformly reflect this
dichotomy.
3. Students will be able to code, run, and debug Matlab programs.
This was successful for a majority of students due largely to
help offered by two experienced teaching assistants. Even if
details were not retained, at least students were exposed to the
control of computers through software.