ACADEMIC PLANNING COUNCIL
Minutes of April 8, 2002
3 p.m., Holmes Student Center – Room 505
Present: Aase, Cassidy, Changnon, Deskis,
Goldenberg, House, Isabel, Jeris, Munroe, Newman-Ryan, Rintala, Scarborough,
Wheeler, Zar
Guests: Craig Barnard, Assessment Coordinator, Assessment Services; Carl Campbell, Assistant Chair, Department of Economics; Ed Furlong, Assessment Coordinator, College of Engineering and Engineering Technology; Eliakim Katz, Chair, Department of Economics; Fred Kitterle, Dean, College of Liberal Arts and Sciences; Vincent P. McGinn, Chair, Department of Electrical Engineering; Jamie Rothstein, Assistant to the Dean, College of Liberal Arts and Sciences; John Shaffer, Chair, Department of Physics; Promod Vohra, Associate Dean, College of Engineering and Engineering Technology
Goldenberg called the
meeting to order at 3:00 p.m. Cassidy announced that the provost and president
are at the Senate Appropriations meetings today.
Goldenberg noted that
Vice Provost Zar is retiring, and this is his last APC meeting. He thanked Zar for all of his hard work and
announced that on April 17 there will be a reception for Zar from 4:00-6:00
p.m. in the Regency Room.
It was moved and seconded
to approve the minutes of March 4, 2002, and the motion passed unanimously.
Goldenberg introduced
Promod Vohra, associate dean, College of Engineering and Engineering
Technology; Vincent McGinn, chair, Department of Electrical Engineering; Ed
Furlong, assessment coordinator, College of Engineering and Engineering
Technology; Fred Kitterle, dean, College of Liberal Arts and Sciences; John
Shaffer, chair, Department of Physics; Eliakim Katz, chair, Department of
Economics; Carl Campbell, assistant chair, Department of Economics; and Jamie
Rothstein, assistant to the dean, College of Liberal Arts and Sciences.
Goldberg stated that the
focus of the meeting today was the follow-up reports on certain aspects of
programs that had previously been reviewed by the APC. Cassidy indicated that
the three departments represented at the meeting today were reporting on progress
on the assessment of their programs, and that the Department of Economics was
also reporting on faculty scholarship and grant funding. Goldenberg then asked
for the progress report from the Department of Electrical Engineering.
Vohra said that based on
the recommendations from the associate vice provost the department felt it
would be wise to submit a supplemental report showing what the department has
done with assessment. McGinn added that
the assessment data represents a new look at how the program might be
envisioned. Our accrediting body, ABET,
has changed its emphasis to a high emphasis on design and the achievement of
objectives. ABET wants us to
demonstrate how these two items meet.
Employers are looking for engineers and scientists who can start doing a
job within two weeks of employment. In
the past, engineers received one to two years of on-the-job training. The
department has stayed with a classical electrical engineering program. We have
received consistent feedback from our advisory board that our alumni can indeed
do the job. Vohra added that assessment in the College of Engineering and
Engineering Technology is ongoing.
Vohra said that ABET only
accredits the primary engineering programs. He added that the college is
careful about closing the feedback loop regarding assessment. McGinn stated that there are peripheral
issues in dealing with some of the information ABET wants. ABET wants a team approach used. When our students were asked if a team
approach was taken, they were only looking at team approach experiences in the
classroom. If you asked students if
they worked as a team on a project, the students would reply yes.
Cassidy said that the
data exemplifies the program very well and that it is an applied program. The report is comprehensive. One question that wasn’t answered in the
report is how all of the pieces fit together as program assessment. In talking about the capstone course you
talk about how the objectives fit with the courses. You need to answer the question, how do the courses fit into the
entire program. The objectives of the
program still don’t seem to be clearly stated.
In the summer program review meeting we talked about what undergraduate students
can do that is different than graduate students, which is needed to
differentiate the programs; this question has still not been answered. Vohra responded that one way we looked at
students’ preparedness is looking at the courses students take for the capstone
experience. We also ask other questions
on the survey that we send them.
Cassidy said that the issue is that the program needs to be using
authentic assessment, not just surveys.
How are you sitting down and looking at the students’ work? Vohra replied that this is done by the
faculty. Cassidy added that the
department needs to talk about the assessment of the program, not courses.
McGinn said that some of the professors in the department think the Fundamental
Engineering (FE) examination is not important, yet the ABET standards use the
FE scores. Vohra added that if he sees consistency in feedback, this makes him
feel comfortable that the department is doing what they say they will do. This feedback is received from students,
faculty, and employers. Furlong said that when you are looking at ultimate
outcomes, the employer feedback is the key. It is difficult to go beyond the
attitudinal level. McGinn said that
there are various jobs that graduates with an electrical engineering degree can
obtain. Cassidy said that she didn’t think there was any question about the
quality of these data or the program.
The issue we are trying to address is that we need direct evidence of
students’ learning. This is also an IBHE mandate. As students are moving through the program, what can you hold in
your hand to show what percentage of the graduates meet this objective on a
standardized scale. May be you can work
with Craig Barnard at looking at some of the ways to do this. Scarborough added that an example would be a
design portfolio. McGinn replied that
the department has this, but I think you are looking for numerical data. Rintala said that the department might want
to design a scoring rubric. McGinn
replied that we have all the data. Is
this the depth of the information you want?
Rintala replied yes, this reflects on the program’s effectiveness. Vohra added that a matrix could be developed
for this. Cassidy replied that the
department, college, and university need evidence that says that the faculty
have looked at students’ achievement of the program outcomes across a period of
time. Vohra added that the department
will work with Craig Barnard on this.
Furlong said that the current protocols look at students’ attitudes, but
you need faculty to look at this.
Cassidy replied yes assessment should include students’ attitudes and
surveys can be used as an assessment method; they cannot be the only assessment
method used, however. The information
needs to be based primarily on the work that students have completed.
Newman-Ryan asked if the
department really wants to say that the ethical dimension does not arise in its
classes. This is a hot topic right
now. McGinn replied that this is what
he did want to say. The courses
themselves are technical. Newman-Ryan
said to take the example of someone drawing a weapon, someone else designing
it, and someone else manufacturing it. These are technical issues that also
have ethical implications. Vohra added that ABET has an engineering code of
ethics, and it is talked about indirectly.
It has been suggested that we have monthly workshops to address ethical
issues. This is one area that was
raised by ABET last time they were here.
We do need to integrate this into the program. Wheeler said that engineering does have a course in the recently
approved emphasis in applied ethics.
Vohra said that ethics is implicit in the program, but it needs to be
addressed more explicitly.
Goldenberg then directed
the council’s attention to the report from the Department of Economics.
Campbell said that the APC has requested a follow-up report from the department
on faculty research, external grant funding, and assessment. Campbell noted that the department currently
has 10 faculty from a total of 14 faculty who are active in publishing. Since June of 1998 the faculty have
published 2 books or monographs, 26 articles, 3 book chapters, and 1 book
review. Faculty have also made 73 presentations
of their work during this time period.
This work was presented internationally. The department has obtained more than $805,000 in external grant
funding since 1998, and 12 of 14 faculty members have submitted grants during
this period. After one more proposal is awarded, half of the faculty in the
department will have received grant funding.
Campbell reported that
regarding assessment, the department has started doing surveys of alumni, exit
interviews, and added questions to examinations. There is also a capstone research requirement for baccalaureate
majors. For the M.A. program the
department also has done a survey of current M.A. students. At the Ph.D. level dissertations are
reviewed by external evaluators in connection with program review. He added that the department found that one
weakness is writing skills, which came across in several assessment findings;
faculty are working on this now.
Another area of weakness identified was that students wish they had more
feedback on job opportunities. The department will work with the Career
Planning and Placement Center to provide presentations to students regarding
job opportunities. Cassidy complimented
the department and Campbell for the excellent plans they have submitted for all
three programs. She stated that the
department is on the right track, and now all of the plans need to be
implemented. Cassidy observed that the
number of presentations significantly increased since the full review of the
program. She asked if presentations
were more valuable than articles in terms of faculty scholarship. Katz replied that attending conferences and
workshops allows faculty to interact with other faculty, and this is very
valuable. It also provides external
monitoring for what faculty are doing.
The bottom line is that faculty want to publish in top rated journals,
and the department is doing this. He
added that he believed that the quality of the scholarship has increased, and
that the newer members of the department are more visible in the publication
list. The future is looking better with
regard to scholarship. Cassidy asked if
the four members who hadn’t published since the full review of the programs
were members of the graduate faculty.
Katz responded that he thought they were, and added that everyone in the
department is working on research.
Rintala noted that in the undergraduate assessment plan the department
talked about conducting exit interviews on a random sample of students. Campbell replied that they try to interview
10 students. One of the students did
not keep their interview appointment. The department has approximately 80
students in the undergraduate program.
Rintala asked if it would be helpful to have some other faculty involved
in the exit interview process. Campbell replied that this would be helpful.
Cassidy added that the assessment of the programs is not totally Campbell’s
responsibility. Katz noted that
Campbell coordinates assessment.
Goldenberg asked for the
report from the Department of Physics. Shaffer said that the department did not
have formal assessment plans in place at the time of the last program review,
and the APC required a follow-up report on the department’s assessment plans.
The assessment plans have been revised by a departmental committee. The
department has a capstone course in place for the B.S. degree program, and provides
general education courses too. We have
some knowledge of our native undergraduate students, but this is not the case
with our transfer students. We pretty
much know what the objectives are. We decided there were some intermediate
points we could evaluate, and the rest we would try to do in the capstone
course. Shaffer asked the faculty to evaluate the student’s laboratory skills,
their attitudes, and their writing skills as they progress in the program; we
will look at this information again when students are seniors. In the capstone course students have to
attend all the colloquia, and there is an examination that covers all 50 areas
that we feel are basic to physics students. There is also an exit interview.
Shaffer added that
students are required to write a research paper and give a 30-minute oral
presentation on a paper at the M.A. level. The program doesn’t have a large
number of graduates, and surveys are not that helpful to us. During the exit interview we want to try to
find a way to contact students in the future.
We will also contact employers.
We are going to have the alumni write a letter to us one year after
graduating, and we feel it is easier to be candid by writing a letter. We will talk to the students about this
during the exit interview. At the
graduate level we already assess students, and the department needs to look at
what we already do and add to it.
Shaffer stated that at the graduate level the department requires all
students to take the GRE examination in physics and pass a qualifying
examination. We need to show what the differences in skills are between the
B.S., M.S., and Ph.D. graduates. At the
M.S. level it is demonstrated that students can do some kind of independent research,
and a research thesis is required. We are considering having someone outside
NIU evaluate randomly selected theses. The dissertation already has a method
for external evaluation; the department thinks that every Ph.D. dissertation
should be sent to an outside reviewer.
Cassidy stated that the department had an excellent start. It would not take too much to turn what
you’re proposing into program assessment.
You need a bit of rethinking, but the pieces that you have here will
serve you very well. Zar added that the
department could have a non-NIU individual be part of a student’s dissertation
committee, which would provide an external validation of the quality of the research.
Cassidy asked how many
students were enrolled in the Ph.D. program.
Shaffer replied that he thought there were now 16 students in the Ph.D.
program. Zar asked if all of these
students were supported financially.
Shaffer responded yes, except for those who are full-time employees. Scarborough noted that it was impressive
that the department is willing to send out dissertations for external review.
Shaffer said that given the fact that the Ph.D. program is new, this would be a
good thing to do; sometimes new programs find that they are too demanding at
the outset. Kitterle said it is very clear that the department has been
earmarked for significant resources.
Shaffer said that at the end of this semester the department will have
some data. We do expect the number of master’s students to go down now that we
have a Ph.D. program.
Wheeler said that in the
spring of 1999 the department instituted pre- and post-tests in several
courses. You now have three years of
data from these courses; have there been any changes made in these courses? Shaffer replied no, these data are very
difficult to interpret, but we have tried to change the way we teach the
classes. Cassidy asked if these were department-based tests. Shaffer replied yes, and we don’t really
change the tests. The results tell us that we are teaching some physics, and
the scores for the second semester are better than the first semester. Zar asked how many of the PHYS250 students
go on to take PHYS251. Shaffer replied
that approximately 150 students take PHYS250 each semester and about 105 to 110
continue in PHYS251. We lose 20 to 25
percent of our enrollment from PHYS250 to PHYS251.
Rintala asked if the
department was concerned about the degree of honesty the department would get
from alumni writing letters. It might
be helpful to add some questions to provide guidance for responses. Shaffer said that one exit interview has
already been done, and the student was very candid. Jeris asked if there was a way to have relatively new majors
contact the alumni and ask them some of these questions. Shaffer replied that he doesn’t think the
department had enough majors, and this would be difficult; faculty need to do
the calling. Aase said that in the UBUS
course in the College of Business we try to do some testing on critical
thinking, and that he assumed that the laboratory writing skills are quite
different from other writing skills.
Shaffer responded that there are not many standard exams for
writing. Cassidy added that
implementation of the assessment plans is important because the university
needs to start reporting annually on where it is with the assessment of
outcomes. Shaffer was encouraged to meet with Craig Barnard and talk about how
to finalize and implement the plans.
Rintala informed the APC
that the NCA Steering Committee is in place.
Goldenberg and Deskis are both on the steering committee. The committee will meet next week for the
first time, but most of the work of the committee will start in the fall. Several steering committee members attended
the NCA meeting in Chicago at the end of March. Assessment was a key element in many of the sessions. Cassidy
added that the Higher Learning Commission is reviewing its accreditation
criteria and has proposed new criteria, which will be implemented in the fall
of 2004. The expectation is that we
will provide evidence of continuous improvement.
Cassidy reported on the
work of the University Assessment Panel. She announced that the Portfolio
Workshop will be held the first week in June.
This is the second year that the workshop has been offered. She added that over the next few years the
university will need much greater detail on reporting the outcomes of our
assessment plans. Over the last few
years programs have been invited to submit status reports to the UAP prior to
their full program review, however not all programs have taken advantage of
this opportunity. The IBHE has stated
that by 2004 all programs will show evidence that they are using the findings
from assessment to improve programs, and it expects annual reports. Because of
this requirement the UAP is putting new processes in place. An annual report on assessment-based
program changes will be required from each program. The annual reports will
then be submitted to the UAP with the four-year status reports, and the status
reports will be incorporated into the program review materials. The faculty
members on the APC subcommittees will see more of this kind of information in
the future. Cassidy added that beginning in fall 2002 assessment plans will be
attached to program review documents, and the program review itself will report
on the findings from assessment initiatives.
Jeris asked if there was ever any resistance to this type of demand from
the IBHE: how do we respond to these types of things? Cassidy replied that the academic officers were consulted by the
IBHE and they support the annual update.
They feel if it is not reported, it will not be done. She noted that
assessment plans need to be kept active, and we need to find a way to hold
people accountable but not by creating voluminous reports. It is unlikely that
the IBHE will mandate what format our report will take. Changnon added that the
general education programs are required to have some type of assessment.
Cassidy said that the academic officers added general education assessment to
the IBHE board item because they believe it is a critical part of the
undergraduate experience. Wheeler noted
that some departments have requested that the general education review be
synchronized with program review. Cassidy said that was a good suggestion and
that linking the review of faculty graduate status to program review has worked
well.
Zar’s poem follows.
As I look back upon year
after year of APC meetings and program reviews,
I think the discussions
make one thing quite clear,
That NIU colleagues have
multiple views.
They tell of their
programs, their missions and goals;
Their chairs and their
deans add support.
The APC questions, and
probes, and cajoles
To assess and improve,
they exhort.
I have sat through these
meetings, but now I am through
And I must say it’s not
clear to me
How I’ll expend energy,
and what I shall do
In future on Mondays at
three.
The meeting adjourned at
4:50 p.m.
Carolyn A. Cradduck
U\Planning\APCMinutes\April82002