ACADEMIC PLANNING COUNCIL
Minutes of April 8, 2002
3 p.m., Holmes Student Center Ė Room 505
Present:†††††††††††† Aase, Cassidy, Changnon, Deskis, Goldenberg, House, Isabel, Jeris, Munroe, Newman-Ryan, Rintala, Scarborough, Wheeler, Zar
Guests:††††††††††††† Craig Barnard, Assessment Coordinator, Assessment Services; Carl Campbell, Assistant Chair, Department of Economics; Ed Furlong, Assessment Coordinator, College of Engineering and Engineering Technology; Eliakim Katz, Chair, Department of Economics; Fred Kitterle, Dean, College of Liberal Arts and Sciences; Vincent P. McGinn, Chair, Department of Electrical Engineering; Jamie Rothstein, Assistant to the Dean, College of Liberal Arts and Sciences; John Shaffer, Chair, Department of Physics; Promod Vohra, Associate Dean, College of Engineering and Engineering Technology
Goldenberg called the meeting to order at 3:00 p.m. Cassidy announced that the provost and president are at the Senate Appropriations meetings today.†
Goldenberg noted that Vice Provost Zar is retiring, and this is his last APC meeting.† He thanked Zar for all of his hard work and announced that on April 17 there will be a reception for Zar from 4:00-6:00 p.m. in the Regency Room.
It was moved and seconded to approve the minutes of March 4, 2002, and the motion passed unanimously.
Goldenberg introduced Promod Vohra, associate dean, College of Engineering and Engineering Technology; Vincent McGinn, chair, Department of Electrical Engineering; Ed Furlong, assessment coordinator, College of Engineering and Engineering Technology; Fred Kitterle, dean, College of Liberal Arts and Sciences; John Shaffer, chair, Department of Physics; Eliakim Katz, chair, Department of Economics; Carl Campbell, assistant chair, Department of Economics; and Jamie Rothstein, assistant to the dean, College of Liberal Arts and Sciences.†
Goldberg stated that the focus of the meeting today was the follow-up reports on certain aspects of programs that had previously been reviewed by the APC. Cassidy indicated that the three departments represented at the meeting today were reporting on progress on the assessment of their programs, and that the Department of Economics was also reporting on faculty scholarship and grant funding. Goldenberg then asked for the progress report from the Department of Electrical Engineering.
Vohra said that based on the recommendations from the associate vice provost the department felt it would be wise to submit a supplemental report showing what the department has done with assessment.† McGinn added that the assessment data represents a new look at how the program might be envisioned.† Our accrediting body, ABET, has changed its emphasis to a high emphasis on design and the achievement of objectives.† ABET wants us to demonstrate how these two items meet.† Employers are looking for engineers and scientists who can start doing a job within two weeks of employment.† In the past, engineers received one to two years of on-the-job training. The department has stayed with a classical electrical engineering program. We have received consistent feedback from our advisory board that our alumni can indeed do the job. Vohra added that assessment in the College of Engineering and Engineering Technology is ongoing.†
Vohra said that ABET only accredits the primary engineering programs. He added that the college is careful about closing the feedback loop regarding assessment.† McGinn stated that there are peripheral issues in dealing with some of the information ABET wants.† ABET wants a team approach used.† When our students were asked if a team approach was taken, they were only looking at team approach experiences in the classroom.† If you asked students if they worked as a team on a project, the students would reply yes.
Cassidy said that the data exemplifies the program very well and that it is an applied program.† The report is comprehensive.† One question that wasnít answered in the report is how all of the pieces fit together as program assessment.† In talking about the capstone course you talk about how the objectives fit with the courses.† You need to answer the question, how do the courses fit into the entire program.† The objectives of the program still donít seem to be clearly stated.† In the summer program review meeting we talked about what undergraduate students can do that is different than graduate students, which is needed to differentiate the programs; this question has still not been answered.† Vohra responded that one way we looked at studentsí preparedness is looking at the courses students take for the capstone experience.† We also ask other questions on the survey that we send them.† Cassidy said that the issue is that the program needs to be using authentic assessment, not just surveys.† How are you sitting down and looking at the studentsí work?† Vohra replied that this is done by the faculty.† Cassidy added that the department needs to talk about the assessment of the program, not courses. McGinn said that some of the professors in the department think the Fundamental Engineering (FE) examination is not important, yet the ABET standards use the FE scores. Vohra added that if he sees consistency in feedback, this makes him feel comfortable that the department is doing what they say they will do.† This feedback is received from students, faculty, and employers. Furlong said that when you are looking at ultimate outcomes, the employer feedback is the key. It is difficult to go beyond the attitudinal level.† McGinn said that there are various jobs that graduates with an electrical engineering degree can obtain. Cassidy said that she didnít think there was any question about the quality of these data or the program.† The issue we are trying to address is that we need direct evidence of studentsí learning. This is also an IBHE mandate.† As students are moving through the program, what can you hold in your hand to show what percentage of the graduates meet this objective on a standardized scale.† May be you can work with Craig Barnard at looking at some of the ways to do this.† Scarborough added that an example would be a design portfolio.† McGinn replied that the department has this, but I think you are looking for numerical data.† Rintala said that the department might want to design a scoring rubric.† McGinn replied that we have all the data.† Is this the depth of the information you want?† Rintala replied yes, this reflects on the programís effectiveness.† Vohra added that a matrix could be developed for this.† Cassidy replied that the department, college, and university need evidence that says that the faculty have looked at studentsí achievement of the program outcomes across a period of time.† Vohra added that the department will work with Craig Barnard on this.† Furlong said that the current protocols look at studentsí attitudes, but you need faculty to look at this.† Cassidy replied yes assessment should include studentsí attitudes and surveys can be used as an assessment method; they cannot be the only assessment method used, however.† The information needs to be based primarily on the work that students have completed.†
Newman-Ryan asked if the department really wants to say that the ethical dimension does not arise in its classes.† This is a hot topic right now.† McGinn replied that this is what he did want to say.† The courses themselves are technical.† Newman-Ryan said to take the example of someone drawing a weapon, someone else designing it, and someone else manufacturing it. These are technical issues that also have ethical implications. Vohra added that ABET has an engineering code of ethics, and it is talked about indirectly.† It has been suggested that we have monthly workshops to address ethical issues.† This is one area that was raised by ABET last time they were here.† We do need to integrate this into the program.† Wheeler said that engineering does have a course in the recently approved emphasis in applied ethics.† Vohra said that ethics is implicit in the program, but it needs to be addressed more explicitly.†
Goldenberg then directed the councilís attention to the report from the Department of Economics. Campbell said that the APC has requested a follow-up report from the department on faculty research, external grant funding, and assessment.† Campbell noted that the department currently has 10 faculty from a total of 14 faculty who are active in publishing.† Since June of 1998 the faculty have published 2 books or monographs, 26 articles, 3 book chapters, and 1 book review.† Faculty have also made 73 presentations of their work during this time period.† This work was presented internationally.† The department has obtained more than $805,000 in external grant funding since 1998, and 12 of 14 faculty members have submitted grants during this period. After one more proposal is awarded, half of the faculty in the department will have received grant funding.
Campbell reported that regarding assessment, the department has started doing surveys of alumni, exit interviews, and added questions to examinations.† There is also a capstone research requirement for baccalaureate majors.† For the M.A. program the department also has done a survey of current M.A. students.† At the Ph.D. level dissertations are reviewed by external evaluators in connection with program review.† He added that the department found that one weakness is writing skills, which came across in several assessment findings; faculty are working on this now.† Another area of weakness identified was that students wish they had more feedback on job opportunities. The department will work with the Career Planning and Placement Center to provide presentations to students regarding job opportunities.† Cassidy complimented the department and Campbell for the excellent plans they have submitted for all three programs.† She stated that the department is on the right track, and now all of the plans need to be implemented.† Cassidy observed that the number of presentations significantly increased since the full review of the program.† She asked if presentations were more valuable than articles in terms of faculty scholarship.† Katz replied that attending conferences and workshops allows faculty to interact with other faculty, and this is very valuable.† It also provides external monitoring for what faculty are doing.† The bottom line is that faculty want to publish in top rated journals, and the department is doing this.† He added that he believed that the quality of the scholarship has increased, and that the newer members of the department are more visible in the publication list.† The future is looking better with regard to scholarship.† Cassidy asked if the four members who hadnít published since the full review of the programs were members of the graduate faculty.† Katz responded that he thought they were, and added that everyone in the department is working on research.† Rintala noted that in the undergraduate assessment plan the department talked about conducting exit interviews on a random sample of students.† Campbell replied that they try to interview 10 students.† One of the students did not keep their interview appointment. The department has approximately 80 students in the undergraduate program.† Rintala asked if it would be helpful to have some other faculty involved in the exit interview process. Campbell replied that this would be helpful. Cassidy added that the assessment of the programs is not totally Campbellís responsibility.† Katz noted that Campbell coordinates assessment.
Goldenberg asked for the report from the Department of Physics. Shaffer said that the department did not have formal assessment plans in place at the time of the last program review, and the APC required a follow-up report on the departmentís assessment plans. The assessment plans have been revised by a departmental committee. The department has a capstone course in place for the B.S. degree program, and provides general education courses too.† We have some knowledge of our native undergraduate students, but this is not the case with our transfer students.† We pretty much know what the objectives are. We decided there were some intermediate points we could evaluate, and the rest we would try to do in the capstone course. Shaffer asked the faculty to evaluate the studentís laboratory skills, their attitudes, and their writing skills as they progress in the program; we will look at this information again when students are seniors.† In the capstone course students have to attend all the colloquia, and there is an examination that covers all 50 areas that we feel are basic to physics students. There is also an exit interview.†
Shaffer added that students are required to write a research paper and give a 30-minute oral presentation on a paper at the M.A. level. The program doesnít have a large number of graduates, and surveys are not that helpful to us.† During the exit interview we want to try to find a way to contact students in the future.† We will also contact employers.† We are going to have the alumni write a letter to us one year after graduating, and we feel it is easier to be candid by writing a letter.† We will talk to the students about this during the exit interview.† At the graduate level we already assess students, and the department needs to look at what we already do and add to it.† Shaffer stated that at the graduate level the department requires all students to take the GRE examination in physics and pass a qualifying examination. We need to show what the differences in skills are between the B.S., M.S., and Ph.D. graduates.† At the M.S. level it is demonstrated that students can do some kind of independent research, and a research thesis is required. We are considering having someone outside NIU evaluate randomly selected theses. The dissertation already has a method for external evaluation; the department thinks that every Ph.D. dissertation should be sent to an outside reviewer.† Cassidy stated that the department had an excellent start.† It would not take too much to turn what youíre proposing into program assessment.† You need a bit of rethinking, but the pieces that you have here will serve you very well.† Zar added that the department could have a non-NIU individual be part of a studentís dissertation committee, which would provide an external validation of the quality of the research.†
Cassidy asked how many students were enrolled in the Ph.D. program.† Shaffer replied that he thought there were now 16 students in the Ph.D. program.† Zar asked if all of these students were supported financially.† Shaffer responded yes, except for those who are full-time employees.† Scarborough noted that it was impressive that the department is willing to send out dissertations for external review. Shaffer said that given the fact that the Ph.D. program is new, this would be a good thing to do; sometimes new programs find that they are too demanding at the outset. Kitterle said it is very clear that the department has been earmarked for significant resources.† Shaffer said that at the end of this semester the department will have some data. We do expect the number of masterís students to go down now that we have a Ph.D. program.†
Wheeler said that in the spring of 1999 the department instituted pre- and post-tests in several courses.† You now have three years of data from these courses; have there been any changes made in these courses?† Shaffer replied no, these data are very difficult to interpret, but we have tried to change the way we teach the classes. Cassidy asked if these were department-based tests.† Shaffer replied yes, and we donít really change the tests. The results tell us that we are teaching some physics, and the scores for the second semester are better than the first semester.† Zar asked how many of the PHYS250 students go on to take PHYS251.† Shaffer replied that approximately 150 students take PHYS250 each semester and about 105 to 110 continue in PHYS251.† We lose 20 to 25 percent of our enrollment from PHYS250 to PHYS251.
Rintala asked if the department was concerned about the degree of honesty the department would get from alumni writing letters.† It might be helpful to add some questions to provide guidance for responses.† Shaffer said that one exit interview has already been done, and the student was very candid.† Jeris asked if there was a way to have relatively new majors contact the alumni and ask them some of these questions.† Shaffer replied that he doesnít think the department had enough majors, and this would be difficult; faculty need to do the calling.† Aase said that in the UBUS course in the College of Business we try to do some testing on critical thinking, and that he assumed that the laboratory writing skills are quite different from other writing skills.† Shaffer responded that there are not many standard exams for writing.† Cassidy added that implementation of the assessment plans is important because the university needs to start reporting annually on where it is with the assessment of outcomes. Shaffer was encouraged to meet with Craig Barnard and talk about how to finalize and implement the plans.†
Rintala informed the APC that the NCA Steering Committee is in place.† Goldenberg and Deskis are both on the steering committee.† The committee will meet next week for the first time, but most of the work of the committee will start in the fall.† Several steering committee members attended the NCA meeting in Chicago at the end of March.† Assessment was a key element in many of the sessions. Cassidy added that the Higher Learning Commission is reviewing its accreditation criteria and has proposed new criteria, which will be implemented in the fall of 2004.† The expectation is that we will provide evidence of continuous improvement.
Cassidy reported on the work of the University Assessment Panel. She announced that the Portfolio Workshop will be held the first week in June.† This is the second year that the workshop has been offered.† She added that over the next few years the university will need much greater detail on reporting the outcomes of our assessment plans.† Over the last few years programs have been invited to submit status reports to the UAP prior to their full program review, however not all programs have taken advantage of this opportunity.† The IBHE has stated that by 2004 all programs will show evidence that they are using the findings from assessment to improve programs, and it expects annual reports. Because of this requirement the UAP is putting new processes in place.†† An annual report on assessment-based program changes will be required from each program. The annual reports will then be submitted to the UAP with the four-year status reports, and the status reports will be incorporated into the program review materials. The faculty members on the APC subcommittees will see more of this kind of information in the future. Cassidy added that beginning in fall 2002 assessment plans will be attached to program review documents, and the program review itself will report on the findings from assessment initiatives.† Jeris asked if there was ever any resistance to this type of demand from the IBHE: how do we respond to these types of things?† Cassidy replied that the academic officers were consulted by the IBHE and they support the annual update.† They feel if it is not reported, it will not be done. She noted that assessment plans need to be kept active, and we need to find a way to hold people accountable but not by creating voluminous reports. It is unlikely that the IBHE will mandate what format our report will take. Changnon added that the general education programs are required to have some type of assessment. Cassidy said that the academic officers added general education assessment to the IBHE board item because they believe it is a critical part of the undergraduate experience.† Wheeler noted that some departments have requested that the general education review be synchronized with program review. Cassidy said that was a good suggestion and that linking the review of faculty graduate status to program review has worked well.
Zarís poem follows.
As I look back upon year after year of APC meetings and program reviews,
I think the discussions make one thing quite clear,
That NIU colleagues have multiple views.
They tell of their programs, their missions and goals;
Their chairs and their deans add support.
The APC questions, and probes, and cajoles
To assess and improve, they exhort.
I have sat through these meetings, but now I am through
And I must say itís not clear to me
How Iíll expend energy, and what I shall do
In future on Mondays at three.
The meeting adjourned at 4:50 p.m.†
Carolyn A. Cradduck