AA - Academic Program Review

Introduction

State law (see Texas Administrative Code in the Appendices) requires that all public doctoral programs and stand-alone master’s programs be externally reviewed on a 10-year cycle. Texas A&M University has elected to review all degree programs, including bachelor’s degrees, within the scope of the Academic Program Review (APR). Programs reviewed by an external accrediting body are not part of the formal APR process described here. However, the findings of any external accrediting body should be submitted to the Provost’s office as evidence of compliance with Texas Administrative Code for the accredited program.

The APR process, coordinated in the Office of the Provost and Vice President for Academic Affairs, aligns with, and complements, existing measures for assessment and quality enhancement. It facilitates discussion about change and strategies for improvement, and it provides the basis for making strategic decisions based on an extensive review from various constituents. APR is an opportunity to realign actions with current priorities of the university, colleges, departments, and programs as these are articulated in mission statements and strategic plans. APR underpins the process of achieving the university’s goal of continuously improving the quality of all academic programs in the pursuit of excellence.

APR Administrative Team

The APR Administrative Team provides a university perspective for review teams and programs undergoing review.

DR. ALAN SAMS

Interim Provost and Vice President for Academic Affairs
(979) 845-4016
asams@tamu.edu
DR. ALICIA M. DORSEY
Associate Vice President for Academic Effectiveness and Planning, and SACSCOC Accreditation Liaison
(979) 845-4016
amdorsey@tamu.edu
DR. FUHUI TONG
Interim Associate Provost and Dean of the Graduate and Professional School
(979) 845-4016
fuhuitong@tamu.edu
DR. TIMOTHY P. SCOTT

Vice Provost for Student Success
(979) 845-4016
t-scott@tamu.edu
DR. NK ANAND

Vice President for Faculty Affairs
(979) 845-4274
nkanand@tamu.edu
MS. BETTYANN C. ZITO

APR Coordinator
Office of the Provost
(979) 845-4016
apr@tamu.edu

Responsibilities

Office of the Provost

The Interim Provost and Vice President for Academic Affairs oversees the APR process.

Approving nominations of external reviewers
Hosting entry and exit meetings with members of the External Review Team
Hosting the post-review meeting
Submitting the Institutional Response to the THECB

The Associate Vice President for Academic Effectiveness & Planning is the primary point of contact for all review activities.

Coordinating the 10-year review cycle in consultation with the Texas Higher Education Coordinating Board (THECB)
Assisting with self-study development
Reviewing self-studies for feedback and final approval before distribution
Preparing Institutional Response in collaboration with program(s), college(s) leadership, and the APR Administrative Team

The Associate Provost and Dean of the Graduate & Professional School and Vice Provost for Student Success work with the Office of the Provost to provide support and guidance to programs undergoing review. They help ensure that reviews of programs and other educational and teaching priorities, faculty expertise and research priorities, and service and outreach programs are appropriate to the individual program undergoing review.

The APR Coordinator in the Office of the Provost provides logistical support for the review. The APR Coordinator works directly with programs undergoing review to assist with preparations for the review and with monitoring progress throughout the entire review cycle. The specific responsibilities of the coordinator include:

Assisting with logistical arrangements
Scheduling academic program review dates in consultation with the program
Scheduling all meetings involving the APR Administrative Team
Contacting potential reviewers on behalf of the Provost
Facilitating external review team lodging and travel

The College(s)

The college dean's responsibilities include:

Approving the program’s nomination of peer reviewers
Reviewing and approving the self-study
Meeting with the External Review Team during the site visit
Participating in the post-review meeting with the APR Administrative Team
Approving follow-up correspondence from the program and monitoring strategies and outcomes in response to the APR and providing additional input from the college’s perspective.

All documentation must be reviewed and approved by the college dean before it is forwarded to the Provost Office.

The Degree Program

Program leadership has significant responsibilities in preparing for the review and in executing the visit. To assist with this effort, degree programs should closely follow the procedures outlined below.

Approximate Time Frame	Responsibility	Comments & Reference
12 months prior	Attend APR Orientation Session
10-12 months prior	Choose review dates	Selected in consultation with the dean’s office and members of the APR Administrative Team.
9 months prior	Nominate Members for the External Review Team	See “Nominating the External Review Team”
8 months prior	Submit the “Program Overview”	As the review team is finalized, the APR Coordinator will forward the “Program Overview” template to the program to complete and return. It should briefly cover the program overview and degrees offered.
7-9 months prior	Organize, Initiate, and Write Self Study	See "Self Study"
2.5 months prior	Submit Draft of Self-Study to the Dean then to the Associate Vice President for Effectiveness & Planning	Submit a final draft of the Self-Study to the dean for review. After dean’s approval, submit to the Associate Vice President for Effectiveness & Planning for review and feedback.
2 months prior	Prepare for Site Visit	Develop Itinerary - See “Sample Itinerary” Reserve Meeting Rooms - Reserve meeting spaces for meetings except for the Exit meeting. If available, the degree may set up a room to allow as office space for the reviewers. Arrange Transportation – Arrange for transportation needs between the airport and hotel as well as any other transportation needs during the review. Arrange Campus and Facility Tours - Tours should be given of any major facilities, centers, labs, etc. associated with the review. Arrange Meals for Review Team - Arrange lunches for the review team.
1 month prior	Submit Final Self-Study to the Associate Vice President for Academic Effectiveness & Planning
	Facilitate the Site Visit
21 days after site visit	Receive Site Visit Report
30 days after receipt of review team's report	Prepare and Submit Program Response to the External Review Team’s Report	The program should receive the final report from the review team within 21 days after the site visit and has 30 days to prepare a draft response for the dean’s review. The College will review and consult with the program as needed before submitting the response to the Provost through the Associate Vice President for Academic Effectiveness & Planning.
Approx. 90 days after review	Attend and Lead Post Review Meeting	The Post Review meeting is scheduled by the APR Coordinator. Unless otherwise approved, only the Provost, APR administrative team, dean, and DH/PC attend this meeting.

The External Review Team

The external review team participates in and conducts the site visit. The role of the review team is to ensure the integrity of the academic program review, provide valid feedback regarding the status of the program, and make recommendations for improvement. The external review members will receive a $1,500 professional fee and will be reimbursed for all travel-related expenses by the Office of the Provost upon completion and transmittal of the final report to the Provost.

External Review Team Charge

Based on the information provided in the self-study report and gathered by the review team during its campus visit, what are the programs’ overall strengths and weaknesses?
How well do the programs’ focus and learning outcomes align with the strategic academic goals and priorities of the department, college(s), and those of Texas A&M University?
How would you compare the program(s) reviewed with those offered by peers? Specifically, do the curriculum, the identified learning outcomes, and overall program outcomes align with similar programs offered at peer institutions? If not, are unique characteristics of the program(s) reviewed a strength or area of concern?
With only current resources or a modest infusion of new ones, what specific recommendations could improve the programs’ performance, marginally or significantly?

External Team will be asked to complete an External Review Team Form that asks about each section covered in the self-study

Academic Review Process

Degree programs scheduled for an Academic Program Review are contacted approximately one year in advance of their review to allow ample preparation time. The process is initiated by participating in a mandatory APR Orientation session approximately 12 months prior to the beginning of the academic year in which the review will occur.

The Academic Program Review process is outlined in detail below. The process can be conceptualized into five stages:

Nominating members for the external review team
Preparing the self-study
Establishing the itinerary for the on-site visit
Generating the response document
Attending post-review meeting and reporting.

Nominating the External Review Team

An important task for the Degree Program is to develop a list of 8 to 10 potential reviewers. Nominees should be professionally prominent individuals who have experience not only with undergraduate and graduate programs, but also with teaching, research, and engagement or service components as appropriate. Programs should identify a diverse group of nominees from peer or aspirant peers. When appropriate, nominees may also be from business or government. A team member who participated in a previous review is recommended, but not required. Programs should coordinate their nominees in consultation with the appropriate college dean.

The list of potential reviewers will contain the following information:

Preferred potential reviewers
Complete job title, rank, and name of a reviewer’s department or program
Nominee's principal area of scholarly activity (related to degree program being reviewed)
Name of university or organization
Contact information (full mailing address, e-mail, and telephone number)
Preference for chair of external review team (top two choices)

Upon receipt of the college-approved peer review candidates, the Associate Vice President for Academic Effectiveness & Planning will forward the list of candidates to the Provost for approval. After approval by the Provost, selected members will be invited by the Provost.

Review team members selected from the list of potential reviewers will be chosen according to the following criteria:

All external reviewers with subject matter expertise must be employed by institutions of higher education outside of Texas.
External Reviewers must be part of a program that is nationally recognized for excellence in the discipline; they will be asked to benchmark Texas A&M’s programs with their peer and aspirant peer programs based on discipline-specific rankings and other publicly available comparisons to graduate programs.
External Reviewers must affirm that there exists no conflict of interest related to the program under review.

Preference will be given to External Reviewers from peer programs at public, preferably land grant, institutions

Preparing the Self-Study Document

The Program’s faculty prepare the self-study prior to review. This study provides detailed information about the program and includes an assessment of the strengths, weaknesses, opportunities, and threats. The self-study is the faculty’s opportunity for self-evaluation and reflection both on historical data (the past five years) as well as a strategic path forward (the next five years). Programs are encouraged to commit themselves to specific, long-range planning in the self-study. An analysis of each major section of the self-study helps reviewers put the program data into context within the college and larger institution.

A final draft of the self-study must be submitted to the Provost’s office two and a half months before the on-site visit.

Programs should closely follow the self-study outline provided in the next section.This outline ensures that key elements required by the Texas Higher Education Coordinating Board (THECB) are addressed.

Contents of the Self-Study Report

The program’s self-study will be your external review team’s primary source of information. In addition to presenting the information requested in tables, charts, and narratives, the program should complete each section with an analysis.

Organize the self-study report according to the order of required items as listed.

Sections of the self-study denoted with * are required by the State of Texas (see “Characteristics of Texas Doctoral Programs” and Texas Administrative Code Rule § 5.52). Additional information will be embedded in the template.

Executive Summary of the Self-Study Report (1-2 pages)

Overview of the program(s) to be included in the APR
Overview of the number of faculty and students across programs for the current academic year

Department Overview and Infrastructure

Briefly describe each of the following:

Mission and strategic plan
- Include a summary of the strategic priorities for the department, the college/school, and the University as they relate specifically to academic programs
Leadership of the department
External program accreditations (if applicable) *
Facilities supporting the program(s) included in the APR, including space and equipment used for the delivery of the programs and related activities (e.g., research)
Finances related to academic programs(s)

At a minimum, supporting documents should include the following:

Institutional Summary – provided by the APR Coordinator
Department Strategic Plan (if available)
School/College Strategic Plan

Academic Program(s) Overview

List the academic degree programs to be addressed in the APR
Identify peer and aspirant programs
Summarize the curriculum by program, including program learning outcomes
Describe how the curriculum is appropriate for the level and type of degree awarded by the program
Specify the admissions criteria (by program)
Provide the number of degrees awarded per year (based on the most recent 5 years) and average time to degree (based on the most recent 5 years) by program*

At a minimum, supporting documents should include the following:

A summary of the program curriculum for each program; documents might include the catalog description of the program and curricular requirements or sample degree plans from the most recent academic year.

Summary of the last APR if applicable (if this is the first APR review for the programs in the department, note this in this section)

Provide the date of the last review and list the members of the external review team
Summarize the recommendations and actions taken to date based on the recommendations, with particular attention to the last 2-3 years since the submission of the last Status Report
Provide a summary of any significant structural or curricular changes to programs offered since the last review (e.g., new programs, program phaseouts, changes to mode of delivery, changes to the administrative location of the programs)

At a minimum, supporting documents should include the following:

The last External Review Team Report
The Institutional Summary submitted to the THECB
The 1-Year Status Report
The 5-Year Status Report

Program Assessment (by program)

Learning Outcomes
Provide the strategies used to assess identified learning outcomes (as outlined in Section 3)
Summarize the results from previous student learning outcome assessments
Discuss (1-3 improvements made based on results of the program’s assessment of learning outcomes (by program) over the last 2-3 years.
Program Outcomes (for the previous 5 years, by program)
Provide data on the identified program outcomes, to include but not limited to the following:
- Retention rates*
- Number of degrees per year*
- Graduate rates*
- Average time to degree*
- Student publications / presentations*
Discuss the degree program’s progress and efforts related to retention and graduation rate, time to degree, degrees awarded and placement upon graduation

Student Profile

Provide the following data for the last 5 academic years (based on fall semester data) by academic program included in the current APR:
- Enrollment by academic year, including % of full-time students*
- Student demographics (gender, ethnicity, domestic v. international, first generation) by level (e.g., baccalaureate, masters, doctoral) *
- Average institutional support provided for graduate students by program/year (e.g., first year master’s students, first year doctoral students) *
- Number of applied, admitted, and enrolled students by program*
- Percentage of full-time students with institutional financial support*
- Employment profile (in field/discipline within one year of graduation) *
Analysis: Discuss the degree program’s contribution to the land grant mission of the university as described in the Strategic Plan. Discuss any discernable trends in the data, including efforts undertaken to address any identified deficiencies.

Additional information could include:

Average SAT/GRE scores for entering students
Average GPA for enrolled students by program
Comparisons with similar programs offered across the college/school or among the identified peer/aspirant programs

Faculty Profile

The information provided in the tables below should reflect the faculty integral to the program(s) such as those who teach courses, mentor students, or serve on advisory committees

	Tenure/Tenure Track Faculty	APT Faculty
Number of Faculty *
Faculty/Student Ratio (full-time faculty to student FTE) by program for the most recent academic year (insert additional rows for programs) *
Faculty Publications (most recent 5 years) – Total number of faculty publications (attach a support document that lists all publications represented in the numbers provided) *
External Grants (most recent 5 years) – Total dollars awarded (attach a support document that lists all grants represented in the numbers provided) *
Average Teaching Load (add rows, if applicable, to reflect differenced by rank) *

Faculty demographics

Demographics of full-time faculty as of the fall semester immediately prior to submission of self-study

	Tenure/Tenure Track		APT		Peer or Aspirant Comparisons (if Applicable)
Gender *	Number	Percent	Number	Percent	Percent
Male
Female
Race & Ethnicity *
White
Black/Multiracial with Black
Latina/o
Asian
Rank
Full
Associate
Assistant
% with a terminal degree

Faculty qualifications (i.e., expected qualifications for faculty by rank)
- Provide a summary of expected qualifications for faculty by rank and type (i.e., tenure/tenure-track, APT). Include a discussion of metrics commonly used to assess faculty performance such as in annual reviews.
- In the table below, list the names of all faculty represented in the tables above and provide the information as indicated (add rows as needed).

Faculty Name (link an abbreviated CV)	Highest Degree Earned (including institution and discipline)	Summary of areas of teaching and research specialization(s)

Analysis: Discuss the extent to which the number of full-time faculty members is sufficient to support the mission of the program(s) under review and to ensure their quality and integrity. Using the degree program(s) standards (i.e., Program Evaluation Metrics for faculty teaching, research, and service, as well as other relevant data such as annual reviews or other benchmarks) discuss the faculty’s overall performance.

Identified Priorities and Opportunities for Improvement as a result of the Self-Study process.

Based on the self-study process,

What opportunities for improvement did the program faculty identify?
What priorities did the program faculty develop?
Are there any specific areas for which the program faculty is particularly interested in ERT feedback/guidance?

Data

Data is to cover the most recent 5-year period and will include the information summarized below.
Data can be found in the Accountability website and in the Faculty Affairs Intranet under the Data tab.

Student Data

Student enrollment by degree program
Student demographics (gender, ethnicity, domestic vs international) by level
Average SAT/GRE scores and GPA for enrolled students by level; also compared to the average for the affiliated academic college (by semester and academic year)
Degrees awarded by degree program
Number of applied/admitted/enrolled students for degree program by degree program and by level (fall semesters only)
Average time to degree by degree program (by academic year)
Average retention rates by degree program (by academic year)
First time in college by Texas high school

Faculty Data

Average annual faculty salary by rank (by academic year converted to nine-month salaries); this will include full-time faculty only and will be divided among tenured/tenure-track individuals and non-tenure-track individuals
Average faculty salary by rank relative to TAMU, and relative to peer institutions comparable degree programs; this will include full-time faculty only
Faculty demographics (gender, ethnicity, age) by rank (divided by full time and part-time)
Teaching load per faculty rank by level (by academic year)
Faculty to student ratio; also compared to other degree programs within the affiliated academic college

Other Data

Semester credit hours taken by major (regardless of degree program) by level (by academic year)
Semester credit hours taught in degree program courses (regardless of major of student taking the course) by level (by academic year)
Courses taken by major
History of courses taught
Outside-degree program students by gender and level
Outside-degree program students summary by gender and level

Establishing the Itinerary for the On-Site Visit

External reviewers arrive in College Station by 6:00 p.m. on the first day of the review and will depart the afternoon of the final day of the review. The site visit consists of Entry/Exit Interviews with the APR Administrative Team, various meetings with the dean of the college (or designee), DH/PC, faculty members, and students, and tours of degree program facilities. Below is a sample itinerary. This is a suggested itinerary and is flexible with the exception of the breakfasts that are scheduled by the APR Coordinator.

Sample Itinerary - 1 1/2 day option (for Bachelor or Master's (or both) or Doctoral program only)

Day 1 - Travel & Welcome

2:00-6:00 pm	Review Team arrives in College Station.
7:00-9:00 pm	Welcome dinner hosted by DH/PC at local restaurant. Orientation/background will be provided at this time. Those in attendance include DH/PC, associate department heads/chairs and/or other relevant people.

Day 2

7:30-8:30 am	Reviewers eat breakfast at The Texas A&M Hotel (a voucher will be provided in the reviewer’s welcome packet to eat breakfast in the Brazos Restaurant at the hotel)
9:00-10:00 am	Meet with college dean at The Texas A&M Hotel
10:00-10:30 am	Meet with DH/PC at The Texas A&M Hotel
11:00 am-12:00 pm	Meet with faculty committees
`12:00-1:00 pm	Lunch (possible brown bag or pizza with grad students)
1:00-2:00 pm	Meet with faculty in sub-discipline areas
2:00-3:00 pm	Meet with students
3:00-4:00 pm	Meet with department heads/program chairs within the same college
4:00-6:00 pm	Tour degree program facilities
6:00-9:00 pm	Return to hotel / Dinner / Work session for review team

Day 3

8:00-9:15 am	Exit Meeting and breakfast with APR Administrative Team + dean (or designee) at The Texas A&M Hotel; (degree program will confirm dean’s attendance)
9:30-10:30 am	Reviewers debrief degree program leadership at the hotel
10:30-11:30 am	Reviewers brief faculty, staff, and students on final report
12:00 pm	Lunch or departure as applicable

Sample itinerary for 2 1/2 day option

Transportation, Meals, and Lodging

Transportation: Members of the external review team will be making their own travel arrangements. The program is responsible for ground transportation while in College Station.

Meals: The program is responsible for arranging lunch for the External Review Team.

Lodging: Lodging is arranged by the APR Coordinator. Reviewers will stay at The Texas A&M Hotel & Conference Center

Texas A&M Hotel & Conference Center
177 Joe Routt Boulevard
College Station, Texas 77840
979-260-2235

The External Review Team's Final Report

The Response Document

The APR Coordinator will forward the external review team’s final report to the degree program, the college leadership, and the APR Administrative Team. The degree program has 30 days to provide a draft written response on the external review team’s report to the dean. During this time, the DH/PC will share the response with the faculty and provide the opportunity for input and discussion. The program will then submit the report to the college dean. The college will then submit the final program response to the Associate Vice President for Effectiveness & Planning through the APR Coordinator no later than 45 days from the site vist.

The program must address all recommendations identified in the external review team’s report. If the external review team has made a recommendation with which the program disagrees, that too should be addressed in the response.

Note that the response must be a stand-alone document such that individuals could read the response without having seen the self-study or the external review team’s final report. Therefore, the recommendations shall be extracted verbatim and copied into the response.

Post-Review Meeting and Reporting

The goal of the post-review process is continuing institutional improvement as a result of the APR. Post-review consists of a post-review meeting and submission of a post-review summary.

The Post-Review Meeting

The post-review meeting is held within 90 days after the site visit. This meeting is led by the DH/PC and attended by the APR Administrative Team and the college dean (or designee). With prior approval, the DH/PC may invite other degree program or college personnel as appropriate. The post-review meeting provides the DH/PC the opportunity to present and discuss the Program Response and the proposed actions based on the external review. The APR Administrative Team, college dean, and DH/PC will subsequently reach consensus on what actions are to be taken, by whom, and in what time frame.

The Institutional Response

The outcomes of the Post Review meeting, as well as the degree program response, will be outlined in the Institutional Response. The Institutional Response includes actions to be taken and discussion of any resources necessary for implementing the plan. Developed in consultation with the program, college leadership, and members of the APR team, the Provost’s Office will submit the Institutional Response to the THECB as required.

Appendices

Texas Administrative Code

In accordance with the requirements of the Southern Association of Colleges and Schools, Commission on Colleges, each public institution of higher education shall have a process to review the quality and effectiveness of existing degree programs and for continuous improvement.
The Coordinating Board staff shall develop a process for conducting a periodic audit of the quality, productivity, and effectiveness of existing bachelor's, master's, and doctoral degree programs at public institutions of higher education and health-related institutions.
Each public university and health-related institution shall review all doctoral programs at least once every ten years.
1. On a schedule to be determined by the Commissioner, institutions shall submit a schedule of review for all doctoral programs to the Assistant Commissioner of Academic Quality and Workforce.
2. Institutions shall begin each review of a doctoral program with a rigorous self-study.
3. As part of the required review process, institutions shall use at least two external reviewers with subject-matter expertise who are employed by institutions of higher education outside of Texas.
4. External reviewers must be provided with the materials and products of the self-study and must be brought to the campus for an on-site review.
5. External reviewers must be part of a program that is nationally recognized for excellence in the discipline.
6. External reviewers must affirm that they have no conflict of interest related to the program under review.
7. Closely-related programs, defined as sharing the same 4-digit Classification of Instructional Programs code, may be reviewed in a consolidated manner at the discretion of the institution.
8. Institutions shall review master’s and doctoral programs in the same discipline simultaneously, using the same self-study materials and reviewers. Institutions may also, at their discretion, review bachelor’s programs in the same discipline as master’s and doctoral programs simultaneously.
9. Criteria for the review of doctoral programs must include, but are not limited to:
  1. The Characteristics of Texas Public Doctoral Programs;
  2. Student retention rates;
  3. Student enrollment;
  4. Graduate licensure rates (if applicable);
  5. Alignment of program with stated program and institutional goals and purposes;
  6. Program curriculum and duration in comparison to peer programs;
  7. Program facilities and equipment;
  8. Program finance and resources;
  9. Program administration; and
  10. Faculty Qualifications.
10. Institutions shall submit a report on the outcomes of each review, including the evaluation of the external reviewers and actions the institution has taken or will take to improve the program, and shall deliver these reports to the Academic Quality and Workforce Division no later than 180 days after the reviewers have submitted their findings to the institution.
11. Institutions may submit reviews of graduate programs performed for reasons of programmatic licensure or accreditation in satisfaction of the review and reporting requirements in this subsection.
Each public university and health-related institution shall review all stand-alone master’s programs at least once every ten years.
1. On a schedule to be determined by the Commissioner, institutions shall submit a schedule of review for all master’s programs to the Assistant Commissioner of Academic Quality and Workspace.
2. Institutions shall begin each review of a master’s program with a rigorous self-study.
3. As part of the required review process, institutions shall use at least one external reviewer with subject-matter expertise who is employed by an institution of higher education outside of Texas.
4. External reviewers shall be provided with the materials and products of the self-study. External reviewers may be brought to the campus for an on-site review or may be asked to conduct remote desk review.
5. External reviewers must be part of a program that is nationally recognized for excellence in the discipline.
6. External reviewers must affirm that they have no conflict of interest related to the program under review.
7. Closely-related programs, defined as sharing the same 4-digit Classification of Instructional Programs code, may be reviewed in a consolidated manner at the discretion of the institution.
8. Master’s programs in the same 6-digit Classification of Instructional Programs code as doctoral programs shall be reviewed simultaneously with their related doctoral programs.
9. Criteria for the review of master’s programs must include, but are not limited to:
  1. Faculty qualifications;
  2. Faculty publications;
  3. Faculty external grants;
  4. Faculty teaching load;
  5. Faculty/student ratio;
  6. Student demographics;
  7. Student time-to-degree;
  8. Student publication and awards;
  9. Student retention rates;
  10. Student graduation rates;
  11. Student enrollment;
  12. Graduate licensure rates (if applicable);
  13. Graduate placement (i.e. employment or further education/training);
  14. Number of degrees conferred annually;
  15. Alignment of program with stated program and institutional goals and purposes;
  16. Program curriculum and duration in comparison to peer programs;
  17. Program facilities and equipment;
  18. Program finance and resources;
  19. Program administration.
10. Institutions shall submit a report of the outcomes of each review, including the evaluation of the external reviewer(s) and actions the institution has taken or will take to improve the program, and shall deliver these reports to the Academic Quality and Workforce Division no later than 180 days after the reviewer(s) have submitted their findings to the institution.
11. Institutions may submit reviews of graduate programs performed for reasons of programmatic licensure or accreditation in satisfaction of the review and reporting requirements in this subsection.
The Coordinating Board shall review all reports submitted for master’s and doctoral programs and shall conduct analysis as necessary to ensure high quality. Institutions may be required to take additional actions to improve their programs as a result of Coordinating Board review.