Guide to the On-Site Evaluation Visit

SECTION I.   INTRODUCTION

The goal of the accreditation process is to determine to what extent the applicant’s psychoanalytic training program meets the Standards of Accreditation of the American Board for Accreditation in Psychoanalysis, Inc. (ABAP, Inc.) and to make a decision on the training program’s accreditation status.

The on-site evaluation visit provides essential data to assist ABAP, Inc. in reaching an accreditation decision through the discernment and recommendation of its Committee on Accreditation (COA). The visiting team provides empirical information and assessments that validate or supplement the program’s self-study, helping the COA make an accreditation recommendation.  It provides information that can be learned only via on-site observation, including discussions with faculty, administration, students, and other appropriate persons such as graduate analysts.

The on-site evaluation visit team serves as the eyes and ears of the Committee on Accreditation to gather the facts indicating whether a psychoanalytic training program is in compliance, partial compliance, or noncompliance with each of the Standards of Accreditation. The COA makes an evaluative discernment and recommendation to the Board of Trustees of ABAP, Inc. based upon these facts.  The Board then takes an action regarding the program’s accreditation status.

This manual is to be used as a guide to ensure that the site team adequately and objectively evaluates the psychoanalytic training program against the Standards for Accreditation, using the program’s Self-Study as a reference. The on-site evaluation team members are professional peers who are expected to conduct the site visit not as an adversarial proceeding, but rather as a constructive and collegial opportunity both for learning about the program and for facilitating quality improvement within the program.

SECTION II. PRE-VISIT PROCEDURES

Upon receipt of the program’s Self-Study, the Director of the Office of Accreditation forwards it to the Committee on Accreditation.  The COA reviews the self-study to determine whether it satisfactorily describes and evaluates the program’s compliance with the Standards for Accreditation.  If the COA either (a) requires more data, (b) believes that the self-study needs to be more fully developed or concise, or (c) in the case of programs applying for Candidacy or Initial Accreditation, concludes that the self-study demonstrates blatant non-compliance with multiple standards, the COA asks that the Director communicate with the program leadership to work on remedying the situation before scheduling a site visit. When the report is satisfactory to the COA, the Director schedules an on-site evaluation visit at a time mutually convenient to the institution and the team.  In most cases, the COA will articulate specific questions or concerns for the site visitors to address.

Well in advance of the on-site evaluation review, the Director of the Office of Accreditation, in close cooperation with the Chair of the Committee on Accreditation and the CEO of the applicant institution or his/her designee, establishes a site-visit date and appoints an on-site evaluation team.  The exact number of visitors and the length of the visit will reflect the complexity of the program being visited. In general, an on-site evaluation visit does not last more than two days, depending on the size and complexity of the institute or program. Visits that require more than two days can be arranged with the approval of the Director of the Office of Accreditation.

The composition of the on-site evaluation team should reflect the nature of the questions and concerns of the COA.  The Director will designate an on-site evaluation team leader who is well versed in the accreditation process of ABAP, Inc.  A typical team includes at least three site evaluators, including at least one practicing certified psychoanalyst and one psychoanalytic faculty member at an ABAP, Inc. accredited program. Other potential site visitors include administrators at institutions with an ABAP, Inc. accredited program or other professionals with specific areas of expertise, such as finance or educational administration.  Qualified visitors have the appropriate education, skills, and experience to assess and benefit the program being evaluated and are trained by ABAP, Inc. on its standards, policies, and procedures.

The team should include representation from a variety of schools of psychoanalytic thought, including the school of thought of the program being evaluated.  The team also includes members from other geographic markets from the program being evaluated.

The Director informs the program in writing of the names and professional affiliations of the on-site evaluation team members. The program leadership is asked to advise the Director if it believes any team member is not eligible to participate due to past, present, or anticipated affiliations with the institute, or due to any apparent conflict of interest. Any question regarding the makeup of the visiting team should be resolved at this point.

An administrative staff member of ABAP, Inc. may serve as a site visit observer to provide consistency in procedures and standards from site visit to site visit.

Once established, the on-site evaluation team meets as a group with the Director of the Office of Accreditation prior to the on-site visit in order to review all materials pertinent to the on-site visit and discuss any questions or concerns raised by the COA. The Director instructs the on-site team members regarding their responsibilities, including the importance of an open and continual interchange between the visiting team and the faculty, administrative staff, students, and other appropriate persons. To assess the program appropriately, the team members should make a thorough review of the program to ensure that they have a grasp of its strengths weaknesses in relation to the Standards for Accreditation.

The Director of the Office of Accreditation provides the institution with a copy of this manual and some brief guidance on the typical agenda for site visit activities.  This allows the program to begin to arrange a schedule that allows appropriate individuals to be present at the visit.

SECTION III. PRE-VISIT RESPONSIBILITIES OF THE TEAM MEMBERS AND TEAM LEADER

After the institute or program has had an opportunity to comment on the composition of the on-site evaluation team, the team leader contacts the applicant institution’s chief executive officer or designee to arrange any logistics for the on-site visit, including accommodations, meals, meeting arrangements, and times for classroom observations. The team leader may schedule a preliminary meeting with the applicant prior to the actual on-site visit. He/she should use the preliminary visit to further clarify the intent of the on-site visit, discuss any areas of focus of the visiting team, and develop a suitable site-visit agenda.

The team leader will communicate with the members of the on-site evaluation team to establish which specific items in the Standards for Accreditation that each team member will cover in-depth.  Team members are then responsible for reading the Self-Study in-depth, focusing specifically on his or her specific standards.  Team members also review supplemental materials such as syllabi, manuals, or financial statements in advance of the visit in order to be prepared to ask questions and learn more about those materials during the visit itself.  Team members may ask the team leader to request additional materials from the program if needed before the visit.

The “Standards Checklist for the On-Site Evaluation Team” may be used before the visit to prepare questions related to the Self-Study and supplemental materials and to jot notes for future use in the evaluation report.  Some team members even find it useful to draft parts of the evaluation report ahead of time, leaving room for revision and answers to questions that arise over the course of the site visit process.

SECTION IV. PUBLIC COMMENT

At least 30 days before the site visit, the Director of the Office of Accreditation will publish an invitation for comment on programs that are scheduled for an accreditation review through email broadcast, the NAAP News, and the agency’s web site, www.abapinc.org. Such comments will be discussed by the Committee on Accreditation as it articulates questions and concerns for the on-site evaluation team.

SECTION V. THE ON-SITE EVALUATION VISIT

Goals of the Site Visit

Site visitors are reminded that, although ABAP, Inc. is the standard-setting and accrediting body, the accreditation on-site review is done at the program’s invitation and request.  On-site evaluators are professional peers who understand the Standards on Accreditation and the challenges in striving to meet them.  The purpose of the visit is to validate the findings of the self-study and provide guidance on adherence to the standards through a salutary, professional exchange. The evaluators are encouraged to be as unobtrusive as possible in classes, the library, and clinical facilities so as not to disturb the usual operations of the institution.

The on-site review team is a fact-finding body which provides ABAP, Inc., through its Committee on Accreditation, with an in-depth report of the program’s strengths and weaknesses in light of its stated mission, goals, and objectives and in relation to the Standards for Accreditation.  The evaluation is not a critique based on the evaluator’s own training, beliefs, or school of psychoanalytic thought.  Rather, the team examines the program’s success in meeting its stated goals.  For example, does a Freudian program demonstrate a Freudian learning matrix and do the learning products of this matrix reliably reflect this orientation?  Does the library sufficiently represent the program’s school of thought?

Functions of the Site Visit

In order to assess the program against the Standards for Accreditation during the visit, the evaluation team views the facilities, conducts meetings, observes classes, examines sample student records, and reviews documents such as publicity, manuals, financial statements, vitae, course syllabi, and students’ written work.  Each member of the evaluation team obtains an overall impression of the program, and each visitor conducts a previously planned study of one or more of the specific Standards for Accreditation, such as the library, curriculum, governance, and finances.

Members of the team interview all appropriate groups and individuals, including the governing board, administration, faculty, staff, and students, with special emphasis on faculty and student meetings.  In addition, every exhibit prepared by the program should be examined by at least one member of the on-site evaluation team, including financial statements, bylaws, grievance procedures, and files.

Appropriate members of the team should observe classes and evaluate the educational process in the classroom, in field work, and in group and individual case supervision.  Is the learning process effective and consistent with the program’s purpose and goals?

The end result of these activities is to validate and supplement the content of the Self-Study with in-person observations, leading to a full assessment of the program in light of its mission and in relation to the Standards for Accreditation.  The “Standards Checklist for the On-Site Evaluation Team” may be used during the visit to take notes related to the Self-Study, supplemental materials, interviews, and observations.  The “Checklist” is a place to document what data and evidence led the team to which specific evaluative conclusions.  This is critical for the development of the evaluation report.

At the end of the visit, the team meets with the chief executive officer, and others whom he/she may invite, to report its findings from the visit.  The verbal report should be organized according to the Standards, should constructively convey what the team learned about the program, and should facilitate quality improvement in relation to the Standards for Accreditation.  At this point, the program leaders are not invited to offer feedback on the report, since the visit is concluding and there is no further opportunity for fact-finding.  (The program will have the opportunity to give feedback in writing.)  In addition, because the COA and ABAP, Inc. Board still need to consider the team’s findings, the team is asked by ABAP, Inc. not to provide program leaders with an explicit recommendation regarding accreditation status.

SECTION VI. THE ON-SITE EVALUATION REPORT

After the visit, the visiting team will write an On-Site Evaluation Report.

Preparation of the Report

The on-site evaluation report provides the program, the COA, and the Board with a detailed written assessment of the extent to which the program complies with the Standards for Accreditation, including areas of compliance, partial compliance, and non-compliance.  The report is concerned largely with specific and unique characteristics of the institute and its program of instruction, setting forth the limitations and difficulties which the institute or program is experiencing and the plan and potential it has for overcoming them.  However, not every aspect of the program needs to be analyzed:  The purpose of the report is to provide a fair and useful estimate of the effectiveness of a program specifically against the Standards for Accreditation, so that ABAP, Inc. may make an objective, fair, and rational decision on accreditation. The report itself does not recommend an accreditation action.

Each member of the team contributes to the report, based on the areas of the Standards on which s/he focused during the visit.  The team members submit their portion of the report to the team leader within fifteen (15) days of the on-site evaluation visit. Within thirty (30) days of the visit, the team leader is responsible for pulling together the team reports, writing his or her own sections, writing the introduction and conclusion, and editing the entire report to improve the clarity and unity of the document. The team report should respect and reflect the assessment of all team members.

It is important to make certain that the report reflects the extent to which the applicant meets or exceeds the Standards of ABAP, Inc., because the Standards are the criteria against which ABAP, Inc. makes accreditation decisions.  Specifically, in its report, the on-site evaluation team should evaluate the extent to which the institute or program complies with each of ABAP, Inc.’s standards, noting strengths, areas of compliance, areas of partial compliance, and areas of apparent non-compliance.  The team may note areas where improvement is indicated and may offer constructive solutions.  However, the program is not required to meet the standard in any one particular way.  Advice should be clearly distinguished as such and may be used by the applicant accordingly. Advice has no bearing on accreditation status.

Process for Distributing the Report

The visiting team leader first sends a draft of the evaluation report to the other team members to ensure that team members’ observations and conclusions are properly represented.  Once the draft is approved by the entire team, the team leader sends a draft to the program director for review, no later than thirty (30) days after the evaluation visit.  At this point, the program director should, within fifteen (15) days, point out any errors of fact for correction by the team chair.  The team leader then sends a final copy of the report to the program director and the Director of the Office of Accreditation, who forwards the report to the Committee on Accreditation (COA).

Upon receipt of the report, the Director of the Office of Accreditation invites the institute’s chief executive officer to respond to the evaluation report in writing within fifteen (15) days of receipt.  The Director then sends the evaluation report and program’s response to the COA, which schedules a meeting to review the program and recommend an accreditation action to the Board.  The COA invites the team leader to be present when the Committee on Accreditation reviews the report.

SECTION VII. ON-SITE EVALUATION REPORT OUTLINE

PART I INTRODUCTION

1. Purpose of the Visit (Focused visit or comprehensive evaluation)
2. On-Site Evaluation Review Team
3. Materials Reviewed
4. Overview of the On-Site Evaluation Visit

PART II  NARRATIVE

1. Mission, Organization and Governance
2. Resources
   1. Financial Resources
   2. Library Resources
   3. Physical and Technological Resources
   4. Faculty Resources
   5. Student Support Resources
3. Public Disclosure and Institutional Integrity
4. Education and Training Program

Please note:  Standard E on Distance Learning should be incorporated into the entire report.

PART III CONCLUSIONS

1. Strengths
2. Areas for Improvement
3. Overall Assessment

SECTION VIII.  Standards Checklist for the Site Visit Evaluation Team