By: Margie Crutchfield, Ph.D., CAEP Consultant
June 2, 2016
Over the last two semesters, I’ve been a CAEP observer on six onsite visits and six offsite meetings (across all accreditation pathways) and I’ve been thinking about what worked well in all these different settings. In my experience there are two basic components to consider--and they both require a lot of preparation. The first, and most important, is focused on the substance or content of the review—how effectively does the educator preparation provider (EPP) demonstrate that it is meeting the CAEP Standards? The second is the planning for logistics of the visit—the nuts and bolts.
I’ve come to realize one of the great strengths of the CAEP review process is that it is, in fact, a series of interconnected events, each one a critical part of the process. It isn’t possible to think about the success of the site visit without fully understanding what I like to call the arc of the review process. Others have called it a journey and it can also be likened to a conversation—albeit a long and drawn-out conversation!
Whatever pathway an EPP has selected, there are several opportunities for the EPP and reviewers to go back and forth—to provide documentation, to give feedback, and then to modify or add documentation. It is this formative review process—this arc--that allows EPPs to demonstrate that they are meeting the standards and are engaged in the critical work of continuous improvement.
Therefore, the answer to “what makes for a successful site visit?” can’t be answered without looking at the formative process that has occurred in the 8 - 12 months prior to the visit. It is that effort that will have the greatest impact on the site visit.
This all begins with the Self-Study Report (SSR)—using either the Selected Improvement or the Inquiry Brief format. In the SSR, the EPP provides its best case for meeting the CAEP Standards. The EPP gathers evidence for the standards and its claims, and, hopefully presents that evidence in ways that are effective and efficient. All the EPPs that I have visited had several committees set up to collect and review evidence. Often an EPP will have a committee for each of the CAEP Standards. Each committee takes the lead for their standard and preparing the draft narrative for that standard. Although this may not be true in all cases, typically an EPP will have one point person who is responsible for directing the work of the committees, for collecting all the documentation and submitting it into AIMS, and for ensuring consistency in data and narrative presentation (especially with helping them work together on issues that cut across standards).
The March 2016 Accreditation Handbook (p. 11) has some excellent tips for preparing the SSR—including carefully following the guidelines it contains! The new handbook also has a great deal of information on each pathway. The many webinars that Deb Eldridge has done, available on CAEP’s Youtube channel, are also excellent EPP resources.
It is important to understand that the Self-Study Report or the draft Inquiry Brief is the first step of an extended process. EPPs need to be as clear and accurate as possible in describing and documenting how they are meeting the standards, how they are analyzing and using data, how they are engaging with partners, and so on. EPPs will receive substantive feedback and be able to provide additional information and documentation.
The formative review process is a little different depending on the pathway selected, but all pathways have a feedback loop built into the timeline. For specific information about each pathway, be sure and review the information in the Accreditation Handbook. The following is a general overview of the process.
The next step in the process, after submitting the first report, is an evaluation by trained reviewers. For the SI and TI pathways, this is called the Formative Review and is completed by the same team that will come on site. For the IB pathway, trained reviewers complete this evaluation. Reviewers carefully read and evaluate everything in the SSR or the Inquiry Brief. They will examine the evidence and then provide extensive feedback. If on the IB pathway, this feedback will come in the form of comments and questions embedded in the draft Inquiry Brief. In either case, reviewers will tell you what questions they continue to have, what evidence needs clarification, what documentation they think is missing, and what tasks they will need to complete on site. SI and TI reviewers will also tell you what they see as potential areas for improvement and/or stipulations.
It may help to take a deep breath before you open the Formative Feedback Report or the annotated Inquiry Brief. That’s not because the news is necessarily bad, but because, if the reviewers have done their job, they are going to give the EPP a lot of feedback, and that might seem overwhelming at first. Remember, this is a formative review—and the reviewer’s responsibility is to let the EPP know anything and everything that they see as potential concerns. This is a good thing (although it might not always feel so good)—because now the EPP has time to respond to their concerns. If the data weren’t presented in ways that were effective for the reviewers, then the EPP has time to re-do those charts. If there are missing data, there is time to provide it. If plans were submitted that the team felt were not sufficiently specific, then the EPP has time develop these more fully. The EPP can provide additional documentation and data. If there is anything in the report that is unclear, the EPP can ask the Team Lead for clarification.
After receiving feedback, the EPP has another opportunity to provide documentation. This is done in the Addendum for the SI, TI pathways; for the IB Pathway, this is done in the submission of the final Inquiry Brief. There are many ways to format your Addendum—but it should be done in a way that is most helpful to the review team. One effective way I’ve seen is for the EPP to follow the same exact format of the Formative Feedback Report, responding to each question and concern in turn. The Site Visit Team members will read the Addendum/Inquiry Brief and review all additional documentation prior to coming on site. This new information may modify some of their tasks or need for further corroboration. Those on the IB Pathway will receive an Auditability Report that responds to the Final Inquiry Brief.
The next important event to help prepare for a successful site visit is the Pre-Visit. This is usually a conference call that includes the Lead Site Visitor, appropriate staff from the EPP and often the State representative. There are many things to discuss during the Pre-Visit but one of the most important is setting up the schedule for the visit. Before the call, the EPP should look carefully at the tasks outlined in the Formative Feedback Report/Auditability Report and focus on what the team needs to know and any issues that may or may not have been resolved in the Addendum/Inquiry Brief. Using this information, the EPP and Team Lead will set up the schedule for the visit. Some questions to think about:
• Who will the team need to interview to answer the questions they have raised? Typically the EPP would include interviews with candidates; recent graduates; field and clinical supervisors; clinical partners including principals, school personnel directors, cooperating teachers and others involved in the clinical preparation; and critical committees and groups of faculty.
• What documentation will the team need to review to address their concerns?
• What school sites, if any, would be the best place for a few members of the team to visit and observe? For example, if one of the concerns raised in the Formative Feedback Report was candidates’ use of technology, the EPP may want to ensure that the team visits classrooms where their candidates are using technology in their instruction.
The EPP will also go through the logistics of the visit with the Team Lead to make sure that the team has the appropriate work spaces, that meals are addressed, transportation needs are taken care of, the team has access to appropriate technology, and so on. All of these logistical details are important and help provide the framework for a successful visit.
At last it is time for the site visit! The good news is that the EPP should be very aware of what will happen. The feedback from the reviewers prior to the visit should make it clear what the team will focus on during their time on campus and what issues they will be investigating. From the team’s perspective, it is during the onsite visit that they are really able to understand all the different aspects and nuances of the EPP’s work. I’ve seen this happen over and over again—the team has gained a great deal of information about the EPP through the offsite review activities. But over the several days of the visit, the team is able to get a real feel for the EPP culture, to get a much deeper and broader picture of all aspects of the EPP program through interviews, observation and documentation review. The visit may seem brief but there is a great deal of information the team is able to gather—because they have done so much preparation before arriving.
Of course, after the visit, the team will post a Final Report and the EPP will have one more opportunity to provide a response prior to the Accreditation Council’s final decision.
So what makes for a successful visit? A successful visit is really the result of the entire review process—one that has several steps, each important, each building on each other, leading towards a positive visit, and a successful Accreditation Council decision.