I have received a response to the previous post about the EPPP Step 2. With permission, I am sharing here. I welcome your thoughts and comments below or you can email me directly. Please title the email: EPPP-2 Blog.

Thanks,
Christine

---

 

Dear Dr. Jehu:

ASPPB appreciates the time and effort you took to communicate APAGS’ thoughts about the development of the EPPP Step 2.  We would like to respond to your comments in an effort to continue the dialog with APAGS about the EPPP Step 2.  We hope you will share this response with the APAGS membership and other colleagues as you see fit.

ASPPB’s mission is to enhance and support our member jurisdictions (that is the psychology licensing boards in the US and Canada) in fulfilling their goal of public protection.  We believe that the development of the EPPP Step 2 is a necessary and critical step in serving that mission and will prove to be a very helpful tool in protecting the public and in advancing our profession.  As the APA Board of Education Affairs recently stated,

“Embarking on this important initiative not only reflects recommended practices but also helps to enhance the profession of psychology and advances the trust society places in the profession.”

In your statement, you commented,

This exam may feel like a massive surprise to students and Early Career Psychologists. Unbeknownst to many of us, our field has been moving toward competency assessment since the Competencies Conference in 2002 and subsequent publications highlighting the importance and value in competency assessments (e.g. Rodolfa, Bent, Eisman, Nelson, Rehm, & Ritchie, 2005). During the 2013 updates to the APA Commission on Accreditation’s Guidelines and Principles, the APAGS Committee provided a comment that supported the development of competencies based assessment, but had concerns about cost, the process of assessing competencies, and the fair implementation of a new exam to psychology license applicants.

We appreciate that you have provided a context to the EPPP Step 2, by discussing the competency movement in psychology.  As ASPPB has discussed the EPPP Step 2, we have tried to state that this examination was not developed in a vacuum, but rather it is another step in this competency movement in psychology.  As most know, the competency movement in psychology is well documented in the education and training literature.  For those unfamiliar with the movement, we have a brief overview of the competency movement and how it relates to the EPPP Step 2 on our website.

The comment APAGS submitted to the CoA supported competency assessment, but also expressed concerns about costs, process and fair implementation. These are also concerns for ASPPB.  In discussing APAG’s concerns about costs, you cite the Doran et al., 2016 study indicating median student debt at $200,000 and that financial issues are a major stressor for students and graduates.  You note the current cost of the EPPP is $687.00 in most jurisdictions and encourage ASPPB to make the cost as low as possible to be fair to future psychologists.

Cost

As you are clearly aware, the cost of the EPPP is a small fraction of graduate psychology student debt.  Regardless of the fee of the EPPP, graduate psychology students will experience a painful debt load.  ASPPB understands this debt is truly stressful and hopes that training programs will be able to develop realistic plans to pay interns and postdocs a working wage and the profession will be able to work with the Federal government to create additional funding opportunities for graduate psychology students.   Having said that, the cost of EPPP Step 2 has not been determined at this time.  Please be assured, though, that ASPPB is fully cognizant and concerned about the financial burdens on those entering the profession and is committed to keeping costs as low as possible.  ASPPB envisions the EPPP Step 2 as a required step in the licensure process and thus by definition a high stakes exam (i.e. a candidate must pass this examination to be licensed).  Developing a high stakes exam, particularly one that will use computer-based simulations, and taped scenarios and vignettes, and possibly even avatars, will require considerable up front costs. ASPPB is prepared to absorb those start-up costs and operate the program in a cost efficient manner that will maintain the program in the years ahead.  There will, however, be some added costs to applicants for this examination.

Sequence of Training Issues

An additional concern expressed by APAGS is about the time that the EPPP Step 2 may add to the sequence of examinations.  APAGS cites the excellent Olvey, Hogg, & Counts (2002) article that psychologists have the longest time to licensure when compared to other health professionals.         

As you may know (and many graduate students may not know), the EPPP can be taken in almost all  jurisdictions after the doctoral degree is awarded, during the postdoctoral year, if the jurisdiction requires a postdoctoral training experience. There are a few jurisdictions (about 6 or so) that require the EPPP to be taken after the postdoctoral year is completed.  ASPPB has encouraged all jurisdictions to allow candidates to take the EPPP once all requirements for the degree are completed.

We are not expecting that the EPPP Step 2 will increase the amount of time to obtain licensure by much, if at all.  We anticipate that a licensure candidate will take the EPPP Step 2 after passing the EPPP and completing all other requirements for licensure (i.e., a postdoctoral experience if the candidate is located in a jurisdiction requiring a postdoctoral experience).  If a candidate is not in a jurisdiction that requires a postdoc, then we believe the candidate will be able to take the EPPP Step 2 soon after passing the EPPP. We must emphasize, however, that the timing of the EPPP and the EPPP Step 2 will be based on the rules each jurisdiction adopts.

APAGS strongly encouraged ASPPB (and in turn jurisdictions) to align the EPPP Step 1 with the United States Medical Licensing Examination (USMLE) Step 1 knowledge based exam to be given after completion of coursework and before the internship.  You noted the Schaffer et al. (2012) article highlighting a higher fail rate the further the EPPP is taken from graduation.  Your suggestion also aligns with recent data presented by ASPPB at the 2016 APPIC Conference noting that the further from graduation, the more poorly candidates do.  It does conflict, however, with other data presented at the APPIC Conference indicating that candidates completing an APA-accredited or APPIC-member internship have a higher pass rate on the EPPP than those that don’t.

The issue of the timing of the EPPP and the EPPP Step 2 is one that ASPPB is interested in and quite willing to discuss with our member licensing boards and other relevant stakeholders such as APAGS, APA, CPA, early career psychologists, and training associations.  As the development process of the EPPP Step 2 moves farther along, we will continue to examine data that is generated by the exams, consult with our member licensing boards, and make appropriate recommendations about this issue.

Highlighting positive financial implications for new graduates, APAGS encouraged ASPPB and the licensing jurisdictions to adopt the APA policy on sequence of training recommendation that the doctoral degree is the entry level into the profession, eliminating the need for a full year postdoctoral supervised experience.

There has been quite a debate since APA made the policy shift to endorse licensure at degree.  Up to this point, this debate has been based on views, thoughts, and feelings. As a result of the 16 years of debate, there has been a split and only 15-17 jurisdictions have changed their licensing requirements to match the APA policy.  This unfortunately has caused confusion in the profession about when a psychology trainee has the knowledge and skills (i.e., the competencies) needed for licensure.  Once the EPPP Step 2 is operational, ASPPB will have a mechanism to contribute to this debate by offering data that can inform the profession, and specifically the licensing boards, about when the skills needed for the independent practice of psychology are acquired. If the data shows limited differences, it may lead to a true resolution of this issue one way or another. However, it is too early in the examination development process for us to comment much beyond that.  As psychologists though, it is important for us to use data to guide and inform our decisions, whichever way those data lead.

Test Development

APAGS mentioned two issues regarding test development:

1. Data on the validity of exams such as the proposed EPPP Step 2, and,
2. The time frame for test development seems fast; and that, in an effort to reach the completion date goal, the test may not be as fully developed or as valid as it could be with more time.

As the EPPP Step 2 is an examination for licensure, content validity is the appropriate standard according to the Standards for Educational and Psychological Testing.  Content validity is typically accomplished through something called a practice or job task analysis.  A job task analysis uses a survey of individuals who already hold the license or certificate for which the test will screen future candidates.  The survey asks questions about the essential nature and importance of a set of knowledge statements for a test of knowledge or a set of skills and abilities for a skills test.  The survey results are analyzed to determine the key content domains or the key skills and competencies that define entry-level competence.  The current EPPP has undergone regular and routine practice or job task analyses, the most recent being completed in 2010.  ASPPB is currently conducting the job task analysis that will validate the essential skills that will become the focus of questions and performance events assessed by the EPPP Step 2.  Below we have added references to an article Dr. Steve DeMers wrote several years ago responding to an article critical of the EPPP (Sharpless and Barber, 2009).  Sharpless and Barber raised many of the questions that you raise about assessing validity in a licensure context, and ironically called for the creation of a competency exam. Dr. DeMers’ response provides a detailed discussion of the validity questions you raise as well as a glimpse into some of the “conversations” that led ASPPB to consider the need to develop a skills exam.

ASPPB has been working with our test vendor (Pearson) on the development of the EPPP Step 2 for over a year now, and they assure us that the current time frame of three years to first administration is more than enough time to develop a valid and reliable exam.

Test Preparation

APAGS stated:

Finally, APAGS is concerned that the EPPP Step 2 is being developed without appropriate training for students in the pipeline. It is unfair to pose a test to students without training materials and information to the education and training community. Therefore, implementing a test would need to come after materials were distributed to the training community so that rising psychologists would be aware of the exam and have experience with components of the exam. Distributing these materials widely also provides accurate informed consent to students entering the psychology pipeline.

ASPPB appreciates this comment and our Implementation Task Force will carefully review the materials needed to adequately prepare students to sit for this examination.  ASPPB would like to emphasize that the EPPP Step 2 will be an assessment of practice skills, based on competencies that are thoroughly described in documents on our website.  These competencies are highly similar and compatible with those required by the APA Commission on Accreditation (CoA).  It is ASPPB’s expectation that licensure candidates will already be prepared to take the exam.  We do not anticipate the need for formal studying for this exam.  Additionally as we move forward with this exam’s development, information and training materials, including sample questions, will be prepared by ASPPB and will be provided to applicants at no charge.

ASPPB is very willing to engage in multiple dialogs with students, trainees, and members of APAGS.  We hope to discuss the developments in the EPPP Step 2 at the APA Convention next year.  We will actively participate in professional dialogues about the licensing process.

We hope that you find our comments responsive and informative.  We appreciate your thoughtful and professional letter.  Please let us know if you have further questions or concerns.

Emil Rodolfa, PhD, Chair
EPPP Step 2 Implementation Task Force

 

References

DeMers, S.T. (2009). Understanding the purpose, strengths and limitations of the EPPP.  Professional Psychology: Research and Practice, 40, 348-353.

Olvey, C.,  Hogg, A., & Counts, W.  (2002).  Licensure requirements:  Have we raised the bar too far?  Professional Psychology:  Research and Practice, 33, 323-329.

Rodolfa, E., Bent, R., Eisman, Nelson, P.,  Rehm,L. & Ritchie, P. (2005).  A cube model for competency development:  Implications for psychology educators and regulators.  Professional Psychology: Research and Practice, 36, 347-354.

Sharpless, B. & Barber, J. (2009). Predictors of program performance on the Examination for Professional Practice in Psychology (EPPP).  Professional Psychology: Research and Practice, 40,208-217.