Increasing Research Transparency in Psychological Science: How Can Graduate Students Participate?

Written by: Michelle Rivers, Rachael Soicher, and Dr. Morton Ann Gernsbacher

What is transparency in research, and why does it matter?

Research transparency, also referred to as “open science,” is the process of making research and its dissemination accessible to all levels of society. Practices that increase research transparency accelerate progress toward solving the persistent problems of society. For example, you may have heard about the “replication crisis” in psychology and other sciences. Engaging in open science practices can help make science more reproducible and thus more trustworthy.

As the field of psychology works toward greater research transparency, how can you get involved? In this post, we talk about some baby steps you can take as a graduate student, and we address some common challenges you may face along the way.

2. How can you get started?

Below we’ve crafted a short list of ways you can make your research more transparent. Think of baby steps. Don’t feel like you have to do all of them at once!

Pre-register your study design, hypotheses, and planned analyses. Place them in a locked file (hardcopy or electronically) or post them on free websites like AsPredicted or Open Science Framework.
Make your study materials, data, preprints, and post-prints publicly available online, such as on your lab’s website or on Open Science Framework.
Avoid “Questionable Research Practices,” such as selectively reporting dependent measures/entire experiments, revising your sample size after viewing your data, or so-called “HARKing” (hypothesizing after the results are known).
Take advantage of research collaborations. Do you have or need access to a particular demographic sample? Check out StudySwap or Psychological Science Accelerator, online platforms for research replication, collaboration, and resource exchanges.
Use free and open-source software so it’s easier for others to reproduce your exact methods and analyses. For example, OpenSesame and PsychoPy are open-source applications for running psychology experiments, LimeSurvey is open-source survey software, G*Power is free software for running power analyses, and R is free software for statistical computing and graphics
Practice disseminating your research to diverse audiences. For example, one of us, Michelle, manages a blog (cogbites.org) run by early-career academics that focuses on translating scientific research about the study of mental processes to a general audience. To learn more about science communication and outreach, consider applying to attend ComSciCon, a workshop for graduate students, by graduate students.
Stay informed! If you have an advanced stats or methods course in your program, encourage the instructor to include recent readings on research transparency. One of us, Morton, teaches an open-access active-learning Research Methods course, and you can find other teaching resources here. Consider starting a reading group with other students in your program!

Continue reading →

The Examination for Professional Practice in Psychology (EPPP): What’s race got to do with it?

This blog post is a part of the series, “CARED Perspectives,” developed by the APAGS Committee for the Advancement of Racial and Ethnic Diversity (CARED). This series will discuss current events and issues in psychology and how these relate to graduate students. If you are interested in contributing to the CARED Perspectives series, please contact Aleesha Young, Chair of APAGS-CARED.

By Fiona C. Thomas

Grad school, that time in your life where you are constantly thinking about what milestone comes next. Course completion, Master’s thesis, comps, practica, dissertation proposal, data collection, analysis, publications…and that final training experience…internship. Although it feels like so much of our preparation in grad school is for this near final stage of the internship, there is one more milestone to meet before we can finally refer to ourselves as licensed psychologists – the EPPP.

The Examination for Professional Practice in Psychology (EPPP) was first administered in 1965 (Rehm & Lipkins, 2006) and is now a key requirement to independent practice as a clinical psychologist. It is required for licensing in all US states, three US territories, and the majority of Canadian provinces (DeMers & Schaffer, 2012). After all our years of training, and unique experiences along the way, it is that one shared experience we will all have as we work towards independent practice.

As I prepare to begin internship in September, this next milestone of licensure is on my mind. When do I begin preparing for this exam? What are the most effective methods to study for it? What is the pass rate? It is also timely to be thinking about this exam as a second part to the EPPP, which assesses professional skill, is a new requirement coming to an undisclosed list of jurisdictions in 2020 (Association of State and Provincial Psychology Boards [ASPPB], 2019).

With these questions on my mind, I recently became aware of a study conducted by Dr. Brian Sharpless which found performance discrepancies based on ethnicity (2018). In other words, failure rates were highest for individuals who identified as Black (38.5%), Hispanic (35.6%), Asian (24%), and lowest for those who identified as White (14.07%) (Sharpless, 2018). Interestingly, it was not the first time these findings emerged. In 2013, Sharpless and Barber identified that doctoral programs with greater ethnic diversity also reported lower program-wide EPPP pass rates. Going back almost four decades to an early study on this topic, Werner (1981) similarly found that Whites had an EPPP pass rate almost twice that of any other ethnic group. Reviewing survey data of individuals who graduated between 2005 and 2015 with a doctoral degree in clinical, counseling, school or combined/integrated psychology, Bowman and Ameen (2018) found similar results. In particular, their analysis indicated that White psychologists were significantly more likely to pass the EPPP on the first administration compared to psychologists of color (92 percent vs. 83 percent). For this study, psychologists of color included individuals who identified as Asian, Hispanic/Latino, multiracial, Black, American Indian/Alaska Native, and Middle Eastern/North African (Bowman & Ameen, 2018).

As someone who identifies as an ethnic minority, these studies surprised me. What variables contributed to these fail rates? What could be done about this? And what did I need to know to be better prepared so that I didn’t become an embarrassing and expensive statistic?

Of course, as with all research, there are limitations to these findings, and this is important to keep in mind. Until recently, demographic information (e.g., ethnicity) was not collected in EPPP administration (DeMers, 2009). This means that these studies are mainly based on self-report information, which is less robust and reliable compared to test administration data (Bowman & Ameen, 2018). Additionally, some ethnic minorities are not properly represented in these studies. In the studies mentioned here, Middle Eastern/North African (MENA) Psychologists and Indigenous Psychologists are not included as distinct groups. We also have limited information about pass/fail rates for ethnic minority licensure applicants in Canada. Despite these limitations, the replication of such results has prompted me to wonder what the root causes of these differences are and what can be done to address this discrepancy.

In good news, the test publisher, ASPPB, has started collecting race and ethnicity data as optional information on the EPPP. This will continue to better inform whether some test takers, namely test-takers of color, are at a disadvantage. Importantly, this information will be collected as optional data after the completion of the exam to avoid priming any stereotype threats (ASPPB, 2019). If questions show differential responding by ethnicity, ASPPB will take specific items to a diversity committee who can then review and decide whether to keep or discard the questions in future tests. Relatedly, graduate and internship programs are also becoming aware of this discrepancy. Such knowledge can lead to initiatives to better prepare ethnic minority students.

As psychology’s clients and practitioners becomes increasingly diverse, it is important to understand how the EPPP and other high-stakes exams may be leaving certain individuals behind. Factors such as Eurocentric item construction, minority stress, or performance expectancy may or may not be playing a role in score discrepancies (Bowman & Ameen, 2018). Such hurdles can only be identified by appropriately collecting data on race and ethnicity, and transparently reviewing it.

As a member of the APAGS Committee for the Advancement of Racial and Ethnic Diversity (APAGS-CARED), we are keen to explore this further and will be hosting a webinar on this very topic on October 1, 2019. We hope you will join us!

If you have experience with taking the EPPP and have tips to share for future licensure applicants, we would love to hear from you! Please feel free to share your experiences below.

References

Association of State and Provincial Psychology Boards. (2019). The Examination for Professional Practice in Psychology (EPPP): Frequently asked questions. Retrieved from https://cdn.ymaws.com/www.asppb.net/resource/resmgr/eppp_2/eppp__part_2-skills__faq_s.pdf

Bowman, N., & Ameen, E. (2018, June). Exploring differences in pass rates on the Examination for Professional Practice in Psychology. Retrieved from https://www.apa.org/pi/oema/resources/communique/2018/06/pass-rates.

DeMers, S. T. (2009). Understanding the purpose, strengths, and limitations of the EPPP: A response to Sharpless and Barber. Professional Psychology: Research and Practice, 40, 348–353. doi:10.1037/a0015734.

DeMers, S. T., & Schaffer, J. B. (2012). The regulation of professional psychology. In S. J. Knapp (Ed.), APA handbook of ethics in psychology: Volume 1: Moral foundations and common themes (pp. 453–482). Washington, DC: American Psychological Association.

Rehm, L. P., & Lipkins, R. H. (2006). The examination for professional practice in psychology. In T. J. Vaughn (Ed.), Psychology licensure and certification: What students need to know (pp. 39–53). Washington, DC: American Psychological Association.

Sharpless, B., & Barber, J. (2013). Predictors of Program Performance on the Examination for Professional Practice in Psychology (EPPP). Professional Psychology: Research and Practice, 44(4), 208-217.

Werner, E. (1981). A review of the examination for professional practice in psychology. Sacramento, CA: California Department of Consumer Affairs.

Navigating Faculty Dynamics in Graduate School

By Lydia HaRim Ahn, MS, Ed

It was my sophomore year of college, and I was eating Korean seaweed soup in my room as quickly as possible before my roommate came home from class. I did not want her to see me eating it and asking about it. This was what it was like for the first few years at a predominately White campus – I felt ashamed of my identity and wanted to fit in to be “like everyone else.”

Things changed drastically when I joined my first research lab with people who had similar experiences, and I was working with a faculty of color. He gave me various types of work responsibilities but also would personally check in with me. I remember before my GRE, we sat down at a coffee shop and he taught me basic math skills that I had not used in Image result for mentorship over ten years. We talked about our experiences of racism and discrimination on campus. For the first time in a while, I felt like I could truly be myself. I did not need to be “on” all the time. This led me to apply to graduate school, where I have been extremely lucky to also have a supportive advisor who has made an enormous impact in both my professional and personal life. I have been fortunate to have advisors and mentors who believe in me and my capabilities. This led me to my decision of wanting to continue in academia and mentoring students.

Research has shown that for Counseling Psychology graduate students, advisor support reduces burnout and predicts career choice satisfaction (Clark, Murdock, & Koetting, 2009). However, not all graduate students are fortunate to have such supportive advisors. Advisees who are unsatisfied with their advisors mention infrequent meetings with their advisors, lack of research guidance, limited encouragement to attend conferences, and often do not feel comfortable discussing professional issues (Schlosser et al., 2003). Working with faculty can be especially difficult when faculty take advantage of the power differential, leaving students vulnerable to feelings of powerlessness and hopelessness. Without support and advocacy from faculty advisors, students are often left feeling isolated and alone. Students of color may be even more at risk when faculty recreate power structures that are happening outside of the classroom.

So how do we as students navigate faculty dynamics in graduate school? Faculty have mentioned that communication problems (Knox, Schlosser, Pruitt, & Hill, 2006) create difficulties in the advisor-advisee relationship. As communication is the key to any relationship, routinely communicating with faculty members about our needs may be beneficial. However, it may be that some professors do not take into account your concerns and stressors in graduate school. Irrespective of the reasons, I suggest finding your support system, your “tribe.” Navigating graduate school and faculty dynamics can be challenging and thus it is important to find people we trust to share our experiences. If it is difficult to find this relationship on campus, try and connect with a mentor elsewhere. For example, Division 45 offers a mentoring program here and APAGS lists other resources for mentorship. Search, email, and reach out to faculty members you are interested in working with. Lastly, if possible, find a trusted therapist who you can open up to and share your experiences. The most important piece of advice I received upon entering graduate school was to find a therapist, and I feel fortunate to have had the opportunity for personal growth and development during this challenging stage of life.

Graduate school can be a difficult experience, and the relationship with faculty members is a unique one, given the close working relationship and the inevitable power dynamics at play. APAGS supports and believes that every future psychologist deserves a quality, transformative mentoring relationship but recognize that this is not always the case. My hope is that everyone can find a mentor who helps them be proud of their work, proud of who they are, and never feels ashamed to be eating seaweed soup. But if this isn’t the case, remember that you are not alone.

References

Kovach Clark, H., Murdock, N. L., & Koetting, K. (2009). Predicting burnout and career choice satisfaction in counseling psychology graduate students. The Counseling Psychologist, 37(4), 580-606.

Knox, S., Schlosser, L. Z., Pruitt, N. T., & Hill, C. E. (2006). A qualitative examination of graduate advising relationships: The advisor perspective. The Counseling Psychologist, 34(4), 489-518.

Kovach Clark, H., Murdock, N. L., & Koetting, K. (2009). Predicting burnout and career choice satisfaction in counseling psychology graduate students. The Counseling Psychologist, 37(4), 580-606.

Additional Resources

https://www.apa.org/apags/governance/subcommittees/student-resources

Self-Care on a Student Budget

When I hear the term self-care I automatically envision myself relaxing at a luxurious spa with cucumbers over my eyes, laying in a mud bath from a natural hot spring. Unfortunately, I have never actually been to a spa, although I definitely plan to treat myself to a fancy spa day when I am no longer a broke college student! Nonetheless, self-care to me has almost always involved spending money whether that be $5 towards a bubble tea, or a bit more on a new shirt or iPhone accessory. The same rings true for many of my friends, particularly my best friend, Sara, who takes multiple trips to the mall every week in the name of self-care and in the spirit of “Treat yourself.”

And you absolutely should treat yourself, right?

You, hardworking YOU, deserves that brand-new item you’ve been eyeing or those concert tickets you’ve been contemplating for weeks. However, it is important for us to understand that engaging in self-care does not require you to spend this week’s pay check. In fact, self-care does not require you to spend any money at all.

Yesterday, for APA’s Active Week 2019, employees were given the opportunity to go to the U.S. Botanic Garden for about an hour and a half in the morning. Eight of us set off at 9:45 am on a mile-long walk to the garden and were back a little bit after 11:00 am. This was my very first time at the Botanic Garden and it did not disappoint (except for the fact that the Butterfly Garden included no actual butterflies but that is a conversation for another time). I have always loved flowers ever since I was a child and the Botanic Garden offered an impressive array of flowers and plants found all over the World. The excursion gave each of us a chance to get away from our desks for a bit during the work day, and the views on the walk there as well as at the actual garden itself were a huge bonus. It was a great experience and I plan to go back and visit soon.

Orange Marmalade Firecracker Flowers at the entrance of the U.S. Botanic Garden

The excursion to the Garden was something I didn’t know I needed but I came back feeling refreshed and content. I was ready to get back to work after the short break and I didn’t realize until after that I was practicing self-care. The term itself is defined as any activity that we do deliberately in order to take care of our mental, emotional, and physical health. Self-care seems easy enough, but it is often forgotten and overlooked, especially when life gets busy. There are numerous benefits of self-care including better physical and emotional health, improved mood, and reduced stress and anxiety, to name a few. Self-care also combats burnout which can occur when you exert all your energy towards taking care of others, while neglecting yourself in the process. So, in conclusion, engaging in self-care is extremely important to live your best, most meaningful life. Although you may not necessarily be able to take a walk to the Botanic Garden, do something that makes you happy, whether that be taking a quick a walk, getting an extra hour of sleep at night, spending some time with a friend, or a taking hot bath before bed. Make sure to check in on yourself, find time to unplug, and take the time today that will have a positive impact on you going forward. Treat yourself!

Sources referenced:

Not sure where to start on your self-care journey? Here are some helpful links:

Sydney Wade is a rising college senior attending school in Austin, TX. She is originally from Maryland and spent this summer working as an intern for APA’s Early Career & Graduate Student Affairs.

Where Science Meets Policy Part 4: Writing a Policy Brief

Mary Fernandes, Elyse Mowle, & Melanie Arenson

Why do we need it?

As mentioned in a previous blog post, there is a well-known lack of consistent translation of scientific research into public policy. Researchers and policy-makers often have differing timescales and incentives that contribute to limited communication between the two groups. For example, policy-makers are often required to work under strict deadlines to produce immediate policy results, and their time-constraints frequently preclude them from staying up-to-date on the vast scientific literature pertaining to the issues they are interested in. As a result, research findings that are easily digestible are more likely to be read, utilized, and acted upon. Researchers have the ability to coherently summarize their work into a short document that provides clear recommendations for policy and practice. In fact, 79% of policy-makers identified policy briefs as valuable communication tools (Jones & Walsh, 2008).

So, what is a policy brief?

A policy brief is a short document that summarizes important research findings about a specific problem to a lay audience and makes recommendations for improving the identified problem. The document is typically focused on a single topic, and is no more than 2-4 pages or 1,500 words long. A policy brief is written with an audience in mind, and should be easy to understand without specialized knowledge or additional reading. Succinct, simple, and smart. The main goal of any policy brief is to convince the audience of the urgency of the problem, as well as to provide feasible, evidence-based solutions to it, and calling the policy-maker(s) to action.

Who is it for?

Policy briefs are used by local, national, and regional policy stakeholders. Other individuals or groups who have access to the policy making process (including nonprofits, government advisers, lobbyists, Think Tanks, and the media) also use policy briefs.

How do I write one?

First, identify the aim of your policy brief. What are you trying to achieve, or communicate to your reader? Next, consider your audience. What is their technical knowledge? How familiar are they likely to be with the issue? What information will they need? With your audience and aim in mind, target the key aspects that a policy-maker will want from your policy brief:

Title- It should be informative and short.
Summary- This is like an abstract, but without the jargon. It summarizes the overview of the problem and makes evidence-based recommendations. This section should appear on the top of the first page of your brief and it should draw the reader in.
Introduction- Provide an overview of the context of the problem. Identify a few salient points that provide support for the urgency of the topic. Also, give an overview of the research objectives, findings, and conclusions. Include why this problem is important to the audience, and if applicable, why previous efforts to solve the problem have failed.
Methods/Approach and Results- Here, explain the methodology used in the presented research as well as the results of the project(s). Distill the results into a few, key take-home messages. Be sure to simplify the language and avoid technical terms.
Conclusions- Provide clear conclusions based on the data, and express them in an assertive manner, without scientific jargon. Solidify the key take-away from the study.
Policy Recommendations- Propose solutions that are relevant to the stated policy problem. The brief should systematically and clearly describe the evidence in support of the proposed solutions. Recommendations should be specific and backed by the evidence you outlined. Ensure that your recommendations are appropriate for the audience. All solutions should be realistic and feasible (cost-effective, reasonable, etc.). State the next steps precisely.
References- Include a list of the references used and provide resources they might be helpful. A large list of references, akin to a research paper, is unnecessary and even unhelpful – this should be a small section of your brief!
Disclaimers- Just like in a presentation, detail any funding sources that contributed to the research presented. Also list contact details and institution disclaimers that might be necessary.
Visual aids/Infographics- Infographics and visual aids often make detail-heavy content more digestible and appealing.

When you’re done, ask yourself if the document answers these questions:

What is the purpose of the brief?
What problem does it hope to address?
What recommendations does it make to address the problem?
Does it pass the breakfast test? A good policy brief should be able to answer all of these questions and be read in the time that it takes to drink a coffee over breakfast.

Style Tips

Draw the reader in by starting with your conclusions. Clearly state what the problem is from the start.
Break up the text – use headings, bulleted lists, font or call-out boxes to highlight key points. Have healthy margins and plenty of white space.
Write using active rather than passive voice
Write for clarity and simplicity. Can you say in one sentence what you just said in two or three? Think “economy of words”. For example:
- Did you use a bigger word when a smaller one would have worked? (“Operationalize” versus “define”)
- Can you cut words or phrases without changing the meaning? (“Due to the fact that” versus “because”)

How to disseminate your policy brief?

Share with key players in the policy field. Have copies ready for policy-makers whom you meet with.
Share with other stakeholders who might influence the policy-making process (see our ‘Science Meets Policy: Part 1’ post for information about stakeholders). If emailing, mention in the body of the email why the brief will be useful to them. Make sure that the subject of your email is concise, says everything that you want to convey, and eye-drawing (ex. “Importance of Science Influencing Policy: A Policy Brief).
Strategically plan the timing of your dissemination (i.e. during preparations for policy shifts, while there is media attention, etc.).
Submit the brief to journals that specify a call for briefs (https://www.apa.org/pubs/journals/ipp/call-for-policy-briefs).
Combine with other communication methods such as posters and presentations at conferences.
Use social media to disseminate your brief (e.g. newsletters, Facebook, Twitter).

Further Resources

Sample Briefs

References
Jones, Nicola & Walsh, Cora. (2008). Policy Briefs as a Communication Tool for Development Research. ODI Background Notes.

gradPSYCH Blog

APA's Food for Thought for Psychology Students

Increasing Research Transparency in Psychological Science: How Can Graduate Students Participate?

The Examination for Professional Practice in Psychology (EPPP): What’s race got to do with it?

Navigating Faculty Dynamics in Graduate School

Self-Care on a Student Budget

Where Science Meets Policy Part 4: Writing a Policy Brief