Category Archives: Graduate School

The Argument Against P-Values

There is concern that a substantial proportion of published research presents largely false findings (Ioannidis, 2005). This problem, in part, stems from social science’s reliance on null hypothesis statistical testing (NHST) given the incentive to achieve statistical significance (e.g., publications, grant funding). Research in the social sciences has historically adopted a Frequentist perspective, primarily reporting results using a dichotomous reject or non-reject decision strategy based on whether some test statistic surpasses a critical value and results in a statistically significant p-value (usually p > 0.05). Although useful in several ways, p-values are largely arbitrary metrics of statistical significance (Greenland et al., 2016), and they are often used incorrectly (Gelman, 2016). The use of p-values encourages a binary mindset when analyzing effects as either null or real, however, this binary outlook provides no information on the magnitude or precision of the effect. P-values can vary dramatically based on the population effect size and the sample size (Cumming, 2008). This reliance on an unstable statistical foundation has been discussed in the literature (Wasserstein, 2016), and while some journals have taken matters into their own hands (for example, Basic and Applied Social Psychology banned p-values and NHST), the field of psychology has largely failed to address the concerns raised by the use of NHST.                                     

Research is moving towards adopting new statistics as best practice, relying instead on estimations based on effect sizes, confidence intervals, and meta-analysis (Cumming, 2014). We, as graduate students in training, are in a position to push towards thinking in terms of estimations and away from dichotomously constrained interpretations. In contrast to the binary nature of p-values, a confidence interval is a set of plausible values for the point estimate. Although perhaps wide, the confidence interval accurately conveys the magnitude of uncertainty of the point estimate (Cumming, 2014), as well as the level of confidence in our results. For example, a 95% confidence interval that includes values for a population mean, μ, indicates 95% confidence that the lower and upper limits are likely lower and upper bounds for μ. The APA Publication Manual (APA, 2020) specifically outlines recommendations to report results based on effect size estimates and confidence intervals, rather than p-values. P-values are not well suited to drive our field forward in terms of precision and magnitude of estimates. Researchers should therefore focus on advancing the field by gaining an understanding of what the data can tell us about the magnitude of effects and the practical significance of those results. It is important for graduate students to adopt practices to produce reproducible and reliable research. One way to do so is to move beyond p-values.

How to move beyond p-values:

  • Prioritize estimation instead of null hypothesis testing or p-values
    • Formulate research questions in terms of estimation. Ex: How large is the effect of X on Y; to what extent does X impact Y?
  • Report confidence intervals and corresponding effect sizes
  • Include confidence intervals in figures (preferred over standard error bars)
  • Make interpretations and conclusions based on the magnitude of the effects rather than a dichotomous decision based on “statistical significance”

References                                                     

American Psychological Association. (2020). Publication manual of the American Psychological Association 2020: the official guide to APA style (7th ed.). American Psychological Association.

Cumming, G. (2008). Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science, 3, 286– 300. doi:10.1111/j.1745-6924.2008.00079.x     

Cumming, G. (2014). The new statistics: why and how. Psychological science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966

Gelman, A. (2016). The problems with p-values are not just with p-values. The American Statistician, 70(10).

Greenland, S., Senn, S.J., Rothman, K.J. et al. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31, 337–350. https://doi.org/10.1007/s10654-016-0149-3                                          

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. Retrieved from http:// www.plosmedicine.org/article/info:doi/10.1371/journal .pmed.0020124

Wasserstein, R.L., & Lazar, N.A. (2016). The ASA’s Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70:2, 129-133, DOI: 10.1080/00031305.2016.1154108

Written by Marianne Chirica, an APAGS Science Committee member and a third-year graduate student in the Psychological and Brain Sciences Ph.D. program at Indiana University. Feel free to reach out to Marianne with any questions you may have!

Writing Commentaries

By Samuel Accuff, PhD and Melanie Arenson, PhD

Many journals offer the opportunity to contribute commentaries, but many graduate students are uncertain about how to write one or think these are reserved for senior career scientists. This couldn’t be farther from the truth (see this call in Translational Issues in Psychological Science), so we wrote a blog post to demystify the process and help you get started! 

What is a commentary and why should I write one?

A commentary is a paper which discusses and expands upon a recently-published article in a peer-reviewed journal. Often commentaries are short (<1000 words), do not require additional data, and focus on a piece of the results or impacts that the original article did not discuss at length. 

So, why write one? Many of us are drawn to research to contribute knowledge and to engage in the conversation of science. Commentaries offer unique advantages relative to other article types to do just that. First, due to their brevity, they require less time. Second, a commentary is a product that emerges from the critical thinking that hopefully naturally occurs while keeping up with the literature. Third, commentaries are opportunities to collaborate with your research community on interesting and stimulating issues. We are in the era of team science, and advancement typically comes through the furnace of critical thinking and friendly, collegial debate. In short, this is an opportunity to engage with your scientific community and question the beliefs of yourself and others, in a brief format that engages you with the literature and results in a publication. What could be better?

What’s the process?

  1. Stay up to date with the current literature: Commentaries discuss recently-published articles (ideally published within 6 months, at most a year). You also want your commentary to reference the most up-to-date literature, and your future directions to be novel. To do so, it is important to stay up to date! 
  2. Think critically: Commentaries are fundamentally papers that critically examine an existing work. This is not to say they are critical in nature; in fact, they are often far from this. However, there is likely a result, application, implication, or future direction the authors did not touch, but you can!
  3. Express interest: Often, journals have a specific call for commentaries and may require that you reach out to express interest in writing one. Draft a brief summary and reach out before you write the commentary. You don’t want to write a commentary that the journal isn’t planning to review!
  4. Get writing: Connect with collaborators to determine the division of responsibilities and  timeline. Then, carve out time in your calendar and get writing!
  5. Go through review: After your commentary is submitted, it will go through a review process that may be slightly different than other articles to foster a faster publication timeline (e.g., involves fewer reviewers, often reviewed by editorial board members).

The nitty gritty – things that help, and things that can hurt

Journals only have so much space, and editors are picky about what they publish. Here are some tips and tricks to enhance your commentary, and some things to avoid. 

  1. Identify an idea that is impactful and builds beyond the scope of a previously published article. Identify a practical or theoretical implication not articulated by the authors, or illuminate a limitation/future direction for research (remember to be respectful). A restatement of the findings is not unique and likely won’t publish. Related, avoid vague statements and instead articulate reasons why your take is important. Be specific, and contribute something novel. Connect the paper to a bigger picture.
  • Avoid lengthy summaries of the study, and instead point readers to the published article. Lengthy summaries take space, and by their nature are not commenting on the existing article.
  • Commentaries often have greater flexibility in voice, with opportunities to show more personality. Use catchy phrases to create brief, memorable summaries of your main idea. That said, you are still a scientist. Do not make claims that cannot be empirically tested; stick to the evidence, and communicate a healthy dose of skepticism. 

This is your opportunity to join the conversation of science. You may have lost touch with this desire due to tests or the stress of managing your dissertation, but isn’t that why we are here? Find a topic or recently published paper and share your thoughts with a colleague. Push the limits of what the study found, and challenge the assumptions. Sit long enough to articulate that feeling you’ve been having about the field that you hold dear. You never know how your engagement might impact the direction of science!

5 Tips to Start Your Graduate School Year Off Strong

By Ashley Wojcik

The beginning of a new school year can bring about a mix of emotions. It is normal to feel excited about the start of a new chapter, but also anxious for what is ahead. Graduate school specifically invites financial, social, personal, and intellectual challenges that can be difficult to manage. In this post, we have compiled some tips for surviving and thriving in a new year of graduate school. 

  1. Communicate With Your Support System Before Starting School 

Before going into a new or stressful environment, it can be helpful to talk with your family and friends about how you are feeling and what they can expect in terms of your availability. Some people struggle to garner support from those around them in their pursuit of graduate education, so you should be prepared for those difficult conversations. Remember to be honest about why it is important for you to go back to school, what you hope to accomplish, and how they can best support you. 

  1. Prepare for a Busier Schedule 

Going back to school as a graduate student can be a different experience than your undergraduate years. In many cases, you may have already spent time working and will have to readjust to the academic lifestyle. If you have a family or children, this can be another commitment to juggle. Even if you have gone straight into graduate school, you may find that the combination of research, courses, and other responsibilities such as being a TA is a heavier load than you experienced as an undergraduate. It is important to set realistic expectations so you can be kind to yourself. Forgive yourself in advance for the late nights and busy days, while also remembering to build in some time for self-care.  

  1. Give Yourself Grace in the Beginning 

Summer breaks notoriously lead to some loss of information learned in the previous school year. This depletion only increases if you have taken more time off from school. When starting graduate school, you should be ready for an adjustment period academically. Prepare yourself to endure a learning curve in the beginning and to allow yourself time to review previous material before you can learn new concepts.  

  1. Understand That Student Life Has Changed 

If you spent time out of school, you may quickly recognize that things have changed since you were last a student. Student culture, expectations, and resources are changing all the time. Additionally, the graduate student experience is inherently different from the undergraduate, so even if you went straight into graduate school, you should prepare for those differences before you begin. For example, many people complete their graduate degrees while working or juggling other things, so the schedule may be different than you are used to. Your curriculum may be more research-based, with smaller classes, and closer connections to your professors than you were used to at the undergraduate level. 

  1. Keep an Open Mind 

Like anything else, your mind is subject to change in grad school, and you should be open-minded when going into graduate studies. Your area of focus may shift or change completely through your education, and you should be prepared for that and welcoming to it. After all, that is the benefit of a graduate education. The stress you endure is the cost of the knowledge you gain about yourself and the impact you want to have on the world. 

APAGS Asked the 2023 APA Presidential Candidates Questions about Grad Student Issues…. Here is What they Shared

2023 is a huge year for graduate students in APA! For the first time, Graduate students that have been APA members for at least one year can vote for the 2023 APA President-Elect, APA Board of Director’s Members-at-Large, Apportionment Ballots, and Bylaw amendments since being enfranchised in Fall 2020. 

To help graduate students learn more about the candidates and their stances on issues at the forefront of graduate student members, the members of the APAGS Elections Work Group have asked candidates to submit their responses to the following three questions:

  • As APA President, what do you envision graduate students’ roles being in APA’s response to ongoing threats to human rights amidst changing societal circumstances (e.g., affirmative action, removal of EDI courses)?
  • Given research data suggesting financial burden is a significant burden for graduate students, as APA President, what would be your role in responding to the growing student debt crisis?
  • With increased graduate student representation across APA, as APA President, how would you foster leadership development for graduate student members in leadership?

As a committee, APAGS thanks all candidates for their responses and commitment to graduate students! Please note we have organized candidates in alphabetical order based on their last names. You can also view all candidate statements here.

As APA President, what do you envision graduate students’ roles being in APA’s response to ongoing threats to human rights amidst changing societal circumstances (e.g., affirmative action, removal of EDI courses)?

Eric Butter

Central.  Graduate students are central to our work on protecting and expanding human rights.

Our graduate students are our most credible spokespeople and advocates for responding to the multitude of threats to human rights. APA’s graduate students are more diverse, more globally represented and connected, and more cohesive in their perspective on human rights than any other generation of career status within the Association.  Additionally, APA’s graduate students have a responsibility to our future that underscores their standing when addressing human rights.

We should not ask graduate students to stand alone however.  We must invite graduate students to co-design our responses to each threat to human rights we encounter and even the ones we can anticipate encountering. These co-designs should be on-going, planned, and ready to execute. We need to respond, not react, to these threats with a verve reflective of the urgency of our next generation of psychologists and the deliberateness of a collaborative, action-oriented response developed in concert with APA staff and all relevant governance groups.

With graduate students centered in this work, we have the opportunity to not only engage our most motivated and impacted stakeholders, but also our most informed by emerging frameworks, models, and research.

Debra Kawahara

With the ongoing threats to human rights amidst changing societal circumstances, I believe that graduate students can be active participants in APA’s response to critical social issues. The direct impact of these issues on graduate students’ lives, physically, emotionally, and mentally, as well as on their education and training, is real. Many may feel invalidated, threatened, and disenfranchised by the current threats, and I have seen firsthand how graduate students can effectively advocate, protest, and be activists. As APA President, I would like to harness this energy and dedication by forming an advisory/consultative group of graduate student members to provide me with the information about how best to give voice and empower graduate students to act. I would seek a diverse group in terms of an intersectionality of identities, specialties, and stage in their graduate studies. I will listen to what graduate student members believe is the best way to mobilize their efforts in promoting change and advancing human rights in APA and beyond and then work collaboratively with the graduate student members to put this into action.

Margaret Kovera

The recent Supreme Court decisions eliminating considerations of race in college admissions just made diversifying higher education much harder. And the attacks on LGBTQIA+ people and DEI material in Florida classrooms is appalling. But as psychologists, we have the tools to fight these trends and I include graduate students in that “we.” As psychologists, we have the expertise to develop new ways of evaluating admissions materials that will allow us to identify a diverse student body that is prepared for graduate study. We must continue to find new avenues for disseminating our science supporting human rights in this very polarized political environment. With attacks on gender-affirming care, DEI initiatives, and reproductive care, we need to make our science known. As inaugural chair of the Amicus Chair Expert Panel, I have been working to involve students in doing environmental scans of our state and district appellate courts for cases that can be informed by our science. We recently submitted a brief in Colorado in an attempt to protect minors from conversion therapies and are actively seeking a case in which we can weigh in with our science supporting access to reproductive health care. We need help with this environmental scanning, and whether I win this election or not, I would welcome APAGS members to contact me if they wish to play a role in our judicial advocacy efforts.

Grant Rich

Graduate students are the future of psychology. They must be involved, and psychology must be more than merely an intellectual exercise.  Our specialized research and journals are valuable, but psychologists must resist siloes and communicate and advocate for positive social change. Such work means utilizing research to challenge threats to affirmative action and DEI courses. This also means communicating with policymakers and the general public. Psychology graduate students are more diverse than a generation ago, in terms of gender, race/ethnicity, religion, national origin, LGBTQAI2S+ status, disability, and age. Involving our diverse body of graduate students in meaningful ways, such as participation as full, voting members of committees and workgroups will be positive steps towards effective advocacy, and for building our leaders for the future. Graduate students who have recently benefitted from affirmative action, DEI courses, and LGBTQAI2S+ materials in high schools/colleges also provide essential points of view to share with psychology leaders of today, policymakers, and other groups involved with the decision-making. I am optimistic, that working collaboratively with graduate students, APA divisional and national leaders, psychology and society can move forward through these challenging societal circumstances. 

I also advocate for more divisional student listservs and social media outreach.

Given research data suggesting financial burden is a significant burden for graduate students, as APA President, what would be your role in responding to the growing student debt crisis?

Eric Butter

Free.  Graduate training in psychology should be free.

While we must continue to advocate for loan forgiveness programs, we must also explore new strategies to mitigate the financial burden of graduate training in psychology to begin with.  Loan forgiveness is only addressing the challenge after debt has been accrued. I have heard no conversation about alternative funding models for graduate education in psychology. 

Private, for profit graduate school tuition is crushing. It hobbles a career and a life before it starts.  There are too few public university, tuition wavier programs available and too few slots in those programs. Our nation’s greatest public universities have abandoned scientist-practitioner and practitioner-scholar programs for clinical science programs with highly selective admissions.  We have allowed our institutions of higher education to ignore their duty to the public good. 

APA can help our country imagine something different.  With the mental health crisis gripping our nation, public funding for professional training in psychology is a compelling issue. With a combination of direct funding to graduate schools and expansion of service learning programs where national and community service could earn tuition waivers, free graduate school is not a delusion. We should aim high and be bold.

Debra Kawahara

Student loan debt is a serious issue for many individuals who pursue higher education, and President Biden has been proactively seeking to address this issue through student debt forgiveness. However, this action does not address the future. As APA President, there will be several actions that I will facilitate. First, there are congressional representatives who are proposing the elimination of federal work study. APA, along with other professional organizations, should immediately advocate against this action as it would significantly hurt students who rely on these monies as they matriculate through their graduate studies. Also, APA can continue efforts in advocating and soliciting for more funding and scholarships for graduate psychology students from both public and private entities such as philanthropic foundations, healthcare organizations, and public and private organizations or corporations. Another possible idea is for APA to advocate for a program that provides tuition support for students instead of students signing educational loans. In return for the tuition, students could agree to work in areas that are in high need of psychological services or with populations that are underserved. Working with APAGS, the APA Advocacy Office, and the APA directorates and offices will be critical in moving these actions forward.    

Margaret Kovera

In the past, I have participated in APA advocacy on the Hill to increase loan forgiveness funding and would continue and expand on those activities if elected President. Unfortunately, there is only so much APA can do to set costs and funding opportunities at universities, but APA could be proactive in providing educational materials for graduate students that address financial literacy, including the tax implications of fellowships, small grants, as well as costs of attendance (e.g., conference travel), so that students have the requisite knowledge to make difficult financial decisions.

Grant Rich

In my view, step one is for psychology faculty and administrators and APA divisional and national leaders to be better informed about the changes in student expenses in higher education, such as tuition and fees, and the student debt crisis. Some psychology faculty may assume that the challenges are similar to what was present a generation ago. The reality is clear; evidence shows dramatic and disproportionate increases in expenses for students recently, far outpacing general inflation and cost of living increases in broader society.

I firmly believe higher education institutions must clearly and transparently communicate costs and fees to students and must be clear as to post-graduation realities. Colleges and universities must annually conduct surveys to assess information about average student starting salaries and pay trajectories for students and alumni in psychology, and facts about student placements (such as in academe, or outside of academe, and visiting vs. tenure track positions).

I will advocate for national APA level committees and workgroups on this topic, and will advocate for student representation on these groups; I believe representation from multiple, diverse graduate students will be most beneficial as student experiences vary widely. APA can disseminate results from such fact-finding inquires.

With increased graduate student representation across APA, as APA President, how would you foster leadership development for graduate student members in leadership?

Eric Butter

Coaching.  Coaching is underutilized for our graduate students.

For graduate student leadership development, we need to support a learning culture across APA governance that goes beyond mentorship.  Mentorship is important and it happens.  Student leaders gravitate toward dynamic, more senior leaders who are generous with their time and resources. We have formal mentorship programs as well. Often mentors become sponsors.  Many graduate students impress more experienced APA leaders as well as senior APA staff who sponsor them for future opportunities.  This is important.

Yet, mentorship implies a hierarchical relationship.  Typically, an “expert” and more senior professional is providing advice, guidance, answers, skills, and solutions to a more “junior” professional in training.  Coaching is more deliberate, student-centered, and amplifies the student voice.  Coaching can be bi-directional. The coach helps the student become the best version of themselves, while the mentor helps the student become a good version of the mentor.

As President, I would initiate a “culture of coaching”.  Our graduate student leaders would have an opportunity for a confidential, coaching relationship where each student could focus on their own development as a leader, developing their own wisdom, learning from their successes and mistakes, and finding their own way in APA.

Debra Kawahara

Fostering leadership development for graduate student members is important as I believe that graduate students are the future of the field and APA. In August 2022, the Leadership Development Institute (LDI) was created, and the next steps were outlined. As APA President, I would work collaboratively with APAGS and LDI to conduct a needs assessment to identify and communicate established and future programs for leadership development of graduate student members. In addition, providing more leadership opportunities for graduate student members within APA and beyond is important as these experiences build leadership skills. These would include, but not limited to, establishing graduate student positions in the APA directorates and offices; key agencies, foundations, and funding sources such as NSF, NIH, NIMH, CDC, VA, DoD, EPA; public and private organizations; and local, state, and federal departments. The leadership opportunities will hopefully cover the wide range of specialties within our field and permeate those diverse areas with psychologists and our science and knowledge. Lastly, for all of my presidential initiatives, I will have at minimum one graduate student member on each of the initiatives to ensure that the graduate student members’ voice and presence are represented.  

Margaret Kovera

I have experience creating opportunities for graduate student leadership during my previous Editorship of Law and Human Behavior (creating a student editorial board, where student reviewers were mentored by editorial board members so that they could learn the ins and outs of peer reviewing). I would include student members on task forces and other leadership groups so that they could learn through participation in leadership. When mentoring my own graduate students, I find that treating them as colleagues from the very beginning helps them grow their leadership abilities. I would hope to continue such an approach if elected. 

Grant Rich

I have an abiding interest in education and student success; this is clearly demonstrated by the fact I have three co-edited books on diversifying/internationalizing the teaching of psychology, and many chapters/articles on the topic, including in American Psychologist, and am a Division 2 Fellow (Teaching). I have extensive experience- over 25 years- teaching both in brick and mortar and online formats, and have received strong positive evaluations for my teaching both in the USA and abroad. I have supervised many dissertations and theses at several institutions.

My experience tells me that we must do a better job of supporting and involving graduate students early on and through their process, and beyond to the ECP (early career professional) stage. I will work at the APA and divisional level to assess graduate student leadership opportunities and endeavor to build more and better such roles!

Providing students with adequate financial and academic support will free up time for graduate students to more actively and fully participate in research, publishing, and committee workgroup opportunities that build careers, by facilitating graduate student participation in activities where they not only work with APA and divisional and faculty leaders, but also are taking on leadership roles now!

We hope that you will find this information useful as you decide how to cast your vote. Your voice matters! Look for ballot information via email. If you do not receive a ballot, please contact Aliza Epstein.

Supporting Transgender and Gender Nonconforming Colleagues and Clients: Why It’s Essential and How to Start

By Liz Deibel, APAGS Committee on Sexual Orientation and Gender Diversity Member

(Curious to learn more about this topic? Check out the APAGS upcoming webinar: Impact with Pride: Leveraging Political Resources to Thwart Anti-LGBT Laws. Register here.

The United States can be a really scary place for individuals of sexual orientation and gender diversity, particularly in the current political climate and rise of anti-trans legislation sweeping the country. It can be difficult for Transgender and Gender Nonconforming (TGNC) students to continue going about their daily responsibilities, as well as the clients we are working with. During this stressful time, it is important that both those of us in the LGBTQ+ community and our allies come together to support each other and advocate for equal rights. With each day, more states are attempting to strip the rights of LGBTQ+ people, particularly transgender and nonbinary individuals. If you know of a colleague or classmate who is being affected by the changing legislation, it may be helpful to reach out to them and ask if there is anything you can do to support them. 

How to Support TGNC Colleagues?

Graduate school is tough enough as it is without political strife piled on top of everything else. It is important that allies or privileged members of the LGBTQ+ community participate in advocacy to help protect their peers’ rights and safety. Emailing your representatives, signing petitions, and sharing resources can be a quick way to contribute to building protections for transgender and nonbinary community members. Additionally, encouraging professors or program directors to incorporate education on LGBTQ+ psychology can increase the number of gender-affirming psychologists going into the field, leading to more support and strength in numbers in the years to come.

Outside of the professional sphere, it can be really encouraging to simply check in with individuals in your life who identify as LGBTQ+ and ask how they are doing and if they need a supportive friend to talk to. Many queer folks feel frustrated with how little these discussions are occurring outside of LGBTQ+-majority spaces, and showing awareness of what is going on and how much it is affecting people’s mental health can provide validation to their experiences.

How to Support TGNC Clients?

The American Psychological Association has released a list of guidelines for working with LGBTQ+ clients, divided into sexual orientation diversity and gender diversity. Many of these guidelines touch on understanding the differences between biological sex, gender identity, and gender expression. They also acknowledge intersectionality, the way that gender and sexual orientation intersect with other facets of identity such as race, ethnicity, religion, and socioeconomic status. It is essential that psychologists understand the ways in which their implicit biases and the power dynamics in society influence the way of life for TGNC people. 

Psychology as a field needs to continue adapting to the changing times to ensure that we can provide affirming and compassionate care. In addition, individuals within our field should continue to address their own unconscious biases and prejudices about gender and sexual orientation to avoid bringing them into the room with a client. At practicum sites, students can initiate conversations about gender-affirming care and ways to ensure the site is adhering to professional guidelines and standards of care for TGNC clients.

In the same way that showing awareness of the situation with colleagues can be validating, it can have the same effect with clients. If the clinician can take the onus to approach the topic, even though it can be uncomfortable to bring up, it removes that additional responsibility from the client’s plate and acknowledges that you see them as a piece of a larger system. Clients do not exist in a vacuum, and being able to address macro-level issues or concerns can instill a feeling of being seen that the client may really need at this time.

References:

Liz Deibel (she/her/hers)
Clinical Psychology Doctoral Student, Roosevelt University