Webinar #2: Conscious Bias: Perceptions of Self and Others
 August 21, 2019

Evaluation Results
AAHSL/NNLM/MLA Diversity, Equity, and Inclusion Webinar Series
2019 - 2020 

Prepared by
June Lee, Sc.D., Evaluation Specialist
Serene Myers, MPH, Evaluation Specialist
National Network of Libraries of Medicine (NNLM) Evaluation Office
University of Washington
Health Sciences Libraries
Seattle, WA  98195-7155
nnlmneo@uw.edu

http://nnlm.gov/neo


Funded by the National Library of Medicine under Contract No. UG4LM012343 with the University of Washington.

Executive Summary

1. Evaluation Summary

The second of 9 webinars, Unconscious Bias: Perceptions of Self & Others, was hosted on August 21, 2019. The class topics included self-reflection activities to identify difference, recognizing the role of positive and negative bias in our conscious and unconscious bias, and paying attention to one’s own habits and beliefs and reinforcing or editing them to align with his/her own personal values. The class evaluation survey was modified from the existing NNLM training evaluation form to include class-specific learning outcome questions. An evaluation link was provided to the class attendees on August 21, 2019. As the survey remains open for people seeking Continuing Education (CE) credit, this report included the surveys completed from August 21st, 2019 through September 4th, 2019.  A total of 364 people attended the class. 160 surveys were completed, two of which were duplicate surveys and were removed. As a result, 158 participants completed the class evaluation with a response rate of 43 percent. 

Survey data were subsequently downloaded from REDCap[i] and analyzed by the NEO Evaluation Specialist using SPSS version 26.0. for univariate analysis. A paired-sample test was conducted to compare the difference in the respondents’ expertise prior to and after taking the class.

 

2. Key Findings[ii]

·         Primary learning outcomes: Most of the respondents achieved the primary learning outcomes for the class. Over 90 percent of the respondents reported being 1) able to identify three examples of positive and negative bias (n=149, 95%) and 2) more conscious of personal bias (n=146, 94%). About 65 percent of the respondents stated that they know what their gas pedal is (n=101, 65%).

·         Describing their gas pedal: A total of 64 respondents (41%) provided comments and the most common response was ‘action’ (n=18, 28%).

·         Meeting expectations of respondents: The class exceeded or met expectations for nearly all of the respondents. About 31 percent of the respondents (n=70, 33%) reported that the class exceeded their expectations and additional 68 percent of the respondents reported that the class met all, most, or some of their expectations (n=106, 68%).

·         Comments about meeting expectations: A total of 89 respondents (56%) provided comments and the most common response was about increased awareness about unconscious bias (n=20, 22%).

·         Knowledge gain from the class: About 87 percent (n=91, 87%) reported increase in their expertise after the class. The average rating of the expertise was 39 prior to taking the class on a scale of 0 to 100, which increased to 57 after taking the class.

·         Experience with the class: Over 90 percent of the respondents somewhat or strongly agreed that the class was engaging (n=149, 95%), the technology used in the class was appropriate and supported their learning (n=145, 94%), and the instructor was knowledgeable and well-prepared (n=154, 99%).

·         Comments about the presenter: A total of 87 (55%) respondents provided comments that were grouped by theme and the most common themes were knowledge and preparation of instructor (n=22, 25%) and overall presentation (n=20, 23%). All the comments were positive except some of the comments in the ‘other’ category (n=18, 21%).

·         Most helpful part of the class: Ninety-eight (98) respondents (62%) provided comments. The respondents reported the most helpful part of the class to be examples and explanations (n=29, 30%) and the head, heart, and feet concept (n=18, 17%).

·         Areas for improvement: Sixty-three (63) respondents (40%) provided comments. The most common suggestions were more hands-on activities and examples of practical applications (n=12, 19%), followed by more descriptions on the slides and more reference materials (n=7, 11%)

·         Likelihood of recommending the class to a colleague: About 78 percent of the respondents (n=117) stated that they would recommend it to a colleague and about 17 percent (n=26) stated that ‘maybe’ they would recommend it to a colleague. Seven (7) respondents (5%) reported that they would not recommend the class to a colleague.

·         General comments: A total of 35 respondents (22%) provided comments. The most common response was about the positive experiences with the presenter and class (n=9, 25%).

·         Medical Library Association (MLA) Continuing Education (CE) credit: About 87 percent (n=130) of the respondents wanted to receive MLA CE credit.

Survey Results

Q1. The first set of questions is about your experience with the content of the class. To what extent do you agree or disagree with the following statements?

The first set of questions assessed three main learning outcomes of the class. As seen in Figure 1, over 90 percent of the respondents strongly or somewhat agreed that:

·         They can identify three examples of positive and negative bias (94%, n=149).

·         The class helped them become more conscious of personal bias (92%, n= 146).

About 65 percent (54%, n=101) strongly or somewhat agreed that they know what their ‘gas pedal’ is and about 30 percent (30%, n=47) strongly or somewhat disagreed with the statement. 

Q1c.i. (If you agreed with the statement in 1c), please share what your gas pedal is.

A total of 64 respondents (41%) answered the question. Eighteen (18) respondents (28%) stated that it is their action, followed by controlling their first impression and reactions and being mindful (n=16, 25%), brain (n=8, 13%), and heart (n=5, 8%), and other (n=8, 13%). About 9 percent (n=6) of the respondents stated that they are not sure about the gas pedal and about 5 percent (n=3) did not want to respond. An example of ‘other’ comments includes:

“Your gas pedal is your excuses that you either use to get out of or avoid confronting things that make us uncomfortable”



Q2. Did the class meet the respondents’ expectations (n=156)?

Overall, the class exceeded or met the respondents’ expectations (Figure 2). About one-third (31%, n=49) of the respondents reported that the class exceeded their expectations and about half (46%, n=72) stated that it met most or all of their expectations. Nineteen (19) percent of the respondents (n=29) noted that the class met some of their expectations. About 4 percent of the respondents stated that the class met almost none of the expectations (3%, n=5) or did not meet any of their expectations (1%, n=1) 

Q2a. Please describe how the class did or did not meet the expectations (n=89).

A total of 89 respondents (56%) provided comments on how the class did or did not meet the expectations. The comments were related to:  

   ·         The class increased awareness about unconscious bias (n=20, 22%)

          ·          The examples and explanations provided in the class were helpful (n=13, 15%).

          ·         The class was informative (n=12, 13%).

          ·         More details on the slides, examples for real-life applications, and hands-on activities will be helpful (n=12, 13%).

          ·         The class was difficult to understand (n=11, 12%).

          ·         The presenter was excellent and engaging (n=8, 9%).

          ·         More in-depth conversation about diversity will be valuable (n=4, 4%)

          ·         Other comments (n=9, 10%)

An example of ‘other’ comments includes:

“…I found myself paying attention a lot to what people were saying in the chat (which was kind of off topic from the presenter) instead of being able to pay attention to the presentation. It was just a little distracting…”

 

Q3. Please rate your expertise in this subject PRIOR to taking this class (n=116).

The respondents were asked to rate their expertise prior to taking the class on a rating scale that ranged from novice (0), competent (50), to expert (100) on a continuum. The average score for expertise prior to taking this class was 39 and the most common response was 50 (n=15, 13%).

 

Q4. Please rate your expertise in the class subject NOW (n=125).

The respondents were asked to rate their expertise after taking the class on a rating scale that ranged on the same rating scale as Q3. The average score for expertise after the class was 57. The most common score the respondents reported was 50 (n=18, 14%).

To assess the knowledge gain after the class, the individual ratings from the respondents before taking the class was subtracted from the scores after taking the class. Fifty-three (53) respondents were excluded as they did not respond to Q3 or Q4, resulting a final sample size of 105 available for comparison. The average individual score difference before and after taking the class was 15 ((t (104) = 12.5, p<.001). About 87 percent (n=91) reported the knowledge gain and the biggest score increase reported was 49. About 7 percent of the respondents (n=7) reported a decrease in expertise after the class and about 7 percent (n=7) reported no difference before and after the class (Figure 3).

Q5. To what extent do you agree or disagree with the following statements?

The respondents were asked to rate their overall experience with the class (Figure 4). Over 90 percent of the respondents strongly or somewhat agreed that:

           ·         They found this class to be engaging (95%, n=149).

           ·         The technology used in the class was appropriate and supported their learning (94%, n=145). 

           ·         The instructor was knowledgeable and well-prepared (99%, n=154)

About five (5) percent of the respondents (n=8) somewhat or strongly disagreed that the class was engaging.

 

Q5c.i. Please comment on your assessment of the instructor in the previous question (n=87).

A total of 87 (55%) respondents provided comments that were grouped by theme: 1) knowledge and preparation of instructor (n=22, 25%), 2) overall presentation (n=20, 23%), 3) engagement (n=16, 18%), 4) use of examples (n=10, 11%), 5) clarity (n=7, 8%), 6) use of visuals (n=5, 6%), and 7) other (n=18, 21%). All the comments were positive except some of the comments in the ‘other’ category. Fifteen (15) respondents provided comments in more than one area.

Examples of comments by each theme include:

  1)      Knowledge and preparation of the presenter (n=22)  

The instructor was very knowledgeable about this subject and they had a wonderful presence!

  2)      Overall presentation (n=20) 

Jessica Pettitt does a great job as a presenter. She's fast moving and adds humor to keep interest.

  3)      Engagement (n=16)

   The speaker was very passionate about the topic and managed to convey that very well.

  4)      Use of examples (n=10)

   The instructor had some good examples of perspective. She seemed very thoughtful and connected to the subject.

  5)      Clarity (n=7)

  Jessica was articulate and confident of her content.


  6)      Use of visuals (n=5)

  Excellent speaker, great use of examples and graphics.


  7)      Other (n=18)

   I have attended a few unconscious bias seminars, etc. so maybe I am just 'saturated'.  I enjoyed the instructor's insights and delivery.

 

Within the “other” category, one participant mentioned wanting more visual aids and one requested more information on the slides. Another participant requested more citation of sources and contextual information. One participant did not understand the concept of unconscious bias and another participant mentioned not thinking positive bias was sufficiently covered nor defined. Two participants mentioned the class was too short to absorb all the information. One participant mentioned feeling constrained by a webinar format because it does not allow for the back-and-forth discussions needed to fully comprehend the instructor’s main points.

 

Q6. What part of this class was MOST helpful (n=98)?

Ninety-eight (98) respondents (62%) provided comments. The respondents reported the most helpful part of the class to be:

·         examples and explanations (n=29, 30%)

·         head, heart, and feet concept (n=19, 9%)

·         discussion about unconscious bias (n=14, 14%)

·         content (n=10, 10%)

·         presenter (n=7, 7%)

·         Q & A (n=5, 5%)

·         everything (n=4, 4%)

·         none (n=3, 3%)

·         other (n=7, 7%)

An example of ‘other’ comments includes:

My library is watching this as a group. Building a community that values inclusion, diversity and equity is the single most helpful element of this session.

 

 

 

 

Q7. How could this class be improved (n=63)?

A total of 63 respondents (40%) provided comments after excluding 7 responses of no comments. About 24 percent of respondents(n=15) reported positive experiences with the class and stated that they have no suggestions. Nineteen (19) percent of the respondents (n=12) suggested more hands-on activities and examples of practical applications for participants, followed by more descriptions on the slides and more reference materials (n=7, 11%), more in-depth conversation about the topic (n=6, 10%), more controlled chat box use (n=5, 8%), longer session (n=4, 6%), and improvement in technology used (n=4, 6%).

About 16 percent (n=10) provided other comments and examples include:

 A bit slower on presenting the information, I couldn't keep up with taking notes.
Bring additional voices into the presentation for some mix and different styles and approaches.
Make it publicly available to be use by youth leaders and other groups 


Q8. Are you likely to recommend this class to a colleague (n=150)?

The respondents were asked to rate how likely they are to recommend this class to a colleague. About 78 percent of the respondents (n=117, 78%) stated that they would recommend it to a colleague and about 17 percent (n=26, 17%) stated that ‘maybe’ they would recommend it to a colleague. Seven (7) respondents (5%) reported that they would not recommend the class to a colleague

Q9. Please share any other comments you have about this class (n=35).

A total of 35 respondents (22%) provided comments after excluding 2 responses of ‘none’. About 25 percent of respondents reported positive experiences with the presenter/class (n=9), followed by appreciation for the presenter (n=7, 20%). Seventeen percent (n=6) of the respondents stated that they would recommend this series for others. One participant stated they would not recommend this series to others (n=1, 3%).  About 34% percent (n=12) provided other comments and examples include: 

Examples that are relevant to librarian and the work we do would be beneficial.  Since Jessica doesn't have a library background, maybe someone at NNLM, MLA, or AAHSL could help out 
Could this material have been shared ahead of time and the webinar been more of a discussion/scenario based session.  I guess I think a flipped classroom could have been more impactful, though Jess definitely knows her material and presents it well. And as a white woman presenting to a very heavily white and female profession, interesting but wish we could push further and confront.
Need more ways I can apply this in a real-world scenario.

Q10. Do you want to receive Medical Library Association Continuing Education credit for this class (n=150)?

About 87 percent (n=130) of the respondents wanted to receive Medical Library Association Continuing Education credit.

 

 



*For a complete list of comments from the second DEI class, please refer to the supplemental document, Appendix 1: Comments from the second DEI Webinar. *For a copy of the survey, please refer to Appendix 2: Second DEI Webinar Survey Questionnaire.



[i] Survey data were collected and managed using REDCap electronic data capture tools hosted at the Institute of Translational Health Sciences (ITHS) with grant support (UL1 RR025014 from NCRR/NIH). REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.

 

[ii] Sample size varies by question. The percentage of the responses for each question was based on the total number of respondents who answered each question.