NNLM Summit Feedback Survey Report

Prepared by NNLM Evaluation Office (NEO)

Submitted to NNLM on 4/11/2019


All Hands Summit for the National Network of Libraries of Medicine (NNLM) staff was hosted at University of Maryland, Baltimore from March 4 through March 5, 2019. A total of 95 people attended the summit in person.  The Summit agenda was developed by the Summit Planning Committee in response to the interest in and need of an in-person meeting for the NNLM staff.  The three objectives of the Summit were:

The two-day program included: 7 general sessions, 7 break-out sessions, and 2 committee meetings, and 4 working group meetings. An online participation option was available for some sessions. Following the conclusion of the Summit, a link to the Summit Feedback Survey was sent out by NEO to the Summit attendees via NNLM-All listserv on March 5. Questions included attendees’ perceptions about meeting the objectives, experiences with the program content, and strengths and weaknesses of how the Summit was executed and might be improved in the future.  The online survey was open for 10 days and closed on March 15. Of the 95 in-person Summit attendees, 58 (61%) completed the survey. For the online Summit attendees, 3 people completed the survey. This report provides an analysis of the 61 survey responses from the in-person and online attendees.  

Key Findings 

 Ten (10) Areas of Summit Evaluation (n=61):

 ·         Overall rating: Seventy-two (72%) percent (n=44) of the attendees who completed the survey stated that overall experience at the Summit was ‘very good’ or ‘excellent.’ Three online attendees rated the overall experience as ‘poor’ or ‘fair’ as all of them commented that they were not able to hear the session or participate in the activity.

 ·         Obtaining a shared understanding of key messages: Attendees received a shared understanding of 3 topics: cultural humility, human subject research, and NLM priorities. Cultural humility (64%) was the most highly rated topic with most ‘well enough’ and ‘very well’ responses in comparison to human subject research (54%) or NLM priorities (56%). However, the cultural humility topic also received the most respondent comments that the training was too safe and that the conversation should be ongoing and needs to cover difficult topics such as “white privilege or fragility.”  

 ·         Strengthening relationships: The highest percentage of respondents (90%) felt that the objective of strengthening relationships was met ‘well enough’ or ‘very well.’ The feedback was further confirmed by the response from another question: eighty-five percent (85%) of the respondents agreed that they were able to build meaningful connections by meeting in person.

 ·         Improving network operations: Sixty-two percent (62%) of the respondents reported that the Summit met the objective of improving network operations ‘well enough’ or ‘very well.’  Ten percent of the respondents stated ‘unsure’ about this objective as some commented that this objective is a long-term goal and needs to be followed by action after the Summit.

 ·         Communication: Eighty percent (80%) of the respondents ‘strongly agreed’ or ‘agreed’ that their voice was heard at the Summit.



·         Time allotted for sessions: When asked if there was enough time to cover the topic at each session, the responses were more varied with the highest percentage of ‘neutral’ (26%) or ‘disagree’ (15%).      

·         Applying learning from the Summit: Eighty-two percent (82%) ‘strongly agreed’ or ‘agreed’ that the information presented at the Summit was useful to their work or service. When asked what they plan to do as a result of the Summit, the responses included applying the learning at their work or organization, reaching out to partners within and outside their region, following up on action items from the Summit, engaging with workgroups and committees, and aligning NLM priorities with their work.

·         Favorite part about the Summit: When asked to identify one thing they liked about the Summit, 50 (82%) respondents provided comments. Thirty-two (32) people (52%) identified meeting in person and five people selected the communication session by Tyler Smith as their favorite part about the Summit.

·         Areas for Improvement: Forty-six (46) respondents (75%) provided comments on what could be improved about the Summit.  The comments were about allowing more time for planning, considering different timing and location for future meetings, longer meeting time, better technology, and specific suggestions about the breakout sessions or meeting organization.

 ·         General feedback: Many attendees (n=26, 43%) complimented the Summit Planning Committee and the host for the event and provided positive comments about the overall in-person Summit. There were suggestions about more in-person meetings in the future or improving on meeting organization, many of which coincided with the comments from the Areas for Improvement. Module

What did Attendees Think of the Summit?

Q1. How would you rate your overall experience at the Summit (n=60)?

Seventy-two percent (72%, n=44) of the attendees who completed the survey stated that overall experience at the Summit was ‘very good’ or ‘excellent.’

Q2. Did the Summit meet the objectives (n=61)?

More than half of the respondents thought the Summit met the objective 1 ‘well enough’ or ‘very well’ for all three topics. About seven to eight respondents stated that the Summit did not meet the objective 1 at all or met it ‘very little.’

Q3. Do you think the Summit met the objectives (n=31)?

Thirty-one respondents provided comments and 12 respondents commented on more than one objective. The topic with the highest number of comments was the cultural humility (n=11) and the objective with the most positive comments was strengthening relationships.

The cultural humility session was too basic. It needs to be demographic specific when we are addressing specific audiences or topics…. 
More than anything I think the summit strengthened relationships among NNLM staff.  There was a feeling of camaraderie in that we're all in this together….

Q4. Please indicate your level of agreement with the following statements (n=61):

Eighty percent or more respondents ‘strongly agreed’ or ‘agreed’ that they were able to build meaningful connections, that their voice was heard at the Summit, and that the information presented at the Summit was useful to their work or service. When asked if there was enough time to cover the topic at each session, the responses were more varied with the highest percentage of ‘neutral’ (26%) or ‘disagree’ (15%).     

Q5. Name one thing that you liked about the Summit (n=50)?

A total of 50 respondents provided comments and 6 respondents identified more than one thing they liked about the Summit. Thirty-two respondents commented that meeting in person or connecting with colleagues was their favorite part about the Summit. Three people selected breakout sessions and 7 people stated that guest speakers (five communication workshop by Tyler Smith, two cultural humility workshop by Dr. Isabel Rambob, and one workshops in general) as their favorite part of the Summit. Examples of other comments include:

 I saw how passionate everyone is about the Network and the Library.
….. I also appreciate the opportunity to make our thoughts and opinions heard.

Q6. What is one thing you plan to do as a result of the Summit (n=45)?

Comments from 45 respondents were categorized by theme as illustrated in the chart below. Examples of comments from applying learning to their work or organization include integrating cultural humility in every aspect of their job or further examining human subject research IRB requirement for their work.

Q7. Do you think you accomplished something at the Summit face-to-face that could not have been accomplished through NNLM virtual meetings? If so, what was it (n=52)?

Fifty-one respondents stated positively and one respondent simply stated ‘no’ to this question.   

The real time back and forth, being able use nonverbal communication to help understand, to have 1:1 conversation or small groups to help problem solve or listen to issues, to place our work in the context of other regions/offices. It also seemed like folks were less restrained to say what they really thought.

Yes, meeting people and discussions worked better in person.  I thought the communication and brainstorming (what works well, challenges and ideas) programs worked particularly well. 

Q8. What is one thing that could be improved about the Summit to plan future NNLM face-to-face meetings (n=46)?

A total of 46 respondents provided comments and six of them provided suggestion in more than one area. 

Q9. Is there anything else that you would like to share about the Summit (n=42)?

Congratulation to the planning team! This meeting was important and we know that they had little planning time.
 I would suggest doing this annually; it's a great way to connect with colleagues cross regions who don't get any other opportunities...

Study data were collected and managed using REDCap electronic data capture tools hosted at the Institute of Translational Health Sciences and LiveStories hosted at the NNLM National Evaluation Office (NEO). REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.

LiveStories at the NNLM National Evaluation Office (NEO) is supported by the National Library of Medicine of the National Institutes of Health under Award Number UG4LM012344.