Web Formative Evaluation Report After reviewing the existing archive of past workshop evaluations, we determined that we needed to revise the evaluation instruments to identify how well the existing website supported the first two user groups (potential workshop participants and accepted workshop participants). The Final Evaluation form for this year’s and future workshops include questions that specifically address how the participants used the website prior to the workshop and if they plan to use website following the given workshop.
We posted an overall follow up survey on the website in March to gather feedback about the impact of the past workshops on the participants’ teaching. To date we have received 95 respondents. 82 of the respondents indicated that they shared the workshop website with their colleagues and/or department. These respondents provided narratives that included the specific resources on the web they recommended or described department meetings devoted to perusing the website’s activities. 3 of the respondents used the website since the workshop and 18 additional respondents who had not yet used the website indicated plans to use it in the near future. For the five workshops held this year (Teaching with Visualizations, Preparing for an Academic Career, Course Design, Early Career, NAGT Geology and Human Health) additional questions were added to the final evaluations to better assess the website. From the final evaluations for these five workshops, nearly all attendees (93% to 100%) reported it easy to find information on the website to prepare for a workshop.
However, these final evaluations also highlight a potential problem in that the website does not seem to compel or make the attendees more aware of the other Cutting Edge resources. For Visualizations, Early Career, and Geo and Human Health, nearly all attendees (96%-100%) indicated that they planned to visit the Cutting Edge website again in the future. However, when the questions were changed and augmented for the Preparing for an Academic Career and Course Design, only half of the participants indicated hat they were aware of the Cutting Edge website. Since these same attendees found it easy to find information on the website to prepare for the workshop, it is unclear if all of the attendees were aware of the workshop being part of a bigger series. When separating out the leaders from the participants for the Academic Career workshop, only half of the leaders indicated that they were aware of and had visited the Cutting Edge website.
The lack of connection between the workshop website and previous workshop websites is even more dramatic. For the Academic Career workshop, only roughly 12% were aware of the other workshop websites and even fewer had used these websites. For the Course Design workshop closer to one third were aware of the other workshop websites. It should be noted that many of the questions pertaining to the awareness and use of the specific workshop websites on these final evaluations had a low percentage of respondents.
So, we only know that 12% indicated an awareness for a given workshop website but we do not know if the other 78% were not aware or just chose to skip the question. We may want to pose fewer questions in future evaluations to achieve a higher response rate. From the respondents to the overall follow up survey, we identified several participants of workshops and one respondent who didn’t attend a workshop but actively uses the website. We developed an interview protocol and are in the process of conducting interviews to focus on these specific cases.
We also plan to revise this protocol to use for a focus group at the upcoming AGU meeting. Hopefully, through these interviews and the focus group we can understand more about how users draw on the website and make connections between the resources. All of the case studies for the project will be used to better understand the long term impact of the website. From these studies we hope to develop a strategy for better disseminating the existence of the Cutting Edge website resources.