CSU Stanislaus - Quality Assurance ePortfolio (2021-2022)

https://docs.google.com/document/d/1FxgnrRDGZP5fpNrZumrKiRvUxpH0k40eFkcN31HdTEQ/edit

Quality Learning and Teaching (QLT)

Faculty Learning Community Portfolio

Academic Year 2021/22

Dr. Daniel Soodjinda, Dr. Cassandra Drake, and Dr. Shradha Tibrewal

California State University | Stanislaus

Proposal Summary: At the conclusion of the 2021/22 AR-QLT Faculty Learning Community project, we were able to (1) increase understanding of quality online teaching and learning that is anchored in anti-racist pedagogy, especially as connected to the elements of the AR-QLT rubric; (2) assist faculty in developing or revising wholly online courses to meet AR-QLT expectations; (3) improve faculty ability to self-assess and peer-evaluate online courses; (4) Continue to develop a peer mentorship program where previous AR-QLT participants supported new members of the 2021/22 Faculty Learning Community.

Campus Need for Quality Assurance 

Campus Need for Quality Assurance 

  • Continued understanding of the components of a quality online (or hybrid) educational experience
  • Introduction to the Anti-Racist (AR) Quality Learning and Teaching (QLT) evaluation tool - and the self- and peer-evaluation process
  • Continued understanding of faculty needs with regard to online (and hybrid) teaching
  • Stronger infrastructure/support for faculty teaching online (and hybrid) courses and students taking such courses

Proposal Goals Based on Need

  • Increase understanding of quality online teaching and learning that are anchored in anti-racist pedagogy, especially as connected to the elements of the AR-QLT rubric
  • Assist faculty in developing or revising wholly online courses to meet AR-QLT expectations;
  • Improve faculty ability to self-assess and peer-evaluate online courses
  • Train a cohort of AR-QLT Peer Mentors; and develop an ongoing program to promote peer-review for online courses
  • Develop an ongoing program to promote peer-review for online courses
  • Help department chairs (and others) better support and determine the quality of online teaching by their faculty.
  • Review AR-QLT participant courses by certified CSU peer reviewers so that the courses can be added to the CSU Quality Assurance Resource Repository
  • Evaluate the student outcomes of online courses developed through previous QLT-FLCs in alignment with our local graduation initiatives and institutional student success plan.

Quality Assurance Leads

  • Daniel Soodjinda, Professor of Liberal Studies
  • Cassandra Drake, Assistant Professor of Liberal Studies 

Supporting Campus Partners

  • Shradha Tibrewal, Director, Faculty Center for Excellence in Teaching and Learning
  • Glenn Pillsbury, Instructional Designer, OIT/Learning Services

Background on Quality Assurance Efforts 

  • Prior to the introduction of QLT at CSU Stanislaus
    • Regular workshops on pedagogy, course planning, and technology tools delivered by the Faculty Center for Excellence in Teaching and Learning and OIT staff each semester.
    • Link to the Stan State Faculty Center for Excellence in Teaching and Learning "Teaching Remotely" workshops. 
  • QLT Introduction at CSU Stanislaus
    • Spring 2020/2021 QLT workshops: Faculty were invited to attend a three-day series of workshops that introduced each section of QLT along with significant resources and tools related to them.  The participants each self-assessed one of their courses that was in the process of being designed or converted to an online format. 
  • Continued QLT efforts at CSU Stanislaus
    • 2014 QLT workshop:  In May, 2014 a workshop was held providing an overview of the QLT evaluation tool, which discussed the similarities and differences between online teaching and learning and face-to-face teaching and learning, and considered the types of supports faculty may need for course development and facilitation. 
    • In Summer 2014 we hosted the "Supporting Quality Online Learning and Teaching Faculty Learning Community." 
      • Ten faculty members chose to participate in the FLC, representing the disciplines of anthropology, art, business administration, communication studies, criminal justice, kinesiology, psychology, public administration, and theatre. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2014, some were also engaging in transformation of summer 2014 courses.
      • Nine of the ten FLC members were able to complete course redesign, and engage in self and peer evaluation. Upon completion of course design/redesign, FLC members identified between 46 and 54 objectives as "relevant" for consideration in the evaluation of their course (since most courses had not yet been taught, objectives related to the facilitation of the course were typically not able to be assessed). Self- and Peer-evaluations indicated 87-100% of relevant objectives were evaluated as "meeting" or "exceeding" expectations on the QLT evaluation tool.
    • 2015 QLT workshops:  In August, 2015 a set of workshops were held for incoming participants of the 2015/2016 QLT Faculty Learning Community (FLC).  The workshops provided an overview of the QLT evaluation tool, which discussed the similarities and differences between online teaching and learning and face-to-face teaching and learning, and considered the types of supports faculty may need for course development and facilitation. 
    • During the 2015-2016 academic year we established a QLT-FLC that consisted of 7 faculty members representing the disciplines of Education, Child Development, English, Social Work, Nursing, and Biology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2015, some were also engaging in transformation of summer 2016 courses. 
      • In an effort to help support the 2015/2016 QLT-FLC during 2015-2016 academic year, we also developed a QLT peer mentor program that consisted of 7 faculty members who were assigned to QLT-FLC participants in a effort to support them in developing an online course.
    • During the 2016-2017 academic year we established a QLT-FLC that consisted of 7 faculty members representing the disciplines of psychology, nursing, art history, kinesiology, political science, and English. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Fall 2017, some were also engaging in transformation of summer 2017 courses. 
      • In addition to supporting the facilitating the 2016/17 QLT-FLC, we also further developed the QLT-FLC mentor program that consisted of 3 faculty members who were assigned to QLT-FLC participants in an effort to support them in developing an online course. 
    • During the 2017-2018 academic year we established a QLT-FLC that consisted of 6 faculty members representing the disciplines of psychology, nursing, art history, kinesiology, political science, and English. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2018 or Fall 2018, some were also engaging in transformation of summer 2018 courses. 
      • In addition to supporting the facilitating the 2017/18 QLT-FLC, we also further developed the QLT-FLC mentor program that consisted of 2 faculty members who were assigned to QLT-FLC participants in an effort to support them in developing an online course. 
    • During the 2018-19 academic year we established a QLT-FLC that consisted of 6 faculty members representing the disciplines of advanced studies in educational technology, English, kinesiology, psychology, and sociology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2019 or Fall 2018, some were also engaging in transformation of summer 2019 courses. 
    • During the 2019-20 academic year we established a QLT-FLC that consisted of 8 faculty members representing the disciplines of Teacher Education, Sociology, Communication Studies, Accounting/Finance, Computer Science, and Psychology. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2020 or Fall 2020, some were also engaging in transformation of summer 2020 courses.
    • During the 2020/2021 academic year we established a QLT-FLC that consisted of 6 faculty members. The FLC membership included assistant, associate and full professors, and faculty with extensive to no prior experience teaching wholly online courses. While most faculty were preparing courses to be offered in Spring 2022 or Fall 2021, some were also engaging in transformation of summer 2021 courses.
  • Transition to the Anti-Racist (AR) Quality Learning and Teaching (QLT) evaluation instrument
    • During the 2020-2021 academic year we decided to intersect the central concepts of the original QLT instrument with anti-racist, inclusive, and equitable pedagogy.  This resulted in the AR-QLT evaluation tool, which supported a new faculty learning community who were interested in developing quality online courses that support their students who belong to historically excluded marginalized communities. 

Quality Assurance Project Results

Fall 2021/Spring 20212 Faculty Learning Community

This year we had a 100% completion rate.  We believe that the high completion rate is attributed to more formalized accountability "check-ins" that occurred every 3-4 weeks via synchronous online meetings, and additional accountability checks completed by the AR-QLT-FLC mentors. 

Sessions were held throughout the academic year, and incorporated face-to-face and online components. Participants were introduced to the AR-QLT instrument, obtained knowledge and access to resources in support of curricular redesign and received training to self- and peer-evaluate courses. Faculty development funds were awarded to 13 faculty participants who developed a course meeting expectations in at least 75% of the relevant objectives indicated in the AR-QLT rubric. These faculty members also completed and received a peer-evaluation. Additional resources that were examined during this particular AR-QLT-FLC are detailed below:

  • Faculty had access to a number of resources and guest speakers that focused on anti-racist, de-colonial, and culturally sustaining teaching practices. 
  • Resources were provided to faculty about Universal Design at our face-to-face meetings and through a self-directed Blackboard Course.  For access to the self-directed course (via CSU Stanislaus Blackboard), please contact Bob Koehler at bkoehler@csustan.edu.
  • An online workshop that highlighted a self-developed QLT course development framework and recommendation matrix.  The matrix was developed to support faculty as they develop their online courses by breaking up the lengthy and cumbersome QLT rubric into different areas that faculty can focus on.  The areas explored include: 
    • Pre-planning:  This is the first thing that faculty should first think about when developing an online course. Faculty don't necessarily have to be detailed here, but should at least have some type of framework for the items listed in this area before they start building the course on their chosen platform.
    • Syllabus development:  Once faculty complete step 1 (pre-planning), a syllabus can then be developed that explains the instructor's policies, course's expectations, assignments , grading, etc. to the items students will find in the online course. This can, and should, be revised after the course has been developed to make sure that things are clearly aligned.
    • Course build (primary): At this stage, faculty can then begin to build their online course by considering the more important elements outlined.  The elements are somewhat in order of importance, but faculty should view all elements before  they begin to build their course so that the course's flow and alignment between objectives/materials/assessments makes sense. 
    • Course build (secondary):  After a framework for the course is built in point 3 above, faculty can then go back and add in these additional elements to supplement what was already developed. 
    • Course delivery/maintenance:  Faculty should be aware of the elements listed in this column, but don't have to address it until the course has begun and students are actively participating. 
  • A presentation about Google Docs suite of applications was provided to faculty participants. Resources relating to the presentation can be found by utilizing the two links below. A discussion about Google Hangouts took place, and a resource was provided relating to Google Hangouts for lecture capture: http://www.eduhacker.net/technology/google-hangouts-lecture-capture.html
  • A presentation was provided to faculty participants about best practices in supporting students with disabilities using the principles of Universal Design. A link is provided here to the presentation's resources:  https://www.csustan.edu/accessible-technology-initiative. A discussion relating to the transformation of Powerpoint or Keynote presentations to video resources took place, and the discussion's resources can be found here:  http://bit.ly/1KTQiHe
  • A presentation was provided to faculty participants about best practices involving Voice Thread in an online course. 

Participating Faculty/Team Feedback 

AR-QLT-FLC Participants reported that the accountability "check-ins" helped keep them on track. During these online meetings, faculty shared out the work they completed in their online courses at different points in the program and learned about an additional topic of interest that they were surveyed on.  

Successes

  • All faculty who participated in the 2021-2022 AR-QLT-FLC met expectations, and they were able to successfully create or revise an online course that meets or exceeds expectations on over 75% of relevant QLT objectives. 
  • Many faculty found the QLT course development framework helpful in organizing the QLT standards in a meaningful way.  
  • As of 6/26/22, CSU Stanislaus has trained and supported 180 faculty members who are now formally trained in using the QLT instrument through the QLT faculty learning community program.  And 19 faculty have been formally trained in using the AR-QLT evaluation instrument. 

Ideas/Lessons Learned

  • Participation in the AR-QLT faculty learning community programs continues to be of interest, so discussions of continued stipends and/or course releases for participation in the program is occurring. 

Next Steps for Quality Assurance for 2021/22

Stan State has a long history of providing online and hybrid teaching workshops and FLCs, including a decade or so of FLCs specifically tied to engagements with the QLT Rubric. In addition, we have 107 faculty who have completed the “Introduction to Teaching Online Using QLT” course, 21 who have completed the Advanced QLT Course in Teaching Online, and 13 who have completed Reviewing Courses Using the QLT Rubric. Participants in the FLCs and QLT courses have had some opportunity to engage in informal or self-review of QLT courses, but to date we have only one course that has gone through formal review (two more in process this summer), and very limited Stan State participation as QLT course reviewers. We do not have an established informal online/hybrid review process at Stan State outside of the FLCs. With the release and implementation of the 3rd edition of the QLT Rubric, even those faculty who have completed QLT trainings need to update their understanding of the QLT objectives.

Having spent much of 2020 and 2021 teaching in online and hybrid modalities, many faculty and students have increased interest in online teaching and learning even after campuses are fully reopened for traditional in-person instruction. Because we are seeing higher numbers of online and hybrid course offerings than in the pre-pandemic period, we wish to better assist faculty as they offer intentionally-designed online and hybrid courses, rather than courses temporarily moved online amid crisis. We are, therefore, designing a year-long Faculty Learning Community designed to enhance our capacity to provide assistance to faculty who are developing and facilitating online courses. Specifically, we wish to get 9 new faculty trained and certified to complete informal and formal QLT reviews, and to better prepare 3 additional faculty members who are certified as QLT reviewers, to engage in reviews using version 3 of the QLT rubric.