Feedback Tool

Feedback is one of the most important ways for teachers to help students improve their work. When provided via a screencast video, feedback becomes even more relevant, engaging, and actionable. This project will build a video feedback experience into Google Docs and Google Classroom designed to streamline the experience of recording, sharing, and watching a video for teachers and their students.

Hypothesis: Showing a “Record Feedback” button on Google Docs/Classroom will increase usage of record through the use case of providing student feedback.

🔬 Research Plan

🥅 Goal

The overall goal of creating a Feedback Tool is to increase the usage of record.

We believe there is a large opportunity to increase the number of videos users are creating by introducing (or encouraging) the use of Screencastify as a tool for feedback.

In order to validate our hypothesis, we will conduct two phases of research:

  1. POC Experiment: Show a “Record Feedback” button on Google Docs and measure its usage. (Validate: Does this placement and functionality increase usage of record)

  2. User Survey: Quantify the behavior and expectations of this button. (Validate: Are users using this button in the way we intend them to)

 
 

POC Experiment

Before building the full ideal flow (large scope), is there a lightweight POC we can build to validate if users have an appetite for initiating a recording from this placement?

 

⚖️  Metrics

Of users that see this button, how many use it to record a video? And how many times do they do this?

Need to measure:

  • How many users see the button?

    • Docs vs Classroom views

  • How many users click on the button once?

  • How many users click on the button more than once?

🧪 Experiment Design

  • What: A “Record Feedback” button will be shown on Docs. Clicking this button will start a screen recording with mic and webcam; clicking the button again will end the recording and open the VMP as with a normal recording

  • Who: Select group of districts will be given access

    • Do we want to focus on Classroom districts (next test)?

  • Two groups: Experiment group and control group:

    • Prior to experiment, we will pull baseline usage data for both groups

    • Build a dashboard/report to track this data week to week

Results and Key Findings

📈 POC Results

Key Takeaways:

  • Of the users who did try the button, 22% had repeat interactions

  • Anywhere from 2-25% of users that saw the button actually clicked on it (approximate guess based on incomplete GA/Pendo data)

Experiment Segment:

  • 27 Districts/Accounts enabled

  • 13k district teachers total

    • 4.5k average unique monthly users

    • 2.1k unique users last month

From Pendo Data, Feb 15 - March 1 2022 (2 week period)

  • Button Shown: 2283 visitors have seen the button

  • 2% Click Rate: 45 visitors

  • 22% Repeat Usage: (10 of the 45 clicked)

  • 21% Dismissed: 484 visitors

📝 Survey Results

Overall, regardless of whether the user clicked or not, there is an interest in providing video feedback. The main pain points/reasons why teachers don't currently record video feedback is that it is too time consuming.

  • Why users clicked on the button:

    • To record video feedback

    • Quicker way to start recordings

  • Why users didn’t click on the button:

    • Don't provide feedback via video (too time consuming, prefer written/live feedback) 67%

    • Didn't understand what the button did 25%

    • Other- Didn’t have enough time 8%

  • More than half of users say they already do/would like to record more feedback via video (54%)

    • 33% said they realistically do not have time

    • 13% said they don’t see themselves recording video feedback

Next Steps

  • Run the same test in Google Classroom and compare data

  • Broader roll-out without changes to UX to collect more clarifying data

    • Potential downsides - users not seeing value in first use, preventing from future use

    • Should this be a limited feature

  • Release to a wider audience (start with paid users or domains)

    • Create Pendo guide to pop-up for user feedback

  • Main areas of opportunity:

    • Survey of Edtech influencers - video feedback as a growing use case

    • Explain the intended use case

    • Expedite the whole process (pre and post recording) to save teachers time

    • Provide a way for users to turn it on/off