Educational Platform for Learning Sketching

SketchTivity is an NSF-funded research project being developed at the Sketch Recognition Lab for teaching basic drawing skills. I lead the design and development of the system and it is the topic of my dissertation. We focus on a practical form of drawing known as design sketching or conceptual sketching, with a special emphasis on teaching engineers the skill.

The sofware has been implemented in 3 high schools and at Georgia Tech's Mechanical Engineering and Industrial Design programs, and we have ongoing studies to evaluate its effectiveness at improving basic sketching ability.

We utilize sketch recognition to give real-time feedback as people learn fundamentals of sketching. We explore different ways of doing this by extracting various features from sketches. We also utlize proven concepts in education theory like mastery learning, and self-regulated learning to motivate students to practice and develop their skills.

COLLABORATORS
  • Matthew Runyon
  • Dr. Tracy Hammond
  • Dr. Julie Linsey
  • Wayne Li
MY ROLE
  • UX Research
  • UX Design
  • UI Design
  • Visual Design
  • HTML / CSS / JS
  • Sketch Recognition
OBJECTIVES
  • Empathize with individuals learning sketching and understand their struggles via semi-structured interviews and focus groups
  • Develop a framework for motivating individuals with sketch-based gameplay
  • Design and prototype an educational system for guiding students through basic sketching fundamentals
  • Develop an algorithm for recognizing perspective accuracy in any rectilinear perspective sketch
  • Conduct user studies to determine how effective software is in improving general ability, motivation, and self-efficacy
KEY RESULTS
  • Proved through user studies with hundreds of students that students can make statistically significant improvements in accuracy, line quality, and speed of basic primitives
  • Proved through user study with 40 participants that automated real-time feedback can help individuals improve perspective drawing accuracy.
  • Developed a motivation framework for motivating students with sketch-based gameplay by interviewing 20 participants
  • Won 1st place (Jury award) CHI Play 2017 Game Design Competition
  • Won 2nd place Student Research Competition
  • Software has been used by over 1000 students at 3 universities and 3 high schools
  • Published and presented work at top HCI conferences around the world including CHI, IUI, SBIM, CHI Play, and Creativity & Cognition

BACKGROUND

Sketching is a skill that can be difficult to learn for many people. The act of drawing is complex and relies on many different physical and cognitive functions working together including fine motor control, spatial reasoning, hand-eye coordination, visual memory, perception, and so on. Mastery of the skill requires a great deal of effort and many hours of practice. Many people have low self-efficacy with regards to sketching, and this hinders their ability to improve and master the skill.

One of the biggest problems with traditional pedagogy - Studio environments is the comparison to peers and lack of feeling any sense of achievement for many people. It often leads to feelings of learned helplessness. It's also difficult for teachers to cater their teaching to individual students. They simply don't have the time our resources to always individualize the learning experience.

MOTIVATION

My research has led to me realize the major difference between beginners who struggle at sketching and their more advanced peers is self-efficacy. I believe there is an important theoretical “threshold” which shifts a beginners mindset and leads to further mastery of sketching. Some people reach this threshold quickly and easily, others struggle. Either way, at this point the learner is:
- Beginning to unlearn any negative thought patterns that kept them from improving drawing skills i.e. “I’m not a good at drawing” or “I’m just not creative / artistic” Beginning to be encouraged by seeing peer’s work, rather than discouraged. (Peers should still be around the same level of mastery)
- Are more likely to take control of their learning experience (self-regulated learning)
- May gain a more intrinsic desire to improve sketching ability without relying on external praise or external motivators.

UNDERSTANDING THE USERS

Since learning sketching is very much about practice, it was important early on to interview individuals with various skill levels (from novice to expert) to understand how motivations to practice sketching shift over time. As it turns out, the motivations largely follow Self Determination Theory - Individuals tend to be more extrinsically motivated at first (seeking praise, positive feedback, or other external rewards), then as their skill improves they need to be more intrinsically motivated to master the skill so that they can utilize it for collaboration, creativity, etc.

This allowed for the creation of 3 personas based on skill level.

ITERATION 1 - LOW-FIDELITY PROTOTYPE AND USABILITY TESTING

The first prototypes were low-fidelity and paper-based. This allowed me to rapidly explore exercise designs with virtually the same interaction, just real pen and paper, not a stylus and a touchscreen device. For the assessment, I used a Wizard of Oz technique to simulate what the software's feedback might be like.

I went on to build interactive prototypes with HTML / CSS / JS. Not long after this our NSF grant was accepted and SRL began working on the project (I joined SRL 2 years later to lead development)

ITERATION 2 - MID-FIDELITY PROTOTYPE AND USABILITY TESTING

I went on to build interactive prototypes with HTML / CSS / JS. Not long after this our NSF grant was accepted and SRL began working on the project to add sketch recognition algorithms (I joined SRL 2 years later to lead development). This version was deployed at Georgia Tech and we were able to prove that the using the tool over a semester can lead to statistically significant improvements in:

Accuracy of basic primitives

Line quality of basic primitives

Speed of basic primitives

ITERATION 3 - HIGH-FIDELITY PROTOTYPE AND ONGOING DEPLOYMENT

After joining SRL I designed a far better summative feedback system, a user profile for visualizing performance over time (to empower both students and instructors), and a novel sketch-based game for line work called ZenSketch. The game won a global game design competition with 30 entries at CHI Play 2017 and has been shown to motivate practice of line work.

The software has ongoing deployment in 3 universities and has also been deployed in 3 high schools.

PERSPECTIVE ALGORITHM AND INTELLIGENT INTERFACE

After practicing basic primitives in perspective, it becomes important to practice one's sketching skills on actual objects. A common exercise is the "city street" in perspective. I developed an algorithm and designed / prototyped an intelligent interface that can provide real-time feedback and assessment on any rectilinear perspective sketch. I also tested it in an experimental split study with 40 participants. Through the study I was able to prove that students receiving the intelligent feedback can make statistically significant improvements in the accuracy of their perspective drawings.