pen

Dissertation now online

| Michelle Annett

Now that all of the research comprising my Dissertation has been published (see here, here, here, here, and here), I am happy to make it publicly available. If you would like to read my dissertation (its a tad long at 144 pages), a .pdf copy of my thesis can be found here. I am also super happy that I was lucky enough to receive the University of Alberta Faculty of Science Doctoral Dissertation Award for my research.

Yes, there was lots of pontificatation during the writing of my dissertation.

Abstract:
Although pens and paper are pervasive in the analog world, their digital counterparts, styli and tablets, have yet to achieve the same adoption or frequency of use. Digital styli should provide a natural, intuitive method to take notes, annotate, and sketch, but have yet to reach their full potential. There has been surprisingly little research focused on understanding why inking experiences differ so vastly between analog and digital media and amongst various styli themselves. To enrich our knowledge on the stylus experience, this thesis contributes a foundational understanding of the factors implicated in the varied experiences found within the stylus ecosystem today. 

The thesis first reports on an exploratory study utilizing traditional pen and paper and tablets and styli that observed quantitative and behavioural data, in addition to preferential opinions, to understand current inking experiences. The exploration uncovered the significant impact latency, unintended touch, and stylus accuracy have on the user experience, whilst also determining the increasing importance of stylus and device aesthetics and stroke beautification. The observed behavioural adaptations and quantitative measurements dictated the direction of the research presented herein. 

A systematic approach was then taken to gather a deeper understanding of device latency and stylus accuracy. A series of experiments garnered insight into latency and accuracy, examining the underlying elements that result in the lackluster experiences found today. The results underscored the importance of visual feedback, user expectations, and perceptual limitations on user performance and satisfaction. The proposed Latency Perception Model has provided a cohesive understanding of touch- and pen-based latency perception, and a solid foundation upon which future explorations of latency can occur. 

The thesis also presents an in-depth exploration of unintended touch. The data collection and analysis underscored the importance of stylus information and the use of additional data sources for solving unintended touch. The behavioral observations reemphasized the importance of designing devices and interfaces that support natural, fluid interaction and suggested hardware and software advancements necessary in the future. The commentary on the interaction – rejection dichotomy should be of great value to developers of unintended touch solutions along with designers of next-generation interaction techniques and styli.

The thesis then concludes with a commentary on the areas of the stylus ecosystem that would benefit from increased attention and focus in the years to come and future technological advancements that could present interesting challenges in the future.

GI 2015 Publication! The last of my dissertation to be published …

| Michelle Annett

I am very happy to announce that the last work from my Dissertation is set to be published at Graphics Interface this year. The publication, “Hands, Hover, and Nibs: Understanding Stylus Accuracy on Tablets” reports on a two-stage user study that evaluated the role hand posture, the information available from the hover cursor, and nib diameters have on the user experience while inking. a user study that was conducted at Microsoft Research during my extended internship. Walter assisted me with the analysis and discussion sections of the work. Once the full paper is available, I will provide a link to it, but for now the abstract is below.

Abstract:

Although tablets and styli have become pervasive, styli have not seen widespread adoption for precise input tasks such as annotation, note-taking, algebra, and so on. While many have identified that stylus accuracy is a problem, there is still much unknown about how the user and the stylus itself influences accuracy. The present work identifies a multitude of factors relating to the user, the stylus, and tablet hardware that impact the inaccuracy experienced today. Further, we report on a two-part user study that evaluated the interplay between the motor and visual systems (i.e., hand posture and visual feedback) and an increasingly important feature of the stylus, the nib diameter. The results determined that the presence of visual feedback and the dimensions of the stylus nib are crucial to the accuracy attained and pressure exerted with the stylus. The ability to rest one’s hand on the screen, while providing comfort and support, was found to have surprisingly little influence on accuracy.

 

Unintended Touch in ToCHI – New Journal Publication from MSR Internship

| Michelle Annett

Two days before my PhD defense, I found out that a manuscript that I submitted to ToCHI on unintended touch (aka palm rejection) was accepted for publication! The manuscript is entitled “Exploring and Understanding Unintended Touch during Direct Pen Interaction” (previously titled, “Is it Intended or Unintended? Palm Rejection during Direct Pen Interaction”)  details a data collection experiment and an algorithmic analysis of various possible solutions to unintended touch on tablets. The work was part of the larger collection of pen-based work that I performed while I was an intern at Microsoft Research (yippee! for publication #4 from my MSR time).

An example of unintended touch information from the perspective of a digitizer. The current stylus location is denoted in blue and the unintentional touch events from the palm and little finger are denoted in varying shades of orange.

The manuscript will not be published until December 2014, so here is the abstract:

The user experience on tablets that support both touch and styli is less than ideal, due in large part to the problem of unintended touch or palm rejection. Devices are often unable to distinguish between intended touch, i.e., interaction on the screen intended for action, and unintended touch, i.e., incidental interaction from the palm, forearm, or fingers. This often results in stray ink strokes and accidental navigation, frustrating users. We present a data collection experiment where participants performed inking tasks, and where natural tablet and stylus behaviors were observed and analyzed from both digitizer and behavioral perspectives. An analysis and comparison of novel and existing unintended touch algorithms revealed that the use of stylus information can greatly reduce unintended touch. Our analysis also revealed many natural stylus behaviors that influence unintended touch, underscoring the importance of application and ecosystem demands, and providing many avenues for future research and technological advancement.

GI 2014 Publication! How Low Should We Go? Microsoft Research Internship paper #2

| Michelle Annett

Today is a big day in the world of Michelle because I have lots of news to share! In addition to the CHI 2014 publication on the High Performance Stylus System, I am happy to report that I also have a follow-up paper that will be presented a Graphics Interface 2014 in Montreal entitled, “How Low Should We Go? Understanding the Perception of Latency While Inking”! (Yaaah again for no cell phone roaming fees to attend the conference!) As the paper is forthcoming, I can’t release all of the details but I can share the abstract and a short video that we shot. Once the full paper is available, I will provide a link to it.

Abstract:
Recent advances in hardware have enabled researchers to study the perception of latency. Thus far, latency research has utilized simple touch and stylus-based tasks that do not represent inking activities found in the real world. In this work, we report on two studies that utilized writing and sketching tasks to understand the limits of human perception. Our studies revealed that latency perception while inking is worse (~50 milliseconds) than perception while performing non-inking tasks reported previously (~2-7 milliseconds). We also determined that latency perception is not based on the distance from the stylus’ nib to the ink, but rather on the presence of a visual referent such as the hand or stylus. The prior and current work has informed the Latency Perception Model, a framework upon which latency knowledge and the underlying mechanisms of perception can be understood and further explored.

Although every accepted paper is cause for celebration, I am especially happy that this paper was accepted because the proposed Latency Perception Model is a nice summary of all the latency work that has been done by myself and others thus far. Here is the YouTube video summarizing our work:

 

CHI 2014 publication! In the Blink of an Eye, Microsoft Research internship paper #1

| Michelle Annett

Woo hoo! It’s finally time that I can talk about some of the awesome work that I did at Microsoft Research while I was on my extended internship! One of my forthcoming papers, entitled “In the Blink of an Eye: Investigating Latency Perception during Stylus Interaction” is going to appear in the proceedings of CHI 2014. The conference will take place at the end of April in Toronto (yippeee no cell phone roaming fees!) but here is sneak preview of the abstract and the video we shot for the project. The follow-up to this work will be published at Graphics Interface 2014, and can be seen here.

The HPSS was built by Albert Ng and Paul Dietz, who collaborated previously on the High Performance Touch System that you can see here on YouTube. The psychophysical just-noticeable difference studies were incepted, conducted, and analyzed by me! Anoop Gupta and my supervisor Walter acted as mentors on the project.

Abstract:
While pen computing has become increasingly more popular, device responsiveness, or latency, still plagues such interaction. Although there have been advances in digitizer technology over the last few years, commercial end-to-end latencies are unfortunately similar to those found with touchscreens, i.e., 65 – 120 milliseconds. We report on a prototype stylus-enabled device, the High Performance Stylus System (HPSS), designed to display latencies as low as one millisecond while users ink or perform dragging tasks. To understand the role of latency while inking with a stylus, psychophysical just-noticeable difference experiments were conducted using the HPSS. While participants performed dragging and scribbling tasks, very low levels of latency could be discriminated, i.e., ~1 versus 2 milliseconds while dragging and ~7 versus 40 milliseconds while scribbling. The HPSS and our experimentation have provided further motivation for the implementation of latency saving measures in pen-based hardware and software systems.