Graphical User Interface (GUI) based Integrated Development Environments (IDEs) provide important information through visual cues.

Sighted developers can glance through code, quickly navigate by scroling, pointing and clicking and, get realtime information from IDEs through visual cues like syntax coloring, etc. This visual information is unavailable to screen reader users. through initial user studies, we identified 4 catagories of issues.

  1. Glanceability.
  2. Navigability.
  3. alertability.
  4. Discoverability.

CodeTalk improves code glanceability, navigability and alertability (realtime access to information) through UI enhancements, speech and non-speech cues. CodeTalk also introduces TalkPoints, an audio debugging experience. CodeTalk supports 3 types of TalkPoints:

One unique feature to TalkPoints is that the developer can choose to not break when these TalkPoints are hit. The developer can listen to the associated audio cue and continue execution.


Project Collaborators

Priyan Vaithilingam, Suresh Parthasarathy, Gopal Srinivasa and Dr. Manohar Swaminathan