Visual Peer Review

Development and history

Visual Peer Review was first described in a 2017 classroom study by Friedman and Rosen, ^[1] which examined how students evaluate peer-produced data visualizations using structured rubrics. Developed within the broader fields of data visualization, information visualization, and educational technology, the system emphasized clear labeling, visual integrity, and reduction of chartjunk. Students assigned rubric scores and provided written explanations, aligning the activity with established principles of peer review.

Follow-up research expanded both the methodological and analytic dimensions of the framework. Friedman and colleagues applied natural language processing (NLP) to peer-review text to analyze part-of-speech patterns, sentence complexity, and comment length. ^[2] These analyses offered insight into how students expressed critique and engaged with core design principles. Later studies incorporated advanced statistical modeling to evaluate system-level behavior, including peer review networks and reviewer typologies. ^[3]

Between 2021 and 2024, the framework underwent iterative refinement through a series of studies that explored interface design, behavioral nudges, reviewer engagement, and social network dynamics. ^[4] The system was influenced by earlier work in computer-supported peer review—particularly My Reviewers, a rubric-based writing assessment platform developed by Joe Moxley at the University of South Florida. ^[5] While Moxley's platform focused on text-based feedback, Visual Peer Review adapted its core structure to support critique of DataVis and visual analytics.

To guide structured analysis and feedback, Friedman and Rosen also drew on the “what, why, and how” framework introduced by Liu and Stasko (2010), which emphasizes understanding a visualization's purpose, task alignment, and encoding strategy. ^[6]

Framework and components

Visual Peer Review is designed to support critique, reflection, and learning in courses focusing on data visualization, visual analytics, and related fields in educational technology. The system consists of interconnected component. Core components include:

• Visual Artifacts: Students generate original visualizations using software such as R (e.g., ggplot2), Tableau, Python, or Adobe Illustrator. These artifacts may include statistical graphics, dashboards, or design-oriented infographics.
• Rubric-Based Assessment: Peer reviewers evaluate submitted visualizations using structured rubrics grounded in visualization theory and design heuristics. Rubric dimensions typically include:
  • Use of labeling and axis scales
  • Minimalization of chartjunk and clutter (following Tufte's principles) ^[7]
  • Optimization of the data–ink ratio
  • Preservation of visual integrity through accurate representation (lie factor)
• Written Peer Comments: In addition to scoring, reviewers provide narrative feedback explaining their reasoning. These comments aim to improve design literacy, strengthen visual reasoning, and support the learning process common to peer review across educational contexts. ^[8]
• Instructor Analytics Dashboard: Instructors access an analytics dashboard that displays peer-review activity across the course. Metrics include comment length, rubric coverage, participation patterns, and potential indicators of disengagement. These features position the framework within the domain of learning analytics, where visualized data helps instructors monitor student progress and identify support needs.

Ongoing development

Current work focuses on enhancing rubric structure, integrating principles from human–computer interaction, DataVis and expanding learning-analytics capabilities. Ongoing studies investigate how interface design, reviewer behavior, and classroom context influence the quality of feedback and overall engagement. Continuing development positions Visual Peer Review at the intersection of data visualization education, peer assessment, and educational technology.

References

1. Friedman, A., & Rosen, P. (2021). Leveraging peer review in visualization education: A proposal for a new model. https://doi.org/10.48550/arXiv.2101.07708
2. Beasley, Z. J., Friedman, A., & Rosen, P. (2021). Through the looking glass: Insights into visualization pedagogy through sentiment analysis of peer review text. IEEE Computer Graphics and Applications, 41(6), 59-70.
3. Friedman, A., Hawley, K., Rosen, P., & Rahman, M. D. (2024, May). Enhancing student feedback using predictive models in visual literacy courses. In 2024 IEEE Global Engineering Education Conference (EDUCON) (pp. 1-7). IEEE. https://arxiv.org/abs/2405.15026
4. Dinh,L., Friedman, A., & Hawley,K. (2024).Examining peer review network dynamics in higher education visual communication courses using ERGM, Computers and Education. https://doi.org/10.1016/j.caeo.2024.100222.
5. Beasley, Z., Friedman, A., Pieg, L., & Rosen, P. (2020, June). Leveraging peer feedback to improve visualization education. In 2020 IEEE Pacific Visualization Symposium (PacificVis) (pp. 146-155). IEEE https://arxiv.org/pdf/2001.07549
6. Liu, Z., & Stasko, J. (2010). Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE transactions on visualization and computer graphics, 16(6), 999-1008
7. Tufte, E. R. (1991). Envisioning information. Optometry and Vision Science, 68(4), 322-324
8. Moxley, J. M., & Eubanks, D. (2016). On Keeping Score: Instructors' vs. Students' Rubric Ratings of 46,689 Essays. WPA: Writing Program Administration-Journal of the Council of Writing Program Administrators, 39(2)

External links

• Official website