Mixed Reality Training System for Teaching Pointers

Problem Statement

Abstract programming concepts, such as pointers, present significant challenges for novice programmers. Pointers are fundamental to data structures like linked lists and binary trees and are essential for memory management, threading, and file handling. However, their complex syntax, the distinction between variables and memory addresses, and the abstract nature of memory operations make them difficult to understand.

The PointARs System

In this paper, we present PointARs, an augmented reality learning system that leverages 3D-printed tangible blocks and projected digital overlays to support the teaching of pointer concepts. The system enables learners to engage with variables, pointer declarations, dereferencing, and memory operations through embodied interaction.

PointARs is an interactive tabletop system that blends physical and digital elements to create an immersive learning environment for teaching pointers. The system combines projected digital augmentations with custom 3D-printed tangible objects, each representing specific programming constructs. A companion display dynamically updates to show the corresponding code representation in real time, establishing a direct mapping between physical interactions and programming syntax.

Design and Concept

The tangible components in PointARs were deliberately designed to map programming constructs into physical form, allowing learners to manipulate them directly while receiving projected AR feedback. Section 3.1 of the paper describes the design of these tangible objects:

Variables

Variables are square cubes labeled with variable names and values. Learners can use these blocks to represent declared variables, which can then be assigned or referenced by pointer blocks. When a variable is declared in the physical environment, the projected overlay highlights its memory address and current value.

These blocks provide a physical anchor for variables in memory. By being able to place and move them, students can clearly distinguish between a variable’s identifier, address, and stored value, which is a major hurdle in pointer learning.

Pointers

Pointers are distinctively shaped to denote references rather than values. When connected to a variable block, the system projects an arrow linking the pointer’s name to the memory address of the variable, visually representing the reference relationship.

This design makes explicit the difference between values and references. The projected arrow demonstrates what is otherwise invisible in memory — how a pointer refers not to data itself, but to an address.

Arrays

Array are elongated and feature rotatable dials along their surface. Each dial represents an index within the array, allowing learners to select and access specific elements. Projected overlays display the index position and value at that location, creating a tangible representation of indexed memory access.

The physical dials provide an intuitive way to navigate through array indices, reinforcing how arrays are laid out in contiguous memory. This design supports both iteration and random access, which students often struggle to visualize.

Evaluating PointARs

We conducted a between-subject study comparing three instructional modalities: traditional lecture notes, visuals-only slides, and the PointARs augmented tangible system. We measured their effects on learning recall, subjective cognitive load, and perceived user experience.

We recruited (n = 35) first-year computer science student across the three conditions: 12 participants in the Tangible and Slides conditions, and 11 in the Lecture condition.

Key Findings

Our results show significant differences in physical and mental demand, as well as in user experience ratings, with PointARs receiving the highest hedonic and pragmatic quality scores.

Tangible condition participants (those who used PointARs) reported the lowest mental demand during the task, followed by the Lecture condition and the Slides condition having the highest mental demand. PointARs provides an engaging and interactive experience with the help of the overlaid augmented projections on the tangibles.

Although no statistically significant improvements in learning recall were observed across conditions, our analysis revealed promising trends in favor of tangible interactions, particularly in user engagement and affective responses.

Design Recommendations

To properly assess the benefits of augmented tangible blocks, we recommend evaluating their effectiveness on more advanced pointer topics, such as dynamic memory allocation, passing by reference, and pointers as arrays. augmented tangible interactions may be more beneficial for advanced programming concepts.

We recommend that future designs focus on minimizing the physical demands of tangible interactions while preserving their interactive benefits. This could be achieved by reducing the size and complexity of tangible objects or integrating adaptive system feedback to help users manage physical strain while maintaining cognitive engagement.