Mixed Reality Training System for Teaching Pointers

Problem Statement

Abstract programming concepts, such as pointers, present significant challenges for novice programmers. Pointers are fundamental to data structures like linked lists and binary trees and are essential for memory management, threading, and file handling. However, their complex syntax, the distinction between variables and memory addresses, and the abstract nature of memory operations make them difficult to understand.

The PointARs System

In this paper, we present PointARs, an augmented reality learning system that leverages 3D-printed tangible blocks and projected digital overlays to support the teaching of pointer concepts. The system enables learners to engage with variables, pointer declarations, dereferencing, and memory operations through embodied interaction.

PointARs is an interactive tabletop system that blends physical and digital elements to create an immersive learning environment for teaching pointers. The system combines projected digital augmentations with custom 3D-printed tangible objects, each representing specific programming constructs. A companion display dynamically updates to show the corresponding code representation in real time, establishing a direct mapping between physical interactions and programming syntax.

Design and Concept

The tangible components in PointARs were deliberately designed to map programming constructs into physical form, allowing learners to manipulate them directly while receiving projected AR feedback. Section 3.1 of the paper describes the design of these tangible objects:

Variables

Variables are square cubes labeled with variable names and values. Learners can use these blocks to represent declared variables, which can then be assigned or referenced by pointer blocks. When a variable is declared in the physical environment, the projected overlay highlights its memory address and current value.

These blocks provide a physical anchor for variables in memory. By being able to place and move them, students can clearly distinguish between a variable’s identifier, address, and stored value, which is a major hurdle in pointer learning.

Pointers

Pointers are distinctively shaped to denote references rather than values. When connected to a variable block, the system projects an arrow linking the pointer’s name to the memory address of the variable, visually representing the reference relationship.

This design makes explicit the difference between values and references. The projected arrow demonstrates what is otherwise invisible in memory — how a pointer refers not to data itself, but to an address.

Arrays

Array are elongated and feature rotatable dials along their surface. Each dial represents an index within the array, allowing learners to select and access specific elements. Projected overlays display the index position and value at that location, creating a tangible representation of indexed memory access.

The physical dials provide an intuitive way to navigate through array indices, reinforcing how arrays are laid out in contiguous memory. This design supports both iteration and random access, which students often struggle to visualize.