MagScribe

The following is my first-authored paper currently under review at ACM CHI 2027.

MagScribe: Extending Mobile Device Interaction into the Adjacent Workspace with Edge-Mounted Magnetometer Arrays

Abstract: Mobile computers support rich interaction within their boundaries but remain largely blind to activity in the surrounding workspace. Extending interaction into this adjacent workspace is challenging: sensing must tolerate occlusion, work without instrumenting nearby surfaces or objects, and maintain precision with compact device-mounted hardware. We present MagScribe, a device-integrated magnetic sensing architecture using edge-mounted magnetometer strips to track passive magnetic markers in 5-DoF at 225 Hz without line of sight. A masking study guides a staggered-strip design for laptops, tablets, and phones. On a laptop, MagScribe creates a 1150× 1050 × 600 mm (0.7245 m^3) working volume with 4.91 mm mean 3D positional error within 400 mm of a sensor, while reconstructing handwriting on ordinary paper with 1.57 mm positional error and recognizing 97 of 98 words. Applications spanning physical-digital editing, paper interfaces, tangible controls, and free-space interaction demonstrate how MagScribe extends mobile-device interaction into the surrounding physical world.

Demo #1

Demo #2

Demo #3

Demo #4