CITE: The Collaborative Interactive Table for Education

Illustration of users engaging with CITE.  Users are manipulating tabletop objects and viewing the results on a screen.

The Collaborative Interactive Tabletop for Education (CITE) allows users to manipulate on-screen data visualizations through the manipulation of hand-held tabletop objects, as illustrated in the diagram above. The system includes sound and touch features that considers users with visual impairments.

Research has shown that interacting with tangible user interfaces (TUI), such as CITE, enhances collaborative behavior, strengthens learning, and improves user experience. While many TUI rely on specialized hardware, our approach is to create a system, a set of verified instructions along with software packages, for adapting standard computer hardware into a highly customizable tangible interactive tabletop.

This work builds on the datablocks project, shown in the video below, which was developed in 2016 by the principal investigator. CITE features new visualizations, new software that is robust and amenable to user edits, digitally modeled tokens, and design considerations for the visually impaired.

This website is intended to provide the resources you need (instructions and software) to build CITE yourself. Please feel free to contact the team members with any questions.

How it Works

The tokens are tracked by means of a camera placed beneath a transparent tabletop. The bottom of each object is marked with a fiducial marker specially designed to be identified by the (open-source) reacTIVision software, and the camera placed below the table captures the image of the fiducial markers in real time. The reacTIVision software outputs the ID of each identified marker and its X and Y coordinates, when they are in the field of view of the camera. This information is interpreted by the TUIO protocol and processed by our software, which constructs the visualizations from a database filtered by the user’s query.

The illustration below provides a schematic of the system.

Schematic illustration showing how CITE works.  Tokens, marked with fiducials underneath, are placed on a clear tabletop.  The fiducials are detected by a camera and interpreted with ReacTIVision along with our software to produce an onscreen visualization.

Our Prototype

Our prototype, for which we provide instructions and software here, uses data from Gapminder, and displays visualizations, similar to the one illustrated below, that are inspired by gapminder's bubble scatter plots. Users interact with the graph by manipulating tokens on the table.

Illustration of one of our data visualizations.

To add or remove countries from the scatter plot, users add or remove the corresponding 'country' token from the table. Our prototype allows users to view data from up to six countries: United States, Brazil, Greece, Australia, Singapore, and Kenya. Below is an illustration of one of our country tokens.

US country token

Users can select from the following list of indicators to plot on the x or y axis: Life Expectancy, Income, CO2 Emissions, Child Mortality, Pump Price for Gasoline, or Population Density. To plot Income on the x-axis, for example, users place the Income token within the slot of the x-value token, as illustrated below.

illustration of indicator and xbucket tokens interlocked

Our current prototype displays data for 2002, but our software can be modified easily to show data from another year or allow users to select the year with a token interaction.

We note that there is also a sound component to accompany the visual display to help guide users that may be visually impaired. The software announces each token as it is placed on (or removed from) the table, and narrates the information displayed on the graph.

Our Team

This work was done at SUNY Polytechnic Institute, and funded by an Innovative Instruction Technology Grant (IITG).

Team members:

Ana Jofre, Principal Investigator

Kristina Boylan, Co-Investigator

Nick LeJeune, Co-Investigator

Michael Reale, Co-Investigator

Ibrahim Yucel, Co-Investigator

Josh Rosenbaum, Student Researcher

Kyle Frenette, Student Researcher

Nicholas Byard, Student Researcher

John Harlan, Student Researcher

Instructions and downloads for building the table

Here we provide instructions and recommendations for replicating our prototype, and we encourage you to add your own modifications as needs or restrictions arise.

We divide the instructions here into 3 steps: Building the table, installing the software, and creating the tokens.

Build the Table

illustration of the table

To build this system, you will need a table with a transparent top and a camera mounted below. We recommend the design illustrated above, where the camera is centered on the lower cross-bar. The camera position and zoom should be adjusted so that its full field-of-view encompasses the whole table surface. Our table was constructed using the following parts.

The full parts list can be found on Amazon here.

A note on lighting

ReacTIVision software requires the right lighting conditions to recognize the token markers on the table. If you are 3-d printing the tokens, you can use an opaque translucent material and place an led light within the token. Otherwise, you will need to add some lighting from below. In doing so, you must be careful to avoid specular reflections from the glass as these will interfere with ReacTIVision's ability to see the tokens. Therefore, lights below the table should be pointed toward the floor, and we recommend that you check the camera to ensure there are no bright spots on the table surface.

Download the Software

Before downloading our software, we recommend that you download ReacTIVision and familiarize yourself with it.

Print out a few of the fiducial markers, place them on the table and make sure that ReacTIVision is able to recognize them. When recognized, a green cross along with the id number will appear on the marker, as illustrated below. We recommend doing this everytime you use the table, to ensure the lighting is right and that the markers are being detected.

image of fiducial marker being detected by ReacTIVision

Our software is available for download on github at this link. You will need Python 3.7 with Virtualenv and Anaconda or Miniconda for installation. The required Python libraries are Kivy, oscpy, pandas, matplotlib, playsound, gTTS (Google Text-to-Speech), and pyserial.

Users can modify the data displayed and the fiducial marker allocation without having to modify the code. The data is read into the software from csv files, and the fiducial marker allocation is input through json files. In our current prototype, as is, the fiducial marker IDs are allocated as follows:

Upon starting the software, if there are no markers on the table, the user will be prompted to start placing markers on the table.

software screenshot - no tokens on table

The Indicator markers work in conjunction with the X and Y Buckets. For example, to plot Income on the x-axis and Life Expectancy on the y-axis, the user must place the Income marker in close proximity to the X Bucket marker, and the Life Expectancy marker in close proximity to the Y Bucket marker.

software screenshot - axis selected only.

The user can now start placing Country tokens anywhere on the table to start viewing the data.

software screenshot showing country data graph

Our software includes sound to help the visually impaired. Each of the instructions are dictated, the graph is textually described, and the software announces which token has been placed or removed from the table in real-time.

Make the tokens

Here we provide the labels and 3d models for the tokens used the in our prototype. We provide two alternatives on building the tokens:

  1. Make your own token object, and attach our labels to the top and the corresponding fiducial to the bottom.

  2. Download our stl files and 3d-print the tokens. Our models include the label and corresponding fiducial embossed within the token.

Our labels

Our label designs each include braille and universal symbols, as illustrated below.

Country Token labels

Click here for a pdf of the Country tokens.

Indicator Token labels

Click here for a pdf of the Indicator tokens.

X and Y Bucket labels

Click here for a pdf of the X and Y buckets.

If you decide to make your own token objects, you should be mindful to design for the interaction between the indicators and the X and Y buckets. To select an indicator to visualize, the user must place it in close proximity to one of the X or Y buckets. If one indicator is selected for one of the axis, you should be mindful to prevent other indicators from being in proximity to selected axis bucket.

Our 3-d models

All tokens have the labels embossed on the top and the fiducials engraved on the bottom. The country tokens are simple cylinders and the indicator tokens are conical.

US country token top view US country token bottom view indicator token top view indicator token bottom view

We designed tokens for the indicators and the X and Y buckets so that they intuitively fit together, as illustrated below.

indicator and xbucket tokens interlocked top view indicator and xbucket tokens interlocked bottom view

The stl files for all our tokens can be downloaded from a single zip file here.