Crisis in Space

Game-based assessment system of collaborative problem solving skills -pinpoints strengths and weaknesses and provides feedback for improvements.

MY ROLE:
Product Management, UX design, usability testing

CHALLENGES

21st century skills such as collaboration, problem-solving, and critical thinking are crucial skills today’s students need to succeed in their careers during the Information Age. However, educational institutions have been focusing on assessing traditional academic skills and, there was no publicly available software to assess them at the time of development. Receiving a grant from Hewlett Foundation, my team at GlassLab/LRNG tackled the challenge of developing the industry's first collaborative problem-solving assessment and feedback system. A combination of an engaging puzzle game, a chat system, and audio volume tracking are used to record the users' input and assess their collaboration skills.

GAME FLOW

Crisis in Space is a two-player system. After starting a game, the Player selects a role and is matched with another player who has selected a different role. The game challenges are only visible to one player at a time and cannot be solved without the information obtained from the partner. Level difficulty progression and when and what to include in the tutorial were crucial to creating an accurate assessment.

COMMUNICATION ANALYSYS

Initially we created a chat system to track and analyse communication between players. Natural language analysis would have been the best solution, however, due to lack of expertise and limited budget, text tree based analysis system was developed using a following process:

Had testers play the game without using the chat system and recorded their verbal communications
Transcribed the recordings
Broke down sentences into segments
Categoraized and added meta data to the segments and keywords
Iconized frequently used words
Developed a smart compose system that offers suggestions as you type.

The challenge was most users came up with their own unique style of naming conventions and it was impossible to catalog all keywords and users often became frustrated when their desired words were not found. As a solution, we decided to allow free typing and The final system consisted of smart compose and free typeing similar to the system Gmail uses.

RESULTS

We created an engaging game-based assessment system that measures users' collaborative problem-solving skills. The assessment data can be used to gauge the current skill level but also to improve communication and collaboration skills. The modified version is now part of LRNG's workforce readiness program.

The success of the application also caught the attention of educational institutions such as UC Berkely and ACT. With my team's support, UC Berkely integrated our app into their social-emotional learning program to close the gender gap in Tanzania. And it is also used in ACT's collaboration study to help students learn or inspire them for a potential career.