• Application due (submit here): August 5, 2019, extended to August 12, 2019,
  • Acceptance notification: August 23, 2019
  • Hardware kits shipped to accepted teams: August 24 – August 26, 2019
  • Development period starts: August 26, 2019
  • Development period ends: October 1, 2019
  • Teams demo at Poster/Demo Reception: October 2, 2019 (during the Poster/Demo reception in the main conference)


For the first time ever, ACM MobileHCI 2019 is organizing a student competition (SC) during the main conference. The challenge will be to augment human capabilities and advance the interactive experience. Is your work pushing the boundaries of human sensing and cognitive capabilities? Are you excited by the prospect of presenting your ideas to the world?

The MobileHCI SC is an outstanding opportunity to showcase your creativity to leaders in the field, turn your ideas into reality, and win fabulous prizes. Projects featured at the poster/demo reception in the main conference of MobileHCI 2019 in Taipei, and contest winners will be announced at the conference. A jury will vote on the Most Creative and Best Implementation and conference attendees will vote for their favorite teams in the People’s Choice. 

Hardware Platform and Software Tool


JiNS MEME is eyewear with frame-mounted sensors to obtain electro-oculography (EOG) data and acceleration/angular rate by simply wearing the eyewear. A blink and eye movement detection can be done with these sensors on the eyewear. The data collected by this system makes it possible for users to characterize their daily activities (e.g., reading, typing, and talking) and gain insight into their mental state throughout the day, which can then be used as information for various applications. JiNS MEME allows users to shift their attention from objects in the external world to the mental operations that characterize their inner mental realm.

Each selected team will receive one or two pairs of JINS MEME ES_R Smart Glasses and a Hauoli MobilTrakk motion tracking system to form the basis of a cognition augmentation system. These groundbreaking systems are outlined in the following:

JINS MEME ES_R Smart Glasses

JiNS MEME eyewear is equipped with multiple sensors:

1. Three-point electro-oculography (EOG) sensor

The EOG sensor in the nose pads monitors patterns in blinking and eye movements in order to characterize the user’s physical condition in terms of concentration capacity, fatigue, and composure.

2. Six-axis (accelerometer and gyroscope) sensor

The six-axis sensors are able to detect even the slightest changes in the orientation of the body to assist in characterizing the user’s physical and mental state.

Each team will also receive a complete development pack (for FREE!). They will also receive one to two pairs of JiNS MEME ES_R providing Android and iOS SDK connectivity to allow the uploading of raw sensor data from the JiNS MEME eyewear to a mobile APP via Bluetooth in real-time.

For details, please refer to the official website for SDK and documentation.

[1] JiNS MEME ES_R Spec: https://jins-meme.com/en/researchers/specifications/
[2] JiNS MEME ES_R SDK: https://jins-meme.github.io

The following Youtube videos provide demonstrations of the JiNS MEME glasses.

[video1] JiNS MEME eyewear looks into you to stop fatigue: https://www.youtube.com/watch?v=kOeANYANz-Q
[video2] JiNS MEME experiment in the driver: https://www.youtube.com/watch?v=KeX6Wi4AlHQ

Hauoli MobilTrakk: Device-Free Motion Tracking SDK

In additional to JiNS MEME eyewear, we provide another sensing technique as an option which you can add on top of the JiNS MEME eyewear. The Hauoli MobilTrakk motion tracking system uses the speaker and microphone built into smartphones to achieve (I) mm-scale accuracy, (II) smartphone connectivity without the need for additional hardware or infrastructure, and (III) functionality in areas with obstructions and variable lighting conditions. The Hauoli motion tracking system is nearly unlimited in terms of applicability. A simple wave of the hand could be used to change TV channels, control the air conditioner, pilot a drone, or participate in video games with fine-grained motion control, such as first-person shooters. It can even be used in VR/AR systems or even controlling drones.

Each selected team in the competition will receive the Hauoli MobilTrakk Device-Free SDK for iOS or Android. The SDK makes it possible to develop mobile APPs that use the distance from the mobile device to one’s hand (or any moving object around the phone). Further details can be found on the official website.

[1] Hauoli MobilTrakk Device-Free SDK: https://www.hauoli.co/mobiltrakk-df.html

MobileHCI 2019 Student Competition: Human Augmentation


Recent achievements in sensor technology, device miniaturization, wireless networking, AI/machine learning, and data analysis methods have advanced the design and proliferation of connected devices. We are just getting the first glimpses of the Internet of Things, wearable devices, AR/VR devices, and highly immersive virtual environments. Unfortunately, many people lack the cognitive and processing capabilities required to keep up with many of these advancements. We envision the development of human augmentation schemes aimed at amplifying human cognition and processing power. Such devices will require advancements in three main areas: (1) Sensing hardware and algorithms: Multimodal sensing is expected to raise sensing capabilities to the next level; (2) Feedback: Researchers will develop feedback schemes that provide intuitive access to human participants; (3) AI: Sophisticated software solutions will make it possible to leverage multimodal sensing data via intelligent information processing.

CHALLENGE: Interactions Without Saying a Word

Your goal will be to formulate a method that enables meaningful interactions with other entities (people or machines) without explicit language verbalization (e.g., using speech recognition) or language transcription (i.e., using a touchscreen or keyboard/mouse). The first problem will be to figure out other ways to express your intent. It will then be necessary to identify tasks that would benefit from an interaction method that does not involve speech or writing. Your system should motivate the use-case that clearly articulates the reasons for discarding conventional linguistic interactions (e.g., speech or writing). Your system should also clearly illustrate how substantive interactions could otherwise be achieved. Keep in mind that multi-user interactions are crowd favorites at MobileHCI, so try not to limit yourselves to single-user situations.


There is really no limit to the configuration of your system. The only constraint is the need to engage in meaningful interactions without saying a word; i.e., a communication channel that does not rely on speech or writing.

– Sensing Space:

Target: Gaze direction? Facial expressions? Gestures? Body language?

Sensors: The proposed system must be based on JiNS MEME eyewear; however, we expect that other sensors will also be required, such as the Hauoli MobilTrakk motion tracker, IR, depth/camera, EMG, frequency analysis, or EEG. The sensors could be implemented in any area of the users’ body, such as in eyewear.

– Output Space:

Machine Interactions: Could gaze direction be used to play games? How difficult would it be to use gestures to control appliances?

Context-Aware tasks: Providing assistance when and where it is needed? Issuing warnings of driver drowsiness?

Multi-user collaboration: Would it be possible to feed users’ data pertaining to the intent, emotional states, or focus of those with whom they are interacting?

Feedback solutions: Outputting inference results (based on sensing data) via visual representations, haptics, and/or tactile feedback.

– Interaction Space:

Form-factor and wearability: Could 3D-printing be used to alter the form of glasses?

Attachments on glasses: Could cameras, 3D Scanners, micro-projectors, or other devices be installed in eyewear?

Simply mixing and matching these three primitives could prompt a multitude of exciting ideas. Feel free to add custom hardware, sensors, and components. The hardware provided for the competition is meant to be used as a minimal starting kit from which to explore new possibilities.


Each selected team will receive a pair of JINS MEME ES_R Smart Glasses (JINS MEME Academic Pack: https://jins-meme.github.io). The development kits of JINS MEME ES_R (Academic Pack) for Windows, Android, and Mac can be found at the following address: https://github.com/jins-meme/ES_R-Development-Kit. Moreover, each team will receive the Hauoli MobilTrakk Device-Free SDK for iOS or Android.

When you provide your initial proposal, please specify the number of JiNS MEME eyewear sets you will need and list some of the sensors you plan to use. This will make it easier for users to evaluate your idea. Online video tutorials and documentation will also be provided to ensure that you have everything you need to realize your vision.


All participating teams must follow the instructions outlined on this form.

The submission deadline is August 12th, 2019.

Your submission should include the following:

  • A description of your idea (roughly 150-words)
  • Supporting document (SIGCHI extended abstract format, 4-page max.)
  • Link to supporting materials (images, video, etc.)
  • Completion of all items on Registration Page (contact info, etc.)

Decisions will be emailed to each team by August 16th.

At least one member from each of the teams accepted into the competition will have to register for the MobileHCI conference and pay the registration fee. If you have already been accepted as a student volunteer, please inform us of this.

Once your MobileHCI conference registration has been confirmed, your hardware will be shipped to you at no expense.

Contest Rules, Details, and Suggestions

Pre-Selection Criteria: Organizers will select a minimum of 15 teams based on usefulness, creativity, complexity, feasibility, and time/resource constraints.

Team Size: The minimum team size is 2, and the maximum team size is 5.

Team Composition: Each team member is limited to membership on only one team.

Eligibility: All participants must currently be enrolled as students (high school, college, university, Master’s program, Ph.D. students). International students are welcome.

Registration Requirement: At least one member from each of the teams accepted into the competition will have to register for the MobileHCI conference and pay the registration fee. If your team is accepted, at least one of your team members MUST register for the conference (or be accepted as a student volunteer). This is for three reasons: 1) you have to demo your idea, 2) you have to claim your prizes, and 3) MobileHCI is an awesome conference!

Demo Ideas: Teams will be permitted to demo only one idea; however, teams will be allowed to demo one idea during the contest voting period. They will be allowed to present multiple demos of that idea.

Demo Code: We encourage you to open-source your code. Conference organizers will create a GitHub organization to which all teams are expected to post their code. Note that teams will not be judged according to the quality of the code they submit.

Demo Safety: Make sure your demos follow reasonable safety guidelines. Setting fire to the jury or inciting an international incident will be frowned upon.

Extra Equipment: Teams will have to bring the computers and hardware required to run their demos, including the hardware shipped by the contest organizers. No additional equipment will be available at MobileHCI.

Power Supply: A powerstrip (Type-B plug) and a 110-volt power supply will be provided at each demo table. Nonetheless, we recommend that you bring your own regional power strip with an appropriate adapter for connecting your hardware.

WiFi: WiFi connectivity is available at the conference; however, please be aware of bandwidth restrictions. Teams are welcome to use WiFi as a part of their demos; however, we recommend having a Plan B, such as a router.

Bluetooth: Please test your Bluetooth system carefully beforehand. Note that some bluetooth firmware is easily stressed when 300 attendees are around your demo with BT enabled devices. Make sure you have a plan B.


Innovation comes in many forms. At MobileHCI SC, awards will be presented in the following categories:

  • Most Creative (Jury): USD $900
  • Best Implementation (Jury): USD $900
  • People’s Choice (Attendee Votes): USD $200

All teams are free to bring home their hardware kits. Winners will receive cash prizes and a MobileHCI SC certificate. Note that we reserve the right to change the awards structure during the contest.


Please contact Student Competition Co-Chairs (mobilehci.student.competition@gmail.com)