Associate Professor Erle Lim, Department of Medicine, Yong Loo Lin School of Medicine, NUS

MediSIM – Medical Simulated Interactive Manikin

Hololens MediSIM is a holographic avatar superimposed on abdominal manikin to simulate various disease states, and to teach the abdominal examination.

Palpation skills are paramount to effective osteopathic practice, but teaching students how to palpate and learning palpation is a real pedagogical challenge as it requires the right combination of knowledge, skills, and attitude.

By visually simulating a 3-D, CG interactive patient and with the HoloLens superimposed on the Abe, including the simulated patient’s facial and auditory reactions to palpation, students will see a moving, interactive person – lending greater verisimilitude to the experience. The student may also need to make decision on what organs need to be palpated by the corresponding medical records.

Video link here

Using Oculus Rift for Healthcare Students to Appreciate Eye Symptoms

The Oculus Rift, created by Oculus VR, is a virtual reality 3-D headset that uses 360-degree head tracking to make it feel like the wearer is inside a different world. The Oculus Rift was initially designed for gaming purposes, but potential uses for education have also been explored. The Oculus Rift has traditionally employed virtual reality – in which the user is in an artificial environment. In this project, I collaborated with CMU’s Entertainment Technology Center (ETC) to develop games- and task-based scenarios in which the user, ostensibly a healthcare student, gets to “look through the eyes” of patients with various neurologic and eye diseases with characteristic eye signs. These conditions include:

1. Migraines – with scotomata (obscuration of visual field), chromatopsia (coloured lights)
2. Colour blindness
3. Optic neuritis – with blurred vision and loss of colour vision
4. Constriction of visual fields – called tunnel vision
5. Hemianopia – with diminution of half the visual field
6. Oscillopsia – apparent movement of stationary objects
7. Diplopia – double vision

In addition to using virtual reality to simulate these optic phenomena, the students at ETC created a device with 2 cameras mounted on a perspex “external spectacles” to fit over the Oculus Rift, allowed the wearer to view the external milieu – and project the above phenomena over them, i.e. augmented reality.

Mr Giam Kok Leng, Computer Centre, NUS

Google Cardboard + 360-Degree Media

Google Cardboard is a virtual reality (VR) platform developed by Google for use with a head mount for a smartphone. 360-degree media are videos or photos that allow you to view in any direction from a single standing position.

In this event, we are going to demo a simple setup using Google Cardboard to create an immersive experience with 360-degree media.

Google Cardboard: How it works! – https://www.youtube.com/watch?v=SxAj2lyX4oU

iBeacon + NUS Virtual Cards

An iBeacon is a device transmitting bluetooth signals that can be picked up by smartphone apps. Strategically-placed, iBeacons can be used to deliver location-based content in smartphone apps.

The NUS Virtual Cards is a mobile app that creates virtual identity of a user, displaying a virtual matric or staff card following a valid authentication. This virtual identity can potentially tap on functions of a mobile device such as bluetooth, QR Code scanning, etc, to verify the identity of a user. For this event, we are going to demo how the NUS Virtual Cards can be used with iBeacons for attendance taking.

Mr Narayanan Shyam, Centre for Instructional Technology, NUS

HTC Vive

Tilt Brush lets you paint in 3D space with virtual reality. Your room is your canvas. Your palette is your imagination. This endless possibilities can be now realised via CIT’s HTC Vive set.

GoPro HERO5

HERO5 Black can capture stunningly smooth video handheld, mounted to your favourite gear and more.. CIT has acquired chest harness, tripod and a head gear to create educational content.

360fly 4K Camera

360fly 4K captures in both 360° and first-person POV modes. Shoot, edit and share videos in 4k resolution. It’s dustproof, shockproof and water resistant. The 360fly 4K camera is now part of CIT’s suite of video equipment available for loan/rent.

Watch demo video

Dr Emelyn Tan, Department of Chemistry, Faculty of Science and
Mr Giam Kok Leng, Computer Centre, NUS

3D Augmented Reality App

“ARMolVis” is an Augmented Reality Molecular Visualiser for everyday products. By hovering smart devices with this app over 2D photos of selected everyday products, the predominant molecule in those products will appear on the device in 3D. The 2D photos can be manipulated in the physical world to view the 3D molecules models from different angles. In addition, the chemical’s name and formula are also displayed.

Watch demo video

Associate Professor Kelvin Foong, Faculty of Dentistry

Bridging clinical skills training and oral/facial anatomy with computer-aided 3-D visualization to enhance operative skills competency

Dental students have difficulty in applying anatomical knowledge learnt to the clinical context in the performance of operative procedures, a common example being the administration of local anaesthesia (LA). To overcome this difficulty, the project team developed interactive platforms through:

• an iBook (Apple Inc.)
• a stereoscopic virtual LA delivery model using the iBench (EON Reality)
• a realistic 3D printed physical model of the facial/oral anatomy

The presentation will give an overview of the development and how these platforms help with learning. It will also highlight the main challenges faced in the development, e.g., the steep learning curve in human anatomy of the technical team, and the approach taken to mitigate these issues.

Ms Amanda Chia, ST Electronics (Training & Simulation Systems) Pte Ltd

Flight Crafter

A virtual reality edutainment game to stimulate interest in aviation, aerospace and aerodynamics.

Flight Crafter is an interactive edutainment solution that helps to foster an understanding of the basic principles of flight. In this two-part experience game, players are offered the chance to design their own aircraft by modifying various components. Then they put on a virtual reality headset and experience their own design through an immersive visualisation of their flight.

Overall through the game, players can experiment with different variables and experience their outcomes, allowing them to learn through discovery. More importantly, by having fun while learning, Flight Crafter serves to ignite greater interest in STEM (Science Technology Engineering Mathematics).

Key Features:

• Create your own aircraft design by selecting up to 600 configurations
• Modify an airliner or F-16 with twin tails, two engines or even elliptical wings
• Take part in a virtual test pilot using VR goggles to see the effect your design changes have against the standard model

Visitor Flow

Dr Kevin Yap, Department of Pharmacy, Faculty of Science, NUS and
Mr John Yap, Computer Centre, NUS

RETROZFECT – 3D Gammification, Virtual Reality, Gesture Computing

Set in a futuristic post-apocalyptic world, the online game requires students to take on the role of a pharmacist avatar and interact with virtual patients while embarking on a mystery mission, such as saving the world from a plague that transforms humans to zombies. Similar to other online games, the game offers varying levels of difficulty. It also allows students to dictate their own fate.

By engaging in role-play, the students gain a better understanding of their patients’ medical conditions, after which they will undertake tasks applicable in real-life pharmacy practice. These tasks include processing prescriptions, prescribing over-the-counter treatments, developing a pharmaceutical care plan for the patient, as well as counselling patients. The problem-solving approach employed in the game challenges the students to develop critical thinking skills in tackling healthcare issues.

Link to video interview by LianHeZaoBao

Dr Rainer Dumke, Director and Dr Claus Dieter Ohl, Director, Edu2VR

EDU2VR

EDU2VR is Virtual Reality for the classroom. The VR platform consists of a cell phone app (binoo) and a web-based teacher console. The teacher designs and controls the VR learning environments from a browser. The students use their cell phones. Head tracking and immersion is based on Google Cardboard technology.

Watch demo video

Mr Tan Kay Chuan, Director, School of Electronics & Info-Comm Technology, ITE

CPR+AED Virtual Reality (VR) Simulation

The CPR+AED VR simulation provides an immersive 3D VR environment in which personnel like new trainee-paramedics, nursing students, volunteers or the community at large can have an alternative means to learn the steps of CPR and AED live saving procedures. The CPR+AED VR simulation can also serve as a refresher training.

Learning Apps for Special-Education-Need Students

iPad Learning Apps for students in APSN (Association for Persons with Special Needs) Schools. Using the rich audio and video tools, the Apps integrate Student-Centered Learning with Teacher-Centered Learning uniquely required for teachers in APSN Schools. The Apps demonstrate scalability of continuously adding or editing lessons, e-submission / e-assessment, and raising curiosity and inspiring learning. The contents include Smart Listening, Smart Visual and Pre-Vocational Cognitive Skills by Activity Levels (Symmetrical & Asymmetrical Body Movement, Praxis, Motor Response, Matching Objects-Numbers-Shapes-Colours, Auditory Memory, Visual Memory and Puzzle.

Smart Living Solutions

An in-house developed electronics kit that provides application learning of the various IOT devices (sensors for temperature, humidity and multi-channel actuators) in a smart living room setting. The system is powered by a 32-bit embedded controller with built-in WiFi and tri-colour LED control modules. It can be configured as an intelligent IOT node or as an IOT Gateway. The system can be demonstrated via a mobile application which is scalable through an Arduino (Integrated Development Environment) software coding.

Assistant Professor Lu Weiquan and Associate Professor Yen Ching-Chiuan,
Keio-NUS CUTE Center, NUS Interactive & Digital Media Institute

Virtual Interactive Human Anatomy (VIHA)

The Virtual Interactive Human Anatomy project or VIHA, focuses on the learning of human anatomy. Students interact with a human body, see animations and are able to remove parts and view them at multiple angles. Learning anatomy also requires the students to understand and visualise the spatial arrangements of different body parts, which VIHA allows for. The interactive tutorial also contains a step by step guide that allows students to understand medical concepts. The steps in the guide require students to interact with the human body, thus aiding the process of learning. Textbook concepts that only show the perspective of the author now comes to life in VIHA.

Virtual Interactive Simulation Environment (VISE)

The Virtual Interactive Simulation Environment project or VISE, focuses on emergency room procedures. These procedures are hard to replicate but VISE makes this possible. During these procedures, decisions have to be made in order to stabilise and diagnose the patient. Students engage a patient and have to perform the required tasks in the virtual environment. The scoring system gives real-time feedback on correct/incorrect actions. At the end of the given task, students are graded and this logged into the system. All tools, equipment and persons are designed and detailed, made to simulate the actual environment. This, along with other features included in VISE make it a valuable tool for doctors and nurses to-be.

Project AILE

The Augmented Interaction Learning Environment or AILE, combines the key features of three ongoing projects developed by the Augmented Learning Group at the Keio-NUS CUTE Centre. The first project is the specimen tracking system, first developed for the Lee Kong Chian Natural Museum, The second project is LeARnus, which is a web-based AR authoring platform for non-programmers. The third project is Whoa, which is a real-time AR demonstration and prototyping platform for Google Cardboard.

Mr Loh Mun Yew, ReVR Pte Ltd and Mr T K Ng, Side Effects Asia Pte Ltd

Tilt Brush

Tilt Brush is a new virtual reality (VR) app that lets you paint from an entirely new perspective on the HTC Vive.

With Tilt Brush, you can paint in three-dimensional space. Just select your colors and brushes and get going with a wave of your hand. Your room is a blank slate. You can step around, in and through your drawings as you go. And, because it’s in virtual reality, you can even choose to use otherwise-impossible materials like fire, stars or snowflakes. You can use different brushes, effects, and tools to paint your virtual creations.

You will have unique Immersive experience by putting on the headset, picking up the controllers and walking around the creations of others or creating your own masterpieces in full 3D.