Interactive Stimulative Anatomy

Interactive Simulative Anatomy (ISA) is a mobile application that allow medical students to study the Human anatomy by providing an augmented view of the organ under study. In general students? study and learn anatomy by diagrams provided to them in books. These diagrams don?t provide the proper inf

Project Title

Interactive Stimulative Anatomy

Project Area of Specialization

Augmented and Virtual Reality

Project Summary

Interactive Simulative Anatomy (ISA) is a mobile application that allow medical students to study the Human anatomy by providing an augmented view of the organ under study. In general students’ study and learn anatomy by diagrams provided to them in books. These diagrams don’t provide the proper information about the organ as they are a 2-dimentional (2-D) view of the organ. Students face problem in understanding the exact and true structure of the organ because of the 2-D view of the organ provided to them  through diagrams of a 3-D model.

Interactive Simulative Anatomy not only provide the real, augmented view of the organ under study but also let the students or the user to see the structure of the organ by providing augmented simulation of the organ and other gesture like pinch-zoom, labeled 3-D view and fifth degree of freedom (FOF). It allows the users to point on the part of body and augment the internal structure of the pointed part organ. Currently ISA will focus on Thorax, neck face and head anatomy.  ISA  application in  education will contribute to the formation of an effective and productive learning environment.

Currently students who learn Human anatomy uses textbox, pictures and statue model. Medical student can also learn from Cadaver surgery. However, learning through Cadaver is proved to be a good method for learning human anatomy however it is so much complex as compare to a 3D model. Learning materials are mostly available in form of book and anatomy mannequin (puppet), but it is still insufficient enough to help students in understanding human body anatomy

through interactive simulative anatomy, we are providing a general schema for learning anatomy and that schema can be further extend and can be modified to enhance the functionalities and organ's detail, by feeding more Data Set to the system and making it learn using machine learning techniques.

Project Objectives

The current learning methods used  for teaching the intricated anatomical regions are considered in-efficient as they lack the depiction of a 3D spatial tissue in a proper manner. Most of the explanation to illustrate anatomy are Diagrammatic, 2D representations of pre-determined angles of depiction. This usually requires several images to provide full description of 3D objects in a 2D way.

Due to these issues learning anatomy education has become a very delicate, complex and difficult area . It is well known that different people learn in different modes. Some people might learn better in, for example, a kinesthetic way. However, this mode is usually restricted because of the current limits of the conventional learning environment including online learning with multi-media resources even with interaction. Due to these restrictions, learners must adapt their way of learning to fit the circumstances of provision.

By implementation of ISA , various conditions with shapes and colors and simulation can be displayed according to users selected options. The main purpose of this project is to make the anatomy learning easy, efficient and more learnable by providing real time augmented view of rather than a 2-D diagrammatic view of the organ, which don’t cover all the aspects and feature of the organ. ISA purpose is to make the learning of anatomy more real rather than conventional learning. As 3-D views have higher cognitive understanding and are more easy to understand and learn.

To provide a generic schema for learning anatomy and which can be further modified, and additional resources can be added.

To make education and learning easy and efficient.

To provide the internal details of how organs works through simulation and pinch zoom-in zoom-out mechanism.

Removing the limitation to understand the anatomy of human organs only by pointing to a certain diagram or to a certain object rather making the system intelligent enough so that it can augment the part of the body to which the use is pointing using marker base and location base AR techniques and generating the pose matrix.

 Increasing knowledge

 Memorizing and reproducing

 Applying

Understanding

engagement

Involvement

processing

Project Implementation Method

The Agile approach with component base incremental development has been selected as the project life cycle. Agile Approach using incremental development will be used as to enhance the quality of proposed project and as agile approach provide:

testing mechanism for each iteration.

Risk management will be taken care using agile approach.

 AR applications have important parts. The implementation of this project is carried out by going through these phases:

 The first part is camera (camera vision). The camera is a use measure the size and position of marker and to capture the marker.

 The second is capture and tracking (identifying the marker). The outputs from the camera are in the form of the position and size of images which will help to determine the initial size of the 3D model to be displayed.

The third stage is identification (identifying) in which the applications will identify the marker, determine whether the marker can be used in accordance with the stored database, and determine which model will be displayed.

The final stage is data appearance (visual display). The application displays the existing data in the form of 3D models and the coordinates of the part models. Appearance models use the marker as a position indicator for the 3D model that will be displayed.

We will use the marker-based AR on android smartphone, so that users of this app can use android smartphone camera to see the view from various perspectives. So, this software product is supposed to help students learn the position of each organ of the body and then observe them from various angles.

Modeling: We will use Blender Foundation, a 3D modelling tool for this purpose. Any 3D polygon that is used to compose the whole structure of the model (i.e. cylinder, sphere, cube) is considered as a starting mesh for the modeling of the organ. The faces and vertices of the polygon are modeled it into the desired shape using techniques like extrusion, contraction, scaling morphing, sculpturing etc

Texturing: After modeling the 3D structure texturing will be applied on the software. For applying texturing on the model, we will use Substance Painter. The brush tool will be used to spray the texture over the model by dragging the cursor over the 3D model. Some preset software can also be used for this purpose to save time.

Animation/ Simulation: The animation and simulation of the modeled organs are done by using the software’s such as:

Cinema 4D: The software we will use to animate the model of the organ will be Cinema 4D. We will use Deformers such as bulge, jiggle, spherify, bend, twist, etc. The parameters for the deformers will be set according to the requirements, and the key points will be set in the timeline of the key frames of animation. Such types of animation, which is done using deformers is Point level Animation (PLA).

Unity 3D: To import the point level animation controller data into unity, Megafires plugin will be used and applied on the loaded animated model

Benefits of the Project

• Solution for effective and efficient anatomy learning.
• better understanding of the human anatomy and its organs through AR.
• detailed analysis and view of organs by providing simulation and 360 view.
• time and effort elimination in searching different diagrams and studying many books in order to get the understanding of organ
• Paper cost reduce (use of 2-D diagrams of organs)
• Less energy used
• Time spent reduce
• Better understandably
• bringing change and innovation in Education
• Cost reduction (cadavers are costly )

Technical Details of Final Deliverable

Minimum required resources to implement or develop this project:

• Processor: core i5 3rd generation or higher
• Ram: 8 GB or higher
• Internal Memory: 128 GB or higher
• Operating System: Windows 10
• Android phone to test and run this app

As the project is an android mobile application the requirement to use this app are:

internet connection to download the application

android phone with minimum  minSdkVersion ? 24 (API Level 24, Nougat, version 7.0) 

The Main Functionalities of the deliverable are :

• Login: The user will login to his/her account in order to access other functionalities using email and password given at the time of registration.
• Signup: registering a new account to avail the functionalities of Interactive Simulative Anatomy by signing up using email, a unique username and password.
• Pointer selection: pointing the device to the person or diagram in order to make the marker learn about the position and shape of the person or diagram and letting the user select from the marked place which organ to augment.
• Organ selection: this means selecting which organ to augment after making the pointer selection and marking.
• Model generation: this means the organ on which user has point to augment, will have its model generated and by using marker-less augmentation techniques of AR the model will be then available to view and simulate.
• Simulation: simulation means the organ whose anatomy has been augmented in the mobile application after user selection has a simulative 360-degree view and organs approximate imitation available in a 3-D model form.
• Pinch-zoom, zoom in/Zoom out: this means the augmented view of the organs will provide featured of zoom-in/ zoom-out to the user to have clear and detail understanding of human anatomy and organs.
• Annotations: this means the organs will have a detailed labelled description of them in animated/simulative or 360-degree view
• On-Click description: once the user select the organ to augment, written description and details about the project will be available for better understanding of organ in detail.

Final Deliverable of the Project

Software System

Core Industry

Medical

Other Industries

Education

Core Technology

Augmented & Virtual Reality

Other Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Quality Education, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com