VR+ EM Training
The VR+ Electron Microscope Assembling Training system is a client-cooperative project. We offer a solution for general laboratory researchers to learn and practice how to operate with sophisticated electron microscopes in a virtual environment. The learning mode will facilitate users to learn each steps of EM assembling while the testing mode can verify and strengthen their ability through an engaging way.
Conducted user flow diagram.
Designed the menu interface, loading interface, system specification interface.
Designed and mapped out the user flow for assembling microscopes.
Created the 3D model for the shelves.
Developed the functionalities for assembling microscopes (learning mode) in Unity.
Developed the system specification, loading, menu interactions in Unity.
Electron Microscope (SEM) is an essential instrument in the scientific research world, using by numerous laboratory researchers in their studies. However, compared with the standard microscopes, it is so sophisticated and fragile that requires the users to be very familiar with the operating process. Plus, for most laboratories, the researchers only have limited access to EM. Our client, Electron Microscopy China, who is a leading manufacturer in the EM field, intended to create a virtual environment for their customers to cultivate their researchers familiar with the operations (assemble and disassemble) of the EM professionally.
1. Since the EM equipments are very sophisticated, the team had no access to physical EM. We were offered the 3D models for each components based on real-scales alongside the instructions by our client.
2.The team had to negotiate the VR equipment to test out our product during the development phase.
Key Findings and Insights
VR Training Environment
According to Dr. Ebbinghaus’s memory research, we forget up to 80% of new information over time. It is related to our declarative memory, which works when we try to remember something from regulations or manuals. But skills transferred to our procedural memory — it works when we do something with our hands — stay there for a long time. Through VR training environment, the users can enhance their practical skills.
EM Assembling Process
From our meetings with our client and the lab researchers, the pain points for them to operate the EM are: 1. The devices are very expensive and limited in each laboratories in general. 2. Some of the researchers need to follow professional classes to cultivate themselves EM assembling skills. 3. The assembling process is a very essential part during experiments, which means they need to be very prepared before using EM. 4. Some of the researchers stated that they might feel dizzy with longtime VR equipment wearing.
The training system would not only benefit for the scientific users but our clients are also eager to provide the VR training system as one of their services to benefit their further customers to bring more revenue. Besides, they would like to cultivate professional technicians to meet the labs’ tight need of this kind of professional talents.
· A realistic, accessible VR training platform, to prepare lab researchers for EM assembling and disassembling in an immersive, adaptable environment.
· Learning Mode - The system will facilitate users all the way to help them assemble the EM step by step. Users can follow the instruction board and find the appropriate components to finish the process. This mode aims at strengthening users memory through practice.
· Testing Mode - Users will be required to finish the assembling process without hints and get their scores accordingly. When users reach the qualified scores, they can start to operate the physical EM.
· Teleportation - In order to reduce the vertigo while using the VR equipment, we will develop the teleportation functionality to save users' movements.
Minimum Viable Product
· Two modes of the VR EM assembling system for users.
· For learning mode, users should be able to see a hint board hovering on the screen, and each time when they finish the previous step, the hint will change to the following one. All the components will be labeled by their scientific names, and be placed on the shelf in sequence. The associated parts on the EM will turn into red and green to suggest users if their assembling process is correct or not.
· For testing mode, user should finish the assembling process as fast as possible within a 3 mins countdown. The score board will update when users assemble one component successfully. When the test end, the system will grade users into three classes (A/B/C) to verify that if they can pass the test or they still need to practice.
· In the both modes, users should be able to be transited to the position by pushing the button on the down side of the right handle.
Based on our MVP, the team divided the tasks and each of us were responsible for different parts. The team have daily scrum meeting in the school laboratory to make sure everyone are aligned and find the problems immediately.
After the team finalized our MVP, in order to better understand how our system will work, I mapped of the user flow above. The diagram elaborated the logical principles of the system.
Preparing UI assets and 3D Model
Before developing the learning mode, I created the shelf model and design the UI elements and components for the main screens(loading page, home page and system instruction page). The EM component models and the lab room are provides by the previous team.
Developing the learning mode
After preparing all the art assets, the team started to develop for different functionalities. I was responsible for the learning mode. In Unity, firstly, I rearranged all the EM components and put them on the shelf. Then, label them according to their scientific name and edit the settings. Also, I place the hint board on the top of the lab table as the default position. After all the settings are down, I develop the learning mode coding to control all the components and make the learning mode work.
Integrate the project
After we developed and implemented different functionalities respectively, we started to move all the projects and files into the laboratory's computer and test if there's any bugs or errors. And also connect to our VR devices to adjust some details regarding to the scenes from the VR device.
In this phase, due to the restrictions of VR devices, we did an internal user test with our professor and 2 of our classmates at the school laboratory. Also, we sent out our project files to our client, they tested out the program with another 2 of the laboratory researchers who are familiar with the EM operations and assembling process.
Feedback and Improvements
· All the users in the learning mode expressed that the hind board is a very effective way for guidance; however, 3 of them said that it would be more noticeable for users if the hint board can stick on the top of the screen instead o floating above the lab table in the scene.
· During the testing mode, both the researchers from the lab were worried about our countdown minutes were too short for beginners. They suggested that normally, beginners will be expected to finish the assembling process within 5 minutes.
· 4 of the testers found the teleportation function were really helpful especially for the tester who were in a very small room. They said in this way, they can minimize the movements and reduce the dizziness.
What I Learned and Gained?
· As our first VR related project, we overcame many obstacles regarding to this unfamiliar area. During the project, I learned a lot through experiments and the team had discovered many functionalities and technical knowledge which we also applied into our project.
· Since it's also our first client sponsored project in our university time, by meeting with them and learning their requirements, I gained and improved my communication and negotiation skills.
· Due to the constraints of the project, we had a hard time of testing our program, but the team always very connected and when we had blockers, we would helped each other get out of the trouble, which made our project moved very smoothly.
The project won the Second-Class Prize in the 6th China Digital Media Technology Works and Creative Contest for Undergraduates.