Augmented Reality And E-NABLE

Just about 4 years ago, the 3D printing industry was taken in a whole new direction when two men on opposite sides of the world, collaborated together to create a functional 3D printed hand for a little boy in South Africa and shared those designs into the open-source community, for anyone, anywhere to use.They watched as the e-NABLE Community formed and started using the technology to create 3D printed assistive devices for those with upper limb differences. Things like toys and Yoda heads are a fun way to learn & build skill, but beneficial projects can create a tangible context for the usefulness of 3D printing. They showed that it was possible to come together from all over the globe to collaborate and find a way to use a new technology for the greater good of mankind in ways that the developers of this industry had hoped people would eventually use the technology as well as in a way that they didn’t anticipate.

rrmanual

Today, I am excited to share what we believe to be the first ever open-source Augmented Reality (AR) assembly manual for a 3D printed upper limb assistive device in the world! Once again, this is a collaborative effort from talented and passionate individuals who are trying to find ways to use another new technology to help improve the lives of others in ways the developers had not expected.

e-NABLE’s Augmented Reality Raptor Reloaded Assembly Manual from e-NABLE on Vimeo.

Introducing the Raptor Reloaded Augmented Reality Assembly Manual, which uses augmented reality to show the hand assembly in 3 dimensional space by using your smart phone or other device with a camera (currently only available for Android) pointed at a target image displayed on another device or printed out onto paper! This free app was created by Derek Delizo, an electrical engineering student in his junior year at University of Washington Bothell, along with his mentors, Rafael Silva and Ivan Owen for the e-NABLE Community.

Instructions can be found here:

UWBarmanual-2
Rafael currently works for UWB IT as a STEM Liaison where he manages the Unix Infra-Structure of CSS Division as well as leading a research group that investigates how Virtual Reality can improve rehabilitation of impaired upper limb patients.

Ivan is the Makerspace Lab Manager and student mentor at UWB and is the co-creator of the first 3D printed hand that was shared into the open source community as a public domain file, resulting in the eventual creation of the global e-NABLE Community of volunteers who have since delivered over and estimated 2000 free hands and arms to those in need in over 50 countries.

It is our hope, that this augmented reality Raptor Reloaded user manual will help volunteers and recipients to be able to see how the parts fit together in a fun, useful and non verbal way, but we also hope that this app will get the global maker community as well as students in classrooms, thinking about situations in which they can use this newer technology in ways that could make a larger positive impact on society.

Rafael shares, “AR (augmented reality) and VR (virtual reality) is an eye catching field that has the potential to attract and motivate several people to develop and use this technology. Most of the development we see today is aimed at the Gaming industry, but there are several other serious applications that need to be considered. The more we use/talk/think about the noble uses of this technology, the more the industry will invest in evolving this application for something other than video games. It’s important that the general public understand the full potential of these tools and not antagonize it as just another video game thing.

He goes on to add, “It took decades for this technology to evolve to be feasible and available for the general public. We are in a exceptional stage where we can finally push boundaries in many fields, by using this technology. We are in contact with experts in the medical field (UW’s Harbourview Medical Center and Dr. Albert Chi) that are really excited to see what we are developing and with their help we are able to make our work available to real patients and people which can significantly improve their lives through this technology. The biggest selling point of this technology, mainly with mobile AR/VR, is the fact that it’s democratizing these solutions, by dramatically decreasing costs and facilitating access to cutting edge treatments.

My main motivation to learn about AR/VR was to develop solutions to real world projects. There are plenty of new video games out there, so I felt the obligation to take advantage of the big technical advances we have in recent years and use it to address challenges that can improve people’s lives,” explains Rafael. “When I met Ivan and showed him my previous work, our interaction resonated with amazing synergy. In one of these conversations he brought up the language barriers faced by people trying to assemble the e-NABLE Community designed hands, all around the world; from this conversation the AR Raptor Reloaded Manual project was born.

ARSCREENSHOT

Ivan shares, “I believe Derek was an absolutely ideal student to pick up the rough proof of concept that Rafael and I created and then carry it forward to a functional mobile application.

He goes on to say, “The passion, drive and effort he brought to the project combined with the excellent ideas and additions he made were incredible. Additionally, this was quite an achievement as it was the first AR application he has ever developed, so he learned an incredible amount along the way.

We had a perfect environment for this to take place in. Rafael has a talent for teaching and his ability to break this complex subject into manageable portions for the students to dig into is excellent. He is also very skilled in guiding the students through the process of project management. The professor of record for the course as well as our guide and mentor, Dr. Pierre Mourad, was instrumental not only in helping Rafael and I to develop a well structured lesson plan, but also through his leadership in modeling interdisciplinary environments at UWB.

The student research group which Rafael is leading, and of which Derek is a member, now has students involved from Electrical Engineering, Computer Science, Mathematics, Mechanical Engineering, Interactive Media Design, Health Sciences, Biology & Business majors. This creates a highly dynamic environment where students learn from their instructors and mentors, but also from each other by learning new skills that fall outside their major. It’s been wonderful to see it all take shape.

Spring quarter 2016 I was apart of a mixed reality class that taught students how to develop augmented and virtual reality applications,” explains Derek. “Ivan and Rafael discussed their idea of creating an augmented reality assembly manual. I was interested in there idea and decided to do it as my final project. I wanted to be challenged.

Derek goes on to add, “It only took a little over 2 weeks to develop the manual and to complete all the publication steps. Creating the animations was the biggest challenge, as in the class we had only learned how to use blender. When exporting the animations in Blender it wouldn’t combine the individual pieces into one animation. Instead there were 100+ clips that couldn’t be used. My first solution to this problem was to code the animations by hand, and after that didn’t work Ivan suggested to use Maya. I ended up teaching myself Maya and completing the animations in 2 days.”

This is actually the second interaction of this project,” said Rafael. “Before working at UWB, I was a graduate research assistant (2013/2015) at the Virtual Experiences Laboratory at UGA, where I got introduced to the Virtual Reality field. When I started working for UWB and first met Ivan, in May of 2015, we started looking into ways to teach students to develop meaningful AR applications and the manual was the first idea we actually tried to evolve.

UWBarmanual-1
We first started with a small group in summer of 2015, but due the lack of institutional support, the project didn’t have much progress at the time and the initial group was dissolved. Ivan and I ended up building a proof of concept that implemented the assembly of just one finger, but stopped there, so we could show the feasibility and spark students motivation to participate.”

FINGER

Rafael goes on to add, “At the end of 2015 winter, Professor Pierre Mourad invited Ivan and I to teach a Mixed reality class for a small group of students on the following Quarter (Derek was one of these students). In the first half of the class, I presented the introductory VR theory and taught the students on how to develop a mobile virtual reality application. In the second half of the class, the students applied what they learned in their final projects and the manual was one of them. Besides the manual, the students developed a training system for robotic hand prosthetics, an instructional app to orient on the use of a mill machine and a game that enabled “real” gravity in a virtual maze.”

We are hoping that like 3D printing, we will start seeing more schools getting AR/VR technologies into their classrooms and finding ways to make a difference in the world.

As consumers, this technology has the potential to improve cognition processes and make learning more engaging,” explains Rafael.

As developers/producers, they need to always have a critical thinking mindset about it,” shares Rafael.  “VR/AR is just a tool. It has a very specific purpose and works better than other tools in some very specific cases. It’s great if teachers and Professors use VR/AR as a motivation tool in their classrooms to learn about programing and engineering but they need to have a lot of content and valid reasons to back it up.

He goes on to explain, “If they just present the technology without a good context, it will not hold for long the students’ motivation and it can be more distracting than beneficial. The key for AR is to find the right applications for it. Also, AR is a concentration of multiple major fields, so it’s important to understand the basic concepts (programming, computer graphics, tracking, etc.) in order to build a solid foundation to master AR.

As we learned with the first group that tried to develop the manual, enough structure and good planning is essential to succeed in teaching people how to develop AR/VR applications.”

We encourage all of those who are able to download and use this app, to please do so and share any feedback that you have for Derek about how he might improve this app for the community or let him know if it was helpful to you, your students or others as they build their first e-NABLE Community Raptor Reloaded hands!

This is still very much a work in progress!

Derek adds, “I will be working on some updates to the application that include adding an assembly step for inserting the wire, allowing the back button to step backwards through assemblies and adding the tensioner clip to the wrist assembly.

ASSEMBLE
Derek explains some of the current features: “There is a stage selection drop down that works as a table of contents and allows the user to choose the different assemblies. There is a highlight feature that can be activated by clicking the target button in the bottom left of the screen. The highlight feature can be used to identify the name of the pieces in the assembly process. For the complex assemblies there is a play/pause button to pause the animations. The arrows in the bottom right are used to navigate through the steps of each assembly. Finally, you can swipe vertically on the screen to rotate the assembly and view all parts of the animation.”

Please share your videos and images of yourselves trying this app out by using hashtag #enablethefuture on Instagram, Twitter and Facebook! We would love to see you playing with it, especially in the classroom setting!

Go forth and “enable the future!”

“Invention is the most important product of man’s creative brain. The ultimate purpose is the complete mastery of mind over the material world, the harnessing of human nature to human needs.”
― Nikola Tesla

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.