Advanced Interactive Design / Task 1: Interactive AR Application
11.4.2023 - 16.5.2023 (Week 2 - Week 7)
Takuto Hozumi / 0354047 / Bachelor Of Design (Hons) In Creative Media
Advanced Interactive Design
Task 1: Interactive AR Application
LECTURE
W1 Module Outline
In this module, we would learn interactive design following the latest trends. The module consists of three assignments:
- Task 1: Interactive AR Application
- Task 2: Interactive Web Application Pre-Production & Prototype
- Final Project: Thematic Interactive Web Application
W2 In-class Lecture (Effect House)
In the second week of class, we tried out the Effect House for a trial
run.
Fig1: Pre-installed "Effect House"
What is Effect House?
Effect House is a video editing software developed by TikTok for creating
and editing visual effects in TikTok videos. The software is designed to
be user-friendly and intuitive, allowing users to create high-quality
effects without extensive technical knowledge. Effect House includes a
wide range of tools and features, such as filters, stickers, animations,
and transitions, that can be applied to videos in real-time. The software
is available on both iOS and Android devices and can be accessed through
the TikTok app.
Fig2: Create an effect from preset
Easily create effects by adjusting the parameters provided. And the visual
scripting window below allows you to configure how the effect will behave
on the actual device. For example, you can set the effect to be applied
when the user taps the screen.
Fig3: sketchfab.com
If using a 3D model, the website above can be utilized.
Fig4: Mini Clones
For this tutorial, we have reproduced the procedure for making mini clones
as described in the guide on the official Effects House website.
The procedure is as follows;
- Add an Image object by clicking the Add button [+] > 3D > Image in the Hierarchy panel.
- With the Image object selected in the Hierarchy panel, go to the Inspector panel and click + Add component > 3D > Look At.
- Add a Head Tracker object by clicking the Add button [+] > AR Tracking > Head Tracker in the Hierarchy panel.
- Select the Image object in the Hierarchy panel, then go to the Look At component in the Inspector panel, and change the Target property to Head Tracker.
- Change the Look At component’s Mode property to Look At Direction.
- Add a Portrait Segmentation Texture by clicking the Add button [+] > Texture > Segmentation Texture > Portrait in the Assets panel.
- Select the Image object in the Hierarchy panel, then go to the Image component in the Inspector panel, and change the Texture to Portrait Segmentation Texture.
- Resize and position the Image in the Scene panel to your desire.
- Feel free to duplicate the Image in the Hierarchy panel and create as many mini clones as you want!
After completed, the effect was output from the QR code in the upper right
corner [preview in TikTok] to confirm its operation in the actual
application.
INSTRUCTION
Project Outline
Description Augmented Reality Experience Filter AR is the latest form of
interactive design nowadays. Students are required to develop AR application
with AR interactivity which will detect user face and expression.
Software Requirement: Effect House
Requirements Students are required to develop an AR application for an
online store. For example, an AR application for glasses, hats or cosmetic
shops. Customers of the shop will use the AR application to try different
options before making their purchase. Students are welcome to propose new AR
application ideas as well, but must obtain the approval from the lecturer.
Submission
- Completed AR interactive application project files to be submitted to Times Portal.
- All processes (concept, wireframes, mood-board, flow chart) have to be documented and posts in your E-portfolio as your reflective studies
- A video recording showing the usage of AR application need also be submitted for verification.
- Students are welcome to publish their filter to Facebook and Instagram platform
Ideas
The first week of lectures showed that to create a good design one must be a good user. Therefore, each student was asked to install TikTok and research effects. W1 first task was to prepare 3 ideas for effects to be created in this assignment.
- Idea 1: Animal face
An effect in which the face becomes a horse was very popular. > Create another animal? Ex.) randomly create an animal face that resembles the person? need further research about 3D modeling
3D / Random
- Idea 2: Using dynamic photo
Ur facial expressions link to the loaded photo? Still have no idea how to use it in a filter, but seems interesting.
Beauty?
- Idea 3: AI illustration filter
Filters to make AI illustrations seemed to be all the rage late last year. Not only such a beautiful one, but I'd like to create a little more realistic one. Need further research about AI Illustration
Effect Ideas Proposal
On W5, I'll consider ideas that are even more specific than W1 ideas.
- Idea 1: Animal Face Effect
Fig5: Horse Effect
I would like to do something similar with a dog motif because I like them.
There is no particular necessity for the motif to be a dog, so it could be
any other animal.
Fig6: Dog Head 3D Model
Possible uses: Enjoyment with family and friends
Target: Age 0 to 35
Genres: 3D, Fun, Cute, Weird
- Idea 2: Creepy Deformation Effect
Human body parts are composed in exquisite proportions, and it with some of
the sizes wrong gives the viewer a very uncanny sensation.
Fig7: Lee Griggs "55"
Fig7: 葛拉姆 "派翠西亞・佩奇尼尼"
Specifically, I plan to extremely distort and duplicate some of the body parts
to create an eerie human body.
Possible uses: People with a unique sense of style
Target: Age 20 to 40
Genres: 3D, Artistic, Creepy, Horror
- Idea 3: AI Illustration Filter
I researched "Stable Diffusion," an image generation AI. Perhaps the TikTok
AI illustration filter I presented in the previous article also uses this
system.
Fig8: Drawing Style Changes
About the drawing style, found that it could be changed by adjusting the
prompts.
Fig9: The video you get every so often
There appear to be several different filters for the AI illustration
generators currently prevalent on TikTok. However, all of them have a very
flat design and animated style, as shown in the image above. So for example,
Picasso's abstract or Monet's impressionism? It would be interesting to create
something in a slightly different style.
*This idea is expected to be very difficult from a programming technical
standpoint on how to couple WebUI and filter.
Possible uses: Enjoyment, Casual Art
Target: Age 10 to 40
Genres: 2D, Anime, Artistic
Fig10: Proposal Slides
Progression on Effect House
After consulting with the lecturer, the decision was made on Idea 2: Creepy Deformation Effect. At this stage, I was studying the technical feasibility of this theme. Firstly, I looked for an option to deform the shape of the back of the head, but it was not present among the options in Effect House.
Fig11: Face inset
The Face Inset template appeared to allow for the attachment of facial parts to the 3D model. So I planned to actually represent the oddly shaped head with a 3D model and paste the face on it.
Fig12: Importing 3D model
Fig13: Human Body 3D
Downloaded a 3D model of the human body from Sketchfab and made a first attempt. .bin file failed to import, and I re-downloaded the .gltf file and succeeded. A list of supported extensions should be placed on the official website.
Fig14: Blending 3D model and Pasted Face
I spent a week looking for a way to reflect the skin color extracted from the face in the 3D model, but could not find it. When I reported this issue to the online class at this stage, the lecturer suggested creating a design that reflected the movie characters such as "Conehead" and "Hellraiser". I asked about the copy right issue and received the response that it is OK if it is a parody, not a copy.
Fig15: Coneheads
Fig16: Hellraiser
Fig17: Research on Hellraiser
I had never seen any of these films, so after researching through image searches, I chose Hellraiser.
Fig18: Nail 3D model
First, a 3D model had to be created of the nails to be placed on the face. Since I could not find a 3D model of a single nail, decided to start with a blender and use it to create a single nail.
Fig18: Blender
3D modeling was more difficult to master than expected, and it took two full days to create a single nail.
Fig19: Placed Nail on Face
Back in the effects house, we placed the nails, which went relatively smoothly, as there was no waiting time to duplicate and place the 3D model.
Fig20: Nail Process
However, the horror of the original film was not enough as it was, so I downloaded a preset for creating make-up filters, edited them in Photoshop, and created my original filters.
Fig21: Makeup filter
Also, as you can see from the image search, the Hellraiser film has a slight blue tint to it, so I adjusted the overall color scheme.
Incidentally, at this stage, the lecturer suggested that I rework the original distorted head idea. He also said that I could use the Hellraiser as a second plan in case we could not create the distorted head.
Results of the consideration, for the reasons below, the suggestion was unacceptable and decided to complete the Hellraiser.
- That I explored technical feasibility during Weeks 6 through 8, but was unable to find a good way to do so.
- On my computer, Effect House requires a long freeze time of 5 to 10 minutes per operation, which makes it difficult to do something complex enough to create from scratch in only 1 week remaining until the assignment is due.
Fig22: W10 Final Outcome (5/6/2023)
The final version is shown above with the 3D model positioned uniformly and the overall color and contrast adjusted.
REFLECTIONS
Experiences
Through this assignment, I learned the uncertainty of learning a new technology. This time, I had to learn a completely new field of software for me, Effect House and Blender, at the same time, which was quite a big challenge. To be honest, I am not at all satisfied with my work this time and consider it very poorly done. In a field I am not very familiar with, I have reaffirmed that it is more reasonable to complete a project by generating ideas within the feasibility of the function, rather than generating ideas first and then determining if they are feasible. Also, I probably will never make a TikTok effect again in my life, but if I should ever have the opportunity to do so again, my biggest lesson will be to buy a better PC.
Observation
When it comes to work, it is absolutely unacceptable to deliver low-quality deliverables. However, I was trying this now as a student. I think this is very beneficial and it is good to be exposed to new fields and new technologies that I didn't know about. I am grateful for this environment.
Findings
Actually, I don't use social media much and have especially avoided TikTok in particular, which is made in China, because it is one of the apps that our government does not recommend installing. After this experience, still I thought that the TikTok culture was not very familiar to me. I also found many of the technical aspects to be frustrating. Software that freezes for 5 to 10 minutes on a single operation, despite meeting the required specifications, is not convenient. And as the official website lacked explanation, it seemed that Discord had a community of effect creators where they exchanged techniques. I am sorry to see this reality for such a well-known and profitable company. I hope that issues like these can be resolved and TikTok can grow into a better service.
Comments
Post a Comment