Concept:
Jeremiah Baker, Ben Hoggatt, and I wanted to create our own relief sculpture to map. While brainstorming we thought of making low poly instruments we could light up with music. I remembered I drew some geometric art months ago and showed them it. Ben quickly made a 3d model of it, Jeremiah unfolded it in Pepekura, and then I laser cut it. Jeremiah and I pieced it together and created what we call the Growth Sculpture as it resembles a vine or organic intrusion on a building.
I planned to create one composition with Max MSP and another using more of the Google Deep Dream Processing I found in experimentation last project.
Process:
Within Max I used the previous setup I used for creating the John Conway's Game of Life for the tape mapping. I replaced the images used with video created by Falak Jamma. I utilized a new command called jit.pix where I could pull in files to modify the video. I utilized the brcosa, infinite mirror, twirl, and pinch files. I sent those videos the MadMapper to record them. I found the way of recording through Max to be unreliable as sometimes it would corrupt or not record anything. I applied the Spiral Galaxy looking one to the actual sculpture then made a dim kaleidoscope effect in the background.
For the Google Deep Dream one I took a picture as close to the projector's view as possible and cropped it to just the sculpture in Photoshop. I do not know enough about the technology of Deep Dream to create an animation the correct way. I instead slowly zoomed in through Photoshop and rendering images out as I zoomed in. I sent all of these images to render into the Deep Dream look. I then stacked all these images together and converted them into a .gif file. MadMapper works well with those. I used a different kaleidoscope from Max and shifted to a yellow-black color to better suit the Deep Dream results.
Concept:
This project endured many different forms in my head. When first announced I wanted to do something interactive with a Kinect. I have no knowledge of the Kinect and how to transfer it to something like this. I instead decided to create multiple animations and a dark theme to go with a song. I wanted a song that had an industrial (like a Workshop/Manufacturing, not the music genre) sound. In the past year I had played a game called Ruiner. I knew that had music very close to what I was looking for and listened to it's soundtrack multiple times over to find a song that I could envision in the space. In context of the game all the music fits perfectly being rough, repetitive, and slightly unpleasing to listen to, because that isn't the focus and fits the environment that's rough, repetitive, and slightly unpleasing to look at (Dystopian, reds, blacks, and strobe lights). I knew I would be listening to this song hundreds of times so I had to at least enjoy it. I knew others would hear this and I chose the one song that stood out as a median between pleasant and industrial. A good test of would my mother listen to this at least once or twice and not skip it after 30 seconds (which is something I considered, love you mom).
Song Selected - Call by Zamilska.
Animations, How can I activate this space. I had many more ambitious ideas than I was able to accomplish either due to time or lack of skill. I knew because of the music and the source of where I found that music I knew to create a dark environment. I knew I wanted rats, from the beginning, to run around the space. I wanted those same rats to look into the space. I added holes to help the rats look into the space. I created two big gears to anchor the space. The backdrop was from an ad campaign for the game that I heavily transformed.
Process:
Going in Order, Song Selected, I knew before I started timing things I needed assets to apply to the timings. The rat running down the ventilation shaft was a goal for me, if I could do that, I could build every other asset I wanted too.
Rats, I got references of rats running (A gif by Karrakas significantly helped me) and created a 10 key frames of a rat running in photoshop. I took these rat templated and a MadMapper Mask I made of the vent and animated the rat running down the vent (50~ frames). I tested it and first try it looked amazing. I knew I could do this. Continuing with the rats, I created two different lengths of straight across running animations (50~ and 100~ frames long).
All the Animations (Excluding backdrop) were frame by frame animated by me in Adobe Fireworks (Which I haven't touched in 6 years) and Exported as gifs.
Gears, The big gear was created first. I've made gears before in Rhino and knew I could recreate them easily. The first Big gear had nine teeth and five inner slots. I learned when animating that in order to create small loops that could be repeated easily the amount of teeth needed to be divisible by the amount of inner slots (Here's some numbers to explain why this happened animating wise. In order to turn the gear one tooth length I had 4 frames planned. If you take 9/5 you cant simplify it into an integer without rounding. With my Second Gear I had 12 teeth and 6 slots, 12/6 can be simplified to 2 or 2/1. The numerator determines how many full teeth turns needed to make it look seamless and able to be repeated. In addition it takes 4 frames to animate and since I'm doing it by hand it matters. First Gear - 36 frames, Second Gear - 8. Work smarter not harder is a good lesson I've learned repeatedly this semester.). The Big Gear has 12 teeth and six slots while the Small Gear has 10 teeth and five slots. I animated one set of revolutions for each. I then lined up a circle with each medallion to make sure it fit right when I was animating it. I recreated the animations and transformed each frame of each gear and placed it correctly.
Holes and Eyes, The holes were created in Photoshop. I planned multiple and different holes but I had a hard time drawing them, as references didn't help me particularly well. I drew one and ran with it knowing I could rotate and flip it. I created 5 key frames for holes. The eyes I created at lightning speed as I was really familiar at this point with Adobe Fireworks. I created a wide open and half open eye in Photoshop and created a mass of them animated in Fireworks.
Gifs, MadMapper accepts .gif files and works well with them surprisingly. I used gifs to make sure early timings were perfect.
Timing, The Music this was all timed to music through Adobe AfterEffects, which I had never used before so it was quite the learning (and frustrating) experience. I timed out all the effects in the song that I wanted to time things too. The gears would turn on certain beats, the rats released on others, the holes were created early on, the eyes appeared with voices. The small gear was timed first and since it was a irregular beat I timed it in Fireworks, Took three different tries because of the massive amount of frames. This is where I tried to export the whole timeline of the Gear throughout the song as a gif, No No don't do this it was a horrible idea and it crashed Fireworks and froze up the computer for a good amount of time. I left Atec and went for lunch at that point hours of work down the drain, worried these gears wouldn't work. Came back restarted lined up one animation and got it timed back to where I had it last time and cropped it so it wouldn't freeze hopefully. I learned the amount I cropped significantly decreased the lag, so all my timings were off by up to 30 milliseconds. I re-timed it. Duplicated it to line the entire gif up with the song and export it, No No bad idea, you think I learned something last time. Crash, Restart, Re-time. I didn't duplicate it every time the gear spun it was like 2800~ frames when it was crashing (what a fool). This is where After Effects came in. I timed it instead in After Effects after converting the gif to a mp4. After this tedious experience it was mostly smooth sailing. I just lined everything up in After effects and exported them all as .avi files (I tried exporting through Media Encoder and was having all sorts of crashing and nonstop errors). There are seven places the rats can appear (15 appearances) , two gears, four holes, six timed eye set appearances. I don't like to point out flaws, but in this report I've mentioned multiple times about animating the big gear and it is still in the show. I animated the key frames and lined them up with the MadMapper template. In AfterEffects when I imported the mp4 I made (did check to make sure the mp4 was correct) it would only show two of the eight frames I animated. I couldn't figure out why and elected to move on and return if I had time. Each set of animations per computer setup was delayed in 5 second intervals so I could start each myself. Of course this is not perfectly accurate and leaves some upsetting discontinuities between computers but I elected it over trying to time 4 different people to click all at once.
(Quick Note about this Gif and Ruiner. The game is violent and about saving your brother from his captors in any way necessary. This gif contains the words "KILL YOU" scrolling across the screen. Given the subject matter I can see how this might upset some people. Since my theme was dark but not necessarily about violence I made sure in Max to make the text on the helmet/face non-legible)
Backdrop, I planned on creating my own video and putting it through Max to distort it with tests I had done earlier. This didn't work very well as I came to this after all of my animating and timing had been finished. Realistically, I had no idea what to do or how to do it. I found an gif of the Ruiner character from an ad campaign for the game that I thought would be interesting and after many Max tests I found a setup wasn't expecting that I loved. The Composition included duplicating the image multiple times, shrinking, scrolling, rotating, blurring (not really, but not easy to explain and blur is close-ish) and adding a VCR line distortion.The recording commands in Max just didn't work. I recorded through Windows recording and recorded the MadMapper Output that was Spout'd from Max.
The Backdrop was placed zoomed out on the far hallway and the two opposing walls. The other five walls I made a near seamless (except when the timing isn't perfect) five projector blend of one texture across three computers. I am very happy with that. I'd like to add the remaining three walls so I could have one unified blend, but I need to figure out how to increase the resolution in Max which is surprisingly elusive.
Additional Comment - Most of my other posts are worded differently. They go straight into it on how I did what, not saying much about trial and error. For this since I for a good part used programs I'm unfamiliar with I wanted to go into more detail about everything I learned and the difficulties I had. I really wanted with this project to expand my bounds and try something new. I could've created something new with Max and only Max. Instead, I want to use my time in this class wisely and try as many new approaches to this world of projection mapping while I have this perfect space available to me.
Thank you Professor Scott and Staff of the Atec, building that kept it open, for letting me spend my Spring Break in the Lab working on this project.
I saw the Cabinets as a perfect setup to create a piano setup. There are 12 cabinets and that results in just one key short of an C to C octave. I created the piano using a standard USB keyboard, using the keys " ` 1 2 3 4 5 6 7 8 9 0 - " I made the piano using Max and it roughly plays the right frequencies.
Jeremiah Baker made a Color randomizer for the tape setup that I could overlay with the piano.
I also used the cabinets as a piano and played the Nationwide Insurance Jingle. I changed the color of the keys to be Orange when I pressed them and it was inverted in MadMapper to create Blue.
Techniques:
In Max, In the Sound Patcher, I recorded the keys as input values and ported them to a bus that would only output specific places based on the input value. When a key is pressed it would output a bang (On/1) and open a gate to let sound through and also trigger a delay to turn the gate back off after a short time. It would also send another bang to a 255 value to an output to input in the visual patcher. In the Visual Patcher, the values sent out from the Sound Patcher would change the color of a visual block to white. All twelve of these blocks would then get assigned a position on a grid (At the same time could be assigned a color for the Nationwide version) and the entire grid would be sent as one texture output through Spout to MadMapper. In MadMapper the grid was inverted in colors to be white by default instead of black. Each Cabinet was assigned to a different section of the grid. When a key was pressed only the corresponding section of the grid would change color and change the Cabinet color
Dual Projectors
Concept:
Within Max I found a built in module to interpret images as starting cells for John Conway's Game of life. I took the line files and recreated them in Photoshop to get the lines darker so they processed better with Max.
Techniques:
I imported the pictures into Max after moving the pictures to a specific Library for Max in Program Data. The picture them got pulled into the Max visual matrix and all pixels that were not completely white or transparent were changed to black. Black pixels served as starting cells for the simulation and the background changed to black and the cells changed to white. The simulation plays out slowly frame by frame and I set a delay to restart it after a short delay so the life cells stayed close to their origin thus maintaining shape. The simulation was ported over to MadMapper via Spout. In MadMapper all I had to do was to align the outputs with the lines since Max had distorted the images.
Llama
Concept:
I really liked creating the interactive piano for the cabinets and wanted something else that was interactive. I thought of the infinity created with two mirrors and replicated that effect with the projector and a webcamera. The first video was filmed during the day.
Techniques:
I put the webcamera closer to level with the tape and masked the output within MadMapper. Using the Tape itself with the camera was too difficult as the camera would just blur any excess light coming in. I added the lines and created a Purple Color with them and created a Purple Tint for the Camera layer as well. How much excess light is present determines how quickly the Purple color blurs and creates the Blue Haze. I tinkered with the color profiles to minimize the Blue and found that a lower Opacity for the camera layer makes all the difference to create the 3D effect. I added a Vertex Noise effect to the Camera Layer in MadMapper to create the swaying effect. I also created two quads that periodically light up for the eyes to help emphasize the backwards movement. The second video was filmed at night and shows how a lower surrounding light level drastically changes the effect.
Concept:
Over Winter break I traveled to Morocco to visit my sister and got plenty of beautiful pictures of the landscapes. I loved the view from my sister's balcony. It peers over part of the city of Rabat and looks almost straight at a Mosque. Although noted that projectors wouldn't reach as far as I mapped I still strived to highlight the building shapes and edges. In addition, to the work in Mad Mapper, I created a type of visual sound synthesizer and imported it live from Max.
Techniques:
I started with two quads on the far left building that I chose to map and set up the perspective. I used the quad mask and control point snaps to mask only the building and take the windows out. I duplicated this process on the building in the middle. The building on the bottom and back right were created each with a single quad and mesh warping. I masked out the buildings and windows but also masked out the red trim on both buildings. On the First Display I used contrasting colors as well as the Max Display. On the Second Display I sampled colors from the buildings around to created projections that blend better with the surroundings.
For the Max Display I split what I made into three separate parts. The First Part creates the background of colors. Its created with noise that is distorted, spun, and blended. The Second Part takes audio input (such as music off of my phone), duplicates it three times, then delays each duplicate. These duplicates get converted from Max's audio language to Max' video language and each duplicate gets applied to a different section of a scope creator that controls the colors. These two videos get meshed together. The Third Part makes the video into a texture and refreshes it live on a plane. It also uses a plugin that Mad Mapper recognizes called Spout that lets the video be transfer through the programs.
