aka.wiiremote- This object is the only external object installed, however it is the most important. It is used to pick up several different types of data from a wiimote via bluetooth. This object gave me a lot of trouble when trying to connect to it, especially when trying to connect 2 or more devices to it. But when all the creases were ironed out, it worked perfectly. hi- The human interface object, previously used in my old patcher was used in this one to connect to a wireless, optical mouse
which was then used to manipulate the play speed of the samples. groove- This object was used twice, once for either foot, to loop one of 12 samples (6 per foot) The sample being played by the groove object was specified by 6 different messages to set them, which were controlled by a 6 way gate, connected to a random number generator. Vst- This object was extremely useful and allowed me to import a library of downloadable effects into the patcher. The parameters of these effects were then manipulated by the tilt data from the wiimote in the right shoe.
Other Objects; Gizmo- Used to alter pitch of samples. Random- Used to generate random numbers. Gate- Used to pass data between several different points depending on the users commands. SfPlay- Used to play one off sounds (not loops) like kick trigger effects.
The supplied 14 samples were all created using garage band, and designed to match in terms of time signature and genre.
It was a tight squeeze and took a lot of hacking with a big knife, but the wiimotes are finaly in the shoes, which means I can test out the functionality of my project without gaffer taping a wiimote to the side of my shoe, which is good. Thanks to Richard Searle for the help with this (and for letting me use his mac mini!) The project is really taking form now, and is working and sounding just how I wanted it to so far.
Before simplifying the technology used in my shoes, I wasn’t sure that I would be able to make 2 of them. However after indroducing the wiimote as the new core technology, the second shoe became a definate go. Connecting 2 wiimotes to the aka.wiiremote object wasnt easy though, there is very little help with this online, not in english anyway, but eventually I worked out that you need to specify the unique mac address of each wiimote in 2 seperate instances of the object.
I have decided to assign different responsibilities to the different shoes. The right shoe controls the drum samples, and the left shoe controls all the vocals, synth etc. By controling different samples with different feet the project takes on a rythm game style playability and has a lot of scope for some really fun beatmatching while dancing.
Vts-
VST are awesome, I cannot beleive I only just found out that these can be manipulated in max/msp.
VST stands for virtual studio technology, a vst file is basically like an audio effects file, you can download loads of different effects from the web and affect parameters of them by passing messages into the vst object in max. VST effects are perfect for manipulating my samples via the wiimotes tilt data, Ive got it all working and they sound great. The effects i’m currently using are a Bitcrusher, a Retuner (works best with vocal samples), the Muffle parameter of a Overdrive VST, and a VST called kontrast that creates a kind of underwater effect. The 4 effects can be manipulated by the tilting of the foot, when the foot is in the resting position, the values of the vst are at default, and then are gradualy manipulated as the foot tilts. It sounds cool and works great.
Heres another sound shoe project that ive found, however this one doesnt seem to focus on music manipulation or creation, but mainly on sound effects, and pressure sensing. It is still an interesting project though, I like how it uses audio to simulate walking on different surfaces. This just goes to show how using movement and shoes as control devices is really cool!
After a rather productive session in the Sound Labs, I finally managed to get the wiimote connected to the mac, and then receive the data via the aka.wiimote object in max. However
the values the accelerometer were sending were very hard to work with, I needed to convert them into usefull numbers for manipulating music (currently experimenting with pitch shifting via Gizmo object). At this point, my friend Alex Ryley suggested I use the ‘Scale’ object. This object was perfect for what i wanted to do, all you need to do is specify the current min and max values your device is sending, then after specify the min and max values you want to convert them to. After doing this, I could then connect the wiimote to my previous project xboxmix, and use the tilting of the wiimote to affect speed and pitch. Here’s a screenshot of the aka.wiimote object in action.
After deciding to use the wiimote for my project, I started to research other artists on youtube who use wiimotes to manipulate sound. One of my favourites was this project by sound artist Daito Manabe
He seems to be using looping as the main form of sound manipulation, as he twists the wiimote the application reloops and stutters, which sounds quite cool! However I plan on using different methods to remix my music.
The technology I originally planned to use for my Project, were a set of LDRs and an accelerometer. This technology would also have to be wireless, as my project is wearable, and a dance tool, it would be heavily restrict the playability if the user was wired up to a computer. Creating a wirless accelerometer and LDRs that can input data directly into Max/MSP may be a little ambitious for my electronics skills.
This is why ive simplified the technology of my project. It seemed pointless to try and create a wireless accelerometer, when I essentialy already had several lying around the house in the form of WiiMotes. I did some research into connecting wiimotes to computers, and came accross a website dedicated to just that- WiiLi If I could connect a wiimote to my pc, and also get info from it into Max, most of my technology problems would be solved!
After a while of downloading different apps that didnt seem to work, it wasnt looking good. Untill i came accross this site Aka.Objects. This is a site of several really clever external objects for Max developed by Masayuki Akamatsu, one of which is dedicated to reciving wireless data from the wiimotes accelerometer, called the aka.wiiremote. This external object however was designed from mac use only. Its starting to make a lot more sense to go down this root due to the bluetooth connectivity of macs, and the functionality of this object. Looks like im switching to a mac!
This project was pointed out to me by my tutor, Dan Livingstone. The concept is very similar to that of my project, In that it aims to create music influenced by dance and movement. I watched some of the videos for this project, and felt that the context of expressive dance being converted into audible forms was well acheived, however the general outputs werent exactly what i had planned for my project. The audio produced was interesting and gathered with some really impressive tech, but it just didnt sound that fun. This made me think a little more in detail about the functionality of my project in terms of changing the model to a remix device, rather than a stand alone instrument, like this one.
Instrument Vs Remix Tool
After researching other projects similar to my idea, like Paradiso’s expressive footwear project, I’ve started to think a little more indepth about my projects functionality, specificallyconverting it into a remix device and sampler. The reason for this decision was inspired mainly by viewing the outputs of Paradiso’s project and realising although the project ideas are similar, thats not what I want mine to sound like at all. I want my project to be fun to use, and generate music that people actually like listening to. The advantage of using samples and using movement to manipulate them is that I can use rythm as a key part of my project, and use dance for beatmatching. Rhythm is such a key part of dancing, it should also be the key part of my project.
My project idea looks into the field of wearable hardware, I intend to create a user interaction device in the form of a shoe, or shoes. An advantage of using such a familiar object in order to collect data, means it can draw on different peoples existing skill sets involving rhythm, dancing or anything else to do with the feet. The shoe device will provide data to manipulate audio or even to create its own sounds using something even as simple as walking.
Technology
The technology I will use to generate data would include a set of light dependant resistors on both the heel and toe of the bottom of the shoe, and also an accelerometer in the shoe itself. The LDR’s would be able detect when both the front and back of the shoe hit the floor as the light source will be completely cut off, so if I were to use a pair of shoes instead of one, the time between steps could be logged, and therefore walking speed could be used as a manipulative tool for the audio (most likely tempo). The accelerometer should be able to detect the height of the shoe from the ground, and also any angling or twisting of the foot. It is a possibility to cut a corner with the implementation of an accelerometer, by using drivers for PC that work with the WiiMote, therefore hopefully being able to generate data from the accelerometer wireless-ly into max.
Max/MSP App
In the previous sound applet project I have acquired several skills useful for live audio remixing, such as pitch and speed manipulation, independent of each other, or together, as well as loop creation and manipulation. These could all be applied to to input data generated by the LDR’s and accelerometer, but the way in which this is this is going to be done is still up in the air. If I were to use the walk speed detection, the Max app would also have to detect the time gap in between 2 LDR’s reading zero, then generate a number from that interval using some simple equations. It is still possible that I may use the data generated to create sounds rather than manipulating (or both) this would change the shoe from a tool into an actual instrument, and would be interesting to deliver an audible representation of movement such as dance or walking.
Collaboration
It Is possible that I will be collaborating with Ben Reynhart, who is also exploring a wearable device in the form of a Zipper/Jacket so we may develop the Project as an interactive sound suit that manipulates the same track rather than 2 separate projects remixing different tracks.
Presentation
As the project is focused on the way in which people interact with the hardware, the best thing to do would be to record a generous sample group of people using it in different ways, as well as an organised live collaborative performance. The pre recorded videos will show the range of what the project can do, and the live performance will show its improvisational and generative qualities.
