Friday, November 11, 2005

Device / Instrument / Tool - Final Project - " The Spoken Hand" - Group : Terence Arjo, Langdon Crawford, Stephen Kerrigan, Adam Samuels

From Wearable Music -aka - The MIDI Suit Project to:

The Spoken Hand - Auditory American Sign Language

What was the MIDI Suit and how did we get to The Spoken Hand ?

Recent History: Our Wearable Music Project was well on it's way. We had gloves, stitched and outfitted with flex sensors, we had programming through Max, sound tests were "conducted" (pun intended), there were soundbanks indexed in Excel ready to go, we had funky schematics, accolades from well-wishers and a wide array of things we could get these gloves and later elbow and knee braces similarly outfitted, to do.............

Here's (see below) a funky image of our data flow and please see Langdon's Site

for a couple more - they're cool !

The MIDI Suit concept grew from the idea of using normal range of motion in body movement to trigger MIDI : sound effect/ musical/ pecussive events. The viability of this concept is good, testing for the project had begun and the early (raw) results were satisfying. Our group came to a decision however, on a point made by our instructor, Andrew Milmoe regarding focusing our efforts into a more narrowly defined area. We, later that evening, came to a decision to use the model of incorporating customarily used movement in our project rather than movements needed to be made to get function out of the hard/soft -wear designed for the project.
Obersvations and user testing that are part of the project work for Wearable Music will be things that I will continue for specific concerns of mine, regarding therapuetic uses for individuals with cognitive disabilities.
We would have loved to have done the "all of the above" type project but these are probably projects to be considered individually, because there is so much that could be done with any "one" of them.

So,....... Onto the New............
For our group's final project, we have decided to work with American Sign Language, making the alphabet of ASL, audible and transforming the customary movement, i.e. of signing letters of the alphabet, into a form of communication understandable by those who are: ( 1.) not ASL conversant, generally (and 2.) not able to observe the use of ASL for the occasion of conversing with someone who signs. By this transformation, a visually impaired individual, for example, might then have the opportunity to communicate with an individual who is speech impaired.

No laments, about the work we placed to the side for now, because this project has it's own great merits and challenges. Given the efforts that were already invested in what we thought was going to be our project, turning on a dime, as it were, speaks (at least) to understanding and flexability and those are great assets in collaborative endeavors. Thanks to the group !

Take a look at the Michigan State University - American Sign Language Browser (below) for observtions of how ASL is physically signed.

American Sign Language Browser


Hand position shown for fingerspelling of letter A.

Hand position shown for fingerspelling of letter Z.

November 17th :my Dad's Birthday!


One thing is certain, none of us in the group would have anticipated the hours needed to bring the "Spoken Hand" even to this point. We have been able so far to make audible, 21 letters of the ASL alphabet, leaving J,P,Q,V and Z for future development. The necessity of incorporating accelerometers for the letters J and Z ; stretch sensors for P and Q and another dedicated circuit, of a yet to be determined form, for V, required more time than we had and so we proceeded with our 21 letter grouping.


Sign Language, in a literal sense, could be thought of as the earliest practice of “digital” communications. The anatomical definition referring to fingers as digits is indicative of the means by which this communication, in part, takes place. The graphical depiction of words, phrases, concepts and letters is accomplished manually and forms the basis of “speaking” in the Signing community.

Our project termed “The Spoken Hand” starts with the manual American Sign Language Alphabet (below) and literally digitizes it. The result we sought and have been able to demonstrate is a transduction to an Audible form of the ASL Alphabet. Beyond graphic to text (another form of graphic) the transduction is from digitized manual graphic to sound. This was the best place to start because to one extent or another, an understanding of the alphabet, regardless of its form, is common to most people. Conceptually this transduction/translation facilitates an ability of independent communication and instruction. Members of the Signing community would now be understood by individuals who are visually impaired and by those who cognitively might not comprehend ASL. The physical/cognitive situations for each of these groups prior to “Spoken Hand” effectively left them exclusive of each other. This exclusivity cannot be seen as anything other than a loss for all of the individuals involved. Once what relegates a person to a group is removed or remediated, the individuals within the groups become the focus. What they may contribute to each other is of far more importance than the means by which they make that contribution. That it is made possible by a translation of either Digitized Speech-to-Text or Sign for those who find fluid manual Signing not practical or as in our project’s case, Sign-to-Speech, is only a means to have communication take place.

History of ASL

In the United States, as in most of the world, hearing families with deaf children often employ ad-hoc home sign for simple communications. Today, though, many ASL classes are offered in secondary and postsecondary schools. ASL is a language distinct from spoken English—replete with its own syntax and grammar and supporting its own culture. The origin of modern ASL is ultimately tied to the confluence of many events and circumstances, including historical attempts at deaf education; possibly the sign used by the indigenous nations of North America; the unique situation present on a small island in Massachusetts; the attempts of a father to enlist a local minister to help educate his deaf daughter; and in no small part the ingenuity and genius of people (in this case deaf people) for language itself.

Standardized sign languages have been used in Italy since the 17th century and in France since the 18th century for the instruction of the deaf. Old French Sign Language was developed and used in Paris by the Abbé de l'Epée in his school for the deaf. These languages were always modeled after the natural sign languages already in use by the deaf cultures in their area of origin, often with additions to show aspects of the grammar of the local spoken languages.

American Plains Indians used Plains Indian Sign Language as an interlanguage for communication between people/tribes not sharing a common spoken language; its influence on ASL, if any, is unknown.

Off the coast of Massachusetts, on the island of Martha's Vineyard in the 18th century, the population had a much higher rate of deafness than the general population of the continental United States because of the founder effect and the island's isolation. Martha's Vineyard Sign Language was well known by almost all islanders since so many families had deaf members. It afforded almost everyone with the opportunity to have frequent contact with ASL while at an age most conducive to effortlessly learning a language.

Congregationalist minister and deaf educator Thomas Hopkins Gallaudet is credited with popularizing the signing technique in North America. At the behest of a father who was interested in educating his deaf daughter, Alice Cogswell, he was enlisted to investigate the methods of teaching the deaf. In the early 1800s he visited the Abbé de l'Epée's school in Paris and convinced one of the teachers, Laurent Clerc, to return with him to America. In 1817 they founded the American Asylum for the Deaf and Dumb (now the American School for the Deaf), in Hartford, Connecticut, to teach sign language to American deaf students.

It was at this school that all these influences would intermingle, interact and what would become ASL was born. Many of the school's students were from Martha's Vineyard, and they mixed their "native" sign language with Clerc's OFSL. Other students probably brought their own highly localized sign language or "home sign" systems to the mix. Undoubtedly, spontaneous lexicon developed at the school as well. If there was any influence from sign language of indigenous people, it may have been here that it was absorbed into the language.

Interestingly, because of the early influence of the sign language of France upon the school, the vocabularies of the modern sign languages in North America and France are approximately 60% shared whereas the vocabularies of ASL and British Sign Language are almost completely dissimilar.

From its synthesis at this first public school for the deaf in North America, the language went on to grow. Many of the graduates of this school went on to found schools of their own in many other states thus spreading the methods of Gallaudet and Clerc and serving to expand and standardize the language; as with most languages though, there are regional variations.

After being strongly established in this country there was a bitter fight between those who supported oralism over manualism in the late 1800s. Many notable individuals of high standing contributed to this row, such as Alexander Graham Bell. The oralists won many battles and for a long time the use of sign was suppressed, socially and pedagogically. Many considered sign to not even be a language at all. This situation was changed by William Stokoe a professor of English hired at Gallaudet University in 1955. He immediately became fascinated by ASL and began serious study of it. Eventually, through publication in linguistics journals of articles containing detailed linguistic analysis of ASL, he was able to convince the scientific mainstream that ASL was indeed a natural language on a par with any other.

The language continues to grow and change like any living language. Currently, as with spoken English, ASL constantly adds new signs in an attempt to keep up with constantly changing technology.

Material from Wikipedia - search ASL

A Brief History Of The American Asylum, At Hartford, For The Education And Instruction Of The Deaf And Dumb
Creator: n/a

DATE: 1893

American Sign Language (ASL) is the dominant sign language in United States, English-speaking Canada, and parts of Mexico. Although the United Kingdom and the United States share English as a spoken language, British Sign Language (BSL) is a different language from ASL, and not mutually intelligible.


American Sign Language (ASL) Signed in: United States, Canada and Mexico

Region: Anglophone North America


ASL is also used (sometimes alongside indigenous sign languages) in the Philippines, Singapore, Hong Kong, Dominican Republic, Haiti, Puerto Rico, Côte d'Ivoire, Burkina Faso, Ghana, Togo, Benin, Nigeria, Chad, Gabon, Democratic Republic of the Congo, Central African Republic, Mauritania, Kenya, Madagascar, and Zimbabwe. As with other sign languages, its grammar and syntax are distinct from the spoken language(s) in its area of influence. There has been no reliable survey of the number of people who use ASL as their primary language; estimates range from 200,000 to 2 million


Creole emerging from Old French Sign Language, Martha's Vineyard Sign Language and home sign

Hardware , Software, Handwear, etc.:
Pictures of the setups:

The first version ofthe glove proved to be too difficult to remove from your hand . It worked but really only with flex sensors. We neded to find a more sturdy glove that could have a fair amount of fabrication done upon it and that could remain fairly flexible.

New Version in stages of fabrication. Fitted with contact switches and flex sensors.

Glove with new resistor array to allow more discrete spacing between hand positioning to correct letter response

Leads from glove sensors and contact switches to PIC and MIDI out to USB 2x2 Midiman and into Max

First glove with MIDI to Max

in to
flex sensor

Here is a screen shot from the freeware audio editing program named: Audacity, of the Alphabet Letter Sound file I recorded and edited for use with our "Spoken Hand" -sensor glove mapping information. The sounds of the letters have been time adjusted to occur at 2 second intervals. This negates the need for an Excel spreadsheet listing of the individual sound files. It is saved as a .wav file in Audacity and will be converted to an aiff file for our uses.

User Tests

Using the glove has a learning curve but with a little work users are able to correctly form letters.

Some users depending upon their hand size have an easier time forming letters.
This glove is not a one size fits all but that might be helpful if an on the fly calibration of individual bendings of flex sensors, etc. could be made.

Only took

User Tests Responses:

USER QUESTIONAIRE for the Spoken Hand

Name : C. High School Senior


Are you familiar with American Sign Language (ASL)? - No

Have you ever had occasion to observe ASL before? -Yes

Have you ever had occasion to use ASL before? - No

Are you familiar with the ASL alphabet? - No


Please rate the ease of use in forming letters of the ASL alphabet


How would you rate the feel of the glove?

Difficult to manage--------------------------Easy to manage


Comments: After a while the glove felt uncomfortable.

Thank You from The Spoken Hand PComp Group for your participation!

USER QUESTIONAIRE for the Spoken Hand

Name : D. - Special Ed. Teacher


Are you familiar with American Sign Language (ASL)? - Yes

Have you ever had occasion to observe ASL before? -Yes

Have you ever had occasion to use ASL before? - Yes

Are you familiar with the ASL alphabet? - Yes


Please rate the ease of use in forming letters of the ASL alphabet


How would you rate the feel of the glove?

Difficult to manage--------------------------Easy to manage


Comments: I found the glove to be a bit loose on my hand and also found that at times the material would stick making it difficult to maneuver into the next letter. I feel that it is a bit too bulky with all the wires but an ingenious project which just needs a little tweaking here and there.

Thank You from The Spoken Hand PComp Group for your participation!

USER QUESTIONAIRE for the Spoken Hand

Name: F. - Kindergartener


Are you familiar with American Sign Language (ASL)? –A little

Have you ever had occasion to observe ASL before? - No

Have you ever had occasion to use ASL before? - No

Are you familiar with the ASL alphabet? - No


Please rate the ease of use in forming letters of the ASL alphabet


How would you rate the feel of the glove?

Difficult to manage--------------------------Easy to manage


Comments: It was hard to form the letters and the glove felt weird. My hand is too small.

Thank You from The Spoken Hand PComp Group for your participation!

USER QUESTIONAIRE for the Spoken Hand


This project has been a wonderful challenge. Our group put a great amount of energy into this endeavor and the result, I believe, has been geater than we had tought it might be. We will continue to develop a more sleek and responsive version making it wireless, enabling a freer user experience. Our ultimate test will be to have an individual who signs as a primary communication find our work useful. The idea of having this device become a usable communication and teaching tool will be better realized under fire, as it were, with a more high powered work-out. Until then there is more than enough work to be done on this project to refine it.

My thanks to our group members. I was greatly privleged to work with them.

Thanks also to our instructor Andrew Milmoe. Had he not pointed out(more than once)the importance of the user and not technology being the center of the idea, I'm sure we'd be off to some spiffy yet less helpful corner by now.

For Additional Info about the different areas of hard work in the Projects, please check out Our Group's Sites, They all like it so, when company comes a callin' :

Terence's Site

Langdon's Site

Adam's Site


Post a Comment

<< Home