Multi-Touch State of Mind

Gesture Learning Research Studies

2012-02-13T14:37:00.004-07:00

Fraser Anderson, and I are looking for participants for our (2) research studies. Each experiment is designed to evaluate how people learn pen-based gestures and apply touchscreen gestures in different contexts. The results of the experiments will help us understand how generalizable gestures are, how people perform gestures in different contexts, and will help guide the design of gesture-based interfaces in the future. You can volunteer for one or both of our studies.

We are currently seeking right handed University of Alberta students who are 18+ to participate. Both studies will take place in the Advanced Man-Machine Interface Lab in the Computing Science Center on the University of Alberta Campus. Your participation will be split over two days. On day 1, you spend 1 hour learning a number of gestures. On day 2 (approximately 24 hours later), you will return for some follow-up tasks. Participation on day 2 takes less than 15 minutes. At the completion of the experiment, you will receive $15 cash.

To be eligible to participate, you must be available for 1 full hour on day one and 15 minutes on day two.

If you are interested in participating, please email hci@ualberta.ca to set up an appointment. Thanks!

** These studies are being conducted by Michelle Annett and Fraser Anderson under the guidance of Dr. Walter Bischof and have been approved by the Research Ethics Board at the University of Alberta. **


Michelle's Experiment	Fraser's Experiment

Vinylmation tumblr. Not school related, but full of fun pictures!

2012-02-09T18:46:00.001-07:00

So when I'm not doing research, I collect Disney figures called Vinylmation. I usually take a lot of pictures of my figures and post them on Twitter, but I figured that it was time to put them someplace that is much easier to access (and doesn't contain all of my @getglue updates or other random retweets). So, A Vinylmation State of Mind was born (yes I know, I like A ____ State of Mind for some reason). Anyhow, if you're interested, hop on over and see my (no-so-great iPhone) photography!

A blast from the past: Resources for Story Structure, Dramatic Elements, and Student-Centered Learning

2012-01-04T14:00:00.002-07:00

Back in 2007, I was fortunate enough to be part of the a Canadian Distributed Mentor Project (CDMP) which is now known as the Collaborative Research Experiences for Undergraduates - Canada (CREUC)). The CDMP is a program that encourages undergraduate women in Computer Science and Computer Engineering to go to graduate school. It matches female students who have finished their 2nd or 3rd year of undergraduate studies with female professors for a summer of research and mentoring. Participating in the program was a really great experience for me, as I got to work with Dr. Eleni Stroulia and on wiEGO,a java based applet that interacts with an open source content management system, Moodle, and a wiki, to assist Junior High students with their group projects. wiEGO supports the inter-play linguistic and spatial-visual intelligences held by the collaborating learners though the of a visualization toolkit. A report that I wrote up about the project is available here.

A few days ago, I was contacted by Audrey Plasse, a teacher in Vermont with the Green Mountain Central School District, whose student's had stumbled a list of resources that I used during the project. They found a new link about Freidman's Pyramid that they (and I) find to be very informative and thought that it should be added to my resources page. I wish that I found it when I was working on the project! Although I no longer have access to the original webpage, I wanted to repost the resources page here so that others could find it and I can easily add to it.

Story Structure & Dramatic Elements:

Collaboration, Student-Centered Learning & Wiki Links:

Medusa (aka the secret project from my internship)

2011-10-19T22:05:00.004-06:00

So now that UIST 2011 is officially over, I can post about the work that I did at Autodesk Research in January of 2010! I'm super excited to finally get to talk about all my hard work and the awesome project (Medusa) that I got to work on. I'll post more on my work on the coming days (including some behind-the-scenes pictures of a few of the technical tricks we used), but for now, I just want to share the video and a brief description of what I did (it was 4 months of work after all!)

Quick summary:

So in short, Medusa is a Microsoft Surface that has been instrumented with 138 proximity sensors (sort of like a 1 pixel Microsoft Kinect). These proximity sensors enable the Surface to sense users as they move around the tabletop, and detect a user's hands and arms above the display area of the Surface. Not only are these sensors inexpensive and simple to configure, but also they enable an integrated hardware solution, without requiring any markers, cameras, or other sensing devices external to the display platform itself.

Medusa, a proximity-aware multi-touch tabletop.

As Medusa has an awareness of users' locations, it can for example, identify touch points by user, and disambiguate between touches made with left or right hands. It can also make use of the touch-based information provided from the Surface to map touch points to specific users (as well as identify which hand they used, right or left), even in multi-user scenarios.

Me, waving my arms above Medusa. The blue oval indicates the sensed location of my body, the orange circle and cone indicate the detection and location of my right arm, and the purple circles and cone indicate the detection and location of my left arm.

Using all of this information, there are an infinite number of ways that multi-touch interaction with a horizontal display can be enhanced and augmented. In the video below, I (along with Tovi Grossman), demonstrate a few of the techniques that we explored.

Please pardon the construction!

2011-09-27T17:33:00.002-06:00

I'm gonna start off by saying that I love the new blogger Dynamic View templates! With next to zero work on my part, my blog now can be viewed/rendered in seven really cool ways (you are currently looking at the 'Magazine View'). If you don't really dig the magazine view, and want to see something else, you can use the drop down box in the upper left corner to see a different view (it all feels so much like Flipboard - which I really think should have a Mac app!!).
Needless to say, with the new template(s) there are things that will disappear, reappear, and need some work (e.g., all my posts need to have pictures now so that they are super exciting and there are cool things to look at in the different views). The next week or so will probably be spent updating old posts (to look cooler), figuring out how to make my static pages not look so dull, and writing up a new paragraphs or so on my hobby, Vinylmations!

Switching gears ...

2011-09-22T16:32:00.000-06:00

It's been a long, awesome summer but I am happy to get back in the research groove. Ever since I completed my internship at Autodesk Research, I have been trying to narrow down the focus of my PhD research (because improving client enjoyment and motivation is still a huge area!!). After lots of reading, contemplating, and staring at the ceiling, I began to realize I was really interested in the basics of human movement: how people were moving their arms and hands on our multi-touch tabletop, why they were moving, why they weren't moving, and how some movements were very similar from activity to activity, while others were completely different.

Is this a two finger gesture? Three finger? 2.5?

These questions lead me towards thinking about the core principles behind horizontal-based gestures, multi-finger interaction, and multi-touch interfaces, as well as motor learning, skill acquisition, and skill transfer. With all these great questions swirling around in my head, I have decided to switch gears and focus my forthcoming research on a number of the unanswered questions that exist with multi-touch gestures and look at them through a motor learning-inspired magnifying glass. Until next time!

(Yes I have also started watching Phineas and Ferb).

More multi-touch in the news!

2011-07-26T09:45:00.002-06:00

I am super excited that the tabletop I built for the Glenrose was featured in an article in the Wall Street Journal today (as well as a few other places)! It's great to not only see the impact that the table has had on the Glenrose, but also see all the great research into technology-based rehabilitation interventions that is being conducted at other institutions. I hope that all this publicity will encourage more HCI researchers to think about entering into this awesome field and providing clients in therapy programs with some exciting, engaging technology to work with!

"Playing on a tablet as therapy". Wall Street Journal, July 2011.
"Smartphones, Tablets Provide Therapy for Autism, Other Disabilities". Mobiledia.com, July 2011. Also appeared on Forbes.com, July 2011.

We need participants for our user study!! [Updated]

2011-07-17T11:51:00.003-06:00

Update: Due to the overwhelming number of responses we have received, we are no longer looking for participants! Thank you very much to all of those who volunteered for our study!

Dr. Walter Bischof, Fraser Anderson, and I have just started running a user study and are looking for people in the Edmonton area to participate! In our study, you will get to wear a super nifty motion capture jacket, have your muscular activity recorded by EMG electrodes, and use one of the AIRTouch multi-touch tabletops that Fraser and I have built. We are using all of these technologies to compare different rehabilitation activities: those performed on a multi-touch tabletop and those performed on a traditional table. The information collected during our study will be used to evaluate the potential of multi-touch tabletops in rehabilitation programs.

Me wearing the motion capture jacket and interacting with the multi-touch tabletop. The grey wires come from the EMG electrodes.

A front view of the motion capture jacket. It's not skin tight or made of spandex.

We are looking for people who are 18 years of age or older who weigh less than 220 lbs (due to the sizes of motion capture jackets that we have). The study will take place in the AMMI Lab (Department of Computing Science, University of Alberta) and will take approximately one hour. Participants will receive $15 CAD for participating in the study and an additional $5 CAD if all tasks in the study are completed. The study is running from July 18th, 2011 to August 5th, 2011.

For more information and/or to schedule a convenient time to participate, please contact our research assistant, Gauri Chaggar, at chaggar1@ualberta.ca.

Convert System.Windows.Media.Brush to System.Windows.Media.Color and vice versa

2011-03-09T22:21:00.014-07:00

I am working on a project that uses C# and WPF and I have been having one hella time on one small technical problem: converting between different classes of colors. All I needed to do was convert from a System.Windows.Media.Brush to System.Windows.Media.Color in one part of my code and then convert from my System.Windows.Media.Color back to a System.Windows.Media.Brush in another part of the code. Needless to say, it was easy to go from Color to Brush but not from Brush to Color (I searched high and low using Google but had zero luck - hence my post on the topic!).

For those out there who need it:
Color -> Brush
Color newColor = Colors.Blue;
Brush imageColor = new SolidColorBrush(newColor);

Brush -> Color
Brush newColor = Brushes.Blue;
SolidColorBrush newBrush = (SolidColorBrush)newColor;
Color imageColor = newBrush;

From: http://angels-overcome.tumblr.com/

Also, a ScatterView does not like to scale Canvas objects.
-M

DIY-ing, Sparkfun, and Adafruit

2010-10-25T20:15:00.008-06:00

After meeting with a number of fellow grad students who are working on course projects for Walter's CMPUT 607 - Introduction to Human Computer Interaction course, I realized that I have been recommending a number of textbooks and manuals to people starting in HCI/with DIY-projects, but I don't have a central place where people can view them. As I order most of my books from Amazon or Chapters (whenever they have them!), I have added a 'My Recommendations' widget to my blog. Currently, the recommendations are all based around books that I use all the time (on Arduino's (my new 'it' thing) and interaction), but I will switch out books whenever I discover new must-have's!

One of the DIY, Arduino-based projects that Fraser and I worked on with a WISEST intern.

If you are starting with the Arduino and electronics for the FIRST time, I highly recommend Banzi's Getting Started with Arduino book. I also recommend Noble's Programming Interactivity: A Designer's Guide to Processing, Arduino, and Openframeworks for anyone new to human computer interaction, motion tracking, basic computer vision, Arduino, processing, etc. Both of these books provide many tutorials, code examples, and pictures. They also are written for beginners and people who aren't technical, who maybe don't know how to code, or who don't know what a sensor is. I found the approaches taken in these books to be very helpful when you are first learning.

As I am giving a shout-out to the books that I love to use for my projects, I also think it's important to share the links to the various sellers/websites that I order components, Lilypad parts, and Arduino boards from. All of the websites below ship to Canada!

Sparkfun: I can't even tell you what I would do if Sparkfun didn't exist. They sell everything Lilypad and Arduino that you could ever need, have a great list of user-submitted projects, code examples, and tutorials, and have reasonable shipping to Canada.
Adafruit Industries: I just found Adafruit two weeks ago and started following them on twitter, but they have a very unique set of products available and I already received my first shipment of a very exciting product I will post about later. Really great Twitter feed and jobs section that just started up.
Sparkle Labs: I found this place three weeks ago from a blog post on Gizmodo about Gizmodo University. Aside from their very good DIY Electronics Kit, they sell some cool parts such as pink LEDs, rainbow LEDs, and a solar cell. They also have very quick shipping to Canada.
RobotShop: A Canadian company that sells actual robots, as well as a number of sensors and actuators, including some really weird things such as a Flame Sensor! Lots of breakout boards for Arduino.
Phidgets Inc.: An Alberta-based company that sells Phidgets, which are like much more precise, high-tech Arduino's. Phidget's sample at a much faster rate than Arduino's, making them perfect for research projects. They sell a number of I/O boards that you can use with virtually any sensor, as well as a number of sensors, motors, switches, LEDs, etc.
Coolight.com: My resource for all things Electro-luminescent wire. EL Wire is cheap and looks super awesome. My favorite example of EL wire are the Daft Punk EL Wire costumes. They have good customer service and reasonable shipping to Canada.
Maker SHED: Although I haven't ordered from Maker SHED in a while (mostly because Sparkfun/Adafruit carry many of the same things), this website is a great resource for books, kits, Make Magazine, and those awesome little BristleBots!

Happy DIY-ing!

Tabletop Research in the News

2010-10-25T19:41:00.006-06:00

On October 20 and 22, the Glenrose held pseudo-press conferences to share the success of my multi-touch tabletop and it's activities with the local Edmonton and St. Albert media. To date between 40 and 50 patients have used the tabletop, and many patients and therapists reported that it has produced many great results. I hope that these positive benefits will continue with new patients and that we will be able to make a few more tabletops for the Glenrose. I also hope that we can get many more tabletops into other hospitals and clinics around Alberta as well. If you would like more information on this project, please don't hesitate to contact me!

Media links:

Interactive table top therapy - grad student speeds up rehabilitation process. 24 Hours Edmonton (front page)
Tabling the motion. Alberta Health Services Website
New Machine Helps Patients in Rehabilitation. CityNews Edmonton
New tool helping patients undergoing rehab. CTV Edmonton (also on 6:00 newscast)
New Rehab tool at clinic. Edmonton Examiner
iPad-like technology aids patients' rehab. Edmonton Sun (video included)
Techno-table gets patients moving. St. Albert Gazette

My lack of updates

2010-10-25T19:17:00.004-06:00

This is what I feel like some days (well, except for the purple hair!)

Over the last few months I have been very busy with supervising interns, getting my tabletop ready for the Glenrose, working on course assignments, etc. I regret that I have not been able to post more because the point of moving my website over to a blog format was so that I would be able to update it more frequently. Starting today, I hope to update it weekly because there are many many exciting things happening! For the next few posts, I am going to try and capture all of the projects and events that have been happening over the last few months.

-m

Internet, Windows 7, Arduinos, Interns

2010-06-30T23:30:00.000-06:00

So last week, I found out that someone hacked into the department's system and as a result, it was decided that everything should be shut down. This meant that Internet access, webpages, ssh'ing, and basically everything was inaccessible. After finally getting access to my files, I decided to pull everything off of cs.ualber.... and instead move it to some other webspace that I have access to. Both cs.ualberta.ca/~mkannett and ualberta.ca/~mkannett will redirect to here so there should be no problems, except with hyper-linked resources. In the process of moving things over, I tried to update as many links as possible, but it is likely that I missed some. If you can't access something, please send me an email (if possible) and alert me so that I can fix them and prevent others from encountering the same problem! I would greatly appreciate it.

This week I have also been preparing for the five interns that will be starting next week. Fraser Anderson and I have been working very hard to get tutorials, equipment, computers, and projects ready for our new lab mates! After everything gets settled down, I will be updating the blog more frequently with information on what I have been doing for the last few months, as well as what I will be doing in the future.

If you haven't seen Toy Story 3, do it! Go Pixar!

Happy Eclipse Release day!!

This week in the news....UBC visitors and scholarships!

2010-06-01T17:55:00.004-06:00

Hello! It is only Tuesday, but it has already been a very busy week. Yesterday and today, the AMMI lab was fortunately enough to have Dr. Alan Kingstone, and two of his post-docs Tom Foulsham and Evan Risko visit our lab, share their research with us, and have a great dinner at Walter's house. I had a great time hearing and watching videos of the projects that Dr. Kingstone's lab has/is undertaking and appreciated all of their comments and ideas about my (potential) work.

Also, I found out that I was awarded an iCORE PhD Graduate Student Scholarship in Information and Communication Technologies (it's a long title, i know) to help me pursue my rehabilitation-based research. I am really excited about this because this means that people outside of my lab actually believe in the potential of my research projects and ideas. A real motivation booster that's for sure!!!

Killam Funding

2010-05-13T14:47:00.002-06:00

Greetings! I have been on vacation in the United Kingdom for two weeks (I had a stellar time). While in the UK, and more specifically in Lancaster, England, I found out that I am now a Killam Scholar! What this means for me is that I have been fortunate enough to receive funding from the Killam Trusts (University of Alberta Izaak Walton Killam Scholarship) for my PhD research and education. I am very thankful that I received this award / honor!!

New CP & CT Papers

2010-02-17T23:26:00.003-07:00

Gyeongbok Palace - Seoul Korea

I just found out that I have two papers (along with Fraser Anderson) that were accepted at CyberTherapy and CyberPsychology 15 to be held in Seoul, Korea in June (during the World Cup of Soccer)! I am very happy that I get to go back to CT & CP again this year, because I had a great time last year when it was held in Italy. As soon as I am able, I will post pictures, videos, and our accepted papers!

AIRWall

2010-02-08T18:28:00.004-07:00

Inspired by the success of our AIRTouch project, Fraser and I wanted to adapt the multi-touch activities that I created for the AIRTouch system to a much larger surface. We used our large rear-projected "Disney" screen, a Wiimote, an off-the-shelf NEC projector, an inexpensive Bluetooth dongle (Deal Extreme is awesome), and a custom IR 'light pen' to make our AIRWall system. We initially used Johnny Chung Lee's Wiimote software (he is the nicest guy) to handle the tracking of the IR 'light pen', and then later built our own.

Fraser's IR Light Pen:

A few of the activities that we tested on the AIRWall can be seen here:

AIR Touch + Publication!

2010-02-08T17:22:00.001-07:00

After much hard work and valuable input from a number of my therapist and research colleagues, the first version of the AIRTouch software was completed in September 2009. So far, we have 12 different tasks that are fully customizable and meet a variety of patient needs. Each of the activities was created using Adobe Flex (which I personally think is awesome) and although they are rather simple graphically, they are very fun to interact with.

The Match Me, memory-esque activity.

Touch Tracing (aka picture tracing)

Touch Puzzles

Finger Painting!

Foggy Windows (aka wash the window)

Pop the Balloons (aka dynmaic target selection)

In November 2009, Fraser and I were lucky enough to travel to Australia and give a conference presentation about our preliminary work at OzCHI 2009. You can read our conference publication here or email me for a copy. The publication was co-authored by D. Goertzen, J. Halton, Q. Ranson, W.F. Bischof and P. Boulanger.

A few of the upper-extremity-based activities are shown here:

Version two of the software is finished but I am waiting until I deploy them at the Glenrose before I begin writing about it and taking videos with my new Flip HD!

Multi-Touch TAR

2010-02-08T17:10:00.002-07:00

After I finished my MSc degree in January, I spent the next few months working on a number of projects in the AMMI lab. One of the most fruitful projects was the design and development of the AIR Touch multi-touch system. Working with Fraser Anderson and a team of researchers and occupational therapists at the Glenrose Rehabilitation Hospital, the AIR Touch system is comprised of a 3 foot by 2 foot multi-touch surface (manufactured by NOR_/D), a FireFly MV IR camera, an off-the-shelf NEC projector, a mirror, the openFrameworks software package, and a suite of rehabilitation-inspired multi touch activities written in Adobe Flex. The lovely black 2x4 stand was created by Fraser and I in my garage (thanks Dad for the supplies!).

Version One of the system:

IR Camera Image (Top), Tracking Image (Bottom):

We chose to use a very open solution, as opposed to the Microsoft Surface or a SMART table, so we could modify or add things to the hardware setup as needed. We are currently in the process of designing version two of the hardware and I will post pictures when we are done!

Technology Assisted Rehabilitation

2010-02-08T12:29:00.004-07:00

For my PhD research, I have been working on investigating the limitations and potential of integrating various technologies into rehabilitation programs. I have been working with a number of occupational and physical therapists at the Glenrose Rehabilitation Hospital in Edmonton, Alberta, Canada to help increase patient enjoyment and compliance with rehabilitation activities. So far, I have been exploring Wii Peripherals, virtual reality, multi-touch surfaces, tangibles, fabric-based computing, and low range RFID and have accumulated quite a collection of devices and prototyping products.

Glenrose Rehabilitation Hospital in Edmonton, Alberta, Canada

The majority of the blog will be a documentation of my experiences with technology-assisted rehabilitation (TAR) and human computer interaction (HCI). I will try to update it as much as possible, especially with youtube videos and pictures. Any questions or comments can always be emailed to me.

I am also always looking for eager high school or undergraduate students to work with me as interns or for ideas about course projects. Please check out the WISEST and HIP programs at the University of Alberta.

Thanks for reading,
-Michelle

Alien Invasion

2010-02-08T11:30:00.008-07:00

Last week, I decided to make a new outreach activity for the WIT and WISEST conferences that I usually volunteer at. This new activity, Alien Invasion, uses the Alice programming environment (available here) to teach elementary, junior high, and senior high school students about basic programming concepts such as loops, events, variables, and methods. The estimated time for the activity is 35 minutes teaching and 15+ minutes of self-exploration.

Backstory: You are the commander of an spaceship that was sent to earth. Your mission is to invade earth, capture as many cows as possible, and bring them back to your home planet. You flew around many different farms in the Edmonton area and found the perfect herd of cows to bring home but due to the warm temperatures on Earth, none of the controls on your spaceship work. You must repair your space ship’s controls so that you will be able to fly around the farm, determine when you are close enough to a cow to ‘beam’ it up to your ship, and then fly home.

I have made a quick YouTube video (below) that illustrates how to teach/complete the activity. The three sections at the end (Adding Animations, Changing World/Object Properties, Adding Additional 3D Objects) are optional, and can be integrated into a lesson if time permits.

If you want to use the activity, or have more questions about it, please email me. I am always happy to work with others who have an interest in STEM! Special thanks to Fraser Anderson for helping with the original Virtools version of Alien Invasion.

The SNaP Framework - A Virtual Reality Tool for Spatial Navigation

2009-12-31T18:30:00.013-07:00

Continuing off of the work I did for my independent study course with Dr. Walter F. Bischof, my thesis project required me to create a software framework (the SNaP Framework - Spatial Navigation Framework) that would enable psychologists to easy create, control, and deploy virtual reality-based spatial navigation experiments. My main goal was to eliminate the hardware and usability issues inherent in current VR systems and create a flexible system that is easy for novices to use. I also had a number of research goals that I wanted to investigate/achieve:

Eliminate issues in using VR for spatial navigation research
Simplify input and output peripheral usage
Enable novice specification and deployment
Decrease design and implementation time
Reduce the volume of incomparable results
Create environments with similar appearances
Include universal interaction metaphors and behavioral recording techniques

Read my thesis

Read the 2009 CyberPsychology and Cybertherapy Paper Here

Read the 2009 SEARIS Conference Paper Here

The software framework contains a number of components (Chapter 1 and 4 of thesis):

Parameter File
User-created XML file that specifies the spatial navigation experiment to be performed
VR Configuration Creator
Python module that transforms each trial specified in a parameter file into a configuration file; determines if a block of trials needs to be run again due to poor performance
Configuration Files
XML-based file that specifies the environmental and protocol configurations for a given experimental trial
VR Launcher
Python module that determines which deployment contexts and input devices are desired
VRPN Server
Open source virtual reality server/client that transforms input devices into generic device types; the data streaming from the server is used to control participant movement
Virtools VR Player
Virtools-provided component that renders virtual environments
Virtools Composition Files
Virtools file that contains a paradigm’s virtual world; contains custom scripts, 3D models, and universal modules to generate and render a paradigm’s virtual implementation
Result Files
Capture a participant’s performance (i.e., path traveled, camera frustum images, or overall experiment results)

There are a number of ways that different groups of users can use the SNaP framework in their research. Novices can use 3D Layout window to add, relocate, or retexture 3D elements, quickly specify desired input and output peripherals, and easily deploy experiments using the provided batch scripts. Expert users can extend the XML parameter and configuration file schemas, design and integrate new 3D models, modify or add new scripting or C++ SDK code, introduce new metrics, modify the VR Configuration Creator to handle new parameter and configuration files, establish new goal monitoring algorithms, navigation metaphors, or aids, and implement new paradigms using template environment. (Chapter 5 of thesis)

To test the efficacy of the SNaP Framework, I implemented a number of popular spatial navigation paradigms using the framework (I also performed a pilot study using a small sample of individuals, those results were done after my thesis was completed so they are present in my publications but not thesis). (Chapter 3 of thesis)

Complex Maze

Bucket World

Scatter Hoarding Task

Cheng Task

Virtual Morris Water Maze

Analyzing the Relationships Between Users in Wiki Settings

2009-12-31T17:43:00.001-07:00

During the summer of 2007, when I was working on the wiEGO project, I supervised my second WISEST student, Joyce Lam. Because there was lots of work in the Software Engineering lab investigating wikis and blogs, we set Joyce up on a project that analyzed the relationships that existed between wiki users. Using the Annoki Software Engineering wiki, she modified an existing wiki visualization program to help users compare the behavior (number of wiki page edits) of any two wiki contributors. As the first picture illustrates, a user of the Java-based visualization program can select two users who have contributed to the Annoki Wiki and then view an orbit graph (Image 2). In the main application panel, you can also choose to view a one person orbit diagram, or a two person diagram. You can also zoom in and out of the graph, hide pages you don't want to see, and view a number of wiki statistics on the right hand side.

In an orbit graph, there is a central orb surrounded by a number of concentric circles. In the center orb are the two wiki contributors which were selected to be compared. In each concentric circle, there are a number of wiki pages (represented by colored dots). Each dot represents a wiki page present in the Annoki wiki. The color of each dot indicates which wiki contributor edited the page (one, the other, or both). The dots which are closest to the center orb have been edited the most, those edited the least (if at all) are the farthest away.

One can also select a specific wiki contributor and view all of the wiki pages that were edited by that contributor (via lines/edges connecting the dots). Although this project was not completed finished, both Joyce and I learned many interesting things about integrating wiki information with visualizations during her summer internship.

Invenio - Tracking Music Trends Using Web Services

2009-12-30T23:24:00.002-07:00

The Invenio project was part of the course requirements for my CMPUT 660 - Web Services course. When I took the course (Winter 2008), I was reading lots of celebrity gossip websites (such as Perez Hilton) everyday and was listening to a ton of top 40 hits in Winamp. It was around this time that Perez Hilton starting posting information about new, unsigned (or little known) artists in the USA and Europe. About once a week I would be finding out about some new person that I had never heard or, or had not yet gotten 'big' in Canada or Alberta. This started me thinking about how music starts to spread across North America and why some artists are popular everywhere, but some are entirely regional.

For my final course project, one of our requirements was to incorporate geographic information into a web service. Coupling this requirement with my thoughts on music trends, I decided to build a web service that would geovisualize radio station music. (At the time, XM and Sirius weren't as big as they are today, and this project was a few months before last.fm [i think that's right] came out with their big analytics software and angorithms). Over a period of 6 weeks (last week in January until 2nd week in March), I collected and organized the music chart information for 190 radio stations that are registered with Nelson SoundScan/Billboard Magazine.

I then took this information, and created a data-intensive, REST-based, RIA entitled, Invenio. Invenio combines a variety of different technologies (Yahoo! Maps, Amazon Associates Web Service, REST, and the Adobe Flex framework) to geographically visualize aggregated music chart information. You can watch a short-ish / long-ish video about Invenio's features below (or on Youtube here). This project was very successful for me - I got a publication accepted into Cascon 2008 (co-authored by the course instructor Dr. Eleni Stroulia). You can read the paper in it's entirely here on the ACM website or email me and I can send you a copy.

If you don't feel like watching the whole video (I know it's long), here are a couple of pictures that illustrate Invenio's main features:

In the main Invenio view, you can select the artist, song, and time period and then view the song's position on each of the 190 radio stations in the US and Canada over this time period. Each circle represents one radio station (or song, or artist, depending upon the view). In some views (Track By Artist), the size of the circle indicates the song or artist's chart position, in other views the color of the circle indicates the genre of music (Track By Success). Pictured above is the Track by Success view, whereby one can view the top of bottom genre that was on year radio station during the selected time period. If one chooses to 'View All Weeks', then they will see a geographic time-lapsed animation of the options they have chosen (e.g., How Alicia Key's song 'No One', fared on the charts for the 6 week period).

In this second picture, instead of circles to indicate song position or genre, we are shown the artist's album cover (pulled from Amazon) that contains the song that is currently on the chart [in this case in the #1 position]). The main window's maps are fully interactive - you can zoom in and out, pan the map, and change it's type (e.g., satellite, hybrid, map). You can also elect to have tool tips appear (that provide additional information about the radio station and link to the radio station's website). Also, you can provide other additional information which is pulled from the Amazon web site (i.e., album price, a link to the album's page, a review of the album, the number of lists that the album is on, the genre of music, etc.).

Another type of visualization that is available in Invenio are the Cover Flows or Display Shelves (when I was making Invenio, Apple hadn't popularized them yet). Each radio station has six display shelves associated to it, and each display shelf visualizes the music that was on each week's chart. One can select a radio station using the combo box (or by clicking on a circle or artist album cover in the main application window) and all of the display shelves will appear. Once they appear, you can choose to flip through each of them individually, or 'lock' them according to chart position or song. This alternative view makes it easy to see how a song has fared using a method that is very different to the main application's map. Similar to the main application window, you can choose to see tool tips and additional artist/album information from Amazon.

The last visualizations that are contained within Invenio are the charting views. Because most people are used to viewing information via charts rather than display shelves or maps, I chose to include three different charting options in Invenio. There are Bubble Charts (the picture directly below this one; the size of each bubble indicates the position of that song on the corresponding radio station's chart), a line chart indicating a chart score (the picture in the middle; the average position a song across all radio stations), and a vertical chart (the picture at the bottom; it indicates if each song/artist has been fairly consistent over time or if the have had a large variance in chart position over the given time period).

AViz - Visualizing Execution Traces for the Dynamic Analysis of Software

2009-12-30T19:37:00.003-07:00

For my CMPUT 666 - Reverse Software Engineering course (Fall 2007), I worked on an individual project relating to the visualization of execution traces for dynamic analysis of a software program.

As many software engineers will attest, one of the most important and time-consuming activities within the software development cycle is the continual maintenance of a software system or program. Contrary to popular belief, roughly 50% of the costs encountered during a typical software development cycle are incurred during the modification and maintenance phases of a system, not during the design or implementation activities. Estimates have shown that 50% of the software maintenance phase is spent trying to comprehend a software system. Because program comprehension contributes so much time, effort and money to the total cost of system development, a logical question to ask is: what are the problems with the methods, tools, and techniques that are being used, and how can they be improved upon?

Although each has their own problems, static and dynamic analyses are very beneficial for discovering the behavior and architecture of a system. One of the most surprising concerns that arises after reading static and dynamic analysis literature is that both of these techniques have not been combined together very often. A hybrid of these approaches would involve aggregating the static and dynamic artifacts together, and creating a new visualization that represents the system’s architecture and behavior. Of the limited research which has tried to create a ‘hybrid’ analysis, the most common visualizations that are created are UML collaboration diagrams (a hybrid UML class and sequence diagram):

Within the context of my final project, I wanted to explore the possibilities of using this hybrid approach to assist maintenance personnel in program comprehension when they are performing an evolutionary maintenance tasks. Motivated by the success of a previously created software program at the University of Alberta (JDEvAn Viewer), I wanted to extend this work to include a dynamic analysis element. To this end, I experimented with the creation of a hybrid analysis system, AViz (Aspect Visualization). I utilized the typical reverse engineering paradigm to extract execution traces from my program (using the AspectJ language and writing to files), analyzed the traces (in Java) and then output the results using a UML collaboration diagram (using the Java SWT and Draw2D libraries).

Thanks to Zhenchang Xing (for the JDEvAn Viewer code) and Mike Smit (for his AspectJ sample code to get me started) at the UofA.

Enulog - Determining and Visualizing the Polarity of Movie Reviews Using Sentiment Analysis

2009-12-30T19:13:00.002-07:00

As part of my CMPUT 500 - Introduction to Natural Language Processing course (Fall 2007), I was required to do an individual course project. Because I was tied to the Software Engineering lab at the University of Alberta, I decided to integrate some NLP into an existing visualization project (eNulog) that was created by the Software Engineering lab. (Some of the following is from my Canadian AI paper which can be found here in its entirety or you can email me and I will send it to you).

The eNulog project aimed to aggregate and visualize RSS feeds and movie blog postings into a simple, easy to use interface. Within the eNulog interface, each movie, actor, director, or movie genre is represented by a node. A user can click on a node and view all of blogs, comments, or RSS feeds that relate to the given movie (or actor, director, or movie genre). The size of each node relates to the number of posts, comments, or feeds about that movie. Once a node has been clicked/selected, all of the nodes which are similar to it will aggregate around it; those nodes which are dissimilar (e.g., the comedy genre is vastly different than the action genre) will move farther away.

For my project, I took the eNulog program, and mined all of the blog postings to determine their relative polarity. In the Candian AI conference paper that I wrote along with Dr. Greg Kondrak, we compared two different sentiment analysis techniques (a lexical/dictionary-based approach and a machine learning/support vector machine approach) using the eNulog blog data set. The stronger approach (using support vector machines via SVMLite) was incorporated into the eNulog visualization program. The paper can be found here. The two pictures illustrate the interface: green nodes indicating that the average movie reviews for the movie in question were positive, red indicating that they were negative, and yellow indicating that they were neutral or split 50-50 (or somewhat close to even). The first picture shows the movie 'Batman Begins' has 14 total blog postings, that generally recommend the movie. In the second picture, the movie 'Charlie and the Chocolate Factory' has been selected/clicked. All of the movies similar to it have moved closer to it, and those that are dissimilar are father away.

Exploring Maya and Virtools, Underwater World

2009-12-30T17:34:00.005-07:00

In the summer of 2008, Fraser Anderson and I supervised a WISEST student, Melissa Hall, in the AMMI lab. We decided to create a project for Melissa that would make use of her amazing artistic skills, whilst not requiring a huge amount of programming. As I was just finished the Caribbean Resort Project, we asked her to make us a large, detailed virtual environment that could be used for some of the spatial navigation research that I was working on for my thesis.

At the end of her 6 week internship, Melissa created a wonderful underwater environment, a la Finding Nemo / Little Mermaid. She used Maya 2008 for 3D object creation and Virtools 4.1 for character behaviors and environment deployment. The environment has a number of character behaviors and contains a vast array of ocean-esque elements such as tankers, a sunken ship, kelp, shells, fish, sharks, etc. The 'Q' and 'E' keys can be used to swim upwards and downwards, the 'W', 'A', 'S'. and 'D' keys are used to swim forwards, backwards, left, and right, and the arrow keys are used to look around. The Spacebar can also be used to turn a flashlight on and off.

We commonly show visitors to the AMMI lab her environment in our CAVE or on a monitor or HMD. If you have been to the AMMI lab in the few years for a demo, then you have most certainly seen her virtual environment! For those of you who haven't seen the underwater world, you can see it on YouTube here, or watch below.

Exploring Maya and Virtools, Caribbean Resort Project

2009-12-29T18:53:00.009-07:00

In December of 2009, I went with Fraser Anderson to Punta Cana in the Dominican Republic. It was my first time out of North America and on an all-inclusive resort. Needless to say, much fun was had and the sun's rays were enjoyed very much. I'm from Canada people, we only have summer for 4 months. While at our resort, the Baha Principe, I noticed that all of the buildings and architecture was uniform and the layout of the resort would make a wonderful setting for a spatial navigation experiment (I know I am nerdy thinking about these things on vacation).Once I got back to Edmonton, the winter semester was starting (at the time I was in the MSc program) and I was scheduled to take an independent study course with Dr. Walter F. Bischof. The purpose of this course was for me to explore virtual reality software and the difficulties inherent in creating virtual environments for psychological applications.

As my final project, I worked with Maya (3D modelling software) as well Virtools (a virtual reality behavior and deployment suite). One of my final deliverables for the course was a 3-D Carribean virtual resort. I based the layout and architectural elements in the 3-D world after the resort that I had been to in the Dominican Republic. While the resort may seem like a toy or trivial environment, I designed following many of the principles that are used to design navigation and way-finding studies and also added in support to gather user metrics (which are used to facilitate understanding about human way-finding behavior).

Although the 3-D models might not be the greatest, I was quite happy with the end result. My virtual resort is rather large, with lots of Carribean-ish items and can be viewed on a monitor, HMD, or in a CAVE. As it requires the Virtools software for deployment, the pictures below gives a brief look at the resort. You can also see the 19MB video on YouTube here or below:

One of the cool things about this project was that Fraser Anderson took one of the 3D models that I created in Maya of a lawn chair and had it 3D printed for me!

wiEGO - Wiki Integrating Electronic Graphical Organizers

2009-12-29T17:26:00.002-07:00

In the summer of 2007, I was awarded a Canadian Distributed Mentor Project award and spent the summer continuing my previous work with Dr. Eleni Stroulia. wiEGO, my summer project, is a java based applet (built off of AnnokiBlooms) that interacts with an open source content management system (Moodle) and a wiki (Annoki) to assist Junior High students with their group projects. Keeping Bloom’s Taxonomy of learning objectives in mind, to enable increasingly rich levels of learning, wiEGO supports a variety of graphic organizer structures of different complexities and enables the association of a single wiki document with multiple graphic organizers. wiEGO is comprised of three main sections:

Moodle

Moodle (Modular Object-Oriented Dynamic Learning Environment) is a platform independent content based management system (CMS) that is used by educators at all levels of education (elementary, secondary and post-secondary) throughout the world. Moodle takes a “constructivist and social constructionist approach to education, emphasizing that learners (and not just teachers) can contribute to the educational experience in many ways” [8]. It is used to post course materials, administer quizzes, create assignments, blogs, message boards, and forums, and manage course content through an online e-learning website. Moodle is not only user friendly for students and teachers, but it is also developer friendly. Everything in Moodle is modular, so any new additions that need to be made, which are called ‘blocks’, are extremely easy to author and implement. All code is written in php.

To implement wiEGO, not many modifications needed to be made to our Moodle installation. I added in a ‘wiEGO Block’ that allows a user to view all of the wiki changes fellow group members have made since their last login to Moodle. A user can select a page title from the list of recent changes and they will not only be taken to this page to view the changes, but also logged into Annoki so they can edit, view or add new pages. This transparent login allows a student to only log in once, to Moodle, and not twice (once to Moodle and once to Annoki).

Annoki
Annoki (Annotated Wiki) is web-based wiki that contains a variety of wiki extensions. A wiki is a website that encourages individuals to collaborate with other via editing and commenting on ‘wiki’ pages. A wiki is an excellant piece of software to use for group project planning because it can track user behavior and allow for an anytime, anywhere collaboration between group members. When a group is writing a final report, this is very efficient because any member can edit a section of the page, and no one will have to email their section of the report to another group member. As well, a teacher can look at the history of a page and determine which group members have collaborated when, and how meaningful their contributions were.

With is in mind, Annoki is based on the very popular Mediawiki software application. Mediawiki is a LAMP system (Linux, Apache Server, mySQL, PHP) that is platform independent. In terms of the wiEGO project, the seamless combination of php and mySQL allows me to easily query the mySQL database and output XML files that can be used in the EGO program.

There are many different extensions and modifications that I added to Annoki to fulfill the wiEGO project. Finally, I needed to update Annoki from version 1.5 to 1.10, which took me roughly 2 weeks. I also added a spellchecker, pdf viewer, task pages, calendar, group access control, special pages to create new group projects (for teachers) and wiEGO extensions to seamlessly allow users to open EGO in a new browser window, to name a few. I also spent a large amount of time on the CSS of the website and changed the wiki skin (Figure 3). Some of the new functionality required me to write some JavaScript to interact with the PHP and mySQL of Mediawiki.

EGO
Because wiEGO contains an extension of AnnokiBlooms it implements a previous toolkit that one of Dr. Stroulia students had created, TOMU. TOMU is java-based applet that reads in an XML XTM file. The XML files TOMU uses are created by our wiki, Annoki. This being said, TOMU queries the database for every wiki pages’ name and associated pages and outputs them to an XML XTM file. This XML file is read by the TOMU toolkit, and TOMU uses a package called TouchGraph to render a topic map.

Keeping with the methods used in TOMU and AnnokiBlooms, EGO also reads in an XML XTM file, but unlike TOMU, EGO can also create new topic maps or edit the existing topic maps. The new graphs can take the form of spider maps, topic maps, timelines, hierarchies or flow charts. Not only are new graphs created in EGO, they are also transformed into new wiki pages, wiki sections or content, meaning that a user can change the wiki content from another source outside of the wiki. By allowing students to do this, we are providing those visual-spatial learners with an alternative method to creating projects, reports and collaborating with their peers.

AnnokiBlooms - An Annoki-Based Tool For Story Planning and Reflection

2009-12-29T15:52:00.004-07:00

In the summer of 2006, I was awarded an NSERC Undergraduate Student Research Award. For my summer research project I worked with Dr. Eleni Stroulia from the Software Services group (formerly the Software Engineering Group). My project required me to visualize a small collection of wiki pages, for example those pages that are from a single project group or a single wiki page that identifies the important sections of a page. During the summer, I also supervised a WISEST student Janice Annett (who also happens to be my sister!). She helped me with the user interface portions of this project and also assisted with the 'Teach Planner' aspect of the project (which isn't mentioned below but allows teachers to create their own graphic organizers for AnnokiBlooms).

AnnokiBlooms is a wiki-based tool, that combines Annotated Wikis (Wiki libraries such as Wikipedia and MediaWiki) with Bloom’s Taxonomy (a hierarchical framework used in education settings to classify the cognitive domain of learning) to create semantic representations of short stories. AnnokiBlooms aims to strengthen a student’s mastery of concepts and ideas they have gained from reading or writing a short story or narrative.

Through my work, I created a Java applet that utilized the University of Alberta Software Engineering wiki (Annoki) and XML files to dynamically visualize a single wiki page (we chose the domain of story writing to test this on). The idea in using story writing allowed me to investigate the multiple semantic graphs that a story could have, such as time-lines, topic maps, spider maps, flow charts and hierarchies. The bulk of my work involved creating wiki parsers that would parse a story and create multiple XML XTM files, and then transforming XTM's into different graphs (such as time-lines, spider maps, flow charts, or hierarchies), complete with different shaped and colored nodes and edges.

AnnokiBlooms is a very user-friendly tool, that allows students to easily see i) how the story they have created is semantically structured and ii) how the story they have read is structured through the use of different types of graphic organizers (time line, spider map, topic map, hierarchy, flow chart), nodes and edges. Teachers can use this tool to view a students progress and story evolution to determine which areas a student needs additional assistance in.

Some images of AnnokiBlooms:

The first version of AnnokiBlooms that I built. The Annoki Wiki and Java program were two separate entities but communicated with each other.

The second version of AnnokiBlooms, where I integrated an HTML browser into the Java program so that the graphic organizers and the wiki could be used simultaneously.

A screen shot of the 'Story Comprehension' use case of the software. In this example, the wiki page and subsequent visualization would be created by the teacher (with specific elements omitted) and the student would be asked to complete the graphic organizer by answering story-specific questions.

Fun Tag Clouds

2009-12-24T13:18:00.002-07:00

I am still continuing with the migration of my old site to my new site. I am currently working on adding information and media about the various projects I have worked on in the past, and on projects that I am currently working on. I suspect that this information will be posted in a week or so (it's difficult to find media for some of my old projects!). In the mean time, I have migrated information on my publications, scholarships, volunteering, supervision activities, and education.

Following a suggestion from Fraser Anderson, I decided to make a tag cloud about the different areas of Computing Science that I am interested in. I make a quick Google search and discovered an awesome website, Wordle, that generates them for you. Here are a few of the different tag clouds that were generated (that I absolutely loved):

Migrating to a new site

2009-12-23T22:57:00.002-07:00

We're migrating dude!

After two years of my plain, vanilla website, I have finally decided that it is time to make a new website for myself. After weighing many options, I have decided to use a blog-style format because well, I read blogs everyday and I figured that if others can easily add new media seamlessly, then this will be the perfect medium for me to use as well. Please bear with me, the migration of my old website's information to this website will take me awhile.

-m