Meeting with one of my assesors

Wow! I was a bit... I don't know the correct word... afraid? I arranged a meeting with Andrew Vande Moore to sign my PhD Plan. Would he criticize too much my PhD Plan? Ok... I know... the PhD Plan went through several iterations with Erik and Katrien, but still I was aware that the PhD was a bit open to face my next two years of PhD.

I am working in learning... and learning is affected by multiple factors. We can control some of them but maybe others we are not aware of them (or we can not control them because are externals). But sometimes I am a bit afraid... I have a really good scenario to get a good PhD. We are experimenting with real students, so I could do really nice experiments, however, I end up after every evaluations with a feeling that I could have got more valuable evaluation (It also relates to my non-sexual usual sadomasochist tendencies, I don't know why but I'm always thinking that I did something wrong). 

Anyway, let's go to the conversation. In the last post I wrote about concepts such as awareness, meaningfulness and usefulness. So after the conversation, I added two concepts to my TODO list: Trust and Robustness. Btw, sometimes, I have the feeling that my thesis will be like the collin dictionary, a collection of concepts with their context-dependent definitions. Or maybe it's end up as an onthology... a self-definition of concepts that nobody uses except the owners (Oooops! Sorry! I don't want to offend pro-onthologists ;))

Andrew pointed out these concepts as something that I should consider in the evaluations. Why? Because it is a really important part in the learning process. An iterative feedback cycle between students and teachers. If students don't trust the teacher, would teaching make sense? We are trying to increase the awareness of our students through some kind of feedback (STEP UP!-the dashboard for those that don't remember the name ;)). I really like this picture, trust is reliability plus delight, and in this case, reliability relies (I know it's redundant) on robustness. How will students increase their awareness if they don't trust the system? And just thinking, the thesis students scenario is ideal for evaluating this. We have students, we have supervisors, and we have the activity of both in the social networks. In addition, neither students nor supervisor have the same motivation to work on the topics. Still master students don't have seen the visualizations so we have the perception of the students on how they are performing before the dashboard. We can show them the dashboard and we can ask them about their perception after the reflection. Afterwards, we can ask supervisors about their perception on how the student is going, and,  finally, we can mix motivation on the topic and social-network activity. Here are included posts, tweets and read/skimmed papers but even more important the comments received on their blogs from their peers and supervisors. Therefore these parameters can show us whether they are performance indicators. Because it was other part of the conversation and it also relates to trust:

What do we visualize?

It also relates to visual storytelling concepts. What is the message? What is the goal of the visualization? I think that I already mentioned before in other posts this concept. Andrew has explained to me how they use visualizations to display spent energy. He has told me that makes no sense to show spent energy whether they don't explain information about the context, for instance, how the sensors are and some additional contextual information.

We talked about more things but I was trying to summarize, although as you can see... I'm not good on it!. Also he offered me an additional testbed, he recommended to read one paper.

So conclusions of the meeting: It was a really productive meeting!

Usefulness, meaningfulness and awareness

Previous week, I had a couple of discussions with my colleagues Sten and Gonzalo about these terms. One common topic in our research is awareness... we try to increase the awareness of the user about what s/he is doing, and we use the (sub-)community to contextualize such activity. To this end, Gonzalo uses activity streams in Tinyarm (Haven't you tried it yet? Do it! It's a really cool tool for research awareness) and I visualize activity streams (Post, comments, twitter, toggl and soon Tinyarm activity).

Ok, both different methods to achieve awareness... but how to evaluate whether our tools provide awareness or not? Most of the papers, that I've read so far, made evaluations about conclusions extracted from the visualization or the tool in general... from my point of view, it is related to meaningfulness of something. Sure! It is a really important step! The user reflect on something meaningful and get aware(?) about their conclusions...

(?) Can we say that someone gets aware of something whether there is no change in her/his behavior?

(?) What is the proof that someone gets aware?

From my point of view, change of behavior can be a proof of awareness, however, someone can change the behavior for different reasons. No change in the behavior does not mean anything... maybe s/he feels ok with her/his behavior and s/he does not have such change... or maybe yes, but the trigger is not enough strong.

So what is a solution?

Sten sent me several user experience/design models (1, 2 and 3) previous week and my conclusion is that we should take a more pragmatic point of view. All of them share (directly or indirectly) concepts such as requirements, satisfaction and user's needs. Concepts that are also related to marketing. And ok... we cannot claim that we satisfy a predefined user need if the user comes back to our tool and use it... but at least, we know that something is going well when the user does it. We cannot claim that all the users share the same needs and we cover them... but we can say that we are providing a service and the users consider it useful for some reasons.

I have to think further on this and even more important, thinking about metrics that can be meaningful for us... I believe that usefulness (not perceived usefulness) is the key.

If anybody have some useful pointers, don't hesitate to tell me about them!

Step Up! Because sometimes the name really matters....

Sometimes I feel a bit... how can I say it? Silly... We try to find a suitable name for our application... Yes! Because marketing is also important in research... in fact, I think that is becoming more and more important... sometimes I think that is even more important than the research itself... but anyway... I drop this ideas off and start to discuss about the name itself.

 So... yes... sometimes I feel a bit weird because we discuss about names before we have our application in a stable version, even before we know that the concept works...

 Yes... it seems obvious sometimes that the concept works... For instance, in our case, most of the people thinks that increasing your awareness is positive! But how you will achieve this goal and whether the people will have enough time to spend on your proposed solution is not so clearly very often.

  Ok... so nobody understands why we look for a name in a so early stage... but it has an explanation... a name can define ideas... can send a message... can explain a concept... for me, a name is like an slogan in an election campaign... Would Obama have won without the slogan "Yes, we can!"? I don't know but it's clear that it had a clear impact on the people.

  So we have been thinking on different names such as:

1. LYA -> Learning from your activity.
2. Step Up!
3. SuP! -> Step Up!
4. Learnograph

  and other different names that can be representative for our application. But the selected one is:

 And maybe the question now is: What does it means for me?

STEP UP YOUR AWARENESS!

Because we are too worried about achieving goals, getting certificates and so on. Out of the educational context is the same... what really matters is to buy a new car, new house, new phone... but what happens with the process? Maybe if we were more aware about the process... our choices, goals and achievements would change...

And this is what STEP UP! tries to change telling to the students:

STOP! LOOK WHAT THE OTHERS ARE DOING!

ARE YOU IN THE CORRECT PATH?

THINK ABOUT IT!
MAYBE YOU CAN LEARN SOMETHING
MAYBE YOU CAN APPLY WHAT YOU LEARNT

And it is all what means STEP UP! for me. It is a tool for students, a tool for teachers, a tool for users that want to enjoy the process instead to achieve something, because in such process is where we spend more time... The success, the achievement, the goals are ephemeral... but we always keep our learned lessons along our life...

Blogs viz and twitter aggregation

After a while, I have just decided to post about the new visualization of blog posts/comments and the twitter integration.

Usually the problem is that depending on which decisions you took (related to architecture/development), they have some consequences on the next iterations, so you have to stop and to think a bit about it.

Now, we are working on visualizing blogs of our thesis students, but they have also to tweet. So the ideal situation is merge both visualizations trying to get an overview of the students activity. You can see the visualization here. And you can find a small screencast bellow.


You have two tables. The first one for the authors and the other for external people, they can be supervisors, colleagues of the authors or simply people that decided to comment on the blogs. You have a legend with three colors (Green means posts, Blue means comments and red means no activity at all).

You can interact with the table in different ways:

• If you click on the headers, the rows are sorted based on the activity.

• Clicking on the header with the right button of the mouse, you can access to the blogs.

• If you click on the cells, you get a tip with extra information. Click again to hide it.

• If you click on the sparklines, you get a bigger visualization that shows the aggregated data of comments and blogs per week of the year.

Now the question is:

How to add twitter information to the current table?

Just thinking that in the end, twitter in this visualization can be like a blog. An additional source where users can do some activity.

However, also we are thinking to aggregate other data from toggl. Ok... it can be another column. And every additional source can be an additional column, but in the end, you can find a huge overview of your classroom activity.

For instance, another use case that I'm thinking about is on one course that I'm teaching. Is about learning management systems and technological resources. The students learn how to manage different LMSs in several weeks. In this period, they have to tweet and blog about web 2.0 tools and/or reflect on their weekly activity. So other scenario could be something like blog activity, twitter activity and activity in the different LMSs. Something like number of activities, resources, lessons and so on. But it is always the same... not sure whether the column system is a good approach. In addition, they are more than 100 hundred students... it means 100 hundred rows in the table... ok... we can minimize the impact adding a filter and sorting functionality, but sometimes you lose a bit of the overview. Maybe the solution is to add additional metrics to take the overview, but the matter here is... what are meaningful metrics? Once that you decide to provide metrics, you are trying to drive the conclusions. As a teacher, maybe this is the goal. But not sure whether students expect that from a learning dashboard and what is the data that they want to look into. Yes... for sure... you can ask them, but it is not so obvious approach because sometimes they never thought about what is meaningful for them. Anyway, life is a matter of decisions... and wrong decisions means a lesson that can be learned for the future... as Erik usually says life is learning.

I will tell you more about my decission and what I have learnt in the next post! Because it is what really matters... What do we learn from our research...