After two years... preparing Phd Plan! (4)

This is my last post regarding the PhD Plan... after this post, I will try to formulate it in a formal way... but before I will answer two questions:

What I am doing now?

Now I am working in more widgets. Our professor, Erik, works with social networks in all his courses, specifically he suggests to the students that they have to tweet and to blog once per week. For teachers and assistants is relatively easy to follow what the students are doing whether they are actively participating in the social networks, for instance, using Google reader for the blogs and your twitter client, but becomes more difficult whether the teacher/assistant tries to know who is actively participating... and visualizations can play an important role to help this situation...

We developed this widget that visualizes per week the number of tweets:


On the top of the table, you can see the number of the week, on the right side the usernames. You can take an easy overview.

However, doing the same for blogs becomes a bit more close because you have to visualize posts but also comments who is posting and where.

These visualizations are not only oriented to teachers, also for students can be useful to be aware what others are doing and whether you are not enough active compare it with other students. One of the main critiques could be... to game the system is easy... you do a silly comment in the specific network and the cell becomes green. This is true... but we should not forget that teachers and assitants have to read all this social network interaction and they can remind students the kind of expected contributions.

So here maybe, we have to evaluate whether this kind of visualizations shared by teacher and students are good triggers for students to reflect on their progress.

Summarizing:

- How can we help students and teacher to drive conclusions using visualizations?
- How can we motivate students to reflect on their activity?
- What visualizations are more useful in a learning context and why?

So let's start to work!

After two years... preparing Phd Plan! (3)

Yes! Paper accepted! So someone of us (Sten Govaerts, Katrien Verbert, Erik or me ) will go next year to LAK Conference. They are really good news, it is a conference indexed in acm.org.

But the question is:

- What did we do to be accepted in this great conference?

It is an easy answer! We tried to explain how we proceed with our first attempt to answer one of the research questions:

- How can we help users to drive conclusions in a generic way?

We thought that term of learning goals could be nice. But what does learning goal means? It is a more difficult question... so we decided to reduce a bit the abstraction level of the word. Let's call it "Goal". What can be a goal? Everything that means achievement... a milestone in the course can be a goal? Sure! And if a students want to spent more or less time focused on an application, is it a goal? Yes! Sure! What am I trying to explain here? That we keep this word a bit away of the pedagogical formal meaning.

We got some inspiration from tools such as Rescuetime, Wakoopa, RunKeeper, JoggyCoach,... applications more oriented to the behavior rather than an abstracted goal. And why? For instance, in RecueTime, you can set up goals such as I want to be more productive! And you start to track your activity... but how RescueTime calculates your activity? They have a classification that scores every application as a productive or non productive... so browsing is not a productive application. It means that you can be editing a deliverable in google docs and you are not productive... sorry dude! You are wasting your time doing such deliverable... ;) Luckily, this classification is customizable... but anyway... blogging can be a non-productive tool and productive depending of the purpose... And It is really difficult to track... but also difficult to get input every time from the user that changes the focus to another application. So let's decrease the level of complexity and we ask:

Do the goals of the course help to contextualize the visualizations?

We developed this dashboard:



We developed a dashboard with visualizations of activity data. The overall goal of this dashboard is to enable students to reflect on their own activity and compare it with their peers. The time spent with different tools, websites and Eclipse IDE documents are tracked by RescueTime and the Rabbit Eclipse plug-in. The collected information is displayed in a dashboard containing goal-oriented visualizations. In the visualizations, the students can filter by different criteria, such as course goals and dates. Such filters allow contextualization of the visualized data for the user. Linking the visualizations with the learning goals can help students and teachers to assess whether the goal has been achieved.

In this course, the students have to develop software and go through the different phases of software development process, such as design, programming and reporting. To this end, they use tools such as LibreOffice, the Eclipse IDE and Mozilla Firefox. They have to share tasks and responsibilities between group members. Controlling the risks and evolution of such tasks is part of the assignment.

Visualization 1 are the goals during the course that can also be defined as milestones. Green color is that the time has expired and blue is that currently is going on.

Visualization 2 is a motion chart where x is the activity of the user and axis y is the average of the members of the team.

Visualization 3 is a timeline that visualizes the time spent on the different activities. To classify this activity we use the Rescuetime taxonomy that the site provides.

Visualization 4 is a barchart that visualizes the same than the previous one but visulized with the total activity.

Visualization 5 is a timeline that visualizes the time spent on the concrete tool.

Visualization 6 is a barchart that compares your activity with the total activity of the group.

Visualization 7 is a barchart that show the time spent on the different document on your Eclipse.

Finally, visualization 8 visualizes the time spent on the different websites.

At the beginning of this month we attended the Quantified Self conference in Amsterdam and there we learn something that was relatively useful... an easy way to present our results answering three simple questions:

- What did you do?
- How did you do it?
- What did you learn?

We have explained more or less the first two questions but...

What did we learn?

The perceived usefulness by the users is good but they don't use the application. It points out that we are doing something wrong... maybe the problem is that we don't stimulate enough the students... there are a behavior model that describes when the people modify their behavior. People needs triggers that motivate the change and the ability of the people to do the action. In this website, Fogg describes Facebook as a good trigger generator... for instance, when someone is tagged in a photo and receives the notification, automatically s/he connects to facebook to see the photo. Maybe this is some of the topics that we should explore...

How can we motivate the students to use learning analytic tools?

And tomorrow a bit more! What we are currently working and future plans!

After two years... preparing Phd Plan! (2)

As I mentioned in my previous post, I shifted a bit my research after the ARIADNE Registry work...

I got involved in ROLE project. It is a project that research about Responsive Open Learning Environments. Specifically, it focuses on Personal Learning Environments (PLE). PLEs enable users to personalize the services required for the different assignments in a learning process, in a technical way, we can define it such as a mashup of services.

Don't hesitate to read our paper about ROLE that we presented at ECTEL'11!!!!!!!!!. The main authors of this paper were my colleagues Sten Govaerts and Katrien Verbert.

Personalization is the key. If users can personalize their own learning environment, we add another possible factor that can influence on the success or fail of the user activities... which resources and services can be the key to understand learners (from a teacher perspective but also from their own perspective).

The understanding of the learner is the key of my current work, the development of tools to reflect on the user activity and visualizations can be one of the possibilities.

The first attempt to do something in this field was this application and we submitted a paper to EFEPLE workshop. You can see a screenshot of the application below.



During the development, we did some evaluations with teacher assitants, and finally had a discussion in the context of the EFEPLE'11 workshop.

The results pointed out some interesting details:

1. Users have different visualization background and they use the visualizations for different purpose. For instance, some of the evaluated users highlighted the fact that bar charts are really easy to understand and they really like it. However, others user with higher skills in visualization tools asked for more relevant information. They were interested for more detailed information.

2. The viewer needs information about the context to understand the visualizations. If you know the purpose/goal of the learners, the activity, the course, the specific student... the visualization makes more sense... now it is a dashboard that visualizes information... but the question that came up is:

How can we provide more meaningful visualizations in a learning environment?

My colleague Joris Klerkx recommended me a paper about narrative visualzation that I really liked it. This paper explains in a few words that visualization can be a good support to explain a story and shows some examples.

Sometimes I heard the discussion about what is better visualizations or recommendations? Simplifying the differences (both are huge fields), one rely on the user the cognitive efforts to achieve conclusions (Visualizations) and the other minimize the user cognitive efforts providing some possible conclusions (Recommendations). Maybe it is not a matter on which approach is the best... depending of the user and depending the quantity and quality of data that we have, one can be better than the other...

So that paper showed me a middle term approach, in a narrative way, the writer was driving the conclusions of the reader and the visualizations were the support of our conclusions (It sounds really similar to me when we use visualizations writing a paper to support our conclusions with quantity data, isn't it?).

Again, my question that was coming up to my mind was:

How can we help users to drive conclusions in a generic way?

Generic way is the key of the question. The systems that I'm working with are very different, different services, different trackers, different quality of data, different formats, so... it makes a bit more complex driving conclusions if you don't know which data and context you will have... again... context is the key... and sometimes we don't have access to this context... for instance, PLEs, LMSs, learning environments in general don't enable external access to such information easily...

How can we provide information about the context?

I finish here my post... but don't worry... I haven't still solved these questions... maybe someday we can read my thesis with useful conclusions about this point... :-) but I think that this question is really interesting to research on: How can we help users to understand visualizations?

Another post comes soon... waiting to know whether my last paper is accepted or not... crossed fingers...

After two years... preparing Phd Plan! (1)

Yes... I know that it can sound a bit weird... It's something that I should have done during the first year of my PhD... but there is always a positive side... now I have deeper knowledge in my field. So now, I have to do a bit of inverse engineering for what I have got so far and a bit of planification for my next challenges... The goal of this reflection is finding the research question that I have answered and the research questions that I am going to answer in the near future.

So let's go to think about what I have so far...

At lirias you can find my publications at KULeuven.

In 2010, I got two publications. The first one was presented at ECTEL conference. It describes the ARIADNE architecture. My contribution to this paper is about the ARIADNE registry and it has a strong link with the second paper due to it is the description of the already mentioned ARIADNE registry that we presented at the SE@M workshop.

What is this registry about?

The ARIADNE Registry is an open source project that enables users and services to discover new repositories of learning resources. Yes... I know... it sounds interesting... isn't it? ;)

What can we do once that we have found a new repository?

First, I should point out that these repositories can work in a federated way... why? Because they can share descriptive information about the learning resources and, to this end, they expose this information through service interfaces. This idea is similar to the WEB 2.0 concept, different services that can interact with others, for instance, facebook has an application that if you install it, all your tweets are posted directly in your facebook wall. It's cool... updating one social network status, it appears in two social networks... so one time work... double impact of your status...

Let's apply the same concept to the repositories... you (as learning resource provider, simple user or teacher) update your learning object once and such information is extended along the network of repositories hence your information is accessible from different access points. (It's important that user remains the control of the object itself... so this federation is not incompatible with business models). The idea follows the same concept: increase the impact of your work!

It seems ideal the situation but...

What is the challenge?

The challenge is that repositories expose their information using different protocols and specifications. For instance, repositories can expose their information using SQI, SRU/W or OAI-PMH protocols and the description or metadata using specifications such as IEEE LOM, Dublin Core and RSS. Therefore, these requirements should be described somewhere enabling access to external services to this information automatically. In this way. they can understand how they can access to such metadata.

Looking for a possible solution in the context of ASPECT project, we used the specification IMS LODE. Such specification allowed us to define different collection of objects and define the protocols to access the repository information.

It worked quite well providing access to more than 700.000 metadata objects in total and the integration fo the ARIADNE Harvester with the ARIADNE Registry.

What are the limitations of this approach?

Currently, learning resources are not static objects such as docs and pdfs. Blogs, tweets, forums and so on can be learning resources. IMS LODE is an specification oriented to define protocols and specifications, however these WEB 2.0 services do not follow specifications and protocols, so most of the definitions are specific to such services and we can lose the process automatization.

What is the research question?

What are the benefits on defining services to share learning resources using IMS LODE?

As you will see in my next posts, I changed a bit my orientation on my research focused on how we can track users and enabling reflection on the activity for teachers and students, but following the same research line 'mashup of services'.