Reveal-it applied in an educational context!

As I mentioned in my previous post, we (Erik, Gonzalo and myself) attended the Chi conference. More concretely, we attended one session about "Tensions in Social Media" due to there was a very interesting paper called "Reveal-it!: The Impact of a Social Visualization Projection on Public Awareness and Discourse" and the fact that the authors of the paper were from my former university (UPF) and one of the co-authors was my assessor Andrew Vande Moere increased my interest to know more about it.

What was Reveal-it about?

Source of the image: blog of the primary author on the paper

I would strongly recommend you to read it. I ask in advance apologies to the authors whether I forget some important details about the paper, but summarizing this paper describes a set of experiments where energy expenses are visualized on an ambient display in different public spaces . The visualization is designed in order to increase the user awareness of energy consumption. The experiment also digs into how these public visualizations can trigger social discourse. Users that pass by the visualization can report their expenses and their information is added to the visualization. Such information is highlighted in order to attract the attention of the user and to trigger reflection. The information of the user can be compared with the expenses average of her/his neighborhood.

What did we do?

We wanted to test this application in our context, education. So we started to think about how we could apply a similar concept to our students. It could also be an alternative to our big table overview.

We wanted to test the application with our students who currently use StepUp! (a bit of explanation  here), Navi (a bit more here) and the activity stream (that aggregates all the activity of the course). So this evaluation could help us to understand how Reveal-it could complement our current work.

Ok, so... what do we have? We have students and we track different activity, tweets, blogs, time, badges... they can work in groups or individually. We wanted to test it with our current students and  the courses are ending... so they do not have to report new activity to the system... but still we wanted some interaction with the visualization with a second device such as highlighting the user activity...

Our first approach was to do the analogy between neighborhoods and groups. But we found two main issues:

•  We had to create one visualization per activity. For instance, for our #chikul13 students, we created four different visualizations (per blog posts, blog comments, tweets and earned badges along the course). Reveal-it aggregates gas and electricity expenses because both are paid with the same currency, but the nature of our data is completely different and to find a common way to measure it was difficult.
• Most of our #thesis12 students do not work in groups, so the analogy of neighborhood and groups did not work for this case study.

 We thought that to keep all the data on one unique visualization made more sense. So we forgot about the analogy and we went for a simpler approach. Neighborhoods are different activities such as tweeting, blogging, commenting, spending time and earning badges. And each user should be represented in every "neighborhood". In this way, the students can see in a glance how their efforts are distributed compared with the others and the mean.

What was the result?


So after Sam and I tweaked the code, this was the result for our #thesis12 and #chikul13 students:

And the students could highlight their usernames using a simple mobile web app:

What did we do?

We evaluated the tool during a poster demo session of our #thesis12 students where also our #chikul13 students were invited to participate. In fact, we did two simultaneously evaluations. Sven that focuses on how we can enhance collaborative reflection with multi-touch and big devices evaluated his tabletop app.

We projected the two visualizations, each one in different walls.

What were the first reactions from the students? A couple of #thesis12 students asked me before the start of the poster session if it was a #chikul13 project or something like that. I told them that the visualization was displaying their data, one stayed for the evaluation, the other almost ran away...

After this, I stayed a bit away of the visualization and I didn't see any student who paid a lot of attention to the visualizations. Consequently they didn't even read the text where was explained that they could interact with the visualization. So before losing the opportunity, I started to ask some students if they could evaluate the visualization.

I introduced them the tool explaining that it relies on a concept of public and ambient displays but that it enables interaction through a web mobile app. I let them use the app with my own mobile and they started highlighting their username and afterwards others usernames.

I highlight some findings got from the interviews (10 interviews):

•  Two groups of two people understood faster the visualization and they had some fun (at least they laughed) while they were comparing with each other.
• All the individuals needed a bit of help to understand completely the visualization.
• Three person understood the bars as a chronological representation of their own activity. Each bar was a week instead of a user. It can be a bias because so far StepUp! and Navi they use weeks as a granularity level to represent the data.
• But the most important perception, at least from my point of view, is that most of them expect to interact with the visualization, for instance, in order to highlight the outlier users. However, when I inquiry them about if they think that was a required feature, they replied that maybe to compare themselves with the mean was already ok.
• Also I asked them what would they rather prefer, whether the big table visualization or Reveal-it. The opinion was quite divided because the big table overview gives them more information, however everybody agreed that it was a fancier and nicer visualization that enables you to get a quick status of your activity compared with the others.

Still we have to research a bit more about it. But Reveal-it is considered more or less equally useful than StepUp! and Navi. On the other hand the #thesis12 students would feel less confident with such public visualizations used in public spaces than #chikul13 students. And they do not find that interacting with the visualization with a second device (in this case a web mobile app makes a lot of sense).

[weSPOT] Personal informatics, workshop, chi conference and weSPOT

This weekend weSPOT (as myself on behalf of the project) attended to the workshop of personal informatics at Chi conference. It was nice how the organizers set up this workshop in a hackaton kind of way.

First we participated on the workshop madness session, a series of 2 minutes presentations where participants could introduce themselves and their work. 2 minutes is a really short period of time but enough to make the others understand what are you working on and what do you expect from the workshop. Sure! It requires pragmatism, simplicity and left aside a bit of the narcissism that characterize to every good (and not so good) researchers ;).

In fact, it was one of the issues that Mara Balestrini brought to the discussion, are personal informatics promoting narcissism? Personal informatics are pretty much about self-knowledge, but this tools also should promote empathy among the users... it is not a matter only to understand yourself, it's a matter also to understand the others. I really liked this kind of reasoning, because in our topic, learning is also important. In fact, we expect that students understand themselves through understanding their peers in the social context.

After this workshop madness session, we started our hackaton. We started to work in a project that we previously discussed through email. The members of my team were Mara Balestrini, Jon Bird, Christian Detweiler, and Mads Mærsk Frost. Basically, our team focused on how truthful are the answers when people replied to a survey due to a sociability bias. It has been a long topic discussed along the years and some people already proposed a simple solution for yes/no questions [1][2].

Jon Bird proposed to develop an app with this system. Basically the system relies on a very simple methodology, the user before answering a question has to flip a coin. If it's tail, you have to say the truth, if it's head, you have to reply yes by default. In this way, nobody knows if you has replied truthfully or not. However, statistically we know how many 'yes' we can drop from the sample and the rest are reliable 'yes'. The theory says that in this way, we can know the real percentages of the answers.

In order to demonstrate whether this system could be integrated in an app, we are going to deploy three different kind of surveys. One survey where the flipping coin methodology is not applied. Another where the user has to flip a physical coin. Finally, a third one where the user has to flip a virtual coin integrated in the system.

The ideal result would be that a social bias exists in the first one but not in the other two. But we'll see the results... we hope to deploy tomorrow during the conference.

Does someone wonder what kind of questions will be? We'll try to balance between very personal ones such as "have you ever had an affair?" and less personal ones where the social bias should be less.

We'll see what comes up from this very interesting workshop! Hope we can report something soon!

In the meantime, let's see if we can get some inspiration from this amazing conference!

Navi, StepUp, OpenBadges and ¿Gamification?

It was a long time ago since my last post... but is always to get back to good habits...

Yesterday there was a really nice discussion in our HCI course where we are evaluating our Open Badges approach.

In this experiment several tools take part:

• Navi: It's the dashboard to display the badges to our students. As you probably know, we are continuously iterating our prototypes and this is not an exception ;) So feedback is welcome! Btw, this app is developed by Sven Charleer who joined our team on January.
• StepUp back-end: If you have read previous posts on this blog, you know that I am working on trackers and visualizing this information in a meaningful way for students (or at least I try)
• Open Badges system: We rely on Mozilla Open Badges System to give the students the possibility to share their badges with the outside world through social networks.
• Analytics layer (sorry, it does not have any URL): The backend that contains all the rules to award the badges.
• Activity Stream of the course: Following the same concept of TinyARM that aims to increase the awareness on what others are reading, we merge the different activity streams of the course such as twitter, blogs and badges in the activity stream, offering to filter by the different actions.

What is the goal of this experiment? Are we gamifying the course?

Badges are game elements but they are representation of achievements. Some students claimed yesterday that applying gamification to master students was a bit childish... and I am aligned with this idea... there are even current research that goes against the gamification of learning because it breaks the real motivation of learning... everybody should have their own intrinsic and extrinsic motivation for learning... however how the teacher teaches the lesson is other point... if he does it dynamically, participative, collaborative or simply boring is up to him or some rules of the institution... and usually is up to the student, to attend the f2f lessons (except if they are mandatory), to be participative, etc... and learning analytics tools can be part of these decissions.

Learning analytics are other additional resource to help the students to steer their own learning process, but is up to the student to use the tools that we provide. We usually test our applications with bachelor and master students and our assumption is that they are autonomous students... they will become engineers and computer scientists soon... so our first assumption couldn't go in different direction.

So... What are badges for us?

Our assumption is that badges are a representation of achievements and a mean to reflect on what is going in the class.

If you (as a student) are not tweeting, commenting or blogging but you see that others are getting badges for it, it may trigger a question:

• why is the teacher giving badges to the students? The answer is clear, we are encouraging positive behavior.

There are some badges considered as neutral, but they are given periodically. Theoretically, they want to increase the awareness of what you or other student has done. We could use some chart instead, whereas badges represents an achievement, visualizations rely on the user the cognitive effort to drive conclusions and we try to simplify this reflection process.

If someone finds a fun element in this process, it's great! It will increase the motivation and it usually have positive effects! But... Learning is already fun by itself!

And what are we trying to figure out from our students? Do they consider them useful? As a reflection mean, as motivational elements, as positive feedback... they decide and:

WE LEARN FROM THEM