Usefulness, meaningfulness and awareness

Previous week, I had a couple of discussions with my colleagues Sten and Gonzalo about these terms. One common topic in our research is awareness... we try to increase the awareness of the user about what s/he is doing, and we use the (sub-)community to contextualize such activity. To this end, Gonzalo uses activity streams in Tinyarm (Haven't you tried it yet? Do it! It's a really cool tool for research awareness) and I visualize activity streams (Post, comments, twitter, toggl and soon Tinyarm activity).

Ok, both different methods to achieve awareness... but how to evaluate whether our tools provide awareness or not? Most of the papers, that I've read so far, made evaluations about conclusions extracted from the visualization or the tool in general... from my point of view, it is related to meaningfulness of something. Sure! It is a really important step! The user reflect on something meaningful and get aware(?) about their conclusions...

(?) Can we say that someone gets aware of something whether there is no change in her/his behavior?

(?) What is the proof that someone gets aware?

From my point of view, change of behavior can be a proof of awareness, however, someone can change the behavior for different reasons. No change in the behavior does not mean anything... maybe s/he feels ok with her/his behavior and s/he does not have such change... or maybe yes, but the trigger is not enough strong.

So what is a solution?

Sten sent me several user experience/design models (1, 2 and 3) previous week and my conclusion is that we should take a more pragmatic point of view. All of them share (directly or indirectly) concepts such as requirements, satisfaction and user's needs. Concepts that are also related to marketing. And ok... we cannot claim that we satisfy a predefined user need if the user comes back to our tool and use it... but at least, we know that something is going well when the user does it. We cannot claim that all the users share the same needs and we cover them... but we can say that we are providing a service and the users consider it useful for some reasons.

I have to think further on this and even more important, thinking about metrics that can be meaningful for us... I believe that usefulness (not perceived usefulness) is the key.

If anybody have some useful pointers, don't hesitate to tell me about them!