As part of our usability testing we have been asking users to fill out user satisfaction questionnaires. In addition to the standard rating scale questions, we have also asked users to fill out a checklist of adjectives that describe the system we are testing (as written about here http://www.userfocus.co.uk/articles/satisfaction.html ).

What we have been finding is that the words users are selecting do not seem to reflect the reality of their experience with the system. For example, users that struggled with the system and could not complete tasks without assistance are selecting terms such as 'Easy to Use' and 'Straightforward'.

Is there a way we can frame this process that would encourage users to be more realistic/honest in their responses?

Thanks!

Maybe its something wrong with questionnaires? Can you show them to us?

It's exactly as taken from the Userfocus article. http://www.userfocus.co.uk/articles/satisfaction.html

I heared that a teacher of the Rutgers University conducted a research on the cognitive factors that explain the difference between the general good feeling a user has for a UI although his user experience was not so good. It could be due to the fact that 95% of our behavior is unconscious, therefore we are not able to express unconscious unsatisfaction.
Does someone have the reference of that study ?

I don't think that any questionnaires are 100% able to tell for developers what is the real problem. Off course they really help us, but there others way to find what's wrong... like the user try a real tour and report some problems! In my mind, this way can help us more