Show me the evidence
I was asked by a clinical researcher today what the point was of social media. To put this in context, this is about site that I set up 10 months ago, in a niche area of neuroscience research. The blog now has a steady 4000 unique visitors a month, 7000 page views all through the use of social media. The point may seem obvious, except it was the owner of the site who was asking me.
I was about to trot off the stats in order to justify my existence but then thought, he’s right. Can I really measure the effect? Can I provide evidence-based results that shows that those 4000 visitors have had a positive impact from visiting the blog, and that it has improved their clinical practice?
I can provide readership stats, but clinical impact. Nope.
Loading ...
Heidi Allen aka @dreamingspires: Stats/traffic to a blog are not the same as clinical usefulness (using social media) http://bit.ly/dciqo4
This comment was originally posted on Twitter
[Reply]
How about comparing it to the model of publishing research into journals? When people publish how do they know that their findings have had an impact? By the number of times it was subsequently cited? Or, is it more complex?
Next time, question their perception of the “traditional” model and see if you get answers.
[Reply]
Heidi Allen Reply:
July 15th, 2010 at 8:44 am
Yes, I’m wondering the same thing, in many ways a journal article will get more readership if it is blogged about. But… reading is one thing, clinical effect is another. Maybe I am thinking about measuring the wrong thing. Blogging is literally about dissemination of information. full stop. maybe.
[Reply]
Mark Hawker Reply:
July 15th, 2010 at 8:57 am
Disagree. That assumes that researchers are reading blogs, plus the converse that bloggers are not reading journals. An interesting subset/superset question. Anyway.
You could add “I tried this and it worked/didn’t buttons” to posts, but clinical effect is just too complex. There would be lots of potential confounders and you could never truly link it back. With citation at least you can say “I used the methods/results from Paper X” but maybe blogs are just like you suggest, about dissemination.
[Reply]
Heidi Allen Reply:
July 15th, 2010 at 9:05 am
Yes you’re right, it does assume researchers are reading blogs (suspect this is not the case, bloggers like you or me will be reading journals but then we’re in this subject area. It’s all about the question….
Jon Brassey of Tripdatabase asks that questions a lot too. But how well do we know clinical impact of Cochrane reviews or National guidelines or any piece of peer-reviewed research? The fact that we can’t tell the impact of these means that either we are negligent in not measuring them, or that we are asking the wrong question.
[Reply]
it could well be that we are asking the wrong question. How to frame the right question. I find myself asking what it is exactly I am doing and to what end.
[Reply]
I suppose you could say that comments are more likely to reflect impact. I should read that blog more often. And comment more often. And share it more often. And if I do think that something impacts on my decision making then I will write a comment. I’m surprised it wasn’t in my RSS reader yet. Oops:(
[Reply]
Very interesting observation indeed and thought provoking. I run a blog and am on the editorial board of two PT Journals. I am beginning to wonder how the Impact Factor, which is a measure of the quality and success of a Journal (essentially an average of the citation count of articles)can be related to the clinical impact. In addition, how does the hit count on a blog relate to clinical impact.
The suggestion by Mark Hawker to add ‘tried it/did not work’ button seems rational, but again is too subjective, as internet marketing research has shown that if people are shown a nice button, they are bound to click on it…
So, I feel that clinical impact can only be measured through follow up studies, which could use both subective and objective measures but can not be answered through the hit count of a blog or the citation count of a Journal article.
[Reply]
Way to go Heidi,
4000 visitors per month from nothing is an outstanding achievement. It would be great if the owner of the website had a survey in place for all clients to answer one simple question. Did they use the website. That would at least show some figures on actual clients getting value from the website.
kind regards
Tony
[Reply]