Social media and peer review

Last night I picked up a few tweets in my stream from people who had gone along to the #media140 event in Glasgow which was organised by the lovely @markofrespect. I find it quite hard to follow an event when not there, even with live blogging and hashtags, I think one of the biggest dangers is that you miss the point, miss sarcasm or context and risk barking up completely the wrong tree.

Photo by dicktay2000

So with that disclaimer out the way I wanted to pick up on two tweets I saw from Steve Berry’s talk. The first tweet that got my notice was a question:

“Why don’t other industries keep a journal of failure, like scientists do, for others to learn from?”

Well I think lots of people do…maybe not all industries, and maybe people aren’t always comfortable talking about failures but they’re one of the best sources of reflection, thoughts and new solutions or knowledge. I’d argue that the vast majority of academic research, in any discipline, is done to find new knowledge and most research reports and dissertations report what happened, good and bad, for people to learn from.

My rambling thoughts on the matter are that on the whole the media, including social media, has a lot to be held responsible for in this regard. There is little encouragement for people to offer what went wrong, even for the sake of learning and the development of new responses, if they are likely to be pulled a part by the media. The recent decision by the new government to publish Serious Case Reviews in full leaves me feeling very anxious as to what is to be gained by the process; I suspect that the reality is likely to lead to yet more trial by media for people working in the already impossibly difficult role of trying to protect vulnerable children and I’m rather doubtful as to what the learning will be for other agencies, partners or workers.

So I guess my reflection, if there is one, is that learning from failure and mistakes is really important but also very difficult. So if anyone has any ideas for how best to do that give me a shout.

The second tweet that got my attention, and if I’m honest raised my blood pressure a little was a statement about peer review! Steve said, again remember I don’t have the context – it could have been ironic – to some extent I’m really hoping it was, anyway he said:

“Social Media is peer reviewed on an almost instanaeous basis”

Eeek. I really, really wish I could agree with this statement but I think it’s overly simplistic and dangerous if people believe it. I have real concerns about the potential influence of social media, the fact that people appear able to build their credentials based on nothing more than repeated use of the retweet feature. Increasingly I see people tweeting other people’s thoughts and views with few, or tenuous links to the original source; I see people who know little about a topic (and I’d include myself in here in some situations) being bandied around twitter as people who know something about it; more and more often I see people billed as ‘experts’ on X, Y or Z but the reality is they may be enthusiasts but there is a big distance between enthusiasm and expertise.

So to some extent, yes, there is an immediate avenue for people to question or debate anything, and the increasing access to social media tools has made it easier than ever; however I think it’s worth acknowledging that it takes a lot to do that in some situations. I suspect that people equate followers with popularity or wisdom (both gross delusional mistakes to make methinks) but the reality is that you’d be foolish to pick a debate with someone unless you knew you were right and/or had nothing to lose.

Peer review works on the basis that people with expertise in an area or profession pass judgement about the credibility of statements, papers or work (at least that’s my understanding of it) – social media is missing (a lot) of rigour, the time and the space, the impartiality and objectivity, and nearly always the anonymity that is sometimes required to peer review effectively.

I’d really love to hear from someone who was at the event, to hear what the context was for the discussion. I’d especially appreciate someone leaving a comment and letting me know if I am currently barking up a very large, scottish tree 😉 ps Thanks to those who were there for tweeting it.


6 thoughts on “Social media and peer review

  1. Expertly put 😉
    In all seriousness I particularly agree with your point about so called experts, or to be precise, social media gurus. I stopped following all of these self promoting, mutually appreciating ‘pioneers’ of social media networking for the simple reason that they all seem to be trying to convice us that this is all new and therefore difficult and that we need their expert help in fully utilising the guineas benefits blah, blah, blah…..and breathe.
    Well, the technology and the medium might be relatively new but social networking isn’t. We’ve been doing it for thousands of years, in pictures, in speech, in words. It’s called communication, and if I can communicate then it ain’t rocket science!
    Well, that’s my bit said, back to the football now.

  2. Hello!

    This is an interesting angle to look at this from George – I’m sorry I can’t remember the context of that first quote to fill you in…

    The one point that jumped out at me was:

    “social media is missing (a lot) of rigour”

    Social Media is exactly what is says on the tin – it’s social – why do we feel the need to move it into a realm that requires rigour? It is organic, every changing and completely new – that’s why I love it – that’s why many are obsessed over it. I don’t want to read an academic paper on it I just want to do it and see what happens! Of course there is always a place for rigour and analysis but I think there are new exciting aspects of SM that should be our priority.

    The other point I was interested in was your point about keeping a journal of failure, you say that the ‘vast majority of academic research’ keep a journal of failure. My interpretation of this was much more personal and open. Academic research is not accessible to the masses and immediately isolates people because of language used etc. I think a journal of failure for academics defeats the purpose of being open about mistakes and failings as it is typically only read by fellow academics ( or those in / from higher education) (.. I think )

    It is brilliant in a personal way, enabling reflection and learnings but to influence others I don’t think it is the way forward.

    I like to think of the journal of failure being much more about a mindset,a way of thinking and talking about your life and the work you do. The way society and the media portray and celebrate mistakes / successes. I would much rather read your moleskin full of writings about your failings than an academic paper…

    I hope this makes sense 🙂 and I am proud to welcome you to the scottish tree 😉 x

  3. Hey Lauren, thanks for the reply and the welcome :). I’ve had some chat over on twitter about this post and feel a little reassured that I wasn’t completely up the wrong tree, albeit that I of course had the luxury to expand ad infinitum on here http://bit.ly/bMBtdY. So thanks to @BrianInkster and you for filling me in.

    I respect your lack of interest in academic papers and your wish to just do it and see what happens. That is of course your privilege and right and that’s great. I’m not sure in my head that I’d make the leap from rigour to academic papers though. The characteristics I see missing so often, and refer to when I use the term rigour, are about informative, accurate, scrutinised information…as opposed to constant recycling or accepting without question. To that end I’m delighted we have a conversation going on here – after all that was my point for blogging about this, wanting to question and discuss, not just accept.

    The point I was responding to was Steve’s comment about peer review – I just don’t think peer review works in social media – for the very reason you flag. People just do it and wait for the impact of their actions. There is very little time for thought or reflection, the nature of social media often pushes a sense of urgency and we’re talking about a form of communication that can go viral in minutes. From where I sit there is very little peer review – far more peer dissemination. With regards to twitter in the last few months I’ve clicked on several links in RTs from people that don’t, and never had worked – if the person RTing it had clicked on the link first they’d have known that – instead they just hit RT. Maybe they intended to read follow the link later, maybe they trusted the original source to be correct and therefore just hit RT (and I understand that mistakes happen) but it has just led to me questioning the RT phenomenon and how people use it. Which in turn has led to me having little faith in the idea of peer review in social media. There was a Guardian article today about how facebook has changed adoption – it is far more articulate than me in discussing some of the concerns I have re social media and the impact of it – it’s available here http://bit.ly/cO86YH. I’m not for a minute suggesting that we shouldn’t use social media, I’m as addicted as you are ;), just that consideration needs to be given to how it is used and the potential impact.

    I obviously misrepresented my point about academic research – I was trying to just suggest that a lot of academic work is intended to create new knowledge and research reports should (not that all do) document what goes wrong as well as what goes right. I think things have moved a long way in the past 15 years (no doubt in part due to changes in media and communications) and research is more accessible (and arguably applicable) than ever before so I’m not sure that I’d agree that research is only read by academics or students – or indeed only conducted by those groups. The point I obviously didn’t make that well, is that learning from failure is difficult – I’m very happy to share my failings, I’m a very open person and my blog is one way in which I try to share my experiences in life (not that I’m deluded enough to think that anyone might learn anything from them ;)) but all the blogs and moleskin notes in the world won’t serve to make the changes that I personally would like to see happen in important areas of my life – new approaches to social care, patients experiences of healthcare, reforming the public sector etc – that is where using evidence/research, and a healthy spoonful of rigour, can have a great impact.

    For now I’ll climb down from my soap box – thanks a million for continuing the discussion Lauren, really appreciate it. Would love to know what you took away from the event given you got to enjoy it in person – any chance of a blog post?

  4. I understand what you are saying. Peer review, any review, takes time. And social media, particularly the expectation that a 140 character response is all that is required, tends to make responses instantaneous rather than reflective.
    Last summer I was at a conference suggesting that social media could be used to support the development of scholarship in medical education; not the scholarship which depends on publication in academic journals- but the kind of reflection and feedback from peers which contributes to a scholarly approach to education. A year later and little progress has been made. I rarely make blog posts about my own teaching for several reasons, and one of the foremost is concerns about confidentiality. But there are ways around this. If I established the right network we could find solutions.
    Then there are issues about reward. How would blogging about success and failure demonstrate excellence in teaching as others consider it (http://scienceoftheinvisible.blogspot.com/2010/06/what-is-excellence-in-teaching.html).
    Instantaneous feedback can keep you going in your work. But to really develop projects or ideas needs a deeper level of reflection which needs time. I still believe that social media can facilitate this. And I think you demonstrate this.

  5. I’m with you here George, although I’d say the peer-review part is just a sign of a larger symptom; the fact that many persons have a tendency to not be critical (enough). Peer reviewing is a way for academia to make sure that quality is maintained – at good academic outlets you will go through a blind peer reviewing process before you get to publish. Then if others think what you said is good enough and want to publish a reference to you, they need to go through the same procedure – so the person citing you needs to make a good argument as well.

    In the twitterverse you can claim what you want without anyone taking responsibility for its truth or quality. And if someone wants to quote you, it takes them 2 seconds. That makes it much easier for something which is not true to be spread. On top of that you know who has said what you want to quote – many times I get the impression that retweets are done just to suck up to someone…

    But those facts (I wouldn’t dare to call them that in an academic paper though), is not a problem in itself. Rather a strength. It’s fantastic that we have different levels of communication. The problem which does occur is when people forget this fact and compare material from different channels and hold them to be equally valid sources of “truth”. Social media is fantastic for conversations, keeping up with what is happening and much more. What it isn’t fantastic at (rather the opposite) is providing what academia can do: a rigorous check of the truth-value in what is published (rigorous in such a way that it is conducted according to standards which the community at large agrees on – in the humanities it might still be done from a perspective which you don’t like; and the difference here is the subjectivity in your values vs the objectivity in a common standard).

    So to reiterative; social media and academia are both great at what they are intended for; problems occur when people forget the difference and compare them to each other as if they were equal…

    (On a sidenote: I can’t really say that I was surprised to find the person behind the peer-review comment isn’t in academia)

Leave a comment