Tag Archives: open science

Hypothify: first outline

I’ve been pondering a hypothesis-based academic social media site for a few weeks now; and talked with a couple of people about it. Ideas are only just beginning to coalesce but now seems the right time to try to outline what I see hypothify doing, and how it might work. I’m conscious that it needs to start very simple, and remain as simple as possible. It’s easy to come up with a massive feature list, but identifying the most important stuff and planning how it might work is key. [edit after writing – there’s still A WAY to go on this!!]

What?

A place to propose,  discuss and aggregate evidence for/against hypotheses. Consensus building through democratic voting up/down of evidence/ discussion items. What I thought Quora was going to be when I first heard about it, but less question/answer and more evidence-based.

In traditional(ish) publishing terms it would represent a platform for ‘living’ review articles on each hypothesis. However, would integrate social media aspects (voting and sharing) and wider evidence than just academic papers.

Not peer-reviewed, or peer evaluated, but peer assembled.

Why?

Hypotheses are fundamental to academic work. They represent the ideas and concepts which propagate through the academic sphere and out into the wider world as our understanding of the natural world/ the universe/ the human condition etc. They are often dissociated from the piece-by-piece evidence in the traditional academic record. Currently academics are supposed  to read everything and make up their own mind on a particular matter. For each individual this is only possible for a limited number of concepts/hypotheses because of the massive time cost of i) finding all the literature and ii) reading it all and ii) keeping up to date. In reality we all take ‘received wisdom’ on many matters on trust from other academics, or tend to disbelieve everything we’re told and argue it out ourselves from first principles! Hypothify would solve the pain of i) and negates ii) and iii) by providing community-maintained consensus instead of ‘received wisdom’ on each given hypothesis.

How?

The platform would allow the proposal of hypotheses by any user. Evidence items (papers [via Mendeley API if poss], unpublished data [figshare, slideshare, blogs or notebook entries], snippets of discussion / reasoned argument [from discussion of this hypothesis or elsewhere via e.g Disqus]) can be presented by any member of the community  as being ‘for’ or ‘against’ the hypothesis. Key to the usefulness of evidence items will be a tweet-length summary of what that evidence item contributes to the assessment of the hypothesis. One will have to be added by the person introducing the evidence, other ‘competing’ summaries may be added. Where necessary, discussion of the evidence can be conducted and this itself can be cited as evidence itself. It is conceivable that a single piece of evidence may be argued to support either side of a hypothesis. Maybe it’s necessary to recognise that evidence can be ‘agnostic’?

Key to the success of the platform will be the voting up/down of content (a la stack exchange). Hypotheses themselves should not be voteable on I think  -i.e. there will be no opportunity for individuals  to vote subjectively/dogmatically for/against a hypothesis, only vote up or down evidence supporting or contradicting the hypothesis. Plus vote up or down particular summaries of evidence items, so the best summaries float to the top for each bit of evidence. So the ‘hypothesis view’ page will show the hypothesis at the top and evidence items for and against (highest voted first), with the best summary pertaining to that hypothesis for each one. Plus a link to the evidence item (i.e. NOT stored on-site). I think this is really neat because a user can find a hypothesis they’re interested in, find what the community thinks it the base evidence for and against, read those bits, and make and informed decision based on comprehensive community review of the field. It may or may not be useful to have a ‘swingometer’ for each hypothesis which  represents the net votes for evidence for and against the hypothesis, which give a ‘community assesment’ of the hypothesis.

Attracting users?

What’s in it for users? Firstly, being seen to propose and and contribute to hypothesis assessment will bring kudos to users. A ‘reputation’ system (also a la Stack Exchange) could be implemented to measure reputation / value of contributions… Even badges etc would probably work for academics, but I think there’s a more instantly attractive ‘doughnut’ (as my good friend Charlie calls them) – promotion of your research output. If you add a good summary of your paper which informs debate on a particular hypothesis it will (if it’s good) float towards the top for that hypothesis. You will be able to engage with other interested parties and discuss your research. Google will love it.

Engage ‘the enemy’. Let’s say I propose a hypothesis, which just happens to be something I’ve proposed in papers in the literature in the past. Great. I put up the hypothesis, provide evidence items. As I’m the only contributor, the hypothesis is only ‘proposed’. To move it to the ‘community debated’ stage I need to get other people involved. So I share it on Twitter, but also invite people I know will be interested, to join the debate. Furthermore, other established hypothify users will be automatically invited to join based on their interests  and other hypotheses they’re active on and the tags which have been associated with the hypothesis in question.

As evidence items are added, the system will attempt to ‘scrobble’ originator details (emails, figshare user details, Mendeley author details) and contact the originators to inform them that their work is being used to inform debate on a particular hypothesis. They will be invited to join the debate. I’m guessing if their work is being ‘correctly’ cited they will be flattered enough to go and have a look, and if it’s being ‘incorrectly’ cited (in their opinion) they will be incensed enough to wade in and put those upstarts right. Thus the experts will hopefully filter in.

Furthermore, as evidence and discussion accumulates and more people vote evidence, evidence summaries and the hypothesis summary  up and down, the ‘top contributors’ will be identified. Those top contibutors, plus the hypothesis proposer (i.e. the proposer on hypothify, not necessarily the originator of the hypothesis in the outside world [who should be cited and acknowledged]) will be identified at the head of the hypothesis view as the ‘authors’ of the synthesis. Thus each hypothesis becomes a peer assembled citeable document (hopefully with a doi assigned). A publication! And as we all know in academia, publications are all. And what’s really nice is that it doesn’t matter which ‘side’ you’re on. If you’re presenting valuable evidence and discussion in either direction, you’ll be listed. So all those old vested interests evaporate away like the eyewash they are.

Synthify?

Not all problems present well as hypotheses. For instance, in my field – marine biogeochemistry – much science is exploratory, and/or based on assessing magnitudes of things : “What is the globally representative concentration of ammonium in the surface ocean“; “What is the annual global carbon uptake by primary producers in the ocean“. Of course, these can be presented as hypotheses “The globally representative concentration of ammonium in the surface ocean is 150nM“, but this is rather pointless. However, the accumulation of evidence leading to an assessment is much the same process as outlined above for hypotheses, only without the FOR and AGAINST argument. And these syntheses could then feed in to wider hypotheses as evidence. Synthify.com has been taken unfortunately, but I think it’s reasonable to conduct such data synthesis work under the hypothify banner. For the field I work in at least, I think the ‘synthify’ function will be as useful as the ‘hypothify’ one.

Anything else?

Moderation will be important and will rely strongly on the community. Controversial topics could get very sticky very quickly. Need to think about policing. Free speech is important, but balanced debate more so. Anthropogenic global warming deniers and intelligent designers are going to be a challenge to the stability and value of the system.

Integration with the rest of the web is obviously very important. All items will obviously be fully shareable, but a proper API would ideally allow full functional push and pull to and from other sites – mendeley, peer evaluation, wikipedia, quora, disqus etc etc.

If all this sounds irresistably interesting, please hit the pre-sign-up  at http://hypothify.kickofflabs.com

Advertisements

I declare my science offically open! Mostly

I’ve been keeping an open lab notebook since late last year, recording my laboratory and related scientific activities. Partly because I believe strongly in the philosophy of openness in science, as in software, and partly because I feel like airing my dirty laundry in public will force me to be a better and more rigourous scientist and ultimately do better work.

I’ve been finding my feet over the past few months – in what format do I put data up, what are sensible (semantic, consistent) names for entries in the notebook, at what level of detail should each individual entry occur, how much descriptive / explanatory writing should accompany raw data / observations? I’m not sure I’m quite there with the answers to those questions yet, but I’m getting there. Also, I wasn’t sure how recording everything online would fit in my working day – would I manage to get round to recording everything I did, or would some things get missed? So far it’s been pretty successful.

As it’s all been going quite well I thought I’d ‘fess up to my PI that I’ve been promiscuously sharing our precious research willy nilly all over the internet (I really should have told him up front but I didn’t want to make a big thing out of it and then it not come to anything). He was fine (he’s pretty chilled out, and we’re not exactly working in a commercially sensitive or massively competitive field), and actually quite supportive of the idea. Phew.

SO now that it’s official I thought I’d put an Open Notebook Science banner on my notebook. This involves making a ‘claim’ as to the degree of openness – does absolutely everything I ever do go up immediately, or is it just some of my stuff, after some period of time. It’s my intention to be fully open, and to put everything up immediately. However, as I’m still getting going with the whole enterprise I’ve decided to go, rather conservatively, you may argue, for claiming ‘some content, immediately’. I.e. you can’t absolutely guarantee that if it’s not in the lab book it hasn’t been done. It’s not the ideal, and my intention is to change to a fully open claim soon, but this is a step in the right direction.

SCI Seclected Content - Immediate

My open notebook claim. Click the image to go to the Open Notebook Science claims page to find out what it's all about

Researchpages code set free!

https://github.com/martwine/Researchpages

Years ago I built a ‘networking for research’ web application / site called researchpages (http://researchpages.net). The site is still running and in use by some hundreds of users, and getting about 100 hits a day I discovered earlier when I visited the google analytics for it for the first time in years! But it’s most definitely not in active development. The best way to describe it is maybe as ‘MySpace for research’. So it was probably a bit behind the times in 2006 when I built it, at the dawn of the Facebook age!

In many ways it has been superceeded by stuff like Mendeley. But I think it still has some useful features, or at least ideas of features. Multiple-author-linked publications, wiki, blog and web space (albeit very basic)  for individuals / projects, ‘resource’ (i.e. images, documents, other files) sharing. Some level of public / private functionality. All in all as my first ever (and only) database driven web app it’s not so bad.

Also in it’s favour is the fact that it has run stably for the last 5 years without me having to make any code changes since I stopped actively developing it! It has quirks, but the only thing that has brought it down is disk full errors. Which does happen quite often due some cron backup ridiculousness that I’ve never found the time to sort out.

Anyway, the future of researchpages hangs in the balance. I don’t really have time to maintain it let alone improve it and it’s so un-integrated with the rest of the social web (no rss, not open id, nothing, no api, no facebook or twitter integration) that it is guaranteed to die a slow death over time. Maybe by setting the code free, it will find some new life somewhere. At least some bits of it might be of use.

If anybody would like to help me with or take over the running of researchpages you’re more than welcome – please get in touch! Otherwise, please enjoy the code in whatever way you see fit.