Our Blog

Don't Let Data Drive Your Dialogue

4 November 2010
2
1605 votes
+
Vote up!

Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies, and statistics.’” - Mark Twain

Why should our dialogue be “driven” by data? I certainly understand that data can provide a provocation, challenge, affirmation or contradiction that sparks dialogue - but why should it “drive” that dialogue? Data itself has no meaning, until it is organized and displayed in charts or graphs that can be interpreted, usually in multiple ways. These interpretations may usefully inform our dialogue, decisions and subsequent actions so data definitely can be valuable, but it often seems to be granted undue reverence simply because it is numerical. Although insight can derive from analysis of data, equally it can arise out of intuition and, in fact, I wonder if some analyses are not actually rationalizations subconsciously imposed on data to justify intuitive speculations.

The notion of “data-driven dialogue” was introduced to counter the perception of an excessive reliance on unjustified opinion and personal preference in educational discourse. I think this correction was required and believe there is more work to be done to reduce the aversion to data that still exists for some. However, evidence can take many forms and wisdom has many roots. All should be valued. Numerical data is only one form.

Some not only revere numerical data but argue that data is only useful if it is derived from the self-declared “gold standard” of randomized controlled trial methodology - for example, the American government’s What’s Working Clearinghouse has this bias. This fanaticism arises from a hyper-rational perspective on human experience. Positivist science can be highly informative, but I cannot imagine why one would not also welcome the insights that have been distilled from experience through more qualitative processes.

At the school level, richer insights will arise through triangulation based on multiple methods than through any one avenue alone. Data should be used to inform our dialogue at every opportunity but it does not speak for itself. Data is created through an imperfect and incomplete recording and codification of experience and only acquires meaning through fallible human interpretation.  

It is in this interpretive process and in the subsequent discussion of what to do about what one has learned that dialogue is required, and this dialogue is the process through which understanding is constructed and commitment to action develops. Data informs the dialogue but does not determine its outcomes. The insights, questions and perspectives that teachers bring to the interpretive process are the source of its power and consequence as much as the data that instigates the dialogue.

So let’s not let data drive us, but also let’s not denigrate it. Professional dialogue is more powerful when it is informed by evidence of all sorts, including numerical data, as well as the intuition and wisdom of experience.

Bruce, Great to connect with

Bruce,

Great to connect with you here.  Recently I have changed my thinking around how we share data, and believe that we should make it more public.  In the school system, we guard it closely, and this, in turn, leads to suspicion as people wonder just what it is we are trying to hide.  While I agree with you that we should not be data driven (and I actually think we are moving past this), I think we should be public about the data.  You are right, it is just data. The richness comes in the conversations that come out of the data. 

Chris, I agree with you.  It

Chris,

I agree with you.  It shouldn't take Wikileaks for the public to know about their schools.  The fear, of course, is that the information made public will not accurately reflect what is happening in schools and may be misinterpreted.  That is a very legitimate concern.  On the other hand, there is no comment more easily taken negatively than "no comment."  The challenge then is to find a way of identifying a potentially meaningful set of data and making it available in a way that can be understood.  Simply unleashing a wave of data would confuse the situation rather than clarify it and would probably reinforce existing perspectives rather than create the potential for more informed conversations.  Unfortunately, I don't know of any examples of effective mechanisms for sharing informative data about the full range of student achievement.  Perhaps one of our readers can direct us to promising practices.