Artichoke

What can recommendation systems in museums teach us about underachievement in school?

It is worth thinking about the detail in Anne Salmond’s opinion piece in this morning’s NZ Herald Anne Salmond Open Entry for Maori a near miss Monday July 6 2009.

In addressing Pita Sharples’ suggestion of open entry for Maori students to universities, Salmond uses research findings from the Starpath Project to make three disturbing claims about education in New Zealand.

Claim #1.  The management of educational data in New Zealand has more to do with the distribution of resources rather than with tracking the long-term success or failure of students.

As a result – “schools are often unaware when bright students begin to fail; or when groups of students (say, Maori boys) begin to follow pathways that lead to failure and early exit.”

Claim #2.  New Zealand educational investment in initiatives aimed at enhancing student achievement are “largely working blind”.  We not only do not know if the investment is making a positive difference we also do not know if the investment is targeting the real problem.  

A result – “leading to many uncoordinated, short-term initiatives (80 in one school that Starpath studied) and a failure to identify those approaches that really work, so that they can be adopted across the education system.”

Salmond also calls for a more careful monitoring on government funded initiatives that claim to enhance learning outcomes for Maori, Pacific and low-income students.

Those initiatives that don't have a positive impact on student outcomes should be dropped, while those that are highly successful should be adopted across the education system.

I find it somewhat disturbing that Salmond feels the need to recommend “dropping initiatives that don’t have a positive impact on student outcomes" – it suggests that this is not currently the case. 

If the Starpath Project has evidence of initiatives that do and initiatives that don’t – I hope they offered this evidence to Anne Tolley before the recent budget  - and that Anne Tolley respected their findings - so that we can be confident that we are no longer funding initiatives that don’t have a positive impact on student outcomes.

Claim #3 – Our qualification system NCEA is so complex that families cannot make wise decisions about which courses to study, which educational pathways to pursue.

As a result, “while most Maori, Pacific and low-income students aspire to gain university entrance (78 per cent in one study), it is too easy for them to find themselves on NCEA pathways that foreclose this option.”

I will admit to being attracted to chaos theory, fuzzy logic and ambiguity in what I read but it unnerves me just a little to realise that the Starpath Project research suggests the New Zealand Government is funding educational initiatives where uncertainty of focus and indeterminate outcomes rule.  In truth it is easier to understand Pita Sharples call for open entry for Maori to universities if you accept that  the current framing/ educational design and funding of the MoE initiatives designed to raise Maori and Pacific achievement is closer to whimsy than anything professionally responsible.

Claim#3 captures my interest this evening.  Our latest budget set aside “$8 million to ensure NCEA assessment tools are of a high-standard and well understood by teachers.” Presumably on the basis that teachers understanding of NCEA assessment leaves something to be desired.  Salmond’s remarks suggest that parents and students similarly lack the understanding needed to make good decisions about NCEA assessment.

It all makes me wonder how we could re-design the NCEA course options available at secondary schools so that students and their parents would find it easier to make wise choices

There is more to it than this of course - Any New Zealand student studying for NCEA can relate instances where they or their friends have been excluded from courses or dissuaded from certain option lines on the grounds that the institutions deems the chance of student success unlikely. 

In New Zealand secondary schools the right to try (and fail) is seldom available.  Like Etruscans divining the future from the entrails of sacrified animals, secondary teachers continue to confidently (and perhaps patronisingly) practice haruspicy on the NCEA course selections of their students.

So parents and students not only need to understand NCEA well enough to make wise choices they also need to understand it well enough to fight the institution for their right to access courses based upon these choices.

Nina Simon in the Musem2.0 Blog has been looking at ways museums can design recommendation systems for their visitors in Designing recommendation systems that go beyond “You’ll like this”  . 

Much of her thinking about customised museum tours can be usefully transferred to educational contexts – where museum exhibits become educational options and courses.  For example Simon’s thinking helps me think in new ways about the design of solutions to help students and their families make sense of the courses and learning pathways available to them.

When it comes to museums, recommendation systems are a natural solution for the problem of the customized tour. How can a museum offer each visitor suggestions for exhibits and experiences that will uniquely serve their interests? There are many lovely example of museums providing quirky tours based on particular interests. For example, The Tate Modern offers a set of pamphlets featuring different tours of the museum based on emotional mood. You can pick up the "I've just split up" tour and wallow in depression, or the "I'm an animal freak" tour and explore your wilder side. And the site I Like Museums lets you find whole institutions of interest based on your preference for trails like "making things," "nice cup of tea," or simply "pigs."

Salmond's call for more careful monitoring of student learning outcomes .... 

Above all, the compulsory education system needs re-engineering. Information systems in schools should be tracking the educational journeys of students, identifying the strengths and potentials of individual students (so that they and their parents get optimal advice), and patterns of success and failure across the student body (so that initiatives are accurately targeted).

... sounds like it could be answered in part by the design principles in the recommendation system Simon suggests for museums – one based upon collaborative filtering   (“like the one used to recommend new songs to you on Pandora or new movies on Netflix?”) –

Perhaps we can design a platform for the monitoring of individual educational journeys in New Zealand – one that could aggregate content about the strengths and potentials of individual students and build it into a Pandora/Netflix like recommendation system – a system alerting students and their families of the educational targets to be met and the course options available.

Then the patterns of success data available might allow students and their families to thoughtfully design learning pathways - pathways that not only meet their aspirations but also extend them to create alternative educational reach.

If school is disturbance, is it virtuous?

Reading Tactical Media by Rita Raley has provided both an escape from the tactical activism expected on the domestic front on a sodden  Sunday afternoon in Auckland and an escape from my current way of imagining the “future” of school.

I enjoy thinking about the future of museums, libraries and school.  They are all institutions that face precarity - uncertainty and challenge - in their current architecture. 

Raley’s critique let me think in a new way about Robert Jane’s questions in  Museums in a Troubled World: Renewal, Irrelevance or Collapse (London: Routledge, May, 2009) cited in Pallazo Strozzi Blog 

 Jane asks

If museums did not exist, would we reinvent them and what would they look like?

Further, if the museum were to be reinvented, what would be the public’s role in the reinvented institution?

 When I replace [museum] with [school] I get a much better start point for thinking about the future of school.

If schools did not exist, would we reinvent them and what would they look like?

Further, if the school were to be reinvented, what would be the public’s role in the reinvented institution?

Raley made me realise that whenever I think about the future of school through questions like Janes  - there is a unacknowledged sense of a permanence of place or places (real or virtual) –

If schools did not exist and I did re-invent them - then in my imaginings I create a spatial identity albeit in some cases a fractured spatial identity – what I mean is there is a sense of permanence in how I imagine school.  

You can see this assumption of permanence – this focus on a space/place is not mine alone - in the provocative thinking in “Witnessing the future”  – and in  “Just the other day I saw the future ...”    or school2.0 

It is interesting that our use of media in education is no different – we seem hell bent on using Web2.0 all that “participatory media” to create Raley’s “ever hardening totems of identity” – both personal and institutional – We use participatory media like attention whores - creating multiple textured “look at me spaces”.

When I read Raley on tactical media the focus is different – her analysis is on the “experiential” – the value of tactical media is to be found in its ephemerality.

Media tacticians challenge even the digital preservation of "the experience" - asking

“How can Tactical Media be preserved and made accessible without altering the value produced by its ephemerality?”  Politics of the Ephemeral: Rethinking the Archive  

This made me laugh for this weekend's media details an instance in New Zealand schools where we see the reverse happening –

Possibly because of our current focus on totemic place in education – our schools are intent on preserving and making accessible their places and space online.

It seems this extends to making claims over the ephemeral use of media for conversation.  

The Sunday Herald newspaper headline reads Dio girls suspended for Facebook comments   –

The irony is that by claiming the right to preserve a selection of their student ephemeral online conversation (by printing out selected conversations (from some but not all Dio students using FaceBook) and by handing these to school authorities); and then by making the ephemeral conversationalists accountable – and by withdrawing access to learning for those students whose conversations were chosen to be preserved by printing, the school has effectively preserved, archived and made the content of those ephemeral conversations accessible to much, much, much, wider audiences than the students themselves could ever imagine or have intended.   

It makes me wonder if the follow up headline will read “School stood down for actions that led to the preservation, publication and digital archiving of the ephemeral Facebook conversations of young people .”  

I am interested in what happens to our thinking about the future of school if we refuse anything that creates Bourriaud’s “ever hardening totems of identity” (p13). 

What happens if we imagine “school” as an experience – a learning experience where learning and the learners themselves are both flexible and ephemeral like the conversations we might hold when walking across a mall.   

“Future School” becomes an experience where afterwards there is little material trace – a concept where “living memory” rather than “products of learning” dominate our discourse.

When school is imagined as “nomadic” experience, then pedagogy becomes a “deliberately slippery and heterogenous practice”?

Raley describes the categorical unity of tactical media as “disturbance”.  What if we understood “school” as disturbance?

Citing Geert Lovink and “The Next Five Minutes” (N5M)  festival of media arts and politics - Raley argues that tactical media is intended to  disrupt dominant ways of thinking so that critical thinking can occur.

What if we understood “school” as any open to anyone at any time experience, where critical thinking can occur?

It is this inclusivity and flexibility of tactical media – that is powerful in reimagining “school” in this way.

Can “school” be imagined as a process – as a “tool for creating temporary consensus zones based on unexpected alliances”.  

And all this makes me wonder will our future questions about “school” reject notions of does it work? – or how well have the learners in an identified  physical or virtual space met “national standards”? – or Greg’s fear that the place of school might be ladder ranked in league tables.   

We would ask instead if the experience is virtuous?

 Virtuosity – described by Virno as “activity which finds its own fulfilment (that is, its own purpose) in itself, without objectifying itself into an end product or an object that survives the performance. p29

Does the experience - “future school” - strengthen interpersonal relationships in society?  

Raley cites Bourriaud who suggests ... the role of art is “learning to inhabit the world in a better way” p27

 So perhaps “future school” can be an experience rather than a place – and we can understand “school” as we do art – as something transitory, precarious, and uncertain that helps us learn how to inhabit the world in a better way.

 

Crack learning, the achievement gap and Sisyphean struggle.

Pruned Blog’s  "The Crack Garden"  post captured my attention right from the start –

 “The interventions into the site of The Crack Garden were primarily actions of removal rather than the addition of new layers and material. By eliminating portions of the existing concrete and exposing the soil beneath, potential is released, and new opportunities for the garden arise.”

“The design is conceived as an intervention that functions as a lens, altering perception of a place rather than completely remaking it.”

This made me think of “crack learning” and how we might understand learning based on actions of removal rather than by constantly adding new layers and materials to our schools, classrooms and students.

I wanted to ask ..

What would happen to learning if we removed "the din"?

“We approach our technologies through a battery of advertising and media narratives; it is hard to think above the din.”  Turkle, Sherry. (Ed.). The inner history of devices. Cambridge, MA: MIT Press, 2008. p4

What would happen to learning if we removed the expectation that "progress" requires unrelenting change and innovation  

One of the most critical problems our schools face is ... “not resistance to innovation, but the fragmentation, overload, and incoherence resulting from the uncritical and uncoordinated acceptance of too many different innovations” Fullan & Steigelbauer 1991 p197

What would happen to learning if we removed "the rush", if we slowed down, learned how to see and took time to realise that all things connect?

Crack gardens/learning made me think of a return to; slow pedagogy, to observation (see think wonder), to Geetha Narayanan like learning spaces squeezed into cracks between city buildings, to looking carefully at exploring and knowledge building around the local (existing) rather than all that costly rip snorting through the screen activity we favour to get to the global, to looking at ways to discover and develop all learning identities of the child rather simply addressing learning identities for the 9am to 3pm child.     

And I wondered if the coherence provided by the stripped back nature of "crack learning" would provide new opportunities for understanding individual potential.    

Whenever I read the latest policy initiative aimed at reducing disparity in New Zealand schools I have high apple pie in the sky hopes ... I imagine the MoE policy makers in Wellington as Pratchett’s “great minds”

These are great minds he told himself.  These are men who are trying to work out how the world fits together, not by magic, not by religion but by inserting their brains in whatever crack they can find and trying to lever it apart.  p199 in Pyramids by Terry Pratchett

and I hope that this time round we will be brokering something that makes a real difference to New Zealand’s alarmingly disparate achievement outcomes.   

That is undoubtedly why it was a little discomforting to read Gladwell's "Outliers" over the weekend.  Now Malcolm Gladwell has been accused of cherry picking his references in Outliers but I could not help but be affected by his description of Karl Alexander’s five year longitudinal analysis tracking the city of Baltimore’s profile of results for 650 first graders on the Californian Achievement Test math and reading skill exams. (pages 255 to 259)

Reading Gladwell made me fret that all our MoE sanctioned interventions to reduce our achievement gap are perhaps a Sisyphean struggle – made me think that perhaps we are doomed to always struggle because in targeting schools we are targeting the wrong intervention.

As Gladwell frames it, when we have disparate achievement outcomes from kids with different backgrounds we are tempted to attribute causality to either

1.  Kids from background X do not have the same inherent ability to learn as kids from background Y.

2.  Our schools are failing kids from background X.

This is certainly what has happened in the conversations about disparity in New Zealand – option 1 – deficit thinking - is rightly rejected leaving us with option 2 – our schools are failing [insert gender, socio economic status, ethnicity] students. Our latest solutions to not failing [insert gender, socio economic status, ethnicity] students is focussing on improving teacher student relationships, engagement, and feedback.

Gladwell makes me ask ... when we focus on reducing disparity in learning outcomes by changing the stuff happening in schools have we misidentified the contribution school makes?   

If for the purposes of this post I accept that The Californian Achievement Test measures something valuable in terms of learning outcome [and I know this may be unwarranted]  ... then using Alexander’s data below I can suggest that the achievement gap between students from “rich” and “poor” homes is exacerbated by attending school.

Californian Achievement Test Data from start of school year (June)

Socioeconomic Class	1st Grade	2nd Grade	3rd Grade	4th grade	5th Grade
Low	329	375	397	433	461
Middle	348	388	425	467	497
High	361	418	460	506	534
Achievement gap between low and high	32 points	43 points	63 points	73 points	73 points

 

Gladwell next reveals additional results from the same CAT testing carried out at the end of the school year (September) – This testing that excludes the summer holidays – and allows quite different conclusions to be drawn about the same group of students.

Socioeconomic Class	After 1st Grade	After 2nd Grade	After 3rd Grade	After 4th grade	After 5th Grade	Total - Cumulative classroom learning
Low	55	46	30	33	25	189
Middle	69	33	34	41	27	214
High	60	39	34	28	23	184

     

It seems that by testing at the end of the school year - the data showing “within school” learning gains between children from low and high socioeconomic backgrounds are not as “gappy” as we first imagined.

Which causes us to ask is “gappiness” due to what is happening in classrooms or is “gappiness” due to  what is happening outside of classrooms?

Is Glawell right?

Should our focus on reducing disparity look at the effect on learning of time spent outside of school rather than what happens within school?

To ask ...Does the break in schooling over the summer holidays differentially affect learning outcomes for children from lower, middle and high socioeconomic homes?

Look at Gladwell's data comparing student reading skill test scores before and after the summer break.

Class	After 1st	After 2nd	After 3rd	After 4th	Total
Low	-3.67	-1.70	2.74	2.89	0.26
Middle	-3.11	4.18	3.68	2.34	7.09
High	15.38	9.22	14.51	13.38	52.49

 

Now Gladwell, using Alexander’s data suggests that ..

 “When it comes to reading skills poor kids learn nothing when school is not in session.  The reading skills of rich kids by contrast, go up a whopping 52.49 points.  Virtually all of the advantage that wealthy students have over poor students is the result of differences in the way privileged kids learn   when they are not in school.”    p258

Leading me to wonder - Do we simply need to increase the number of days students attend school to reduce disparity?

Trying to validate Gladwell’s claims led me straight to Hattie’s Visible Learning meta-analyses where I checked out the number crunching on Summer Vacations (d=-0.09) p80 and 81.  

Hattie’s metanalyses on summer vacations confirmed that students “lost some achievement gains over the summer” and that “middleclass students appeared to gain on grade level equivalent reading tests over summer compared to lower class students".  And he also notes that the “negative effect of summer did increase with grade level.”  

However, Hattie doesn’t call this like Gladwell does - he suggests instead that the magnitude of these effects when compared to other achievement influences “are minor indeed”

Hattie concludes

“It may be that if teachers were more attuned to the proficiencies that students bring into their classrooms, then the first month of the school year could be used to recapture the losses from the school break reasonably quickly.”

I hope Hattie is right because, whether it is happening within schools or outside of school over the summer break, we have an awful lot riding on the inequality we are building into New Zealand society.

 

Fear is fungible.

I felt like an extra in Richard Scarry’s Busy Busy World yesterday morning.

Filling up with gas at the local garage I shared the forecourt with a huge red fire truck packed with firemen. Given the cell phone risk signs posted all around I was disconcerted to see one of the crew leap down from the truck and start texting in the pump bay.

Lowly Worm insight: The public’s perceived risk of taking or making any calls in a gas station forecourt is less than the actual risk.

Leaving the gas station I went to our local GP to pick up a script for one of the kids living off the corridor, only to be greeted by a laminated notice and a closed door.  To paraphrase - “Before you open door pause and consider whether you are feeling dodgy in a way that could be ‘flu like – if this is possible remain outside and Tom Jones like “knock three times” at a window to alert us inside of your possible H1N1 incubating self outside.”

Lowly Worm insight: The actual risk of catching something serious in GP’s waiting room is high enough to abandon those too enfeebled to knock three times and those too short to reach the window but not so high as to exclude those without the literacy and or the language to make meaning from the text.  [that is the 380,000 New Zealanders without the literacy to understand the instructions on a fire extinguisher and anyone else without the ability to make meaning from English]  

I then visited a friend who had a Richard Scarry “Great Pie Robbery” experience on Wednesday, where all her electrical goods stolen.  The home and contents insurance was up to date, the burglary happened when she was out, the forced windows and doors were quickly repaired and the police visited and detectives Sam and Dudley dusted for fingerprints.  But she feels vulnerable in a place that the week before had been her refuge.

Lowly Worm insight: The actual risk is not important – the perceived risk of Horace Wolf and Croaky Crocodile revisiting the house creates high levels of anxiety and outrage.  

To understand how we use risk management and risk communication, [or how it uses us] I reckon you cannot better reading Max Brook’s World War Z  but reading Peter Sandman  Talking about “Risk Communication Before and During Epidemics”  provides a framework that has helped me analyse our response to “swine ‘flu”, national testing and all the other stuff that frightens us. 

For example, I didn’t know that “how much harm a risk does” and “how upset people get” has a correlation coefficient of only 0.2. 

Those of you who remember your statistics know you can square a correlation coefficient to get the percentage of variance accounted for: If you square 0.2, you get 0.04, or 4% of the variance.
That is, the risks that kill people and the risks that upset people are completely different. If you know that a risk kills people, you have no idea whether it upsets them or not. If you know it upsets them, you have no idea whether it kills them or not.

Sandman frames these two as “hazard” and “outrage” and goes on to show that

“It doesn't seem to matter what your measure of harm is. Whatever your measure of harm, across a wide range of risks, the correlation between how much harm [a risk is] going to do and how upset people are going to get is this absurdly low 0.2 correlation.”

He elaborates on this research to show that “the correlation between hazard and perceived hazard is also very low, but the correlation between outrage and perceived hazard is very high.”

This analysis interests me as I try to understand the high levels of outrage at the Minister Anne Tolley’s suggested introduction of National Standards – outrage that is based on the “perceived risk” of league tables.

Sandman is smart ... he knows that correlation is not necessarily causality ... but that when it is ... then the directionality is contestable. 

Now, as soon as you have a high correlation, of course, what you want to know is: What's the direction of the causality. That is the question we're asking when we look at the high correlation between whether people get upset and whether they think [a risk is] dangerous. Are they upset because they think it's dangerous, or do they think it's dangerous because they're upset?
That's an important question, because if you want to manage the system, you have to know which one is the cause and which one is the effect. You don't want to be in the awkward position of trying to manage a cause by manipulating the effect. That's not likely to work. So you need to know the direction of the causality. This is much studied, and as usual in social science, it turns out to be a cycle, but one of the arrows is very robust and the other arrow is very weak. The strong arrow is from outrage to hazard perception. That is, for the most part, it is not true that people are upset because they think [a risk is] dangerous; it's much more true that people think [a risk is] dangerous because they're upset.

And I love the way he explains the negative

The same is true in the negative: It's not true that people are calm because they think [a risk is] safe; it's much more true that people think [a risk is] safe because they're calm. It follows, [therefore], that if you want people to think [a risk is] dangerous, then you'd better get them upset, and if you want them not to think [a risk is] dangerous--if you want them to think it's safe--then you need to calm them down.

If the outrage is the driver ... and the hazard perception is the result the message to Anne Tolley is quite clear – introduce communication strategies to reduce the outrage. The message to the NZEI is equally clear – introduce communication strategies that increase the outrage. 

However, it was the second part of Sandman’s analysis that I most enjoyed reading.

Here he talks about how risk management is also used by those with institutional authority to shelter the public from high risk information on the grounds that the ensuing panic will be more damaging than the risk itself.

All those “damage control”, “on a need to know basis” conversations that obscure transparency within institutions

The Argument: If we scare people this fear will escalate into panic.  Panic will exacerbate the situation.

However Sandman counters by arguing that our experience in New Orleans and 9/11show that we overrate panic. He distinguishes “feeling panicky” from “acting panicky” and he introduces a new idea – that of “panic panic”

Panic, in short, is rare. But official "panic panic" is common. That is, officials often imagine that the public is panicking or about to panic. And in order to allay panic, officials sometimes do exactly the wrong thing from a crisis communication perspective: They withhold information, they over-reassure, they express contempt for public fears, etc.

And this is a problem because in risky situations fear is tolerated, has an “adjustment reaction” which because it is a rehearsal has positive outcomes in terms of appropriate future reaction ...

• The underestimation of the frequency with which fearful people rise to resilient, pro-social, and even heroic behavior. We had ample evidence of that in 9/11, and I won't belabor the point.
• The failure to recognize the positive value of fear in encouraging preparedness, vigilance, tolerance of inconvenience and expense, and so forth.

And it seems contrary to what we commonly hold as true - that communication that makes people fearful in risky situations is a good thing.

So it's completely inconsistent to say we want the public to prepare, [but] we don't want the public to be frightened. The main incentive for people to prepare is becoming frightened.

The bit I liked best was Sandman’s notion that fear is fungible.

Greenpeace wants us afraid of genetically modified food, and the Christian Right wants us afraid of gay marriage, and I want us afraid of H5N1. You should not think of any of those three as trying to make people more afraid. What we are doing is competing with each other for our slice of the fearfulness pie.

Which brings me right back to thinking about  – what are the things that are currently competing for a slice of the fearfulness pie in education?

We like our flour canisters larger than our sugar.

Michael Doyle often nudges me into new thinking ...  Take this thought from his Science Teacher Blog.

 Humans like boundaries. We like borders and lines and straight thoughts. We like to categorize and sort. We like our flour canisters larger than our sugar. The Edges of the Sea Post

Michael Doyle is onto something .... I like thinking about boundaries of all kinds

This morning in Ponsonby Rd I was captured by the number of people I saw balanced on the strip of white paint that is only a few mm thick.  Clutching biodegradable trays loaded with cups of takeout coffee they were watchful in the precariousness of their boundary - waiting for an opportunity to swap their boundary for another albeit more elevated boundary in Auckland City  - the pavement strip. 

Those boundaries marked by white painted surfaces are not protected by armies, but they are marked by agreement and convention. The boundaries of white paint road markings might seem a little different from the boundaries of nationalism but many of the same rules apply –

John Stewart Mill describes it “nationalism” as when

“a portion of mankind, is united among themselves by common sympathies, which do not exist between them and any others.”

Those coffee clutching pedestrians standing on the painted strip boundary are united by common sympathies that do not exist between them and the motorist barrelling down Ponsonby Rd trying to get to work.  The motorists occupying a different boundaried surface have common sympathies that are not shared with the Ponsonby Rd already at work - office gopher.  

Common sympathies always make me anxious -  they suggest the existence of common antipathies.
Looking at “why we belong” requires us to identify those others that don’t ... and this is probably why the boundaries of nationalism make me anxious – as do the other boundaries people use to form collective identity.

I guess I can never escape that boundary ideas  that Tom Lehrer so cleverly captures in Who’s Next.

Those coffee clutching boundary perchers in Ponsonby Rd would have to re- position themselves if they attempted the same activity on the Auckland Harbour Bridge -  The lane boundaries on the harbour bridge are movable boundaries – great hunks of interconnecting concrete that make me think of global warming and those national boundaries drawn across glaciers that are not as immutable as we pretended when we talked about them in school.

Italy and Switzerland are preparing to make—or rather to recognise—alterations to the border that runs through the Monte Rosa massif of the Alps. Despite what romantically minded locals may say, the name of the massif has nothing to with the pink blush its peaks acquire at sunset. It comes from a dialect word meaning glacier.

Recognising that global warming will make any line based on the watershed of a glacier temporary, the understanding with Austria has for the first time introduced the concept of a movable border. Experts from both sides will be empowered to alter it at regular intervals. Until, presumably, the glaciers disappear altogether. A movable border

All of which makes me think again about the boundaries we have built between individual schools in New Zealand.

In education the boundaries and borders we create with our points of difference, our principals as robber barons, our school based curricula etc allows us to measure what fits and what doesn’t – allows us  to discriminate based upon assessment – both formative and summative – allows us Anne Tolley like national standards – allows us NZEI feared league tables.

In the L@S09 keynote Andy Hargreaves talked about creating “culture of collective responsibility” in education  – a culture based on

“sampling rather than a politically distorted insistence on testing every student” – a culture where “schools must support and learn from each other, become collectively responsible for all the children and youth in their city or community and commit to systems and dispositions where the strong help the weak.”- Hargreaves, A., and D. Shirley The Fourth Way. Educational Leadership. October 2008. p60

Collective responsibility – requires movable boundaries – responsibility without boundaries - it is a novel notion in a world predicated on right indexed to the might of a nation state, right indexed to the “successful school”,  and right indexed to the powerful individual. 

Collective responsibility is a certainly a novel notion in New Zealand education where we are encouraged to identify and market our boundaries with other schools as points of difference, to establish zones and where the NZC sees us exhorted to develop our own curriculum boundaries – aka our own school based curricula.

In such a boundary based educational landscape we should not be at all surprised to see principals talking like robber barons about owning the schools that employ them; to see principals passing around personal business cards that could be mistaken for those of our most entrepreneurial real estate agents; and to see as a consequence of this boundaried thinking - teachers and principals made anxious and defensive by talk about national standards and measurement targets.  

The boundaries of teaching and learning allow us vainglorious ambition but they also allow measures of boundaried accountability –

“Humans like boundaries. We like borders and lines and straight thoughts. We like to categorize and sort. We like our flour canisters larger than our sugar.”

... and yet it is the wantonness of learning sans boundaries that we need to imagine and make real.

When I imagine learning without boundaries in schools across New Zealand – I imagine that all that collective responsibility will be not unlike lovemaking, not unlike art

My feeling about technique in art is that it has about the same value as technique in lovemaking. That is to say, heartfelt ineptitude has its appeal and so does heartless skill; but what you want is passionate virtuosity. — John Barth

So how do we get movable boundaries and collective responsibility in New Zealand education – how do we achieve passionate virtuosity? How do we raise achievement outcomes for all young New Zealanders?

Two straight lines crossing over in the middle.

The Easter holiday represents a pause, a chance to catch breath and enjoy the texture of my world ... I have mostly caught up on promises made and promises broken  – and I am at last free to think about some of the approaches to building deep maths understanding and mathematical innovation  observed in the secondary schools I worked in this term - and from there the relationship of creativity to educational policy.

I know it’s only an impression but it seems that ever since the launch of the New Zealand Curriculum, we have been awash with calls to “bring back” creativity in classrooms.

"Creativity is now as important in education as literacy, and we should treat it with the same status". — Ken Robinson

I always smile grimly at this quote from Robinson - our adult literacy results - "380,000 Kiwi adults' whose “literacy skills are so poor they would be unable to determine how to use a fire extinguisher from the instructions written on the bottle.” (Prime Minister Helen Clark 2007) suggest that Robinson is not aiming high enough if he wants to lift creative potential/capital through educational endeavour. 

Still it seems the “call creative” is the new cry of the educational wild.  And it has grown louder and increasingly wilder with our Education Minister Anne Tolley’s call for the introduction of national standards. A call, which despite Tolley’s best efforts, (see Questions for oral answer 9 April 2009 below), is being widely and possibly unfairly represented in New Zealand schools, as a call for the introduction of league tables.

Hon ANNE TOLLEY: Firstly, I did not say in the Christchurch Press that I had a favourite. I want to make it very clear to that member that the Ministry of Education will not be publishing league tables. I will say that again to the member. The ministry will not be publishing league tables. This Government will be using the information that we gather from the national standards policy in a responsible way, to help to lift the literacy and numeracy standards of young New Zealand children. That is what this policy is about.

... and it is hard to counter the need for improving the ways in which we understand student achievement given the ERO statement that despite all the measuring going on in New Zealand schools - 56 percent of them were not using worthwhile achievement data to look at their student learning outcomes. 

Hon ANNE TOLLEY: I have seen a report that suggests that a number of schools are using formative assessment, and the data produced by it, to help their students. For those schools the national standards policy will complement the excellent work they are already doing. The same report from the Education Review Office states that 56 percent of schools were not using worthwhile achievement data. The national standards policy is about ensuring that those schools do use good assessment practices to help our young New Zealanders to read, write, and do maths at a much higher level than they do at present.

ERO’s 56 percent statistic reminded me of John Hattie’s key question at the end of Visible Learning.

"The key question is whether teaching can shift from an immature to a mature profession, from opinion to evidence, from subjective judgements and personal contact to critique of judgements." P259 Hattie, J. 2009.  Visible learning

In the New Zealand schools I work with there has been a lot of talk about national standards leading to league tables which if I take taxi drivers as a measure - seems a prospect that bothers educators much more than it does the rest of the country. 

There has also been the suggestion that creative endeavour and creativity, which is widely and for the most part approvingly received by teachers, will be lost if we adopt national standards.

To sum up these lunchtime conversations: we have Tolley’s - National standards  “a very bad thing – don’t let them in” , Robinson’s - Creativity in classrooms – “ a very good thing – let’s bring it back” 

The weird thing about any discussion about “creativity” in New Zealand is not the “let’s bring it back” sentiment – though that’s kind of weird since it never went away in a lot of the places I work - it is the creative conversation default setting.

When you join a conversation about creativity in education in New Zealand  - instead of finding yourself interrogating what creativity might mean in 2009, or evaluating how we might judge creative practice in New Zealand classrooms - you end up revisiting the writings of Elwyn Richardson in the 1950’s early sixties.

This default to the activities of just one male teacher and his mostly self described creative practices from nearly sixty years ago makes me suspicious.

I’d like to ask -

Why are we privileging Richardson’s descriptions of his classroom experiences?

Why was it that Richardson was published and promoted and other creative educators didn’t and aren’t?    

I ask this because I do not find it plausible that in a culture that is built upon “number eight fencing wire innovation” Richardson was, (or for that matter is), the only educator designing worthy creative adventures in learning in New Zealand classrooms.

I don’t find it credible that nothing creatively worthy was going on in New Zealand classrooms before Richardson, at the same time as Richardson, or since Richardson. 

And I also wonder why - if creative teaching did occur in the past and does occur in the present - we don’t we refer to it today? Why we don’t appear to even know about it?

You might jump in here and declare Richardson an iconic educator on the basis of his writings and the memories of teaching colleagues – but it does not explain why in our discussion and commentaries we fail to reference any creative educators before or since Richardson – For instance I have never climbed a mountain but I know the names of more New Zealand mountain climbers than I do New Zealand teachers famous for their creative teaching practice or creative student outcomes.

It makes me wonder if anyone at NZCER has done a historiography of creativity and creative practice in New Zealand education; has anyone looked at the way creativity has been represented and understood and the way creativity has been written about?

It seems more likely that we have neglected and undervalued the stories and practices of other creative teachers in New Zealand in a way that we haven’t done for mountain climbers both before and after Hillary.

If we did privilege Richardson in the past and if we continue to do so  - is this because of gender, or ethnicity, is it simply because our current day commentators are mostly contemporaries of Elwyn, or is it something else?

I suspect we need to clarify “creativity” and to more carefully distinguish creative acts of teaching from creative achievement outcomes if we are to progress.

To ask –
What are the assumptions we make when we identify a teacher and or their practice as creative?
Does creative teaching result in creative achievements by students who in turn become creative adults?
Should it?
Is a measure of the success of a creative teacher the measure of the creative success of their students?
Should it be?
What is the measure of the success of a creative teacher?
Without a measure of achievement success how can a creative teacher who is teaching creatively tell what they are doing, whether it is going well or not, and what they should do next?
Ditto for the student in a classroom with a teacher who identifies themselves as creative or who describes their teaching strategies as those developing creative outcomes.
Who do we judge to be creative adults in New Zealand today? 
Who taught the New Zealanders we judge as creative today, and how did they teach them?
What did these creative New Zealanders remember learning from their teachers that they believe enhanced their own creative abilities?

If we accept Ken Robinson’s definition of creativity

“I define creativity as the process of having original ideas that have value.”

Then J.J Thompson  must surely have been a highly successful in enhancing creative thinking in the science students he taught 

“I’d like to cite the example of J.J. Thompson, who had nine Nobel prize-winners, thirty two fellows of the Royal Society, and eighty three professors of physics among his pupils … Yet when you look – what were his rules of thumb?  How did he teach? You find practically nothing.”(Dedijer, 1966 cited in Root- Bernstein 1982, p.200) 

 And Bill Manhire -  jumps out as a New Zealand educator who is highly successful in enhancing creative writing in the writers he teaches.
.
Perhaps the researchers of a historiography of creativity in New Zealand education could start by asking the contributers to Manhire at 60: A Book for Bill (ed Fergus Barrowman and Damien Wilkins; 2007, VUP) to identify “How did he teach?” – "what were his rules of thumb?"

Published in a limited edition of 500 copies for Bill's birthday, this is an anthology including memoirs, essays, poems, stories and extracts from work-in-progress which have been contributed by over 40 writers who have been inspired by Bill as writer, teacher and friend: Michele Amas, Barbara Anderson, Angela Andrews, Hinemoana Baker, Fergus Barrowman, Rachel Barrowman & R.A.K. Mason, Jenny Bornholdt, William Brandt, James Brown, Kate Camp, Catherine Chidgey, Geoff Cochrane, Nigel Cox, Jim Crace, John Davidson, Kate De Goldi, Stephanie de Montalk, Ken Duncum, Laurence Fearnley, Cliff Fell, Bernadette Hall, Dinah Hawken, Janet Holmes, Ralph Hotere & Mary McFarlane, Keri Hulme, Eirlys Hunter, Andrew Johnston, Elizabeth Knox, Robyn Marsack, Paula Morris, Gregory O’Brien, Vincent O’Sullivan, Emily Perkins, Chris Price, Jo Randerson, Michael Schmidt, Iain Sharp, Elizabeth Smither, Kathryn Walls, Peter Whiteford, and Damien Wilkins.  http://www.bookcouncil.org.nz/writers/manhireb.html

Those staffroom claims – both simplistic and expansive, that our current educational focus on accountability, assessment and compliance are dangerously hampering the innovative and original classroom teacher need to be more carefully unpacked.

Because I don’t know that they can reliably or validly be used to reject the Minister of Education Anne Tolley’s push for a national standards policy.

Hargreave’s I’m stuck to the floor keynote address at L@S09 - “The Fourth Way” revealed a man and a mind that was funny, provocative and ever so smart – a great pick by the conference programme organisers to launch New Zealand teachers at the start of our school year.  His claims over the freedoms and innovation The First Way afforded teachers seem kind of relevant here

The First Way:  Over dependence on the state.
“These policies provided unprecedented levels of support for the poor, but they also fostered long term state dependency without providing any real foundation for long term civic engagement.  The First Way granted state professionals, including educators, considerable freedom. In education, it fostered innovation but also allowed unacceptable variations in quality.”

Rather like topology we shouldn’t be looking at the objects involved but rather at the ways in which they work together.

Creativity and creative endeavour is advanced by deep learning, and conceptual understanding and if national standards mean knowing what you are doing, whether it is going well and what to do next then it would seem that national standards and creativity may well be two straight lines that cross over in the middle.

One supports the other and X marks the spot.  

Teaching: working for the government and stealing chickens.

People were usually quite pleased to see them.  They taught children enough to shut them up, which was the main thing after all.  But they always had to be driven out of the villages at nightfall in case they stole chickens. The Wee Free Men Terry Pratchett P26

You won’t find Pratchett like imaginings about possible futures for teaching and learning in Time Magazine’s Annual Special Issue on “10 ideas changing the world right now”.

In fact you won’t find anything about teaching and learning in the ten ideas listed – except perhaps in the “seeking security in risk adverse careers is the new cool” suggestion woven into  #1 Jobs Are The New Assets –

“In this new era, a predictable salary is more appealing than the chance of scoring big with bonuses and stock options.  And having a government job – one of the last bastions of security – looks even better.” P 27 

 

The Annual Special Issue "10 Ideas Changing the World Right Now" claims - The global economy is being remade before our eyes. Here’s what’s on the horizon

• WHY YOUR JOB IS YOUR MOST VALUABLE ASSET
• REPURPOSING THE SUBURBS
• SURVIVAL-STORE SHOPPING
• BIOBANKS: SAVING YOUR PARTS
• NEED LAND? RENT A COUNTRY
• THE NEW CALVINISM
• ECOLOGICAL INTELLIGENCE
• AMORTALITY: FOREVER YOUNG
• AFRICA: OPEN FOR BUSINESS
• REINVENTING THE HIGHWAY 

The irrelevance of ideas around changing education (teaching and learning) in Time Magazine’s “changing the world” list captured my attention.

It worries me that children and their learning seem so easily excluded from these imaginings over remaking the global economy.

Are we as teachers so professionally predictable that when global remaking is going on we have nothing new/relevant to contribute?

Has our secure government salary meant that “paradigm shifting” edu_conferences and edu_ un_conferences – our “future focussed Web2.0” edu_blogs and edu_twitter streams –our “best evidence synthesis based” edu_professional learning communities – and our “knowledge waved” edu_policies and edicts allowed us a false sense of our own relevance?

Has being pre-disposed to risk adverse behaviours – behaviours like choosing to: train for a job with a predictable salary, apply for a job with a predictable salary, and work in a job with a predictable salary – excluded us from 10 ideas changing the world right now.

How would Time Magazine feel about us/ write about us if teachers were filchers of Black Orpingtons  ...  rather than Chomskiian ”tamers of the young of the bewildered herd”?

How might we alter our pedagogical approach if we thought we were working in uncertain careers in perilous times?

How might we alter our pedagogical approach if what we could offer was not needed every day?

How might we alter our pedagogical approach if what we could offer was only occasionally useful?

“The teachers were useful there.  Bands of them wandered through the mountains, along with the tinkers, portable blacksmiths, miracle medicine men, cloth pedlars, fortune-tellers and all the other travellers who sold things people didn’t need every day but occasionally found useful.  The Wee Free Men P26

In a remade global economy wandering teachers will undoubtedly have stalls offering Pratchett’s -

The Wonders of Punctuation and Spelling

1.  Absolute Certainty about the Comma.
2.  I before E Completely Sorted Out.
3. The Mystery of the Semi-Colon Revealed.
4.  See the Ampersand (Small extra charge)
5.  Fun with Brackets.

Will accept vegetables, eggs, and clean used clothing.

But the temporarily erected tent stall in The Wee Free Men – A Story of Discworld that interests me the most is the one with the sign outside that reads - “I can teach you a lesson you won’t forget in a hurry”

If I was a wandering teacher – living on baked hedgehogs and stealing chickens - whilst the global economy is being remade - what would I offer as a lesson that wouldn't  be forgotten in a hurry?

"Take me to your leader", truants and the classroom as a space between things.

Whenever I watch those “take me to you leader” science fiction movies I am less worried by the essentially bipedal nature of the aliens than I am by the large number of kids in the crowd scenes who must obviously be truanting from school.

You have just got to know that if aliens landed in New Zealand between the hours of 8.30am to 3.30pm they wouldn’t realise that earthlings in New Zealand had a nymph form at all – all our 5 to 18 year olds are shut away in classrooms – the irony in this sequestering of our young is that we shut them away from the outside world in classrooms so that they can learn about the world outside the classroom.

I have always argued - that a significant weakness of doing all our teaching and learning in schools and classrooms is their very isolation from the real – I believe that one of the reasons schools cannot ever deliver what they promise in terms of learning what it is to be human is because of their disconnect from the real. Our schools are places identified in part by their isolation and the way they remove a large chunk of the 5 to 18 year olds in society from everyday life between the hours of 8 to 4pm each day.

I like having the things I hold most dear shaken and undermined by new thought. Which is why I am enjoying Sherry Turkle’s start up essay in The Inner History of Devices.

Turkle’s essay looks at the relationship between an ethographer and the subject. In exploring this relationship she refers to Virginia Woolf’s description of “the writer’s space’ as “a room of one’s own.”

I am always looking out for metaphors that help me understand teaching and learning. If learning at school is all about learning how to reflect on the strengths and weaknesses of our own thinking about the curriculum of what it is to be human, then there is something in Woolf’s “room of one’s own” sentiment, that might help me better understand how classrooms might work.

“In its safety and containment, writers open themselves to the kind of reflection, [.... ] where feelings don’t conform to a predetermined script.” Turkle The Inner History of Devices P6

“Safety” and “containment” are good descriptors for the physical classrooms we have dotted across the country. And providing learning experiences that encourage students to reflect upon their learning in a way that does not “conform to a predetermined script” seems a worthy endeavour.

Turkle develops Woolf’s idea further when she describes psychoanalysis as needing

“a space between things,” “a space that is “transitional,” a space removed from everyday life; it is a liminal space located between things. There, relationships are not based on time worn hierarchies but develop in new meaning filled encounters.” Turkle p6

This confronts and undermines my thinking. For Turkle argues that a space removed from everyday life is essential for relationships to develop - in meaning filled encounters between ethnographer and subject.

Can I extend this argument for a space removed from everyday life to classroom spaces where relationships can develop – in meaningful encounters between teacher and student/s?

I was interested in Turkle’s description of how an ethnographer creates environments where “what is there can emerge” and in thinking how this differs from what we do as teachers in classroom environments.

Turkle emphasises that

“Ethnography is not a passive practice; understanding the experience of others demands active listening. Nor are ethnographers trying to get something out of their subjects through clever questioning. They are trying to create an environment where what is there can emerge.”

Active listening versus “telling,” and getting rid of all that initiation elaboration response teacher clever questioning – now that would be an interesting change in classrooms.

I have to think about this more deeply because developing meaning filled encounters is such a sought after outcome in classrooms.

P.S. Turkle’s distancing from the real reminds me of Sidorkin’s dialogue thinking. I have always loved Sidorkin’s argument for a removal from the real in his three wine dialogue theory where wine allows deep conversation and dialogue by removing the individual from group identity.

The first drink conversations establish a group, they provide “a common text, a shared experience, an initial conversational event. ….It establishes a common set of references, a shared language for the following conversations.”

The second drink conversations occur when individuals within the group “…. challenge, deconstruct, actively agree or disagree with it, they commend and ridicule. We understand things by breaking them, turning them upside down, taking a bite, or dissolving with saliva—literally with edible objects, figuratively with texts. The idea is to enmesh the self into the text, to break down the whole, to salvage whatever is left from a common meaning for individual sense-making. We understand by trying to co-author the text, to interpret it, and to offer our interpretations to those with whom we listened together. “

By the third drink – the conversations see individuals escaping the group identity “People take things lightly, they give up on convincing each other, they talk with their emotions, while often pretending to make sense of each other. …Talking nonsense, and having a good laugh about it is obviously better than endless discussions and polarization of opinions. When people miss the third drink phase, their conversation ceases to be a source of happiness, and becomes a beginning of their misery. For different opinions to coexist, there needs to be a nurturing broth of a carnival, where all things seem to be possible, and all become laughable.”

There is obviously more to creating space between things than I first thought.

Welcome to the Learningdome: Thinking about school as a theme park

Bill, raises some good questions about physical design of spaces for learning (specifically the ability for teacher and students to give and receive feedback Hattie effect size d=0.72).

One problem is that computer labs and the way they are organised in schools are often not very good environments for the nuanced development of feedback. ie. since the computer lab is an expensive, time restricted resource then there is great pressure on the teacher to keep students on computer tasks and not interrupt those tasks for other matters. Also the physical layout of some computer rooms is poor, especially those with computers in rows. Best layout is all computers around the walls. I would argue that if students had netbooks 24/7 then very different interactions could develop in computers use. So, perhaps Hattie's research confirms that the computer revolution hasn't happened yet?

His questions made me think about what school might be like if we could align all the teaching interventions with effects sizes that made the most difference, with the learning spaces that best supported them.

And then imagining what this might look like in a physical environment, in a virtual world like Second Life and in a game based simulation like WOW.

We have an anxiety about panopticon-like experience, where everything is designed, observed and measured, forgetting we are all chemical, we are unsettled by the idea that the complexity of human experience can be reduced to a database

 Fantastic Journal has a great post Welcome to the Pleasuredome that starts to tease out how we understand designed environments and theme parks. The post exposes our fears and hopes for virtual worlds in the context of the sci fi movies Westworld, Logan’s Run and The Truman Show.

A recurrent anxiety about theme parks is that this carefully controlled environment denies us the ability to act independently. To visit somewhere like Disneyworld is to take part in a minutely choreographed experience where little deviation is allowed from the script. This is most apparent in the rides themselves where the same experience is repeated for each guest exactly, ad infinitum, like a Fordist approach to having fun. But it also occurs in the landscape between the rides where boredom, lethargy or other forms of deviant behaviour is frowned upon. Cast members constantly coax visitors into immersing themselves more fully into the concept. These cast members, like the technicians in Westworld, have their own separate circulation system from the public, appearing fully dressed and smiling as if from nowhere.

The description could well have been written about school. 

Schools are carefully controlled environments both in a real sense and in the connections we allow to the virtual.  To attend does require the student and the teacher to take part in an increasingly choreographed experience where little deviation is allowed from the script or grammar of school. There is an expectation that students, in different classes at the same curriculum learning area and level, will experience the same learning experiences, and we monitor this through assessments that target the expected learning outcomes.  Boredom, lethargy and other forms of deviant behaviour in students and staff is frowned upon, and cast members (who may be student leaders, staff, senior management, boards of trustees or even the MoE) constantly coax visitors to the theme park of school to immerse themselves more fully in the experience.  And the technicians who appear fully dressed and smiling as if from nowhere is something any student who has tried to have a quick smoke behind the caretakers shed will affirm.

It made me wonder if an outcome of designing and measuring everything we do will be a Truman like experience for both teachers and kids ... some of you will undoubtedly argue that school already has this effect on students ... school the ultimate Learningdome theme park, with an experience so manipulated that it has corrupted forever our sense of what it is to learn without the institution.    

Don't shake the system, cause the system works ...

I am having fun trying to imagine how Hattie and his colleagues at Visible Learning Labs would make a mashup between  Freddie Fiction’s  You Tube Video Big Corporation Man and the shake the system ideas in Visible Learning.   

Using those multiliteracies to get the message across ...

There is heaps of wiggle room in the lyrics given Hattie’s claim that

In many classrooms and schools, there is evidence of low effect sizes, reliance on poor methods and strategies, a dependence on “war stories” and anecdotes, and an agreement to tolerate different and sometimes poor teaching.  We beseech these teachers to be evidence based but so many government agencies and departments, teacher educators, and others are not evidence-based, and seem reluctant to accept evidence if it is contrary to current policies.  There is a preference instead to make changes to structural and working conditions.  Visible Learning P257

And that

If the criterion of success is achieving effect sizes greater than 0 then nearly all teachers could be considered effective.  But this is a false comparison and assumes that any achievement is better than none!  Students are more discriminating about teachers ....

Perhaps it is no wonder there is an increasing set of problems relating to student engagement. AS Steinberg, Brown, and Dornbusch (1997) claimed, so many students are “physically present but psychologically absent”(p67). They also cited that about 40 percent of students are “going through the motions” and say they neither try hard nor pay attention.  So many cut class and are truant, so many admit to cheating to get through, so many lose interest because they cannot keep up, and so many are bored by the lack of appropriate challenge.  So many do not learn that ability is not enough, and effort is critical. About half who do drop out of school claim that classes were not interesting or inviting, and two thirds claim that not one teacher was interested in their success in learning at school (Bridgeland, Dillulio, & Morison, 2006) All is not rosy with teachers, teaching , and schooling.  Visible Learning P250   

    

Visible Learning: The Gathering, and what to do when everything works.

I am enjoying reading all the summaries of the interventions for learning in Hattie's Visible Learning .  They are a little like reading “Wittgenstein in ninety minutes” –  the summaries are extraordinarily useful overviews of the key research papers informing learning interventions from homework, to acceleration, to bilingual programs, to simulations and gaming etc.

In truth I think Routledge missed a marketing moment here.  The research summaries and natty little d=0.33 effect size coloured dials are crying out to be captured on collectible trading cards.  They could be effortlessly developed into a “Magic: The Gathering” type trading card game to be played online or with decks of collectible cards by teachers, educators, parents and students all over the world.  I am already imagining what influences I’d want to collect for my deck.

I am trying to make connections between Hattie’s thinking about the effect sizes of different influences on achievement, and the Best Evidence Synthesis on Professional Development, to better inform what I do with teachers in the ICT_PD Cluster programme. 

 All the reading makes me realise that;

 

Almost everything has an influence on learning.

 

As a teacher I can be a significant influence on the learning of others

 

The strategies I use to teach with vary widely in their effectiveness.

 

Learning results in some kind of a learning outcome.

 

Learning outcomes vary – let us call it a continuum between shallow and deep.

 

Effective teaching and learning occur when both my students and I can tell you; what we are doing, how well it is going, and what we need to do next.

 

Effective teaching and learning requires that my students and I can distinguish between surface and deep learning outcomes.

 

Part of the day job sees me working within the structure of the ICT_PD cluster model to change teacher practice when using ICTs so that the teacher practice that results makes significant changes to student learning outcomes. 

Changing teacher practice is an interesting challenge.  There is an irony in this activity that is kind of like being a life coach for a life coach.

When thinking about the effectiveness of any professional learning I do, or do to others, I have always been a fan of Thomas Guskey’s thinking. I have used his 5 levels thinking as a personal effectiveness audit for many years.  Evaluating Professional Development by T.R Guskey

  

1. Participants’ Reactions

2. Participants’ Learning

3. School Organisation Support and Change

4. Participants’ Use of Knowledge and Skills

5. Student Learning Outcomes over the period of the professional learning

 

The Teacher Professional Learning and Development Best Evidence Synthesis Iteration (BES)  by H. Timperley, A. Wilson, H. Barrar and I. Fung (published in December 2007) means I now know heaps more about how best to change teacher practice in a New Zealand context.

Timperley’s work means we also know heaps about what doesn’t change teacher practice For example, the BES suggests that bringing in a charismatic speaker, giving teacher’s release time, holding a TOD, taking everyone away to a conference, enrolling people in an online community, or having visionary leadership may well be valued activity but they are the equivalent of “busy work” when it comes to professional learning for they do not lead to changes in teacher practice that are reflected in changes in student learning outcomes.

Being able to live with paradox means that within the ICTPD cluster programme, I am charged with and encourage all of the above.

When you read Hattie and Timperley et al it is apparent that, despite all the literature framing adult learners as having different learning needs from students stuff, there are even bigger similarities between changing teacher practice and helping kids achieve deep learning outcomes – both require hard work.

I want to learn how to teach teachers in ways that are more effective, in ways that are most likely to make big shifts in improving teacher and student learning.  The BES helps but not enough.  Without some clarity, over the strategies that make the biggest differences to student learning outcomes when using ICTs, it is hard for me to tell what I should be targeting, and then what I am doing, how it is going and what I should do next.

I don’t want anecdote to drive the decisions I make about the way I teach.  But, after reading Hattie and Timperley et al I acknowledge that I do not have enough research based evidence to judge the effectiveness of what I do when teaching teachers how to teach using ICTs in classrooms and schools.

The BES suggests that if I want the teachers I work with to have deep learning about the strategies that are most effective when using ICTs in teaching and learning I need to provide them with,

 

1.  multiple opportunities to learn through a range of activities.

2.  activities focused on content aims, eg translating theory into practice or demonstrating how assessment could be used to focus and refine teaching.

 

Part 1 is achievable but part 2, well part 2 is problematic.

The thing is that the facilitators in the ICTPD clusters don’t have any agreed upon content aims, theory, practice or demonstration of how assessment could be used to focus and refine teacher practice when teaching through ICTs.

I guess it is a whole lot easier to provide a conference for teachers to attend each year than it is to provide clearly identified professional standards for effective teacher practice when using ICTs in teaching and learning.

Unlike the numeracy, literacy or assessment for learning contracts the ict_pd cluster programme pretty much leaves clusters and facilitators to develop their own content aims.

Our cluster’s content aims are based around helping teachers;

• identify the student learning outcome in the learning experiences they plan for their students,
• identify the ICTs that might enhance the conditions of value of that student learning outcome. 
• learn how to use the ICTs identified to help student learn, and
• develop self assessment rubrics and success criteria to allow students to self assess their learning outcome and know what to do next.
  

They had always seemed defensible until I read Hattie.

Reading "Visible Learning" makes me realise that my content aims and process are based upon probability not evidence.

It is probable/ likely that the approach I take will enhance student learning outcomes but I have no evidence based practice to support this in the context of using ICTs. I have no professional standards or success criteria to let me know if I am successful or what I should do next. 

However, more significant was the realisation that what I usually take as proof of our success – all those pages and pages of milestone reporting on improvements in student learning outcomes as a consequence of using ICTs with students in our cluster schools ... is actually no big deal ... for improvements in learning outcome are a given when innovations are introduced to schools.

In fact the only result worth reporting on in a milestone would be if there was no improvement in student learning outcome as a result of using ICTs in teaching and learning.

Which means that providing evidence in a milestone report that suggests that student academic achievement has been enhanced because I have encouraged teachers to use an IWB, student blogs, an online community, or VoiceThread is not as exciting as we make it out to be.  At best it only affirms that teachers are in a cluster looking at how best to use ICTs in teaching and learning. Because the very act of looking at what we do enhances learning outcomes .

To put it another way, providing evidence of enhanced student learning outcomes is no big deal because Hattie’s research into influences on student learning outcomes identifies that most everything I do as a teacher will improve learning outcomes.

 Realising that “everything works” is a bit of a wakeup call.  It is also why the trading card game idea is a winner. 

If as Hattie describes just “having a teacher’s pulse” in a classroom can be shown to improve student learning outcomes, with an average effect size of d= 0.2 to d= 0.4 growth per year, then it is uncomfortably apparent that we need to look for other ways to discriminate between all the things that influence learning outcomes.

Because if everything works, then everything can be and is defended, and we have no professional yardstick to tell us what to do next, or how to improve the effectiveness of our practice.

I reckon we are especially vulnerable to this “everything works” effect in the ICT_PD Cluster programme.  This is because in the ICTPD Cluster programme we are asking teachers to change their existing practice, and as Hattie shows just being involved in an innovation that alters an existing practice can lead to improvements in student learning outcomes.

“the mere involvement in asking questions about the effectiveness of any innovation may lead to an inflation of the results”

 Hattie uses this to argue that we should be looking for an effect size of d=0.4 or greater when deciding on where to put our energies in teaching and learning, when deciding on the strategies to best influence our effectiveness. Noting that effect sizes may not be uniform across all students, Hattie suggests that “effects lower than d=0.4 indicate the need more consideration (costs, interaction, facts and so on)”. 

Look at the effect sizes associated with ICts and teaching and learning.

Computer assisted instruction: d= 0.37

Web- based learning: d= 0.18

Interactive video methods: d= 0.52

Audio/Visual methods: d=0.22

Simulations: d=0.33

Programmed instruction: d= 0.24

From Hattie, J. (2009) Visible Learning p 220 to 232

I sense some great discussions ahead.  Also worth thinking about is that the effect size for computer based instruction has remained much the same for the last thirty years - between 1975 and 2007.  Meaning that all the improvements in technologies, hardware, software, infrastructure etc, that we have purchased and introduced to schools, have made no significant improvements in the effect size on student learning outcomes.  How will the paradigm shifters and marketers make sense of that one?

There are four questions that I particularly want to hang onto, before I read more.

Englemann (cited in Hattie 2009 p253) challenges teachers and schools to ask four critical questions about the innovations we are asked to adopt in school. The kicker is in the third question.

 

• Precisely where have you seen this practice installed so that it produces effective results?
• Precisely where have you trained teachers so they can uniformly perform within guidelines of this new system?
• Where is the data that show you have achieved performance that is superior to that achieved by successful programmes (not simply the administrations last unsuccessful attempt).
• Where are your endorsements from historically successful teachers (those whose students outperform demographic predictions)?
  

When we are thinking about ICTs in teaching and learning they become:

• Precisely where have you seen teaching and learning through ICTs installed so that it produces effective results in enhancing student achievement?
• Precisely where have you trained teachers so they can uniformly perform within guidelines of this new system?
• Where is the data that show teaching and learning using ICTs achieves performance that is superior to that achieved by other successful strategies used in teaching and learning?
• Where are your endorsements from historically successful teachers – those whose students outperform demographic predictions for student achievement?
  

 

Collision theory, class size, and those deliberate acts of teaching.

"I've tried to avoid teaching, which for all its charm takes a lot of your energy and makes you doubt yourself."   John Updike cited in Robert Genn Letters 

A much anticipated copy of John Hattie’s “Visible learning A synthesis of over 800 meta - analyses relating to achievement”    was sitting in the corridor in its yellow bubble wrapped Amazon envelope waiting for me to get home from the Pakuranga Schools TOD yesterday.  I have been hanging out to read the book that summarises 15 years of his research and thinking about influences on student achievement.

I will admit here that Hattie is one of my most favourite educational thinkers.  I regard him, as I do Illich, as someone who has brought an unusual and much needed clarity to my own teaching and learning. 

For example, take Hattie's thinking on class size effects on student learning.

It seems plausible to claim that reducing class sizes will improve learning outcomes, and for as long as I can remember this has been the claim of lobbying by teachers and teacher unions for improving working conditions. Refer the Scoop piece from the NZEI.  And until I read John's take on this a while back I had never challenged the assumption.  It seemed highly plausible.

However, Hattie notes that although reducing class sizes might improve the probability that the environments for improving learning will occur; he has research that shows that it does not improve student learning outcomes per se.  

Teachers get very agitated by this claim.  But if they listened carefully to Hattie’s argument and read his research they would understand why he can validly claim that class size has minimal effect of learning outcomes.  

Improving student learning outcomes through reducing class sizes requires that teachers moving to smaller class sizes “reconfigure the interaction, curricula, and strategies” they used with large classes to change the student interactions happening within the smaller classes. 

And the provocative bit is that Hattie’s research shows that in many instances this does not happen.

We continue to use the same interaction, curricula and strategies with the students in our care regardless of how many students there are.  If as teachers we do not change our practice when working with smaller class sizes , the quality of teaching, student learning and student interaction does not change when class sizes change.

It reminds me of the way I used to explain collision theory when I taught science.   The student text put it like this:  

 A reaction occurs when particles collide.  Not all collisions between particles result in a chemical reaction.  If a reaction is to occur, the particles must collide with sufficient energy for a reaction to occur.  They must also collide in the correct orientation (or position).

Finally, it is a fact that the more frequently effective collisions occur, the faster the rate of reaction.  There are four factors that affect the rate of a reaction, by affecting the frequency (and possibly the energy) with which collisions occur.  These factors are concentration of reacting solutions, surface area of solids, temperature and use of catalysts.

As a consequence I used to teach the kids to talk about increasing the rate of effective collisions rather than simply increasing the rate of collisions.

 I have to unpack the analogy a little more but it starts with “learning happens when teachers interact with students.  Not all interactions between teachers and students result in learning.  If learning is to occur, the teacher and students interactions must be of sufficient quality for learning to occur.  They teacher student interactions must also occur with  .... ”

So Hattie’s argument is that not all collisions between teachers and students will result in learning.  Simply increasing the frequency of those collisions without attention to changing the teaching and learning equivalent of the orientation of the collision and possibly the energy of the collision does not guarantee a reaction or improved learning outcomes.

As a consequence as teachers we need to talk about increasing the rate of effective interactions with students rather than simply increasing the rate of interactions. 

Hattie's work alerts us to re-look at ourselves and our practice.  To ask how can we assess and improve the quality of our teaching? 

Opening “Visible learning”   was an experience that reminded me of clasping one of those famous double scoop icecreams whilst standing in the sunshine on the banks of the Waikato River overlooking the power station in Huntly on Wednesday afternoon.

The dilemma of where to start when dealing with a rapidly melting double scoop chocolate ice-cream cone was not unlike the dilemma I faced when deciding where to start reading Visible Learning. 

There was so much that I wanted to read that I was made momentarily uncertain ...about where to begin.  

I decided on a strategy of selecting pages at random; and of reading whatever text first caught my eye as the book splayed open. And this is a book that splays well.  I was immediately rewarded by the following statement.

Constructivism is a form of knowing, and not a form of teaching, and it is important not to confuse constructing conceptual knowledge with the current fad of constructivism (Bereiter, 2002; Small 2003). Excerpt from Visible Learning “The empirical quest for explanations” on p243.

Hattie notes that his meta analyses findings back those of Kirschner, Sweller and Clark (2006); (researchers I’ve blogged about before) who reviewed minimal guidance during instruction teaching methods and found them wanting.

I liked the text clarifies the assumptions within inquiry and problem based learning approaches.

Assumption 1: Setting authentic problems leads to students constructing their own solutions.

Assumption 2: Knowledge is best acquired through experiences based on procedures of the discipline.  

And I liked how the text explains that these assumptions are not supported by the data.

“...each new set of advocates for these approaches seem either unaware of uninterested in previous evidence that unguided approaches have not been validated.” P243

Hattie affirms that claims of engagement and enjoyment are not evidence of student learning. 

All that “I know this works because “my students’ enjoy” or “the class is so engaged”, when we do [discovery learning, problem based learning or inquiry learning]” that I hear when working in schools. 

Inquiry and problem based learning are not evidence based practice because the evidence that they work is lacking.

A key understanding from p243 in Visible Learning is that believing that students construct conceptual knowledge DOES NOT REQUIRE us to adopt minimally guided methodologies like inquiry learning and problem-based learning.

Forms of knowing are not forms of teaching.  

If we believe that students learn through constructing conceptual knowledge, and that this is an active, social process where learners need “to create and recreate knowledge of themselves”, then direct and active teaching for this learning is the smart solution - both possible and appropriate.

Just look at the effect size data Hattie provides. 

Table 11.1 Effect sizes of teacher as activator and teacher as facilitator

Teacher as activator	d	Teacher as facilitator	d
Reciprocal teaching	0.74	Simulations and gaming	0.32
Feedback	0.72	Inquiry based teaching	0.31
Teaching students self verbalisation	0.67	Smaller class sizes	0.21
Meta-cognition strategies	0.67	Individualised instruction	0.20
Direct instruction	0.59	Problem-based learning	0.15
Mastery learning	0.57	Different teaching for boys and girls	0.12
Goals-challenging	0.56	Web based learning	0.09
Frequent/effects of testing	0.46	Whole language – reading	0.06
Behavioural organisers	0.41	Inductive teaching	0.06
Average activator	0.60	Average facilitator	0.17

Table 11.1 from Visible Learning (2009) Page 243

 The data in the table provides a powerful insight for teachers who want to make a difference to student learning; teachers who want to know where they should best put their energy.

When shared outside of the classroom it will also provides powerful argument for parents, Boards of Trustees, schools and communities who want to work together to improve student learning.

And what I like best is that, students learn significantly more when teachers adopt direct, active teaching methods.

Hattie’s effect size data resolves all that ambivalence between offering “deliberate and purposeful acts of teaching” or adopting that currently much acclaimed role of being a “guide on the side”, that stepping back and overseeing student inquiry.

And the question Hattie tackles in the next few pages (p244 to 247) is to explain the active teaching and learning strategies that teachers might work to improve their practice.  Things like improving the way we use backward design, learning strategies for surface deep and constructed knowledge, feedback, helping students know where they are going, how they are going and where to next etc.  

The effect size data and the elaboration on how we might improve the direct active teaching methods we adopt will undoubtedly help thoughtful teachers who like John Updike find that teaching takes all their energy and makes them doubt themselves.

I must continue reading.

Sustainability and Jamie Oliver drizzle words in education.

We have been writing content framed around the New Zealand Curriculum this summer break.  The process of working so intensively with the NZC document has alerted me to the things we privilege/ legitimise through inclusion in a national curriculum, and the things we omit and thus undermine.

For example in the NZC vision statement the second out of five bullet points (and yes I am alert to the irony of bullet pointing a vision) is for young people:

• who will seize the opportunities offered by new knowledge and technologies to secure a sustainable social, cultural, economic, and environmental future for our country;

 “Sustainability” is currently a Jamie Oliver drizzle word.  Be it in the context of a “Nan’s Lemon Drizzle cake” drizzle or a “drizzle over a little balsamic vinegar and season lightly with salt and pepper” drizzle, the result of drizzling “sustainability” or “sustainable outcomes” over all language symbols and text in any document you are working on in education is currently a “good thing” and guaranteed, much like Jamie Oliver’s creative drizzles in the kitchen, to find favour with your patrons.  It is so popular at the moment that I am waiting for the release of a drizzle index.

Because of the drizzle thing, I was not surprised that he NZC vision statement suggests that it is important to secure a social, cultural, economic and environmentally sustainable future for New Zealand.

But what did catch my eye was the way in which the NZC vision statement suggests these sustainability outcomes will occur.

It seems that the NZC believes that sustainability will be “secured” through “seizing” opportunities offered by “new knowledge and technologies”.

I don’t know what seize or “secured” means to you ... but my guess is that what is being alluded to by “seize” is different from to clasp gently, to linger over, or to to nudge up against, and that “secure” is something different from sharing, or distributing.

The vision statement suggests that not only will sustainability be resolved through a technological solution but also that this technological solution will need to be pursued, pinned and consumed through competitive activity. 

It makes me want to ask ... who will be doing the seizing? And who will be advantaged by the seizing?

It is noteworthy that there is no expectation in this bullet point that changes in human values – the what it is to live, love and be kind, will play a role.  And this seems strange given the emphasis on the key competencies later in the document.

Sustainability is all about discovering what is important and then discovering how we can protect what we have identified now and in the future – it is about balancing human needs with other needs, preservation, guardianship, and kaitiakitanga.

But it seems that preservation bit is not as easily understood as it might first appear.   Is preservation all about seizing?  Is preservation all about finding a technological solution?

Thinking that technology will rescue us, and that what will rescue us can be seized and secured, suggests that the collective writers of the NZC believe we do not have to take collective individual responsibility for preserving the social, cultural, economic and environmental resources in New Zealand in the 21^st Century. 

And that is kind of frightening because the problem with thinking that “technology will rescue us” in these situations  was well explained and found wanting over forty years ago in 1968 by Hardin (refer Tragedy of the commons  - The article is well worth a read and I note not without its own critics). 

Hardin’s argument was that sustainable outcomes cannot be resolved by technological solutions, if individuals are advantaged by pursuing their own interests then we will need “a change in human values or ideas and morality” not a technological solution to change things.   

Picture a pasture open to all. It is to be expected that each herdsman will try to keep as many cattle as possible on the commons. Such an arrangement may work reasonably satisfactorily for centuries because tribal wars, poaching, and disease keep the numbers of both man and beast well below the carrying capacity of the land. Finally, however, comes the day of reckoning, that is, the day when the long-desired goal of social stability becomes a reality. At this point, the inherent logic of the commons remorselessly generates tragedy.

As a rational being, each herdsman seeks to maximize his gain. Explicitly or implicitly, more or less consciously, he asks, "What is the utility to me of adding one more animal to my herd?" This utility has one negative and one positive component.

1) The positive component is a function of the increment of one animal. Since the herdsman receives all the proceeds from the sale of the additional animal, the positive utility is nearly +1.

2) The negative component is a function of the additional overgrazing created by one more animal. Since, however, the effects of overgrazing are shared by all the herdsmen, the negative utility for any particular decision-making herdsman is only a fraction of 1.

Adding together the component partial utilities, the rational herdsman concludes that the only sensible course for him to pursue is to add another animal to his herd. And another; and another. .But this is the conclusion reached by each and every rational herdsman sharing a commons. Therein is the tragedy. Each man is locked into a system that compels him to increase his herd without limit--in a world that is limited. Ruin is the destination toward which all men rush, each pursuing his own best interest in a society that believes in the freedom of the commons. Freedom in a commons brings ruin to all.

The NZC suggests that if we want sustainability we must seek a technological solution that has been wrested and secured - control and ownership stuff.

Hardin’s argument suggests that if we want sustainability we must manage how individuals and corporations use the commons – more control.

McKenna suggests yet another alternative – that we should unburden the individual from all that wresting and securing or being managed by others stuff and instead trust in the collective for sustainable outcomes. Less seizing and securing and technology and more changing behaviours and values for trusting and sharing.  (Thanks to Nix for the link)

I have thought a lot about preservation, and what it means to preserve something, an insect, a person, a story, a fantasy.  The instinct to preserve something sometimes means that we covet it and  keep it hidden away, but I have found that objects can exert their will on us and preservation can become a burden.   Rather than covet my father’s collection I’d like to let go of it.  For this reason, when I first exhibited this work in PlaySpace Gallery at CCA, I gradually gave away every image in the piece to gallery visitors.  The act of scattering these images was meant to diffuse the responsibility of their preservation and to give these objects a second life. Perhaps this gesture of dispersal can be a form of preservation, one that lifts the weight off one person and trusts in the collective. Klea McKenna The Butterfly Hunter

John Francis Fien (2003) describes what preservation through dispersal might look like in an educational context: (Thanks to RB for the reference)

 “Enhancing our abilities to learn, to live sustainably and to love is the only way we will be able to address the ecological and social imperatives we face as we seek to build a fairer, less troubled and sustainable world for our children.”  From Learning to care: Education and compassion. In Australian Journal of Environmental Education, Vol. 19, pp. 1-13

[I note that although social, cultural, economic, and environmental sustainability are in the NZC vision on page 8 by the time we hit the values “to be encouraged modelled and explored” on page 12 the social cultural and economic have been lost.  NZC Values: excellence, innovation, diversity, equity, community and participation, ecological sustainability, integrity and respect.   And that is a shame because I don’t know if it is possible nowadays to think critically about ecological sustainability without a concomitant valuing of social and economic sustainability.]

How should we measure visitor satisfaction in museums and engagement in learning in schools?

The NDF conference presentation on Web metrics by Seb Chan from the Powerhouse Museum who blogs at fresh + new(er) blog had some valuable insights for institutions like NZCER  who produce surveys claiming to measure student engagement in school.  

Me and My School: student engagement survey

This year we developed and launched a survey tool aimed at finding out what students in years 7-10 think about their school and their learning. The tool was trialled on more than 8000 students before we launched it in the third term this year. Many schools have told us how useful they have found it, so much so we are looking at expanding it to years 11-13. We hope to have that work in development next year and an extended tool available in 2010. Meanwhile, the years 7-10 version will be available again in 2009, with schools able to run it in the third term. We make it available at the same time each year in order to ensure the national norms are valid.

 How we understand engagement is something that I have fretted over before on Artichoke   ... .  

I’d much prefer we put our thinking and energy into measuring student versatility and control.   Engagement is also something that I doubt can be measured on a survey so I was especially alert to the current enthusiasm in New Zealand schools for NZCER’s new measure designed to profile  student engagement  –and to listening to how principals are talking about the use of the NZCER engagement data as an evaluative measure for school programmes.  

At the NDF conference Seb Chan’s presentation was on the validity of different measurements of visitor satisfaction  used to evaluate the success of the work of Museums.

His presentation provided much to challenge our NZCER survey measurement of student engagement used to evaluate the success of the work of schools.

Chan started by looking at current measures of visitor engagement and how little they really tell us.

For example when Chan claimed that changes in the way people interact in online environments

makes traditional Web analytics and metrics that museums have used to measure and track success on the Web for the past decade increasingly inadequate. Occasional user surveys and server-side log analysis can no longer be relied upon by Web teams to guide them towards making museum sites more user-centric and effective.

The  “more user-centric and effective” bit reminded me of our NZC claim to  “put students at the centre of teaching and learning".

When Chan claimed that   

Whilst basic reporting currently satisfies government and sometimes corporate benefactors, far more complex analysis is required for museums themselves to more effectively evaluate and refine their on-line offerings for their users.

I was interested in how this might also relate to the conclusions gained from a self report survey on engagement.   

Chan was an entertaining conference speaker.  He well exposed the flaws and deceit in commonly used web analytics - “Where counting has no point” - through “A Summary Of Old Problems”;

• The Problem With Log File Data,
• The Problem With Page Tagging Data,
• The Problem With 'Unique Visitors',
• The Problem With 'Visits' And 'Time Spent On Site',
• The Problem With 'Page Views'. 

Even the number of visitors who click on an interactive such as a video talk or download a podcast was exposed when more detailed analysis shows so few of them watch the whole video or listen to the whole podcast.  

“In many ways the best measure of the success of a podcast is how much feedback and discussion it generates. This is far more valuable than the total number of downloads”.

Of much more interest to Chan was how we might measure the stuff that really shows visitor satisfaction. 

If just turning up on a website was not enough then  ....Seb argued for  third party web metric measures of visitor behaviour using  RSS feed tracking, comments on the museum website, but also on other blog posts and comments, tagging and comments on museum content on Flickr Commons photos and how these are used in other conversations in communities and blogs,  Technorati trackback, and Facebook friends, fans and profile comments, gave a better indication of the success of museums and exhibits and events than number of visitors/page views.  

He suggested combining qualitative and quantitative measures when we measure visitor comments online.  

Again, it is far better to measure interactions – comments, trackbacks – and then qualitatively assess them. Blogs should ideally be generating conversation and discussion, and blogs will rank differently depending upon your choice of what to measure (Chan & Spadaccini, 2007).

Chan identified these alternative web analytics as a way of collecting “measures of recommendation” – a kind of “how likely is it that you would recommend [the company/ experience] to a friend or a colleague? – a broader sense of those net promoter score stuff.  He suggested that recommendation (and hence allowing recommendation and sharing) is how we should understand the way people interact with museums.

It all made me think of our current excitement in education over measuring engagement.

If engagement in learning is important then counting the numbers of students who claim to be engaged in response to questions in a survey will tell us very little.

We should look carefully at Seb Chan’s museum analytics thinking and look for measures of recommendation.

1.  How likely is it that students would recommend [the school, the teacher, the learning experience] to others?

2.  How could we find this out using Web metrics?

My thinking starts with mentions of learning on student social networking sites, blogs, Rate my teacher ....

And then

3.  How could we use technology to allow for/ enhance the conditions for recommendation and sharing of learning in school?

More reading:

Sebastian Chan : Towards New Metrics Of Success For On-line Museum Projects

Understanding knowledge, George Oates, Flickr and building learning communities in school.

I spent Thursday and Friday at the National Digital Forum 2008 Conference with Nix.   

It was liberating for two teachers to go undercover at a conference for uber_librarians, (e)_historians, anarcho_archivists, web designers and museum_istas.  We spent two glorious and anonymous days learning about knowledge, ownership, access and authority, NZ cultural copyright, what this means when things are digitised, quantitative and qualitative measurement of audience engagement, what website analytics really show, the fabulous  Digital NZ  site with its Digital NZ Memory Maker remix editor and editable Coming Home search widget and and and ...

The tensions in the discussions in the Owen Glenn Building so often came back to how we understand knowledge – and the artificial polarising of the alternatives.  Those traditionalists  worried about digitisation betraying institutional authority and expertise – and what happens to knowing when we blur the privileging of particular experiences or interpretations.  The modernists  argued for the experiential basis of knowledge – that knowledge is both a social and historical product stuff, and that digitising can leverage knowledge by opening access and interpretation to all.

Moore and Young were helpful in not dismissing those with reservations, or rejecting those without.

The neo conservative position may be flawed, but it is not false.  It reminds us that (a) education needs to be seen as an end in itself and not just a means to an end (the instrumentalists position), and that (b) tradition, though capable of preserving vested interests, is also crucial in ensuring the maintenance and development of standards of learning in schools, as well as being a condition for innovation and creating new knowledge.  More generally, neo conservatives remind us that the curriculum must, in Matthew Arnold’s words, strive to,

Make the best that has been thought and known in the world current everywhere! (Arnold, 1960, p70)

From "Knowledge and the Curriculum in the Sociology of Education: towards a reconceptualisation by Moore and Young. British Journal of Sociology in Education Vol 22 No 4 2001 p449 and 450"    Thanks to cj for suggesting this as mind food.

The irony being that different interpretations of "knowledge" means that Arnold's quote could have been a catch cry for either group.

Much like conversations and claims over how knowledge should be produced or acquired at an  educational conference, the conversation over digitising knowledge in the two day NDF conference could have also been framed by how the various speakers and organisations understood knowledge. 

However, there were some significant difference between the educational conferences I get to attend and the NDF08.  The first thing I realised was that librarians, historians and archivists flock differently from teachers. They dress differently, they queue differently, and they question differently.  For example The NDFers had to be encouraged to take freebies like fractured fragments of greenstone from the registration desk and ice cream from the Trade Exhibit Area.  And unlike my experience in educational conferences the end of each keynote and forum session was marked by thoughtful challenge and critique offered. The NDFers asked difficult questions, provided intriguing analogies, offered significant alternatives, and contested espoused institutional values.

The NDF2008 keynotes were notable for their focus on real achievement.  The NDF keynoters had all done the stuff they were talking about.  We heard about what had worked and what had failed; we heard about real outcomes and actual achievement. There was an absence of all that futuristic visionary rhetoric we have become so accustomed to in educational conferences in New Zealand; an absence of those paradigm shift_ers, digital native_rs, generation Y_ers, knowledge is a verb_ers, perfect education storm_ers, and  guide on the side_rs.   Conference circuit junkies, (e) learning futurists and prophets didn’t get a look in at the NDF08 conference. 

 In the opening keynote on Thursday, George Oates Senior Program Manager, Flickr talked about “Human Traffic, General Public.”

Flickr has grown to an archive of over 3 billion photos in just under 5 years. What?!?!? Once upon a time it was just a start-up with a handful of members. How did it become the world-famous photo sharing site it is today? By building a passionate community - or, more accurately, lots of co-existing communities, all bustling around the same place. What better place for public institutions to share their collections? It turns out the enormous Flickr community is very interested in The Commons project on Flickr. The key goals of The Commons (http://www.flickr.com/commons) are to “firstly show you hidden treasures in the world’s public photography archives, and secondly to show how your input and knowledge can help make these collections even richer.”

A founding member of the team that built Flickr, George Oates was the Lead Designer of flickr.com for four years, and recently moved into the role of Senior Program Manager, leading The Commons on Flickr. Her keynote presentation at NDF is called “Human Traffic,” about how designing for community might actually be able to help public institutions can create digital value through platforms like Flickr, by creating an engaged, conversational and generous community.

 George Oates described how Flickr started .. The Origin Myth ... and what Flickr is today - five years on. This thinking about Flickr was very useful for educators thinking about different ways of doing school and different ways of defining knowledge. Her keynote was all about conversation, collaboration and contact networks.

George identified two key ideas learned from Flickr;

People don’t like being told what to do.

People do like to feel that they belong.     

You could tell in the post keynote conversation as we queued for coffee at morning tea that in talking about The Flickr Commons, George respectfully disrupted the ways some in the audience made meaning of their day jobs.  The coffee line conversation was all about the perceived loss of institutional authority, loss of archival context, the authentication of comments made, and control over the digital copies shared. Although, The National Library of New Zealand had obviously thought through all of this and officially joined The Commons Project www.flickr.com/commons on Thursday afternoon.

I took something different from the archivists and their concern over access, ownership and control.

Learning how Flickr had designed and then built a community provided an insight for thinking about new ways of designing learning communities in schools and between cluster schools. 

It the success of Flickr (3 billion photos archived in just under 5 years) tells us anything about human interaction and I think the sheer scale of Flickr means it does, our challenge is to build flexible places/spaces online and face to face where we change our current focus on compliance reporting.  If we are genuine in building a learning community then we need to reduce all the telling people what to do stuff and rark up all the opportunities for belonging – the contributing and participating stuff.  

I much enjoyed the opening keynote, I liked the way “historical authenticity” is understood on Flickr, how it is not the end of the world if something happens that is not controlled, how the best  protection may well come from proliferation, how Flickr increases public access to public things, but best of all I liked George Oates reference to hand crafted objects and Malcolm McCullough

The handcrafted object reflects not only an informal economy of energy (as opposed to one of process efficiency), but also pleasure. Its production involves some play, some waste and above all some kind of communion. P10 Abstracting Craft: The Practiced Digital Hand 

The quote has such resonance in understanding the work we do in schools with knowledge building.

so what happens when we look at student learning outcomes against the criteria for identifying a hand crafted object and when we can do this using digital platforms?

Can we create learning experiences where we scaffold for both an economy of energy and the opportunity for pleasure?  Where in planning for a student learning outcome we ask ourselves;

Where is the opportunity for play?

Where is the possibility for waste?

Where is the prospect for communion?