Our contributing editor Clarissa Morawski sent me this article from the weekend edition of the Globe & Mail newspaper. This is the latest in a series that columnist John Barber has written about (and against) waste incineration proposals in the Greater Toronto Area based on information first reported in our magazine in the cover story by Clarissa of the April/May 2007 edition, plus further information she's dug up. Enjoy.

INCINERATION: THE NUMBERS GAME

It's hard to see the truth through all the smoke?

The Globe and Mail

Sat 23 June 2007

JOHN BARBER

There's nothing like a good, clean hit to enliven either a hockey game or a public debate, and Peterborough consultant Clarissa Morawski landed a beauty this spring when she looked at the emissions data for the necklace of large garbage incinerators our suburban neighbours plan to build around Toronto.

The data, supplied by the vendors of incinerators and published as an appendix to the environmental assessment of the plant that Hamilton and Niagara Region hope to build, showed an entirely different picture from the rosy propaganda the vendors and their agents had spun about their wondrous technology.

Unlike the old incinerators that were once considered safe - until they weren't - the new ones are said to be advanced "energy-to-waste" facilities that turn household waste into clean energy. But the data submitted by the vendors themselves showed that this new technology was one of the dirtiest imaginable ways to produce power - far worse even than coal-fired power plants in terms of heavy-metal and greenhouse-gas emissions.

"I was absolutely shocked that incineration is still under consideration, given the pollution profile alone," Ms. Morawski said at the time. So were a lot of people when she published her findings in Solid Waste Magazine. In the time since then, concerns about the huge costs and potential hazards of incineration have led Halton Region to cancel plans to build a facility, while Niagara and York are slipping free of the partnerships they once entered to do the same.

But Hamilton and Durham still appear determined to go it alone with their big burners, doubling down on what their counterparts considered a losing bet. They do have one new advantage: The inconvenient facts that helped deter the others no longer exist.

Within weeks of Ms. Morawski's critique, the "comparative emission study" she relied on disappeared from the website documenting the Hamilton-Niagara environmental assessment. Within months of the date one of the facilities is scheduled to be built - thanks to the McGuinty government's recent decision to fast- track incinerator projects - there is no agreed-upon data about what will come out of their stacks.

The reason, according to the consultant who advised that the data be "taken down," is that they were incorrect. "We've found more recent information that corrects it," said David Merriman of Genivar Inc., the firm advising both the Hamilton and the Durham teams on their projects. The Niagara document, which was posted for more than a year, was only a draft, according to Mr. Merriman.

"We found, having posted it, there were some incorrect things," he said. "We removed it and we're now working on a corrected version that we will be presenting in September."

Mr. Merriman wouldn't say when he discovered the data was incorrect, but acknowledged the review was inspired in part by incinerator vendors "who told us the emissions coming out of the new technologies are lower than they have been historically."

So they get to supply new numbers, based on their fondest hopes for the very latest technology, to update the image of facilities that were once considered state-of-the-art, low-emission power plants - until they weren't, sometime last week, at which point they reverted to being dirty old mass-burn incinerators.

Technology advances - and so does the tricky business of calculating greenhouse-gas emissions, which represent another image problem for the nasty old incinerators that were so clean and modern last week. Thus the consultants also plan to introduce new, radically downgraded estimates about their carbon-dioxide emissions to replace the ugly numbers that disappeared.

The change is necessary not because there is new technology that reduces carbon emissions from garbage burners, according to Mr. Merriman, but because there is new thinking about how to count them. The actual emissions will stay the same, but the numbers reported in September will likely be halved.

Incinerator vendors have long supported such an approach, which is used by the Intergovernmental Panel on Climate Change to calculate national carbon inventories - and assumes, among many other things, that all the paper and wood burned as garbage will come from renewable sources. But applying such factors to emissions from actual burners with real smokestacks is highly controversial.

"It's totally inappropriate that anyone would apply IPCC guidelines when measuring emissions from thermal stations," Ms. Morawski said. "We just want to know what comes out of the stack."

But we no longer do - and likely never will, if the new arithmetic prevails and reported carbon emissions fall dramatically this September. In the meantime, numbers swirl headily behind the scenes.

What a spectacle. It's enough to make you realize why the McGuinty government exempted garbage incinerators from the Environmental Assessment Act. Learning the truth about them is such a confusing business.

jbarber@globeandmail.com

Posted by Guy Crittenden at 12:16 PM | Permalink | Comments (0)

I thought readers would appreciate this sobering evaluation of the peer review process, which (like so many other things) isn't quite the model process some think. This interesting article is from the Financial Post, FP Comment section, Friday, June 22, 2007.

Lessons of figure skating

Peer review is a crucial part of science funding, but scientists could learn from the skating world that more than two opinions are needed for a good judgment

REINHOLD VIETH

Financial Post

Scientific peer reviewers are the best specialists that editors can find to read the manuscripts they receive. Peer reviewers usually serve as unpaid, hardworking experts. In essence, journal peer reviewers stand on guard for society as a whole, to ensure that only scientifically credible articles get published.

But long before any journal peer review, the research needs to be financed, so a different kind of peer review takes place. To apply to publicly supported granting agencies, researchers need to describe why an idea needs investigation and how they would conduct their experiments. Instead of a worldwide pool of experts, funding agencies usually must rely on committees, or groups of scientists from various fields. Those who serve as peer reviewers for funding agencies are also volunteers, giving of their time in the often thankless task of reading many applications for funding. The goal of their peer review is to provide a score that agencies will use to rank who gets funding. In theory, peer reviews applicable to grant applications ensure that limited research dollars support the best science.

The peer review of research-grant applications is a huge problem for all concerned. Not counting the thinking and the groundwork, a typical medical researcher spends an absolute minimum of a month of full-time work writing a grant application. After that, according to the Canadian Institutes for Health Research (CIHR), our federal government’s largest medical and health funding agency, the odds of funding success are about one out of six. If the grant application aims to fund brand-new research, the odds of success are even worse. Peer review should ensure that good research stands a better chance of success than a roll of the dice.

Almost five years ago, Warren Thorngate, a statistician from Carleton University, examined the statistics about the peer-review process at CIHR. His “Thorngate report” paints a very sad picture. He shows that the scoring of scientific grant applications is no different from other situations in which humans need to “score” something, whether it is judging figure skaters, or choosing a paint colour. Any two people can agree or disagree, just by the luck of the draw of which two people are selected to judge. I have long wondered how many people who apply for research funding have read the Thorngate report with care.

Basically, every grant application is given to two members of a committee to read, each assigning a score out of five. That score is a judgment based on quality guidelines about the possible importance of the research to the health of Canadians, the quality of the experiment proposed, etc. The result, according to Thorngate, is that “perceived differences in the quality of the applications accounted for less than 25% of the variance of internal reviewers’ ratings. Individual differences among the internal reviewers seemed to account for the rest.”

This means that 75% of what compares your score with everybody else’s is just plain randomness. Everyone who has applied repeatedly for research grant money knows this, and it applies to just about any peer-review system, not just CIHR. For example, I sent exactly the same grant proposal to two funding agencies at the same time. Agency A scored the proposal so badly it was not even worth discussing. Agency B scored the proposal as the best of the 20 it considered. When I complained to Agency A about this discrepancy, it replied that its low score simply reflected a difference of opinion.

For researchers who need support, the random gamble of the way applications are scored and ranked is a huge problem. Researchers now accept that peer review is junk science, because it is not science at all. Applicants for grants know that, despite the sincere efforts of peer reviewers (all of whom have also been applicants), the opinion-based judgments of peer reviews do end up functioning like a lottery. And just like any lottery, the only way to be sure of winning is to keep on buying a ticket.

For funding agencies, the randomness of peer review has created an ever-growing problem. As applicants keep recycling grant applications into the lottery, the number of applications climbs and the success rate drops. New research ideas entering the pool are quickly watered down into a sea of applications. The burden of dealing with applications that need reviewing increases. With that, the mental capacity of peer reviewers becomes ever more strained. It becomes difficult for them to do justice to every application, they are overworked, and they become quick to toss proposals out of competition.

To outsiders, peer review is a mysterious scientific system that serves as our ultimate way to determine research quality. Warren Thorngate tells us with evidence that the quality of judgment in peer review is no more reliable than for any other kind of judgment call.

In the field of figure skating, performances are scored and averaged from several judges, with the highest and lowest scores tossed out. The scoring system for figure skating is more scientific, because ranking for a given performance is designed to be reproducible. Figure skating has minimized the lottery effect. The problem for those of us who apply for medical research funding is that, with only two reviewers to score applications, the scoring system that compares each applicant with the competition is just too noisy. I am by no means criticizing peer review of research grants because there really is no better alternative. But we need to make the system less of a crapshoot for applicants.

In science, the usual way to make things less random is to average more inputs. This means to average scores from more than the usual two peer reviewers who sit on committees. However, according to Thorngate, even though our CIHR sends proposals to outside experts for peer review, their opinions “matter little in the adjudication process” and “the usefulness of external reviews remains a mystery.” In other words, to a statistician, it looks like the extra peer reviews available are wasted because there is no evidence that they count toward the ranking for funding. This is not good science.

Counting the input from a greater number of judges in the average score works for figure skating. Those responsible for designing the way research grant applications are ranked need to borrow a page from the world of sports and make the system as reliable as it would be if an audience were watching.

Reinhold Vieth is Professor, Department of Nutritional Sciences and Department of Laboratory Medicine and Pathobiology at the University of Toronto, and Director, Bone and Mineral Laboratory Pathology and Laboratory Medicine at Mount Sinai Hospital in Toronto.

Posted by Guy Crittenden at 3:46 PM | Permalink | Comments (0)

I thought readers would be interested in John Barber's recent column that's critical of incineration plans in regions that ring the Greater Toronto Area. I also invite waste management professionals who live in Southern Ontario to consider attending next week's MWIN conference and, especially, the panel on thermal treatment that I'm chairing on the final day (Thursday, June 14 in Cobourg). You can get the details and register at www.mwin.org

The evidence is in: Halton's incinerator folly is toast

The Globe And Mail -- Wednesday, June 6, 2007

JOHN BARBER

Once again, on behalf of all Torontonians, allow me to extend sincere gratitude to the suburban municipalities now flirting seriously with incineration and similar "thermal treatments" of household wastes. Every step forward they take reconfirms the folly of their path.

But the department headed by Bob Nosal, medical officer of health for Halton Region, deserves special credit for offering the most important public service so far: a scarlet-red flag warning the easily deceived that building any such device, despite prevailing happy talk about "acceptable" levels of pollution, will hurt people - or, to use the phrase preferred by Halton bureaucrats, "be associated with some increase in adverse health impacts."

Until now, the folly of incineration has emerged in the form of inconvenient truths popping out of the environmental assessments of impending new incinerators in Durham and Niagara - hard evidence about emissions, costs and alternatives to replace the easy assurances heard earlier on the sales floor.

Dr. Nosal's intervention is the first rebellion to emerge from within the ranks of the promoters.

It takes the benign form of a peer review of "Step 4a" of the region's plan to build an incinerator, in which it purported to identify and describe the prospective facility's "potential health and environmental effects." Written by medical scientist David Pengelly, recent recipient of a City of Toronto Green award for his work on air quality, the review gently but thoroughly demolishes official assurances that modern incinerators are benign.

"I'm a scientist," Dr. Pengelly said in an interview. "I'm not convinced by assertions, I'm convinced by evidence." The Halton report, he added, offered no evidence to support the contention that modern incinerators, despite being cleaner than their predecessors, are in fact safe. They emit the same dangerous pollutants as earlier incinerators, albeit less of them. But how much is that? Step 4a doesn't say.

"I'm prepared to accept that things are better than they were," he said. "My problem was that there wasn't very specific scientific evidence brought out to show how much better they are."

Dr. Nosal, the official who commissioned the review, is already advocating strict abatement of existing pollution in Halton's already "taxed" airshed - a position unlikely to herald approval of new sources of dangerous pollution. He and his crew deserve "a great deal of credit for taking an active role in making sure that these health issues are addressed right from the very beginning," Dr. Pengelly said. "I can tell you that's not happening in other municipalities."

Leaving aside its welcome expose of incineration's health hazards, the Halton report includes more than enough latent ammunition to destroy any hope a burner might soon be built there. The idea is absurd on its face: Halton's existing landfill is big enough to last until 2030, long before which it could easily be expanded to take garbage until the last person alive today is gone.

Mercifully, the bureaucrats have abandoned their nutty idea that Halton should "take a leadership role" by building a giant incinerator to compete with facilities throughout the province. Unlike some of their colleagues elsewhere, they acknowledge that recent developments - especially the sudden appearance of 50 million tonnes of new landfill capacity in Southern Ontario - have destroyed the viability of such schemes. Faced with the disappointing fact that Halton has no need for an incinerator, they are reduced to recommending a teeny tiny one.

This ongoing retreat is a fascinating event for which suburban taxpayers - and everybody who breathes - should be grateful.

Stripped of its rationale, its hazards exposed, the current push to incinerate is revealed as a kind of infrastructure adventurism, led by a tunnel-visioned cadre of engineers and consultants, that can be brought to a halt with no negative consequences.

jbarber@globeandmail.com

Posted by Guy Crittenden at 10:45 AM | Permalink | Comments (0)