« Living in a demand-side world | Main | Flows in and out of employment during the recession »


Feed You can follow this conversation by subscribing to the comment feed for this post.

You can also find the reference through the national library AMICUS link which takes you to
I found the AMICUS link using the wonderful google scholar link.
However, the loss of government research papers is a real problem that we should keep after.
Disk space is cheap. SOme of the universities should consider archiving key research.

Jciconsult - are there any particular papers that you have in mind?

Also - does anyone know - does it violate copyright to scan a government document and post it on-line?

Everything (with the rare exception of data made available under alternative, open licenses) produced by the government is under Crown Copyright: "Permission to reproduce Government of Canada works, in part or in whole, and by any means, for personal or public non-commercial purposes, or for cost-recovery purposes, is not required, unless otherwise specified in the material you wish to reproduce" (http://publications.gc.ca/site/eng/ccl/aboutCrownCopyright.html).

There is a philosophy in the #goc web manager/comms community to remove old content from websites, particularly if web metrics indicate they are rarely visited pages. The progressive side of this philosophy is about improving usability of websites so users can more easily accomplish their tasks. The regressive side is that niche information gets lost over time. The assumption is that improvements in usability will make it easier to find niche information, but the reality is that the web of government web properties is enormous, of dubious quality (from a technical web design and from a metadata perspective), and web governance is inconsistent, so usability improvements aren't enough to surface old information.

There is even a current challenge in the #goc web community to "reduce the ROT (redundant, outdated, trivial content)", see: http://aboutness.ca/2011/04/20/of-rot-and-puffery/ , http://spaghettitesting.ca/2010/11/04/reduce-the-rot-or-not/ , http://usability4government.wordpress.com/2010/05/05/reduce-the-rot/

The paradox is that while it makes sense to refine web content to make it easier for users to find what they are looking for the majority of the time, there will always be someone looking for the old, archived version of a document because it contains some information that a user needs.

My concern about this practice of reducing ROT is that it lends itself to revisionary, summary narratives of government-produced information. While the motives for the practice are noble, the result is a constant "memory hole" effect perpetuated by well-intentioned bureaucrats adhering to evaluative criteria for archival decisions that naturally lead to old reports getting lost because they don't attract much web traffic.

Jesse - thanks so much for this comment, very thoughtful and informative.

People frequently complain about JStor and similar organizations, and the difficulty of accessing academic work behind paywall.

However it is a non-trivial business to find old articles, scan them, and make them searchable and machine readable. I know that when JStor was putting the back issues of Canadian Journal of Economics and Canadian Public Policy on-line finding a complete set of the journals was a non-trivial business - certainly the Canadian Economics Association didn't have a complete set.

There are a few researchers who have devoted a lot of time and effort to scanning and preserving and making accessible old government documents - see, e.g., the historical census project - but not a huge number.

Coincidentally, this week the World Bank announced an Open Access policy for all of their manuscripts and accompanying data sets.

Though as Jesse points out, "unprotected" doesn't necessarily mean "accessible".

In the last month, I have experienced missing publications from both CMHC and NRCan. Very frustrating. AANDC has changed all their URLs to a numeric string, resulting in a real headache finding stuff, understanding the content of a page (say from an email link), and of course, a significant number of broken links.

On a related note, yesterday I was talking to a friend who works at Statscan. According to her, the recent cuts are going to decimate the service. Not only is its budget cut by 10% but many more of their "government clients" are going to cut back on their contracts to Statscan. She heard as many as 20% of the staff might be laid off...

FYI I haven't commented here in a while but I am going to work in Ottawa starting in June. Hopefully I'll run into some of you WCIers in the near future!

MDB - do you have any specific examples?

Guillaume - wonderful (on you coming to Ottawa), I hope that we'll have a chance to connect.

Interesting news on Statscan... the question is whether surveys will be cut, or analysis capability? My guess is the later.

Did you try (older technology) Canadian Research Index?
"The Canadian Research Index (formerly known as Microlog) includes all depository publications of research value issued by both the federal government and the ten provinces and three territories; hard to find non-depository publications issued by hundreds of Canadian government agencies and departments; scientific and technical report literature issued by research institutes and government laboratories; policy, social, economic, and political reports; Statistics Canada monographs and serials; selected statistical publications from other government agenices; and theses and dissertations(1990 - 1998 only) from Canadian universities.
All documents found in the database are available in the Microlog microfiche collection"

Your library subscribes. Provides detailed citations to monographs and serial publications in the Microlog microfiche collection located in MADGIC. Authorized Users: Carleton students, faculty and staff
Access Notes: To find the item in the library, record the MN number from the database and take it to the Maps, Data, and Government Information Centre.

Your (Carleton) library has the report in print format and e-formats (all 280 pages) It is part of the Canadian Public Policy Collection.
See http://preview.tinyurl.com/72rapul

Carleton gets it via ebrary and Scholars Portal
"Connect via Scholars Portal
via ebrary"
"A collection of publications from Canadian public policy institutes, government agencies, advocacy groups, think-tanks, university research centres and other public interest groups. The organizations included in this collection represent the leading edge of primary research and opinion in all areas of Canadian public policy. Their publications are vital to the understanding of developing issues in every arena of Canadian public life. There are over 12,000 titles in the current (2004-2009) collection, and the service provides more than 3,000 additional current documents to subscribers each year." - from the publisher's website. Date Coverage: 2004 - present

Have a good browse.

Bill, thanks for that link. As the former business editor of Canadian Public Policy (the journal) I'd been aware of the existence of Canadian Public Policy (the collection) but hadn't really thought about it other than as something that messed up Canadian Public Policy (the journal)'s search results.

The Canadian public policy collection does have the Polygamy in Canada discussed in the post. It doesn't seem to have older titles, that is, it doesn't seem to have gone back in time and scanned material. E.g. there is a frequently cited and very hard to find publication called "The state of Canada's climate: temperature change in Canada 1895-1991" published by Environment Canada in 1992 - that's not in there.

On the Canadian research index - microfiche is better than nothing, but worse than any other form of access.

But "state of Canada's climate: temperature change in Canada 1895-1991" is part of the Canadian Research Index, and fiche is at Carleon, the Athens of the North. CRI goes back beyond the Canada Public Policy Collection start date of 2002.

And you can use your digital pocket (or DSLR larger camera) to take the "standard" high resolution image from the microfiche projector and then put it up to some OCR tool.

The free IrfanView image browser/mild editor has an OCR attachment for people who don't want to spend money. Most good OCR cost more and people don't want to pay for occasional use. Middling OCR come with the cheap $100 scanners these days.
Ctrl+F OCR on the page http://www.irfanview.com/plugins.htm
You have the Carleton Library Kirtas 2400 high speed scanner for books and such.

While "The state of Canada's climate: temperature change in Canada 1895-1991" may seem rare to you, a quick search of Amicus [ Most people do not notice the first/second AMICUS page and the little radio button for the nation-wide search, not using the default and narrower NLC-BNC option (nowadays the LAC (Lib. Archives Can.) button ] show 73 locations for it giving call numbers in Library of Congress or some Guelph government file system. 23 in Ontario, including many Federal government agencies and such things as OOC (Ottawa Public Lib.) 10 in Quebec province.

Not rare at all.

Worldcat.org will show you D.W. Gullet and J. W. Skinners's "The state of Canada's climate: temperature change in...", and if you answer the right radio buttons will show local instiutions who have it and their distance from you, in a widening circle of North American libraries.
Though the policy of de-accesionning books under the Last Copy in Region or such as LOCKSS policies might mean local physical copies are "lost".

Mr. Anonymous might scan it and upload to Scribd.com as many people do some documents.

R (the free open-source implementation of the S programming language) 's CRAN package CHCN Canadian Historical Climate Network scrapes climate data from EnviroCan is you needed numbers. SEAS Seasonal analysis and graphics, especially for climatology does some similar things. There are others for world or NOAA data.

There are ways of making it talk.

Bill - Scribd.com is cool, thanks for linking to it.

The climate change document is actually in the Carleton library in hard copy, and that's where I'd go to read it - I've used a microfiche about once in the last ten years.

My point isn't so much that these documents are lost for ever (though some might be), it's more that they're so hard to find - stored on microfiche, for example - that people are unlikely to ever encounter them.

I recently had to root around to find that OTHER Drummond report (about renewing labour statistics). It WAS on HRSDC, but it seems to have vanished, only to resurface on this obscure corner of the web: http://www.flmm-lmi.org
The study seems destined to be forgotten given the Statscan and HRSDC cuts.

Statscan itself put in an archive policy for old publications and releases last year -- fairly innocuous given how digging up what you want on that site can be a challenge at the best of times. The archived material is still available, as far as I've seen.

What's perhaps of greater concern are the preservation practices for microdata and survey documentation at Statcan. I sense some slippage there. In the old days, "Data Stewardship" was the mantra. Now, microdata seem to be viewed as disposable a few years after publication. Since some of the more profound trends play out over very long time frames, such as four full business cycles for capital investment, this slippage might have severe repurcusions later on. Now that a good deal of knowledge is about to walk out the door at Statscan, I fear we could lose some things of immense value.

I wonder if the Open Government initiatives recently announced ( http://www.open.gc.ca/open-ouvert/ap-pa03-eng.asp#toc4 ) may be an opportunity to address this, for instance through the "Open Government Directive" (which could include requirements on archiving)or the "Virtual Library" (by requiring that the repository include older documents). There's a role for civil society in the implementation and monitoring of the Canadian action plan...

Anonymous, thanks for that tip.

Today's Guardian (London, UK) might be of interest
Linkname: UK open government data: the results of the official audit |
News | guardian.co.uk

"The National Audit Office has just published its verdict on the UK government's open data project. Find out what the report said - and get the indicators here:
o [72]Timeline of UK government data
o [73]Get the data

" This should be a good week for open [77]government data in the UK. The British government is one of the key drivers in the [78]Open GovernmentPartnership, presently [79]meeting in Brasilia, where it is being lauded for the way it has released a "tsunami of data"."

"And yet, according to the [80]National Audit Office, all is not
entirely rosy. Read between the lines of its report out today,
Implementing Transparency, and you will see a government which has been
chucking out tonnes of data, that no-one looks at and without a
complete strategy. Oh and it's cost an awful lot of money.

Crucially, it found the Cabinet office seemed to have no idea of how
much the transparency agenda would cost or what it would do:

The Cabinet Office ... not yet systematically assessed the costs and
benefits of the Government's specific transparency initiatives

Also that the Cabinet Office hadn't done enough to make sure
departments knew how to release the data, making it less useful:

The Cabinet Office has not yet defined how departments should
prepare and disclose data inventories to facilitate wider use

How bad are things? These are the key findings. [ more ]

And I found a copy of D. W. Gullet's "The state of Canada's Climate: Temperature change in Canada 1895-1991" and find that it has pretty coloured maps but few data tables.

Where's the data set?
I found one of Gullet's datasets, as differentials, in the U.S. as:
Canadian National and Regional Temperature Departures (1895-1992), in CDIAC, Trends '93
"Abstract: This dataset represents Canadian national and regional average temperature departures (from a 1951-1980 reference) derived from data contained in the Historical Canadian Climate Database (HCCD) for the period 1895-1992. The HCCD contains homogeneous maximum, minimum, and mean temperature data sets for 131 locations in Canada. "

"The individual ... station departures series were combined into regional series representing 11 Canadian climate regions, outlined in the State of the Environment Report, SOE 92-2 (Gullett and Skinner, 1992). The 11 regions are: ..."
At: http://gcmd.nasa.gov/records/GCMD_CDIAC_CANADA_TEMPS_TRENDS.html

I was trying to find absolute numbers and found this (google-wise)
The www.cccma.ec.gc.ca/hccd/
AHCCD: Adjusted Historical Canadian Climate Data

But it is now moved to

Adjusted and Homogenized Canadian Climate Data (AHCCD)


From a Twitter feed

Canada Joins Intl Open Gov Partnership http://www.tbs-sct.gc.ca/media/nr-cp/2012/0418-eng.asp announces Open Gov Action Plan: http://www.open.gc.ca/open-ouvert/ap-pa02-eng.asp#toc3 (run by Ministry of Truth?)

Bill Lee, thanks for the Guardian article, it's fascinating and very timely. The Canadian action plan is what I linked to earlier.

The comments to this entry are closed.

Search this site

  • Google

Blog powered by Typepad