Everyone knows - and should have known - that the numbers from the National Household Survey (NHS) would be dodgy. Statistics Canada has always claimed that the NHS numbers would be useful for many purposes, and this line has been swallowed by many. After all, Statistics Canada has a deserved reputation for professionalism, and their work deserved the benefit of the doubt.
No longer. It took Frances Woolley only a couple of hours to come across some seriously wonky numbers from the very first NHS release. Moreover, StatsCan isn't planning to release the technical documentation for the NHS until 2014. (In the ordinary course of things, methodology is presented before results.) And now there's today's last-minute delay in the NHS income numbers.
Chief Statistician Wayne Smith was telling us a few weeks ago
It’s irresponsible to try and dissuade Canadians from using what is an extraordinary rich and powerful database. To make them nervous about that is I think irresponsible.
It's entirely reasonable to be skeptical about numbers generated by an untried and untested methodology in the absence of technical documentation. Many people - me included - were prepared to give StatsCan the benefit of the doubt. They no longer deserve it.
I don't think it is anomalous that the NHS is releasing technical docs after the data. That was the norm with the Census, as I recall.
Here is 2006 doc with some release date info. (I couldn't find the full release date schedule.)
http://www12.statcan.gc.ca/census-recensement/2006/ref/preview-avantgout/pdf/812x11-eng.pdf
On page 22 it says "Remaining guides to be released approximately 1 and 3 months after the applicable major day of release. " which I interpret to mean they release the guide 1 to 3 months after the data.
P. 21 also says that tech reports on sampling and weighting weren't out until Winter 2010 for the 2006 Census.
I agree that it would make most sense to have the technical docs along with the data, I wouldn't infer anything from the fact that the technical docs are lagging the data releases from the NHS.
Any Statcan lurkers out there want to explain why technical doc releases lag the data releases?
Posted by: Kevin Milligan | August 12, 2013 at 07:15 PM
Har. Statcan lurkers will want to remain that way, unless they're interested in getting canned.
Posted by: Andrew F | August 12, 2013 at 08:13 PM
You guys might want to take note that since the loss of the mandatory long form census, StatsCan had been proudly rebranded, in the Harper "I make the rules" tradition, as "TrendsCan".
Oh their data is reliable, just view the findings as "trends" and don't try to apply rigorous statistical analysis to them (a colleague of mine used to admonish me not to waste my good thoughts on dirty data, so see, I am just passing on her pearls of wisdom to you). I can further assure you that the large majority of the taxpaying proletariat would not know the difference between a biased and nonbiased sample. This is why Wallin is in over her head, referring to "flawed" analysis and so forth in her complaint about the auditor's report, but that is another issue.
So, my friends, we should all rejoice that people like the Harper appointed Wayne Smith are still toiling to squeeze every ounce of stats., er, trend, from the NHS database. lol
Posted by: Anon in Harperland | August 13, 2013 at 11:45 AM
It's actually really sad.
Posted by: Sina Motamedi | August 20, 2013 at 06:45 PM