The economics of census data

One of the surprising things about the census fiasco is that of all the publicy-provided services that small-government advocates could target, the census is very near the bottom of the list of priorities. Many of the services provided by governments could and perhaps should be produced by the private sector. But the economics of databases such as the census aren't the same as the goods that inhabit most economic models, and so the standard market-is-best results of the First and Second Welfare Theorems don't apply. When it comes to things like the census, markets fail.

Standard public finance theory offers two dimensions in which goods can be classified: rivalry and excludability. If a good is a purely private - that is, rival and excludable - then the welfare theorems apply, and there's no obvious case for government intervention. But a database is different. Although intellectual property laws can provide a mechanism for owners to control access, it is non-rival: many users can benefit from a database just as easily as one can.

You can see why this means that markets would generate inefficient outcomes here: firms that require certain information would be obliged to produce multiple data sets, all serving pretty much the same purpose. The inefficiency takes the form of this duplication of efforts. It's more efficient to simply have one database and allow access to multiple users. In other words, the census is a natural monopoly.

But what pricing strategy should the census monopolist adopt? Usually, we'd try to set price equal to marginal cost. In this case, the marginal cost of providing access to a database is essentially zero: once it is set up, maintenance costs are trivial. So in an ideal situation, the census would become a pure public good: universal access at zero cost.

This is pretty much the model adopted by the United States. A trip to the websites of the Bureau of Economic Analysis or the Bureau of Labor Statistics will get you a wealth of data at the price of a couple of mouse clicks. And over at the Census Bureau, fees only seem to be incurred by those whose projects require working at their offices in Washington; I can't see any mention of fees for access to the data per se. (Readers more familiar with these matters will correct me if I've read things incorrectly.)

Things are different in Canada. If you go looking for data at the Statistics Canada website, you end up looking at a price list. For reasons going back to the Mulroney-era budget cuts, Statistics Canada is expected to generate a significant portion of its revenues. In 2009, some 20% of its $500m budget came from selling access to its data. Private-sector clients are not the only people who pay for StatsCan data: government departments and agencies at all levels must pay for access as well. (A significant chunk of the Parliamentary Budget Office's budget goes to Statistics Canada.)

The effect of this pricing model on research activity is considerable, setting the stage for a complicated shell game that would be comical if it wasn't so detrimental to the advancement of our understanding. A Canadian professor who works at a publicly-funded university must obtain research grants from publicly-funded institution in order to pay for data - that have already been collected! - from yet another government agency. Too often, the game ends with the researcher abandoning the project or choosing to use US data instead.

And now things will get much, much worse.

Comments

[Frances Woolley]

Too true - and at least the Canadian professor has the know-how, library facilities and potential public support to access the library.

What about non-profit and advocacy groups? Say you're a person advocating for the disabled, and you want to know 'how many disabled people are there in my community who might potentially use a dedicated swim time?' This is not a left-right issue - everyone from Christian home schoolers to G-20 protestors to G-20 protesting home schoolers needs information.

[Jim Rootham]

I think we should take advantage of Harper poking this hornet's nest to press for free access to the data. Lean on the opposition parties.

[Jacques Marcil]

Stephen, isn't Laval part of a Cansim consortium agreement with StatCan? I know that UofT and UofC are. Through those agreements, non-profits pay an annual flat fee and in return they get "all-you-can-download" Cansim access. Not free, but still cheaper than $3 a pop.

Jacques

[Stephen Gordon]

Yes, but that's the point: there's a fee.

[Jacques Marcil]

OK, I see your point.

Note however that based on my experience (at StatCan and Ontario Finance), the maintenance fees are not trivial. In the case of Cansim, there is a permanent team of about 10 people running the thing, plus the IT support, plus IT development (to stay ahead of the curve) plus the time spent by staff in all the data-producing areas loading and verifying the data on a daily basis.

Additionally, there is more and more data available for free on the StatCan website, however not under Cansim.

From an external user's point of view though, the odd animal in this is Industry Canada's Trade Data Online service, which is free for all to use but based on StatCan data.

But then, maybe this is a bit too specialized for this forum. We can continue this chat offline if you want.

[ThomasL]

This post misses on important point. It takes for granted that the person wishing access to information has a more compelling interest and right than a person that would not like their information disclosed.

If the controversy of the census were simply that some people (the government) get access to the data for free while other parties must pay for the same access, I would tend to agree with your assessment - treat it as a public good and let everyone have full access.

However, the controversy over the census is really about increasingly intrusive questions that must be answered on penalty of jail.

The implication is that the curiosity of any bean counter, actuary, or researcher must be satisfied whether the subject of their study is a willing participant or not.

[Frances Woolley]

Jacques Marcil wrote:

Additionally, there is more and more data available for free on the StatCan website, however not under Cansim.

Jacques - as a long-term and avid user of the free data on www.statcan.gc.ca, I beg to differ. Yes, there is a lot of data there.

But data that you'd think would be basic information is really hard to find. E.g. a while ago I was looking for average age at first marriage going back over the past 50 years or so - perhaps it's on the Statcan website somewhere, but after 1/2 hour or so of looking I ended up using US data instead. And I'm an experienced and fast user.

And it's really hard to find what you want - it's like looking for a needle in a haystack. I was just trying to pull up the community census profile for Burnaby, BC - it took me about 10 minutes to find that, and I knew exactly what I was looking for.

[@Wendy_Waters]

Because it's so hard to find stuff on the Stats Can website, the private sector can play a role here.

In my job(strategy research division of a company that manages pension fund real estate assets) I regularly purchase census derived data and software from several different private sector clients. Sometimes we pay someone to both find and analyze the census data. This is often more efficient than me spending hours looking for it--they know exactly where it is, and how to interpret it.

If there is no long form census data, going forward, I/we will be hiring fewer economic and demographic consultants, for example. We will not be buying data updates for demographic mapping software (that uses the long form), etc. If there is no data to analyze, we won't be paying someone to find it and help us to analyze or interpret it.

We do also occasionally pay for polls, on much narrower topics and to understand motivators, but these polls and questions are also based on knowledge from the long form. If we no longer have that knowledge, then it may not help us much to do a poll.

There's an entire industry or many industries based around the long form census data. The long form is great for the private sector. An opaque stats can website is great for the private sector too.

[Another civil servant]

The comment by ThomasL and others reveal what might be a key issue in this debate: a lot of people seem to think that their personal information will be available (eg "than a person that would not like their information disclosed".)

I know it's been stated clearly by Stephen and others, but perhaps it should be repeated every other sentence or something: the census data does NOT contain personal information per se. Nobody using census data can guess who responded what to a given question.

There is a valid concern (IMO) that has not been properly addressed until now, the fact that census responses are reviewed by people from the same community. I don't think I'd care in a city, as the odds that I know the reviewer are very slim. But if I knew the reviewer, I'd be uncomfortable about some of the questions. I haven't yet come up with a good alternative to the local reviewer - use other sources (eg municipal records?) to validate the information about the house? I'm sure others will have better ideas on how to work around that problem.

[ThomasL]

@Another civil servant

I think you are misunderstanding privacy. You define it as if it were the same as anonymity, but the two are not perfectly interchangeable states.

If some one comes up with a clipboard and asks me, "Hey, have you ever had an STD? I'm compiling statistics for incident rates to better allocate education and health funding. But don't worry, it is all anonymous."

My answer is not, "Oh, ok, as long as it's anonymous." It is, "Get the $@##$%! away from me, pervert."

I made up a deliberately offensive question that a very high number of people would refuse to answer even if they knew the results were anonymous. Why? Because it is none of your business. It is my business. The census questions are not (yet) that offensive, but they are getting there. Most importantly your argument that "anonymity is privacy" is of equal use to justify any question, no matter how personal or offensive. With that argument, there is no imaginable question too offensive for the state to ask and to require an answer.

I don't think these questions should be asked as part of the census at all--the census is, after all, about counting people; the only answer I should need to make is, "Present"--but if they are asked, it certainly should be my prerogative to say, "Get out of my face," without going to jail.

Just because some researcher came up with some interesting question and convinced some bureaucrats that the answer is worth knowing, does not make it his right to know, nor my obligation to tell him.

[Paul Tonita]

ThomasL wrote:

"However, the controversy over the census is really about increasingly intrusive questions that must be answered on penalty of jail."

How is it increasingly intrusive? There is an awful lot of subjectivity in this debate, and the government is so far loathe to comment on the objective facts. Facts that this blog and others have routinely been hammering the government with for two weeks now.

There are far more intrusive actions by the government and arguably with programs that do not produce the same utility as the census data do. The Ottawa Citizen had a good article today titled How the federal government can really defend privacy, which details one such provision the government uses, one which the privacy commissioner has warned us of.

[Determinant]

@Another Civil Servant:

Statistics Canada doesn't use local reviewers as far as I know. 98% of all census forms are mailed out to households by Canada Post and respondents mail them back. You can also do it online which was used by 18% of people last time.

Enumerators are only sent out to Indian Reserves which are known to be hostile to anything related to the Government of Canada (often they don't vote either) and downtown neighbourhoods with large numbers of transients.

Forms are centrally processed.

Besides, StatsCan employees are bound by law to confidentiality. You could level the same argument at Elections Canada staff. I was a poll clerk last time. If a staffer keeps a list matching ballot numbers to names, the secrecy of the vote would be blown to bits. I served people I knew quite well. They trusted me and I obeyed the law.

Speaking of intrusiveness, I got a long-form census last time and was flagged for further discussion because I am a Type I diabetic. I got a battery of questions which were actually relevant and intelligent. You get used to a lot of ignorance and stupidity when discussing diabetes with the general public. The only really annoying thing was that they kept my family home when I was at University.

[(first) Another civil servant]

Determinant: in the comments to SG earlier census posts, someone said that even forms mailed back or filled online were sent back to the area's reviewers (or whatever they're called). That person commented that he knew the reviewer and felt uncomfortable knowing that person would be seeing (and challenging) his answers - which I can undertand, so I'm trying to think of a way around it. Even if the reviewer is bound by, and respects, the confidentiality clause, many people can feel discomfort at revealing these details to someone they see at their supermarket or the kids soccer games.

(Added (first) to my signature because someone else started using "Another civil servant" yesterday)

[westslope]

'Cost recovery' is a whole other debate.

There is probably more basic data that StatCan should make available at no charge. Otherwise the use fees serve the purpose of disciplining demand and guiding resource allocation decisions within StatCan.

[Ed Seedhouse]

Logically, if you object to the long form then surely you should also object to the short form census as well. Both are presently compulsory, and if the long form being compulsory is objectionable on that grounds, then clearly so is the short form.

The argument for the census is that it is necessary, and a justifiable infringement of liberty. If the claim that it is necessary is valid then it is hard to see why the long form would not be justifiable on the same grounds.

If the government were consistent, they would dump the short form census as well. But to proceed with a compulsory short form census and dump the compulsory long form seems irrational.

[David Gendron]

"One of the surprising things about the census fiasco is that of all the publicy-provided services that small-government advocates could target, the census is very near the bottom of the list of priorities. "

In a statist minachist capitalist premise, I agree. Maybe that's another reason why libertarian-conservatives are so confused on this debate.

Their cause is for a small capitalist government for the wealthy class.

My cause is for no government.

[Stephen Gordon]

No government? No criminal code? No contract enforcement? Survival of the most ruthlessly violent?

Sorry, I rather prefer civilization, but thanks for sharing your world view.

[20th century workforce]

Hey David, why don't you move to Somalia?

[20th century workforce]

...is an honest question. David, do you believe a stateless state like Somalia is preferable? Reformers and 1/2 our Cabinet want to go halfway to Somalia, simply because they get to be the warlords with tar revenue. I'm curious how far the GOP would take the world to the edge before remaking themselves into a PC or UK right-wing form of sanity. I'm hoping our RW will loyally follow them to sanity just like they are following GOP to insanity.

[20th century workforce]

Political discourse in USA has been very dumb ever since the free marketeers got strong during Reagan. So much effort wasted just trying to deprogramme people that market forces are only one component of economics, they don't even realize their country has turned into a corporatist shithole and only a severe recession can shock them temporarily out of electing retards.

We are on that same path; Albertans were hypnotized by market forces and that's who we have in government. Health Canada just raised alarm bells about healthcare and we are debating Harper et al's attempt to undercount poor people in providing public and private services (which costs 1.75x as much for healthcare according to USA model).