Chalk Up Another for the Economics of Google Searches

The Google search engine continues to demonstrate its use as a tool for predicting human behaviour and activity based on the frequency of topics searched by its users.  Google data has been used to track flu activity and even fertility behaviour.  A Banca D’Italia November 2012 working paper by Francesco D’Amuri and Juri Marcucci is the latest installment on using the predictive power of Google searches and applies it to forecast US unemployment.  As the authors note, such an approach has already been examined for unemployment in Germany, Italy and Israel.

D’Amuri and Marcucci construct a Google Index(GI) of internet job-search intensity and use it as a leading indicator to predict the US monthly unemployment rate.  Indeed, they argue it is the best leading indicator for predicting this variable when comparing it with the use of Initial Claims which is a widely accepted leading indicator for the US unemployment rate.  This they do by estimating standard time series (ARMA) models of monthly unemployment that are grouped into models that include or exclude the GI over assorted time spans.  Generally, the Google-based models perform with lower MSEs.  They conclude:” Notwithstanding its limited time availability (Google data are available since January 2004) we believe that the GI should routinely be included in time series models to predict unemployment dynamics. We fully expect that the use of internet-based data will become widespread in economic research in the near future.” Of course, the rather short time range for which a Google Index (GI) can be constructed for is an issue given that the Initial Claims (IC) data are available going back to 1967. 

The construction of the Google Index (GI) in this case is focused on the use of the keyword “jobs”. According to the authors: “First, we found that the keyword “jobs” was the most popular among different job-search-related keywords… the second reason why we chose the keyword “jobs” is that we believe that it is widely used across the broadest range of job seekers, and as a consequence is less sensitive to the presence of demand or supply shocks specific to subgroups of workers that could bias the values of the GI”.  Combinations of terms such as “public jobs” are also used.  In an attempt to improve precision, authors even subtract the keyword searches for “Steve Jobs” which would undoubtedly be quite important especially given the time range for the GI is post 2004.

Given the prediction made by the authors that the use of internet-based data will become more widespread in economic research in the future, one cannot help but wonder if indicators constructed by such techniques will not be more susceptible to “manipulation.”  Frances Woolley on Twitter recently drew my attention to the “gaming” of Google Scholar citations.  Would it not be possible for indicators of unemployment or retail activity to be “gamed” by having computer programs written and set loose on the internet that input certain search terms on a massive scale in order to create “trends” that can be exploited for marketing or stock market purposes?  Do we want to tie changes in money supply and interest rates to expansions or contractions in economic activity picked up by Google searching if we are not sure where the source of the activity is coming from?  What safeguards can be put in place to filter out any such effects?  I’m not saying older or traditional economic indicators are the best – all data sources have their issues.  However, I think we need to anticipate what the issues of new indicators might be especially in the case of internet-based indicators.

Comments

[Determinant]

Google has no more problems than initial claims, which presumes an unemployed person is eligible for EI. As another piece of information, especially one that helps track those who have fallen through the cracks other methods, it is to be lauded.

The "jobs" search is basic for anyone looking for work.

[Ben J]

Dr Di Matteo, I might be a little late here, but I thought I might add something.

"Would it not be possible for indicators of unemployment or retail activity to be “gamed” by having computer programs written and set loose on the internet that input certain search terms on a massive scale in order to create “trends” that can be exploited for marketing or stock market purposes?"

I find it unlikely that Google would be unable to determine the difference between a legitimate increase in searches for 'jobs' and some manipulated surge in searches. They have an enormous vested interested in preserving the validity of search results and data, as that validity is an integral part of the product they provide to advertisers. Determining what people are 'really' looking for is one of the most important functions, and from all the previous work on flu-trends etc, they obviously have a lot of experience dealing with search inputs.

Were someone to attempt to 'game' a Google-based job indicator, I think Google would be fighting hard to stop them from the get-go. Furthermore, there are already large financial incentives to discovering an effective method of gaming Google in some shape or form, almost all of which would be more concrete (and more immediately profitable) than goosing a leading indicator and going long on stocks.

Very interesting post though, I look forward to seeing how the indicator works in future!

Cheers.

[Livio Di Matteo]

@Ben
Not late at all! That is a good point re Google's likely response.

[Nick Rowe]

Possibly dumb question: I wonder if it would also be possible to use this Google search data to test economic theories. Especially for things we don't have other data on. For example, my hunch is that Google searches for "barter" would be positively correlated with "jobs".

[Ben J]

Nick,

The Google Trends tool doesn't seem to have fine enough data for me to get any useful comparison between jobs and barter. I.e. the scale is too large to show up searches for barter. The tool is fun, but for serious academic work you obviously need the full data set.

Here is "jobs" and "barter". The big peak in October 2011 was the death of Steve Jobs (obviously why it needed to be removed).

http://www.google.com.au/trends/explore#q=jobs%2C%20barter&cmpt=q

Barter alone has a very gentle trend upwards in 2009:

http://www.google.com.au/trends/explore#q=barter&date=3%2F2008%2055m&cmpt=q

Jobs does as well, but it is crying out for seasonal adjustment:

http://www.google.com.au/trends/explore#q=jobs&date=3%2F2008%2055m&cmpt=q

Can you imagine a future where the BLS pays Google a monthly fee for this data? I can! It's an exciting thought.

[Nick Rowe]

Cool! Thanks Ben. I tried to extend the "Barter" series for more years, but I screwed something up, and it said I had reached my quota, or something?

[Ben J]

That is strange Nick! Perhaps you have to be logged into a Google account to use it without limits.

Here is the series for as big a range you can get, 2004 to present:

http://www.google.com.au/trends/explore#q=barter&cmpt=q

Any idea what that huge spike in July 2012 would be? None of the news stories Google picks up are around that date.

For me, that big spike was washing out the trend slightly, so I made another graph where the series ends in May 2012 (is that cheating?):

http://www.google.com.au/trends/explore#q=barter&date=1%2F2004%20101m&cmpt=q

I see a small decline from 2004 onwards, reversing around the middle of 2008. That seems to fit the business cycle alright. Another interesting search might be through Google Books, checking the frequency of the word 'barter' in US books published over the last century against the business cycle.

[Ben J]

Oh! I think I found the July 2012 spike:

http://www.google.com.au/trends/explore#q=barter%2C%20barter%20kings&cmpt=q

Barter Kings is a US tv show that premiered in June 2012. That's probably what the spike is. Shows just how low volume 'barter' searches are.

This almost feels like Detective work.

[Frances Woolley]

This is a dramatic one...

[Frances Woolley]

sorry that link didn't work, how about this >here.

[Frances Woolley]

O.k., that didn't work either. Just search for fiscal cliff.

[DePuy Lawsuit Updates]

That's cool. It really works, thanks for sharing.