Sunday, April 30, 2017

One Millionth Birthday...

Image result for 1 year birthday cake
 ...in event time.  It's true, yesterday No Hesitations passed 1,000,000 page views.  Totally humbling.  I am grateful for your interest and support.

Thursday, April 20, 2017

Automated Time-Series Forecasting at Google

Check out this piece on automated time-series forecasting at Google.  It's a fun and quick read. Several aspects are noteworthy.  

On the upside:

-- Forecast combination features prominently -- they combine forecasts from an ensemble of models.  

-- Uncertainty is acknowledged -- they produce interval forecasts, not just point forecasts.

On the downside:

-- There's little to their approach that wasn't well known and widely used in econometrics a quarter century ago (or more).  Might not something like Autobox, which has been around and evolving since the 1970's, do as well or better?

Friday, April 14, 2017

On Pseudo Out-of-Sample Model Selection

Great to see that Hirano and Wright (HW), "Forecasting with Model Uncertainty", finally came out in Econometrica. (Ungated working paper version here.)

HW make two key contributions. First, they characterize rigorously the source of the inefficiency in forecast model selection by pseudo out-of-sample methods (expanding-sample, split-sample, ...), adding invaluable precision to more intuitive discussions like Diebold (2015). (Ungated working paper version here.) Second, and very constructively, they show that certain simulation-based estimators (including bagging) can considerably reduce, if not completely eliminate, the inefficiency.


Abstract: We consider forecasting with uncertainty about the choice of predictor variables. The researcher wants to select a model, estimate the parameters, and use the parameter estimates for forecasting. We investigate the distributional properties of a number of different schemes for model choice and parameter estimation, including: in‐sample model selection using the Akaike information criterion; out‐of‐sample model selection; and splitting the data into subsamples for model selection and parameter estimation. Using a weak‐predictor local asymptotic scheme, we provide a representation result that facilitates comparison of the distributional properties of the procedures and their associated forecast risks. This representation isolates the source of inefficiency in some of these procedures. We develop a simulation procedure that improves the accuracy of the out‐of‐sample and split‐sample methods uniformly over the local parameter space. We also examine how bootstrap aggregation (bagging) affects the local asymptotic risk of the estimators and their associated forecasts. Numerically, we find that for many values of the local parameter, the out‐of‐sample and split‐sample schemes perform poorly if implemented in the conventional way. But they perform well, if implemented in conjunction with our risk‐reduction method or bagging.

Monday, April 10, 2017

BIg Data, Machine Learning, and the Macroeconomy

Coming soon at Bank of Norway:

CALL FOR PAPERS 
Big data, machine learning and the macroeconomy 
Norges Bank, Oslo, 2-3 October 2017 

Data, in both structured and unstructured form, are becoming easily available on an ever increasing scale. To find patterns and make predictions using such big data, machine learning techniques have proven to be extremely valuable in a wide variety of fields. This conference aims to gather researchers using machine learning and big data to answer challenges relevant for central banking. 

Examples of questions, and topics, of interest are: 

Forecasting applications and methods
-Can better predictive performance of key economic aggregates (GDP, inflation, etc.) be achieved by using alternative data sources? 
- Does the machine learning tool-kit add value to already well-established forecasting frameworks used at central banks? 

 Causal effects
- How can new sources of data and methods be used learn about the causal mechanism underlying economic fluctuations? 

Text as data
- Communication is at the heart of modern central banking. How does this affect markets? 
- How can textual data be linked to economic concepts like uncertainty, news, and sentiment? 

Confirmed keynote speakers are: 
- Victor Chernozhukov (MIT) 
- Matt Taddy (Microsoft, Chicago Booth) 

The conference will feature 10-12 papers. If you would like to present a paper, please send a draft or an extended abstract to mlconference@norges-bank.no by 31 July 2017. Authors of accepted papers will be notified by 15 August. For other questions regarding this conference, please send an e-mail to mlconference@norges-bank.no. Conference organizers are Vegard H. Larsen and Leif Anders Thorsrud.

13th Annual Real-Time Conference

Great news: The Bank of Spain will sponsor the 13th annual conference on real-time data analysis, methods, and applications in macroeconomics and finance, next October 19th and 20th , 2017, in its central headquarters in Madrid, c/ Alcalá, 48. 

The real-time conference has always been unique and valuable. I'm very happy to see the Bank of Spain confirming and promoting its continued vitality.

More information and call for papers here.

Topics include:

• Nowcasting, forecasting and real-time monitoring of macroeconomic and financial conditions.
• The use of real-time data in policy formulation and analysis.
• New real-time macroeconomic and financial databases.
• Real-time modeling and forecasting aspects of high-frequency financial data.
• Survey data, and its use in macro model analysis and evaluation.
• Evaluation of data revision and real-time forecasts, including point forecasts, probability forecasts, density forecasts, risk assessments and decompositions
.

Monday, April 3, 2017

The Latest on the "File Drawer Problem"

The term "file drawer problem" was coined long ago. It refers to the bias in published empirical studies toward "large", or "significant", or "good" estimates. That is, "small"/"insignificant"/"bad" estimates remain unpublished, in file drawers (or, in modern times, on hard drives). Correcting the bias is a tough nut to crack, since little is known about the nature or number of unpublished studies. For the latest, together with references to the relevant earlier literature, see the interesting new NBER working paper, IDENTIFICATION OF AND CORRECTION FOR PUBLICATION BIAS, by Isaiah AndrewsMaximilian Kasy. There's an ungated version and appendix here, and a nice set of slides here.

Abstract: Some empirical results are more likely to be published than others. Such selective publication leads to biased estimators and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study's results, the first based on systematic replication studies and the second based on meta-studies. For known conditional publication probabilities, we propose median-unbiased estimators and associated confidence sets that correct for selective publication. We apply our methods to recent large-scale replication studies in experimental economics and psychology, and to meta-studies of the effects of minimum wages and de-worming programs.

Tuesday, March 28, 2017

Text as Data

"Text as data" is a vibrant and by now well-established field. (Just Google "text as data".)

For an informative overview geared toward econometricians, see the new paper, "Text as Data" by Matthew Gentzkow, Bryan T. Kelly, and Matt Taddy (GKT). (Ungated version here.)

"Text as data" has wide applications in economics. As GKT note:
... in finance, text from financial news, social media, and company filings is used to predict asset price movements and study the causal impact of new information. In macroeconomics, text is used to forecast variation in inflation and unemployment, and estimate the effects of policy uncertainty. In media economics, text from news and social media is used to study the drivers and effects of political slant. In industrial organization and marketing, text from advertisements and product reviews is used to study the drivers of consumer decision making. In political economy, text from politicians’ speeches is used to study the dynamics of political agendas and debate.

There are three key steps:

1. Represent the raw text D as a numerical array x

2. Map x into predicted values yhat of outcomes y

3. Use yhat in subsequent descriptive or causal analysis.

GKT emphasize the ultra-high dimensionality inherent in statistical text analyses, with connections to machine learning, etc.

Tuesday, March 21, 2017

Forecasting and "As-If" Discounting

Check out the fascinating and creative new paper, "Myopia and Discounting", by Xavier Gabaix and David Laibson.

From their abstract (slightly edited):
We assume that perfectly patient agents estimate the value of future events by generating noisy, unbiased simulations and combining those signals with priors to form posteriors. These posterior expectations exhibit as-if discounting: agents make choices as if they were maximizing a stream of known utils weighted by a discount function. This as-if discount function reflects the fact that estimated utils are a combination of signals and priors, so average expectations are optimally shaded toward the mean of the prior distribution, generating behavior that partially mimics the properties of classical time preferences. When the simulation noise has variance that is linear in the event's horizon, the as-if discount function is hyperbolic.
Among other things, then, they provide a rational foundation for the "myopia" associated with hyperbolic discounting.

Note that in the Gabaix-Laibson environment everything depends on how forecast error variance behaves as a function of forecast horizon \(h\). But we know a lot about that. For example, in linear covariance-stationary \(I(0)\) environments, optimal forecast error variance grows with \(h\) at a decreasing rate, approaching the unconditional variance from below. Hence it cannot grow linearly with \(h\), which is what produces hyperbolic as-if discounting. In contrast, in non-stationary \(I(1)\) environments, optimal forecast error variance does eventually grow linearly with \(h\). In a random walk, for example, \(h\)-step-ahead optimal forecast error variance is just \(h \sigma^2\), where \( \sigma^2\) is the innovation variance. It would be fascinating to put people in \(I(1)\) vs. \(I(0)\) laboratory environments and see if hyperbolic as-if discounting arises in \(I(1)\) cases but not in \(I(0)\) cases.

Sunday, March 19, 2017

ML and Metrics VIII: The New Predictive Econometric Modeling

[Click on "Machine Learning" at right for earlier "Machine Learning and Econometrics" posts.]

We econometricians need -- and have always had -- cross section and time series ("micro econometrics" and "macro/financial econometrics"), causal estimation and predictive modeling, structural and non-structural. And all continue to thrive.

But there's a new twist, happening now, making this an unusually exciting time in econometrics. P
redictive econometric modeling is not only alive and well, but also blossoming anew, this time at the interface of micro-econometrics and machine learning. A fine example is the new Kleinberg, Lakkaraju, Leskovic, Ludwig and Mullainathan paper, “Human Decisions and Machine Predictions”, NBER Working Paper 23180 (February 2017).

Good predictions promote good decisions, and econometrics is ultimately about helping people to make good decisions. Hence the new developments, driven by advances in machine learning, are most welcome contributions to a long and distinguished predictive econometric modeling tradition.