In this post we describe the autoregressive (AR) time series model. We define it, describe steps to take before fitting one, how to choose the number of lags, and how to interpret the resulting fitted model.
The autoregressive model posits that values of a time series are correlated with and depend on recent past values. Intuitive examples may include: moods tend to persist but return to baseline and sales tend to persist. In particular, the model describes how recent past values or ‘lags’ linearly affect the current value of a time series.
AR Model: Equation
Let be a univariate time series. An AR(p) model states
here is a white noise process. If it is an independent white noise process then i.e. we have a Markov property.
First Steps When Fitting an AR Model
Plot and Check for Stationarity
We should first visualize the time series and check for stationarity. We want both constant mean and constant autocovariance. Without stationarity the resulting model will be wrong: often either the white noise process assumption will be violated or the process will be explosive. Let’s load the BJsales dataset and plot it.
library(tseries) data('BJsales') plot(BJsales)
This clearly has a trend. Let’s first difference and look again.
This looks somewhat better. The variance seems to be changing over time though: sometimes it looks small for a while and then we see periods with larger jumps. Let’s do an augmented Dickey Fuller test to check more formally.
print(adf.test(diff(BJsales))) Augmented Dickey-Fuller Test data: diff(BJsales) Dickey-Fuller = -3.3485, Lag order = 5, p-value = 0.06585 alternative hypothesis: stationary
It’s on the border with a p-value of 0.066. It’s not perfect but we can use it. We could difference again but for this exercise we won’t.
Compare ACF and PACF
The next step is to compare the ACF and PACF plots. In order to decide to use an AR(p) model, we should see approximately geometric decay in the ACF plot, and p significant terms in the PACF plot.
It looks like the ACF plot has approximately geometric decay, while the PACF plot has two significant terms. This suggests an AR(2) model.
Fitting and Interpreting the Model
There are two standard ways to choose lags. One is to use the PACF plot as we did above, and one is to use AIC. Based on the pacf plot we choose an AR(2) model. We also fit a model fit using AIC to choose lags.
ar_model_pacf Call: ar(x = y, aic = FALSE, order.max = 2) Coefficients: 1 2 0.2493 0.2005 Order selected 2 sigma^2 estimated as 1.832 ar_model_aic Call: ar(x = y, aic = TRUE) Coefficients: 1 2 3 4 0.2123 0.1493 0.0776 0.1383 Order selected 4 sigma^2 estimated as 1.8
Interpreting the Model
The first model, chosen using the pacf plot, tells us
this says that the fitted difference between this period’s sales and last period’s sales is 0.2493 times the difference one period ago plus 0.2005 times the difference two periods ago.
Undoing the Differencing
However let’s say we want to predict the actual sales and not the changes in sales, or describe the model for actual sales. Let be the sales at time . Then