Data Transformation and Frequency Conversion

The data transformation and the frequency conversion functions convert a data series into a different format.

Data Transformation Functions

Function

Purpose

boxcox

Box-Cox transformation

diff

Differencing

fillts

Fill missing values

filter

Filter

lagts

Lag time series object

leadts

Lead time series object

peravg

Periodic average

smoothts

Smooth data

tsmovavg

Moving average

Frequency Conversion Functions 

Function

New Frequency

convertto

As specified

resamplets

As specified

toannual

Annual

todaily

Daily

tomonthly

Monthly

toquarterly

Quarterly

tosemi

Semiannually

toweekly

Weekly

As an example look at boxcox, the Box-Cox transformation function. This function transforms the data series contained in a financial time series object into another set of data series with relatively normal distributions.

First create a financial time series object from the supplied whirlpool.dat data file.

whrl = ascii2fts('whirlpool.dat', 1, 2, [])
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In ascii2fts (line 64) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
whrl = 
 
    desc:  Whirlpool	(WHR)				
    freq:  Unknown (0)

    'dates:  (1108)'    'Open:  (1108)'    'High:  (1108)'    'Low:  (1108)'    'Close:  (1108)'    'Volume:  (1108)'
    '03-Jan-1995'       [        50.25]    [        50.88]    [       50.13]    [         50.38]    [      159000.00]
    '04-Jan-1995'       [        50.00]    [        50.13]    [       49.63]    [         49.75]    [      365800.00]
    '05-Jan-1995'       [        50.00]    [        50.00]    [       49.25]    [         49.88]    [      410500.00]
    '06-Jan-1995'       [        50.00]    [        50.25]    [       49.88]    [         50.25]    [      192900.00]
    '09-Jan-1995'       [        49.88]    [        50.38]    [       49.63]    [         50.38]    [      180800.00]
    '10-Jan-1995'       [        51.25]    [        51.25]    [       50.25]    [         50.25]    [      157800.00]
    '11-Jan-1995'       [        50.38]    [        50.63]    [       49.75]    [         49.75]    [      266000.00]
    '12-Jan-1995'       [        49.88]    [        50.13]    [       49.25]    [         49.88]    [       70000.00]
    '13-Jan-1995'       [        50.00]    [        50.25]    [       49.75]    [         50.13]    [      128000.00]
    '16-Jan-1995'       [        50.25]    [        50.38]    [       50.00]    [         50.00]    [      228200.00] ...

Fill any missing values denoted with NaNs in whrl with values calculated using the linear method:

f_whrl = fillts(whrl)
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/fillts (line 213) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
f_whrl = 
 
    desc:  Filled Whirlpool	(WHR)				
    freq:  Unknown (0)

    'dates:  (1108)'    'Open:  (1108)'    'High:  (1108)'    'Low:  (1108)'    'Close:  (1108)'    'Volume:  (1108)'
    '03-Jan-1995'       [        50.25]    [        50.88]    [       50.13]    [         50.38]    [      159000.00]
    '04-Jan-1995'       [        50.00]    [        50.13]    [       49.63]    [         49.75]    [      365800.00]
    '05-Jan-1995'       [        50.00]    [        50.00]    [       49.25]    [         49.88]    [      410500.00]
    '06-Jan-1995'       [        50.00]    [        50.25]    [       49.88]    [         50.25]    [      192900.00]
    '09-Jan-1995'       [        49.88]    [        50.38]    [       49.63]    [         50.38]    [      180800.00]
    '10-Jan-1995'       [        51.25]    [        51.25]    [       50.25]    [         50.25]    [      157800.00]
    '11-Jan-1995'       [        50.38]    [        50.63]    [       49.75]    [         49.75]    [      266000.00]
    '12-Jan-1995'       [        49.88]    [        50.13]    [       49.25]    [         49.88]    [       70000.00]
    '13-Jan-1995'       [        50.00]    [        50.25]    [       49.75]    [         50.13]    [      128000.00]
    '16-Jan-1995'       [        50.25]    [        50.38]    [       50.00]    [         50.00]    [      228200.00]
    '17-Jan-1995'       [        51.00]    [        51.00]    [       50.00]    [         50.63]    [      196200.00]
    '18-Jan-1995'       [        50.50]    [        51.00]    [       50.13]    [         50.13]    [      369700.00]
    '19-Jan-1995'       [        50.25]    [        50.50]    [       50.00]    [         50.50]    [      254300.00]
    '20-Jan-1995'       [        50.50]    [        51.25]    [       50.50]    [         50.50]    [      183400.00] ...

Transform the nonnormally distributed filled data series f_whrl into a normally distributed one using Box-Cox transformation:

bc_whrl = boxcox(f_whrl)
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/boxcox (line 36) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
bc_whrl = 
 
    desc:  Box-Cox: Filled Whirlpool	(WHR)				
    freq:  Unknown (0)

    'dates:  (1108)'    'Open:  (1108)'    'High:  (1108)'    'Low:  (1108)'    'Close:  (1108)'    'Volume:  (1108)'
    '03-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.15]
    '04-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.62]
    '05-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.68]
    '06-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.26]
    '09-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.23]
    '10-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.15]
    '11-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.44]
    '12-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           8.68]
    '13-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.03]
    '16-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.36]
    '17-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.27]
    '18-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.62]
    '19-Jan-1995'       [         0.56]    [         0.51]    [        0.60]    [          0.60]    [           9.42]...

Compare the result of the Close data series with a normal (Gaussian) probability distribution function and the nonnormally distributed f_whrl:

subplot(2, 1, 1);
hist(f_whrl.Close);
grid; title('Nonnormally Distributed Data');
subplot(2, 1, 2);
hist(bc_whrl.Close);
grid; title('Box-Cox Transformed Data');
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/subsref (line 106) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/hist (line 37) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/subsref (line 106) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/hist (line 37) 

Box-Cox Transformation

The bar chart on the top represents the probability distribution function of the filled data series, f_whrl, which is the original data series whrl with the missing values interpolated using the linear method. The distribution is skewed toward the left (not normally distributed). The bar chart on the bottom is less skewed to the left. If you plot a Gaussian probability distribution function (PDF) with similar mean and standard deviation, the distribution of the transformed data is very close to normal (Gaussian).

When you examine the contents of the resulting object bc_whrl, you find an identical object to the original object whrl but the contents are the transformed data series. If you have the Statistics and Machine Learning Toolbox™ software, you can generate a Gaussian PDF with mean and standard deviation equal to those of the transformed data series and plot it as an overlay to the second bar chart. In the next figure, you can see that it is an approximately normal distribution.

Overlay of Gaussian PDF

The next example uses the smoothts function to smooth a time series.

To begin, transform ibm9599.dat, a supplied data file, into a financial time series object:

ibm = ascii2fts('ibm9599.dat', 1, 3, 2)
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In ascii2fts (line 64) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
ibm = 
 
    desc:  International Business Machines Corporation (IBM)					
    freq:  Unknown (0)

    'dates:  (1108)'    'OPEN:  (1108)'    'HIGH:  (1108)'    'LOW:  (1108)'    'CLOSE:  (1108)'    'VOLUME:  (1108)'
    '03-Jan-1995'       [        36.75]    [        36.91]    [       36.66]    [         36.88]    [     1167900.00]
    '04-Jan-1995'       [        37.00]    [        37.28]    [       36.63]    [         37.16]    [     1994700.00]
    '05-Jan-1995'       [        37.13]    [        37.63]    [       36.88]    [         37.00]    [     2270900.00]
    '06-Jan-1995'       [        37.25]    [        37.88]    [       37.03]    [         37.53]    [     3040500.00]
    '09-Jan-1995'       [        37.50]    [        37.91]    [       37.41]    [         37.75]    [     1713000.00]
    '10-Jan-1995'       [        38.00]    [        38.63]    [       37.78]    [         38.28]    [     3420800.00]
    '11-Jan-1995'       [        38.53]    [        38.53]    [       37.63]    [         38.00]    [     2450300.00]
    '12-Jan-1995'       [        37.91]    [        38.16]    [       37.75]    [         38.00]    [     2275900.00]
    '13-Jan-1995'       [        38.25]    [        38.41]    [       37.88]    [         38.16]    [     2030300.00]
    '16-Jan-1995'       [        38.16]    [        39.00]    [       38.16]    [         38.75]    [     2716000.00]
    '17-Jan-1995'       [        38.63]    [        39.00]    [       38.53]    [         38.75]    [     1596300.00]
    '18-Jan-1995'       [        38.88]    [        38.91]    [       38.28]    [         38.63]    [     1842600.00]
    '19-Jan-1995'       [        38.53]    [        38.78]    [       38.16]    [         38.25]    [     1713900.00]...

Fill the missing data for holidays with data interpolated using the fillts function and the Spline fill method:

f_ibm = fillts(ibm, 'Spline')
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/fillts (line 213) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
f_ibm = 
 
    desc:  Filled International Business Machines Corporation (IBM)					
    freq:  Unknown (0)

    'dates:  (1108)'    'OPEN:  (1108)'    'HIGH:  (1108)'    'LOW:  (1108)'    'CLOSE:  (1108)'    'VOLUME:  (1108)'
    '03-Jan-1995'       [        36.75]    [        36.91]    [       36.66]    [         36.88]    [     1167900.00]
    '04-Jan-1995'       [        37.00]    [        37.28]    [       36.63]    [         37.16]    [     1994700.00]
    '05-Jan-1995'       [        37.13]    [        37.63]    [       36.88]    [         37.00]    [     2270900.00]
    '06-Jan-1995'       [        37.25]    [        37.88]    [       37.03]    [         37.53]    [     3040500.00]
    '09-Jan-1995'       [        37.50]    [        37.91]    [       37.41]    [         37.75]    [     1713000.00]
    '10-Jan-1995'       [        38.00]    [        38.63]    [       37.78]    [         38.28]    [     3420800.00]
    '11-Jan-1995'       [        38.53]    [        38.53]    [       37.63]    [         38.00]    [     2450300.00]
    '12-Jan-1995'       [        37.91]    [        38.16]    [       37.75]    [         38.00]    [     2275900.00]
    '13-Jan-1995'       [        38.25]    [        38.41]    [       37.88]    [         38.16]    [     2030300.00]
    '16-Jan-1995'       [        38.16]    [        39.00]    [       38.16]    [         38.75]    [     2716000.00]
    '17-Jan-1995'       [        38.63]    [        39.00]    [       38.53]    [         38.75]    [     1596300.00]
    '18-Jan-1995'       [        38.88]    [        38.91]    [       38.28]    [         38.63]    [     1842600.00]
    '19-Jan-1995'       [        38.53]    [        38.78]    [       38.16]    [         38.25]    [     1713900.00]
    '20-Jan-1995'       [        38.13]    [        38.25]    [       37.50]    [         37.66]    [     3333400.00]...

Smooth the filled data series using the default Box (rectangular window) method:

sm_ibm = smoothts(f_ibm)
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/smoothts (line 44) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/display (line 66) 
 
sm_ibm = 
 
    desc:  Box-smoothed of Filled International Business Machines Corporation (IBM)					
    freq:  Unknown (0)

    'dates:  (1108)'    'OPEN:  (1108)'    'HIGH:  (1108)'    'LOW:  (1108)'    'CLOSE:  (1108)'    'VOLUME:  (1108)'
    '03-Jan-1995'       [        22.18]    [        22.36]    [       22.03]    [         22.21]    [     1086700.00]
    '04-Jan-1995'       [        29.63]    [        29.94]    [       29.44]    [         29.71]    [     1694800.00]
    '05-Jan-1995'       [        37.13]    [        37.52]    [       36.92]    [         37.26]    [     2037400.00]
    '06-Jan-1995'       [        37.38]    [        37.86]    [       37.14]    [         37.54]    [     2487980.00]
    '09-Jan-1995'       [        37.68]    [        38.11]    [       37.34]    [         37.71]    [     2579100.00]
    '10-Jan-1995'       [        37.84]    [        38.22]    [       37.52]    [         37.91]    [     2580100.00]
    '11-Jan-1995'       [        38.04]    [        38.33]    [       37.69]    [         38.04]    [     2378060.00]
    '12-Jan-1995'       [        38.17]    [        38.54]    [       37.84]    [         38.24]    [     2578660.00]
    '13-Jan-1995'       [        38.29]    [        38.62]    [       37.99]    [         38.33]    [     2213760.00]
    '16-Jan-1995'       [        38.36]    [        38.69]    [       38.12]    [         38.46]    [     2092220.00]
    '17-Jan-1995'       [        38.49]    [        38.82]    [       38.20]    [         38.51]    [     1979820.00]...

Now, plot the original and smoothed closing price series for IBM® stock:

plot(f_ibm.CLOSE('11/01/97::02/28/98'), 'r')
datetick('x', 'mmmyy')
hold on
plot(sm_ibm.CLOSE('11/01/97::02/28/98'), 'b')
hold off
datetick('x', 'mmmyy')
legend('Filled', 'Smoothed')
title('Filled IBM Close Price vs. Smoothed Series')
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/subsref (line 106) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/plot (line 63) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/subsref (line 106) 
Warning: FINTS will be removed in a future release. Use TIMETABLE instead. 
> In fints/plot (line 63) 

Smoothed Data Series

These examples give you an idea of what you can do with a financial time series object. This toolbox provides some MATLAB® functions that have been overloaded to work directly with these objects. The overloaded functions are those most commonly needed to work with time series data.

See Also

| | | | | | | | | | | | | | | | | | | | |

Related Topics