1.2.1 Categorical Predictors

Let us begin by investigating whether categorical variables reveal any relationship to our main variable of interest - miles per gallon (MPG). Figure 1.1 shows their distribution and corresponding counts. We can see little to no difference between two, and four Doors cars with respect to MPG. Similar can be said for Body, even though there is a slight difference between the types. Drive, Cylinders and Fuel System (FuelSys) seem to have some explanatory power, as the differences between boxes are more apparent. However, we must bear in mind that the majority of occurences are captured by four of less (<=four) cylinders and multi port fuel injection (mpfi) system for Cylinders and FuelSys respectively. Let us consider the plot of the CompRatio (Figure 1.2). It turns out that petrol-powered vehicles’ compression ratio varies between 6 to 10, whereas for diesel engines, it lies between 14 and 22. This explaines why two clusters occur (with one extreme values of 11.5 for petrol). Furthermore, cars with one/two barrel carburettor system (1bbl and 2bbl) tend to have more MPG than cars with multi port fuel injection system (mpfi), which can also be deduced by investigating the last plot of Figure 1 FuelSys boxplot. Moreover, the black line of best fit looks fairly flat when taking into account all points. However, if we split the data into two engine types groups, the relationships are much stronger. In addition to CompRatio, PeakRPM and Stroke are also related to fuel type, with diesel cars generally having less PeakRPM and longer Stroke. It is worth noting that after removing extreme petrol value (11.5), its corresponding \(R^2\) would increase from 0.18 to 0.25, i.e. about 25% of MPG variability could be explained by the CompRatio petrol engine type alone. In the next section we take a closer look at numerical variables and how they interact with MPG.

1.2.2 Correlation

To help us identify the variables’ impact on MPG, we plot the correlation matrix (Figure 1.3). We can immediately notice a triangle with moderate to high correlations between variables. Even at this stage of the analysis we can deduce that the data is highly multicollinear, and we will need to address this problem before modelling. In the next plots, we try to demonstrate why such strong relationships occur. Surprisingly, all numerical predictors but CompRatio are negatively correlated with MPG. By referring back to the brief document, we can understand why, for example, WhlBase is highly correlated with Len - they are very similar measurements.

1.2.3 Horsepower

The correlation bewteen Horsepower and MPG is considerable. Figure 1.3 also shows a correlation of up to 0.8 between Horsepower, EngSize, and CurbWt, We explore how they influence each other in a functional sense. Horsepower is a unit of power, it is highly correlated with EngSiz since larger EngSiz implies more cylinders, and the more cylinders there are, the more petrol is used and more horsepower is generated. Also, the more cylinders there are, of course, the heavier the vehicle will be. Figure 1.6 shows the relation between these variables coloured by the number of cylinders. Cars with more horsepower usually consume more fuel which we can clearly see on the plot. Moreover more cylinders result in more horsepower - the majority of cars have four cylinders (only one car has 3). Moreover, cars with more than four cylinders tend to have a rear drive system, which can be seen on Figure 1.4. We also note an inverse relationship between Horepower and MPG.

1.2.4 Cars Size

A significant number of highly correlated variables constitute cars size or their engine measurements. The bigger the engine, the bigger we expect a car to be. Indeed, Figure 1.5 confirms this presumption. Furthermore, the heavier the engine, the larger it must be, which can be seen on Figure 1.6. Also, the number of cylinders influences the curb weight, and therefore engine size. Even though four cylinders cars dominate, there is an observable pattern for cars with more cylinders - not surprisingly, they tend to increase engine size and weight. When we recall that WhlBase and Len are similar measurements, and Bore measures a diameter of each cylinder, we begin to understand the relation between these variables.

Using only Len, Wid, WhlBase and CurbWt to predict MPG reveals that only CurbWt and WhlBase are influential. This suggests that Len and Wid affect MPG not because of wind resistance or otherwise, but mainly because we need more volume to accommodate more cylinders (more weight) and the larger frame also causes more weight. Our intuition would agree that when the vehicle is heavier, it does more work to travel the same distance and therefore uses more fuel, so we expect Len and Wid to not be important in our model while CurbWt will retain predictive power. Since the correlation between WhlBase and MPG is only 0.5, and it is largely due to correlation with CurbWt, we do not expect to keep this variable either. We will see whether these predictions agree with our model choice during variables selection in the modelling section of this report.

1.2.5 Price

The last variable that seem to highly impact MPG is Price. Intuition tell us that it should not be a direct factor when it comes to predicting MPG. However, the high correlation suggests otherwise. Obviously, many other predictors also highly influence Price, for example cars size, engines, or horsepower. The latter being particularly good at predicting Price. Figure 1.4 confirms this belief as we can clearly see that the higher the car costs, the more horsepower it tends to have. However, there is one interesting outlier

Table 1.1: Outlier

MPG	Doors	Body	Drive	WhlBase	Len	Wid	Ht	CurbWt	EngSiz	FuelSys	Bore	Stroke	CompRatio	Horsepower	PeakRPM	Price	Cylinders
22.5	two	hatchback	Rear	98.4	175.7	72.3	50.5	3366	203	mpfi	3.94	3.11	10	288	5750	10295	>=eight

which is significantly cheaper than cars with similar characteristics (a distinct point the furthest to the right on Figure 1.4) (we will see later in our report that outliers in terms of their effect on how the Price variable is treated in the model will be an issue to consider). In our analysis we discovered an intriguing fact. The dataset contains identical observations up to their price - one example of such a pair below.

Table 1.2: Identical Values

MPG	Doors	Body	Drive	WhlBase	Len	Wid	Ht	CurbWt	EngSiz	FuelSys	Bore	Stroke	CompRatio	Horsepower	PeakRPM	Price	Cylinders
21	two	other	Rear	89.5	168.9	65	51.6	2756	194	mpfi	3.74	2.9	9.5	207	5900	32528	six
21	two	other	Rear	89.5	168.9	65	51.6	2756	194	mpfi	3.74	2.9	9.5	207	5900	34028	six

It is hard to tell why such duplicates occur, one may speculate that they differed by some other characteristics our data did not record. In total, there were 9 such pairs.

2.2.1 Model Diagnositcs

We now fit a model to this transformed data and present some of the resulting model output. The values of parameters and their significance has been ommited for brevity, but we will pay particular attention to the diagnostic plots to verify that our modelling assumptions hold, and that our transformation does indeed improve the quality of our model.

Our new model’s \(R^2\) increases to 0.94 with adjusted \(R^2\) to 0.93, whilst having far less significant heteroscedasticity, residuals more evenly spread around zero, and a better Q-Q plot. This gives us the best of both worlds with a better fitting model which also obeys the modelling assumptions more. A drawback to using this transformation is that our model is no longer predicting the original MPG, but instead the inverse of this: GPM. However, we believe that GPM is still an intuitive idea that can be understood and we can still explain the trend in MPG since an increase in GPM indicates a decrease in fuel efficiency, i.e. MPG.

2.2.2 Outliers and Influential Observations

We do however note some concerning potential outliers. Looking at the Residuals vs Fitted plot on Figure 9, we see that observation \(54\) has an usually large residual, as well as \(125\) and \(156\) to a lesser degree. Further, \(54\) and \(125\) appear as outliers on the Q-Q plot. Lastly, \(48\), \(54\) and \(125\) have a large Cook’s distance which suggests that they may be outliers, influential observations that distort our model, or both. We will keep these observations in mind as we investigate the model further.

We will now consider a “Dffit” plot which shows by how much the predicted value of an observation changes when we fit a new model with that observation removed. Observations which have a large value are observations that are likely influential and distort the model as the large value shows that the model prediction changes significantly when they are not accounted for.

In this plot we notice that observations \(54\) and \(125\) appear again. This adds to our concerns that these observations may be affecting the model.

We will also consider “added variable” plots which show how much we can predict the \(GPM\) with a specific variable after accounting for all other variables. These allow us to identify unusual observations and whether an explanatory variable provides useful information in our model. One such variable we will look into is Price. Whilst there is a strong correlation between MPG and Price, it is also reasonable to suspect that Price itself is a function of all the other data available about a car, and so, once other variables have been included, Price itself no longer provides any useful data, so the added variable plot should show no overall trend. Further, if Price were to have an effect, our model would give the conclusion that two otherwise identical cars could have widely different fuel efficiencies if one is sold at a different price to another which is clearly nonsense. However, inspecting the added variable plot for Price we do see a clear trend.

We also notice that observations \(54\) and \(125\) again stand out in this diagram. In particular each point serves to draw the line of best fit towards them and without these points, it appears as if there would be a reduced trend between the \(x\) and \(y\) values. If we remove these observations and refit the model we see that there is a reduced effect of Price on GPM. These two observations were already highly unusual due to their large residual values, position on the Q-Q plot and their large Dffit values. Also, since their inclusion leads us to the strange conclusion that merely changing the price of a car arbitrarily could change its fuel efficiency, we may wish to remove these from the analysis.

Ideally, we would carry out two analyses in parallel, one including these two data points and one excluding, so that we may compare the results. However, in the interests of brevity, we shall only focus on the latter of these two as we find the model that does not suggest that Price has an effect of fuel efficiency more appealing.

2.2.3 Variable Selection

Now that we have decided to remove the most prominent outliers, we wish to see whether a simpler model adequately explains the data. There are certain explanatory variables which, based on prior knowledge about cars, we would expect to not affect the fuel efficiency of the car, and so we will check if a model excluding these variables is no worse than the full model. We have the following variables, with reasons, which we expect to not be important in our model:

• Price - Keeping all other variables constant, the price a car is sold at should not change the fuel efficiency.
• Height, Width, Length, Wheel Base, Body - Dimensions of the car, as its shape (Body), should effect fuel efficiency only though the increased weight for larger cars, which is already accounted for, and drag, which we expect to be small.
• Doors - Keeping all of other variables constant, adding or removing doors should not change fuel efficiency by any appreciable amount.
• Engine Size, Bore - The size of the cylinders (Bore) serve as a factor in determining Engine Size. For engine size, we only expect the size of the engine to effect fuel efficiency through increased weight, which is already accounted for as a variable, and the fact that larger engines tend to be more powerful, which is accounted for by horsepower.

To test whether these variables should be included in our model or not, we carry out a nested ANOVA test which aims to conclude whether there is sufficient statistical evidence to suggest that at least one of the extra terms in the larger model does indeed have an effect.

Table 2.3: ANOVA test

Res.Df	RSS	Df	Sum of Sq	F	Pr(>F)
185	0.0007838	NA	NA	NA	NA
174	0.0007174	11	6.64e-05	1.463305	0.1492022

In the above table, we are primarily interested in the bottom right-most entry which provides the p-value for the hypothesis test. We see that the p-value of \(0.149\) is larger than \(0.05\), hence there is insufficient evidence to suggest that any one of these variables should be included in the model. As such, we remove these variables from our model simplifying it in the process.

For variables which we cannot discount on the same grounds as above (i.e. those that we intuitively think should have no effect), we instead carry out individual nested ANOVA tests to see if certain categories of variables are collectively significant or not, these categories being Drive, Fuel System and Cylidners. We omit the ANOVA results for brevity, but in all cases, the remaining categorical variables were all had a p-value less than \(0.05\) and thus are collectively significant, i.e. we will retain them in the model.