Abstract

While there is a burgeoning research literature on crime trends, much of the extant research has adopted a relatively narrow approach, efforts across studies are highly variable. In this paper, we outline a method to establish the relation between crime rate in the two years of 1959 and 1960 to the political, social and economic factors of that time, whether the crime does depend on the poverty state and mindset of the people in that era. The correlation between crime rate and all the factors being studied is visualized using a scatterplot matrix (SPLOM). Multiple linear regression models under ANOVA framework are performed to evaluate which factors and their interactions affect the crime rates significantly. Hence, what should be done in order to reduce the crime rate and help in the development of the country in all aspects. Statistically, the factors which influence crime rate the most are police expenditure in both the years, Gross Domestic Product (GDP), State population and probability of imprisonment.

I. Introduction

A. Background

Crime is an illegal act which is subjectable to punishment by the government. It involves violating the standard law code prescribed by a country’s parliament and judiciary. It is an unlawful act and a grave offense against human morality. The topic covered in this paper is the data analysis of crimes in the United States. It is a systematic way of detecting and investigating patterns and trends in crime.

The crime in the U.S. has been recorded since the early 1600s. The crime rates have varied over time, with a sharp rise after 1900 and reaching a broad bulging peak between the 1970s and early 1990s. The range of these criminal activities vary from pickpocketing to serial killing and assassinations. The basic aspect of a crime considers the offender, the victim, type of crime, severity and level, and location. These are the basic questions asked by law enforcement when investigating any situation. This information is formatted into a government record by a police arrest report, also known as an incident report.

Society has a strong misconception about crime rates due to media aspects heightening their fear factor. The manner, in which America’s crime rate compared to other countries of similar wealth and development depends on the nature of the crime used in the comparison. Overall crime statistic comparisons are difficult to conduct, as the definition and categorization of crimes varies across countries. Thus, an agency in a foreign country may include crimes in its annual reports which the United States omits, and vice versa. However, some countries such as Canada have similar definitions of what constitutes a violent crime as America. Overall, the total crime rate is more in the US than it is in other developed countries across Europe.

B. Problem Statement

The criminal behaviour has traditionally been linked to the offender’s presumed unique motivation which might be various factors such as unemployment, family circumstances due to poverty. It also depends on the dark figure of crime which is the gap between reported and unreported crimes calls the reliability of an official crime statistics into question, but all measures of crime have a dark figure to some degree. The gap in official statistics is largest for less serious crimes. It also varies from region to region or as a matter of fact, state to state. It depends on the type of community living in that region as well. The crime in metropolitan statistical areas tends to be above the national average; however, wide variance exists among and within metropolitan areas.

In this report, we will deep dive into various factors contributing/influencing the rate of crime, which serves as our dependent variable and how it depends on 15 independent variables, which are: % of males aged 14–24, indicator variables for a Southern state, mean years of schooling, police expenditure in 1960 and 1959, labour force participation rate, number of males per 1000 females, state population, number of non- whites per 1000 people, unemployment rate of urban males age 14–24 & 35–39, gross domestic product per head, income inequality, probability of imprisonment, and average time served in state prisons, that affect the rate of crime in a specific category per head population.

II. Methodology

A. Descriptive

In this section, we are going to describe, explore and confirm how the economic, social and political factors are affecting the crime rate using the MASS Library in R.

Each row represents a state in the USA and Unit are shown in Table 1.

The rate of crime serves as the dependent variable which depends on a number of independent variables such as police expenditure, mean year of schooling, unemployment rate, income inequality, imprisonment probability and so on.

A SPLOM plot is used to show the basic information of the dataset. It is a collection of scatter plots being organized into matrix. It gives us the direction, strength, and linearity of the relationship between the dependent and independent variables and also helps in determining the correlation between the independent variable and dependent variable as seen in the Figure 1.

B. Exploratory:

The data is first explored and wrangled to check for the presence of outliers and those which are present are removed.

The potential correlation between the variables is further evaluated via analysis by 14 simple regression models, to examine which independent variables are significant factors. Based on this set of simple regression models, 4 variables are chosen which have a statistically significant correlations with the dependent variable Crime Rate.

Regression is used to predict a continuous outcome based on one or more continuous predictor variables. Regression line is the straight line passing through the data that minimizes the sum of the squared differences between the original data and the fitted points.

An aov (analysis of variance) model is performed to check the interrelation between the 4 independent variables from the simple regression model. This analysis is performed iteratively on the various combinations in which the least significant combination (the one with highest p value) is removed after each step. Multiple regression analysis is performed on these 4 independent variables with respect to the dependant variable that is crime rate.

C. Confirmatory:

ANOVA model is used for deciding the best multiple regression model among the given combination of the 4 independent variables. ANOVA and step aic are used to find out the interdependency between the 4 variables. Likelihood ratio test is performed to check goodness of the fit of the nested regression models.

ANOVA is fundamental for all statistical approaches. ANOVA, which stands for Analysis of Variance, is a statistical test used to analyses the difference between the means of more than two groups.

ANOVA is used in the analysis of comparative experiments, those in which only the difference in outcomes is of interest. The statistical significance of the experiment is determined by a ratio of two variances. This ratio is independent of several possible alterations to the experimental observations: Adding a constant to all observations does not alter significance. Multiplying all observations by a constant does not alter significance.

A one-way ANOVA uses one independent variable, while a two-way ANOVA uses two independent variables.

In ANOVA, the null hypothesis is that there is no difference among group means. If any group differs significantly from the overall group mean, then the ANOVA will report a statistically significant result.

So, ANOVA statistical significance result is independent of constant bias and scaling errors as well as the units used in expressing observations. This makes it the ideal model.

We use ANOVA, when we want to test a hypothesis. From our 3 models of different combinations, we choose the model (independent variables) with maximum influence on the dependent variable that is crime rate as an ideal model. Finally, the model with all 4 variables is plotted. Following this, the best model is chosen from all the 3 models.

The likelihood-ratio test compares the goodness of fit of two nested regression models based on the ratio of their likelihoods, specifically one obtained by maximization over the entire parameter space and another obtained after imposing some constraint. A nested model is simply a subset of the predictor variables in the overall regression model.

III. Results

A. Descriptive

In this section, we will be depicting the various results of our analysis of regression and ANOVA Models.

From Figure 2, we can see that police expenditure in 1959 and 1960 have high correlation with one another. This condition is not ideal for linear regression because no predictor variable should strongly correlate with one another, hence we are combining both into a single variable.

SPLOM PLOT

As we can see for police expenditure, state population and gdp vs Crime Rate from Figure 3, the data points are closer to the line moving in the upward direction indicating a strong positive linear relationship with crime rate. Similarly, for probability of imprisonment, it is in the downward direction indicating a strong negative relationship with Crime Rate.

From the plot, we can see that four independent variables with most correlation with the Crime Rate is Police expenditure, GDP, Probability of imprisonment and State Population.

B. Exploratory:

Data Wrangling

In Figure 4, the data is plotted with and without outlier.

The predictor variable during a simple linear regression must always have high linear relationships with target variables. So, for this data we need to first check for the presence of an outlier and if there are any, we must remove them.

If we look at the abline of the plot, we can see that the outliers of our data have a high influence over our model. Therefore, it is safer to remove the outliers from the data. The removal of outliers increases the R-squared of the model ~0.6 points, and the p value has decreased along with the value of intercept which shows the removal of outlier improves the model.

A simple linear regression model is estimated for each of these 14 independent variables and their result has been tabulated below.

From these simple regression models, we can conclude that the relation between police expenditure, GDP, probability of imprisonment, state population and crime rate is highest as their p value is less than 0.05 with r being 72% ,44% ,43% and 36% respectively. So, with these 4 variables, a multiple regression and stepAIC is plotted. As next step, ANOVA is used to find the best model.

C. Confirmatory:

Interrelation between variables using multiple regression and stepAIC

AOV is performed to check which variables among the 4 variables have the most interrelation amongst one another and hence affect the crime rate.

Multiple regression and stepAIC are performed for the 4 interdependent variables as shown in Figure 7

The highest p value (gdp:state_pop and gdp) are iteratively removed and performed multiple regression and stepAIC as shown in Figure 8 and 9

ANOVA is performed to check the best model.

ANOVA
Likelihood ratio test is performed on the 3 nested models, since the p value is greater than 0.05 as seen in the figure, we have to accept the null hypothesis which means there is not much difference between the models and the best fit is the first model (pol_exp * gdp * prob_imp * state_pop)

D. Model Validation

So, from the Shapiro test we can see that p value is more than 0.05 and hence confirms the data is normally distributed.

IV. Discussion

From the simple linear regression, multiple regression and ANOVA model, we can conclude that 4 factors which crime rate depends the most are police expenditure, GDP, State Population and probability of imprisonment.

Out of these 4 it depends on police expenditure the most followed by probability of imprisonment, which has a negative relation and then GDP followed by state population. From, the interrelation model, we can see how closely these 4 are interrelated and affect crime rate. As we can see the economic factors especially the police expenditure and the GDP which affected crime rate. Population factors also affected the crime rate as highly populated states experienced more crimes than the other states.

This was closely followed by putting fear into people’s mind by the probability of imprisonment playing a key role. It is an extremely surprising fact that crime rate is more influenced by the economic factors than unemployment.

Change in population in turn influences the gdp. The gdp of a state decides its expenditure on police. Despite the significant findings, there were several limitations to note in the study. The first limitation was the medium sample size.

V. Conclusion

From the simple linear regression, multiple regression and ANOVA model, we can conclude that 4 factors which crime rate depends the most are police expenditure, GDP, state population, and probability of imprisonment. Out of these 4 it depends on police expenditure the most followed by probability of imprisonment, which has a negative relation and then GDP followed by state population. From, the interrelation model, we can see how closely these 4 are interrelated and affect crime rate. As we can see the economic factors especially the police expenditure and the GDP which affected crime rate. Population factors also affected the crime rate as highly populated states experienced more crimes than the other states. This was closely followed by putting fear into people’s mind by the probability of imprisonment playing a key role. It is an extremely surprising fact that crime rate is more influenced by the economic factors than unemployment.

Change in population in turn influences the gdp. The gdp of a state decides its expenditure on police.

The state must allocate funds towards police training, enforce strict law & order and improves GDP by increasing Government spending, Export and Investment. This helps to bring down the crime rate.

VI. Reference list

1. Venables, W. N. & Ripley, B. D., Modern Applied Statistics with S, Fourth edition (2003)
2. Isaac Ehrlich, Participation in Illegitimate Activities: A Theoretical and Empirical Investigation (1973)
3. Geoffrey R Norman and David L Streiner, BIOSTATISTICS: The Bare Essentials, Third Edition (2008)
4. Adarsh S, GitHub Repository, https://github.com/AdarshS20/UScrimeDataAnalysis (2022)\

Acknowledgements
I would like to take this opportunity to express my profound gratitude and deep regards to my mentor Dr. Hong Pan for his exemplary guidance, monitoring and constant encouragement for this research paper. I would also like to take this opportunity to express my gratitude to Ms. Danielle Voorhies for her cordial support and guidance. Lastly, I thank my parents and friends for their encouragement and support to complete this research paper.

About the author

Adarsh Sasikumar

Adarsh is a 12th Grader at Sri Kumaran Children’s Home of Educational Council, Bangalore, India. He discovered the subject of Mathematics at the age of 6 and it was love at first sight. As he grew up, he felt like he had encapsulated himself into the number, angles, variables, and equations. He is planning to pursue an undergraduate major in Mathematics and is interested in predictive analysis.

The post United States Crime Data Analysis Using Modern Applied Statistics Methodologies appeared first on Exploratio Journal.

1.  Introduction

With the booming of global economies and the develop of societies, the world is changing every single minute. Thus, many considerable issues come to the center of the stage. An important issue is the allocation of public resources to education and progress in the improvement of education across countries of the world. Education, as the base of the society, is a huge topic related to resources allocation and social progress. According to the importance of being educated, not only civilians, but also governments strive for high primary completion rate. To attain the goal, governments in world-wide regions and civilians in different classes pay great prices and obtain interesting results. However, some people may question if the money they put can really promote the education. According to World Data Bank from 1991 to 2016, the progress of the development on education answers the question. 

2.  Governments’ Expenditure and Primary Completion Rate

2.1 East Asia and Pacific

The track of developing education in each region is different. Because of the diverse basics, experiments, and governments, regions around the world show multiple data according to the governments’ expenditure and primary completion rate. In the region of East Asia and Pacific, the governments’ expenditure gains the development of education. From 1991 to 1999, the government is keeping putting effort on education, but the primary completion rate of different countries is not all comparable. The most typical example is in 1991: some countries’ expenditure is around 7.6%, and their primary completion rate is around 98%, which is greatly high; some countries expenditure is around 16.3%, but their primary completion rate is only around 71%, which is unpredictable low. 

From 2000 to 2006, the government expenditure shows a strong linear association with the primary completion rate. The general completion rate is close to 100%. Take 2006 as an example: some countries’ expenditure is around 9%, and their completion rate is around 90%; some countries expenditure is around 22%, and their completion rate is around 98%. The data presented is the lowest one and the highest one in 2006. Both of them show the appreciation of governments to education, and the high primary completion rate. 

However, start form 2007, and end in 2009, the situation is not fluent as before. Probably because of the economic crisis in 2007, the primary completion rate begins to decline though governments expend large amount of money on it: 

In 2007, some countries’ expenditure is around 17%, which is considerable under the crisis, but the primary completion rate is still around 87%, which is lower than the lowest completion rate in 2006. 

After the crisis, which is from 2010 to 2016, the situation is saved. The relationship between governments’ expenditure and the primary completion rate is direct. The more governments spend, the higher completion rate grows. For instance, in 2014, the lowest governments’ expenditure is around 7%, and the primary completion rate is around 92%; the highest governments’ expenditure is around 20%, and the primary completion rate is around 100%. Although the governments’ expenditure from 1991 to 1999 does not win the growth of the primary completion rate, from 2000 to 2006, the rate raises according to the expenditure. During the three year of crisis, to the declined rate, the expenditure is rewarding. Before 2000, governments put lots of money to support education, and the expenditure generally increases with the primary completion rate after 2000. 

2.2 Europe and Central Asia

The condition in Europe and Central Asia is a different picture. Because of the difference cultural background and growing process, the data from 1991 to 1994 shows that the completion rate is unstable. The primary completion rate is generally low. Moreover, although the governments’ expenditure is high, it does not help the primary completion rate. Take 1991 as an example: the governments’ expenditure is around 17.5%, but the primary completion rate is only 70%, which is similar to that of East Asia. The same-level primary completion rate between Europe and Central Asia and East Asia illustrates the parallel background they share from 1991 to 1994. 

The data from 1995 to 2006 becomes stable to develop. The governments pay attention on education, and the primary completion rate is high. In 2004, for instance, the governments’ expenditures are between 12% to 23%, and the primary completion rates are between 88% to 100%. The high primary completion rate responds to the expenditure. 

Different with East Asia, Europe and Central Asia is not impacted by the crisis from 2007 to 2009. Oppositely, the data is quite stable with the almost 100% primary completion rate. From 2007 to 2009, the primary completion rate slightly increases. In 2007, the general rate is around 97%; in 2008, the general rate is around 98%; the general rate is around 99%. The reason behind it maybe the financial relationship between the US and Europe and Central Asia is not as tight as East Asia. 

From 2010 to 2016, the governments’ expenditure has direct relationship with the primary completion rate. Take 2015 as an example, the lowest governments’ expenditure is around 8% with the primary completion rate around 97%. The highest governments’ expenditure is around 20% with the primary completion rate around 100%. The development of the expenditure and the education in Europe and Central Asia is generally fluent and stable—notably it is not harmed by the crisis from 2007 to 2009. The primary completion rate respond immediately once governments start to focus on education.

2.3 Latin America and Caribbean

The region of Latin America and Caribbean shows considerable development in education. From 1991 to 1998, the education level between each country is dramatic. The governments’ expenditure on education is balance, but the primary completion rate of each country is not.  In 1992, for example, the association between governments’ expenditure and the primary completion rate is positive, and straight with a high outlier, and strong. The lowest governments’ expenditure is around 12%, and the primary completion rate is only 40%; the highest governments’ expenditure is around 15%, and the primary completion rate is around 100%. The close expenditure of each government causes variant rate. Furthermore, according to the data, there are only few countries on it, which is probably because of the imperfection of the education does not allow formal data collecting, or just the data is not collected at the period. 

The situation becomes optimistic from 1999 to 2006. Though the plot is moderately strong because of a few outliers, the general rate grows. Take 2003 as an example, the association between governments’ expenditure and the primary completion rate is positive, and straight with a few high and low outliers, and moderately strong. The lowest governments’ expenditure is around 7%, and the primary completion rate is around 92%; the highest governments’ expenditure is around 98%. ‘

From 2007 to 2008, slightly reduce in the primary completion rate is shown. It seems like the economic crisis effects the education in a negative way. Especially in 2008, the association between two variables is negative and straight with a high and a low outlier, and moderately strong. The plot shows that some countries are harmed by the crisis, but others remain stable development. 

After 2008, the primary completion rate keeps growing. Although the expenditure is not efficient enough, the general rate is close to 100%. Take 2010 as an example, the association between the governments’ expenditure and the primary completion rate is negative and straight with a few outliers, and moderately strong. The governments’ expenditure is between 13% to 23%, and the primary completion rate is between 90% to 100%. The efficiency of the development on education is not high enough that the governments’ expenditure does not cause absolute high primary completion rate, but the growth on rate is substantial in the region of Latin America and Caribbean. 

2.4 Middle East and North Africa

Not only the region of Latin America and Caribbean gets progress on education, but also that of Middle East and North Africa. From 1991 to 1995, the data shows both positive and negative association for each year, which means that the region has typically uncertain primary completion rate. Also, the differences between each country is huge on the governments’ expenditure. Take 1994 as an example, the association between the governments’ expenditure and the primary completion rate is positive and straight with no outlier—because there are only two countries in the plot—and strong. One country has the government’s expenditure that is around 7%, and the primary completion rate that is around 71%; another country has the government’s expenditure that is around 22%, and the primary completion rate that is around 80%. It seems like that one country is trying to develop their education by putting great effort by government, so their expenditure is as high as 22%. However, the general rate is still low. 

From 1996 to 1997, it is the most stable period before 2002 that the governments’ expenditure and the primary completion rate is considerable. The association between the expenditure and the rate is positive, and straight with a few outliers, moderately strong. The governments’ expenditure is between 8% to 27%, and the primary completion rate is between 87% to 91%. 

From 1998 to 2002, the situation becomes undesirable again. For instance, in 2001, the association between the governments’ expenditure and the primary completion rate is negative, and straight with a low outlier, and moderately strong. The lowest governments’ expenditure is around 10%, and the primary completion rate is around 94%; the highest governments’ expenditure is around 30%, but the primary completion rate is around 60%. The outlier in 2001 is incredible that it has the expenditure of 27% with the rate of 26%. 

After 2002, the associations between two variables are generally positive and strong, except for those of 2007 and 2008. In 2007 and 2008, the association is negative with few outliers, and moderately strong. It shows that Middle East and North Africa is also one member of victims from the crisis. Nevertheless, the general growth on education is still great accompanying with the help of governments. Take 2013 as an example, the governments’ expenditure is between 7% to 17%, and the primary completion rate is between 95% to 100%. The decline of the governments’ expenditure indicates the steady of primary completion rate. As another region with huge development on education, it is fair to say that governments’ expenditure cannot be ignored. 

2.5 Conclusion 

The four regions show the development on education and governments’ expenditure from 1991 to 2016. Every region has its own track: some have unstable education in the beginning, and become stable recently by the support of governments; some have a good beginning with the help of governments, but suffer from the crisis, and overcome it then; some have a government that with sense of paying attention on education all the time that ensure their education is always progressive. Nonetheless, starting from 2009, the development is substantial: 

Overall, the governments’ expenditure helps the growth of primary completion rate. Before 2000, the data usually shows inverse relationship between them, which indicates that governments try to increase the rate because the primary completion rate of each region is all close to 100 in the end. Thus, as the time past, governments’ efforts on education promote the primary completion rate.

3. People’s Expenditure and Primary Completion Rate

3.1 Introduction

Governments’ expenditure on education significantly increases the primary completion rate, though it sometimes takes years to see the development. Similarly, different income groups, high income, upper middle income, low middle income, and low income, have different degrees of expenditure. Is their expenditure on education also helpful?

3.2 High Income Group

High income group, as the group of people who are most able to complete high-level education, always remains high primary completion rate. From 1991 to 1999, the general primary completion rate is high, but the associations are not strong enough between the expenditure ad the rate. For instance, in 1997, the association is positive, straight with many outliers, and moderately strong. The plot on this graph is scattered, instead of tight. It indicates the differences between different families. Some high-income families at this time do not have realize the importance of education, so they do not spend great amount of money on it. However, as the arrival of 21th century, the difference is fading. 

From 2000 to 2016, the expenditure and the primary completion rate is impressively high, with straight relationship. Take 2013 as an example, the association is positive, straight, and strong. The lowest expenditure in this year is around 6%, and the primary completion rate is around 99%; the highest expenditure in this year is around 22%, and the primary completion rate is around 100%. The expenditures of different families are still discrepant. The reason behind it might be the wish of new high-income families start to put effort to education under the background of the enlargement of high income group. High income group successfully reduce the differences between each family on the primary completion rate; the low expenditure illustrates the stableness on education of some high-income families, and the high expenditure illustrates the development on that of new high-income families.

3.2 Upper Middle Income Group

To middle income group, most of them are trying to become one of high income group members. They pay great attention on education, trying to create chances for their children to attend higher class. Predictably, the expenditure and the primary completion rate is quite high in upper middle income group (because of the lack of information, the analyze only covers from 1998 to 2016). The data shows the stable and high expenditure and the rate of completion from 1998 to 2016. It is similar to that of high income group that they share high primary completion rate that is close to 100%. However, compared to high income group, upper middle income group does not have the association of two variables of upper strong enough. For example, in 2006, the association is negative, straight with some outliers, and moderately strong. The high income group in 2006, on the other head, has the association that is positive, straight, and strong. Upper middle income group already have a general high-level sense of education, but there are still some families do not realize that. The immature sense causes the differences between upper middle income group and high income group. 

3.3 Lower Middle Income Group

Lower middle income group presents interesting data, which indicates the complicated and diverse background of different lower middle income group. From 1991 to 1995, the primary completion rate is low, though the expenditure is moderate. Take 1994 as an example, the association between the expenditure and the primary completion rate is negative, straight with few outliers, and moderately strong. The expenditure is between 6% to 23%, and the primary completion rate is between 89% to 60%, which is close to 90%, but is still not a high rate. 

Start from 1996, the rate increases that even close to 100%. However, the association between two variables is weak, or there is no association. Some families in lower middle income pay attention on education, but some do not. Take 2015 as an example: 

According to the plot, it is hard to analyze the association between the expenditure and the primary completion rate. Every family put different effort on supporting education, and the rate is also unrelated with each other. Although the general primary completion rate is not low, it is still hard to tell if the situation is positive or negative because of the gap between each family. 

3.4 Low Income Group

Low income group, as a group with weakest financial condition, reflects some essential social problems. Not surprisingly, the group have low primary completion rate. However, the expenditure is in the average. It is probably because of their low income, so affording common education is not easy actually. Until 2016, their general completion rate is still around 70%. There are many countries have already made the primary education free, so why the rate is remains low, and the expenditure remains in average? It is reasonable for other income group that they want to give their children better education, like going to private schools for education, and so on. However, why the situation comes to the low income group people? The question probably can be given to the government. 

3.5 Conclusion

In a society that some groups can have the primary completion rate nearly 100%, and some groups can only have the primary completion rate around 70%, there must be some problems. Are the resources allocated well? Does the education system need to be improved? Yet we are still in the progress of development, but as we can see, though we need to spend huge amount of money for years, the association between governments’ and people’s expenditure and primary completion rate is ultimately straight upward. Thus, because of the optimistic result of the expenditure, the society is encouraged to put more effort on education. It is unrealistic to keep the primary completion rate 100% to every income group, but trying to balance each group is what we should focus on. 

Works Cited

Mentor: Dr. Peter Kempthorne
Massachusetts Institute of Technology

“GDP (Current US$) – High Income.” Data, data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=XD.

“GDP (Current US$) – Low Income.” Data,  data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=XM.

“GDP (Current US$) – Lower Middle Income.” Data,  data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=XN.

“GDP (Current US$) – Upper Middle Income.” Data,  data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=XT.

“Government Expenditure on Education, Total (% of Government Expenditure).” Data,  data.worldbank.org/indicator/SE.XPD.TOTL.GB.ZS.

“Primary Completion Rate, Total (% of Relevant Age Group).” Data,  data.worldbank.org/indicator/SE.PRM.CMPT.ZS.

About the author

Dawn Diao

Dawn is a rising senior at the Mercyhurst Preparatory School in Erie, Pennsylvania.

The post Expenditure and Education appeared first on Exploratio Journal.

Abstract

In investment, timing of buying and selling is crucial to profits. In the real estate community, there is often debate amongst investors on the perfect timing to sell real estate. While many contend that the perfect timing is during the summer or spring, there has been little statistical support of these theories. The purpose of this paper is to locate the optimal time for real estate sales in California based on length of time on the market. In this paper, multiple graph types are used in order to determine a concrete pattern that can be used as a basis for future real estate sales. Cyclical functions are used for the majority of the paper as they are a commonly used as a tool to analyze monthly variances. These functions are based on a twelve month scale and vary in percent change of average days of California property on the market. This way, the analysis can emphasize more on the individual changes from month to month rather than year to year. This means that the data compiled across multiple years is analyzed at the same time for each month. To analyze this data, a mean, median, or even a LOESS curve can be utilized to find the trend of the average length real estate is on the market. The mean refers to the value of all variables divided by the number of variables, whereas the median refers to the value that separates the upper half of the data from the lower half. In a LOESS curve, the points are sectioned off into different groups. Within these groups, points are calculated using a focal point, and are determined based on the points closest to the focal point. The closer the point to the focal point, the larger the weight. This is done for every point in the model until the LOESS curve is complete. This can make the model less influenced by one or more outliers. Through the use of these methods, the optimal time for selling real estate based on average days on the market can be found.

Overall Changes:

To look at the optimal sale time in California, aggregated real estate data was pulled from Zillow ranging all the way from January of 2010 to December of 2019 (“California Home Prices”). As can be seen through the following charts, the average days of real estate on the California market has seemed to drop over the course of ten years from almost 100 days on the market to slightly above 50.

This change could have been caused by a variety of factors, including the Great Recession from the end of 2007 until 2009. While the data only tracks the changes starting from 2010, the effects are still great enough to be seen. The Great Recession was caused by the housing market booming then busting when financial institutions over-lended and over-marketed mortgage backed securities at exorbitant levels to sometimes unqualified borrowers (Hall). After the recession, consumers were then hesitant to invest in the very market that had caused the crash. The high levels of days in which real estate was on the market demonstrates the tentative investors. However, as can be seen from 2012 onwards, the days on the market fell as people slowly gained confidence in real estate, with the days on the market generally stabilizing from 2013 to 2017, followed by an even further decrease.

When looking at initial data displayed in Figure 1, one can almost instantly recognize the cyclical form taking place from 2013 to the end of 2019. This initial instinct led to the creation of a graph depicting the percent change from one month to the next over the course of these ten years. From this chart, my initial theory was confirmed and it can be seen that the average days on market of real estate indeed has a cyclical pattern.

Monthly Basis:

After the assurance of a repetitive, reliable pattern from year to year, the next step was finding a way to demonstrate the differences in monthly averages of length of California real estate on the market. To do this, the x-axis was set to a month by month basis (Figure 3). Each year is represented by a scale of colors, with the oldest data from 2010 being the darker blue, and the most recent from 2019 being lighter blue. This way, the data would emphasize each month and the differences between those months.

There were no significant outliers from year to year. For example, the darkest blue colors did not show that the percent change in days on the market to stay the same from January to February, rather, for almost all years, as can be seen through the tight clusters, there seemed to be specific months where the percent change would be similar, whether it be 2010 or 2019. The month of December demonstrates this. From 2010 to 2019, the month of December has stayed between zero to a little above 0.1 percent change in average days of real estate on the California market.

The data points were usually in close clusters, which caused the mean and median to be relatively similar and consistent throughout the twelve months. In Figure 4, the red line represents the median percent change of each month in average days on the market for California real estate while the blue represents the mean of the percent change of each month. The average standard deviation of the points from each month is 0.0375. Some months varied more than others; months such as March had relatively larger standard deviations of 0.083, while months like June had smaller deviations of only 0.015. Overall, the standard deviation is still quite minimal. The green line depicted below demonstrates the mean of the percent change from year to year, which is-0.00034%. When the mean and median (red and blue) are so closely related, it can be said that the data has minimal outliers and that both the mean and median are accurate measures of this set of data points.

Both the mean and median in Figure 4 display the lowest point to be in March, followed by a sharp increase in April and a more gradual incline through to January, where it once again drops dramatically. This has high implications in that it could potentially mean that the month with the shortest days on the market for real estate is March. The high increase from November to January is likely due to low demand during holiday season as most are busy with familial obligations (Fuscaldo). On the other hand, the dip in April and March is most likely caused by the high demand as families start to search for a home before the school year begins (Thorsby).

Figure 5 denotes a similar chart to Figure 3 and 4, except instead of a mean or median, it uses a LOESS residuals curve to find the curve for average days on the market for Californian real estate. In Figure 5, the blue line demonstrates the LOESS curve which is based on the weight of local points, while the gray area following the blue line represents the confidence band. The confidence band is the uncertainty in a curve based on limited data. As can be seen in the chart below, the confidence band is fairly thin, meaning that the difference between the highest and lowest points are significant.

As can be seen in Figure 5, the LOESS curve conveys similar, but not exactly the same information. It still shows that November through January have the longest durations for average days on the market, as well as significant drop in the spring. However, recall that in Figure 4 both the mean and medians displayed March to be the lowest point, but according to the LOESS model it seems that April is the lowest point. The real life theory is still supportive of why consumers have higher demand for housing, yet what is causing the numerical differences? This all leads back to the way LOESS curves are constructed. A LOESS model can be more accurate than the mean or median because it is more influenced by the local points near the focal point, whereas the mean and median can shift quite easily due to a couple of low values, such as in Figure 3 where March is quite low compared to Figure 4. As mentioned earlier, March’s standard deviation is larger compared to the other months, which could be the reason the LOESS curve was only different from the mean and median in this month and not the others.

Conclusion:

Although there are a few minor differences between the median/mean and the LOESS model, the general shape of the curve still suggests that the lowest point is in the spring from March to April. This means that the lowest average days California real estate is on the market is during those months. The decrease could be caused by a rise in demand due to increased pressure to purchase in anticipation of a new job after the summer or new school. The worst time for real estate sales in regard to length of time on market is typically in the winter, where families and individuals are busier with the holidays and are more reluctant to see open houses.

Mentor: Dr. Peter Kempthorne, Massachusetts Institute of Technology

About the author

Renee Wu

Renee Wu is a rising Senior who has a passion for economics, real estate, and investment. She is from California, but currently resides in Shanghai, China. In her free time, she loves to fence or relax with a movie and some caramel popcorn.

The post Analysis of the Duration of California Real Estate on the Market appeared first on Exploratio Journal.