Introduction

Millions of people return to video games after a day of work or school. Most of them replay the same game over and over, even after they’ve “finished” it or reached a high level. According to the Entertainment Software Association (ESA), 61% of the U.S. population (ages 5–90) plays video games at least one hour per week. Among many reasons, one of the main things that makes video games so popular is the competitive system behind them. These systems can take forms like ranked ladders, leaderboards, or PvP (player versus player) matches. They turn gaming from a solo hobby into a social competition. Competitive systems give players goals, opponents, and recognition, and because of this, they hold attention more than other hobbies.

Take League of Legends as an example. The game builds everything around competition. Players team up with four others and face another team of five. After each match, a player’s rank points go up or down based on whether they win or lose, which places them in tiers from “Iron” all the way to “Challenger.” This system gives players a clear goal to work toward and a platform to prove themselves. When players improve, they see it directly in their rank. They also get matched with others at similar skill levels. These features help explain why League of Legends has stayed popular for so long.

Studying how competition keeps people engaged in games can help us build healthier systems: not just in video games, but in companies, schools, and other real-world situations. Competition becomes toxic when it increases hostile behavior, stress, and frustration, or when it pushes players toward unhealthy addiction. Research on online multiplayer communities shows that unclear feedback and harsh punishment systems lead to more hostility and verbal aggression (Kordyaka, Jahn, & Niehaves, 2020; Kou & Nardi, 2014). A healthy competitive system, on the other hand, offers fair challenges, clear rewards, and chances to cooperate. These features tend to build intrinsic motivation and teamwork, helping players stay engaged in positive ways (Deci & Ryan, 2000; Hamari & Keronen, 2017). By understanding what keeps people playing, we also learn what pushes them away. Knowing how competition works in games can help us create environments where people grow under pressure instead of getting hurt by it.

The results show that different competitive systems create different patterns of engagement. PvP attracts the most players, while Ranking leads to the highest time and money investment. Systems that emphasize status pull players in deep, but they also raise emotional pressure: stress, burnout, and addictive behaviors are clearly higher in Ranking and PvP. Timing-based and achievement systems draw smaller but dedicated groups who care about self-improvement and finishing goals, though they struggle with repetition and quitting tied to cost. Non-competitive systems, while showing the lowest engagement levels, give casual players important spaces without pressure. Getting the right mix between these systems matters, and choosing the right combination could encourage sustainable, healthy engagement in competitive environments. 

This study used an anonymous online survey distributed through social media. The survey included structured questions designed to capture participants’ gaming behaviors, preferences, and motivations. To measure gaming intensity, the survey asked about weekly playtime. Genre preference came from asking participants to name their most-played game type. Financial investment was measured through a question about average monthly spending on games and in-game purchases.

To understand competitive orientation, the survey asked whether participants preferred games with strong competitive features like ranked ladders, leaderboards, or PvP modes. Game attachment was measured by asking how often players replay the same game instead of switching to new ones. Motivational factors came from asking respondents to pick their main reason for playing: competition, story immersion, social interaction, or relaxation. The survey also included questions about emotional responses to winning and losing, capturing how competitive play affects players emotionally. Another question asked about how much players value visible recognition systems like rankings, badges, or achievements. Finally, the survey collected demographic information: age group, gender, and participation in other activities: to use as control variables. The whole survey took about three minutes to finish.

Participants came from a Chinese college population. Recruitment happened mostly through online channels commonly used for campus surveys, including student social media groups and course-related platforms. In total, the study collected 1,114 valid responses. The sample was 60.14% female (n = 670) and 39.86% male (n = 444). Age skewed young: 58.89% were 18–25, followed by 17.06% under 18, and 9.96% aged 26–30. The rest were spread across older groups. Education levels matched the college setting: 44.34% were undergrads, 23.25% had associate degrees, and smaller groups reported junior high or below (10.68%), high school or technical school (7.56%), or graduate degrees and above (12.39%). This makes the sample mostly young, educated, and slightly more female, which fits what you’d expect in Chinese universities today.

Literature Review

To better understand whether my observations about competitive games are shared by researchers, it is helpful to look at how previous studies have examined competition, motivation, and gaming behavior.

Previous research has shown that competition plays an important role in shaping player behavior in digital games. Studies on online multiplayer communities suggest that competition system can significantly increase player engagement, but may also lead to negative outcomes such as stress, frustration, and hostile interactions when design elements are unclear or overly punitive (Kou & Nardi, 2014; Kordyaka, Jahn, & Niehaves, 2020). These findings indicate that competition itself is not inherently harmful, but its effects depend largely on how it is structured and experienced by players.

From a motivational perspective, research has emphasized that players are driven by more than simple external rewards. According to self-determination theory, motivation in games can be supported when players experience autonomy, competence, and relatedness (Deci & Ryan, 2000). Building on this idea, studies on game engagement and gamification suggest that transparent feedback, achievable challenges, and meaningful recognition systems are more likely to promote sustained and positive participation (Hamari & Keronen, 2017). In contrast, competitive environments that focus excessively on ranking, punishment, or exclusion may undermine intrinsic motivation and increase emotional pressure.

Research on adolescents and online games further highlights the complexity of game engagement. Large-scale studies in China have shown that online games have both positive and negative impacts on minors. On the one hand, games can provide enjoyment, stress relief, and opportunities for social interaction; on the other hand, a proportion of adolescents display signs of excessive use and difficulty disengaging from games (Tian & Wang, 2022). Importantly, this research suggests that problematic gaming behavior cannot be explained by a single factor, but is related to a combination of individual experience and game structure.

While existing studies have examined gaming addiction, motivation, and social effects separately, fewer studies focus specifically on how differentcompetition system —such as player-versus-player modes, ranking systems, or goal-based challenges—shape patterns of engagement and risk. Most discussions treat competition as a general feature, rather than distinguishing between its different forms.

Based on this gap, the present study focuses on how various competition system are associated with player engagement, motivation, and potential addictive experiences. By comparing different gameplay structures, this research aims to provide a more detailed understanding of how competition operates within games, especially among younger players.

Method and Participants

This study employed an anonymous questionnaire-based survey to examine the relationship between competition system and player engagement in digital games. 

Gaming intensity was measured through self-reported weekly playtime, while genre preference was assessed by asking participants to identify the type of games they played most frequently. Financial investment in gaming was measured using a question on average monthly spending on games and in-game purchases. To examine competitive orientation, the survey included items assessing participants’ engagement with competitive features such as ranked ladders, leaderboards, player-versus-player modes, achievement systems, and time-based challenges. Participants were also asked to report their primary motivation for playing games, including competition, narrative immersion, social interaction, relaxation, and self-improvement.

In addition, the questionnaire included items measuring emotional responses to winning and losing, as well as the perceived importance of visible recognition systems such as rankings, badges, or achievements. Experiences related to gaming addiction were assessed using a series of frequency-based items capturing behaviors such as excessive play, difficulty stopping, emotional dependence, and perceived impact on daily life. Most attitudinal items were measured using five-point Likert scales ranging from 1 (strongly disagree) to 5 (strongly agree). Demographic variables, including age group, gender, educational background, and participation in other activities, were also collected to serve as contextual variables.

Participants were recruited using a non-probability convenience sampling method, primarily through online distribution channels commonly used for campus surveys, including student social media groups and course-related communication platforms. In addition, some responses were collected through school-based channels, allowing access to younger participants. As participants were recruited based on accessibility rather than random selection, the sample does not represent a probability-based population.

A total of 1,114 valid responses were collected. The sample was 60.14% female (n = 670) and 39.86% male (n = 444). The age distribution was heavily concentrated among young people, with 58.89% aged 18–25, followed by 17.06% under 18 (n = 190) and 9.96% aged 26–30, while the remaining respondents were distributed across older age groups. Educational attainment reflected the recruitment context, with most participants reporting undergraduate or college-level education.

Due to the relatively smaller number of adolescent respondents, comparisons between adolescents and adults in this study are presented as exploratory analyses, focusing on identifying patterns and tendencies rather than making population-level generalizations. As the sample was not randomly selected, the findings of this study should be interpreted with caution and are intended to provide insight into possible relationships between competitive game design and player engagement, rather than to represent all player populations.

Results

The study yielded 1,114 valid responses from adolescents and young adults (under 30), providing a broad overview of how players experience different competition system. When categorized by primary competition type, 578 respondents reported most frequently engaging in player-versus-player (PvP) modes, followed by 417 respondents engaging in cooperative-task or player-versus-environment (PvE) modes. Smaller proportions reported primarily engaging in ranking systems (n = 184), achievement-based systems (n = 215), timing-based challenges (n = 130), and non-competitive modes (n = 473). The analysis incorporates both behavioral indicators—such as time spent and money spent—and psychological indicators, including motivation, quitting intention, and addictive tendencies.

Player Investment Across Competitive Systems

Table 1

As shown in Table 1, different competition systems are associated with distinct levels of player investment. Among all gameplay modes, ranking-based systems exhibit the highest average playtime (41.25 minutes) and the highest average financial spending (522.66). These values are notably higher than those observed in the two most common systems, PvP (33.32 minutes; 431.39) and PvE (32.55 minutes; 426.08).

PvP and PvE represent the two most fundamental competition systems in the dataset, capturing direct social confrontation and cooperative performance respectively. They also function as foundational building blocks for other systems, as ranking, timing, and achievement-based modes often incorporate elements of either PvP or PvE. Therefore, comparisons below focus primarily on these two systems, while highlighting how ranking systems amplify their competitive intensity.

The higher levels of time and monetary investment associated with ranking systems suggest that structured competition, particularly when tied to progression ladders and status indicators, is especially effective in sustaining prolonged engagement.

In contrast, achievement-based and non-competitive systems show lower investment levels. Achievement players spend an average of 33.14 minutes and 411.78, while non-competitive players spend 34.22 minutes but only 377.23, the lowest spending among all modes. This suggests that without competitive pressure or visible status rewards, players are less motivated to invest financially.

Motivational Structures in Competitive Gameplay 

Table 2

Beyond behavioral investment, Table 2 illustrates that competitive engagement is driven by multiple motivations rather than the pursuit of victory alone. In ranking systems, players report high motivation not only for ranking and winning (3.76), but also for emotional immersion (3.98), stress relief (4.11), and recognition (3.54). Motivation related to learning and self-improvement (3.35) also remains relatively strong.

PvP modes show a similar but slightly less intensified pattern, with particularly high scores in achievement (3.99) and immersion (3.83). PvE systems, in contrast, emphasize collaboration and immersion, with immersion reaching 4.10 and friend-related motivation at 3.99, reflecting a more stable and predictable engagement structure.

These findings suggest that competition operates as a multi-dimensional motivational environment, fulfilling players’ needs for progress, validation, emotional release, and social connection, rather than functioning solely as a win–lose mechanism.

Timing-based systems stand out for their strong learning motivation (3.846) and freedom (3.7), reflecting their appeal to players who enjoy self-improvement and efficiency. Achievement systems show more moderate scores across all motivational dimensions, while non-competitive modes score lowest on most items except freedom (3.691), confirming their role as pressure-free spaces for casual play.

Addictive Experiences and Competitive Intensity 

Table 3

In addition to increased engagement, competitive and ranking-based systems show a higher concentration of addictive experiences, as presented in Table 3. Players primarily engaged in ranking modes report higher levels of excessive play (2.89), perceived impairment in daily life (2.30), and emotional escape (2.87) compared to other gameplay structures. Financially related addictive behavior is also most prominent in ranking systems, with spending reaching 2.36, the highest among all modes.

PvP modes also display elevated scores on several addiction-related indicators, including overplay (2.79) and craving (2.25), though these values remain lower than those observed in ranking systems. Importantly, while average scores across all modes remain below the threshold of severe addiction, the clustering of higher values in ranking-based gameplay suggests that competitive intensity may increase the likelihood of addictive tendencies emerging, particularly when repeated match cycles and ranking pressure are present.

Among less intensive systems, achievement-based play shows moderate addiction scores, with overplay at 2.753 and craving at 2.163. Timing-based systems have relatively low spending addiction (1.885), the lowest among all modes, suggesting that players focused on efficiency are less likely to spend money excessively. Non-competitive systems consistently score lowest on most addiction indicators, including impairment (1.907) and withdrawal (1.879), supporting their image as healthier play options.

Quitting Intentions and Structural Pressure 

Table 4

Finally, Table 4 demonstrates that players’ intentions to quit are more closely associated with structural pressures than with a simple loss of interest. In ranking systems, players report relatively high levels of stress (3.59) and financial cost (3.92) as reasons for considering withdrawal. The pressure of unmet achievements is particularly notable, with ranking modes scoring 4.02 on “no achievement,” the highest among all systems.

PvP and PvE modes show similar patterns, where quitting intentions are more strongly linked to cost, stress, and unmet progression rather than boredom alone. In contrast, purely non-competitive modes display higher quitting scores related to difficulty and lack of achievement, but comparatively lower stress levels.

These results indicate that players are often driven away not because games become uninteresting, but because competitive structures introduce sustained emotional, financial, and performance-related pressures that gradually outweigh enjoyment.

Summary of Results

The results of this study suggest that competitive and ranking-based game modes are strongly associated with higher player engagement. One possible explanation is that these systems provide players with clear goals and immediate feedback. When players win a match or improve their rank, they receive a sense of progress that encourages them to continue playing. Even after losing, the desire to recover lost points or improve performance may encourage players to play “one more round.” Both of these responses happen in real time during and right after the game. From the players’ perspective, ranking systems make progress visible and measurable, which may help explain why they are linked to longer playtime and higher investment.

Another important finding of this study is that competition is driven by multiple motivations rather than only the desire to win. The data shows that players in competitive modes report strong motivations related to improvement, recognition, emotional release, and immersion. This suggests that competitive gameplay satisfies different psychological needs at the same time. For some players, competition offers a way to test their skills and gain recognition, while for others it provides a way to relieve stress or feel more emotionally involved. As a result, competitive systems may be especially engaging because they combine achievement, emotion, and social interaction into a single experience.

However, the findings also indicate that higher engagement in competitive systems is often accompanied by increased pressure. Players in ranking modes report higher levels of stress, spending, and addictive experiences compared to other gameplay structures. This suggests that competition has pros and cons. While it makes games more exciting and meaningful, it may also increase the difficulty of stopping or taking breaks, especially when progress and performance are constantly evaluated. In addition, the results related to quitting intention show that players are more likely to consider quitting due to stress and cost rather than a lack of interest. This implies that players do not necessarily lose enjoyment, but may feel overwhelmed by the demands of competitive systems.

Beyond the major competitive systems, timing-based and achievement-based modes serve more specialized functions. Timing systems appeal to players who value efficiency and measurable self-improvement, as shown by their high learning motivation scores. However, their narrow player base and high cost-related quitting suggest they work best as complementary features rather than core gameplay loops. Achievement systems, while effective for short-term goal completion, struggle with long-term retention once objectives are exhausted. This explains their high quitting scores related to lack of interest, suggesting that players simply run out of things to do.

Non-competitive systems, despite their low engagement intensity, play an important balancing role in the gaming ecosystem. They provide pressure-free spaces for relaxation, creative expression, and casual social interaction. The high freedom scores in this mode suggest that some players actively seek environments without performance evaluation. However, the elevated quitting motivations related to lack of achievement indicate that pure sandbox experiences may need occasional structure or events to maintain long-term interest. For game developers, this suggests that offering a mix of competitive and non-competitive modes within a single game could support a wider range of player needs: intense ranked play for some, casual exploration for others.

This study has several limitations. First, the data was collected through an online questionnaire, which means the results rely on self-reported experiences and cannot determine cause-and-effect relationships. Second, although the sample size is relatively large, the number of adolescent respondents is smaller compared to adults, so comparisons between age groups should be interpreted cautiously. Future research could use interviews or experiments to better understand how players experience competition over time, or focus specifically on younger players to explore how competitive systems influence their gaming habits.

Overall, this study suggests that competition plays an important role in shaping player engagement. Competitive systems appear to increase both enjoyment and pressure, making them powerful but complex elements of game design. Understanding how different competitive structures influence player behavior may help players, developers, and researchers better reflect on how games are designed and experienced.

This study explored how different competitive systems in digital games are related to player engagement, motivation, and potential addictive experiences. Based on data collected from 1,114 players, the results show that competitive structures: especially ranking-based systems: are associated with higher levels of time investment, financial spending, and emotional involvement. Compared to other gameplay modes, ranked competition appears to intensify both engagement and pressure.

One key finding of this research is that competition motivates players in multiple ways. Rather than being driven solely by the desire to win, players in competitive systems also report strong motivations related to self-improvement, recognition, immersion, and emotional release. This is especially true in timing-based systems, where learning and freedom emerge as primary drivers, and in achievement systems, where goal completion provides structure and direction. Even non-competitive modes serve a purpose, offering players autonomy and relaxation without performance pressure. This suggests that different competitive structures fulfill different psychological needs, and no single system works for all players.

At the same time, the results indicate that higher engagement in competitive modes often comes with increased stress and addictive tendencies. Players in ranking systems are more likely to report excessive play, difficulty stopping, and concerns related to spending and pressure. Importantly, quitting intentions are more closely linked to stress and cost than to a loss of interest, suggesting that players may still enjoy the game but feel overwhelmed by the demands of competitive structures.

This study considers competitive systems as both helpful and potentially dangerous. They can stimulate enjoyment and engagement in the gaming process, but they may also increase the risk of unhealthy play behaviors associated with higher levels of addiction. Based on an online questionnaire survey of 1,114 participants and further analysis, this research provides a clearer understanding of how different competitive systems influence player engagement.

References

Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268.

Hamari, J., & Keronen, L. (2017). Why do people play games? A meta-analysis. International Journal of Information Management, 37(3), 125–141.

Kou, Y., & Nardi, B. (2014). Regulating anti-social behavior on the Internet: The example of League of Legends. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), 616–628.

Kordyaka, B., Jahn, K., & Niehaves, B. (2020). Towards a unified theory of toxic behavior in video games. Internet Research, 30(4), 1081–1102.

Tian, F., & Wang, L. (2022).  A study on minors’ online game use and its impacts.  Youth Research, (3), 45–57.

About the author

Charles Shang

Charles is a 12th grade student based in San Francisco, CA.

The post The Role of Competition in Player Engagement: Evidence from Different Competitive Systems in Video Games appeared first on Exploratio Journal.

Introduction

“87% of neighborhoods in San Francisco undergoing gentrification were once redlined as hazardous” (“From Redlining to Gentrification: The Policy of the Past that Affects Health Outcomes Today”). Redlining was a discriminatory practice in which banks and government agencies denied services such as insurance and mortgages to residents of neighborhoods with large African American and other minority populations. Over time, this practice caused significant disinvestment and economic deterioration of neighborhoods it affected, leaving many people in a fixed place of poverty that prevented upward socioeconomic mobility.

Established in 1933 as a part of Franklin Delano Roosevelt’s “New Deal” program to lift the country out of economic depression, the Home Owners’ Loan Corporation (HOLC) provided mortgage relief to homeowners at risk of losing their homes through foreclosure, in order to stabilize the housing market. As a part of this process, the HOLC created numerous residential security maps, grading different neighborhoods A-D (A being a highly secure and desirable neighborhood & D being a hazardous neighborhood) in about 200 different U.S. cities. Though aiming to create stability in the housing market, these letter grades were primarily based upon the socioeconomic, racial, and ethnic makeup of the neighborhoods’ residents, which facilitated the development of discrimination in the U.S. housing market. With the Johnson administration’s passage of the Fair Housing Act in 1968, any means of housing discrimination on the basis of race, sex, familial status, nationality, and disability were prohibited, thereby marking the closure of the HOLC’s rampant redlining practices of the mid-20th century. However, despite the practice’s prohibition, redlining had already caused a large amount of socioeconomic damage on the communities it had impacted, through a ripple effect of discrimination prevalent in many of society’s pillar institutions (ie schooling, employment, healthy food access)

In this paper I aim to answer the following question: What are the long-term economic impacts of 1930s-era HOLC redlining in the San Francisco Bay Area? By focusing on the quantitative relationships between redlining tract coverage and census data of racial demographics, household income, educational access, and employment, this paper aims to quantify the extent of redlining’s association with economic disparity throughout the late 20th and early 21st centuries through distributional impacts, categorical comparisons, and regression analysis.

From conducting distributional, categorical, and regression analyses, there are several patterns that can be concluded about the long-term economic impacts of New Deal-era HOLC redlining in the Bay Area. Firstly, the distributions holistically showed minimal change in shape over time indicating that the economic indicators measured stayed consistent over the course of several decades. This consistency goes to show that the economic impacts caused by redlining are solid and aren’t weak enough to change over time. Secondly, the regression analysis showed a negative relationship between income and redlining tract coverage with an increasing slope magnitude over time and a statistically significant relationship between the two variables indicated by the very low p-values.  

Section 1: Distributional Impacts

To examine the persistent long-term economic impacts of redlining across different HOLC-graded neighborhoods (graded on desirability), I first generated histograms of median household income (1980-2000), unemployment (1980-2000), and white-occupied housing (1980-2020) across HOLC grades A-D, which reveal several long term distributional economic trends.

Histograms were chosen because they allow one to see the persistence or change of these economic patterns over time by noticing a persistence of change in the spread and shape of the census data across HOLC grades. The incorporation of histogram series by time period with the four different grades per serie allows one to see variation within each grade and how the socioeconomic indicator of one grade changed over time relative to another. Each histogram plots the relative frequency of a given census variable across HOLC tracts A-D as defined by the 1930s HOLC redlining maps. By using relative frequency histograms rather than raw count histograms, the distributions are normalized which allows for fair and accurate comparison across grades that have a different number of census tracts. To ensure accurate comparability over time, the relative frequency histogram that plots median household income (1980-200) across HOLC grades A-D were converted from nominal US dollars to real 2020 US dollars using the Bureau of Labor Statistics’ annual average CPI values.

Subsection 1A: Median Household Income

The first socioeconomic indicator is real median household income. This variable measures the typical earning level of all households in a neighborhood, which is a strong indicator of overall economic opportunity in that neighborhood. Because nominal incomes change with inflation, the median household incomes were converted from nominal to real 2020 U.S. dollars using annual average CPI values from the U.S. Bureau of Labor Statistics. 

The first observation that can be drawn from Figure 1.1 is that the shape of the distributions of median household income for each HOLC grade remains approximately the same consistently across all three dates of measure. This indicates that aggregate household income for each grade persisted over the period of measure.

The second observation that can be drawn from Figure 1.1 has to do with the differences in intergrade clustering and spread of the data. Grade A’s distribution has a slight right-skew with a moderately-large income range that persists across the three dates of measure. Moving from the Grade A distribution to the Grade D distribution, the data gradually shifts left with each grade closer to D. Grade D’s distribution shows one, a high level of clustering at the left side of the left side of the histogram and two, an income range that is significantly smaller than that of Grade A’s distribution. These two observations continue throughout the three dates of measure.

From these observations, it can be concluded that Grade A Bay Area neighborhoods have a larger range of median household income values with fewer observations of lower median household income values. This conclusion is in accordance with the inference that those in a grade with greater security and desirability tend to have higher job opportunity, educational access, and healthcare access. The subtle right-skew in Grade A’s distribution indicates that a smaller proportion of tracts have exceptionally high incomes which increases the mean and pulls the distribution’s tail rightward. The gradual leftward shift of the distributions from Grade A to Grade D indicates that as the magnitude of risk assessed by the HOLC for the tracts is inversely proportional to the median household income for the tracts. Moving to Grade D, it can be concluded that tracts judged to have lower security and desirability by the HOLC have a smaller range of median household income values and a high proportion of observations of lower median household income. This conclusion is in agreement with the inference that individuals in this grade tend to have lesser job opportunity, lesser educational access, and lesser healthcare access, traits which undermine ability to achieve high household income.

In addition, the moderately-large range of income denoted in Grade A’s distribution hints at a more discrete but key principle of 1930s HOLC redlining: racial prejudice. As observed in Grade D’s distribution, there was a visible clustering of tracts toward the left of the histogram around the lower income values. Intuitively, this logic should apply to Grade A as well but in an opposite fashion: clustering of tracts toward the right of the histogram around the higher income values. However, in Grade A’s distribution there is a significantly-sized range and spread of income, much larger than that of Grade D’s distribution, indicating that Grade A neighborhoods contained households with a variety of median household income values. From this distributional observation, it can be understood that HOLC redlining grading was deeply influenced by racial prejudice, more so than objective economic data for a lot of the time.

Subsection 1B: Unemployment

The second socioeconomic indicator is percent of civilians unemployed (16+). This variable measures the share of the working-class population per grade that are actively seeking employment but cannot find work. Redlined neighborhoods often see disinvestment and lower amounts of socioeconomic mobility so distributional unemployment is a strong indicator of the severity of redlining experienced by grade.

Similar to the distribution of median household income across HOLC grades, the first observation drawn from Figure 1.2 is that consistently across all three dates of measure, the shape of the distribution for each respective HOLC grade remains approximately the same, with only a few minor shifts, which indicates that unemployment shares for each grade persisted over time. This persistence in unemployment suggests that the socioeconomic division caused by 1930s HOLC redlining has stayed relatively intact in the latter half of the 20th century, with census level unemployment shares continuing to mirror structural inequalities imposed by HOLC redlining maps.

The second observation drawn from Figure 1.2 is concerned with the differences in intergrade clustering and spread of the data as well. Generally, these differences are the same as in Figure 1.1 but a mirror flip. Grade A’s distribution shows high left-clustering, a small right-skew, and a very high proportion of observations on that side of the histogram. Moving from the Grade A distribution to the Grade D distribution, the data gradually gains overall spread/range and magnitude of right-skew while also losing peak height. Grade D’s distribution shows a range and median greater than that of Grade A. This pattern of inter-grade shifting from A to D remains approximately the same over the period of measure.

From these observations, it can generally be concluded that the magnitude of risk assessed by the HOLC for the tracts is directly proportional to the percent of civilians (16+) unemployed for the tracts. The association between these two variables is consistent with the inference that redlined neighborhoods tend to have lower economic opportunity, in this case job access, resulting in a higher unemployment rate in these tracts. While the median percent of civilians (16+) unemployed increases moving from Grade A to Grade D, the magnitude of right skew also increases which means that there is a higher degree of variability in tracts deemed less secure and desirable by the HOLC. Economically, this increase in variability in percent of civilians (16+) unemployed from Grade A to Grade D means that the magnitude of risk assessed by the HOLC for the tracts is associated with a higher degree of economic volatility in the tracts. These tracts that have experienced a high amount of redlining have also seen uneven patterns of disinvestment, reinvestment, demographic/population change, gentrification, industrial restructuring, factors which are greatly responsible for a high variation in unemployment in these tracts. In addition to an increase in magnitude of right skew from Grade A to Grade D, a decrease in peak height is also observed. This means the magnitude of risk assessed by the HOLC for the tracts is inversely proportional to the proportion of observations made. Similar to an increase in spread, a decrease in peak height also denotes a proportional relationship between degree of redlining and degree of economic volatility observed. From a statistical standpoint, the decrease in peak height, or flattening, of the distribution means that a fewer proportion of tracts share a common unemployment rate for civilians (16+). This flattening represents a growing internal heterogeneity suggesting that redlining produced a fragmented and nonuniform economic landscape in the areas it greatly impacted.

Subsection 1.C: White Share

The third socioeconomic indicator is percent of white-occupied houses. This variable measures a specific racial composition of occupied housing units. Historically, redlined neighborhoods in the mid 20th century saw a large amount of urban flight in which primarily caucasian residents departed urban areas for suburban neighborhoods. As a result of this exodus, over time, redlined neighborhoods saw a change in white residents so by measuring this variable, we will see how the magnitude and direction of this change across grades and the fluctuations in change over time. 

Similar to the previous two distributions, the first observation that can be drawn from Figure 1.3 is that across the five dates of measure, the shape of the distribution for HOLC grades A, B, and C remain approximately the same, with only a few minor shifts. This indicates that for these grades, the distribution of percent of white occupied homes persisted over the course of 4 decades. Likewise to the other socioeconomic indicators measured in Figures 1.1 and 1.2 respectively, the persistence of shares of white occupied homes suggests that the census-level outcomes of the racial bias  ingrained into HOLC redlining maps has seen very minimal change in the highly to moderately secure and desirable tracts (A-C), indicating that housing segregation on the basis of race has persisted over time in the Bay Area.

The second set of observations that can be drawn from Figure 1.3 concerns the shape of each grade’s distribution over the period of measure: what the shape means and for Grade D, what a shift in the distribution’s shape means. Firstly, moving left to right from Grade A to Grade D, there is a gradual flattening of the distribution, most significantly moving from Grade A to Grade B, and there is a shift from a right-skew to a left-skew. Moving vertically down the figure from 1980 to 2020, the distributions of Grades B, C, and D see a subtle and gradual increase in aggregate peak height over time. In Grade D, there is a sharp increase in proportion of observations over time at the 7-20% range of the 2020 distribution.

From these observations, it can be concluded that magnitude of risk assessed by the HOLC for the tracts is inversely proportional to the percent of white-occupied houses for the tracts over time. As the HOLC grade declines in security and desirability, the distribution becomes increasingly concentrated toward lower percentage values of white-occupied houses. This inverse association, shown by the shift of a right-skew to a left-skew from Grade A to Grade D in each date of measure, confirms our understanding of HOLC redlining having a basis of racial prejudice, as lower-tier grades and areas with higher levels of redlining tend to have a greater minority demographic and a lower majority, in this case caucasian, demographic. The flattening of the distributions in the earlier dates of measure (1980 and 1990 primarily) indicate an increase in demographic variability as tracts become more “hazardous.” While secure and desirable areas remain racially homogenous with a high proportion of caucasians, areas with lower security and desirability are more racially heterogenous reflecting both racial segregation’s influence in HOLC redlining as well as subsequent population shifts driven by disinvestment and suburbanization. The increase in overall peak height for Grades B, C, and D but retainment of a roughly flat shape relative to Grade A’s distribution indicate that there is a greater amount of census data concerning white-occupied housing moving forward in time, but the demographic trends revealed by this data remain persistent as mentioned in detail above. The sharp increase in peak height in 2020 at the 7-20% range of the Grade D histogram indicates that there is a significantly higher proportion of observations concentrated around the lower percent of white-occupied houses in recent times, a trend that can be explained by “white-flight” and suburbanization. This migratory occurrence is defined as the migration of primarily caucasian residents from urban to suburban areas, which results in the economic degradation of urban areas through decreased funding and overall maintenance.

Section 2: Categorical / Group-Level Comparisons

To further explore the economic impacts of 1930s HOLC redlining in the Bay Area, I used categorical plots to visualize redlining’s impact through median household income and educational attainment across HOLC grades A-D.

Definitionally, a categorical plot allows for comparison of a single numerical variable across four levels of categorical variables, in this case, each level corresponding to a HOLC grade A-D. The important distinction that needs to be made in order to understand what the differences are in the displaying of categorical variables (HOLC grades) is the difference in interpretation over time for the histograms vs. the categorical plots. The histogram focuses on distributional change within a singular HOLC grade over time, which provides insight into intra-grade socioeconomic change over the period of measure. On the other hand, the categorical plot focuses on the change in inter-grade variation over time for a given socioeconomic indicator.  Due to this difference, I chose to incorporate categorical plots to supplement the distributions in order to visualize comparative disparities between HOLC grades.

Furthermore, in connecting the two models, a single bar within a categorical plot can be interpreted like a distribution for the HOLC grade that the bar represents, through the spread of data points within the bar.

Subsection 2A: Median Household Income

The first socioeconomic indicator to be analyzed categorically is median household income (real 2020 USD) across HOLC grades A-D. In the figure that assesses this variable, there are three categorical plots, one plot for each period of measure, 1980, 1990, and 2000 respectively.

The first observation that can be drawn from Figure 2.1 is the clear downward trend from Grade A to D that is consistent across all three dates of measure, with Grade A having the highest peak and Grade D having the lowest peak. The second observation that can be drawn from Figure 2.1 concerns intra-grade variability indicated by the distribution of data points within each HOLC Grade’s bar. Consistently across all three dates of measure, Grade A tracts show the greatest intra-grade variability, indicated by a greater spread of data points within the bar. As the HOLC grade declines however, there is a decrease in intra-grade variability, indicated by the increasing clustering of data points within each bar. The third observation that can be drawn from Figure 2.1 is an increase in the peak height for each HOLC Grade’s bar over time, while the vertical separation between each bar remains approximately equivalent over time. 

From the first observation, it can be roughly concluded that magnitude of risk and insecurity determined by the HOLC and the median household income are inversely proportional for the tracts. This observation is consistent with the basis of redlining, in which redlined neighborhoods saw high levels of disinvestment and economic deterioration, preventing upward socioeconomic mobility for residents of these tracts, therefore explaining why residents of redlined tracts tended to have lower median incomes than those of non-redlined tracts. From the second observation concerning intra-grade variability, the higher levels of variability in tracts deemed less risky and more secure by the HOLC indicate that in these tracts, there is a greater degree of economic opportunity and mobility, providing an explanation for a wider spread in Grade A and a tighter clustering in lower grades, where many residents of redlined tracts are stuck in a lower socioeconomic position and unable to advance upwards. From the third observation concerning the increase in relative heights of each HOLC Grade’s bar over time, we can conclude that across all tracts, redlined or not, general development over the period of measure through technological innovation and globalization promoted the median household incomes for each HOLC Grade. Despite this overall increase, the vertical separation for each catplot between each HOLC Grade remains approximately equivalent suggesting that relative disparities between grades persisted despite overall income growth over the period of measure.

Subsection 2B: Educational Attainment

The second socioeconomic indicator to be analyzed categorically is educational attainment across HOLC grades A-D. This variable will be measured by the percent of adults 25 and older with 4 or more years of college education. In the figure below, there are three categorical plots, one plot for each period of measure, 1980, 1990, and 2000 respectively.

The first observation that can be drawn from Figure 2.2 is the clear downward trend from Grade A to Grade D across all three dates of measure, with Grade A having the highest peak and Grade A having the lowest peak. The second observation that can be drawn is the gradual upward shift in each grade over time, shown by the increasing peak height for each HOLC Grade’s bar. In connecting these two observations, we see that the difference in peak heights between the HOLC Grades decreases over the period of measure, with the bars for Grades B-D increasing by a greater amount than the bar for Grade A, revealing a flattening of the downward trend from Grade A to D.

From the first observation, it can be concluded that magnitude of risk and insecurity determined by the HOLC is inversely proportional to the percent of adults 25 years of age and older with 4 or more years of college education for the tracts. This conclusion aligns with the economics of redlining in which neighborhoods that experienced the negative implications of the practice saw less economic opportunity as well as overall disinvestment in their communities. Tracking back to Figure 2.1, we see that neighborhoods in lower tier HOLC grades have generally a lower aggregate median household income. Due to the aggregate financial status of these neighborhoods, we can infer that one of the reasons that households in redlined neighborhoods saw lower degrees of educational attainment was they comprehensively had less disposable income to spend on privileges like textbooks, tutoring services, or in this case, a college education.

From the second observation, it can be concluded that the moderate to lower tier grades saw upward educational mobility over the period of measure. This pattern can be primarily attributed to neighborhood redevelopment and gentrification throughout the Bay Area, particularly in tracts close in proximity to university centers, like UC Berkeley, or booming industries that have attracted more college-educated residents in search of work on a domestic and international level, like Silicon Valley and the South Bay. Coinciding with these inferences, the Bay Area saw several urban redevelopment projects in parts of Oakland, Emeryville, and San Francisco’s Mission District in which communities experienced infrastructural improvement in housing and schools. In a research report titled Engaging Schools in Urban Revitalization: The Y-PLAN (Youth – Plan, Learn, Act, Now!), authors Deborah L. McKoy and Jeffrey M. Vincent define the Y-PLAN initiative in the context of the urban Bay Area: 

“West Oakland, California, is an industrial area suffering the abandonment and blight common to other neighborhoods after the loss of manufacturing employers, a process that began in the 1950s…Stepping into this environment in 2000 was the Y-PLAN (Youth—Plan, Learn, Act, Now!), a model for youth civic engagement in city planning that uses urban space slated for redevelopment as a catalyst for community revitalization and education reform. Sponsored by the Center for Cities & Schools at the University of California (UC), Berkeley, Y-PLAN facilitates positive community outcomes by partnering graduate student “mentors,” local high school students, government agencies, private interests, and other community parties to work on a real-world planning issue. The Y-PLAN is both a pedagogical tool and a planning studio that addresses specific issues in local communities” (McKoy & Vincent, 2007, p. 1).

Due to the implementation of programs such as the Y-PLAN that educationally mobilized the Bay Area’s youth in impoverished areas, many of these neighborhood areas saw educational improvement reflected by the increasing peak heights for tracts in HOLC Grades B-D in the categorical plot. As a result of the flattening of the downward trend, it can be concluded that the educational disparities within the Bay Area were ameliorated over the period of measure, in part due to youth initiatives and other programs seeking educational improvement.

Section 3: Regression Analysis of Raw Median Household Income

To determine the impact of redlining tract coverage on median household income, I also conducted ordinary least squares (OLS) linear regressions for both the raw and logarithm of median household income as a function of redlining tract coverage from a 1930s HOLC map. While the histograms and catplots provide distributional insight and categorical comparison, respectively, into disparities across HOLC grades, the regression allows for high quantitative precision in determining the extent to which redlining coverage can accurately predict income for a Bay Area neighborhood. More specifically, the categorical plots allowed us to see a general slope trend based on the peaks of each bar, but the regression expands upon this observation by making it more specific numerically.

Because nominal incomes change with inflation, the median household incomes were converted from nominal to real 2020 U.S. dollars using annual average CPI values from the U.S. Bureau of Labor Statistics. The purpose of conducting a log-transformed regression was to linearize the relationship and allow for the slope coefficient to be interpreted as a percent change in income for a one-unit increase in tract coverage.

The two key regression outputs that will be analyzed in this section are slope coefficient and p-values, which together, can assess the magnitude and certainty, respectively, of the relationship between redlining tract coverage and median household income over time. For the raw income model, the slope coefficient is interpreted as the predicted change in real median household income by the OLS regression line per one-unit (or 100%) increase in redlining tract coverage. For the log-transformed income model, the slope coefficient is interpreted as the predicted percent change in real median household income by the OLS regression line per 1.0 (or 100%) increase in redlining tract coverage. Given that the distribution of income is skewed, taking the logarithm of income compresses high-income and low-income outliers, which creates a more symmetric and homoscedastic residual distribution. The patterns observed in the logarithm-transformed regression are largely identical to those observed in the untransformed regression, so I decided to include logarithm-transformed regression analysis in the first appendix of the paper. The p-values for each model (raw income and log-transformed income) represent the probability of observing each respective slope by random chance. A low p-value (less than the alpha level of 5%) indicates that the observed relationship is unlikely due to random chance.

Several neighborhood wealth-related trends can be concluded from OLS regression analysis of raw and log-transformed real median household income (2020 dollars) as a function of redlining tract coverage from 1980-2000. While quantitative, these patterns share similarities with the distributional trends derived from the histograms showing real median household income (2020 dollars) across HOLC grades A-D described in detail earlier in the paper.

Here, our parameter beta represents the true slope of the population regression line relating the explanatory and response variables of redlining tract coverage and raw income respectively. The null hypothesis is that beta is equal to zero and the alternative hypothesis is that beta is less than zero.

Figure 3.1 Raw Income as a function of Redlining Tract Coverage

Figure 3.1 confirms a negative association between income and redlining tract coverage based on the slope coefficients. Firstly, the slope coefficient for the 1980 OLS regression is -15,205.29. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a $15,205.29 decrease in real median household income (2020 dollars) in 1980. Secondly, the slope coefficient for the 1990 OLS regression is -29,742.12. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a $29,742.12 decrease in real median household income (2020 dollars) in 1990. Thirdly, the slope coefficient for the 2000 OLS regression is -46,819.90. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a $46,819.90 decrease in real median household income (2020 dollars) in 2000.

While I noted that the association between income and redlining tract coverage remains negative, from the slope coefficients in Figure 3.1, it can also be observed that the magnitude of the decrease in income increases with each subsequent date of measure. From a graphical standpoint, this can be understood as an initially negative regression line that gets steeper and steeper in the negative direction over time. So what does this mean in the context of redlining? From this observation, it can be concluded that the negative impact of redlining coverage on household income worsens overtime, as the same increase in coverage is met with a greater magnitude of decrease in income over the period of measure. This trend can be best explained by the extensive impact of disinvestment in higher redlined areas. Following the HOLC’s classification of neighborhoods as “hazardous” via their redlining maps, banks and financial institutions chose not to lend money to businesses and individuals, insure mortgages, or fund development in these areas due to the high-risk attributed to these areas by the HOLC. As a result of this lack of financial and infrastructural support from banking institutions, these areas deteriorated over time through an aggregate decrease in property values as well as less socioeconomic mobility and in this case, access to high-paying jobs. Due to the extensive decline of the economic makeup of these areas caused by disinvestment, it makes sense why the slope coefficient of the income vs. redlining tract coverage regression decrease over time.

Figure 3.1 also displays very low p-values that all fall significantly below the general alpha level of 5%. Therefore, we can ascertain a relationship between redlining tract coverage and income. Firstly, the p-value for the 1980 OLS regression with raw income as the dependent variable is  . Assuming that there’s no association between redlining tract coverage and real median household income in 1980, the probability of obtaining a sample of this size and observing a negative relationship between redlining tract coverage and real median household income (2020 dollars) as or more extreme than a magnitude of approximately $15,205.29 per percentage point of redlining tract coverage by random chance is less than approximately   (rounded 0.0002%). Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower real median household income in 1980 for this sample. Secondly, the p-value for the 1990 OLS regression with raw income as the dependent variable is  . Assuming that there’s no association between redlining tract coverage and real median household income in 1990, the probability of obtaining a sample of this size and observing a negative relationship between redlining tract coverage and real median household income (2020 dollars) as or more extreme than a magnitude of approximately $29,742.12 per percentage point of redlining tract coverage by random chance is less than   (rounded 0.0008%). Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower real median household income 1990. Thirdly, the p-value for the 2000 OLS regression with raw income as the dependent variable is  . Assuming that there’s no association between redlining tract coverage and real median household income in 2000, the probability of obtaining a sample of this size and observing a negative relationship between redlining tract coverage and real median household income (2020 dollars) as or more extreme than a magnitude of approximately $46,819.90 per percentage point of redlining tract coverage by random chance is less than   (rounded 0.0002%). Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower real median household income 2000.

The consistently low p-values across all three dates of measure (less than the alpha level of 5%) confirms the high statistical strength between redlining tract coverage as an independent variable and median household income as a dependent variable. Furthermore, this aspect of the regression corroborates the trends observed from the slope coefficients of the decreasing negative association between these two variables, indicating that it is extremely unlikely this relationship occurred by random chance.

Conclusion

From distributional impacts, to categorical plots, to regression analysis, we can conclude several long term economic impacts that vary by extent to which a given census tract was redlined and how it changed over time. With the histograms, we saw that a lot of the economic patterns surrounding income and unemployment maintained shape for a given grade distribution over time indicating that a lot of the economic impacts stayed stagnant over the course of a few decades. With the regression analysis, we saw that the magnitude of negative slope for raw income as a function of redlining tract coverage greatly decreased in the 20 year period from around $30,000 to $42,000 in 1980 and 2000 respectively. With the log transformed regression, the slope coefficients were approximately equivalent for each year indicating that income dropped by the same percentage each year which shows relative stability in the influence of redlining, not due to external factors.

Looking forward, I would love to dive deeper into how gentrification interacts with these socioeconomic patterns of redlining. By definition, gentrification is the process in which a poorer area is infrastructurally improved as a result of a wealthier demographic of people moving in. This economic improvement is seen over time through improved housing, healthcare, and new business. By exploring gentrification in the context of redlining, I would ask whether modern reinvestment in previously redlined areas resulted in economic growth, ameliorating the negative effects of 1930s-era HOLC redlining.

While in this paper, I primarily studied solely socioeconomic indicators measuring the effects of redlining (unemployment, income, racial demographics, etc.), I would be curious to explore more health-related effects of this same era of redlining. By conducting research on food deserts, urban areas with limited access to good-quality fresh food, through measuring, for example, the number of fast food restaurants in a given census tract as well as illnesses through measuring, for example, the number of diabetes occurrences or hospital visits in a given census tract, I would bring in a new dimension of human health and biology in association with the redlining I explored in this paper. 

Bibliography

De los Santos, H., Jiang, K., Bernardi, J., & Okechukwu, C. (2021, May 26). From redlining to gentrification: The policy of the past that affects health outcomes today. Harvard Medical Journal. Retrieved October 28, 2025, from https://info.primarycare.hms.harvard.edu/perspectives/articles/redlining-gentrification-health-outcomes

Jonathan Schroeder, David Van Riper, Steven Manson, Katherine Knowles, Tracy Kugler, Finn Roberts, and Steven Ruggles. IPUMS National Historical Geographic Information System: Version 20.0 [dataset]. Minneapolis, MN: IPUMS. 2025. http://doi.org/10.18128/D050.V20.0

McKoy, D. L., & Vincent, J. M. (2007, June). Engaging schools in urban revitalization: The y-PLAN. Association of Collegiate Schools of Planning. https://doi.org/10.1177/0739456×06298817

Nelson, R. K., Winling, L, et al. (2023). Mapping Inequality: Redlining in New Deal America. Digital Scholarship Lab. https://dsl.richmond.edu/panorama/redlininghttps://dsl.richmond.edu/panorama/redlining.

Appendix A: Regression Analysis of the Logarithm of Median Household Income

While the raw-income regression shows the dollar change in income as a function of redlining tract coverage, by taking the logarithm of income and performing the same regression function, we see the percent change in income as a function of redlining tract coverage for the same three dates of measure. Though not visible from the regression outputs, the log-transformed regression mitigates the effect of outliers and influential points that undermined the strength of the relationship between raw-income and redlining tract coverage, providing us slope coefficients and p-values that are optimal for ensuring an accurate relationship between the two variables.

Here, our parameter beta represents the true slope of the population regression line relating the explanatory and response variables of redlining tract coverage and logarithm of income respectively. The null hypothesis is that beta is equal to zero and the alternative hypothesis is that beta is less than zero.

Figure A1 Logarithm of Income as a function of Redlining Tract Coverage

Similar to the raw income regression, the results of the log-transformed regression shown in Figure A1 confirm a negative association between redlining tract coverage and income as well. The slope coefficient for the 1980 OLS regression is -0.30. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a 30% decrease in real median household income (2020 dollars) in 1980. The slope coefficient for the 1990 OLS regression is -0.28. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a 28% decrease in real median household income (2020 dollars) in 1980. The slope coefficient for the 2000 OLS regression is -0.32. This means that for each additional one-unit (or 100%) increase in redlining tract coverage, the OLS regression line predicts about a 32% decrease in real median household income (2020 dollars) in 1980.

Similar to the raw income regression, the results of the log-transformed income regression evidenced in Figure A1 show very low p-values. Therefore, we can ascertain a relationship between redlining tract coverage and income. The p-value for the 1980 OLS regression is  . Assuming that there is no relationship between redlining tract coverage and the logarithm of real median household income in 1980, the probability of obtaining a sample of this size and observing a linear relationship between redlining tract coverage and the logarithm of real median household income (2020 dollars) with a slope coefficient of -0.30 or less by random chance alone is less than  %. Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower proportional levels of real median household income in 1980. The p-value for the 1990 OLS regression is  . Assuming that there is no relationship between redlining tract coverage and the logarithm of real median household income in 1990, the probability of obtaining a sample of this size and observing a linear relationship between redlining tract coverage and the logarithm of real median household income (2020 dollars) with a slope coefficient of -0.28 or less by random chance alone is less than  . Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower proportional levels of real median household income in 1990. Lastly, the p-value for the 2000 OLS regression is  . Assuming that there is no relationship between redlining tract coverage and the logarithm of real median household income in 2000, the probability of obtaining a sample of this size and observing a linear relationship between redlining tract coverage and the logarithm of real median household income (2020 dollars) with a slope coefficient of -0.32 or less by random chance alone is less than  . Therefore, the OLS regression model provides significant statistical evidence that tracts with higher redlining coverage are associated with lower proportional levels of real median household income in 2000.

For the most part, these two outputs of the regression reveal largely similar or identical patterns as the raw-income regression. Firstly, the slope coefficients reveal a negative relationship between redlining tract coverage and median household income that steepens over time. This steepening means that the percent decrease in median household income per 100% increase in redlining tract coverage increases with each subsequent date of measure.

Again, the p-values for this regression are far below the alpha level of 5% meaning that a relationship between redlining tract coverage and the logarithm of income is statistically significant across all three dates of measure, and not due to random chance.

About the author

Rishi Haldar

Rishi is a senior at Miramonte High School with interests in economics, mathematics, statistics, and history. He plans to attend Cornell University in the fall, where he will be studying economics in the College of Arts and Sciences. Apart from academics, Rishi is a guitarist in a band that plays local gigs (restaurants, fundraisers, etc.) and plays soccer for a club team and his high school team.

The post Redlining: Quantifying the Economic History of the San Francisco Bay Area appeared first on Exploratio Journal.

Abstract

This paper explores the complex relationship between tariffs on Gross Domestic Product (GDP) and the U.S. trade balances with its major trading partners. It investigates if imports and greater trade balances changed between the U.S and its top trading partners after tariffs were placed. The conclusion is no significant change in trade balances since the implementation of the Trump administration’s tariffs.

The evidence shows that in the months of April to July, the tariffs have not significantly changed U.S. net trade. The effects studied in this paper are a result of new trade policies by the Trump Administration which put retaliatory tariffs on most of the world. As a result, many firms and businesses frontloaded the tariffs which caused a 40% increase in imports in Q1 2025. Also in Q1, real GDP decreased by 0.5% (U.S. Bureau of Economic Analysis, 2025) according to the most recent estimate. However, imports do not reduce GDP and are only included in the calculation to support accounting principles. As a result of this misconception, many news articles written by journalists who are not economists have had misleading claims with regards to the GDP decrease. The current results of the findings could potentially be attributed to the uncertainty in the administration’s tariff policy or simply that not enough time has passed for significant changes to be observable in the data.

Introduction & Literature Review

The role of imports in shaping a nation’s economy has become increasingly significant following President Trump’s trade wars and tariffs on countries across the world. Many of these tariffs were put during the first quarter of 2025 while the others were placed on April 2, 2025, also known as Liberation Day. However, news of President Trump’s intention of using tariffs has been clear before his Inauguration and use of tariffs on foreign countries was common in his first term as well. Since his election, many businesses and firms have increased inventories and the amount of imported goods in anticipation of high tariff rates to go into effect soon.

As noted above, the heart of GDP measurement is the widely cited expenditure formula: GDP = C + I + G + (X-M) where C denotes consumption, I investment, G government expenditures, X exports, and M imports. The superficial glance at this equation shows imports as a direct drag on GDP. However, economists consistently clarify that this superficial glance is quite misleading as the negative sign simply represents an accounting principle to prevent double counting.

Bill Conerly (2025), a writer at Forbes, clarifies that “U.S. imports are neither added nor subtracted conceptually” (para. 3) for GDP. He explains that with perfect data available to statisticians, imports wouldn’t be included in a GDP calculation (Conerly, 2025).

Looking at the subtraction, Greg Mankiw notes, “this subtraction is made because imports of goods and services are included in other components of GDP,” (Mankiw, 2001, p. 499) Mankiw also notes how a purchase of an imported good raises consumption, investment or government expenditures.

The St. Louis Fed adds that “imports (foreign production) should have no impact on GDP,”(Wolla, 2018, para. 9). They explain the variable M as an accounting variable rather than an expenditure variable. It is also important to note that the imported goods will have an effect on the GDP of the country that produces them. Since it isn’t the United States, they don’t affect U.S. GDP. However, it can take into account if the goods are intermediate or partially produced in the U.S. Since the expenditure variables of C, I, and G only take final goods into account, GDP will be affected based on the amount of the goods that was domestically produced.

Keshav Srikant, a writer with Econofact, supports the net-zero effect on imports on GDP. However, he also notes how imports can potentially indirectly reduce GDP if they replace domestic consumption or if domestic government expenditures are reduced as a result of higher purchases of foreign goods (Srikant 2025). Further study of macroeconomic trends are required to make an argument for this situation as these latent variables could drive import growth and GDP declines when those two variables are not correlated.

Ultimately, imports do not directly reduce GDP and their inclusion in the components of GDP is a measure to prevent double counting.

There are many researchers who have explored the growth of imports and its, relationship with the overall economy. In fact, many specific case studies have found that an increase in imports often leads to an increase in real GDP .

A study by Peter Saunders, focused on a time series analysis of the role of imports in the economic rise of India from 1970 to 2005, analyzes the long term relationship between imports and India’s real GDP. Saunders establishes that both variables, imports and real GDP, are cointegrated using Johansen’s test of cointegration (Saunders 2010). This test proves if two variables have a long term equilibrium relationship, meaning that despite short term deviations or outliers, the variables have a long term observed relationship. A Vector Error Correction Model (VECM) examines the relationship between cointegrated variables. In the VECM used by Saunders, the results indicated that imports have positively impacted India’s economic growth in the short-term. Saunders highlights how this result defies traditional expectations that imports could be a drag on the economy (Saunders 2010).

In another study by M.Y Khan et al, about the relationship between imports and economic growth in Pakistan, a similar conclusion was reached. This study used data from 1975 to 2014 with the methodology of a Granger Causality Test. This test focuses on proving directional relationships between time series variables. The results showed that there was a bi-directional relationship between imports and economic growth in Pakistan, meaning that both time series variables mutually supported one another (Khan et al, 2019).

Research focusing on the relationship Rwandan economic growth with imports and exports showed a positive long run relationship (Al Hemzawi & Umutoni 2021) . The study concluded that a one percent increase in imports led to a 0.32% rise in Rwandan GDP. To get that correlation, the authors used a multivariate Ordinary Least Squares regression which is a way to minimize variance between variables. They also used quarterly time series data in the regression.

Immediately after the Q1 GDP contraction was announced, many news publications released misleading or false articles regarding the cause behind this result. They accredited the cause to be the tariff jumping effect and the dramatic import surge that occurred because firms and businesses rushed to purchase foreign goods before tariff prices were assigned to goods. The underlying demand was quite consistent to previous levels while business investment surged as an offset to the imports. Despite the fact that imports do not directly reduce GDP, news outlets continued to push that narrative.

For example, an AP News article stated “First-quarter growth was weighed down by a surge of imports, ” (Wiseman 2025, para. 2) while The Hill said “GDP shrank in the first quarter mostly because of lower consumer spending and a pull-forward in imports ahead of President Trump’s tariffs, ” (Burns 2025,para. 4). Many other outlets made misleading claims regarding the import surge. Although journalistic misconceptions are not uncommon, even the Federal Open Market Committee has made mistakes with regards to the effect of imports on GDP (Lemieux 2018).

On the other hand, many top economists have had different opinions. Many economists have attributed the contraction to the economic activity as a result of the imports, not by the imports directly. For example, Paul Gruenwald, a global chief economist for S&P Global Ratings, mentioned that Q1 GDP data was “distorted by the front-running of tariffs,” (2025). Gregory Daco, a chief economist at EY , added “the contraction was largely a function of economic activity being pulled forward as importers, business, and consumers rushed to get ahead of tariff implementation,” (2025). Economist Preston Caldwell ofMorningstar added that imported goods could be stored in inventories but “it just didn’t show up in the data because of measurement error,” (2025).

Some top economists also challenged the fear that this GDP result was the first domino in a potential recession. Caldwell added that this result “doesn’t mark the beginning of a recession,” (2025). Others mentioned potential for economic uncertainty further down the line as more policy was unveiled. “Demand in the first quarter looks to be driven by businesses battening down the hatches before the storm,” Chief economist Luke Tiley of the Wilmington Trust said (2025).

One potential explanation for the GDP decrease is a phenomenon called the substitution effect, a phenomenon that suggests that the tariff induced frontloading substituted for domestic purchases. If this is the case, GDP would decrease since less money would be spent toward domestic production. This has been prevalent in the past as well.

In the 1990s, the Northern American Free Trade Agreement (NAFTA) contemplated potential tariff reductions. 96% such reductions were announced far in advance, giving consumers and firms the chance to act on this information ( Khan & Khederlarian 2021). A study found that in anticipation of an upcoming tariff reduction of 1%, imports dropped by a sizable 6% in the months before the tariff implementation when compared to regular months. The study used an Herfindahl-Hirschman Index, a method to measure market concentration, and applied it to the spread of imports. Their final result articulated that firms shift their purchases to periods when lower costs can be attainable and that these anticipatory dynamics are true (Khan & Khederlarian 2021).

A potential alternate explanation is that the small decrease in government expenditures was the key factor in the GDP decrease.

There are some potential gaps in data which limit the study of GDP accounting. For example, these accounting principles say nothing about potential causality with latent variables or economic impacts. There is also difficulty in GDP data collections since it can be difficult to only count final goods. Finally, GDP data could be fixed-weighted calculations that can add error as the economy changes and price structures evolve. However, when calculated in a chain-weighted approach to account for economic evolution, there are still struggles with new goods being added.

Methodology

This analysis uses monthly trade data and monthly effective tariff rates for the United States with its largest trading partners. It also uses the same data for the European Union to use as a control. The source for monthly trade data values were the U.S. Census Bureau and Eurostat. The effective tariff rate values were gathered from trusted sources and reflect prior US tariffs and changes as newer tariffs went into effect. This analysis employs a linear regression test with net trade balances and effective tariff rates to analyze the potential correlation between the two.

Results

The data shows a minimal negative relationship between tariff rate and change in trade balances. This means that since the tariffs went into effect, there hasn’t been a significant increase or decrease in U.S. trade balances with main trading partners. The correlation coefficient was 0.0806 which confirms that in the three months since the tariffs went into effect, there weren’t any significant changes in trade balances that were caused by the changes in effective tariff rates. The coefficient of determination is 0.0065, or approximately 0, which meant that any changes that did occur in trade balances were not from the changes in effective tariff rates. Finally, the t-score value equals 0.0977 and indicates that the observed result aligns with the null hypothesis and the difference between the sample data and the population data is not statistically significant.

Discussion

Ultimately, the analysis proves that there is no correlation between the vast increases in effective tariff rates and changes in net trade balances. By contrast, as shown in Table 2, the U.S. trade deficits actually became larger for some countries such as Vietnam or the United Kingdom despite increases in effective tariff rates. Other countries such as China or Italy faced similar decreases in trade deficits despite having large differences in net trade and change in effective tariff rates. Furthermore, some countries were levied with larger tariffs than others, making predicting the change in trade balances harder to anticipate.

A potential explanation for the results of the analysis is the extreme volatility in tariff policy during the study period. Following the implementation of the “Liberation Day” reciprocal tariffs in early April, several countries experienced rapid and significant changes in their tariff rates. For example, China briefly faced tariff levels exceeding 140%, while Brazil was subjected to a 50% tariff following political disputes with the U.S. administration. In addition to these enacted measures, frequent public threats of new tariffs introduced further uncertainty into global trade markets. Simultaneously, reports of partial or full trade agreements with major partners like the EU, China, Japan, and South Korea, led to subsequent reductions in effective tariff rates. This pattern of escalation followed by negotiated de-escalation likely diluted the measurable macroeconomic impact of tariffs, complicating attempts to identify stable relationships between tariff levels and trade or GDP outcomes.

Another potential reason can be shown through the pressures of the markets. Financial Times commentator Robert Armstrong coined the current administration’s trade policies as “TACO Trade”. The acronym refers to some of the administration’s sudden reversals of tariffs. Armstrong coined the term when describing the pattern of placing large tariffs on countries which led to economic panic, shock, and stock market hits. He then explained how later reversals of these tariff policies have led to market comebacks. Additionally, the market uncertainty can be explained as how stocks look like they are trending upward and then stop due to a social media post or claim by the government. It’s possible that many firms and businesses believed that the tariff rate changes wouldn’t be in place long term and thus, no changes were found in the U.S. trade balances.

Implications for Policy and Future Research

Investigating the nuanced economic effects on GDP is key for future policy regarding tariff measures and potential trade deals. As occurred in Q1, there are potential short term distortions in GDP measurement so it’s important to keep these in mind. An area for future research is on the study of tariffs-driven import behavior and with the substitution effect’s prominence in the short and long term. This would provide key insights into how firms react to the government policy and how both parties can better facilitate economic policy.Finally, it’s important to continue to analyze changes in trade balances to see if significant changes will be present with the passing of time and more recent data.

Works Cited

Al Hemzawi, B., & Umutoni, N. (2021). Impact of Exports and Imports on the Economic Growth. MSc. Thesis, Jönköping University. Buckling up for a long ride: chief economists add detail to a downbeat outlook. (2025, May 28). World Economic Forum. https://www.weforum.org/stories/2025/05/wef-chief-economists-uncertainty-global-outlook

Burns, T. (2025, June 26). US economy shrank faster than expected, new data shows. The Hill. https://thehill.com/business/5371005-us-gdp-revised-lower-consumer-spending

Conerly, B. (2025, March 11). Understanding GDP: Why Imports Don’t Actually Reduce Economic Growth. Forbes.https://www.forbes.com/sites/billconerly/2025/03/11/understanding-gdp-why-imports-dont-actually-reduce-economic-growth/

Daco, G. (2025, May). LinkedIn. https://www.linkedin.com/posts/gregorydaco_inflation-fed-fomc-activity-7323335677599793152-c–l/

Freund, C., Pierola, M. D., & Rocha, N. (2021). How Does Trade Respond to Anticipated Tariff Changes? Evidence from NAFTA (Policy Research Working Paper No. 9561). World Bank. Gross Domestic Product, 1st Quarter 2025 (Third Estimate) | U.S. (2025, June 25). Bureau of Economic Analysis. https://www.bea.gov/news/2025/gross-domestic-product-1st-quarter-2025-third-estimate-gdp-industry-and-corporate-profits

Khan, M. Y ., Akhtar, S., & Riaz, S. (2019). Dynamic Relationship Between Imports and Economic Growth in Pakistan. Journal of Economics and Sustainable Development, 10(10), 70–77.

Lemieux, P. (2018, September 6). The St. Louis Fed on Imports and GDP. Econlib. https://www.econlib.org/imports-as-a-drag-on-the-economy/

Mankiw, N. G. (2001). Principles of Economics. Harcourt College Publishers.

Saunders, P.J. (2010). A Time Series Analysis of the Role of Imports in India’s Phenomenal Economic Growth. Indian Journal of Economics and Business, 91, 101-109.

Schonberger, J. (2025, April 30). Shrinking GDP and elevated inflation put Fed in tough spot. Yahoo Finance. https://finance.yahoo.com/news/shrinking-gdp-and-elevated-inflation-put-fed-in-tough-spot-142211609.html

Sekara, D., Dzuibinski, S., & Caldwell, P. (2025, July 16). Morningstar’s Q3 2025 US Market Outlook: Has the Storm Passed, or Are We in the Eye of a Hurricane? Morningstar. https://www.morningstar.com/markets/morningstars-q3-2025-us-market-outlook-has-storm-passed-or-are-we-eye-hurricane

Srikant, K. (2025, May 14). Fact Check: Does an increase in imports directly reduce GDP? Econofact.https://econofact.org/factbrief/fact-check-does-an-increase-in-imports-directly-reduce-gdp

Wiseman, P., & Rugaber, C. (2025, April 29). U.S. economy shrinks 0.3% in first quarter as Trump tradewars disrupt businesses. AP News.https://www.ap.org/news-highlights/spotlights/2025/u-s-economy-shrinks-0-3-in-first-quarter-as-trump-trade-wars-disrupt-businesses/

Wolla, S. A. (2018, September 4). How Do Imports Affect GDP? | St. Louis Fed . Federal Reserve Bank of St. Louis. https://www.stlouisfed.org/publications/page-one-economics/2018/09/04/how-do-imports-affect-gdp

About the author

Ishaan Bafna

Ishaan Bafna is a 12th grade student at Kingswood Oxford School with strong academic and research interests in economics and mathematics. Ishaan actively pursues opportunities that integrates analytical thinking with critical reasoning and problem-solving. Known for his intellectual curiosity and work ethic, Ishaan wishes to pursue a career at the intersection of economics, mathematics and technology.

Outside of the classroom, Ishaan is a leader of his schools Math Team and Mock Trial Team, a lead peer tutor, and a varsity golf athlete. Ishaan has interned with The Hartford Insurance as a Lean Portfolio Management Intern. He is also a National Merit Commended Scholar and a recipient of various awards at his school.

The post The macroeconomic effects of tariffs on GDP and trade balances, through the lens of Q1 2025 GDP change appeared first on Exploratio Journal.

Abstract

Multi-armed bandit algorithms are decision-making algorithms that select between a set of options a specified number of times. Each option, or arm, has a specified, fixed cost and gives a reward which is randomly sampled from a fixed distribution. There exists an optimal action to take, which is choosing the cheapest arm which gives a reward above the predetermined threshold. Choosing a suboptimal arm can lead to cost regret, reward regret, or both. The function of the algorithm is to minimize these regrets. The combinatorial variation of bandits allows for the selection of multiple arms in a single round, whose costs and rewards are averaged. This can potentially lead to lower regrets and higher efficiency. This paper investigates the different parameters of the combinatorial cost-subsidy bandit algorithm and determines its effectiveness. These parameters are tested in several experiments run in VSCode using the MovieLens 25M dataset.

Keywords: Arm, cost, reward, cost regret, reward regret, round, threshold, horizon, logarithmic, combinatorial

Introduction

At its core, a multi-armed cost-subsidy bandit algorithm contains a horizon, a threshold, and a set of arms, each with its own cost and reward distribution (Juneja et al., 2025). The horizon gives the desired number of rounds, and in each round, the algorithm selects an arm (or multiple, in the case of a combinatorial bandit). The threshold gives the desired reward from the selected arm(s). Based on this threshold, an optimal cost is calculated, using the mean rewards of each arm. This optimal cost represents the cheapest arm or combination of arms that will meet the reward threshold. This is then used in the calculation of the cost regret. Each round, if the cost of the selected arm(s) is larger than the optimal cost, the difference is added to the total cost regret. With a horizon of n, a cost during round t of c_t, and an optimal cost of c_*, this can be represented as follows (Juneja et al., 2025):

The reward regret is calculated using the threshold. Each round, if the mean reward of the selected arm(s) is smaller than the threshold, the difference is added to the total reward regret. It is important to note that it is the mean reward that is considered, not the reward achieved in the particular round. With a horizon of n, a threshold of μ_*, and a reward during round t of μ_t, this can be represented as follows (Juneja et al., 2025):

The purpose of the algorithm is to minimize these two quantities. Several different types of bandits achieve this in different ways, but the one used in this paper uses the upper confidence bound (UCB) method. It begins by selecting each arm once and keeping track of the received rewards. Each round after this, it selects the cheapest arm that has reasonable potential to be at or above the threshold. The potential of each arm, also known as its UCB index, is calculated by adding a confidence term to the average reward received from its prior results (Lattimore & Szepesvári, 2020). It is the upper limit of a confidence interval on its mean reward. This term is dependent upon the max difference in potential rewards across the dataset, which in the case of the movielands set, is 4. It is also dependent on the number of times the specified arm has been selected. With a horizon of n, a max difference of Δr, and the number of times a given arm i has been selected by round t of T_i(t), this can be represented as follows (Ganesh, 2019):

The equation contains a constant l, which can be varied, usually in the range 1-4. If multiple arms are being averaged together, each individual upper bound is used. As an arm is selected more times, its interval narrows, as there is more certainty around its expected value. The algorithm learns and improves over the rounds, and it results in the total regret graphs taking a logarithmic shape. 

One of the most beneficial aspects of using a bandit algorithm is that it gives each arm the chances it deserves; no more, no less. If a person was manually making decisions based on rewards, they may neglect arms which initially failed to offer satisfactory returns. However, because each arm has a UCB index which is inversely proportional to the number of times it has been pulled, it can still be chosen even if it has a bad start. If it continues to disappoint, it will be phased out, but if it turns around, it can potentially provide value. 

Combinatorial bandit algorithms allow for the selection of multiple arms in a single round (Xu & Li, 2021). The weighting of each arm can be varied, but this paper will use a uniform average.

The goal of this paper is to test the combinatorial cost-subsidy bandit algorithm. The effect of changing the parameter l will be tested. Additionally, the effectiveness of using multi-arm solutions will also be tested. This will be done by using a specific set of costs whose optimal solution requires multiple arms and seeing how the algorithm improves when it is allowed to use more arms.

Method

The experiment is conducted using the Movielands 25M dataset, which contains ratings of movies of 19 different genres. A cost is assigned to each genre for the experiment. The genres, mean ratings, and costs are shown in the following table:

The first aspect to be tested is the effectiveness of using multi armed solutions. This will be tested by running the algorithm on three settings: one arm allowed, two arms allowed, and three arms allowed. For each setting, it will be run 100 times, and the results will be averaged. For this experiment, the reward threshold is the average rating across the entire set, which is 3.6251287887254984. Using this table of costs, the optimal cost for this threshold using only one arm is 0.71, which is the crime genre. However, the optimal cost goes down when two arms can be selected and averaged. It goes to 0.625, which is the average of the crime and thriller genres. When it is further increased to three selected arms, the optimal cost drops to 0.6, which is the average of the crime, thriller, and romance genres. 0.6 will be used as the optimal cost for all three settings in order to test if the three arm setting can do better than the two and one arm settings. 

Figure 1 shows the graph of the round-by-round cost for each setting. The one-arm setting is blue, the two-arm setting is orange, and the three-arm setting is green. The shaded regions represent the range created by adding and subtracting one standard deviation from the means. The one-arm setting is significantly above the other two, which was expected given that its optimal cost was much higher. The two and three-arm settings are very close together, though there is some difference, especially towards the end. The three-arm setting can find an optimal combination, while the two-arm setting is unable to.

Figure 2 shows the graph of the cumulative incurred cost regret for each setting. All three settings can stay under the optimal cost at the start, as they have more options to choose from due to the confidence bounds being so wide. However, as these bounds narrow and the true value of each arm becomes more apparent, the one-arm and two-arm settings start to incur regret. However, the three-arm setting can stay flat, as it has access to the optimal combination. Overall, the one arm setting incurred an average of 485.6 cost regret by the end, the two arm setting incurred 47.3, and the three arm setting incurred just 2.0. 

Figures 3, 4, and 5 show the graphs of the round-by-round rewards for each setting. Included in each graph is the threshold, represented by the orange line. The single arm setting performs slightly better, while the double and triple are a little behind. This is likely because the single arm setting has fewer overall options to choose from, meaning there are fewer suboptimal solutions to fool it. 

Figure 6 shows the graph of the cumulative incurred reward regret for each setting. Once again, the one-arm setting is blue, the two-arm setting is orange, and the three-arm setting is green. The two-arm and three-arm settings performed extremely similarly, finishing with regrets of 2232.1 and 2216.2, respectively. The one-arm setting was more effective, finishing with a regret of 1338.9. All three settings were able to learn and improve across the horizon, and were able to achieve the desired logarithmic shape. 

Overall, the three-arm setting incurred a reward regret that was 99.2% of the two-arm setting and 165.5% of the one-arm setting. However, it was extremely effective in cost, as it incurred a cost regret that was 4.2% of the two-arm setting and 0.4% of the one-arm setting. 

The second parameter to be tested is the l value. As mentioned in the introduction, the l value is a factor in the calculation of the confidence bounds around each arm’s expected reward. The three values of l that will be tested are 1, 2, and 4. Once again, 100 experiments will be run and averaged with each setting, and one standard deviation above and below the mean will be shaded on the graph.

Figure 7 shows the graph of the cumulative incurred cost regret for each l value. All three are exactly the same, accumulating regret during the initial exploration phase before staying flat the rest of the way. This was expected, as these experiments were run using the three-arm setting, which is extremely effective in minimizing cost regret.

Figure 8 shows the graph of the cumulative incurred reward regret for each l value. This graph is more varied. The blue represents an l value of 1, the orange represents an l value of 2, and the green represents an l value of 4. The results show that all three l settings are able to learn and improve, but an l value of 1 performs the best. In general, a lower l value is better, as larger values give too wide estimates for the rewards of the arms. However, if it goes too low, then an optimal arm may not get chosen after a few pulls if it gives suboptimal results. 

Conclusion

The results of this paper show that the combinatorial bandit algorithm is extremely cost-effective in decision making. While it gives up a little bit in terms of rewards, it makes up for it by reducing the costs. 

The algorithm can be utilized in any situation with decisions involving costs and rewards. For example, in the context of the dataset used in this paper, the algorithm can be used to choose movies to show to audiences. Each arm can be a genre, and the playing of a movie of the genre represents the pulling of the arm. The rewards are the reviews, and using these reviews, the algorithm can decide which movies to show. It can also be used in an educational setting. Each arm can be a specific lesson plan or style, and rewards can be test scores, student feedback, or another evaluation metric. Using the evaluations, the algorithm can decide the most effective lessons to teach.

A further direction for exploration could be the testing of an algorithm that can pull multiple arms with different weights. Instead of simply averaging them out evenly, it could take more from one and less from another. This could potentially allow for even better combinations of arms.

Bibliography

Lattimore, Tor and Szepesvári, Csaba. “Bandit Algorithms.” Cambridge University Press, 2020

Juneja, Ishank, et al. “Pairwise Elimination with Instance-Dependent Guarantees for Bandits with Cost Subsidy.” International Conference on Learning Representations, 2025

Ganesh, A.J. “Multi-armed Bandits.” University of Bristol, 2019

Xu, Haike and Li, Jian. “Simple Combinatorial Algorithms for Combinatorial Bandits: Corruptions and Approximations.” Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, 2021

About the author

Aman Kumar

Aman is a high school senior studying at Wayland High School in Massachusetts. He is interested in math, computer science, and data science. Outside of academics, he plays the clarinet, and is a big football fan, supporting the New England Patriots.

The post Combinatorial Cost-Subsidy Bandit Algorithms appeared first on Exploratio Journal.

Abstract

This study investigates the impact of technology use and urbanization on cultural practices in Brazilian communities, with a focus on family routines, media consumption, and community engagement. Using a survey of 35 respondents from rural and semi-urban towns, data were collected on cultural habits during primary school and adulthood, technology usage, and urbanization-related variables such as commuting and town migration. Composite scales were created to measure cultural engagement and technology use. Results indicate a modest decline in traditional cultural practices, such as sit-down family dinners, while engagement with global media (European and U.S. films, music, and digital platforms) has increased significantly. Urbanization shows strong associations with decreased family meal frequency and community interaction, while technology correlates moderately with increased media consumption and online engagement. These findings highlight the nuanced role of modernization in reshaping cultural life: while traditional practices persist, they are increasingly supplemented or substituted by digital and urban influences. The study underscores the importance of considering both generational and technological factors when analyzing cultural change in contemporary Brazil.

1. Introduction

Latin American culture is quite diverse, complex, and representative of the region’s rich history. It is an amalgamation of indigenous, African, European, and Asian cultures. Of all the countries in Latin America, however, there is only one that does not have a Spanish colonial legacy. Brazil is now the only Latin American country where Portuguese is the official spoken language. It has long been known as a family-based culture, in which Brazilians widely celebrate Carnival, samba, and the national soccer team. Culture is a collection of the beliefs, practices, symbols, and rules of a people, which can be spread verbally or through practice. Culture is ever-evolving, shaped by rapid changes in economic, social, and political environments. Given these shifts, it is likely that new beliefs have emerged as part of this dynamic culture in recent years. These transformations raise an important question: to what extent have modern influences affected traditional cultural practices? Traditional practices include the impact of family-centered values on media representations and the emphasis on culinary traditions and festive holidays. Even though these traditional cultural norms continue to predominate in some parts of the country, globalization has introduced new influences. Consumerist culture, international mass media, and rapid advances in digital technology have altered consumption patterns, entertainment patterns, and even language use patterns, particularly among the younger population (Graham). Urbanization has also changed the way of life, blending traditional lifestyles with modern urban life and restructuring work, social, and cultural norms (Aldrich). As a result, traditional practices are being modified and redefined rapidly. While scholars generally agree that cultural practices are undergoing significant transformations, the extent of these changes and their relative significance—especially in Latin America—remain poorly understood (Caldeira). This study investigates how rapid economic, social, and political changes in Brazil have affected traditional cultural practices through ethnographic survey data.

Though modern consumer culture has certainly penetrated the economic markets of Brazil, introducing international brands, fast food, and digital technology, the exact extent of their influence on daily life—be it diet, leisure activities, or language—is unclear. Consumer culture refers to the lifestyle and social practices centered around the consumption of goods and services, heavily influenced by capitalism, advertising, and mass media in modern societies. Urbanization, particularly in São Paulo, Rio de Janeiro, and Brasília, has catalyzed a mixture of traditional Brazilian ways with cosmopolitan, contemporary lifestyles. At the same time, traditional cultural events such as Carnival continue to celebrate the classic parade, but incorporate international music, fashion, and lighting and stage design technology advances (Abreu). While Carnival is not native to the Brazilian indigenous population, it remains a popular celebration that is considered culturally important. Social media and the World Wide Web are connecting Brazilian society to the rest of the world more than ever before, allowing Brazilians to experience multicultural exchanges where they are both shaping and being shaped by global trends. This essay will examine the various degrees to which these forces of globalization, urbanization, and technology are influencing Brazilian culture in terms of lifestyle changes and time-honored practices. I will attempt to shed light on the extent to which these cultural changes are taking place, clarify their causes, and explore whether there are other hidden forces at play.

In this study, I will assess cultural change through four key concepts and their impact on rural Brazilian societies. The first concept evaluated is Cultural Practices, which include traditional norms, rituals, and social routines. To measure this, I will examine respondents’ frequency of having family meals, attending cultural events in their area, and engaging in religious or community activities. These behaviors will be coded into a scale to capture the strength and frequency of such behaviors, with survey questions asking how frequently participants have dinner with their families, attend cultural events, or visit local religious services or community events.

The second critical construct is technology use, reflecting the extent to which people use modern-day technologies, primarily in the context of media consumption, social media use, and access to information digitally. This will be measured via a “Technology Use” scale, for which participants’ responses related to the level of their media consumption—such as TV watching, browsing on social media, or reading news online—will be evaluated. Through these responses, I intend to measure the degree of technological embeddedness in participants’ lives and its potential influence on cultural habits.

The third concept is the effect of urbanization on individuals and the scale of migration from rural towns to urban cities. This will be reflected through analysis of factors such as how often respondents travel to cities, respondents’ migration experiences (e.g., whether they have migrated from villages to cities), and historical patterns of population density, which can serve as an urbanization proxy. It is essential to determine the extent to which migration and experience in urban settings could impact cultural practices and the adoption of technology in rural communities.

Lastly, I will analyze the effect of modern globalization on indigenous culture, specifically regarding the impact of media and technology on individuals’ lifestyle choices and values. Lifestyle refers to the habits, attitudes, and economic level that together constitute the mode of living of an individual or group. This will be quantified through respondents’ exposure to Western media (e.g., American movies, TV shows, or music) and activities most commonly associated with Western lifestyles (e.g., reading global media materials or keeping up with Western fashion). These factors will ensure an understanding of how Western cultural values and globalization can affect traditional cultural patterns. Globalization refers to the process of increasing interdependence and integration of the world’s economies, cultures, and populations. On the other hand, urbanization is the process of people moving from rural areas to cities and towns, leading to population growth and the expansion of urban centers.

The four key concepts—cultural practices, technology use, urbanization, and modern globalization in rural Brazilian societies—will help identify patterns in societal changes driven by these dynamic external influences. The survey conducted will investigate different correlation patterns between overarching variables to begin determining whether they impact cultural changes currently occurring.

2. Background

Brazil’s current cultural ecosystem is shaped by a complex interplay of urbanization, modern global influences from around the world, and technological advancements. While it is commonly agreed that such forces influence cultural practices and social norms, the question remains: specifically, how have they affected Brazilian culture, and among these forces, which are the most dominant? This is a significant question because even though cultural change is recognized, the exact mechanisms driving this change—the relative contributions of globalization, urbanization, and technology, in particular—are not yet adequately explored (Aldrich, Goldman, and Lipman 2004). Comprehending these nuances would help illuminate the magnitude of these changes and their impact on shaping the future of an important country such as Brazil.

With the early dominance of a slave and plantation economy and the 19th-century emergence of industrialization, Brazil has faced many shifts in its identity. Like the rest of Latin America, Brazil has been significantly influenced by global powers, such as the United States and Asia. Globalization, through media, consumerism, and a host of ideals, has contributed to a sense of cultural dissonance in Brazil (Madukwe and Madukwe). Notwithstanding, some developments, both present and future, influence Brazilian society in unique ways. For instance, multinational companies have promoted a more consumerist economy and materialistic culture, with both positive and negative consequences. This, in turn, has affected market dynamics and social interactions (Graham). The adoption of fast foods, fashion, and technology has brought drastic changes to society. Although these influences have contributed to the erosion of Brazil’s ethnic and indigenous cultures, they also reflect broader cultural adaptation. The breakdown of family structures, religious influence, and the impact of other cultures brought by globalization has been documented by numerous scholars (Caldeira). In Brazil, the weakening of extended family connections and the rise of individualism have profoundly affected lives, resulting in diminished communal relations, a shift toward more autonomous lifestyles, and the transformation of social support mechanisms (Hutchinson). This shift reduces the role of the extended family as a source of social support, encouraging individualism and reshaping community-based welfare.

Western legal systems, particularly in modern contexts, emphasize individual rights, whereas many traditional Western cultures (e.g., pre-industrial Europe) prioritize communal solidarity. However, the interaction of law and culture remains complex. Young generations are most vulnerable to cultural dissonance as they navigate global trends and traditions (Sibani). This manifests in their increased consumption of foreign films, street foods, and fashion. Older generations, conversely, may struggle to adapt to rapid cultural changes as they clash with traditional practices. Although youth are usually at the forefront of adopting new customs, they may also experience negative effects from losing valuable social elements native to their culture while embracing a more globalized lifestyle.

Urbanization has also transformed Brazilian culture, particularly in urban hubs such as Rio de Janeiro and São Paulo, where rural-influenced culture mixes with urban ways of life (Chauvin). Migration from rural towns to cities has resulted in both cultural assimilation and conflict between rural customs and urban norms. The acceleration of urbanism is accompanied by changes in family structures. New nuclear family types have emerged due to increased mobility and independence (Liu). Traditional dependence on extended families, which historically provided both economic and emotional support, has decreased, leading to a more individualized family system. The heightened emphasis on nuclear families, particularly in urban areas, can contribute to social disintegration, as individuals rely less on extended families and more on nuclear families for support (Lai).

Urban lifestyles and attitudes continue to challenge conventional family values and communal affiliations, promoting ideologies centered on individualism and autonomy. Beyond socio-cultural effects, urbanization is associated with economic changes, such as a shift from pre-market to market economies (Villa). With Brazil’s transition to urban settlements, reliance on barter exchange or informal social support systems is diminishing, replaced by material culture. The transition to a cash economy and emphasis on personal wealth promotes competitiveness and social differentiation.

Technological innovation, particularly through social media and digital platforms, has had a significant influence on Brazilian cultural practices. Such innovations have introduced global culture into Brazilian households, accelerating the alignment of local practices with Western norms (Chung). Cross-cultural exchange has occurred for centuries via books, television, and film. However, digital media makes this exchange more direct, interactive, and immediate, with platforms such as Instagram, Twitter, and YouTube enabling real-time participation. Brazil both absorbs and projects global culture in ways not possible with older media (Stuenkel). Traditional festivities, like Carnival, have adapted to incorporate contemporary technologies such as advanced lighting and stage design (Graham). Digital communication devices have also altered social composition, particularly among younger generations, changing methods of communication, socialization, and relationship formation. Mobile technologies and the internet have revolutionized socialization and idea sharing, sometimes supplementing or substituting face-to-face interaction (Lai 2016). While these platforms provide global connectivity, they do not entirely replace in-person interactions but offer alternatives, facilitating rapid cultural exchanges. These changes have introduced new cultural patterns, including increased use of English in daily communication and alternative modes of cultural learning through videos and other digital media (Liu).

Tension persists between maintaining traditional Brazilian values and adopting globalized norms. Traditional practices remain valued but are increasingly modified or replaced by contemporary consumer practices and individual lifestyles. Younger generations, especially in urban centers, are more receptive to Western cultural norms, while older generations retain traditional values. This generational gap permeates family life. Despite globalization, some Brazilian cultural aspects, such as Carnival, demonstrate resilience.

Globalization is not merely the transfer of ideas, technologies, and goods across borders but represents a fundamental transformation of societies through the diffusion of global economic, cultural, and political structures (Sibani). Western values are often dominant, influencing not only material life but also local beliefs and traditions. The imposition of global political, economic, and cultural norms, particularly from the United States and Europe, has profoundly reshaped societies, frequently at the expense of local cultures and identities. One consequence has been the reconfiguration of traditional religions, belief systems, and social structures, which must adapt to remain relevant as new routines emerge. Although Brazil’s culture has always been dynamic, the pace of change induced by globalization, technology, and urbanization warrants closer examination. In Brazil, globalization is particularly evident in the shift toward individualism, materialism, and a consumerist lifestyle. The introduction of global media and consumer culture has significantly impacted daily life, particularly in urban areas such as São Paulo and Rio de Janeiro. Exposure to global ideologies, fashion, and technology has contributed to a decline in historically prioritized collective values, including familial reliance and communal unity. International brands, fast foods, and global attire have shaped social spaces and identity formation (Madukwe and Madukwe).

Social scientific literature suggests that technology, particularly social media and digital platforms, plays a larger role in promoting cultural change in Brazil than urbanization or Western influences. While urbanization and globalization contribute to cultural change, technology accelerates it by making global trends accessible and influential across all aspects of society, from communication to fashion. Digital technologies and social media enable the rapid dissemination of cultural practices and norms, altering social interactions and self-representation.

Younger generations are especially adept at adopting global influences, blending them with local practices, and modifying traditions such as Carnival or family norms. Technology fosters individualism, challenging Brazil’s traditionally communal values. Research indicates that young Brazilians, in particular, align with global digital cultures and adopt individualistic identities. Older age groups are more likely to adhere to traditional practices, and scholars suggest that this generational divide significantly shapes Brazilian culture. Despite these changes, Brazilian culture demonstrates resilience, balancing the adoption of modern technology with the preservation of traditional values. The ongoing negotiation between technological adoption and cultural preservation remains a critical area of research in understanding Brazil’s evolving cultural identity.

In conclusion, Brazilian culture has undeniably transformed in response to these forces, yet it retains remarkable resilience. The central challenge is reconciling modernizing influences, such as technology and globalization, with traditional cultural practices. As Brazil engages with globalization, it must navigate adopting modern innovations without sacrificing its cultural essence. Understanding how Brazilian culture adapts and evolves amid technological progress, while preserving its core customs, is essential for grasping the future trajectory of this vibrant society.

3. Data and Methods

To answer the main research question in this study, I conducted a survey to gather original data from Brazilian citizens living in rural towns and from diverse backgrounds. The survey aimed to collect information regarding cultural practices, social habits, technology use, and demographics. It was also designed to compare participants’ behaviors during their primary school years with their behaviors today. The survey was distributed online using a convenience sampling method and was completed by 35 participants. Respondents ranged in age from 18 to over 50 years old. The survey was available in both English and Portuguese to facilitate completion by participants with different linguistic backgrounds, including Brazilian and Brazilian-American individuals. All responses were collected anonymously. As a convenience survey, the sample was not randomly collected, which constitutes a potential limitation affecting the generalizability of the results.

The questionnaire was designed to capture information on cultural practices, social habits, technology use, and demographics, with a focus on comparing participants’ early primary school habits to their current behaviors. Questions addressed social habits, including family meals, media consumption, news habits, music preferences, attendance at cultural events, community engagement, religious practices, and technology use. The survey was divided into three sections: cultural practices, technology use, and demographics. All questions were formatted as multiple-choice items with pre-set response options. Completion time was approximately 10 minutes.

Demographic questions covered participants’ place of origin, gender, age, education, occupation, and marital status. Respondents were from various Brazilian towns, including Goiás, São Paulo, and Rio de Janeiro, as well as a small group of recent Brazilian immigrants to the United States, primarily in Georgia and Florida, who had immigrated within the last five years. The responses of immigrants were included in the analysis due to their frequent returns to Brazil and their careful understanding of the survey’s objectives. Regarding education, 33.3% of participants held a bachelor’s degree, and 22.2% worked in business or service occupations. The gender split was 60% male and 40% female, and most participants (55.2%) were married.

To assess the impact of technology use on cultural changes, I created scales for key behaviors related to cultural practices and technology use. This involved combining multiple questions measuring similar behaviors into composite variables. For example, responses on the frequency of family dinners, participation in cultural activities, and religious observance were aggregated into a “Cultural Engagement” scale. Similarly, responses on media use (movies, music, news) and social media activity were combined into a “Technology Use” scale. Urbanization was incorporated using variables such as proximity to large cities and migration history (e.g., moving from rural to urban locations). Population density over time was also used as a proxy for urbanization.

Once the scales were constructed, two-way tables were used to analyze data and examine relationships between technology usage and cultural behaviors to determine whether increased exposure to global media or technology influenced traditional cultural habits, such as family meal frequency or religious participation. The dependent measures focused on cultural change indicators, including frequency of family dinners, media consumption, and engagement in local community routines. Correlations between technology use, globalization, and urbanization scales with these dependent measures were analyzed. Descriptive statistics provided response frequencies for all variables, while two-way tables allowed assessment of relationships between technology use (e.g., frequency of social media use or online news consumption), cultural practices, and globalization. Additional variables such as education, age, and migration experience were examined to determine whether they shaped these relationships. This approach allowed identification of patterns indicating how technology, urbanization, or exposure to global cultures affected current cultural practices.

Several limitations of the study must be noted. Convenience sampling, small sample size, and the self-reporting nature of the survey constrain the generalizability of the findings. The non-random sample may not fully represent the broader Brazilian population, and the sample size of 35 limits statistical power. Additionally, self-reported data may introduce bias, as participants could respond inconsistently or inaccurately.

Analysis of the survey findings revealed trends in technology adoption, urbanization, and cultural practice interrelations in Brazilian rural areas. However, the extent of observed cultural change appeared less pronounced than anticipated. This may be a result of the limited sample and may not reflect the full scale of cultural change experienced in more isolated or traditional contexts. The sample, which included respondents already exposed to modern technology and urban influences to varying degrees, may underrepresent the magnitude of cultural shifts. All participants were raised in rural areas but had experienced modernization, either through media exposure or urban migration, potentially diluting the observable impact of cultural change.

This distinction is important when considered in the context of the theoretical model. Individuals raised in small, traditional rural villages—similar to those existing in the 1960s—would likely show more pronounced cultural changes due to globalization, technology, and urbanization. By contrast, participants in this sample, who were already influenced by these factors during childhood, exhibited only modest changes. Thus, the muted cultural change observed is likely due to gradual exposure rather than the absence of change.

The relatively moderate findings should be interpreted cautiously. While the data provides insight into the effects of technology and urbanization on cultural practices, the scope of change may be less dramatic than in populations with more traditional lifestyles. Future studies involving participants from isolated communities or longitudinal studies tracking individuals over time would provide stronger evidence of the magnitude of cultural changes driven by globalization and technological advancement. Despite these limitations, the survey data offers valuable preliminary insights into technology use and cultural practices among rural Brazilian communities. This exploratory research can serve as a foundation for more representative studies and provide informative background for understanding trends within these populations.

4. Results: Descriptive Statistics

To begin examining cultural change, particularly regarding family dinners, 80 percent of respondents reported having a sit-down family dinner each day during their primary school years. This figure decreases slightly in adulthood, with 73.3 percent now having family dinners regularly, either daily or a few times per week. However, this shift may not directly indicate a cultural transformation. It could instead reflect life-stage factors. As individuals grow older, increased responsibilities—such as work, extracurricular activities, and travel—often interfere with the consistency of shared family meals. Therefore, while there is a measurable decline, it is crucial to consider whether this change results from cultural globalization or simply natural changes in daily routines.

Concerning media consumption, which reflects aspects of globalization, noticeable changes are evident. During their primary school years, 46.7 percent of respondents watched European or American shows a few times per week. Today, 50 percent watch them regularly, with 36.7 percent doing so daily. Music consumption from the same regions shows a similar pattern. While 40 percent listened daily in childhood, 53.3 percent now do so either daily or several times per week. These trends suggest a deepening integration of Western cultural products into daily Brazilian life, likely facilitated by increased access to global streaming platforms and digital media.

However, international news consumption requires more critical interpretation. Only 20 percent of respondents engaged with news from the United States and Europe daily during childhood. This figure increases to 40 percent in adulthood. While this appears to represent a meaningful shift, it is important to recognize that young children typically consume limited news, regardless of cultural context. To strengthen this analysis, comparing this with Brazilian news consumption during the same early period would help clarify whether the difference reflects content preference or age-related behavior.

In terms of traditional cultural practices, consumption of Brazilian food has remained consistently high. About 93.3 percent reported eating traditional meals daily during primary school, and 80 percent continue to do so today. Religious participation, however, shows a notable decline. During their early years, 43.3 percent of respondents attended services regularly. Today, 20 percent report that they rarely or never participate in religious activities. This shift may indicate a move toward secularization, though further demographic analysis would help clarify whether age or gender plays a role.

Perceptions of Carnival have also shifted over time. While 70 percent of respondents considered it important in childhood, only 13.3 percent now rate it as “very important.” Sixty percent still consider it “important,” but the overall decrease in enthusiasm may reflect generational changes in values or evolving regional traditions. Breaking this data down by demographic variables such as location, age, or exposure to global events could help identify factors influencing this change.

Community interaction offers additional insight. During primary school, 73.3 percent of respondents regularly interacted with neighbors and local communities. This strong sense of engagement likely stems from the rural or close-knit environments in which many participants grew up. Today, only 31 percent report daily engagement with their communities. This reduction may be attributed to urbanization and lifestyle changes rather than globalization directly. The shift from face-to-face interactions to digital connections may also play a role, particularly as individuals move to more urbanized and individualistic settings. In this case, community interaction functions as a dependent variable, reflecting cultural change within local environments.

Perceptions of migration trends further illuminate social and cultural transformation. For example, 13.3 percent of respondents strongly agree that people from their hometowns often move to larger cities such as Brasília or São Paulo. Additionally, 44.8 percent estimate that about half of their current community members are not native to the area. These migration patterns suggest increasing urbanization and mobility, which can shift local cultural identities and increase exposure to global cultural practices. Exploring whether individuals who have migrated themselves show higher engagement with globalized media or technology would add depth to this analysis.

Technology use and social media engagement provide some of the most striking evidence of globalization. Figure 1 illustrates the distribution of responses related to technology use, urbanization, and Western cultural influence, offering visual support for the trends discussed. About 43.3 percent of respondents agree that technology has significantly impacted their communities. Daily social media use is widespread, with 40 percent spending two to three hours per day on these platforms and another 40 percent spending five or more hours daily. Furthermore, 69 percent obtain their news primarily from the internet, and nearly all respondents (93.1 percent) conduct banking online or via mobile applications. These behaviors highlight how global technology platforms have become deeply embedded in daily life. To better understand the scope of this influence, breaking down the data by age and gender would clarify whether specific groups are more affected by or engaged with these digital tools.

5. Results: Associations Between Globalization, Urbanization, and Technology Use with Cultural Change.

In order to determine relationships between variables, I compiled three pairs—MoviesNow vs. MoviesThen, FamDinnerNow vs. FamDinnerThen, and NewNeighbors vs. OldNeighbors—representing major shifts in behaviors over time. These dependent variables reflect changes in media consumption, family routines, and community engagement, respectively. Their variation over time provides insight into how broader social forces, particularly urbanization and technology use, might be influencing cultural practices.

MoviesNow vs. MoviesThen examines the frequency of watching European and American films or television shows now compared to during primary school. While the majority of responses initially showed stability, indicated by zero values, there was a noticeable upward trend in later responses marked by increasing “1” scores. This trend suggests a shift toward more frequent engagement with global media content. One possible explanation for this change is the proliferation of digital streaming platforms such as Netflix, YouTube, and other on-demand services, which provide wide access to international films and television. These services reduce barriers such as geography, cost, and scheduling, making global media more accessible than ever. While the correlation between technology use and this cultural change was moderate (0.51), it is important to consider the mechanisms by which technology facilitates cultural exposure. Personalized content recommendations and algorithmic tailoring foster deeper engagement with diverse media, which may gradually influence preferences and behaviors. Although causation cannot be confirmed without further statistical testing, these factors provide a plausible explanation for the observed association.

FamDinnerNow vs. FamDinnerThen shows a more complex pattern, with a mixture of zero and negative values. Negative values suggest a decrease in the frequency or enthusiasm of family dinners over time. This shift may be strongly associated with urbanization and the increasing consumption of technology. Urbanization is typically associated with fast-paced lifestyles, longer commutes, and changing family structures, particularly in urban environments where dual-income households and staggered work schedules are more common. These structural changes can disrupt regular family mealtimes. In addition, digital distractions such as smartphones and tablets can interrupt shared meals or reduce their perceived value. Family members may choose to eat alone while watching individual screens or engaging in online activities. The high correlation between urbanization and cultural change (0.84) supports the idea that these lifestyle factors are influencing communal routines. While statistical significance has not been established in this analysis, the observed association aligns with well-documented trends in urban life and digital media consumption.

NewNeighbors vs. OldNeighbors reveals a significant downward trend in neighborhood social interaction. Negative scores dominate the responses, indicating a decline in engagement with local communities over time. This trend may be attributed to both urbanization and increased technology use. As more individuals move into densely populated, transient city environments, community ties often weaken. People may relocate more frequently and feel less incentive or opportunity to build lasting relationships with neighbors. Technology also contributes by providing alternative forms of socialization, such as social media, online forums, and messaging apps, which often replace in-person interaction. These tools allow people to maintain connections beyond their immediate geographic area, further reducing the importance of local social networks. The strong correlation with urbanization (0.84) and moderate correlation with technology use (0.51) reinforce the possibility that these broader structural and technological changes are influencing how individuals relate to their local communities. However, the absence of statistical significance testing means these results should be interpreted cautiously.

The strongest associations are seen with urbanization, particularly regarding declining family meals and reduced neighborhood interaction. Technology use is moderately correlated and appears especially influential in shaping media consumption and changing modes of social engagement. Globalization shows a much weaker correlation in this dataset, suggesting that while global influences are present, they may not be as directly or measurably impactful in this specific sample. The influence of globalization may also be more subtle or indirect compared to the immediate lifestyle impacts of urban living and daily digital technology use.

Taken together, the changes observed in MoviesNow vs. MoviesThen, FamDinnerNow vs. FamDinnerThen, and NewNeighbors vs. OldNeighbors indicate that technology use and urbanization are the most significant external factors associated with changes in cultural practice. These shifts in entertainment consumption, family routines, and neighborhood interaction illustrate how larger structural and technological developments are influencing cultural values and social behaviors. While the data reveals strong associations, it is important to note that correlation does not imply causation. Furthermore, this study did not perform statistical significance testing, so while the trends appear meaningful, they cannot yet be considered definitive. Future research should apply inferential statistical methods to test whether these observed relationships hold across broader populations and to rule out the possibility of confounding variables.

6. Discussion and Conclusion

This research highlights the complex ways in which urbanization and technological advancement are associated with changes in Brazilian cultural and social life. While the survey data shows correlations between these external forces and shifts in cultural practice—particularly in areas such as family dining and neighborhood interactions—it is important to recognize the limitations of the methodology. Correlation does not imply causation, and the self-reported, retrospective nature of the survey cannot definitively explain the causes of these cultural shifts. However, when interpreted alongside existing literature and broader global trends, the findings offer insight into possible mechanisms through which urbanization and technology may be reshaping cultural norms.

The decline in family meal frequency and the weakening of neighborhood social ties observed in the data are patterns consistent with existing research on the social effects of urban living. Scholars have long documented that as societies urbanize, traditional communal structures tend to erode due to increased geographic mobility, demanding work schedules, and spatial reorganization of daily life. In Brazil, the rapid expansion of cities has introduced pressures that disrupt conventional family routines and reduce the frequency of informal community interactions. The strong correlation (0.84) found between urbanization and cultural practice change in this study supports these findings, suggesting that people living in urban environments may be more likely to adopt lifestyles that prioritize individual autonomy over collective ritual.

Technology use also appears to play a significant role in reshaping cultural practices, albeit with a more moderate correlation (0.51). As digital devices and internet access become increasingly integrated into everyday life, patterns of media consumption and social engagement shift accordingly. The rise of social media platforms, online entertainment, and personalized digital environments can create a sense of virtual connection that may, paradoxically, reduce face-to-face interactions within the immediate community. This aligns with the observed decline in neighborly interaction and the substitution of communal activities with individualized digital experiences. Although the survey does not directly measure time spent on digital platforms or social network engagement, these trends are widely reported in other studies and offer a reasonable explanation for the patterns seen in the data.

It is important to emphasize that the observed associations should be interpreted with caution. The study does not include tests of statistical significance, and the retrospective nature of the survey introduces possible memory bias or inaccuracies in self-reporting. In addition, the sample may not represent the full diversity of Brazilian society, particularly rural communities or regions with lower technological penetration. Thus, while the findings suggest important patterns, they should be considered exploratory and preliminary rather than conclusive.

Nonetheless, these results contribute to a growing body of work examining how globalization, urbanization, and technology intersect to transform cultural life. For example, although urbanization and technology appear to be contributing to the erosion of traditional practices, they also present opportunities for cultural reinvention. Technological tools can support cultural preservation through digital archiving, online storytelling, or virtual celebrations of national events like Carnival, enabling new forms of participation and innovation. This dual nature—where culture is both challenged and renewed—emphasizes the need for adaptive cultural frameworks that can absorb change without losing core values.

To build on this research, future studies should investigate generational patterns in cultural adaptation, especially how different age groups experience and negotiate technological and urban pressures. Comparative studies across urban and rural contexts within Brazil could reveal the extent to which these dynamics vary based on infrastructure, community size, or economic development. Moreover, qualitative methods such as interviews or ethnographies could help uncover the lived experiences behind these trends, providing richer context that surveys alone cannot capture.

Beyond Brazil, the patterns observed here are relevant to many societies undergoing rapid urban growth and digital transformation. In global terms, the tension between tradition and modernity is increasingly shaping policy debates about cultural preservation, social cohesion, and identity. Understanding how these macro-level forces influence daily life can inform initiatives aimed at promoting cultural resilience. Resilience, in this context, refers not only to the preservation of traditions but also to the capacity to adapt cultural expressions in ways that remain meaningful amid social change.

In conclusion, while the findings of this study suggest that urbanization and technology use are strongly associated with cultural changes in Brazilian society, further evidence is needed to establish causation and clarify the mechanisms at play. Still, the observed trends reflect broader transformations occurring in contemporary life, and they highlight the urgent need to understand how societies can adapt to change without losing their cultural coherence. These insights are especially vital for shaping cultural policies that strike a balance between embracing innovation and safeguarding community bonds.

References

Abreu, M., & Brasil, E. (2020, August 27). Toward a history of Carnival. Oxford Research Encyclopedia of Latin American History. https://oxfordre.com/latinamericanhistory/view/10.1093/acrefore/9780199366439.001.0001/acre fore-9780199366439-e-820

Aldrich, B. W., Goldman, F. P., & Lipman, A. (2004). Urbanization and familism. International Journal of Sociology of the Family. https://www.jstor.org/stable/23027036

Caldeira, T. P. R. (2015). Social movements, cultural production, and protests: São Paulo’s shifting political landscape. Current Anthropology, 56(S11), S126–S136. https://doi.org/10.1086/681927

Chauvin, J. P., Glaeser, E., Ma, Y ., & Tobio, K. (2016). What is different about urbanization in rich and poor countries? Cities in Brazil, China, India and the United States. NBER Working Paper No. 22002. National Bureau of Economic Research. https://doi.org/10.3386/w22002

Dessen, M. A., & Torres, C. V . (2011). Family and socialization factors in Brazil: An overview. Online Readings in Psychology and Culture, 6(3). https://scholarworks.gvsu.edu/orpc/vol6/iss3/2/

Duarte, R. (2022). The culture industry in Brazil: From the “classic” model to the digital media. Brazilian Research and Studies Journal, 1(1). https://journal.bras-center.com/bras-j/article/view/3

Fix, M., & Arantes, P. F. (2021). On urban studies in Brazil: The favela, uneven urbanisation and beyond. Urban Studies, 58(4), 1–24. https://doi.org/10.1177/0042098021993360

Gomes, S., Pereira, G. M. L., & Chagas, C. L. (2025). The impact of globalization on changes in cultural aspects. New Science Journal of Social Science, 2(2), 1–10. https://periodicos.newsciencepubl.com/arace/article/download/4299/6291/18045

Hauge, G. M. H., & Magnusson, M. T. (2011). Globalization in Brazil: How has globalization affected the economic, political and social conditions in Brazil? (Master’s thesis, Copenhagen Business School). https://research.cbs.dk/files/58430566/gina_marie_helland_hauge_og_marie_therese_magnusson .pdf

Kelly, J. (2020). The city sprouted: The rise of Brasília. Columbia University Academic Commons. https://academiccommons.columbia.edu/doi/10.7916/d8-4s5w-ry93/download

La Rosa, T. (2013). Cultural behavior in post-urbanized Brazil: The cordial man (Master’s thesis, Portland State University). https://pdxscholar.library.pdx.edu/cgi/viewcontent.cgi?article=1666&context=open_access_etds

Maldonado-Mariscal, K. (2020). Social change in Brazil through innovations and educational change. SAGE Open, 10(2), 1–10. https://doi.org/10.1177/2158244020916332

Martine, G. (2010). Brazil’s early urban transition: What can it teach urbanizing countries? International Institute for Environment and Development. https://www.iied.org/sites/default/files/pdfs/migrate/10585IIED.pdf

Perlman, J. E. (2007). Globalization and the urban poor. EconStor. https://www.econstor.eu/bitstream/10419/63592/1/558988636.pdf

Pardo, I., Willaarts, B. A., & De La Mora, G. (2012). Urbanization, socio-economic changes and population growth in Brazil: Dietary shifts and environmental implications. International Union for the Scientific Study of Population. https://iussp.org/sites/default/files/event_call_for_papers/IUSSP%20Willaarts%2C%20Pardo%2 0y%20de%20la%20Mora.pdf

Regis, A. M. de S., Gomes, S. C., & Georges, M. R. R. (2025). The impact of digitalization and technological human capital on the performance of the Brazilian PYME: An empirical study. Journal of Technology Management and Innovation, 20(1), 74–85. https://doi.org/10.4067/S0718-27242025000100074

Salas-Guerra, C. R. (2021). Skills-based on technological knowledge in the digital economy activity. arXiv. https://arxiv.org/abs/2102.01711

Seto, K. S. (2025). Emerging collective actions in Brazil’s tech community. ScienceOpen. https://www.scienceopen.com/hosted-document?doi=10.13169%2Fworkorgalaboglob.19.1.0050

Silva, D. M. (2022). Digital activism and democratic culture: Can digital political participation strengthen democratic culture in São Paulo? Brazilian Political Science Review, 16(1). https://doi.org/10.1590/1981-3821202200010004

Stuenkel, O. (2011). Identity and the concept of the West: The case of Brazil and India. Revista Brasileira de Política Internacional, 54(1), 178–195. https://doi.org/10.1590/s0034-73292011000100011

Wimmer, A., Bonikowski, B., Crabtree, C., Fu, Z., Golder, M., & Tsutsui, K. (2024). Geo-political rivalry and anti-immigrant sentiment: A conjoint experiment in 22 countries. American Political Science Review, 1–18. https://doi.org/10.1017/s0003055424000753

“Vai-Vai: Carnival is fashion, history and resistance.” (2024). Journal of Textile Engineering & Fashion Technology, 5(1). https://medcraveonline.com/JTEFT/vai-vai-carnival-is-fashion-history-and-resistance.html

“Carnival crowds.” (2013). SAGE Open, 3(1), 1–12. https://doi.org/10.1177/2158244013475524

“Carnivals, rogues, and heroes: An interpretation of the Brazilian Carnival.” (2024). Journal of Latin American Studies. https://doi.org/10.1017/S0022216X23000012 Appendix Data collected and visualized by the author through a personal survey

About the author

Lilya Elchahal

Lilya Elchahal is a junior at Westminster in Atlanta, where she has been a dedicated student and community leader since transitioning from the Atlanta International School in sixth grade. Raised in Atlanta, Lilya developed a global perspective early on, fostering a deep love for language and cultures. She is fluent in four languages: English, French, Spanish, and Arabic, and has always had a particular passion for Latin America. Her curiosity about the region was sparked in her sophomore-year Spanish class, where studying Latin American culture inspired her to explore Latin American studies. Researching the political and social environments of various countries deeply resonated with her learning style, and now she hopes to pursue this field as a college major.

At Westminster, Lilya is an active participant in academic and extracurricular life. She is the founder of Angels to Angels, a charitable initiative supporting access to education for underprivileged youth globally, and co-founder and co-president of the school’s Mock Trial program. Lilya leads the Women’s Empowerment and Leadership Club, serves as Business and Outreach Lead for the Robotics Team, and is a committed Lead Admissions Ambassador. She also contributes as a section editor for the Bi-Line, Westminster’s student newspaper, and serves on the Student Alumni Council Board.

Beyond school, Lilya is a Presidential Service Award recipient, a John Locke Essay Competition Finalist, and a member of the National Spanish Honors Society. She has also been recognized by the Scholastic Art & Writing Awards. Lilya is passionate about public service and has held internships with various community organizations while continuing to lead service-based and educational projects.

The post Between Roots and Progress: How Globalization, Urbanization, and Technology Reshape Brazilian Culture appeared first on Exploratio Journal.

Abstract

I evaluate a technical trading strategy that combines the Relative Strength Index (RSI) with a moving‑average (MA) trend filter. Using daily data for the 30 Dow Jones Industrial Average (DJIA) constituents over July 18, 2015–July 18, 2025, I test RSI deviation thresholds of 10, 15, and 20 points from neutral and apply a quarterly walk‑forward selection of RSI and SMA lookbacks. Returns are computed with next‑day execution and later adjusted for transaction and borrow costs. Across all specifications, Sharpe ratios are not statistically different from zero at the 5% level, and realistic costs eliminate small gross gains. Return distributions display high excess kurtosis and low skewness, consistent with exposure to higher‑order risks. I interpret the RSI and SMA combination less as a price‑forecasting tool and more as a risk filter that concentrates tail exposure. The strategy underperforms unconditionally, but the profile suggests it could perform better in specific market regimes in which such risks are compensated. I discuss implications for market efficiency and for using technical rules to target or avoid priced co‑moment exposures.

1. Introduction

Can a double-signal technical trading strategy generate consistent, risk-adjusted returns over the past decade?

To answer this, I run a 10-year backtest on three RSI deviation thresholds (10, 15, and 20 points from neutral), each paired with MA-based trend confirmation. Performance is measured gross of transaction costs and then adjusted using basis-point cost scenarios. I compute risk-adjusted metrics, including Sharpe ratios with Newey-West standard errors, Calmar ratios, and Sortino ratios, for each variation.

Multiple trading influencers promote variations of the RSI and MA strategy as a ‘magic bullet’. These influencers and channels have large followings and often promote their strategy in tandem with a course or funded trading partnership. Some examples are The Moving Average (YouTube, 1.07M+ subscribers), Trading Lab (YouTube, 1.74M+ subscribers), LuxAlgo (TikTok, Instagram, YouTube, 1.2M+ total followers), and many others. They create videos with bright, attention-grabbing colors and enticing thumbnail covers that draw inexperienced audiences in, promising to teach the “right way” to use these indicators and generate unrealistic returns. While these videos focus on anecdotal success and visual appeal, this study tests the strategy in a systematic, data-driven way.

This paper finds that none of the tested variations deliver statistically significant Sharpe ratios, with 95% confidence intervals spanning zero across all strategies. Some configurations yield small positive gross returns, but realistic transaction costs erase these gains. Our results suggest that this RSI–SMA strategy does not provide persistent predictive power and that any observed profitability likely stems from random variation rather than structural inefficiency.

Many investors employ technical trading strategies, such as the RSI and MA, despite decades of academic skepticism. If markets are efficient and past prices contain no useful information, such strategies should not be effective; yet, they persist in practice. We contribute to the literature by testing a simple yet popular combination of RSI and MA to evaluate risk-adjusted performance and exposure to certain risk profiles.

We focus not only on whether this strategy performs well, but also on whether any consistent performance challenges the Efficient Market Hypothesis (EMH), which states that past price data should not offer predictive advantages. McInish & Puglisi (1980) find that markets are efficient through runs tests of utility-preferred stocks. Brock, Lakonishok, and LeBaron (1992), however, find profitability in technical trading rules through nearly a century of DJIA data and contradict the EMH. This conflict shows how empirical results vary based on methodology, asset class, and time horizon, creating a rift between academic theory and real-world trading behavior. Understanding whether this tension reflects outdated testing methods, behavioral inefficiencies, or changing market dynamics is important for both researchers and investors.

The Relative Strength Index identifies overbought and oversold conditions, while the moving average smooths price action to show trend direction. Online trading communities often promote these indicators as part of a ‘signal confluence’ approach before entering trades. As retail trading grows with the rise of zero-commission trading and options contracts, underexperienced traders often rely on signals as primary strategies. These indicators, widely circulated in trading literature and online forums, influence trading decisions despite limited evidence of predictive power, making them a useful case study for examining whether such strategies expose traders to specific market conditions.

In addition to evaluating strategy performance, we reframe technical analysis as a potential filter for hidden risk exposures. This approach considers whether consistent performance could arise from taking on unattractive risks that most investors avoid, and therefore the most rewarding. We account for tail risk and other exposures not well captured in metrics like beta and volatility. In that case, the strategy would not forecast price action but would filter for market conditions where the trade-off between risk and return is most favorable.

The next section reviews the literature on technical analysis, while section three discusses the theoretical context behind risk exposure in asset pricing. Section four describes our methodology, and section five presents the results. Section six discusses the implications, section seven concludes the study, followed by references.

2. Literature Review

2.1 Early Academic Skepticism

Historically, there has been skepticism towards technical analysis in academia. Technical analysis was originally conceptualized around observed patterns rather than formal theory. Some studies, such as Tabell and Tabell’s (1964), emphasize Dow Theory and the Elliott Wave Principle, highlighting cycles and human behavior, themes still relevant today. Levy (1966) offers a more neutral perspective, critiquing both technical and fundamental approaches to trading. While he finds more empirical support for fundamental analysis, his openness suggests that even early researchers saw potential for technical signals to reflect meaningful information. Other studies go further by testing technical analysis and find no edge, concluding that price behaves randomly. Van Horne and Parker (1968) test a moving average and do not find excess returns, and are often cited as one of the earliest academic rejections of technical analysis. Their paper, however, is methodologically limited by a small sample size and lacks out-of-sample testing, limitations that this paper will address with modern tools and risk framing.

2.2 Conditional Effectiveness and Market Context

More modern studies suggest that technical analysis may work in certain market conditions. Han, Yang, and Zhou (2013) and Brown, Crocker, and Foerster (2009) both find that using moving average and buy-and-hold strategies, assets with higher risk components like volatility and turnover earn higher returns. This supports the idea that investor behavior and market dynamics may create price movements that technical analysis is better suited to capture than fundamental analysis. This may imply success in specific market conditions, particularly conditions that are traditionally avoided.

Building on this conditionality, other studies explore when and why technical analysis might work. Bessembinder and Chan (1998) similarly show that technical trading rules may have statistical predictive power, but lack economic significance due to trading costs and nonsynchronous trading effects. This counters the notion of technical analysis as a magic bullet for market context. Likewise, Qi and Wu (2006) test thousands of trading rules and find conditional effectiveness in the earlier half of their sample period, indicating that technical analysis may exploit inefficient markets. Together, these studies suggest that technical analysis can be used as a lens for inefficiency, a concept that will be explored further in this paper.

2.3 Behavioral and Structural Explanations

Many papers support the idea of technical analysis as a filter for behavior and structure. Smith et al. (2016) and Moosa and Li (2011) argue that technical levels correspond with real behavioral structures in different markets. The former shows that hedge funds using technical analysis have higher returns during periods of high sentiment, highlighting the connection to irrational investor behavior. The latter investigates the disproportionate success of technical strategies in Chinese markets, examining the distortion of markets by government intervention and high retail participation. These behavioral patterns may create a unique environment suitable for technical analysis where rational models fail. Other studies, such as Blume et al. (1994) and Kavajecz and Odders-White (2004), find correlations between specific signals and market information. Blume et al. argue that volume reflects trader confidence, while Kavajecz and Odders-White find an alignment between technical indicators and changes in liquidity. These findings imply that, in addition to behavior, structural market components may help inform the conditions where technical analysis could be successful.

2.4 Agreements and Differences

Across the literature, two general camps emerge. Early research, represented by Van Horne and Parker (1968), contends that technical analysis lacks predictive ability, even after accounting for randomness and naïve benchmarks. More recent research, however, finds pockets of conditional efficacy, whether due to market regimes (Han, Yang, & Zhou, 2013; Qi & Wu, 2006) or behavioral/structural patterns (Smith et al., 2016; Moosa & Li, 2011). Where the camps converge is in emphasizing the role of context: even the most ardent supporters recognize that strategy efficacy relies on particular circumstances, whereas skeptics admit anomalies may appear in particular settings. The disagreement is in whether those circumstances are exploitable after costs, and whether findings represent persistent inefficiencies or fleeting artifacts of market design.

This divide directly shapes our methodological choices. First, to address concerns over sample bias and parameter overfitting, we evaluate a multi-signal strategy rather than isolated rules, mirroring how traders filter entries. Second, we incorporate transaction cost adjustments to test whether any gross profitability survives realistic frictions, a point stressed by cost-sensitive studies like Bessembinder and Chan (1998). Finally, inspired by the state-dependent findings in both behavioral and structural work, we adopt a rolling quarterly parameter optimization to account for changing market conditions. This design allows us to test not only whether the strategy performs overall, but also whether it behaves differently across varying environments, bridging the empirical gap between one-size-fits-all tests and purely context-specific studies.

3. Theoretical Context

3.1 Strategy Risk Profile and Testable Claim

We test whether a dual-signal RSI+MA strategy inherently loads on tail risk, marked by high kurtosis and near-zero or negative skewness, because it systematically buys into short-term weakness and sells into short-term strength relative to prevailing trends. In its mean-reversion phases, the strategy takes long positions when RSI signals oversold conditions in uptrends and short positions when RSI signals overbought conditions in downtrends. This pattern can produce many small gains interrupted by rare but significant losses (a “knife-catching” profile), yielding elevated kurtosis and potentially negative skewness. The SMA trend filter mitigates some of this tail risk but does not fully eliminate it. If these return characteristics align with risks that most investors avoid, the strategy may command a risk premium in expected returns.

3.2 Why Higher-Order Risks Matter

In finance, investors are not only concerned with the magnitude of returns, but also with the distribution of gains and losses. Traditional models like the Capital Asset Pricing Model (CAPM) focus on volatility and market sensitivity as the primary sources of risk. However, this framework often overlooks more complex behaviors in return distributions that describe the shape and extremity of gains and losses. For example, two assets might have the same average return and volatility, but one may suddenly crash while the other steadily gains, with investors generally preferring the latter. By intentionally taking on the types of risk that most investors avoid, a strategy may earn higher expected returns as compensation for bearing that discomfort. This paper explores the idea that technical analysis, though often dismissed as unscientific, can function as a filter for the risk exposures that investors implicitly price. This could allow strategies to profit by taking on return profiles that are mispriced or systematically avoided. Central to identifying these patterns are two higher-order risk moments, skewness and kurtosis.

3.3 Definitions and Investor Preferences

A growing body of research suggests that co-skewness and co-kurtosis are crucial in asset pricing. Skewness captures asymmetry in the return distributions of a portfolio. Positive skewness represents large gains with frequent small losses, while negative skewness represents frequent small gains with the potential for large losses. Kurtosis measures the tailedness of the 0 return distribution, where high kurtosis represents a higher probability of extreme values, while low kurtosis represents a lower probability of extreme values.

Co-skewness and co-kurtosis, then, are the relationships between these metrics when comparing one variable to another, generally an asset compared to the market portfolio. Co-skewness describes how the asymmetry of one variable’s distribution is connected to movements in another variable. Co-kurtosis describes how the tendency for extreme values in one variable is connected to movements in another variable. A portfolio with negative co-skewness and high co-kurtosis, for example, tends to suffer its worst losses at the same time as the market, and experiences extreme returns in the same period, amplifying drawdown. However, CAPM suggests that assets performing poorly when investors most value consumption must offer higher expected returns as compensation. Because bad co-skewness and co-kurtosis are so unattractive to investors, the risk may be compensated for.

3.4 Pricing of Higher-Order Risks

Some studies explain this correlation between investor behavior and higher-order risk moments. Conrad, Dittmar, and Ghysels (2013) argue that skewness isn’t only measurable in hindsight, and that investors price ex ante skewness through the options market. They find that stocks with more expected negative skewness earn higher average returns, consistent with the theory that investments with more inherent risk require more compensation. The authors also find that stocks with positive expected skewness earn lower average returns, but the potential for large positive price changes remains attractive to investors. Frugier (2014) adds to this conclusion, arguing that these higher-order moments are embedded signals of trader psychology. He examines past stock data and finds that negative skewness and high kurtosis arise in times of 1 unresolved uncertainty and fear, indicating these psychological factors. These studies show that skewness and kurtosis must be priced in because investors prefer certain shapes of risk, like the frequency and size of gains and losses. These findings suggest that technical analysis could succeed if it accurately filters undesirable risk exposures that demand higher returns.

3.5 Institutional and Practical Evidence

Similar filters seem to be used in institutional trading. A study by Malkiel and Saha (2005) investigates risk exposure in hedge funds. They find that an overwhelming majority of hedge funds in the TASS database exhibit returns with low skewness and high kurtosis, generally avoided by traditional investors. This is supported by Elkamhi and Stefanova (2015), who show that in volatility-based portfolios, even small exposures to skewness and kurtosis can lead to outsized losses. Because of this, portfolios would benefit from explicit identification of these moments to hedge against them. If technical analysis can isolate exposure to higher-order risks, it should be viewed as a way to expand the investor’s span, to broaden the set of priced risks a portfolio can intentionally target or avoid. Technical strategies can be evaluated not just on signal accuracy, but on whether their return profiles reflect exposure to hidden risks. This perspective will be tested in the analysis that follows.

4. Strategy and Methodology

This strategy will test multiple combinations of the Relative Strength Index (RSI) and moving average (MA). The RSI(14) and MA(200) are the most standard values, and these period lengths are commonly referenced in social media posts. Other lengths modify the quantity, and 2 potentially ‘quality’ of each signal as trends become smoother with greater lengths. However, market conditions may affect this ‘quality’, so we perform a quarterly walk-forward selection of RSI and MA combinations to optimize for shifting regimes.

4.1 Universe & Data

We retrieved daily price data for all 30 constituents of the Dow Jones Industrial Average (DJIA) over ten years (July 18, 2015, to July 18, 2025) from the Yahoo Finance Historical Data API (see Appendix for the full list of tickers). Results are non-point-in-time, backfilled from the most recent members. We used close prices adjusted for dividends and splits.

4.2 Signal Construction

The RSI is computed on adjusted closes using Wilder’s smoothing as:

Where AU_t and AD_t are the exponentially smoothed average gains and losses over n days. The SMA is calculated as:

4.3 Position Sizing and Execution Convention

Signals are generated and executed at the close on day t. Each stock may trigger at most one entry per day, and each position is allocated 1/10 of the total portfolio value to avoid concentration risk. Position weight per stock is fixed-weight per trade. Buy and sell signals are processed independently for each stock.

4.4 Walk-Forward Selection

To adapt the strategy to evolving market conditions, we partitioned the 10-year sample into 40 quarterly segments. For each RSI deviation threshold (10, 15, and 20), an initial quarter runs with default parameters of RSI(50) and SMA(200). At the end of each quarter, we evaluate RSI and SMA combinations based on their Sharpe ratios. We apply the combination with the highest Sharpe ratio to the next quarter. This process repeats in a walk-forward fashion across all 40 quarters, producing a series of returns in which each quarter’s parameters reflect the most profitable configuration from the preceding quarter.

4.5 Shorting

This backtest assumes that short sales are permitted for all DJIA constituents at all times, with full borrow availability. A borrowing cost is applied to all open short positions, representing the annualized interest rate charged for borrowing shares. This cost, denoted c_b , is assumed to fall within the range of 30-50 basis points per year (0.30%-0.50%), with the base case using = c_b 0.50%.

This daily borrow charge is calculated as:

Where ShortNotional_t is the dollar value of the short position on day t . Borrow costs t are deducted from portfolio equity each day the short position remains open. Dividend payments owed on short positions are already incorporated in the adjusted price series used for return calculations, so no separate adjustment is required.

4.6 Transaction Costs

Transaction costs were incorporated by first calculating daily portfolio turnover as the average absolute position change across the 30 DJIA stocks.

We multiplied this turnover by a per-dollar transaction cost rate c to estimate daily cost drag.

We then calculated the Compound Annual Growth Rate (CAGR) from the net return series, with results reported for costs of 0, 2, 5, and 10 basis points per side.

All other performance metrics are based on gross returns, with the impact of transaction costs summarized separately in the CAGR table.

4.7 Risk and Return Metrics

We evaluate portfolio performance using a set of risk-adjusted and distributional metrics. The Sharpe ratio is computed using Newey-West adjusted standard errors to account for serial correlation in daily returns.

r o represents the daily average return of our portfolio, while NW represents the standard deviation of returns that has been adjusted for autocorrelation (the similarity between a variable’s values at different points in time) and heteroskedasticity (the spread of standard deviations of a variable).

We use a Calmar ratio to measure the trade-off between annualized return and maximum drawdown.

Where CAGR is the Compound Annual Growth Rate of our portfolio over the testing period, and the denominator is the absolute value of the maximum drawdown of the portfolio.

The Sortino ratio isolates downside volatility as opposed to both sides.

Where T^— is the number of days with negative returns, and r_t the return on day t. Wholly, the denominator represents downside deviation, or the standard deviation of negative returns.

Skewness and kurtosis are computed to assess asymmetry and tail heaviness in the return distribution.

Where T o represents the total number of days in the sample and represents the standard deviation of all returns. Skewness represents asymmetry of the return distribution (whether it is more tilted to gains or losses). Kurtosis represents the “tailedness” of the distribution (the likelihood of extreme gains or losses). All ratios are computed using gross daily returns unless otherwise noted, with transaction-cost-adjusted returns reported separately.

5. Results

This section presents the performance of an RSI and MA strategy under three different threshold configurations: DEV 10, DEV 15, and DEV 20. These values represent the entry condition deviation in RSI. Results are based on a portfolio spanning July 2015 to July 2025.

5.1 Cumulative Returns

The cumulative return chart shows diverging performance across the three RSI deviation tests, all ending with negative returns. Differences in performance begin to emerge in mid-2017, early in the testing period. DEV 10 (blue) showed brief periods of positive returns and was the most volatile, ending with the highest return of the three. DEV 15 and 20, although considerably more stable in return trajectory, finished with even lower returns. Returns tended to move in the same direction for most of the period, suggesting positive covariance between strategies.

5.2 Annualized Metrics

Annual returns were negative across all three configurations. Annual excess returns versus SHY and SPY were also negative, the lowest being DEV 20 (-2.129%) and the highest being DEV 10 (-1.514%). V olatility followed the same pattern, from 2.162% (DEV 20) to 3.154% (DEV 10). The Sharpe ratio also decreased with higher deviation thresholds, ranging from -0.480 (DEV 10) to -0.985 (DEV 20). However, 95% Newey-West adjusted confidence intervals for the Sharpe ratios include 0 in all three strategies, implying that the Sharpe ratios are statistically insignificant. Small, negative Sortino and Calmar ratios suggest poor performance compared to risk-free assets and drawdown, indicating greater downside risk.

5.3 Transaction Costs

To address the effects of transaction costs, we recalculated CAGR for each RSI Dev strategy after applying per-dollar transaction costs of 0, 2, 5, and 10 basis points per side. Across all three strategies, higher transaction costs reduced CAGR approximately linearly with cost magnitude. For further context, we calculated total transaction costs as a percentage of portfolio value. Dev 10 exhibited the highest turnover (5.25% at 10 bps), confirming that smaller deviation thresholds are more sensitive to cost assumptions.

The above graphs show trade logs for each strategy, blue representing long trades and orange representing short trades. This log contextualizes higher transaction costs for the lower threshold strategy, Dev 10, which had the highest trade volume.

5.4 Return Distributions

Returns for all three strategies were density-adjusted and compared to normal distributions. Return distribution plots for each test show a sharper central peak and heavier tails than Gaussian (high excess kurtosis). All three strategies display high kurtosis with values ranging from 18.006 (DEV 15) to 21.788 (DEV 20). Although most returns are clustered around the mean, this shows that these strategies experienced a significantly higher frequency of extreme return days than a normal distribution.

Skewness values ranged from low (DEV 10 and DEV 15) to low-moderate (DEV 20). DEV 10 and DEV 20 had positive skew, implying a higher probability of large positive gains, while DEV 15 had a negative skew and a tendency towards large negative gains.

6. Discussion

This section interprets the performance of the RSI and MA strategy beyond raw returns. While the strategy underperformed, its results reveal insights into market efficiency, hidden risk exposures, and the conditional role of technical analysis.

6.1 Underperformance and Market Efficiency

The RSI and MA strategy tested in this paper underperformed across all annualized metrics in the ten-year testing period, but its implications suggest a broader role of technical analysis in risk exposure. All three strategy variants show negative risk-adjusted values when compared to SHY and display negative Sharpe, Calmar, and Sortino ratios. These outcomes are consistent with the weak form of the Efficient Market Hypothesis, which holds that past price data contains no predictive or actionable information. However, this underperformance does not imply that the signals were entirely random or meaningless. The return paths of DEV 10, 15, and 20 were similar for most of the ten years. Positive covariance in the three strategy variations suggests exposure to similar structural patterns in the market. The strategy may not unconditionally generate alpha, but it might expose investors to certain risks hidden in asset prices.

6.2 Hidden Risk Exposure and Conditional Opportunity

The most compelling pattern in the results is the consistently high kurtosis. This indicates frequent extreme returns, which traditional investors tend to avoid due to financial and psychological discomfort. Relatively low skewness values indicate minor asymmetry in returns, but the heavy-tailed return distributions suggest that the RSI and MA strategy filters for hidden, 2 non-diversifiable risk. This aligns with research by Conrad et al. (2013) and Frugier (2014), who argue that investors price these risks because they reflect the large, abrupt losses during ‘bad times’.

Consistency across all three thresholds suggests that the strategy systematically targets certain classes of risk that are not rewarded over long periods, but may be under unique conditions. This helps explain the underperformance, as strategies with undesirable risk (high kurtosis and low skewness) should not be expected to outperform in all periods, especially in a mostly efficient market. However, it might work over certain regimes where these risks are mispriced, such as volatility clusters and trend reversals.

6.3 Reframing Technical Analysis

Instead of interpreting the negative returns as evidence against technical analysis, these results suggest a more nuanced framing. The value of the RSI and MA strategy is likely not in forecasting price but in allowing investors to bear discomfort in exchange for higher compensation. This is only possible, of course, with research into exactly when and where these strategies conditionally succeed. This can be tested across periods segmented by different macroeconomic factors, and doing so might reveal how technical analysis’s payoffs align with inefficiency-prone conditions.

Ultimately, although this study does not contradict market efficiency, it reveals how technical analysis could account for shortcomings of traditional investment strategies. It might add value by identifying risks that investors expect compensation for. In this way, it can be used as a tool for portfolio construction to access unconventional sources of returns, or simply to hedge against risks in traditional portfolios.

7. Conclusion

In conclusion, this study tested a widely used technical trading strategy, a combination of the Relative Strength Index and moving average. The analysis spanned ten years and evaluated three different RSI deviation thresholds. While the strategy underperformed across all key return metrics, its characteristics offer important insights into the academic and practical discourse around technical analysis. The strategy’s exposure to extreme returns, seen in its high kurtosis, suggests that it filters for hidden, non-diversifiable risk. This finding is relevant in today’s markets, where signal-based strategies are frequently promoted across social media platforms and adopted by traders in search of an edge. Understanding what these strategies are actually capturing, even when they fail to generate alpha, is essential to evaluating their role in constructing portfolios and risk management.

For researchers, these findings support a reframing of technical analysis as a mechanism for identifying priced risk exposures. Further study should test performance in different market conditions where these risks might be mispriced. For traders and portfolio managers, these findings emphasize the risk structure behind technical analysis. While signals like the RSI and MA might not yield consistent profits, they can help in designing portfolios that take advantage of risks that are compensated under certain conditions. This understanding can also help retail traders avoid misusing signals as forecasters of price action. In this way, the lasting contribution of technical analysis is not its forecasts, but its ability to expose the risks that markets reward.

References

Bessembinder, H., & Chan, K. (1998). Market Efficiency and the Returns to Technical Analysis. Financial Management, 27(2), 5–17. https://doi.org/10.2307/3666289

Blume, L., Easley, D., & O’Hara, M. (1994). Market Statistics and Technical Analysis: The Role of V olume. The Journal of Finance, 49(1), 153–181. https://doi.org/10.2307/2329139

Brock, W., Lakonishok, J., & LeBaron, B. (1992). Simple Technical Trading Rules and the Stochastic Properties of Stock Returns. The Journal of Finance, 47(5), 1731–1764. https://doi.org/10.2307/2328994

Brown, J. H., Crocker, D. K., & Foerster, S. R. (2009). Trading V olume and Stock Investments. Financial Analysts Journal, 65(2), 67–84. http://www.jstor.org/stable/40390354

Conrad, J., Dittmar, R. F., & Ghysels, E. (2013). Ex Ante Skewness and Expected Stock Returns. The Journal of Finance, 68(1), 85–124. http://www.jstor.org/stable/23324392

Elkamhi, R., & Stefanova, D. (2015). Dynamic Hedging and Extreme Asset Co-movements. The Review of Financial Studies, 28(3), 743–790. http://www.jstor.org/stable/24465726

Frugier, A. (2014). Higher-order Moments and Investor Sentiment (Alles’ Model Revisited). Quarterly Journal of Finance and Accounting, 51(3–4), 45–70. http://www.jstor.org/stable/qjfinacct.51.3-4.45

Han, Y ., Yang, K., & Zhou, G. (2013). A New Anomaly: The Cross-Sectional Profitability of Technical Analysis. The Journal of Financial and Quantitative Analysis, 48(5), 1433–1461. http://www.jstor.org/stable/43303847

Kavajecz, K. A., & Odders-White, E. R. (2004). Technical Analysis and Liquidity Provision. The Review of Financial Studies, 17(4), 1043–1071. http://www.jstor.org/stable/3598058 5

Levy, R. A. (1966). Conceptual Foundations of Technical Analysis. Financial Analysts Journal, 22(4), 83–89. http://www.jstor.org/stable/4470026

Malkiel, B. G., & Saha, A. (2005). Hedge Funds: Risk and Return. Financial Analysts Journal, 61(6), 80–88. http://www.jstor.org/stable/4480718

McInish, T., & Puglisi, D. J. (1980). Technical Analysis and Utility Preferred Stocks. Nebraska Journal of Economics and Business, 19(3), 55–63. http://www.jstor.org/stable/40472670

Moosa, I., & Li, L. (2011). Technical and Fundamental Trading in the Chinese Stock Market: Evidence Based on Time-Series and Panel Data. Emerging Markets Finance & Trade, 47, 23–31. http://www.jstor.org/stable/27917672

Neely, C. J., Rapach, D. E., Tu, J., & Zhou, G. (2014). Forecasting the Equity Risk Premium: The Role of Technical Indicators. Management Science, 60(7), 1772–1791. http://www.jstor.org/stable/42919633

Qi, M., & Wu, Y . (2006). Technical Trading-Rule Profitability, Data Snooping, and Reality Check: Evidence from the Foreign Exchange Market. Journal of Money, Credit and Banking, 38(8), 2135–2158. http://www.jstor.org/stable/4123046

Smith, D. M., Wang, N., Wang, Y ., & Zychowicz, E. J. (2016). Sentiment and the Effectiveness of Technical Analysis: Evidence from the Hedge Fund Industry. The Journal of Financial and Quantitative Analysis, 51(6), 1991–2013. http://www.jstor.org/stable/44157641

Tabell, Edmund W., & Tabell, Anthony W. (1964). The Case for Technical Analysis. Financial Analysts Journal, 20(2), 67–76. http://www.jstor.org/stable/4469619

Van Horne, J. C., & George G. C. Parker. (1968). Technical Trading Rules: A Comment. Financial Analysts Journal, 24(4), 128–132. http://www.jstor.org/stable/4470382

Yahoo Finance. (2025). Dow Jones Industrial Average historical data [Data set]. Yahoo. Retrieved July 18, 2025, from https://finance.yahoo.com/

About the author

Roshan Shah

Roshan is currently a senior at Glenbrook South High School in Glenview, Illinois. He intends to complete an undergraduate degree in Applied Mathematics. He is particularly interested in the intersection of mathematics, computer science, and finance, and looks forward to pursuing a career in financial technology. In his free time, Roshan enjoys a wide variety of extracurricular activities, including Model United Nations, digital music production, working out at the gym, and spending time with friends. He also enjoys helping his community by providing food and clothing to the homeless in Chicago.

The post Technical Trading and Higher-Order Risks: An Empirical Evaluation of Relative Strength Index and Moving Average Strategies appeared first on Exploratio Journal.

Abstract  

For my research project, I wanted to combine my interest in hockey with my curiosity about data and math. I explored how a Kalman filter—a tool often used in engineering and statistics—could be applied to track NHL player performance during games. Instead of relying on end-of-season stats, which can sometimes be misleading, I built a system that updates each player’s rating shift by shift. It takes into account who was on the ice and the expected goal differential, giving a more accurate and real-time picture of how players are performing. This helps smooth out the effects of one unusually good or bad game and offers more meaningful insights. I hope this project not only helps fans and fantasy players better understand player impact but also shows how math and data science can be used to analyze the sports we love.  

Introduction  

In hockey, stats like goals and assists only tell part of the story. They don’t show what a player does without the puck or during defensive plays, which are just as important. That’s why analysts use a stat called expected goals (or xG), which measures how likely a shot is to become a goal based on things like where it was taken from and what kind of shot it was. While xG is more helpful than just counting goals, it still doesn’t tell you how much each individual player contributed to creating or preventing those chances. For my project, I wanted to build a better way to track player performance during games. I used a tool called a Kalman filter, which is great for situations that change quickly and randomly—just like in hockey. It allowed me to update each player’s rating shift by shift, giving a more accurate and real-time view of how they’re playing. 

Background on Kalman Filters  

The Kalman Filter, introduced by Rudolf E. Kálmán in 1960 [1], is widely used in control systems and aerospace to estimate hidden states over time from noisy measurements. Its ability to extract a “true” underlying value from uncertain data makes it well-suited for tracking player performance shift-by-shift in hockey, where single-game results can be noisy or misleading[3].  

At its core, the Kalman filter is a recursive algorithm that estimates the state of a system over time[2]. The “state” is something we want to track (like a player’s impact), and the filter uses a combination of past estimates and new observations to improve that estimate. 

The Kalman filter works by treating what we’re trying to measure—like a player’s skill— as something that exists but can’t be directly seen. Instead, we get noisy signals (like xG differential from shifts) that give us partial information. The Kalman filter then tries to  “guess” the true skill level over time by combining what it previously thought (the prediction) with the new noisy measurement (the update). 

Think of it like trying to track a car in fog. You can’t see it clearly, but you occasionally catch glimpses of it. Each glimpse is uncertain, but by combining all the glimpses and accounting for how much things could change or how noisy the observations are, you can make a better estimate of where the car is. That’s essentially what the Kalman filter does —but with math. To better understand why this works, we’ll break down the math and where it comes from. 

The Kalman filter is grounded in two core ideas: prediction and correction. It begins with a guess (called the prior) of a hidden variable—like player skill—and as new, imperfect data arrives, it updates that guess. The power of the filter lies in how it mathematically balances how much to trust the old guess versus the new data using probabilities. 

In mathematical terms, it minimizes the mean squared error of the estimate, assuming  Gaussian (bell-curve) noise. This gives it optimality in linear systems, which is why it’s used in everything from radar tracking and robotics to sports analytics.

The filter runs in a loop; every time new data comes in—in this case, each new shift in a game—and updates player ratings by combining what we already know with what just happened. 

The two main steps are called the prediction step and the update step. 

1. Prediction 

In this step, the filter predicts the next state of the system and how uncertain that prediction is. 

    •    State prediction: 

  xₖ₋₁ → x̂ₖ⁻ = x̂ₖ₋₁ 

    •    Uncertainty prediction: 

  Pₖ⁻ = Pₖ₋₁ + Q 

Here: 

    •    x̂ₖ⁻ is the predicted state estimate before seeing new data 

    •    Pₖ⁻ is the predicted uncertainty 

    •    Q is the process noise (random natural changes) 

2. Update 

Now the filter corrects its prediction using the actual observation from the new data.  

   •    Kalman Gain (how much we trust the new data): 

    Kₖ = Pₖ⁻ Hᵀ (H Pₖ⁻ Hᵀ + R)⁻¹ 

    •    Updated estimate:  x̂ₖ = x̂ₖ⁻ + Kₖ (zₖ − H x̂ₖ⁻) 

    •    Updated uncertainty:  Pₖ = (I − Kₖ H) Pₖ⁻

 where:

Zₖ: is the new observation (like xG_diff for a shift) 

   H:  is the matrix that maps the true state to what we can observe     

   R: is the measurement noise (how noisy our observation is) 

   Kₖ: is the Kalman Gain 

   x̂ₖ: is the updated estimate of the state 

   Pₖ: is the updated uncertainty 

These equations come from linear algebra and probability theory. The Kalman filter assumes the state evolves according to a linear model and that both the system and the measurement noise are Gaussian (bell curve-shaped) with known variances. 

The matrices Q and R represent how much randomness or noise we expect in our system and observations. If Q is high, we assume the underlying player skill changes a lot between shifts. If R is high, we assume our xG measurements are very noisy. Tuning these helps the model balance trust between past beliefs and new data. 

Where the Equations Come From  

The Kalman filter is based on the idea of minimizing uncertainty. It assumes that the current state (player rating) follows a simple rule: it either stays the same or changes slightly due to random factors. Observations (like xG_diff) are linked to the state through a matrix (H) and contain noise. The equations are derived by using Bayes’ Theorem to find the most likely state given the prior estimate and the new observation.  

In simple terms, each equation is designed to balance two forces:  

• “The prior (what we thought before this shift)”  
• “The measurement (what the shift data just told us)” 

The Kalman Gain (K) adjusts how much we move toward the new observation. If the data is noisy, we move less. If the prediction was way off, we move more. The math ensures this is done in the optimal way to reduce error. 

The update formula (posterior = prior + gain × error) is essentially a Bayesian update. It  answers: ‘Given what I used to believe and what I just observed, what’s the best new  belief?’ The Kalman gain determines how much we shift our belief, and this is computed based on how uncertain we are about both the prior (P) and the measurement (R).

Why the Kalman Filter Works  

The Kalman Filter works best in situations where you need to track something uncertain over time using noisy information. In hockey, a player’s true impact can’t be directly measured—we only see glimpses through stats like xG_diff. By combining the model’s previous belief with the new evidence from each shift, the Kalman Filter gives a smarter,  smoother estimate of how good each player really is. 

It also adjusts how much to “believe” each new data point based on how noisy it thinks that data is. If one shift has a weird result, the filter won’t overreact. But if lots of shifts show a consistent trend, it will gradually adapt the rating. 

Data and Setup  

I used public 5v5 shift data from Natural Stat Trick [5]. For each shift, I collected:  the list of offensive players and defensive players, and the expected goals differential (xG_diff)  

Each player was given a unique index number. Everyone started with a rating of 0, and I set small initial values for uncertainty and noise because I assumed performance doesn’t change too quickly. This setup allowed the model to slowly adapt as more shifts occurred.  

I used scalar values for the process noise and measurement noise. Specifically, Q=q⋅I and R=r, where I is the identity matrix. I manually tuned q and r to be small (e.g., q=0.01, r=0.1) to ensure stability and avoid overreacting to noisy shifts early in the season.  

Assumptions  

I assumed that the system is linear, noise is Gaussian with a zero mean, and that the covariance is known. 

Method: Using the Kalman Filter  

For every shift in the dataset, I built a vector H, where:  

 Offensive players = +1 

 Defensive players = –1 

This vector H∈R^1xn represents the players on the ice during a shift, where n is the total number of players in the dataset. Each element of H corresponds to a player’s index:  players on offense are set to +1, players on defense are set to –1, and all other values are  0. When we multiply H x X, we get the predicted xG differential for that shift, based on current ratings. 

Then I used the xG_diff from that shift as the observation (z) and ran the Kalman filter to update player ratings. The state vector x∈R^n holds the current rating of every player. Although only a small number of players are involved in any one shift, the Kalman Filter maintains this full vector across all players. The update only affects players present in the shift.  

Importantly, this means we’re not calculating each player’s contribution in isolation. The Kalman filter sees how the group of players on the ice performed together, then uses that to adjust everyone’s ratings just a little. So if a defensive player is consistently on the ice when their team allows fewer expected goals, their rating will slowly increase. This teamwork-based approach is one reason the filter gives more reliable ratings than just counting goals or points.  

How the Kalman Filter is applied to Player Ratings  

In each shift, I record the expected goals for and against (xG differential). Because multiple players are on the ice, the xG differential is influenced by everyone’s performance. I model the team-level xGD as a sum of the ratings of the skaters on the ice.  Using a Kalman Filter, I estimate and update each player’s rating over time. The filter updates a player’s performance estimate based on how the team performed while they were on the ice and how reliable that measurement is.

Let xk be a player’s latent rating at shift k, and zk be the observed xG differential for that shift. 

Prediction Step: 

x̂k | k − 1 = x̂k − 1| k − 1 

P_k|k−1 = P_k−1|k−1 + Q 

Update Step: 

K_{k =} P_k|k−1 

P_k|k−1 + R 

x̂k | k = x̂k | k − 1 + K_k(z_k − x̂_k|k−1) 

P_k|k = (1 − K_k)P_k|k−1

• x^k∣k−1: predicted rating before seeing the new shift 
• P: uncertainty in our estimate 
• Q: process noise (how much a player’s performance might vary) • R: measurement noise (uncertainty in xG) 
• K: Kalman Gain (how much we trust the new shift vs prior rating) 

I chose the Kalman Filter because it balances past performance with new shift data in a mathematically consistent way. Traditional models either rely too much on averages or overreact to single games. By using xG differential and updating shift by shift, this method smooths out variance while still responding to meaningful changes[6]. The assumptions I make include linear player contributions and Gaussian noise, which simplify computation while still producing stable ratings. 

For each shift, we follow this sequence:” 

1. Build matrix H, encoding who is on the ice (offense = +1, defense = –1).
2. Predict the expected xG_diff using current player ratings. 
3. Observe the actual xG_diff for that shift. 
4. Update each involved player’s rating based on the difference between prediction and observation. 
5. Repeat for every shift throughout the season. 

1. Prediction Step  

We first predict the current state before seeing the new data:  

x_prior = x 

This means we start by assuming the ratings from the last shift are still correct.  P_prior = P + Q 

 P is our current uncertainty (a covariance matrix), and Q is the extra uncertainty we add to account for performance possibly changing naturally. 

So this step gives us our best guess before seeing new data and tells us how confident we are in that guess.  

2. Update Step  

Now we use the new shift data to update our estimates:  

K = P_priorH^⊤S⁻¹

This tells us how uncertain we are about our prediction for the xG_diff on this shift.  

 H · P_prior · Hᵀ: how uncertainty in player ratings translates into uncertainty in xG_diff.  

 R: The noise in our observation of xG_diff.  

K = P_priorH^⊤S⁻¹ 

This is the Kalman Gain, which tells us how much to trust the new observation.  

Here, we assume the underlying ratings change slowly (low Q), and that the measurement (xG_diff) has moderate noise (R). The filter adapts depending on these values:  

 High R = less trust in new data, more weight on prior estimates  • High Q = belief that ratings can change quickly, so the model updates faster  

x_updated = x_prior ^{+ K} ₍z − Hx^prior) 

This is the main update:   z is the actual xG_diff observed.  H · Xprior is what we expected the xG_diff to be, based on current ratings. The difference is the “error,” which we scale with K and use to update ratings.  

P_updated = (I − KH) P_prior

Finally, we reduce our uncertainty. After seeing new data, we’re more confident in our updated ratings.

How It All Comes Together  

For every shift, we loop through the following sequence:  

1. Predict ratings and uncertainty 
2. Observe actual xG_diff 
3. Compare prediction to reality 
4. Update ratings and reduce uncertainty 

This repeats for every shift in the season, and over time, the ratings become smarter and more stable. Before applying the Kalman filter, it’s useful to see how shot share (CF%) compares to expected goal share (xGF%) across all players. 

Suppose a shift has three players: Player A, B, and C. A and B are offense, C is defense.

 The vector H=[1,1,−1], and the observed xG differential z=0.3. If we assume that the prior ratings x=[0.1,0.1,−0.1]. The predicted xG is H⋅x=0.1+0.1+0.1=0.3, which matches the observation, so the update is small. However, if z=0.6, the Kalman Gain would increase the offensive ratings and penalize the defensive rating accordingly. This is how the model incrementally adjusts player values. 

This example shows how, over time, each player’s rating becomes a reflection of their repeated impact. A player who always ends up on the ice during good shifts will see their score rise—even if they’re not scoring themselves. 

This process also prevents overreaction. If a player has one really strong shift, their rating only adjusts slightly. But if they consistently influence the xG_diff over many shifts, the filter learns to trust that signal more. This is what gives Kalman ratings their strength— they balance recency with consistency. 

Experiments and Results  

I ran this model on a full NHL season of shift-by-shift data[4]. As the season went on, each player’s rating changed depending on how much they helped or hurt the expected goals during shifts.  This yielded  the following observations:

 Players who weren’t big scorers still rated highly because they consistently created chances or prevented goals.  Players with big goal totals but poor defensive play sometimes ranked lower. Finally, the ratings were more stable than raw xG—less noise, fewer random spikes.  

 For example, Matty Beniers rated higher than expected due to consistent defensive contributions despite modest goal totals, while Patrick Kane had a lower rating due to poor xG_diff when on the ice, even though he scored often. 

As shown in Figure 2 below, some players with strong overall performance (based on xGF%) ranked highly in the Kalman model even if they didn’t lead in goals.. This demonstrates how the model can highlight impactful players who may not lead in traditional scoring stats.  

Implications  

This type of model has real potential:   NHL teams could use it to find undervalued players or make smarter trade decisions.  Scouts could see who consistently impacts the game beyond goals. Fantasy hockey tools or betting models could use it to get an edge.  The method could work in other sports, for example, basketball or soccer.  The filter could be improved by adding more features for example ice time, face-off zones, or goalie performance.  

Conclusion  

Kalman filters offer a powerful, more balanced way to rate players. They combine old data with new information and avoid overreacting to one outlier shift. Compared to just looking at box scores, this model gives a fairer and more complete picture. This is a solid starting point—I didn’t include power play data or goalie stats— providing considerable potential for development. 

Code

The code is publicly  available in the repository below: https://github.com/Dimi Baguette/Kalman-Filter 

References  

[1]Kalman, R. E. (1960). A New Approach to Linear Filtering and Prediction Problems.  Journal of Basic Engineering, 82(1), 35–45.   https://doi.org/10.1115/1.3662552

[2]Welch, G., & Bishop, G. (1995). An Introduction to the Kalman Filter.  University of North Carolina at Chapel Hill.   https://www.cs.unc.edu/~welch/media/pdf/kalman_intro.pdf

[3]DraftKings Engineering. Kalman Filters for NBA Player Ratings.  https://www.draftkings.com/playbook/nba/kalman-filters-nba-player-ratings  

[4]GitHub Repository – Kalman Filter for NHL Player Ratings:   https://github.com/Dimi-Baguette/Kalman-Filter

[5]Natural Stat Trick – NHL Shift and Player Stats:   https://www.naturalstattrick.com

[6]Simon, D. (2006). Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches.  Wiley Publishing.   https://onlinelibrary.wiley.com/doi/book/10.1002/0470045345

About the author

Dimitri Thivaios

Dimitri is a UK born French citizen living in the US. He is currently studying at Mamorenck High School in NY, expecting to graduate in 2026. Dimitri has a strong interest in computer science, applied mathematics, and data analysis and he’s a passionate ice hockey player and captain on the varsity team.

The post Using a Kalman Filter to Rate NHL Players appeared first on Exploratio Journal.

Abstract

Previous research has demonstrated that NBA payroll size positively impacts team championship contention and team success. However, the 2025 NBA Finals, which featured the Indiana Pacers and the Oklahoma City Thunder, provided the necessity to re-examine this correlation because neither teams’ total payroll was near the top of the league in the 2024–25 season. This paper investigates the impacts of NBA team payrolls, relative to the league salary cap, on team success as measured by regular season wins. Utilizing 11 years of financial and performance-related data from the 2014–15 to the 2024–25 season (inclusive), the study quantifies the correlation between spending and on-court results, and measures how well each team outperforms or underperforms their expected seasonal win totals based on annual spending. The study also investigates how the impacts of payroll efficiency on expected wins differ across various markets. Simulation results on different market sizes reveal that although higher payrolls remain generally associated with more wins, efficient resource management and strategic innovation are becoming increasingly more important.

1. Introduction

Over the past 11 years, the financial environment of the National Basketball Association (NBA) has undergone a dramatic transformation. This has altered the framework in which teams operate both on and off the court. The league, once characterized by predictable hierarchies based on spending, now reflects a more dynamic and uncertain battleground where new rules, economic incentives and roster strategies are changing. Not only has the salary cap steadily risen, but evolving collective bargaining agreements have also introduced mechanisms to promote parity and fiscal responsibility, such as luxury tax penalties and cap-smoothing regulations. In turn, team-building philosophies are continually adapting in response.

At the core of NBA team strategy lies a tension between financial investment and competitive success. Ownership groups have long discussed the merits of exceeding salary cap thresholds and incurring luxury tax payments in pursuit of a championship. Historically, empirical research has supported this perspective: a strong positive correlation between payroll spending and regular-season wins suggested that money could indeed provide an advantage in the standings (Gao, 2017), as exemplified by the Golden State Warriors’ championship success from the 2014–15 to the 2017–18 NBA season. The prevailing wisdom supported a team’s decision to stack rosters with elite talent in order to maximize the probability of postseason success.

Yet, the NBA is not static, and recent seasons have supported this idea. The 2024–25 campaign, specifically, delivers a striking counter-example: for the first time in the modern era, both NBA Finals teams, the Indiana Pacers and the Oklahoma City Thunder (OKC), reached the championship without paying any luxury tax charges. This development signals a broader shift: careful roster construction, player development and organizational discipline can offer a viable path to contention despite financial limitations. The 2025 NBA Finals raise questions about the key factors of success: has the relationship between payroll and success weakened, or is this season nothing more than an anomaly?

By tracing the relationship between team payroll and regular-season wins across eleven seasons, this research dives into whether spending is still a predictive factor for competitiveness. Each season is analyzed as a unique competitive environment, accounting for contextual changes that affect both individual team behavior and overall league economics.

The analysis compares the recent triumphs of non-luxury-tax teams within this historical and strategic context, illuminating larger trends in NBA parity and fiscal management. By looking into both the enduring and changing aspects of the payroll-to-wins paradigm, this study provides insights to help understand the future of basketball competition.

2. Related Works

Previous research on NBA payroll and performance has primarily focused on the overall relationship between team payroll size and winning percentage, generally confirming a positive correlation where teams with higher payrolls tend to win more. Several studies examine how salaries differ between teams, supporting the theory that salary inequality can reflect strategic investments in star players to maximize outcomes (Leon, 2025). Other research uses methods such as data envelopment analysis to evaluate how well teams convert financial and human resources into organizational success, including both on-court performance and franchise value (HarvardSports, 2023). These investigations highlight how intelligent roster construction, player development, and management practices significantly influence results beyond raw payroll numbers. Additionally, changes in Collective Bargaining Agreements (CBA) and luxury tax regulations have been studied for their impact on spending behaviors and competitive balance.

Most prior work, however, treated the league as a relatively homogenous entity or only broadly controlled for market effects without segmenting teams by detailed market tiers. Despite market size being a key variable in NBA economics and sports analytics, explicit analysis of payroll efficiency differences across large, medium, and small-market tiers remains limited.

This paper first studies the general relationship between payroll efficiency and wins, then further explores the effect of market sizes by introducing market tier as a moderating factor, offering a more detailed perspective on how spending effectiveness varies across different market sizes. It also demonstrates the analytical challenges posed by unequal team distributions among tiers and suggests more nuanced comparisons near market boundaries. These contributions provide new insights into the role of market size in NBA financial strategy and team performance.

3. Methodology

In this section, we discuss data collection and simulation methodologies.

3.1 Data Collection

Relevant data was scraped from two primary sources: Spotrac for financial data, including team payroll and salary cap figures, and Basketball-Reference for team win-loss records and additional season info (2025–26 NBA team salary cap tracker, 2025; Basketball statistics and history, n.d.). Python was used for web scraping owing to its flexible libraries and compatibility with structured data collection. Python libraries, such as requests and BeautifulSoup, were used for data scraping and cleaning of payroll, salary cap and win/loss data. Focusing on the most recent CBA and modern markets, without loss of generality, our data traces back to the history of each NBA team from the 2014–15 season to the 2024–25 season. We chose to start from the 2014–15 season to allow the immediate effects of the 2011 CBA to settle. Moreover, data sanity checks were applied using pandas to ensure year-over-year consistency and match teams across sources, resolving naming discrepancies, and verifying that payroll values aligned with reported league cap figures. Once compiled into a structured dataset, the information was imported into R for the analysis phase. Combined with R tidyverse packages, scatterplots were created to visually explore the relationship between each team’s payroll, as a share of the NBA salary cap, and regular season wins. These graphs revealed visible trends and helped identify potential correlations or outliers, making it easier to observe how spending efficiency and team success varied across different market sizes and eras.

3.2 Variable Construction

To investigate the potential effect of market size, teams were classified based on NBA market valuations and metropolitan statistical areas, according to HoopSocial (Burns, 2025). Specifically, large markets reach over 2 million homes, medium NBA markets reach between 1.5 to 2 million homes and small NBA markets reach less than 1.5 million homes. Market size categorizations were assigned to each franchise and included as categorical variables in the final analysis. Top-market (e.g., New York Knicks, Golden State Warriors, Los Angeles Lakers) and small-market franchises (e.g., New Orleans Pelicans, Indiana Pacers, OKC) were explicitly tagged based on local economic market size.

4. Experiment Results and Discussions

This section analyzes the findings on the relationship between payroll efficiency, market tier and overall cap efficiency in the NBA from 2014–15 through 2024–25. It examines the connection between payroll efficiency and wins, then compares results across market tiers, and finally evaluates advanced cap efficiency metrics to assess how teams convert spending into competitive success.

Cap efficiency is how effectively an NBA team utilizes its financial resources to achieve on-court success. It’s quantified by the residuals from a regression model that predicts team wins based on the percentage of the salary cap used. Specifically, it’s formulated as

Payroll Efficiency = team payroll / Salary Cap

4.1 Payroll Efficiency vs. Wins

Figure 1 is a scatterplot that shows NBA teams from the 2014–15 through 2014–25 seasons. Overall, teams that allot a large portion of their available salary cap to payroll generally achieve more wins during the regular season. With a correlation coefficient of r=0.52, Figure 110 illustrates a stronger relationship between payroll as a proportion of the salary cap and the number of regular season wins compared to previous studies. The figure also illustrates the growing advantage of increased payroll spending in order to maximize wins in a season. For example, in 2024, the Boston Celtics won the NBA championship with the fourth-highest season payroll at a value of $184,845,028. By handing out large contracts to star players such as Jaylen Brown and Kristaps Porzingis, the Celtics invested in impactful contributors, helping elevate the team’s overall potential.

Despite this moderate correlation, the spread of data points is significant. Teams with average payrolls have, at times, recorded high win totals through smart trades, effective coaching strategies, and strong player development. This is most clearly seen in the 2024–25 season, when neither NBA Finals teams paid the luxury tax, as shown in Figure 2.11

Figure 2 highlights the OKC and Indiana Pacers (IND) as outliers. Each team’s payroll efficiency value was situated close to league average, indicating that their payrolls relative to the salary cap weren’t among the highest in the league. Nevertheless, OKC achieved the most wins across the NBA, a surprising feat given their average payroll efficiency. This suggests that OKC was able to optimize its roster and performance well beyond spending expectations. The Pacers also secured a playoff spot by recording more wins than most teams with similar payrolls. Collectively, these two teams demonstrated exceptional efficiency and management, helping them convert average payroll investment into winning seasons and playoff appearances. As both teams adapt to changes in payroll regulation, the strength of each team’s development system also increases.

The graphs show that while increasing payroll can translate into potentially more wins, it doesn’t guarantee elite performance. Occasionally, teams that spend a lot underperform, and some mid-payroll teams overachieve. Hence, payroll is important, but not determinative.

Figure 1 and 2 underscore the relationship between payroll efficiency and wins while identifying notable outliers like the Thunder and Pacers. However, league-wide trends may differ depending on a team’s market size. To account for these structural differences, the next section explores results by market tier.

4.2 Market Tier Distribution

Categorizing teams based on market tiers allows for fairer comparisons among similar units. This enables more meaningful interpretations of trends such as payroll efficiency vs wins.

Figure 3 displays how payroll increases across different market tiers affect expected wins. Large- and medium-market teams experience greater gains in wins compared to smaller-market franchises when they increase spending. Notably, medium-market teams are projected to achieve bigger improvements when increasing payroll by 1% compared to large-market teams. Medium-market teams earn 0.334 additional wins compared to large-market teams that earn an additional 0.321 wins, a 0.013 win difference. This may be because large-market teams already operate near the limits of payroll efficiency, which means less room for further gains. Medium-market teams, on the other hand, often have more flexibility and potential to capitalize on increased spending. Small-market teams show smaller increases compared to the other two, reflecting a lower sensitivity to payroll changes. These differences could be due to factors such as small market teams’ limited access to top-tier talent, and limited flexibility for large- and small-market teams to grow because the former already maximizes payroll efficiency and the latter continues to face financial constraints. While spending more on players generally benefits teams, the exact extent varies based on market size and underlying competitive dynamics.

Figure 4 indicates that, in general, teams increasing their payroll spending by 1% can expect to win roughly 0.303 additional wins in the same season. This quantifies the practical impact of efficient payroll spending, showing how even a small increase in payroll can translate into tangible competitive benefits over the course of a full season.

Although market tier explains some variation in payroll efficiency’s effect on wins, it doesn’t fully capture a team’s performance relative to payroll expectations. For this, we examine the Cap Efficiency metric, quantifying value gained per payroll dollar against expected wins.

4.3 Cap Efficiency Metrics

Cap Efficiency represents the percent difference between expected wins and actual wins relative to payroll. A team with high Cap-Efficiency wins more games than expected for its spending, suggesting they are getting more value per dollar spent in terms of team regular season success and roster construction. Conversely, low Cap-Efficiency signals that a team is underperforming relative to the amount they spend. This metric sheds light on financial management’s strategic role in competitive outcomes.

Figure 5 visualizes the Cap-Efficiency Index for NBA teams across different seasons and Nielsen market tiers. Each heatmap block corresponds to a market tier, with teams listed along the y-axis and seasons on the x-axis. The color scale represents cap efficiency, where 100 is the league mean. Blocks that are lighter in color represent values above 100, indicating that a team is getting more wins per payroll dollar than the league average. Dark-colored blocks illustrate values below 100, suggesting less efficiency. Teams are sorted from high to low mean cap-efficiency within each market tier to highlight patterns and identify consistently efficient or inefficient franchises.

Based on Figure 5, clear outliers stand out in each market tier. For example, the Washington Wizards, despite being a large market team, win significantly less than expected. Meanwhile, teams like the Boston Celtics have won much more than payroll spending would predict compared to other large market teams. This further illustrates that wins are affected by more factors than just payroll efficiency. Additionally, among small-market teams, the OKC wins a lot more than expected in a given season, making them the most consistently successful team in the classification. In contrast, the Charlotte Hornets have underperformed relative to their spending. This highlights that teams within each market size show significant variability in how effectively their spending correlates to wins. It also allows us to conclude that additional factors beyond payroll spending play a huge role in a team’s success in any given season.

Teams across different market tiers gain varying numbers of expected wins for each 1% increase in payroll spending. The linear regression in graph 6 indicates that teams with payroll efficiency values near the league average tend to record win totals close to the league average of 41 wins. As payroll efficiency rises, so does the trend line: around every 10% increase in payroll ratio corresponds to about 2 to 3 additional regular season wins from the 2014–15 season to the 2024–25 season (inclusive). For example, teams that spend 10% above the average relative to the salary cap typically achieve a win percentage of around 0.540, or 44 to 45 wins.

5. Conclusions

This research illustrates that, historically, higher payrolls relative to the NBA salary cap are correlated with greater regular season success. However, when considering the success of non-luxury tax teams such as the Indiana Pacers and the OKC, this relationship is in flux. The evolution of the CBA has introduced stricter spending controls and harsher penalties for teams that exceed cap and tax thresholds. Now, success requires creative navigation of the CBA, emphasis on clever roster construction, internal talent growth and careful financial planning.

The insights from this analysis are valuable as the NBA’s financial environment becomes increasingly restrictive. Teams across all market sizes benefit from understanding that efficient cap management can still produce competitive and even championship-level squads. As the new CBA encourages parity and places increasing constraints on heavy-spending teams, franchises willing to innovate and maximize value are in a better position to thrive. Ultimately, the findings emphasize that while payroll remains an important ingredient, resourcefulness and a deep understanding of league rules has become even more critical for sustained NBA success.

Future research can benefit from more granular data on roster construction beyond payroll, such as player age, length of contract, injury history and performance metrics. Examining strategic adjustments to the tax penalties and stricter tax escalators would also clarify how teams optimize beyond salary allocation. Moreover, it’s important to note that the distribution of teams across market tiers is uneven, leading to skewed comparisons. More refined analyses between borderline-tier teams or pairwise comparisons would better capture market size effects on payroll efficiency.

Furthermore, while this paper considers market size as a moderator of payroll efficiency, other factors such as ownership philosophy, coaching impact and player development likely influence success. Advanced econometric models or machine learning could help better understand the complex, nonlinear relationships between financial decisions and performance. Monitoring the relationship between spending limits and competitive results will be increasingly important as the CBA tightens financial constraints. Teams able to innovate and maximize spending will be in the best position to succeed.

Acknowledgements

I would like to thank Professor Paramveer Dhillon of the University of Michigan for his inspiring guidance and encouragement throughout this research endeavor. Dr. Dhillon’s support is indispensable for the study to be completed the way it is.

References

Basketball Reference. (n.d.). Basketball statistics and history. Basketball-Reference.com. https://www.basketball-reference.com

Burns, M. (2025, June 5). 2025 NBA Team Market Size Rankings. HoopSocial. https://hoop-social.com/nba-team-market-size-rankings

CBA – National Basketball Players Association. (2023) Collective Bargaining Agreement (CBA). Nbpa.com. https://nbpa.com/cba

Gao, J. (2017). Exploring the impacts of salary allocation on team performance. Academia.edu. https://www.academia.edu/96573049/Exploring_the_Impacts_of_Salary_Allocation_on_Team_Performance

Harvardsports. (2023) Pay to Play: An Analysis of Payroll and Performance in the MLB and NBA. The Harvard Sports Analysis Collective. https://harvardsportsanalysis.org/2023/02/pay-to-play-an-analysis-of-payroll-and-performance-in-the-mlb-and-nba/

Leon, S. (2025, March 4). NBA payrolls: Spending big doesn’t always mean winning big. The Sports Cast https://thesportscast.net/2025/03/04/nba-payrolls-spending-big-doesnt-always-mean-winning-big/

Spotrac. (2025). 2025–26 NBA team salary cap tracker. Spotrac.com. https://www.spotrac.com/nba/cap

About the author

Arik Zhang

Arik is a senior at Millburn High School from Millburn, New Jersey. He is deeply passionate about statistics and its applications in business modeling and sports analysis. He also enjoys pursuing a variety of scientific research inspired by his classroom studies. He is a scholarship-holding member of the American Chemical Society and presented his work on molecular synthesis at ACS Fall 2025 in Washington, D.C.

In his capacity as a student leader, Arik is committed to fostering interdisciplinary work. As the head editor of the Sports section of his school newspaper, he integrates statistics and analysis into reporting to expand the depth of complexities of stories . As co-president of the Science Olympiad team, Arik combines rigorous scientific thinking with team-building to cultivate a collaborative, high-achieving culture.

Arik is also a strong believer in community service and education. As captain of the Millburn Limited Prep speech team, he coordinates and coaches year-round speech and debate classes at local schools. He also directed a summer speech camp for elementary- and middle-school students during the summer of 2025, the proceeds from which were donated to organizations serving local communities in-need.

The post Maximizing Wins per Dollar: A Systematic Analysis of Payroll Efficiency in the NBA appeared first on Exploratio Journal.

Review of Literature

Predicting the stock market, especially with simpler statistical analysis methods, is challenging because it exhibits extremely complex and nonlinear trends (Sawale & Rawat, 2022). Despite inherent unpredictability, numerous studies have been conducted on the ability of AI and other statistical tools to predict the market (Lin & Lobo Marques, 2024). Prediction techniques like Machine Learning (ML), Deep Learning (DL), Neural Networks (NN), Support Vector Machines (SVM), and Sentiment Analysis have been found to perform well in this nonlinear environment. These methods come at the cost of high computational needs and low interpretability (Lin & Lobo Marques, 2024).

It still needs to be determined which of these prediction techniques is best. One study achieved the highest performance with SVM when forecasting stock market returns (Arrieta et al., 2015). In contrast, a 2023 study analyzed nine different ML models in predicting the direction of Tesla’s stock price and found that the simpler method of Logistic Regression had the highest accuracy. In the study, the Random Forest model and NN also performed well (Khan et al., 2023).

Recently, more complex predictive methods like Long Short-Term Memory (LSTM) have been studied for stock market prediction (Tiwari & Chaturvedi, 2021). LSTM models have been found to outperform other ML and NN models in stock market forecasting, and they present a promising option for modeling the stock market (Sawale & Rawat, 2022).

Although advanced techniques like LSTM can make accurate predictions, they generally rely on technical analysis, with less than 25% of studies using fundamental analysis to construct their models (Lin & Lobo Marques, 2024). Creating models based on fundamental analysis is challenging because it requires explaining the reason for the movement of a stock price (Lin & Lobo Marques, 2024). However, models based on fundamental analysis can detect simple relationships between specific factors and a stock price. A study by Sohdi (2024) tested the ability of financial metrics, like growth rate, return on assets, quick ratio, and price-to-earnings ratio, to predict stock price using multiple linear regression.

The study found a significant negative correlation between quick ratio and stock price. It also suggests future research to study an expanded set of variables in predicting stock price. In a separate study, Ganguli (2011) explored the relationships between fundamental accounting metrics like earnings and valuation. Using regression, the study found a correlation between abnormal earnings, book value, and the market value of a company. It was also found that in the presence of abnormal earnings and book value, operating cash flow does not contribute to modeling market value. Ganguli (2011) recommends additional research that includes additional variables besides abnormal earnings and book value.

This project seeks to expand on the previous research exploring the relationships between accounting metrics. Specifically, the project will use linear regression to conduct fundamental analysis on publicly traded US companies. While numerous effective statistical and machine learning methods have been found to predict the stock market, many are complex, hard to interpret, and computationally intensive. Linear regression was chosen for this project because, despite its simplicity, it remains a powerful tool that can detect subtle underlying relationships. In addition, linear regression was selected because of its interpretability, which is crucial for fundamental analysis. With linear regression, this study will search for connections between a company’s market capitalization and other important data metrics like earnings per share. Ideally, linear regression will discover new relationships that can used to predict a company’s market capitalization.

Data Selection

Before fitting the first model, many features irrelevant to Market Cap were removed from the dataset. These features included Change from Open, Country, and 50-day Moving Average. Producing a model with these features would not contribute to accuracy, but it would cause a significant increase in complexity. Only predictors relevant to Market Cap were kept to keep the model interpretable. Additional features with too many missing values were also removed from the dataset. While having some missing values is inevitable, having too many would cause a higher error rate because there is less data to train on. After removing all unnecessary features, there were 15 predictors left to be used in a model.

All of the companies without Market Cap data were removed from the dataset, as the model can’t be trained without it. This means more than 3300 data entries were removed, bringing the number of entries down to ~6200. Additional entries were deleted because they contained missing data from the predictor Outstanding Shares or the predictor Performance by Half Year.

One feature absolutely integral to predicting Market Cap is a company’s earnings. This was a metric not included in the initial dataset. A model without earnings as a predictor would likely struggle to make accurate predictions. To include earnings as a feature, each company’s earnings had to be calculated using its price/earning (p/e) ratio and its number of outstanding shares. To perform this calculation, more than 3000 entries with missing p/e ratios had to be removed, bringing the final number of data entries down to 3086. This is a less-than-ideal amount of data, but still enough to create an accurate model.

With the feature Earnings calculated and added to the dataset, a few final data metrics were dropped from the set, including Earnings Per Share, Price, Price to Earnings, and Forward Price to Earnings. The final number of predictors became 10, one of which, Industry, is a categorical predictor.

Single Variable Analysis

To begin analyzing the data, a correlation heatmap was created to visually look for relationships between the features themselves and Market Cap.

The heatmap illustrates a correlation between Market Cap and Earnings and Market Cap and Outstanding Shares. Earnings and Outstanding Shares also appear to be correlated, and many of the performance statistics appear to be correlated with each other.

Scatterplots were used to look for relationships between individual predictors and Market Cap. The plots above demonstrate almost no relationship between Beta and Market Cap but a positive relationship between Earnings and Market Cap.

Now, to mathematically test each feature’s ability to predict Market Cap, models were created using simple linear regression on each predictor. Each of these models was trained using training data and then tested on a validation set of data. To split the whole dataset into a training set and a validation set, the train_test_split() function from the Sklearn module was used. Roughly 2400 entries were left for the training set, while 600 were used in the validation set. The function parameter random_state was set to 0 for all of the single variable analyses to ensure consistent results even when re-running the program.

The first feature to be tested was the categorical predictor Industry. There are 147 different possible classes a data entry can have for Industry. According to the results from training a model on Industry, almost every class was statistically insignificant in predicting Market Cap. When using the trained model with the validation set, Industry was not able to accurately predict Market Cap. In addition, a categorical variable with 147 possible categories would add significant complexity to a multi-variable model, so Industry was dropped from the dataset.

For every other feature, this process was repeated. A model was trained to predict Market Cap using that predictor, and then the model was tested on the validation set. The feature Outstanding Shares proved to be statistically significant, with a p-value of 0, and able to slightly predict Market Cap, with a testing R-squared of 0.35. The features Relative Volume and Beta had high p-values and testing R-squared close to 0. Performance by Year and Earnings both had p-values of 0, meaning they are statistically significant for predicting Market Cap. The variables Performance by Week, Performance by Month, and Performance by Quarter all had high p-values and low R-squared values. The final predictor, Performance by Half Year, had a low nonzero p-value and a low R-squared value. It may be statistically significant, but its ability to predict Market Cap may be limited.

Multi-variable Analysis

After analyzing each feature individually, a model was created that used all of the features to predict Market Cap. When tested, this model produced a testing R-squared value of 0.624 and a training R-squared value of 0.680. According to the results from training the model, Performance by Week, Performance by Month, Performance by Quarter, Relative Volume, and Beta were all statistically insignificant features. These five predictors were removed to simplify the model, leaving just four features. The cost in accuracy of removing these predictors was minuscule, with a 0.001 drop in both the training and testing R-squared values.

To continue improving this model, nonlinear and interaction terms were tested. Adding nonlinear terms to the model greatly decreased the model’s testing accuracy. The greatest decrease in accuracy was seen when a nonlinear Earnings term was added. Nonlinear terms were tested on all of the model’s features. Plots like the one below were used to look for a nonlinear trend in the data.

Interaction terms benefitted the model and improved its accuracy. An interaction term between Performance by Year and Earnings improved the model’s testing accuracy and had a p-value of zero. The interaction term between Performance by Half Year and Earnings also improved the testing accuracy while having a p-value of zero. Adding an interaction term between Earnings and Outstanding Shares did not improve the accuracy of the model’s predictions, and the term had a high p-value, meaning it is statistically insignificant.

So far, all of the training and testing have been done with the same training and validation sets. The random_state parameter of the train_test_split() function controls these specific training and validation sets. To ensure that the model sustains its accuracy with a different validation set, a for-loop was added that would iterate through multiple values for random_state. This would create different training and validation sets. The model would then be trained on the training set and tested on the validation set. The testing R-squared value was recorded for each random_state value. The mean of these R-squared values was calculated to give a more accurate idea of the model’s accuracy. For 200 different values of random_state, the average testing R-squared value was 0.629.

Standardization and Outliers

Some datasets have outlier points. These points can be outliers in the output or in the inputs. An outlier in output means that for its given inputs, a point has an output that differs greatly from what is expected. An outlier in input means that compared to the rest of the dataset, a point’s inputs are very different. Outliers have very high influence (also known as leverage) when training a model, so their presence can significantly affect the model’s estimated parameters, occasionally causing extra error. The plots below were used to check for these outliers visually. It is apparent that there are quite a few points with extremely high leverage and others with Market Cap values very far from the average (studentized residual values far from zero).

These outlier points may be affecting the model, so a new model was created that was trained off of data without these outlier points. Points with leverage greater than 0.05 and points with studentized residuals above five or below negative five were removed from the training set for this model. This model was tested using the same for-loop to iterate through different training sets and validation sets. For the same 200 different values of random_state, the model’s performance decreased. The average R-squared value fell from 0.629 (for the model trained with outliers) to 0.611 (for the model trained without outliers).

Standardization is another method that can improve model accuracy by equalizing all features. When a dataset is standardized, each data value is put in terms of its predictor’s mean and standard deviation. Since all of the data in the dataset has a similar value once standardized, every predictor is weighted equally. One predictor cannot dominate by having naturally higher values.

Another model was created and tested with the data standardized. It was tested with the same for-loop and 200 different values of random_state. This model produced an average R-squared value of 0.629, the exact same as the initial model. A fourth model was created with standardized data and with outliers removed. This model was tested using the same process and produced an average testing R-squared value of 0.611. This is the same as the model trained without outliers and without standardized data, and it is still lower than both models trained with outliers. The four plots below were created to visualize the R-squared values produced by all four models.

Both models fitted with outliers have the same higher average R-squared of 0.629. Both models trained with the outliers removed had an average R-squared value of 0.611, worse than those trained with outliers. The plots demonstrate the higher performance of models trained with outliers included.

Conclusion

This project has found that Earnings, Performance by Year, Performance by Half Year, and Number of Outstanding Shares can be used to model a company’s Market Cap. It was also found that, in combination, Earnings and Performance by Year produce an additional positive increase in Market Cap. This was modeled through an interaction term between the two. Earnings and Performance by Half Year together create a negative increase in Market Cap. This was also modeled through an interaction term between both features. This project found no accuracy benefit to adding a nonlinear term to the model. Nonlinear terms were tested for every predictor, and there was decreased accuracy every time. Finally, it was found that the model does not benefit from being trained on standardized data nor from being trained on data with outliers removed.

Outstanding Shares was an unexpected feature. This project’s hypothesis was proved correct by Outstanding Share’s presence in the most accurate model. Theoretically, Market Cap is only based on Earnings and project growth, but in practice, it can be predicted using other features.

This research was limited by a lack of usable data points. Less than one-third of the observations included in the initial dataset were suitable for the analysis. Another limitation is a lack of data from different time frames. All data was collected on December 11th, 2023, so some of this project’s conclusions may not apply to other time periods. Many patterns found in the Stock Market may be too subtle or complex to be captured by linear regression. As this project only used linear regression, its ability to detect patterns or trends was limited.

Future research would benefit from studying a wider range of data metrics and factors in predicting company valuation. A study on data across multiple years would also expand on this project and determine if the conclusions from this research can be applied in the future.

References

Arrieta, ibarra, I., & Lobato, I. N. (2015). Testing for Predictability in Financial Returns Using Statistical Learning Procedures. Journal of Time Series Analysis, 36(5), 672–686. https://doi.org/10.1111/jtsa.12120

Ganguli, S. K. (2011). Accounting Earning, Book Value and Cash Flow in Equity Valuation: An Empirical Study on CNX NIFTY Companies. IUP Journal of Accounting Research & Audit Practices, 10(3), 68–77.

Khan AH, Shah A, Ali A, Shahid R, Zahid ZU, Sharif MU, et al. (2023) A performance comparison of machine learning models for stock market prediction with novel investment strategy. PLoS ONE 18(9): e0286362. https://doi.org/10.1371/journal.pone.0286362

Larcher, Jeremy. (2023). US Stock Metrics & Performance [Data set]. Kaggle. https://doi.org/10.34740/KAGGLE/DSV/7187831

Lin, C. Y., & Lobo Marques, J. A. (2024). Stock market prediction using Artificial Intelligence: A systematic review of Systematic Reviews. Social Sciences & Humanities Open, 9. https://doi.org/10.1016/j.ssaho.2024.100864

Sawale, G. J., & Rawat, M. K. (2022). Stock Market Forecasting Using Metaheuristic LSTM Approach with Sentiment Analysis. Special Education, 2(43), 1800–1806.

Sohdi, L. R. (2024). The Influence of Growth Rate, Profitability, Liquidity, and Company Valuation on Stock Price. Jurnal Riset Akuntansi Dan Bisnis Airlangga (JRABA), 9(1), 1–23. https://doi.org/10.20473/jraba.v9i1.56477

The Finance Storyteller. (2018, November 14). Market Capitalization explained. YouTube. https://www.youtube.com/watch?v=k-Rp32j0uj8

Tiwari, S., & Chaturvedi, A. K. (2021). A Survey on LSTM-based Stock Market Prediction. Ilkogretim Online, 20(5), 1671–1677. https://doi.org/10.17051/ilkonline.2021.05.182

About the author

Andrew Jain

Andrew is a Junior in high school with strong interests in math and science. At school, he competes on the math team as well as the cross country and tennis teams. His work in statistics has been greatly assisted by a mentorship under Dr. Mohammadreza Kalan, a postdoctoral researcher at Columbia University. With the dream of using his knowledge to help the world, Andrew plans to pursue an undergraduate degree in Engineering.

The post Modeling Market Capitalization Of Public US Companies Using Publicly Available Stock Metrics And Performance Data appeared first on Exploratio Journal.

Introduction 

Objects and planets in space are much bigger than daily objects we encounter on Earth and, therefore, they experience much larger gravitational forces that cause them to orbit around, collapse on, or escape from another object. The motion of extraterrestrial objects has always intrigued me, especially the NASA DART project, which is a mission to protect the Earth from potential asteroids impacting its surface. I find the collision of objects in space very interesting because the trajectory of the objects after colliding has to take in so many factors like the mass of the objects, their velocities, and any surrounding objects. 

Before I can explain more about the NASA DART project, however, I need to introduce the basics of gravitation and space physics. I will explain the different parts of space physics, like Newton’s universal law of gravitation, the acceleration of objects due to gravitational forces of the Earth and other objects, and escape speed, the speed it takes for an object to escape an object’s orbit. I will also go into the concept of gravitational potential energy, the energy an object has while in orbit, the energy required to place an object in orbit, and the nature of objects orbiting Earth, also known as Earth satellites. Additionally, I will explain Johannes Kepler’s famous 3 laws of planetary motion for a better understanding of how planets move in space. 

Finally, I will introduce the NASA DART (Double Asteroid Redirection Test), a mission where NASA tries to develop technology to protect the Earth in the unlikely event that an asteroid is headed for Earth. Their goal is to make an object, like a satellite, hit the asteroid, thus changing the trajectory of the asteroid and making it miss the Earth. 

This report is also complemented by Python code that simulates planetary motion. It is available for download on my GitHub here: https://github.com/alyang21/solarsystem 

2. Gravitation 

2.1 Universal Law of Gravitation

On Earth, the acceleration at which an object falls toward the Earth is a constant 9.8 m/s². However, this rate is different on other extraterrestrial objects. This is because the force of gravity exerted on an object depends on its mass as well as the mass of the objects around it. Knowing this, famed physicist Sir Issac Newton derived the Universal Law of Gravitation in 1687 [8]. His equation is

where G is the universal gravitational constant, at 6.67 x 10^-11. Furthermore, this equation suggests that the force depends on both objects’ masses and how far apart they are separated.  In vector form, the equation can be written as 

Furthermore, the sum of the forces on an object by the surrounding objects is just the vector sum of all the forces. 

Figure 1.1 The sum of two vectors is found by placing the two vectors tail to tip, and the resulting vector is from the tail of the first vector to the tip of the second. [1]

2.2 Acceleration Due to Gravity of the Earth

The Earth can be visualized as a number of spherical shells centered at the same point. Since the mass of all the shells combined is the mass of the Earth, and the force of gravity by the Earth comes from the center of the Earth. By taking into account the Earth’s density using its volume and mass, we can derive that the force of gravity by the Earth on an object is F_g = (GM_Em)/R_E² [8]. Since F_g = mg where g is the acceleration by the Earth according to Newton’s second Law, 

2.3 Gravitational Potential Energy

The gravitational potential energy of an object on Earth depends on its distance from the center of the Earth. We also know that work equals force multiplied by displacement, so the work done by the Earth to bring a body of mass m from the height h2 to the height h1 is given by:

In other words, the work done on an object is the difference of potential energy from the initial to final positions of the object. If we say that the potential energy W(h) at a height h above the surface of the Earth so that W(h) = mgh + W₀ where W₀ is a constant, then W₁₂ = W(h₂) – W(h₁) [8]. It is also important to note that h = 0 means points on the surface of the Earth.

If we lift the particle along a vertical path where r₁ is the distance from the center of the Earth at its first point and r₂ is the distance from the center at its second point, then we get

And as a result, 

2.4 Escape Speed

Using the law of conservation of energy, we can find the escape speed for an object out of a planet, or the speed it needs to break through the pull of the planet [8]. If we can find the distance where the object has no more potential energy and only kinetic energy, we can set the energies of the object at those two points equal to each other, thus allowing us to find the initial velocity that the object has to leave the planet with. 

As long as the final velocity is greater than or equal to 0, the object can reach infinity. So,

The initial velocity is the minimum velocity for the object to escape the atmosphere, so

If the object is thrown from the surface of the Earth, h = 0, and 

Thus, we come to the equation 

where R_E is the radius of the Earth. This means that the escape speed is independent of the object’s own mass. Additionally, with the knowledge of the Earth’s radius, we can find that the escape speed is 11.2 km/s.

2.5 Earth Satellites

Earth satellites are objects which revolve around the Earth, usually in the shape of an ellipse. The Moon is the only natural satellite of the Earth, and it has a near-circular orbit. Other satellites have been sent up by humans for telecommunication, geophysics, and meteorology. To find the period that these satellites orbit around the Earth once, we can use the equation for centripetal force, where m is the mass of the satellite and V is its speed [8].

This centripetal force is provided by the gravitational force, similar to equation (1.1) but after substituting the variables for the mass and radius of the Earth, we get 

Setting the two equations together, we find that 

A satellite travels a distance 2πR_E with speed V if the satellite is so close to the Earth’s surface that h can be neglected. The time period the satellite takes to orbit the Earth therefore is

and using the relation g = GM/R_E², we arrive at the equation

Substituting the numerical values, we get

Which is about 85 minutes.

2.6 Energy of an Orbiting Satellite

Notice how K is positive and U_g is negative. When added up, the total energy of the satellite is 

It makes sense that the satellite’s total energy is negative because if the total energy is positive, it would leave the orbit and escape to infinity. 

2.7 Energy Required to Orbit a Satellite

The energy required to put a satellite into Earth’s orbit is the difference between the satellite’s total energy in orbit and its energy at Earth’s surface. For example, if we want to lift the 9000-kg Soyuz vehicle from the Earth’s surface up to the ISS, which is 400 km above the Earth’s surface, we would have to find its energy at the Earth’s surface, as well as its total energy in orbit at the ISS. Using Eq 1.19, we get that the total energy of the Soyuz in the same orbit as the ISS is   where m is 9000 kg and h is 0. Plugging the numbers in, we get that E_orbit is -2.65 x 10¹¹ J. The total energy at the surface is just -GmM_e/R_e because E_surface = K_surface + U_surface and K_surface is 0. Plugging the numbers in, we get E_surface = -5.63 x 10¹¹ J. As explained earlier, the energy required is the change in energy, so the energy required is   = -2.65 x 10¹¹ – (-5.63 x 10¹¹) = 2.98 x 10¹¹ J [8].

2.8 Kepler’s Laws of Planetary Motion

After German astronomer Johannes Kepler obtained the data collected by Tycho Brahe, he was able to analyze the positions of all the known planets and our moon. He realized that the orbits of the planets around the sun were elliptical, and was able to come up with three basic laws of planetary motion [8].

Kepler’s first law states that all planets orbit along an ellipse, where the Sun is one of the foci of the ellipse. An ellipse is the set of all points where the sum of the distance from each point to the two foci is a constant. 

In an elliptical orbit, the point where the planet is the closest to the Sun is called the perihelion, which is represented by point A in Figure 1.4. The figure also shows point B, the farthest point from the Sun. This point is called the aphelion. 

The ellipse is a specific example of a conic section, given by the equation

The variables r and θ from Eq. 1.20 are shown in Figure 1.5. The other two variables,   and e, are constants determined by the total energy and angular momentum of the satellite at a point on the ellipse. The constant e is the eccentricity, which determines how close to being a circle the ellipse is. The closer to 0, the more circular the ellipse is, and the closer to 1, the flatter it is.

Kepler’s second law states that over equal periods of time, a planet will sweep out equal areas. In other words, the area it sweeps divided by the time, also known as the areal velocity, is a constant.

This makes sense when you consider that when the planet is closer to the Sun, it is moving faster. Since the energy of the planet-sun system is conserved, when the planet gets closer to the sun, its gravitational potential energy decreases, so its kinetic energy and velocity must increase.

Figure 1.7 The area ∂  swept out during time   as the planet moves through angle  . The angle between the radial direction of r and   is  . [4]

Kepler’s third law states that the square of the period is proportional to the cube of the semi-major axis of the orbit. For this law, we have the equation

In this equation, a is the semi-major axis of the ellipse and T is the period. Interestingly, this law can also be derived from Newtonian principles and the principle of conservation of energy [8]. Additionally, his equation applies to any satellite orbiting any large mass, not just our Sun. If we use this equation for a circular orbit of r about the Earth, we get

3. DART Project NASA

It is widely believed that millions of years ago, the dinosaurs were put into extinction when a meteoroid hit the surface of the Earth. Although no meteor has gotten close enough to Earth since then to cause humans to panic, the scientific community agrees that another meteor will eventually cross paths with the Earth. To combat this, NASA started the Double Asteroid Redirection Test, or DART, to see if it is possible to alter the course of an asteroid by sending an object to impact it. 

I first learned about DART when I visited the Kennedy Space Center in Florida and watched a video about its mission. I was immediately intrigued by DART because I had an interest in object collisions from playing pool and baseball. The DART mission added an interesting element that wasn’t involved when playing on a flat billiards table: the gravitational force of other extraterrestrial objects. This mission pushed me to learn about gravitation, planetary motion, and overall space physics in order to understand the DART mission from a scientific perspective.

DART’s target is the binary asteroid system Didymos. Since Didymos is not on a path that would impact the Earth, it is the ideal candidate for the first planetary defense experiment. The impact would be safe, even if something were to go wrong. The asteroid system consists of two asteroids: the larger asteroid named Didymos, and its moonlet, Dimorphos. DART’s plan was to collide with the moonlet Dimorphos, and then we would examine the changes in Dimorphos’ orbit as a result of the impact. 

The journey to Dimorphos was complicated and required many different state-of-the-art technologies. One was the Small-body Maneuvering Autonomous Real Time Navigation (SMART Nav), developed for guidance, navigation, and control (GNC). The system had to be autonomous because NASA cannot control a satellite when it is 11 million kilometers away from Earth. The system was able to distinguish between Didymos and Dimorphos, and accurately navigate to the moonlet, eventually colliding with the smaller asteroid. DART was also equipped with an ion propulsion system that is solar-powered and incredibly fuel-efficient. Speaking of solar-powered, DART had a Roll-Out Solar Array (ROSA), extending 8.5 meters in length on each side. These solar arrays were used before on the ISS, but DART was the first to use them on a planetary spacecraft. Finally, the LICIACube allowed the DART team back on Earth to see images of the impact and the ejecta cloud, helping them assess the impact and its effects on Dimorphos. These technologies, paired with great antennas to send and receive data from the satellite allowed the DART mission to be incredibly successful.

Figure 1.8 The three images above show the various technologies the DART satellite used throughout its mission. SMART Nav (left) helped the satellite accurately impact Dimorphos. ROSA (center) gave the satellite its power for its ion propulsion system (right). [5]

Figure 1.9 DART would impact Dimorphos from the direction Dimorphos is moving towards, slowing it down. This would cause Dimorphos’ new orbit to be closer to Didymos since its orbiting velocity decreased. At the same time, the LICIA Cube, which DART would eject 15 days before impact, would be able to capture images of the impact and send them back to Earth for NASA to examine. [6]

Overall, the DART project was a massive success, lifting off in November 2021 and colliding with Dimorphos in September 2022. However, the mission is not complete. The DART team is still examining the data from the impact in order to explore all the effects of the impact on Dimorphos. You can watch videos about the mission at this link: https://dart.jhuapl.edu/Gallery/ [7]

References 

[1] https://tikz.net/vector_sum/

[2] https://openstax.org/books/university-physics-volume-1/pages/13-3-gravitational-potential-energy-and-total-energy

[3] https://openstax.org/books/university-physics-volume-1/pages/13-4-satellite-orbits-and-Energy

[4] https://openstax.org/books/university-physics-volume-1/pages/13-5-keplers-laws-of-Planetary-motion

[5] https://dart.jhuapl.edu/Mission/Impactor-Spacecraft.php

[6] https://dart.jhuapl.edu/Mission/index.php

[7] https://dart.jhuapl.edu/Gallery/

[8] This work is partially based on the content of this book: NCERT Books for Class 11 Physics, https://www.ncertbooks.guru/ncert-books-class-11-physics/amp/

Appendix

The following code computes the planetary motion of the Earth, Mars, and a fictional comet orbiting around the sun according to gravitational physics. The trajectories of these planets are calculated using Newton’s Universal Law of Gravitation, with given initial conditions for the position and velocities of each object. These trajectories are computed over a 5-year period and are visualized using an animation. The code is written in the Python language and is taken from this blog post: https://towardsdatascience.com/simulate-a-tiny-solar-system-with-python-fbbb68d8207b

Available on my Github page here: https://github.com/alyang21/solarsystem

# Ensure the right backend for Spyder

import matplotlib

matplotlib.use(“Qt5Agg”)

import matplotlib.pyplot as plt

from matplotlib import animation

# Constants and initial setup with constants and the objects’ masses, velocities, and gravitational constants.

G = 6.67e-11                  # constant G

Ms = 2.0e30                   # sun

Me = 5.972e24                 # earth        

Mm = 6.39e23                  # mars

Mc = 6.39e20                  # comet

AU = 1.5e11                   # earth sun distance

daysec = 24.0*60*60           # seconds of a day

e_ap_v = 29290                # earth velocity at aphelion

m_ap_v = 21970                # mars velocity at aphelion

commet_v = 7000               # comet velocity

gravconst_e = G*Me*Ms

gravconst_m = G*Mm*Ms

gravconst_c = G*Mc*Ms

# Starting positions

# earth

xe, ye, ze = 1.0167*AU, 0, 0

xve, yve, zve = 0, e_ap_v, 0

# mars

xm, ym, zm = 1.666*AU, 0, 0

xvm, yvm, zvm = 0, m_ap_v, 0

#comet

xc, yc, zc = 2*AU, 0, 0

xvc, yvc, zvc = 0, commet_v, 0

# sun

xs, ys, zs = 0, 0, 0

xvs, yvs, zvs = 0, 0, 0

t = 0.0

dt = 1*daysec

# these lists store the points that the objects are at

xelist, yelist, zelist = [], [], []

xmlist, ymlist, zmlist = [], [], []

xclist, yclist, zclist = [], [], []

xslist, yslist, zslist = [], [], []

# save the initial position in their respective lists

#earth

xelist.append(xe)

yelist.append(ye)

zelist.append(ze)

#mars

xmlist.append(xm)

ymlist.append(ym)

zmlist.append(zm)

#comet

xclist.append(xc)

yclist.append(yc)

zclist.append(zc)

# Simulation

# The new radii, forces, velocities, and positions are calculated at each second for 5 years. The new position is then added to the object’s list. 

while t < 5*365*daysec:

    ################ earth #############

    # compute G force on earth

    rx,ry,rz = xe – xs, ye – ys, ze – zs

    modr3_e = (rx**2+ry**2+rz**2)**1.5

    fx_e = -gravconst_e*rx/modr3_e     

    fy_e = -gravconst_e*ry/modr3_e

    fz_e = -gravconst_e*rz/modr3_e

    # update quantities how is this calculated?  F = ma -> a = F/m

    xve += fx_e*dt/Me

    yve += fy_e*dt/Me

    zve += fz_e*dt/Me

    # update position

    xe += xve*dt

    ye += yve*dt 

    ze += zve*dt

    # save the position in list

    xelist.append(xe)

    yelist.append(ye)

    zelist.append(ze)

    ################ mars #############

    # compute G force on mars

    rx_m,ry_m,rz_m = xm – xs, ym – ys, zm – zs

    modr3_m = (rx_m**2+ry_m**2+rz_m**2)**1.5

    fx_m = -gravconst_m*rx_m/modr3_m

    fy_m = -gravconst_m*ry_m/modr3_m

    fz_m = -gravconst_m*rz_m/modr3_m

    xvm += fx_m*dt/Mm

    yvm += fy_m*dt/Mm

    zvm += fz_m*dt/Mm

    # update position

    xm += xvm*dt

    ym += yvm*dt 

    zm += zvm*dt

    # save the position in list

    xmlist.append(xm)

    ymlist.append(ym)

    zmlist.append(zm)

    ################ comet ##############

    # compute G force on comet

    rx_c,ry_c,rz_c = xc – xs, yc – ys, zc – zs

    modr3_c = (rx_c**2+ry_c**2+rz_c**2)**1.5

    fx_c = -gravconst_c*rx_c/modr3_c

    fy_c = -gravconst_c*ry_c/modr3_c

    fz_c = -gravconst_c*rz_c/modr3_c

    xvc += fx_c*dt/Mc

    yvc += fy_c*dt/Mc

    zvc += fz_c*dt/Mc

    # update position

    xc += xvc*dt

    yc += yvc*dt 

    zc += zvc*dt

    # add to list

    xclist.append(xc)

    yclist.append(yc)

    zclist.append(zc)

    ################ the sun ###########

    # update quantities how is this calculated?  F = ma -> a = F/m

    xvs += -(fx_e+fx_m)*dt/Ms

    yvs += -(fy_e+fy_m)*dt/Ms

    zvs += -(fz_e+fz_m)*dt/Ms

    # # update position

    xs += xvs*dt

    ys += yvs*dt 

    zs += zvs*dt

    xslist.append(xs)

    yslist.append(ys)

    zslist.append(zs)

    # update dt

    t +=dt

print(‘data ready’)

# Animation setup

# grid size

fig, ax = plt.subplots(figsize=(6,6))

ax.set_aspect(‘equal’)

ax.grid()

# earth is blue. The text “Earth” follows point_e as it moves

line_e, = ax.plot([], [], lw=1, c=’blue’)

point_e, = ax.plot([AU], [0], marker=”o”, markersize=4, markeredgecolor=”blue”, markerfacecolor=”blue”)

text_e = ax.text(AU, 0, ‘Earth’)

# mars is red. The text “Mars” follows point_m as it moves

line_m, = ax.plot([], [], lw=1, c=’red’)

point_m, = ax.plot([1.666*AU], [0], marker=”o”, markersize=3, markeredgecolor=”red”, markerfacecolor=”red”)

text_m = ax.text(1.666*AU, 0, ‘Mars’)

# comet is black. The text “Comet” follows point_c as it moves

line_c, = ax.plot([],[], lw=1, c=’black’)

point_c, = ax.plot([2*AU], [0], marker=”o”, markersize=2, markeredgecolor=”black”, markerfacecolor=”black”)

text_c = ax.text(2*AU,0,’Comet’)

# the sun is yellow

point_s, = ax.plot([0], [0], marker=”o”, markersize=7, markeredgecolor=”yellow”, markerfacecolor=”yellow”)

text_s = ax.text(0, 0, ‘Sun’)

ax.axis(‘equal’)

ax.set_xlim(-3*AU, 3*AU)

ax.set_ylim(-3*AU, 3*AU)

exdata, eydata = [], []

mxdata, mydata = [], []

cxdata, cydata = [], []

# The points for each object are put into their respective data sets to be plotted on grid

def update(i):

    exdata.append(xelist[i])

    eydata.append(yelist[i])

    mxdata.append(xmlist[i])

    mydata.append(ymlist[i])

    cxdata.append(xclist[i])

    cydata.append(yclist[i])

    line_e.set_data(exdata,eydata)

    point_e.set_data(xelist[i],yelist[i])

    text_e.set_position((xelist[i],yelist[i]))

    line_m.set_data(mxdata,mydata)

    point_m.set_data(xmlist[i],ymlist[i])

    text_m.set_position((xmlist[i],ymlist[i]))

    line_c.set_data(cxdata,cydata)

    point_c.set_data(xclist[i],yclist[i])

    text_c.set_position((xclist[i],yclist[i]))

    point_s.set_data(xslist[i],yslist[i])

    text_s.set_position((xslist[i],yslist[i]))

    ax.axis(‘equal’)

    ax.set_xlim(-3*AU,3*AU)

    ax.set_ylim(-3*AU,3*AU)

    #print(i)

    return line_e,line_m,line_c,point_s,point_e,point_m,point_c,text_e,text_s,text_m,text_c

anim = animation.FuncAnimation(fig, func=update, frames=len(xelist), interval=1, blit=False)

plt.show(block=True)

About the author

Alexander Yang

Alex is currently a 12th grader at the Livingston High School. He is a dedicated singer-student-athlete with a passion for Math and Physics who is fascinated with data analysis and calculations related to aerospace. He founded his high school’s Rocketry Club, competing in the American Rocketry Challenge and also holding educational community launches to spark interest in rocketry and aerospace. Alex has been a part of his school’s Math Team for all four years of high school, and rising to the Math Honor Society’s Vice President in his Junior year. He was also a camp counselor at the Delaware Aerospace Academy, teaching young students about aviation, space, and rockets. He taught the students to construct and launch model rockets, maglev trains, and solar robots.

In addition to these activities, Alex also plays varsity baseball for his school, being the starting second baseman and starting shortstop in his sophomore and junior years respectively. He has also been an active singer, singing in his school chorus, select chorus, and an outside volunteer chorus. He has auditioned into the NJ All-State Chorus both of the last two years, and he is currently ranked 6th in the state in the Tenor 1 voice part. He is deeply interested in math, data science, physics, and computer science and would like to apply his math and physics knowledge to improve technology. Alex looks to further his knowledge and interest in STEM by studying data science related topics in higher education.

The post Space Physics: The motion of extraterrestrial objects appeared first on Exploratio Journal.