Last Updated: 2/6/2023

Forecasting

What is Forecasting?

Machine learning forecasting is a way to use computers to make predictions in the future. To do this, we feed the computer a bunch of data from the past, and then the computer uses that data to learn how different things might be related. Then, when we want to predict the future, we give the computer some new data, and it uses what it has learned to guess what might happen next (in a few hours, days, months, and so on). For example, we want to use machine learning to forecast the weather. We might give the computer a bunch of past weather data, such as temperature, humidity, and wind speed. The computer can then learn how these different factors are related and use that knowledge to predict what the weather will be like in the future. Another example might be using machine learning to forecast stock prices. We could give the computer data about the stock market from the past, such as the prices of different stocks and the overall state of the economy. The computer can then learn how these factors are related and use that knowledge to predict what stock prices might do in the future. Overall, machine learning forecasting is a way for computers to use past data to make intelligent guesses about what might happen in the future.

"A forecast is a prediction of some future event or events." — Time Series Analysis and Forecasting, Douglas C. Montgomery, Cheryl L. Jennings, Murat Kulahci.

Forecasting or Time Series Analysis predicts future events or values based on knowledge from the past. If a prediction problem involves a DateTime variable, it should probably be solved with forecasting. Forecasting problems or time series analysis consist of sorted data series. The algorithm's job is to take a data sequence as input and predict the sequence's next value(s). This sequence can be position-sorted like text but typically is sorted by DateTime.

If we want to predict tomorrow's weather, we need weather information from previous days. If we're going to predict the number of users who will visit a website in the next hour, we need to know who has visited the website in the last hours or even days. The most common way to train these models is to divide the time series into intervals called windows. These windows can be in minutes, hours, days, or the unit we want to predict. Each window takes a specific number of values from the series; the label is the value located right after this window. In Forecasting problems this is called Features extraction but don't worry for now, we will return to this later.

Although we already mentioned animals, classifying birds and amphibians are an excellent example to explain that not only images could be used as input data, but audio signals can also be used to classify species or detect endangered species in a forest. Currently, sound classification algorithms allow audio to be classified according to the species or even to detect any human intervention, such as vehicles and chainsaws in protected areas or national parks.

By nature, many phenomena have repetitive behaviors. Stock market share prices or weather can repeat the same behavior every week, month, or year; however, it is common to have some historical trends. In the case of the weather, it can be a rise in temperature due to global warming. In stocks, it can be a downward trend if a company is in financial trouble or a rising trend if it is growing. Analyzing time series is an exciting task, especially when it comes to people's behavior at certain times of the year. In forecasting, several key terms are often used to describe different aspects of the time series:

• Level: The level refers to the baseline value of the data. It can be thought of as the "average" value of the data over a long period of time. The level is the baseline value if the series were a straight line, and every time series has one.
• Noise: Noise refers to random fluctuations in the data that are not part of any underlying trend or pattern. These fluctuations can make it challenging to forecast the data accurately. A minimum knowledge of the problem's context is required to identify them with security.
• Trend: A trend refers to a long-term pattern in the data, such as an upward or downward movement. Identifying trends can be helpful, as they can help to indicate which direction the data is likely to move in the future.
• Seasonality: Seasonality refers to regular patterns in the data that occur at specific times of the year. For example, a retailer might have a seasonal pattern in sales data, with higher sales during the holiday season and lower sales during other parts of the year. Understanding seasonality is extremely important for making accurate forecasts.

Forecasting problems can also be classified depending on how far into the future they make their predictions:

• Short-term forecasting problems involve predicting events only a few periods into the future: days, weeks, and months.
• Medium-term forecasts could extend from 1 to 2 years into the future.
• Long-term extend many years into the future.

Examples

1. Artificial intelligence (AI) can be used to solve a wide range of forecasting problems. For example: Weather forecasting: AI can help us to analyze historical weather data and predict what the weather will be like in the next days or months.
2. Stock market forecasting: AI can analyze financial data and make predictions about how stock prices will change over time. Also, Investors can forecast the returns from their investments. These assets include, but are not limited to, stocks, bonds, and commodities.

3. Sales forecasting: AI is a great tool to analyze sales data and predict how much a company will sell in the future. Organizations can use product sales forecasts to schedule production, control inventories, manage the supply chain, determine staffing requirements, and plan capacity

4. Demand forecasting: AI can work with consumer demand data and predict how much of a product or service a store will need in the future.
5. Traffic forecasting: they can also analyze traffic patterns and predict how traffic will flow in the future. With this information, we could predict and solve bottlenecks before they occur.
6. Agricultural forecasting: AI can study crop yield data and predict how much food a field will produce in the future.

As humans, we always try to approximate the weather based on the previous days. For example, if the last three days it has been sunny, we use fresh clothes to leave the house, if, on the contrary, it has been raining the previous days, we have to make sure to take an umbrella before leaving our house, even if it is not raining at that moment. Forecasting can be important in many different fields and can help people make informed decisions about a wide range of activities.

Forecasting in Real Life

1. [Kaggle] Store Sales - Time Series Forecasting
2. [Keras] Walmart Sales Forecast
3. [Kaggle] Forecasts for Product Demand
4. [Kaggle] Food Demand Forecasting
5. [Kaggle] Time Series Forecasting with Yahoo Stock Price
6. [Keras] Timeseries forecasting for weather prediction

If you want to learn more about Forecasting you should check out the following resources:

Textbooks

Videos

Podcasts