Juggling Accuracy and Stability Goals
As artificial intelligence (AI) continues to revolutionize demand forecasting, much of the current conversation has centered around maximizing forecast accuracy (minimizing forecast error). However, while AI-driven forecasts offer advanced capabilities, businesses frequently face significant adoption challenges when these forecasts fluctuate dramatically from cycle to cycle for a given future period (e.g., April 2024 cycle forecast for June 2024 month widely different from the January 2024 cycle forecast for the same June 2024 month). These sharp and often unexplainable swings create substantial distrust and disruption within critical business processes, particularly in supply chain operations such as procurement and production planning.
The primary issue arises when AI-generated forecasts shift violently between cycles, often without any clear justification (backed by credible new data, such as customer intent, competitor activity, or market conditions). These unexplainable changes wreak havoc on operational plans, resulting in substantial inefficiencies and misalignment. For example, shifts in forecasts are particularly disruptive when they do not account for actual shifts in demand but rather stem from overfitting—where random noise or minor variations in the most recent data are treated as signals. This can lead to erroneous model selections or adjustments to hyperparameters that cause instability. When AI-driven forecasts fail to deliver consistency alongside accuracy, businesses lose trust in the system, making adoption of these advanced forecasting methods more difficult.
What businesses truly need is more than just accurate forecasts; they require forecasts that strike a balance between accuracy (low error), neutrality (low bias), and stability (low churn). The demand for stability becomes especially apparent in industries with long lead times, such as manufacturing and supply chain management, where raw materials often need to be procured months in advance. For example, consider a scenario where a business has a 6-month lead time to procure raw materials and a 3-month lead time to manufacture a product. In such cases, the lag 5 forecast (which drives raw material procurement) and the lag 2 forecast (which drives production) must be consistent to avoid operational chaos. Any major disconnect between these forecasts can lead to either under- or over-procurement of resources, resulting in significant inefficiencies and costs.
Consider the following scenario: in the January 2024 forecast cycle, a business anticipates the need for 1000 units of a finished good (FG) for the month of June 2024, and raw materials are procured accordingly. However, by the April 2024 forecast cycle, the forecast unexpectedly jumps to 1500 units for the same month. Without the necessary raw materials to support this new forecast, the business faces a costly scramble to expedite materials or risks lost sales. Conversely, if the forecast drops to 500 units in the April cycle, the company may find itself with excess inventory, forcing it to cancel orders and deal with irate suppliers. These types of forecasting swings, particularly when they occur without a clear explanation, create a lack of confidence in the AI forecasting system. Businesses require credible insights into what caused these fluctuations and a consistent forecast to plan effectively.
To address these challenges, vyan.ai approach to developing forecasts goes beyond simply optimizing for low error. While accuracy is essential, businesses also need forecasts that are neutral (minimizing bias) and stable (minimizing churn) across cycles. Our approach allows users to define a demand forecast optimization objective function that incorporates all three of these critical components: accuracy (low error), neutrality (low bias), and stability (low churn) and define the mix differently for different parts of your product portfolio. This multi-objective optimization ensures that forecasts are not only accurate, but also reliable and consistent over time, in-sync with your business objectives.
We explore all the candidate forecasts within the forecasting universe, ranging from traditional statistical models to advanced machine learning (ML) and deep learning (DL) techniques. Our approach evaluates these forecasts across all relevant dimensions: at various hierarchy levels, across different time buckets, and in a lag-specific manner. By examining historical performance across key indicators of error, bias, and churn for each forecast, we can measure how well each model/level/time bucket performs in different conditions.
To provide a comprehensive measure of forecast quality, we define a composite Key Performance Indicator (KPI) called Forecast Discrepancy—a measure that quantifies the performance of each forecast model based on inherent error, bias, and churn. By blending the results from all candidate forecasts and ensemble forecasts, we generate the AI Optimal Forecast—the forecast that has the highest Forecast Quality score (lowest Forecast Discrepancy). This ensures that the forecast is accurate, balanced, and stable, addressing the full range of challenges businesses face in demand forecasting.
This AI Optimal Forecasting approach significantly reduces the uncertainty and unnecessary nervousness that often plagues supply chain and procurement planning. By delivering more stable and reliable forecasts, businesses can plan more effectively without sacrificing accuracy. Additionally, this approach helps foster trust and adoption among business teams, who are often skeptical of AI-driven forecasts when they fluctuate dramatically. As forecast quality improves, businesses gain greater confidence in the AI models, leading to smoother operations, better decision-making, and improved overall performance.
To further explore how AI Optimal Forecasting approach can help your business, we invite you to join our upcoming webinar on September 24th, 2024. In this session, we will dive deeper into our methodology and demonstrate how our solution can enhance the performance of your forecasting processes. Register here to secure your spot.
