Measuring the Forecast Accuracy

Updated: Aug 18

Agenda:

  1. What is the probability of a YUCE-8 forecast?

  2. Is there an overview?

  3. How does a 1-hour forecast work with a 4-hour forecast etc.?

  4. What if I have more questions?




What is the Probability of a YUCE-8 Forecast?

Let's have a look at the current chart of the week. On June 12th the randomly selected asset is LTC-USD with a time period of 1 day:

  • The forecast was created on 1st of June, 23:59 UTC

  • It forecasted a very bearish scenario for LTC-USD until the 18th of June

  • If you zoom into the chart you can see that the last closing price of LTC-USD was approx. 182 USD:




This is the full chart. As you can see the forecast of LTC-USD on 06-01 | 23:59 was extremely correct (in contrast to many, many Twitter influencers who predicted a massive bull run).



Now, how accurate is a forecast? The short answer: there's no guarantee. The only thing we can do is to measure the success of the previous forecasts. And that's calculated on a regular basis. The latest results of this calculation can be found in this heatmap:

The success of a model is determined by using a backtesting algorithm. After a new model is trained the algorithm will check how precise the forecast was in the last 90 timesteps.

Let’s say today is t=-90 and a new model finished training. Then the backtesting algorithm will match the forecast of t=-89, t=-88, … with the real closing prices.

If the forecast of t=-70 is bearish for let's say t=-60 and both, the real closing price of t=-60 and the forecasted closing price of t=-60 is lower than the price of t=-70, then "timestep +10" will receive 1 point.


For LTC-USD 1d(ay) the resulting value is 70. That means that in 70 out of 100 cases during backtesting the YUCE-8 AI model predicted the a right direction of LTC-USD at timestep +10.




Is there an overview?

Whenever our AI models get (re-)trained the backtesting result is published to this website. Every backtest tests the last 90 timesteps. The heatmap groups all models into 4 classes:

  • LOW: The backtesting result is < 54%. That is just a little bit better than a flip coin.

  • MEDIUM: The backtesting result is between 54% and 70%. These results are OK.

  • HIGH: The backtesting result is between 70% and 80%. These results are really good!

  • ULTRA HIGH: The backtesting result is >= 80%. A theoretical money press.



How does a 1-hour forecast work with a 4-hour forecast etc.?

Let's have a look at QQQ and let's start with the 1-hour forecast. The forecast starts at 3am. We can see a mixture of red and yellow zones. The forecast was was executed when the price is calculated* to be 365.53. There are 2 supports (dotted lines) and the forecast starts above both dotted lines. The majority of the forecasted data points is yellow. The red dips indicate a test of both support areas.


This forecast reaches from 3am to 3pm. The backtested accuracy of QQQ's 1-hour forecast reaches from LOW to LOW. That means, that the forecasts of the last 90 hours were not better than a flip coin. It doesn't mean that the current forecast is random and not true. The correctness of this forecast will be measurable after 3pm.


QQQ 1-hour

Now comes the 4-hour forecast. It's starts at 3am, too, but with a slightly different starting price**. From a 4-hour perspective we see the price moving between 2 dotted trend lines. The upper one acts as a resistance line, the lower one is a support line.

The forecasts sees similar minimum and maximum prices for QQQ but within the range of the next 48 hours. The majority of all data points is bullish (yellow). It expects 2 dips to re-test the support line.


The backtested accuracy of QQQ's 4-hour forecast is in the MEDIUM range. That means that all within the last 4 x 90 hours the majority of all forecasts were correct between 54% and 70%. It doesn't mean that the current forecast is true. The correctness of this forecast will be measurable after 08-20 3am.


QQQ 4-hour

Finally we have a look at the 1-day forecast. The 1-day forecasts are executed at 3:30am CET. The starting price is at approx. 365 and is within 2 dotted trend lines, too. There a now forecasted red dips. The complete forecast is bullish with a maximum expected price of 372 USD.


The backtested accuracy of QQQ's 1-day forecast is in the MEDIUM - HIGH range. That means that all within the last 90 days the majority of all 1-day forecasts were correct between 54% and 80%. It doesn't mean that the current forecast is true. The correctness of this forecast will be measurable after September, 3rd.


QQQ 1-day


What if I have more questions?

Whenever you have a question don't hesitate to raise it. We created a Discord channel for your questions, remarks etc.





* we call it calculated because QQQ is not traded during after hours. We estimate the price with various methods.

** this happens when the forecast scheduler executes a specific forecast some secondslater (GPU limits etc.)