How to forecast the short-term future of stocks & cryptos with AI

The nature of stock market data is determined by ups and downs. Geopolitical effects and investor sentiments influence prices. Today you have access to Terabytes of data in real time. But the human brain is not capable of processing this amount of information. As a result, traders and risk managers get overrun by data.

Whilst the human brain gets overrun by the amount of data that is produced per second another technology is waiting for its deployment: AI. AI is able to learn from the past in order to predict the future. But financial data is difficult to predict. Every asset is different and operational success requires experienced traders. Building an AI driven platform requires lots of time, lots of developers and years of patience.


Historical vs. future data

There vast majority of available datapoints are historical datapoints. Every Terabyte of data was created in history. An analyst needs to access the data, filter the datapoints and finally run the analysis. The result is an individual assessment and mitigation of investment consequences. We call that future data.

Every farmer knows that the best time to get seeds in the ground is when the soil is relatively dry, but right before a good soaking rain. Competition exists in agriculture, too. If farmers were today’s investors every farmer would collect datapoints such as air pressure or humidity to calculate his own probability of a rainy tomorrow. And then they would wonder, why the big farmers are always winning with their AI.


About Y8 and IEX Cloud

Y8 has built several AI models that predict the most likely future of various assets on a 24/7 basis. The main input parameters are market data, technical analysis data, asset specific data (such as blockchain data) and a secret sauce. The models are continuously improved and back tested.

Searching for a reliable and easy to integrate market data provider leads to IEX Cloud. Within 1 hour our Python based backend was connected to IEX Cloud’s data pipeline.


Findings

The day-to-day trading market is driven by approx. 70% algorithmic trading. In 2020 Citadel said, that retail is 25% during peaks. Both participants leave traces in historical data. These traces explain how and when most market participants reacted to certain data points in the past. The sum of these traces is a “hidden” script, that was written with invisible ink. Y8 aims to reveal these messages.

Y8 has different models for the same asset. An algorithm chooses the right model for the right situation. A situation is determined by several parameters such as regime class and volatility.

Let’s have a look at a specific model trained for QQQ and 4-hour time periods. Y8 always forecasts 13 future data points. The black dots are the real changes from one 4-hour candle to the next. The blue dots are the forecasted minimum changes by Y8.

Here’s a scatter plot of real price changes for timestep +3 in a back testing scenario where Y8 didn’t know what’s coming next. 74% of all blue dots (= forecasted price changes for this future timestep) are above / below 1 when the black dots (= real price changes) are above / below 1. That means, that in 74% of all cases QQQ closed higher 3 timesteps later when the forecast predicted a bullish scenario. In 36% of these back tested cases the Y8 prediction was wrong.



The back test shows these results for QQQ / 4-hour for the last 90 timesteps:

Y8 combines these forecastes with a predicted volatility, 2 trend lines that a number of support zones. The more exact the real prices (blue lines) move within the forecasted red and yellow areas the more exact was the secret trace predicted by Y8. Here’s a QQQ 4-hour forecast published on 08–19 | 17:29:


Connecting to IEX Cloud and Y8 to obtain future data

Developers can access all forecasts by using Y8’s REST API. In the following code we are going to use Y8’s Python API and combine it with the PyPI module iexfinance.

Let’s start with importing the necessary libraries:



from iexfinance.stocks import get_historical_data
from y8_client_lib import Y8_API_Client as y8_client
from y8_client_lib import Y8_API_helpers as y8_helpers
# You must initialize your client with the subscription username:
client = y8_client.Y8_API_Client(‘your subscription token‘)
# Once initialized you can invoke its functions:
version = client.get_backend_version()
symbols = client.get_supported_symbols()

The supported symbols have a unique identifier and an array of supported time steps:


{AAPL: [1d”,4hour”,1hour”],BTCUSD: [1d”,

The API let’s you query the latest forecast per asset as well as one of the last 100 historical forecasts published per asset. We’ll use an historical example now to explain the trigger points.


Here’s the visualized chart of QQQ from the Y8 website which was forecasted on the 28th of July for the next 13 trading days:

  • The forecast was executed with a closing price of $362.04 and indicates a dip to $358 (red zone) on the 30th of July, followed by a bullish trend up to $375 (yellow zone) until the 13th of August.

  • There’s a solid support line calculated by YUCE-8 at $362.65 (solid line)

  • The current scenario sees a supporting trend (dotted line #1) at approx. $355 and a bullish trend coming from $358 and ending at approx. $377 (dotted line #2)

  • The real prices that occured after the forecast (blue lines) show the accuracy of the forecasted „most likely“ scenario that was calculated on the 28th of July.

We can collect the same information by calling the API. The Y8 symbol for QQQ is NASDAQ-QQQ. We are requesting forecasts on an end of day basis (1d) and the 28th of July is 8 trading days back in history:

f_0 = client.get_historical_forecast(NASDAQ-QQQ,1d’, 8)

Print the forecast start and the starting price:

print(‘start of forecast:, forecast_start_Date)
print(‘price at forecast:, forecast_start_Close)
start of forecast: 20210730 01:00:00
price at forecast: 362.04


Further we can extract the calculated support and trend lines from the returned JSON:


support_levels = f_0[‘extra’][‘support_levels’]
trend_1 = y8_helpers.extend_trendline(f_0[‘extra’][‘trend_1’])
trend_2 = y8_helpers.extend_trendline(f_0[‘extra’][‘trend_2’])


The red bear and yellow bull zones can be calculated as follows:


bull_bear_zone = []
for i in range(0, len(top)):
  max_ = top[i] if top[i] >= bottom[i] else bottom[i]
  min_ = top[i] if top[i] < bottom[i] else bottom[i]
  bull_bear_zone.append([min_, max_])


The output of these zones is an array:


 [[360.23, 362.04], [358.01, 362.04], [362.04, 370.88],

Or it can simply expressed with +/- symbols:

bb_description = []
for bb in bull_bear_zone:
  if bb[0] < forecast_start_Close:
    bb_description.append(-)
  else:
    bb_description.append(+)
Output:
[-,-,+,+,+,+,+,+,+,+,+,+,+]

Now, let’s combine the Y8 forecasts with IEX Cloud’s data:


df = get_historical_data(QQQ, start=y8_helpers.to_datetime(forecast_start_Date)timedelta(hours=24), token=token)


The result can be combined in order to analyse the past, train bots or find the best entry and exit point.


Links