I Know First Evaluation Report On Currencies Forecasts – Predictability And Signal Together Beat Benchmark
Executive Summary
In this live forecast evaluation report we will examine the performance of the forecasts generated by the I Know First AI Algorithm for the Forex market that we sent to our customers on a daily basis. Our analysis covers time period from January 2, 2018 to August 24, 2018.We will start with an introduction to our asset picking and benchmarking methods and then apply it to the currency pairs universe covered by I Know First’s “Currencies” package and compare it to the benchmark performance over the same period.
About the I Know First Algorithm
The I Know First self-learning algorithm analyzes, models, and predicts the stock market. The algorithm is based on Artificial Intelligence (AI) and Machine Learning (ML), and incorporates elements of Artificial Neural Networks and Genetic Algorithms.
The system outputs the predicted trend as a number, positive or negative, along with a wave chart that predicts how the waves will overlap the trend. This helps the trader to decide which direction to trade, at what point to enter the trade, and when to exit. Since the model is 100% empirical, the results are based only on factual data, thereby avoiding any biases or emotions that may accompany human derived assumptions. The human factor is only involved in building the mathematical framework and providing the initial set of inputs and outputs to the system. The algorithm produces a forecast with a signal and a predictability indicator. The signal is the number in the middle of the box. The predictability is the number at the bottom of the box. At the top, a specific asset is identified. This format is consistent across all predictions.
Our algorithm provides two independent indicators for each asset – Signal and Predictability.
The Signal is the predicted strength and direction of movement of the asset. Measured from -inf to +inf.
The predictability indicates our confidence in that result. It is a Pearson correlation coefficient between past algorithmic performance and actual market movement. Measured from -1 to 1.
You can find the detailed description of our heatmap here.
The Asset Picking Method
The method in this evaluation is as follows:
We take the top X most predictable exchange rates pairs, and from them we pick the top Y highest signals.
By doing so we focus on the most predictable assets on the one hand, while capturing the ones with the highest signal on the other.
For example, a top 30 predictability filter with a top 10 signal filter means that on each day we take only the 30 most predictable assets, and then we pick from them the top 10 assets with the highest absolute signals.
We use absolute signals since these strategies are long and short ones. If the signal is positive, then we buy and, if negative, we short.
The Performance Evaluation Method
We perform evaluations on the individual forecast level. It means that we calculate what would be the return of each forecast we have issued for each horizon in the testing period. Then, we take the average of those results by strategy and forecast horizon.
For example, to evaluate the performance of our 1-month forecasts, we calculate the return of each trade by using this formula:
This simulates a client purchasing the asset based on our prediction and selling it exactly 1 month in the future.
We iterate this calculation for all trading days in the analyzed period and average the results.
Note that this evaluation does not take a set portfolio and follow it. This is a different evaluation method at the individual forecast level.
The Benchmarking Method
The theory behind our benchmarking method is the “Null hypothesis“, meaning buying every asset in the particular asset universe regardless of our I Know First indicators.
In comparison, only when our signals are of high signal strength and high predictability, then the particular currencies pairs should be bought (or shorted).
The ratio of our signals trading results to benchmark results indicates the quality of the system and our indicators.
Example: A benchmark for the 3d horizon means buy on each day and sell exactly 3 business days afterwards. We then average the results to get the benchmark. This is in order to get an apples to apples comparison.
Asset universe under consideration – Currencies
In this report we conduct testing for the 50 currencies pairs covered by I Know First in “Currencies” package. The package includes the major worldwide traded currencies such as USD, EUR, UK pound, etc.
Evaluating the predictability indicator
We conduct our research for the period from January 2, 2018 to August 24, 2018. Following the methodology described in the previous sections, we start our analysis with computing the performance of the algorithm’s long and short signals for time horizons ranging from 3 days to 3 months without considering the signal indicator. Therefore, we applied filtering by the predictability indicator for 5 different levels to investigate its sole marginal contribution in terms of return as different filters are applied. Afterwards, we calculated the returns for the same time horizons for the benchmark using the currencies universe and compared it against the performance of the filtered sets of assets. Our findings are summarized in the table below:
Figure 6 -1 Currencies Predictability Effect On Return
From the above table we can observe that generally the marginal predictability effect is double increases of the average return with the increase of the time horizon for both Top 50 and Top 30 assets subsets filtered by predictability for periods all periods. The maximum performance was recorded for Top 50 currencies pairs at 3-months’ horizon – 0.97%. Based on the above we continue our study in order to identify whether the results could be improved in case of Top 30 currencies pairs when we apply filter by signal indicator.
Evaluating the Signal indicator
In this section we will demonstrate how adding the signal indicator to our asset picking method improves the above performance even further. It is also important to measure the outperformance relative to the benchmark and for that we will apply the formula:
Therefore, we applied filtering by signal strength to the Top 30 assets filtered previously by predictability. The results of the testing showed that there is a significant positive marginal effect on the assets’ return, especially in the case of the 1-month’s investment horizon. We present our findings in the following table and charts (Figure 6-2).
Figures 6-2 Currencies Key Performance Indicators Summary
Average returns per time horizon (3 days to 2 weeks), predictability & signal filters
Average returns per time horizon (1 month and 3 months), predictability & signal filters
Out-performance per time horizon (3 days to 3 months), predictability & signal filters
Average hit ratio per time horizon, predictability & signal filters
From the above set of charts, we can clearly see that if we apply signal strength filtering to the currencies’ universe, the subsets of Top 10 and Top 5 assets will start to produce greater returns than the benchmark with increase of time horizon. As soon as we start to consider longer time horizons, we see that the return of the Top 5 subset at 3-months’ period make significant jump comparing to the shorter ones and ultimately reaches 4.03%. At the same time, we observe that the returns of the Top 10 subset for all the considered periods demonstrate increasing trend going above the benchmark’s results for all horizons up to 1-month period reaching its absolute maximum at 1.94% for 3-months’ period. As a result, the highest out-performance over the considered benchmark was produced by Top 5 assets by signal with some 190% at 1-month time horizon. Finally, the hit ratio follows similar to out-performance pattern and we observe its peak values for both Top 10 and Top 5 sets on 1-month time horizon – 61.483% and 61.50%, respectively, comparing to the benchmark’s 57.11%.
Conclusion
In this analysis, we demonstrated the out-performance of our forecasts for the currencies pairs from Currencies universe picked by I Know First’s AI Algorithm during the period from January 1, 2018 to August 24, 2018. Based on the presented observations we record significant out-performance of the Top 5 and Top 10 currencies pairs when our predictability and signal indicators are coupled to be used as an investment criterion. As shown in the above diagram, the Top 5 currencies pairs filtered by predictability and signal yield significantly higher return than any other asset subset on all considered time horizons spanning from 3 days to 3 months. Therefore, an investor who wants to critically improve the structure of his investments into Forex market within his portfolio can do so by simultaneously utilizing the I Know First predictability and signal indicators as criteria for identifying the best performing currency pairs.