The challenge of tackling illegal transactions using AI: Narrowing down ”suspicious transactions” from tens of millions of securities transactions per day
May 07, 2018
The number of share orders in the Tokyo Stock Exchange market exceeds tens of millions per day, and could reach at most a hundred million a day. The challenge lies in how to identify potentially illegal ones from among all these transactions. Tokyo Stock Exchange operator Japan Exchange Group (JPX) has launched full-scale operations of an artificial intelligence (AI) system to support market surveillance operations for securities transactions in March 2018. The system uses NEC’s AI technology centered on RAPID Machine Learning as its platform AI technology.
Value differs depending on which particular market surveillance operation AI is used
Financial instruments exchanges managing share trading markets are focusing on market surveillance operations to identify market price manipulations and other illegal transactions in order to guarantee fare and impartial trading. The Japan Exchange Regulation, which performs market surveillance operations for the Tokyo Stock Exchange market, has been actively incorporating IT in its operations by using a screening system to identify suspicious transactions based on certain criteria. It was Mr. Takashi Watanabe, however, of the Department of Market Surveillance and Compliance of the Japan Exchange Regulation who realized that there is a substantial need for adopting newer technologies in the market surveillance field.
”With the changing environment surrounding the stock market, and particularly with advancements in IT, there has been a steady increase in stock market orders. Major events that affect stock prices, such as the exit of UK from EU and the US Presidential elections, also lead to surge in orders to almost 100 million a day for the Japanese stock market. Recently, the use of computer software to perform algorithmic trading for automatically issuing orders has become commonplace. In response to the increase in trade volume due to use of technology, I think it has become imperative to further improve efficiency of operations by also using systems that leverage technology for market surveillance.” (Mr. Watanabe)
Thus far, market surveillance operations have been performed by first extracting a wide range of ”potentially illegal transactions” based on certain criteria from the massive amount of trading data using existing screening systems. From these possibilities, an inspector then checks the individual trading conditions to eliminate transactions that can be judged without doubt as legal in ”preliminary investigations” and identify transactions that remain suspicious for further screening in ”full-scale investigations.”
”By incorporating previous market surveillance knowhow, existing screening systems that extract suspicious transactions based on certain criteria have been able to identify a significant number of doubtful transactions from tens of millions or even a hundred million transactions. The rise in the number of transactions, however, leads to an increase in the number of suspicious transactions extracted based ona fixed set of criteria, thereby leading to a heavier burden on inspectors in manually checking and analyzing individual transactions. Thus, based on the recent increase in trade volume, it is foreseen that there will be a considerable burden on inspectors in performing checks even only for the preliminary investigation stage.” (Mr. Watanabe)
Despite the increasing number of transactions that need to be processed, the mission of financial instruments exchanges to catch all illegal transactions remains unchanged. This is where the use of AI in preliminary investigations was conceived, in particular, to assign scores for the level of doubtfulness of transactions, in order to achieve efficiency in operations.
”Ideally, it would be best if the coverage of AI operations would also include making the report to the Securities and Exchange Surveillance Commission. Full-scale investigations, compared to preliminary investigations however, involve a more comprehensive understanding and analysis of a wider and deeper range of trading conditions to be able to determine the possibility of illegality—a task that we were skeptical that AI could do. Also, I believe that the Japanese stock market is not a black market that is rampant with illegal transactions, and, in fact, the number of illegal transactions discovered by the Securities and Exchange Surveillance Commission is not substantial. Due to the limited amount of learning data available for determining the illegality of transactions at the full-scale investigation level, it was not realistic to use AI for learning the decisions made by inspectors during full-scale investigations. Thus, we took a different approach and aimed at optimizing operations by having AI handle the preliminary investigations to enable inspectors to focus more on full-scale investigations.” (Mr. Watanabe)
As a result of applying AI, it had been possible to reduce the burden of inspectors by having them refer to AI-generated scores indicating the level of doubtfulness (e.g. 23% or 87%possibility of illegality).The role delegated to AI, therefore, is to catch all indications of illegality and assign scores that approximate the decisions of inspectors during preliminary investigations as closely as possible.
Pre-processing of crucial data and collaboration of experts from different fields
Naturally, there were a lot of hurdles that must be overcome before a practicable AI system could be established. Kenji Fukuda, the leader of NEC’s AI Team, said that one of the hurdles was the pre-processing of data used for AI machine learning.
”Market surveillance data are stored in relational databases. There was a need to process these data using aggregate function and other methods to create the data to be used in learning. Determining which particular data items and how to process them to create learning data required sophisticated decisions.” (Mr. Fukuda)
”In particular, designing the ”explanatory variables,” which are important parameters used in machine learning, took a lot of hard work. NEC’s AI data analytics engineer, Toshiko Fujii, also experienced great difficulty in determining how to combine raw data and generate values that highly correlate with the expected target values. This is because it’s not possible to completely automate the pre-processing for learning even with the use of deep learning methods.”
Mr. Fukuda added, ”Although there are many deep learning engines out there, unless an engineer with a clear understanding of AI attributes creates a well-designed engine, it would not be possible to achieve sophistication of AI.” This is because you cannot just thoughtlessly throw in data accumulated from operations for machine learning and expect AI to learn on its own. There is a need to link advanced levels of knowledge that cover both the fields of consulting and data science.
In this regard, Yutaka Kashima, of the Tokyo Stock Exchange’s IT Development, noted that the vendor must effectively work together with experts from other fields.
”Operations divisions and engineers use widely different jargons and have dissimilar cultures. Connecting the Market Surveillance and Compliance Division, which is an operations division, with technical representatives of vendors had to start from a coordination of terminologies used. For example, unless operations and technology people see eye to eye on the basic point of ”what AI can do to optimize operations,” as part of setting the parameters for indicating the effectiveness of AI—which is just one of the tasks involved, it would be impossible to move forward with properly maximizing AI.” (Mr. Kashima)
Mr. Watanabe, who represents the operations division, recalled that ”If Mr. Kashima was not there to act as an intermediary, we would have ended up concluding that AI is useless.” These days, although many companies are pursuing AI utilization, it seems that a lot of efforts end up in failure. Many of these failures stem from the misconception that AI can do anything, and that it can automatically learn and grow wiser on its own. Project members must have a shared understanding of the strengths and weaknesses of AI. This is a simple yet often neglected fact that determines the success or failure of developing an AI-based system.
Aiming to also incorporate ”explainable AI” in the future
Full-scale operations of the AI system constructed for this project began in March 2018, and the system is now currently being used in actual operations. Going forward, Mr. Watanabe hopes to develop a more sophisticated system for market surveillance.
”Currently, owing to the AI learning outcomes achieved thus far, the scores generated by AI in preliminary investigations are comparable to the results of analysis by experienced inspectors. These scores, however, are still not that specific. Thus, we would like to continue the AI learning process so that we can develop a system wherein we can say, for example, that a 10% or less possibility of illegality would not require checks by an inspector, and then, gradually, increase this threshold to 20%, 30%, and so on.” (Mr. Watanabe)
In regard to the ways of using AI, Mr. Fukuda explained, ”The key lies in how AI is integrated into the tasks.” It might be easier to grasp the picture if you think of it this way: AI grows wiser as it works along with humans to eventually become better able to support humans.
In addition, results generated by AI may otherwise reveal previously unnoticed perspectives for humans. Another future goal, therefore, is to incorporate these new viewpoints as feedback into operations. NEC’s ”heterogeneous mixture learning,” a white-box type of AI technology, is expected to play a pivotal role in making this a reality.
”Current deep learning systems are faced with the problem of being unable to explain the reasons for the decisions that AI makes. White-box AI, on the other hand, is clear about ”what algorithm was used to arrive at a particular score,” making it suitable for confirming the basis of the decisions.” (Mr. Fukuda). He added, ”We hope to contribute to enhancing the utilization of AI in creating an equal and fair society.”
Although the deployment of AI by the Japan Exchange Group had been a major challenge for the operations division, the IT division, as well as the IT vendor, full-scale operations became possible through the combination of knowhow from experts in different fields. Using human wisdom in creating the AI to support humans—this is the true state of the AI that is finding practical use in the real world.