It's no wonder that The Power of Now has sold over 2 million copies worldwide and has been translated into over 30 forei. Wiley Trading. ERNEST P. CHAN. How to Build Your Own Algorithmic Trading Business. Quantitative. Trading. HAN. Q uantitative. Trading. Ho w to B uild Yo. Library of Congress Cataloging-in-Publication Data: Chan, Ernest P., – Algorithmic trading: winning strategies and their rationale / Ernest P. Chan. pages.
|Language:||English, Portuguese, Dutch|
|Genre:||Business & Career|
|ePub File Size:||21.84 MB|
|PDF File Size:||9.39 MB|
|Distribution:||Free* [*Registration Required]|
Algorithmic Trading Strategies And Modelling Ideas. 22 can read the “ Algorithmic Trading: Winning Strategies and Their Rationale” book by Dr. Ernest Chan. Praise for Algorithmic Trading "Algorithmic Trading is an insightful book on quantitative trading written by a seasoned practitioner. What sets this book apart from. Source: Ernie Chan (), “Algorithmic Trading: Winning Strategies ://www3. volwarmdilanmi.cf
Before trying to build a predictive model using this feature matrix, we compared their features importance to other existing features using boosted random forest, as implemented in LightGBM.
These categorical features are nowhere to be found in the top 5 features compared to the price features returns. That is a common fallacy of using train data for feature importance ranking the problem is highlighted by Larkin.
The proper way to compute feature importance is to apply Mean Decrease Accuracy MDA using validation data or with cross-validation see our kernel demonstrating that assetCode is no longer an important feature once we do that.
Alternatively, we can manually exclude such features that remain constant through the history of a stock from features importance ranking. Once we have done that, we find the most important features are Compared to the price features, these categorical news features are much less important, and we find that adding them to the simple news strategy above does not improve performance.
So let's return to the question of why it is that our simple news strategy suffered such deterioration of performance going from validation to test set. Most other kernels published by other Kagglers have not shown any benefits in incorporating news features in generating alpha either.
Complicated price features with complicated machine learning algorithms are used by many leading contestants that have published their kernels.
The other possibilities are bad luck, regime change, or alpha decay. Comparing the two equity curves, bad luck seems an unlikely explanation. Given that the strategy uses news features only, and not macroeconomic, price or market structure features, regime change also seems unlikely. Alpha decay seems a likely culprit - by that we mean the decay of alpha due to competition from other traders who use the same features to generate signals.
A recently published academic paper Beckers, lends support to this conjecture. Based on a meta-study of most published strategies using news sentiment data, the author found that such strategies generated an information ratio of 0.
Does that mean we should abandon news sentiment as a feature?
Not necessarily. Our predictive horizon is constrained to be 10 days. Certainly one should test other horizons if such data is available.
When we gave a summary of our findings at a conference, a member of the audience suggested that news sentiment can still be useful if we are careful in choosing which country India?
Search This Blog
We have only applied the research to US stocks in the top 2, of market cap, due to the restrictions imposed by Two Sigma, but there is no reason you have to abide by those restrictions in your own news sentiment research.
We will walk you through the complete lifecycle of trading strategies creation and improvement using machine learning, including automated execution, with unique insights and commentaries from our own research and practice. It will be co-taught by Dr.
Ernest Chan and Dr. See www. They have built their businesses and vast wealth not just by sitting in front of their trading screens or scribbling complicated equations all day long, but by collaborating and managing other talented traders and researchers. Simons declared that total transparency within Renaissance Technologies is one reason of their success, and Lopez de Prado deemed the "production chain" assembly line approach the best meta-strategy for quantitative investment.
One does not need to be a giant of the industry to practice team-based strategy development, but to do that well requires years of practice and trial and error. While this sounds no easier than developing strategies on your own, it is more sustainable and scalable - we as individual humans do get tired, overwhelmed, sick, or old sometimes.
My experience in team-based strategy development falls into 3 categories: 1 pair-trading, 2 hiring researchers, and 3 hiring subadvisors. Here are my thoughts. From Pair Programming to Pair Trading Software developers may be familiar with the concept of "pair programming".
According to software experts , this practice reduces bugs and vastly improves the quality of the code. I have found that to work equally well in trading research and executions, which gives new meaning to the term "pair trading". The more different the pair-traders are, the more they will learn from each other at the end of the day. One trader may be detail-oriented, while another may be bursting with ideas.
One trader may be a programmer geek, and another may have a CFA. Here is an example. In financial data science and machine learning, data cleansing is a crucial step, often seriously affecting the validity of the final results. I am, unfortunately, often too impatient with this step, eager to get to the "red meat" of strategy testing. Fortunately, my colleagues at QTS Capital are much more patient and careful, leading to much better quality work and invalidating quite a few of my bogus strategies along the way.
Speaking of invalidating strategies, it is crucial to have a pair-trader independently backtest a strategy before trading it, preferably in two different programming languages. As I have written in my book , I backtest with Matlab and others in my firm use Python, while the final implementation as a production system by my pair-trader Roger is always in C.
Often, subtle biases and bugs in a strategy will be revealed only at this last step. After the strategy is "cross-validated" by your pair-trader, and you have moved on to live trading, it is a good idea to have one human watching over the trading programs at all times, even for fully automated strategies.
For the same reason, I always have my foot ready on the brake even though my car has a collision avoidance system. Constant supervision requires two humans, at least, especially if you trade in international as well as domestic markets. Of course, pair-trading is not just about finding bugs and monitoring live trading. It brings to you new ideas, techniques, strategies, or even completely new businesses. I have started two hedge funds in the past. In both cases, it started with me consulting for a client, and the consulting progressed to a collaboration, and the collaboration became so fruitful that we decided to start a fund to trade the resulting strategies.
For balance, I should talk about a few downsides to pair-trading. Though the final product's quality is usually higher, collaborative work often takes a lot longer. Your pair-trader's schedule may be different from yours. If the collaboration takes the form of a formal partnership in managing a fund or business, be careful not to share ultimate control of it with your pair-trading partner sharing economic benefits is of course necessary.
I had one of my funds shut down due to the early retirement of my partner. One of the reasons I started trading independently instead of working for a large firm is to avoid having my projects or strategies prematurely terminated by senior management, and having a partner involuntarily shuts you down is just as bad. Where to find your pair-trader?
Publish your ideas and knowledge to social media is the easiest way note this blog here. Whether you blog, tweet, quora, linkedIn, podcast, or youTube, if your audience finds you knowledgeable, you can entice them to a collaboration. Hiring Researchers Besides pair-trading with partners on a shared intellectual property basis, I have also hired various interns and researchers, where I own all the IP.
They range from undergraduates to post-doctoral researchers and I would not hesitate to hire talented high schoolers either.
The difference with pair-traders is that as the hired quants are typically more junior in experience and hence require more supervision, and they need to be paid a guaranteed fee instead of sharing profits only.
Due to the guaranteed fee, the screening criterion is more important. I found short interviews, even one with brain teasers, to be quite unpredictive of future performance no offence, D. We settled on giving an applicant a tough financial data science problem to be done at their leisure.
I also found that there is no particular advantage to being in the same physical office with your staff.
We have worked very well with interns spanning the globe from the UK to Vietnam. Though physical meetings are unimportant, regular Google Hangouts with screen-sharing is essential in working with remote researchers. Unlike with pair-traders, there isn't time to work together on coding with all the different researchers. But it is very beneficial to walk through their codes whenever results are available. Bugs will be detected, nuances explained, and very often, new ideas come out of the video meetings.
Powerpoint presentations are also much more time-consuming to prepare, whereas a code walk-through needs little preparation.
Generally, even undergraduate interns prefer to develop a brand new strategy on their own. But that is not necessarily the most productive use of their talent for the firm. It is rare to be able to develop and complete a trading strategy using machine learning within a summer internship.
Also, if the goal of the strategy is to be traded as an independent managed account product e. On the other hand, we can often see immediate benefits from improving an existing strategy, and the improvement can be researched within 3 or 4 months.
This also fits within the "production chain" meta-strategy described by Lopez de Prado above, where each quant should mainly focus on one aspect of the strategy production. This whole idea of emphasizing improving existing strategies over creating new strategies was suggested to us by our post-doctoral researcher, which leads me to the next point. Sometimes one hires people because we need help with something we can do ourselves but don't have time to.
This would generally be the reason to hire undergraduate interns. For example, despite my theoretical physics background, my stochastic calculus isn't top notch to put it mildly.
This is remedied by hiring our postdoc Ray who found tedious mathematics a joy rather than a drudgery. While undergraduate interns improve our productivity, graduate and post-doctoral researchers are generally able to break new ground for us.
For these quants, they require more freedom to pursue their projects, but that doesn't mean we can skip the code reviews and weekly video conferences, just like what we do with pair-traders. Some firms may spend a lot of time and money to find such interns and researchers using professional recruiters.
In contrast, these hires generally found their way to us, despite our minuscule size. That is because I am known as an educator both formally as adjunct faculty in universities, as well as informally on social media and through books. Everybody likes to be educated while getting paid. If you develop a reputation of being an educator in the broadest sense, you shall find recruits coming to you too.
Hiring Subadvisors If one decides to give up on intellectual property creation, and just go for returns on investment, finding subadvisors to trade your account isn't a bad option. After all, creating IP takes a lot of time and money, and finding a profitable subadvisor will generate that cash flow and diversify your portfolio and revenue stream while you are patiently doing research. In contrast to Silicon Valley startups where the cash for IP creation comes from venture capital, cash flow for hedge funds like ours comes mainly from fees and expense reimbursements, which are quite limited unless the fund is large or very profitable.
We have tried a lot of subadvisors in the past. All but one failed to deliver. That is because we were cheap.
We picked "emerging" subadvisors who had profitable, but short, track records, and charged lower fees. To our chagrin, their long and deep drawdown typically immediately began once we hired them. There is a name for this: it is called selection bias.
If you generate geometric random walks representing the equity curves of subadvisors, it is likely that one of them has a Sharpe ratio greater than 2 if the random walk has only steps.
Here, I simulated normally distributed returns series with bars, and sure enough, the maximum Sharpe ratio of those is 2. The first 3 readers who can email me a correct analytical expression with a valid proof that describes the cumulative probability P of obtaining a Sharpe ratio greater than or equal to S of a normally distributed returns series of length T will get a free copy of my book Machine Trading. At their option, I can also tweet their names and contact info to attract potential employment or consulting opportunities.
These lucky subadvisors are unlikely to maintain their Sharpe ratios going forward. To overcome this selection bias, we adopted this rule: whenever a subadvisor approaches us, we time-stamp that as Day Zero.
We will only pay attention to the performance thereafter. This is similar in concept to "paper trading" or "walk-forward testing".
Chan E.P. Algorithmic Trading. Winning Strategies and Their Rationale
Subadvisors with longer profitable track records do pass this test more often than "emerging" subadvisors. But these subadvisors typically charge the full 2 and 20 fees, and the more profitable ones may charge even more. Some investors balk at those high fees. I think these investors suffer from a behavioral finance bias, which for lack of a better term I will call "Scrooge syndrome". Does one begrudge Jeff Bezo's wealth? Does one begrudge the many millions he rake in every day?
No, the typical investor only cares about the net returns on equity. So why does this investor suddenly becomes so concerned with the difference between gross and net return of a subadvisor? As long as the net return is attractive, we shouldn't care how much fees the subadvisor is raking in. Renaissance Technologies' Medallion Fund reportedly charges 5 and 44, but most people would jump at the chance of investing if they were allowed.
Besides fees, some quant investors balk at hiring subadvisors because of pride. Nobody would feel diminished downloading AAPL even though they were not involved in creating the iPhone at Apple, why should they feel diminished paying for a service that generates uncorrelated returns?
Do they think they alone can create every new strategy ever discoverable by humankind? Hire well. It enables easy parallel computations. I enjoyed reading it very much. The concepts of backwardation and contango will be illustrated graphically as well as mathematically. The chapter on mean reversion of currencies and futures cumulates in the study of a very special future: the volatility VX future, and how it can form the basis of some quite lucrative strategies.
In the momentum camp, we start by explaining a few statistical tests for times series momentum. The main theme, though, is to explore the four main drivers of momentum in stocks and futures and to propose strategies that can extract time series and cross-sectional momentum. Roll returns in futures is one of those drivers, but it turns out that forced asset sales and downloads is the main driver of stock and ETF momentum in many diverse circumstances.
Some of the newer momentum strategies based on news events, news sentiment, leveraged ETFs, order flow, and high-frequency trading will be covered. Finally, we will look at the pros and cons of momentum versus mean-reverting strategies and discover their diametrically different risk-return characteristics under different market regimes in recent financial history.
I have always maintained that it is easy to find published, supposedly profi table, strategies in the many books, magazines, or blogs out there, but much harder to see why they may be flawed and perhaps ultimately doomed. So, despite the emphasis on suggesting prototype strategies, we will also discuss many common pitfalls of algorithmic trading strategies, which may be almost as valuable to the reader as the description of the strategies themselves.
These pitfalls can cause live trading results to diverge significantly from their backtests. As veterans of algorithmic trading will also agree, the same theoretical strategy can result in spectacular profits and abysmal losses, depending on the details of implementation. Hence, in this book I have lavished attention on the nitty-gritties of backtesting and sometimes live implementation of these strategies, with discussions of concepts such as data-snooping bias, survivorship bias, primary versus consolidated quotes, the venue dependence of currency quotes, the nuances of short-sale constraints, the construction of futures continuous contracts, and the use of futures closing versus settlement prices in backtests.
We also highlight some instances of regime shift historically when even the most correct backtest will fail to predict the future returns of a strategy. I will survey the state of the art in technology, for every level of programming skills, and for many different budgets. In particular, we draw attention to the integrated development environment for traders, ranging from the industrial strength platforms such as Deltix to the myriad open-source versions such as TradeLink.
As we will explain, the ease of switching from backtesting to live trading mode is the most important virtue of such platforms. The fashionable concept of complex event processing will also be introduced in this context.
I covered risk and money management in my previous book, which was built on the Kelly formula—a formula that determines the optimal leverage and capital allocation while balancing returns versus risks. I once again cover risk and money management here, still based on the Kelly formula, but tempered with my practical experience in risk management involving black swans, constant proportion portfolio insurance, and stop losses.
Supreme Court Justice Robert H. Jackson could have been talking about the application of the Kelly formula when he said we should temper its doctrinaire logic with a little practical wisdom. We especially focus on finding the optimal leverage in realistic situations when we can no longer assume Gaussian distribution of returns.
Also, we consider whether risk indicators might be a useful component of a comprehensive risk management scheme.
One general technique that I have overlooked previously is the use of Monte Carlo simulations. Here, we demonstrate using simulated, as opposed to historical, data to test the statistical significance of a backtest as well as to assess the tail risk of a strategy. This book is meant as a follow-up to my previous book, Quantitative Trading.
There, I focused on basic techniques for an algorithmic trader, such as how to find ideas for new strategies, how to backtest a strategy, basic considerations in automating your executions, and, finally, risk management via the Kelly formula.This whole idea of emphasizing improving existing strategies over creating new strategies was suggested to us by our post-doctoral researcher, which leads me to the next point.
Business Finance Nonfiction Praise for Algorithmic Trading "Algorithmic Trading is an insightful book on quantitative trading written by a seasoned practitioner.
It turns out that the loss averse "layman" is the one acting rationally here. This presents difficulties that are not present in industrial-strength databases such as CRSP, and requires us to devise our own algorithm to create a unique identifier. This book is a practical guide to algorithmic trading strategies that can be readily implemented by both retail and institutional traders.
Posted by Ernie Chan at AM 33 comments Friday, June 29, In his famous book " Thinking, Fast and Slow ", the Nobel laureate Daniel Kahneman described one common example of a behavioral finance bias: "You are offered a gamble on the toss of a [fair] coin.