Archive for probability
What a Simple Bingo Game Can Teach Us?
Posted by: | Comments
Kids playing with a MMORPG (in case you do not know what that is – it is the kind of game Wikipedia founder Jimmy Wales played and probably obsessed with many years ago) found a mini-game within the game that essentially plays like bingo.
Each player can get a game board with 5X5 grid filled with numbers from 1 to 25. All cells are unique. Boards are generated by random so it is not likely two players are getting the same number arrangement on their boards.
To play the mini-game, a player has to stay online connected to the game where every 30 minutes 1 token will be given. Another way to get a token is to explore around the virtual environment and hunt the monsters for a small chance per kill to get an extra token. One token is required per lucky draw for a number to be matched in the bingo board. If you get a row or a column completely matched, you will be rewarded with a special prize. Matching all numbers on the board will earn you a very special grand prize.
The problem – kids just discovered that it is very frustrating to get the board filled.
Somehow they feel like the game is cheating on them and not letting them to get that very special prize at all. Draw after draw that very last number on the bingo board just refuses to be filled. Why?
Statistics vs. How We Feel
Following is a table showing the number of draws needed on average to get at least one more number filled based on how many numbers are left at the moment. For example, when the board is new (all numbers are still there), the expectation is that you have 100% chance of getting one of the numbers there with just 1 draw. By the time the board has only 5 numbers left, it takes at least 5 draws on average to get one of them filled.
This table helped the kids realizing the truth – even though the goal of matching another number looks the same, the conditions keep changing. In fact, to fill the last 4 numbers to get the grand prize means double the number of tokens needed! So instead of complaining, off they go happy grinding (to mindlessly killing monsters within a game to gain something). Issue resolved.
We Cannot Control Our Own Swings in Winning Probability
Our performance seldom happens in a straight line. It is not like 30 trades taken you get 70% winning rate and then the next 30 trades you get 70% winning rate again.
One of the things that setback many traders is that when their overall performance is sitting at about 60% to 65%. It is the usual zone where traders start to become profitable. It is also the most vulnerable time in their trading skill development.
Think about the emotional swings they have to deal with. At times, they are "in the zone" and performing at 70% or better winning rate so every 4 trades 3 are winners. Sometimes, however, they swing to 50% or maybe a bit lower in winning rate, which implies every 3 trades 1 to 2 of them are losers. It is very difficult to deal with, especially when they think they have it figured out finally.
How to maintain a healthy attitude with proper perspective of the trading results is very important. Making a connection between your performance swings and how that affect the outcomes you are experiencing should help building your confidence in yourself without the unnecessary emotional trouble.
Counter-Trend Methods vs. Strong Trend Development
In our premium report Hot n Cold, a number of classic trading models are tracked. Those ones with style "CT" are the counter-trend trading models. Those having the style "TF" are trend following models.
Counter-trend trading models tend to have higher winning percentage (the WP column in the report tables) during sideway markets. Then as the market develops into trending mode, counter-trend models will have lower winning rate but still healthy readings. Once a market started to go into strong trend mode, counter-trend models will suffer and winning rate will easily dip below 50%.
The bingo statistics above is equally applicable to winning percentage of a trading model. It gives you a good reason to pay attention when you know CT models in general are dipping below 55%. First, you will likely get more losers than winners if you trade counter-trend. Second, isn’t it better to join the strong trend in the making?
By knowing that a particular CT model is weakening, you know what to avoid in your own trading. For example, if the 90 days rolling performance of Bollinger Band OB/OS model is in a declining phase, then you know using range based (and confirmation based) counter-trend methods will be less reliable. If Relative Strength Index is still going strong, that implies you should instead focus on time based (and extreme fading) methods if you like trading counter-trend.
I do not track the Hot n Cold report everyday. Instead, I review them once a week spotting for subtle changes in the performance of the trading models. If I suspect that we are witnessing something big coming in the underlying market, I will go back to the chart to confirm what I found out from the report. It is like my insurance policy against my own personal bias. For example, if the objective statistics on the models confirm the trend is weakening, then my swing trade directional bet should focus on taking profit at target zone as oppose to holding for more.
Clarifications to Some Basic Concepts in Probability
Posted by: | CommentsI wrote many articles on statistics over the years (Teaching a Grade 4 on Probability, Understanding the Market in a Statistical Way, etc.) and advocate traders to pay more attention to proper application of statistics in trading and model development.
I came across this educational video by Peter Donnelly from TED that explains several basic concepts in statistics clearly.
There is no point for me to repeat his work, so I will just include the video here.
In case your browser does not support HTML5 embedded video, here is the link directly to the lesson itself,
http://ed.ted.com/lessons/peter-donnelly-shows-how-stats-fool-juries
Relating to the Video
The HTH vs HTT occurrence frequencies – It is one of the most overlooked and misunderstood basic pattern recurrence behaviour.
Many rookie traders and trading model designers often assume that certain common patterns (price, indicator, geometry lines) are useless because of their potential frequent recurrences. What they have overlooked, is that, first the assumed common patterns are not that common (like HTH) and second, the importance of any recurring pattern is not how common or rare it is, but the consistencies in the expected outcome following the occurrence of such pattern.
Here is a good teaser – How often does S&P make new year high after the last occurrence has led to 1% (or more) drop in price?
The 99% interpretation issue – You think your trading setup has a 80% win rate is pretty good, think again.
Most successful traders know their own performance pretty well. They are often better than computers in analyzing real-time scenarios. In their heads, they can picture the possible scenarios and reduce that to actions that improves their profitability.
Such mastery is not just a plain 80% win rate or other simple measurement can do to discover or qualify a method.
80% win rate is an overall measurement. It does not take into account the separate performance of a correctly identified setup and a falsely identified one. Often traders cannot even distinguish the two scenarios for trading setups they use all the time.
For example, a 1-2-3 sell setup can fail and results in a bull flag upside breakout pattern. 1-2-3 sell setup works best if the trader can identify the strong resistance area to key off the setup.
A good combination can be 90% of the time you identify the trading setup at a good resistance zone where 90% of those identified setups result in profits, and that in the 10% of falsely identified setups, you still edge out 50% winners. The reason why this is a good combination is that your expected performance will likely to have good consistency.
A bad combination could be 50% of the time you identify the trading setup correctly and 90% profitable on those correctly identified ones. And then in the other 50% of trading setups, 70% of the time you edge out a profit. In this case, even though the historical performance is around 80% win rate, you do not really know why your setups are producing profit. In fact, that 70% winners out of the falsely identified setups could be just random outcomes. Worst yet, traders often try to improve the win rate by adjusting the stop loss which simply curve fit the trading setup to perform better on historical data. Those extra winners in the particular 70% is likely a result of that.
The rare case independency issue – Traders often make the mistake in thinking that since a (huge) losing trade based on a particular setup is very rare, why not double down, triple down, or bet the farm on the next occurrence of the trading setup. People justifying such measures are likely in a losing streak. In such stressed situations, bad ideas often pop up and this is one of them.
Even if the huge losses are rare and likely independent, that does not change the potential of the very next trade being a loser. Betting the farm or sudden increase in bet size in general will significantly affect the expectancies with your trading method. If it is not something you have anticipated and well planned out, it is just plain stupid doing that knowing the consequence can wipe out your trading account.
Professionals in various fields often misuse statistics – In the video Peter Donnelly pointed out this very important issue that infested our society.
Doctors, lawyers, economists, security analysts, etc. are not statisticians. Yet these professionals frequently try to draw conclusions and inference from data they have compiled. Often these inference are incorrect and misleading. Unluckily there is no rule or regulation to, say, revoke their professional licenses for misuse of statistics. At the receiving end of these analysis and conclusions, it is difficult for the public to protect themselves from these misleading information.
Market Bias Detective: S&P 500 Daytrading Time Map Vol. 2
Posted by: | Comments
A collection of the time based statistical biases often mentioned in real-time commentaries on trading S&P 500 related instruments (e.g. e-mini S&P, SPY, etc.). Volume 2 focuses on weekly bias and strategic planning.
Time Map is the concept of utilizing strong time based statistical biases to improve a trader’s existing trading setups/methods.
In this second volume, the focus is on weekly biases. Many daytraders do not notice that there are a number of strong recurring behaviours in S&P on weekly level that can provide important clues in improving their entry timing and price targeting. By knowing these weekly biases, a trader is better prepared without being surprised by the market that often again.
Written by Lawrence Chan
| Regular Price: $25 | Member Price: $20 |
* copies downloaded before mid April 2012 please read the correction notice
Nasdaq 100 Down 2% Trading Setup
Posted by: | CommentsOn a day when Nasdaq 100 is down 2% from its previous day close, something very interesting is likely to happen within a day or two.
Following is a table of Nasdaq 100 emini from September 2003 to March 2011 showing the statistics related to the 2% drop phenomenon.
Members only content, login now or sign up to view the rest of this article.
Essence of Trading: Only Man-made Roads are Straight and Smooth
Posted by: | Comments
Ever observe a tree in detail?From afar, a tree may look tall and sturdy. That is just our impression of the tree. By looking at a tree carefully, you would notice that it is not quite accurate. Its branches are never grown in straight lines. The tree trunk is not a one piece metal pile cut from steel. Every year the tree has to endure tough weather changes and all the suffering it took shows on its bark.
A good trader is like a grown up tree who can withstand tough environment and survive.
Talent may be born with, just like a tree coming from a strong species. But that only gives one a better head start.
What really shapes a trader is the path that the trader must walk through before maturing into someone who can accept themselves as a trader and survive being one. Mental toughness does not stick with a person after just one day or one month of fighting an adverse scenario. Being able to beat the market in a changing environment may come from one awakening after reading a book, talking to a mentor, or just an idea that hits the trader like a lightening strike, but that seed in the mind still takes time to grow into a mature state so that the instinct or knowledge acquired becomes second nature (or part of the person’s character).
Do yourself a favour – Embrace the journey.
Accept the fact that it may not be a man-made road that goes straight and smooth. Expect the ride to be a bumpy one and keep yourself upbeat even if the situation looks grim. Enjoy winning your tough battles but remember that pride comes before the fall. Learn from the mistakes and move on.
All the marks left on you from your daily battle in trading will toughen you up and make you stronger as a better trader.
Be proud of who you are.
SPY Range Expansion Behaviour
Posted by: | CommentsThis is a study on the probability of SPY expanding its range as a multiple of the average range over 10 days period.
The study was originally done by our member smilingsynic. I am just reproducing the results here using data from 1993 up to last Friday Feb 25, 2011.
First, a comparison based on similar starting point in percentage basis with fixed increment up to 1 times average range away.
1DR / AR is the ratio of 1 day range and the average range over 10 days period from the trading day before.
2DR / AR is the ratio of 2 day range and the average range over 10 days period from 2 trading days before.
The reason for adjusting the average range reference point is to avoid leakage of the measured range into the reference average range.
So, according to the table below, for any 3-day period, SPY has 25.17% chance to reach 2.3 times or more of the average range.
Next is the data presented in a graph.
Second, a comparison based on fixed probability range to show the extend of the changes in average range multiples.
Note: The results I got is slightly different from smilingsynic’s results because the range expansion behaviour in last 2 years has skewed the results towards stronger range expansion bias in the tail end of the distribution.
What to Expect After 2 Down Days?
Posted by: | Comments
One of my favorite daily level Time Map patterns is that Emini S&P dropped for 2 days significantly. It provides something so consistent to work with over the years you have to wonder why no one talks about it at all.
Members only content, login now or sign up to view the rest of this article.
Teaching a Grade 4 on Probability
Posted by: | Comments
Before summer started, my son was learning probability at school and his teacher gave his class some interesting questions to solve. The background is that they’ve just learned fraction, decimal numbers, etc.
Continue reading »
Market Bias Detective: S&P 500 Daytrading Time Map Vol. 1
Posted by: | Comments
A collection of the time based statistical biases often mentioned in real-time commentaries on trading S&P 500 related instruments (e.g. e-mini S&P, SPY, etc.). A useful reference if you are not already familiar with the time based trading techniques.
Time Map is the concept of utilizing strong time based statistical biases to improve a trader’s existing trading setups/methods.
In this first volume, the most important/foundational biases that are useful for S&P 500 trading are provided so that any daytrader can easily incorporate the biases into his/her decision making process.
A must read for those daytraders who want to improve their trading performances and also for those who find standard trading techniques failed to satisfy their needs.
Written by Lawrence Chan
| Regular Price: $35 | Member Price: $30 |
Market Internals Update 2011-05-03
Posted by: Lawrence Chan | Comments (0)After I posted the breadth bias warning last week. Someone asked for some samples of the potential outcomes from history. Here they are.
First one, the one that simply go sideway for a month.
Second one, if there is a turn, it should happen quickly in first 2 weeks of May.