The Lawrence Chan Blog

Since the introduction of Windows 7, there is this interesting bug that keeps annoying people everyday with no real solution provided by Microsoft. Will explain what it is and how to deal with it.

In A Nutshell

If your computer has the following,

1. Windows Vista, 7 (and at least early versions of Windows 8)

2. 64-bit

3. more than 4G of RAM

4. multiple cores (4 or more)

then when you copy files from one USB drive to another you will likely experience random disconnect of one of the drives. You would not know that one of the drive is disconnected. Windows would hang up, report to you files missing, not recognized, etc. Worse yet, you have to turn off the drive to force Windows to recognize the disconnect and sometimes reboot the computer is necessary.

It does not matter whether you have the best hardware you can buy, Windows will just fail at random times.

This does not happen to the 32-bit versions, nor in Windows XP, 2000, etc.

This also rarely happens to 64-bit versions if you have up to 4G RAM only.

Root of the Problem

The USB driver for 64-bit Windows somehow not working as it should be when the computer has a lot of RAM.

The problem is there all along and Microsoft never fix it.

As of today this problem still exists even with all the latest updates from Microsoft.

Solutions / Workarounds

The file copy utility I mentioned last time cannot fully resolve this problem although it can make it happen less frequently.

If you have access to an older computer with Windows XP, then just plug the USB drives there and do the copying with the older computer. Just remember to install the file copy utility mentioned above to speed things up. Problem solved.

If you do not have access to an old computer, you need to copy the files from the USB drive to the internal hard drive first, and then copy the files from the internal drive to the other USB drive. I know it sounds stupid and a waste of time, but you do not have a choice.

No comment yet

I have been working on my new trading server lately and one of the old problems of Windows just gave me a hard time. Luckily I already know what to do and the problem is solved quickly. Here is the whole story.

Windows Fail Badly When Dealing With Large Files And Large Number Of Files

If you ever try to copy files of 500+ Mb in size from one USB drive to another on Windows, you will experience random failures from one of the hard drive disconnecting, copy failure, to corrupting the hard disks.

Remember it is not your fault and it is also not necessary that you have a bad hard disk.

Windows have this problem for eons and people at Microsoft never fix it.

The other problem is that Windows would use more time to copy a group of files than copying them one by one. If you choose a large number of files to copy, the total time it will take is much more than necessary. It does not matter if you have the latest fastest top of the line hardware. Windows would just eat up a lot of your computer’s processing power and take forever to copy the files.

Solution 1: Get TeraCopy

TeraCopy is how file copy should be done if Microsoft were doing its job.

It is a free utility that you can install onto Windows and then the file copy problems will go away most of the time.

Following is a link to the site,

http://codesector.com/teracopy

Solution 2: Use Disk Imaging

As mentioned last time when my old trading server was toasted, I have image of my disk drives to restore from.

That is the other way to copy large files and large number of files.

Simply image your hard disk if you are about to copy most of the content to another drive or computer, then restore the files from the image. It is way faster and more reliable than using Windows itself.

There are many disk imaging apps available on the market. Choose one you like as they varies in features and workflow.

No comment yet

A buddy of mine told me the other day that many book authors also write their book reviews directly at Amazon for the benefit of exposures. He thinks it is a good idea and suggested that I should give it a try. The short conversation turned into a lengthy discussion of posting reviews online in general that I think worth writing about.

The Amazon community is big and obviously many readers would read the reviews there to assist them in making their purchase decisions. As someone who reads all the time, I am sure if I post all my reviews there I will have presence in many trading books. The problem, however, is the ongoing discussion there that I have to deal with.

I don’t mind engaging in discussions about the books I have read and the reviews that I have written about them. The problem is that I would not have enough time to go through all the past reviews I posted to handle the discussions scattered everywhere. For a few reviews, I can probably handle that. For a dozen or more, it will become a very time consuming task.

The other concern is that my ebooks are not just available through Amazon. There are other online ebook retailers featuring my ebooks. Posting my reviews only at Amazon will make my readers who bought my work elsewhere not able to see my reviews.

The lazy way out is to centralize all the reviews here so that anyone interested in my research work will also get to read the reviews. It also makes it easier for me to engage in discussions on my reviews as I do not need to check on hundred different places.

Not the best solution but nonetheless a solution.

No comment yet

Right before US stock market was about to close on Jan 26, 2013, a huge block of Apple shares was dumped. We are not talking about 10K, 50K or even 100K block. We are talking about 800K shares of Apple got dumped at close.

So what happened?

Why the rest of the stock market keep going up while Apple got hammered yet never dropped much throughout the day?

I guess it is time to explain one important aspect of the stock market dynamics – institution unloading. I just posted a new article, The War Of The Bot, explaining this messed up situation.

Personally I have seen my share of nasty market making both on and off floor.

For example, a market making brokerage that handles a large sell order on a large cap from an institution demanding a price higher than previous day close by $0.50. Then you would see the issue magically pop $0.50 and likely more, inducing a short squeeze of sort and then the cross is done. For anyone looking at the incident that was the very reason why the institution chooses to use the brokerage firm to execute the order because they know the brokerage firm can make it happen. Yet, the market maker was just doing his job.

Manipulation?

I am not in a position to judge.

Not a fair game?

I just know that it happens everyday.

No comment yet

Many retail traders learned about the evil market making bots through mainstream media. Yet, they are not the ones that rule the game. Their nemesis, institution bots, are the ones even market making firms are afraid of.

Institution Level Trading Is Different From Retail Trading Completely

Institutions participating in the stock markets cannot just go out and buy (or sell) a particular stock at market. Especially when the intention is to accumulate or unload a stock with market moving size, they have to do that in stealth mode. If these institutions simply put a market order out there, they will move the particular issue wildly. The end result is very bad average price for the order. It may even halt the issue due to the extreme volatility induced. No one wants to be put under the microscope of the authorities when it is totally unnecessary.

Many boutique firms are specialized in handling this type of large size orders for the institutions. Their traders (and their bots) are highly specialized in breaking the large orders into small chucks so that no one can tell if such pending supply (or demand) is there. All kinds of strategies are employed to hide the true intention. For example, if an institution needs to unload a large block of shares, the traders handling the order do not just unload the issue all day long. They would at times buy the shares back to nudge the market higher so that they can unload at a better price.

Secrecy Has Always Been A Top Priority

Think about those who some how gain the knowledge of a particular institution’s intention and the particulars of its trading instructions. Due to the sheer size of the pending order, it will imbalance the short term dynamics of the particular stock, and sometimes it would affect the related indices too if its weighting is significant (like Apple). These players who learned the intention can lean on the imbalance to gain absolute advantage for (almost) sure win.

The leak of the intention and the actions taken by the other players as noted above will hurt the institution as they are the one that will be at the receiving end of worse fills on their order. Hence, institutions like to guard their orders and intentions as secretive as possible.

Within the past 10 years, the introduction of specialized client side bots built for these institutions partially solved the problem. For average large size orders, they can be handled without human intervention, thus reduces the chance having the intention leaked to unnecessary parties.

Here Comes The Catch

The use of specialized bots by the institutions induced a technology war with the market making firms. When the institutions having orders entering the markets, the market making bots are the ones that has to absorb the liquidity imbalance. No one wants to lose money simply because some institution decided one day to throw large size orders around. Thus, a race between these 2 classes of bots had started.

Market making bots are now smarter and faster then ever in identifying potential intention of orders coming from a particular brokerage. The goal is to at least not being burn by the institution. Of course, early detection of the intention would not only save the market making firms a lot of money, they can also design their bots to take advantage of the situation to profit from it.

For example, if an institution successfully disguised its intention and unloaded so many shares to the market, the market making bots would have been lured to keep buying as they failed to recognize the unusual situation. When these market making bots are loaded up with too many shares, they have no choice but to dump whatever they have bought. In another words, the institution successfully transferred the risk to the market makers.

In the same situation, if the market making bots figured out the intention of the institution early and not taking the bait to accumulate the shares, the institution would have no choice but to dump at worse price level, unless, they can wait for another day to execute the plan.

2 comments