Flash Boys

I read Michael Lewis’s Flash Boys in a single sitting at the Cambridge Public Library.  For context, I rarely finish books and it usually takes months.  I read it because I needed to understand the pros and cons of routing trades to IEX and couldn’t find enough detail online.  Flash Boys turned out to surpass my expectations, but left me wondering why IEX doesn’t publish a more concise explanation of how the market works and why trades should be routed to them.  Claiming they have a coil of fiber to introduce a delay that slows down HFT is hardly an explanation.

I should mention that I was positively surprised to see Lewis mention both Nanex and Themis Trading, because they really are the best resources for understanding market microstructure and how HFT works.  I’ve been following these problems with the markets for 4 years now, and never imagined it would get so much public attention.

Lewis identifies a few ways in which HFT takes money from investors.

  1. High speed links between exchanges beat customer orders from one exchange to another.
  2. Varying rebate structures cause brokers to route customer orders to where the broker gets paid the most and the customer is therefore least likely to be filled.
  3. Orders in dark pools can be executed at stale prices because they aren’t displayed as part of NBBO.
  4. Sub-pennying.  Lewis actually fails to explain how sub-pennying hurts traders.  There’s a single reference to sub-pennying in the book, but it says nothing about how it works.  See http://www.nanex.net/aqck2/3519.html and http://www.defendtrading.com/cfm.v21.n1.2.pdf

I wish the book had a technical summary of all the phenomena seen, so I’ll try to write one here.

First, some background for non-traders.

There are two basic types of trades: market orders and limit orders.  Market orders are for traders who have strong convictions about the direction a stock is moving.  They are designed to get executed immediately at the best available prices in the market.  They’re also called “market taking” because they take whatever prices are given.  Limit orders are for traders who want to guarantee a particular price or are willing to wait for someone else to trade in the opposite direction.  They’re known as “market making” because they make up the prices you see posted in the market.  A market order is like selling your car to a dealer.  You get instant money, but not as much.   A limit order is like listing the car for sale and waiting for a willing buyer.

Secondly, the order in which trades are executed matters a lot.  Imagine you and a dozen other people are trying to sell something of uncertain and changing value.  If you’re all offering the same price, you don’t want to be the last person to sell.  If that happens, it’s because a buyer came and bought everything, which implies that the thing you sold is probably now worth more.  On the other hand, if you’re the first to sell and everyone else is still offering to sell at the same price, that’s great.  You still have the option of undoing the sale by turning around and buying from another seller at the same price.  Furthermore, imagine a buyer comes and offers one of the sellers a dollar less to buy the thing and the seller agrees.  If you can jump in after the agreement, push the seller aside by offering to save the buyer an additional penny, the buyer probably wouldn’t mind, you’d still be able to undo your trade at a 1 cent loss, and the seller you displaced would feel unfairly treated.

Now, on to the 4 HFT phenomena.

  1. Phenomenon 1 (fastest links between exchanges) only affects market taking orders, not market making orders.  Katsuyama complains that when he sees shares offered on multiple exchanges and sends an order to buy all of them, only some of them get filled.  For years, traders have known that the way to get around this is to release the orders such that they all arrive at nearly the same time.  But maybe Katsuyama discovered it first and it just took a long time for Lewis to write the book.  By the way, you also have to flag orders as ISO (intermarket sweep order) so that the receiving exchange doesn’t wait to confirm that there isn’t a better price at another exchange.  By that time, the available price would have disappeared.  IEX helps a lot here by providing such sweep order functionality to traders who don’t have direct connections to all the exchanges.  I’m surprised nobody mentioned the idea of using synchronized clocks on machines located outside each exchange that release a large order within light meters (tens of nanoseconds) of each other.  I’m sure there are firms doing it.  Maybe Katsuyama didn’t tell Lewis.  But it should be an obvious solution to anyone who thinks about the problem a lot.  Another thought on this phenomenon is that it’s nothing new.  It’s called cross market arbitrage and they used to use telephones.  It’s an unavoidable fact that whoever has the best communication method will keep markets in line with each other.  The unanswered question is, “Why aren’t all customers bringing their trades to the same venue?”  If new exchanges start and pay brokers extra to bring orders there, why don’t all brokers end up directing orders to the single exchange providing the highest kickback “rebate”?
  2. A lot of exchanges provide rebates for market making orders and charge market taking orders.  This incents brokers to direct limit orders to exchanges with the highest positive rebates.  Obviously, those orders will be the last to be filled, since it costs more to take them.  I’m not sure how IEX helps here.  If I post a limit order on IEX that matches the NBBO, it’s not going to get filled before the one on BATS.  Small market orders prefer BATS because they offer a negative rebate (get paid to take markets).  Big market orders often hit BATS first too, as described in the book.  To help limit orders get filled quickly, IEX needs to change the fee structure and pay for market orders while charging limit orders the way BATS does.  And as things are right now, I have no incentive to route small market orders to IEX.  I’m best off sending them to the exchange with the highest negative rebate.  The only counterargument is that the prices on certain exchanges could be stale and no longer available, but here we’re talking about prices that stay stable for multiple seconds at a time and your customer order is the first event causing the price to move.
  3. The dark pool incident in which Rich Gates sells stock on an exchange at a price below the price he’s bidding in the dark pool is interesting.  It wouldn’t happen on a lit exchange because the sell order would automatically be re-routed to your standing best bid.  Maybe the lesson is to not leave limit orders unattended in dark pools?  It’s like being willing to sell a car, but not posting it to craigslist or AutoTrader.  You only tell your local dealer that you’re willing to sell the car and wait until he has a customer willing to pay more, at which point he buys it off you.  Why would you do that?  IEX is a dark pool, but it prevents such occurrences using the coil of fiber that delays dealer trades.  What happens is that while the dealer’s attempt to take your stale limit order is looping around in the fiber, IEX learns about the new prices being offered in the market and then blocks your limit order from executing with the dealer because the order could get a better price now on an “away” exchange.  That’s pretty cool.  But it’s not well explained in the book.  I bet most readers don’t think through exactly how introducing a delay helps at all in protecting against HFT because the fast traders will always be faster than the slow traders.  Lewis should have been very clear that the delay effectively solves the problem of the SIP being too slow and ensures that the exchange knows the NBBO before the trades are executed instead of learning of it too late.  In summary, IEX seems to be a properly functioning dark pool, but I can’t think of any compelling reason to post limit orders in a dark pool instead of publicly at an exchange.  Dark pools seem to be for traders unable to mask their order size by breaking up the order.
  4. Sub-pennying occurs when a market order is about to take a limit order, but then an HFT jumps in front of the limit order by 1/100th of a cent as “price improvement” for the market order.  The market taker saves a penny while the original market maker remains unfilled and protects the HFT from the price immediately worsening.  The end effect is similar to what happens in dark pools (an HFT disrupts two traders from trading with each other).  But sub-pennying happens on public exchanges.  I don’t know a way around it.  It really sucks when you’re at the front of a queue of limit orders and an HFT takes your fill for a meaningless amount of money.  It’s not clear how IEX helps here.  I assume trades executed on IEX only occur at whole cents, so there is no sub-pennying.  But market takers don’t really care so much about sub-pennying because they’re still getting nearly the same price.  And not having your limit orders sub-pennied only helps if people are routing market orders to the exchange where your order is posted.  Therefore, I think IEX should offer negative rebates and pay market takers to bring business to the exchange.

The midpoint peg order offered at IEX seems like an appealing concept.  Bad trade executions are ones where the counterparty immediately knows he got a good deal.  When you take part of a posted market and don’t push the price to the next level, the market makers are happy, which is bad for you.  When you are the last limit order to get filled at a certain price, you have an immediate loss.  The right way to trade is to take the market when you have high conviction about which way the market is going or have limit orders near the top of the book.  If neither of those are possible, having your trades execute halfway between the bid and offer is nice because you know your counterparty is trading out of a need to trade and not to immediately make a profit.  The trouble is that midpoint peg orders can be discovered by probing the market and then traders can manipulate the bids or offers to change the midpoint before executing against your trade.  For instance, say I’m bidding for 10,000 shares and the stock is 24.82 at 24.84.  Someone can send a midpoint buy and a midpoint sell to see which way there’s trading interest to determine that I’m a buyer.  Then, buy all the offers at 24.84, pushing the market to 24.84 at 24.85.  Immediately send a midpoint sell to get rid of the shares just purchased for a half cent profit on each.

There’s actually nothing wrong with the old-fashioned market model of price-time priority and a single exchange.  Your limit orders are treated fairly.  IEX could be that market, except that it’s a dark pool right now.  It also has to deal with a lot of the good customer market order flow being internalized or going off to exchanges that pay negative rebates.

Finally, here are some excelllent posts by Dennis Dick that clearly explain the most egregious flaw in how the stock market currently works.  Until there’s a legislative fix, do the right thing and switch to a broker who allows you to directly route your orders to public exchanges.  We’ll all save money and free up the country’s top technical talent for solving real problems.

(1) http://premarketinfo.com/2012/02/23/dark-secrets-where-does-your-retail-order-go/

(2) http://premarketinfo.com/2012/07/23/exploring-the-hidden-costs-of-retail-price-improvement/