What To Do When a Strategy Flips
Okay, everyone. Welcome to an episode of Line Your Own Pockets. A very interesting one that I know that both me and Dave have been thinking a little bit about trying to formulate an opinion before we started, but I don't think either of us quite have landed there. So came from another user question. We always love to see this.
Michael:So I'm gonna let Dave read it out and we'll kinda start spitballing back and forth.
Dave:Yeah. So this came from member of my email list named Andre. So it's a really good question here. I'll just read it. I'm just gonna read exactly what he said in the email.
Dave:One question I had in mind for you is whether you have already been exposed to the following situation. You have an idea. You build the proper backtest over different long enough periods of time, you put the resulting strategy to work, you refine it with some extra columns, it's going well, very well even. And then all of a sudden, because of an external factor, the P and L is heading south and it becomes clear that taking the exact opposite side of the trades makes far more sense given the new environment.
Michael:Okay, so
Dave:I What's think there's your initial thought here?
Michael:Well, so one thing, right, that struck me just reading that is his comment on because of an external factor. Right, I think that is probably the hidden crux of the of the statement, because it almost seems like the person knows what the factor is, or at least has
Dave:some Yeah, he did. So he so he had some things in parentheses here. I'll just read this because this might be important.
Michael:Okay.
Dave:Because of an external factor, parentheses, political shift, war, reduction of marketmakers, ..dot. So I can't I'm not sure if this is directly related to his the strategy he's talking about or if these are just things that come to his mind about things that could affect a strategy in this way.
Michael:Yeah, that makes it seem a little bit more like a hypothetical that he's proposing as opposed to a specific example because he's putting out many things that could happen. So my first comment right away is I don't think you ever know. You can guess why a strategy may have changed, but I don't think you ever know. I always look at it like how, I know you don't watch CNBC and I don't either, but you know that the end of every day they go the markets were up for this reason or the markets were down for this And it seems like they have like probably two articles ready to go. And it's like the market was up because of job reports or the market was down because of job reports and they just wait for the market to end and then they publish one.
Michael:So it's, you know, I think you could, you could maybe in some scenarios have a decently good idea, but I don't think you ever truly know why. I don't think it's really important either, but I that's kind of the first thing that stuck out for me is that if you knew the why, I think it would change the equation entirely. Like if you knew for sure my strategy worked because of this one specific reason, and now it doesn't work, and the opposite works because of this one specific reason, I think I'd be kind of open to flipping the strategy, but I just don't believe that you can know. Does that make sense?
Dave:Yeah. So I have lots of thoughts about this. But first reaction to this was, okay, he's not creating the strategy in the right way. And what I mean by that is I don't think I have ever been in this situation with any strategy I've ever created, where the version I get live, all of a sudden, literally the opposite has the edge, the opposite side. That seems like a big red flag to me that he's doing something like something is wrong with his strategy creation process in the first place.
Dave:Or not necessarily wrong, but there's some steps missing.
Michael:Well, so let's contextualize it, I think, with an example. Say for example, you had a strategy that, I don't know, it bought a MACD cross, just to make it simple, right, every time the MACD cross you bought it. And then all of a sudden you found out that you should actually be shorting every time the MACD cross is up, where you were buying it before. In that scenario I totally agree, what will probably happen was your strategy wasn't that good to begin with, it was tested wrong, maybe it was tested on a little bit of, you know, a smaller amount of data. That was the one thing that really came to me is that a very short backtest window I think could do this.
Michael:Because say you had three months of amazing performance and the equity curve looks up, and then three months later you run the same backtest and you're only getting three months worth of data and the equity curve is pointing straight down. You would say okay, the opposite was true. But then if you take those two backtests and put them together, you have probably a really good period of time and then maybe a natural pullback in a drawdown of equity curves. So that's the kind of the initial reaction to me. If it's just pure, right, worked on month one doesn't work on month two, it's like, well, is that just the ebb and flow that normally happens with any kind of equity curve that you're gonna trade.
Dave:Yeah. I think I've worked with a lot of traders that have that exact scenario that they're using trade ideas. Trade ideas has a three month window for back testing. At some point they realize, okay, three months is too short for this particular strategy because of exactly what you've seen or what you just described. They've run it for a while, it looks great, then they wait three months and it looks very different than it did.
Dave:Yeah, I totally agree that once you get a large enough backtest window, alleviates this problem to some extent. The main thing, the main crux of this, well, I've got several things to say. Hope I can say this coherently. First But thing is when I get a strategy live, I've gone through the process of figuring out what rules to add that make a lot of sense, that are coherent, and that I think are going to continue to be predictive. So if I run into this situation, I would feel like I have failed somewhere in my strategy creation process because this is an absolute aberration.
Dave:If I've gone through my process right, I understand that the edge is on one side and that I've looked at a long enough window, I've thought about it deeply enough, I've added the right rules that make sense with the signal. There's just when you go through that the right way, this is not going to happen. You're not going to come into a situation where literally the opposite side has the edge. So I would say you need to go back to your process and think more deeply about it because this is actually a pretty valuable spot for him to be in because when you get to the process of creating a strategy, the person that you're trying to convince that what you're doing is correct is the person that he is right now when the strategy falls apart. What steps can you take now to convince that future self that's skeptical, they're in a drawdown?
Dave:Literally, the opposite side of the strategy has the edge now. All right, something's wrong. What can you do when you're creating the process to prevent this situation?
Michael:Right. Because let's say you took the strategy that looked good on month one and looked bad on month two, and you did the just the long backtest window and you realize month two was just a normal pullback. You know, it just doesn't work as well for some market condition or whatever. You still have to convince yourself to trade it in month two, which is a different beast entirely. Have to be able to sit there and say, Okay, I had you know this, it's one thing to look at the you know, the equity curve dip down a little bit.
Michael:It's another thing to be there in that moment, and to make sure that you're actually hitting the button kind of correctly. So the first and then I think the most important thing is that the person definitely should not be flipping the strategy and really shouldn't be trading the strategy period. Right? If you are not confident that the edge is going to stick around, then it's just not a strategy you should trade. Because if you're thinking that the edge is flipping from long to short, then when is it going to flip long again, right?
Michael:So even if it's working to the short side for a period of time, and then all of a sudden you go into a little bit of a drawdown, are you going flip the strategy again and just buy the signal that it told you to sell right away? So making sure that you can kind of understand that right off the bat, I think that's the most important thing. Or, you know, you're probably going to just be flipping back and forth, and you're probably going to be doing it at the possible time at, you know, at the time you should really be kind of sticking to your guns.
Dave:Yeah. And, you know, I'm gonna bring up my what's becoming my favorite phrase, and that's path to confidence. What is your path to confidence with a strategy like this? It's easy. You could say, you could trade this, flip the switch, but you're never going to be able to trade it with any significant size.
Dave:And that's what you're really looking for. What's the idea? What's the strategy where you're confident enough in to trade it with significant size? Anything else, you're just trading for no real reason. You're not trading seriously.
Dave:So that's the really important thing. And like I said, I'm an optimist. My wife says I'm too optimistic lots of times. But I think he's in a good spot here because there's a very valuable lesson to learn here. And he's in the very perfect position to learn it because he's done all this work and this thing has happened.
Dave:Figure out why this has happened and how he can fix this process so that this doesn't happen again.
Michael:Yes, because like we talk about is, you know, one strategy is not enough, right? Especially for any kind of long term career. So whatever root cause is causing, whether again, it's a short backtest window or some error in the strategy building process, by kind of fixing it and understanding it now, it's going to help you hugely in the future as you're able to go through and say, Okay, well, you know, now I know that, you know, the backtest window was too short or whatever you find the problem being, I just need to apply that fix on everything that I'm going to build going forward, and then no longer have this kind of flip flop from good strategy to bad. I just keep in my head trying to think of any scenario in which I could see this occurring, and I just really can't. I just can't see a world in which, you know, there's plenty of times where a good strategy could lose a little bit of edge or lose edge entirely or something like that.
Michael:Could see that happen. But to do the exact opposite thing, to me, the only thing I could think of is that's a short term window of the majority of trades you're taking failing. That's why if you were to put it in a back test and flip it, all of a sudden it would work well is because you're just into some period of time where, you know, maybe the last 20 trades you made just absolutely sucked. And so then if you took that small amount of backtest window and flipped it, then all of a sudden you feel great.
Dave:Yeah. So maybe we should talk about exactly what I would recommend for how to proceed because, you know, he's put a lot of work into the strategy, obviously. Maybe there's some way to salvage it and let's imagine there isn't, and let's go through some steps to see how you could salvage it or how you could get the answers that might get you a path to more confidence here. The first thing I would say is get more data for your back test. Because the real question does this happen often?
Dave:Like how often does this happen? Like go back and see when's the last time it happened? How long ago? How long did that last? And so that's the first thing to see.
Dave:How many times has this happened before? What did those look like? How long did it last? Imagine what it would have been like. Were you trading through that at that point?
Dave:What would you have thought about? So that's the first thing I would do.
Michael:Yes. No. That so I just the only thing that's interesting about that is I could see a world, say the guy backtests the same thing for five years. What if this was the first time it ever happened? Right?
Michael:So he's got the backtest window now, which again, in my brain, that's that's probably what's going on. But let's say it's the first time it's happened. In what scenario do you, you know, what what would you do with that in that situation?
Dave:Okay. So if if that's the case, then I would not be confident in the rules I've applied to the strategy. So what I would do is, you know, get the log get the bigger back testing window, remove those rules that I went through to create. Start from scratch. You're starting from beginning.
Dave:Go back to square one. You're starting with a backtest with a lot of trades, way more than you're going to take. Make sure you've got the columns in your column library that get added to the backtest. He said he was using columns, so it sounds like he's backtesting the right way. Make sure, you know, go through each one of the rules that he applied and think more deeply about it.
Dave:Something one of those rules is there's probably one of those rules that's causing this issue, that's doing the filtering such that this set of trades is presenting itself now that's bizarre. I put my detective hat on and figure out which ones those are and really examine that and think about that deeply.
Michael:Yeah, and that's exactly what I was trying to tease out for you is because that's, I think ultimately step one is that does the base strategy do the same thing as your optimized strategy, right? If the simplest process I would see is one, again, make sure you're getting a long enough backtest, and then two, is just compare. If your base strategy has degraded a little bit, and the columns that you're using is wrong, then the columns could make the strategy look awful. Whereas if you add a different set of columns in there, it wouldn't have completely flipped, it probably would have just degraded a little bit with the overall backtest, because that's the only other scenario that I've seen it happen is where, you know, you have a little bit of a pullback in your overall equity curve in most recent data, but the columns that you've picked in order to kind of optimize that were the wrong ones, so it's really kind of so if you picked a different set of columns, so the first step is again to go back and make sure you have all the data, and make sure that that base strategy, the underlying idea behind your trading is still profitable.
Michael:And if it is, then you know it's just a different set of columns or something has to change there as well. What I would imagine that you would see is that yeah, there's probably just a natural pullback or something occurring in the base strategy, but the columns you have selected are kind of exacerbating that.
Dave:Yeah. And I think it's important to point out that you're saying sets of like the wrong set of columns. Well, they're not like right or wrong necessarily, but I think it's important to think about the rules you're applying and those are gonna correspond to columns. So it's not, I don't think it's, I don't want give people the impression it's like, okay, this sets right, this sets wrong. Like, you're going to have to go through each one of those and really convince yourself.
Dave:And the way to do that is be very skeptical of the rules you apply because the situation could occur and you want to be able to have confidence enough in the strategy to trade through a drawdown. And if you're concluding that the opposite side has the edge, then that's a problem. Mean, that's a big problem.
Michael:So, I used to have this one guy that I used to trade with at the at my old prop firm, and he would always talk about the Costanza trade, where he was doing so bad for a period of time that he just decided he was gonna go in and do the opposite of whatever his natural inclination was and end up making money. But, obviously, that's not how you wanna do it for the long run at least.
Dave:Yeah. That's funny. I remember that from Seinfeld. But, yeah, I think it's you know, it would be this is not a situation you wanna find yourself in. You want to You want your strategy to have continued and worked a little bit closer to how you had planned and how you had made your rules.
Dave:So I think he's actually in a good spot here. You know, because he's already gone through this process once of optimizing the strategy. So, it's very often, it's very valuable to go back and start from scratch. So, your backtest to current, go back and do the optimization again. I guarantee you will learn something doing that.
Dave:And the time to do that is not just when something's falling apart or the opposite side of the strategy has the edge now. It's often very useful to to do that in many cases.
Michael:I think that's actually probably another podcast we should write down is when to optimize your re optimize your strategies. But I agree is that sometimes I find that if it's a strategy I've been running for long enough, want to re optimise it, not because it's bad or because even I've discovered a new column or something that'd be interesting, but I feel like I'm better than when I built it, if that makes sense. Like I have actually gotten better at the process of building strategies and knowing what's important and able to go back and do it. Then also, like we mentioned, the hardest part of this game for for us usually is idea generation is coming up with that new thing that could be a good strategy. So he's already come up with a standard idea, a base that, you know, showed promise.
Michael:And then at some point he feels that it has fallen apart and the exact opposite's working. But I think that probably a combination of the two, the fact that you have that idea that at least worked at some period of time, and so will most likely continue to work again. And since you've done that, you've probably learned a whole lot that it would make sense to kind of strip it down to bare bones and just start building it back up again.
Dave:Yeah, I mean, this is the more you do it, better you'll get at it and the more robust your Yeah. Strategies will So, you know, this is the thing to do here is not panic, but, take a step back and pick up the pieces and figure out what parts of the strategy do still have validity and how you can modify the strategy such that you know, this has this is more robust. And it's he's probably you know, it might sound like he's, you know, miles and miles away from having something that works here. But my guess is he's closer than he thinks.
Michael:Well, and that would shake out with whatever the base idea is, right? If you go in and you you strip everything out and you test, you know, we talked about how it might just not be enough data looking back or something like that. But if you strip it all out, you've got enough data going back and you test, you should be able to get to the point where you have, again, some comfort in the overall idea again, and you're not at the stage where you're like, hey, I have to do things, you know, I want to take the opposite side of the trade or something like that. And then it's just, yeah, it's just that process again of, you know, maybe you need to increase your column library in some way or maybe you need to, you know, if if the example doesn't immediately strike out at you, then, yeah, maybe it's time to just review some of those potential inputs you can change and see if there's there's something that you can you can pull out of the other side there.
Dave:Yeah. I mean, the other important thing about this is so when I designed a strategy for myself, I'm designing it so that this doesn't happen. This exact thing does not happen. Right? And of course, you want things to stay forever and go well forever.
Dave:Sometimes that happens, sometimes it doesn't. But the better you get at creating strategies like this, the less you're going to have to optimize, which is really valuable. You get better at it. You can do it once. You don't end up having to go back and reoptimize over and over and over and over again.
Dave:You can do it once in a smart way knowing that you're not going to have to do this very often and you want to be doing it less often. That means your strategies are working well. That means they're robust. They're not going to all of a sudden fall apart for a reason you had not thought of. That's the goal.
Michael:Well, I'd say that's the inherent laziness that you always joke about with developers, right? If you don't, if you have a wide basket of strategies that only need each strategy only needs a touch up very infrequent, then all of a sudden you're decreasing your workload dramatically. If you had to go in once a month and do, you know, 10 strategies over and over and over again, that in and of itself is going to be a full time job. So, getting to that point where you're not having to do that isn't just to make the strategies better, which of course is the goal, but it saves work, saves time.
Dave:Yeah, and also, you know, there's something when this happens to a strategy, kind of hurts, right? This is something you've built. You've put your heart and soul into this strategy. You've done a lot of work and then you put it live, you traded it. Mean, this is like your baby at this point.
Dave:Once something like this happens, it really hurts. You really want to prevent the situation and do a better job creating this baby again so that you don't have to do it. The other thing to think about is it's one thing if you have one strategy and this happens. Maybe reoptimizing more frequently is not that big of a deal. But when you get more strategies, when something like this happens, it's almost like an emergency out of nowhere.
Dave:This all of a sudden is the top priority, get the strategy back in action. It's always going to come a point where it's going to come at an inconvenient point, especially the more strategies you have. This is not something you want to have to work on right now. Probably got a large priority list. There's probably other ideas that you want to make into strategies.
Dave:So when you have to come and do this rework, that really of throws a wrench in your process. I mean, it happens. Yeah, you have to deal with it, but it's good. You can set up your strategies in such a way where you try to minimize this.
Michael:So let's just for and again, I think we should do a whole podcast on this, but how frequent do you think is probably too frequent? Know, and like, again, obviously perfect world, you never have to touch it again. You build the strategy once, it just keeps going, you know, that way forever. You know, we don't live in that perfect world, so it's going to be some variation of it. But when would you say, Man, I have to go back to this strategy way too much.
Michael:There must be something underlying a problem here that's underlying. It's just not natural touch ups.
Dave:Yeah. The way I think about it is, so you want to have a better finger on the pulse of the strategy. So what do I mean by that? Well, if the first time you notice that something's wrong with the strategy is when the edge is literally on the opposite side, you've let too much time go by. You don't have your finger on the pulse of what's going on.
Dave:Yeah. You have been alerted to this a long time ago. And so the first thing I would be looking at is reconciling with the backtest to see. You can get a much earlier indication of when something is starting to deteriorate before you get to this point. So you can get an early indication knowing that, Hey, it's not an emergency right now, but I can see that something is happening here that I didn't predict with this strategy, so let me put it on my list to get to.
Dave:That's a much different situation than, Okay, this thing's falling apart. I'm losing money with it. This is a hair on fire emergency. We got to do something about it now. It's a very different thing.
Michael:Just like everything, if you're able to address it when you're in that more calm state of, Oh, right, it's starting to it's off its all time equity high that something might be up. You're in a better mind state, you can, you know, you're not chugging a coffee at ten at night and trying to work through the night to get the strategy kind of up and running and fixed again, you're in a more kind of calm and it gives you that relaxed ability to think about potentially what could happen. Whereas just like anything, just like if you're in a trade and panic is sitting in, you're not making good decisions. If you're panicking about you actually your setup and trying to fix that, you're probably not going to make the best decisions anyway, which is why, you know, tweaking these things or looking at these things before that happens, it's just saying, Oh, this one's a little bit, you know, maybe you got five strategies going. You just noticed that this one's like really starting to tick and you're like, well, let me take a look at it and see see what's going on.
Michael:And that's why it's so important to run what you've done and then run what the system has done and then go back and make sure that that's kind of constantly aligned and that you're looking at your backtest periodically. Because yeah, if you're just there's never a time in your life where you're like, Man, I really I wish I was more panicked while I was doing that thing. Would have I would have done it so much better.
Dave:Yeah. Yeah, I've seen some traders that and it's tempting to do this. Just look at the top line P and L across all your strategies. You're not really diving deep into each one. You're not even looking at very closely how each strategy is performed.
Dave:Sometimes you could be making money, but a strategy could be falling apart and doing really poorly relative to the other ones. It's really important to have a close look at that and have a good system for doing that. That's where a lot of automation you can put in place to alert you or to give you a view that's automatically produced that you can just eyeball and automatically comes up on your machine at the end of the day. There's a lot of stuff you could do with automation to to make this very easy on yourself.
Michael:Well, the business point of view there would be like someone running a bar and just like looking in at the at the end of this month, I have more money than than I wanted to. Meanwhile, you've got like a bartender who's just fleecing you the whole time and you're just not paying attention. Yeah. Right? So you know, you're like, oh, you know, my bar made $50 this month.
Michael:Like, yeah, but the so the bartender, right? So if you if you had been paying closer attention, you would have made closer to $100 this month. So it's yeah, using the business analogy, it just you got to keep an eye on all these things. This is in theory your business, right? And because we're not the ones pushing the buttons, we're not spending all day staring at a million charts to try to figure out what to trade and when, that should, you know, create the time.
Michael:So again, for that same business analogy, because you're not the one pouring the drinks to the customers coming in, doesn't mean you just check out, right? You've given that responsibility to someone else, in our case, a bot, and then your job is to, you know, sit back and kind of keep an eye on everybody.
Dave:Yeah, I like that analogy. I think, yeah, I think that's a good one. I mean, you are running a business and you are fully responsible for your own trading. There's nobody that's going to come run the report for you and show you what you should be doing. I think it's part of, like I always say, your path to confidence.
Dave:What can you do now to make yourself a better trader tomorrow, next month, next year? Part of that is automating a lot of this stuff that's kind of a pain to do without automation. I mean, it's not it's like not hard, but it's kind of a pain to do. It's the perfect kind of thing that I like to automate.
Michael:Yes. An annoying, repetitive task. Yeah. Right. You can clear those up, then you're more you're in a better headspace to to start looking at the bigger picture as opposed to as opposed to the smaller one.
Michael:So any more words of wisdom for the the gentleman? I know you had. I know you had some notes there, but anything else you wanted to cover?
Dave:I think that covers it. You've made me pretty thirsty with all this talk about bars time for beer.
Michael:Well, always, I appreciate, we appreciate the questions. You know, I think some of the best, I think some of the best podcasts we do are from user questions, because they're things that we're probably not thinking about. And it doesn't mean that we have it figured out. It just means that it's just sometimes someone can present a really, a question that your initial response is obvious, which this one was, and a couple of the other ones was. But the more you think about it, you realize that the answer that you've just kind of been sitting with maybe your whole life might be something that deserves a little bit more thought because it's just it's obvious because you've been doing it so long doesn't mean it's right.
Dave:Yeah, I love these questions always make me think. Yeah, I think they're really valuable. I know if one person's wondering and asking the question, you know there's others that aren't speaking up and are benefiting, I hope, from the answers and the questions themselves. So yeah, thanks so much for these questions. They're really valuable for everybody, I think.
Michael:Absolutely. So as always, I'm Michael Noss.
Dave:And I'm Dave May. Thanks for joining us online. Your own pockets.
Creators and Guests
