Line Your Own Pockets

Michael: 00:00

Okay, everyone. Welcome back to another episode of Line Your Own Pocket. So by the time you're watching this, we did a episode three weeks to a month ago on a user question about flipping a strategy, a strategy that worked really well for a while, and then the exact opposite strategy seemed to work. And we kind of pontificated onto why and had a cool back and forth on that. And then we released the episode, and then that same user saw it and said, oh, this is really cool, and provided Dave with a whole bunch of follow-up information that is really interesting.

Michael: 00:35

So now we're going to go back through and we're going to look at that. If you missed that episode, that was basically the synopsis of it anyway, but we'll link it in the description below and in the show notes and all of that so you can go back and and listen to it. So what was the follow-up here, Dave?

Dave: 00:49

Yeah. So this is list member, Andre. And he listened to the episode and and wrote me back really quickly, thankful for the episode. But he added some interesting follow ups, which sort of put it more into context what he was doing. And I asked him some questions about it.

Dave: 01:13

And I think it'd be some good discussion here about what we would suggest that he do. So I think the key points here were that he backed So I've just made some notes here for myself. So he back tested this over fifteen years.

Michael: 01:33

Okay, because that was in sample period. Thing was the we said is, you might not have enough back test window in, but it looks like that that wasn't fifteen years, especially assuming it's an intraday strategy was was more than enough, I think.

Dave: 01:48

Yep. And then so he paper traded it out of sample between 2003 and 2000 or no, 2023 and 2024. So it sounds like a couple of years of paper trading. He started trading it live in January 2025. And he scaled it up in February and maybe changed the sizing a bit in March as the market conditions changed.

Dave: 02:17

So his point here is, he didn't just sort of get lucky with this strategy. He put some thought into it, and there's, you know, there's a good bit of effort went into back testing this, which in what seems like a good process for doing that.

Michael: 02:38

Well, and what I would say is not only that, that's even above and beyond, right? Just hearing that that's fifteen years of backtest, what, two years of paper trading. Yeah. That to me is just way more than I would ever do. So which again was the main thing that we were kind of wondering is that was it a situation where, you know, you just didn't have enough data, but that completely dispels that for me personally, because especially the paper trading for two years, that's not only a lot of data, that's just overall impressive to have the patience to paper trade something for two years.

Dave: 03:17

Yeah, it tells me he's probably he probably has some other strategies that he's running because yeah, if if you're paper trading for two years, that's that takes some patience. The other thing, one of the first things I asked, in fact, the first question I asked after he wrote me after the episode was, how much does this strategy trade? How often does it trade? That's the very first thing I want to know about these things. And so he responded with, what's your guess like like, Michael, I want to hear what your guess is.

Dave: 03:55

How many trades he gave to me in trades per year? How many trades per year was your impression or would your impression be now?

Michael: 04:02

Well, now I'm in the if it's not enough backtest data, then now I'm in the camp that it would have to be like really infrequent. So I'm thinking like maybe a couple of times a week or once a week, if that for it to be to have that much data and to have it instantly flip. I'm so I'm just going to guess like once

Dave: 04:24

a week. You're in the you're in the ballpark. So he said between 2008 and 2022, it averaged 200 trades a year. So that's four trades a week. Four trades a week, yeah.

Dave: 04:40

From 2023 to 2024, it averaged two sixty trades a year. Okay. All right. So it's growing. And then in 2025, it's already had three seventy trades.

Michael: 04:59

Okay.

Dave: 05:02

So pretty interesting.

Michael: 05:05

Well, and just as of recording this, we're two months till the end of the year. So it's already done way more trades, and we we still have some room to go until the end of the year. So we'll definitely be knocking on the door of double of what it normally does, if not, you know, 80% more than normal. So that's not, that's, it's still not as bad as I thought. So the more questions I'm getting, the more kind of stumped I guess I'm getting with it.

Michael: 05:32

Now it's still not a super frequent strategy, but, you know, four trades a week, that's almost one trade a day. There's, you know, I trade strategies, especially from the swing trading side of things that are way more infrequent than that. So I guess my follow-up to him would be like, how bad has it has it become? Like, how big has the the switch happened that you think kind of the opposite is starting to work?

Dave: 05:57

Yeah. So that's roughly the same thing I asked him as a follow-up here. So I said, Okay, what's the profit factor over the years for this strategy? What does the deterioration look like? And we clarified what profit factor was.

Dave: 06:16

He didn't know the definition right off the top of his head, so I clarified what I meant.

Michael: 06:20

And

Dave: 06:22

he gave me the winning percentage, and he gave me the profit factor in a percentage, which wasn't probably a little bit of mismatch in the definition I asked him for. So the win percentage was in that first period, 2,008, twenty twenty two, 67%. 2023 to 2024, 65%. And then in 2025, the win rate is 67 again. So back to the previous winning percentage.

Dave: 07:00

But the profit factor has dropped quite a bit. Think he's So given to me in percentage terms, and I'm going to sort of translate here, I think. I'm pretty sure he means from 2028 to twenty twenty two, two point three is the profit factor.

Michael: 07:20

Okay.

Dave: 07:20

2023 to 2024, 1.7. And then 2025, 1.55. And he's given me this other number two that it's with costs. So once he includes his costs, I think this year it's basically flat or like did significantly below the performance of the previous periods. Let's just put it that way.

Michael: 07:56

Yeah, Okay. So it's it's not to the point there where, you know, flipping the strategy would end up being profitable. You're still right, two thirds of the time, like overall, and again, for people who are new to this, those are those are amazing numbers, right? To be right, you know, two thirds of the time and a two profit factor strategy, that's that's a winner, right? That's one that's even with slippage and commission, all those things would be really hard for it to turn negative.

Michael: 08:25

But if he's now it's come down to costs, and things are kind of breakeven, kind of my next probe would be okay, well, what are the costs? And have they changed from what your backtest or paper trade was to the real world, and then how significantly? So right off the top of my head, I'm thinking, you know, shorting, like low float penny stocks or something like that, where a lot of people will do that backtest, but then they won't incorporate things like borrowing fees and all of this where a strategy could flip. But because he said he was trading it successfully in the beginning part of the year, and it's degrading a little bit now, it makes me think that cost probably isn't the issue either, because he would have had to unless they've changed somewhere throughout the year, right? Which I don't really know how they would have.

Dave: 09:12

Yeah. I don't think I think costs are typically like that's not going to make or break your strategy unless something's like really wrong.

Michael: 09:21

Right.

Dave: 09:24

So my first thought here so let me give you a little bit more of the background. It's not a low float shorting strategy. It's following. It's basically a trend following some sort of macro trend following strategy. So the external shocks he thinks his sense was that caused some of the deterioration was the Trump tariffs earlier in the year, at least the talk about those.

Dave: 10:03

So yeah, I think that covers the background here, the new information that we have.

Michael: 10:13

So not a day trading strategy, right? I just assume because you're a day trader, I think most of the people you work with are day traders. So that might have just been an incorrupt assumption that it was a day trading strategy. If you're using macro, you're probably not day trading.

Dave: 10:27

My sense is it's not a day trading strategy, but I don't know that for sure. That'd be a good question to ask him.

Michael: 10:36

Well, I'm going to assume, right? If you think that the tariff news or some sort of global macro event changed your strategy, it wouldn't make sense that it's a day trading strategy because generally speaking, you know, if that could change like inter market relationships or it could change, you know, the way things work from a longer term point of view. But, know, if you're trading an opening range breakout strategy, it really doesn't matter who the President is or what his ideas are. It's just whether or not there's like statistically enough events to go through. So I'm gonna assume that that's kind of the case.

Michael: 11:16

With a macro strategy, that makes me think that there might be other inputs associated with whatever he's doing besides general price and volume, right? When I hear macro, I think that there's some amount of, GDP or PCI or any of these health of economy numbers that are going into an input and that's deciding whether or not to be long or short to market.

Dave: 11:43

Yeah. When it boils down to it, he's trading a strategy, he's trading a system. There's various inputs you can put into it. But whether your timeframe is long or short, if you're systematically doing it, a lot of the same principles apply. And the first one that I think of is the number of trades increasing so dramatically this year, which is I think that's where I would focus.

Dave: 12:23

And so usually when you have strategies that deteriorate like this, when you look at the performance, it's typically the opposite where you have fewer trades that are happening over time. So it is curious that all of a sudden there's a whole bunch of trades that are happening. There are periods where when you look at the history of trading over the last several years, you see big bumps in numbers of trades, like during COVID. One of my strategies just killed it during COVID, several of them did, just because there was so much opportunity there, so many more trades to take. I don't get the sense that that's the case here.

Dave: 13:13

I don't feel like this tariff thing was a big enough thing by COVID to really just change the trajectory of a lot of strategies. What's your take on that, Michael?

Michael: 13:27

Well, earlier in the year, and generally speaking, what you say bodes true with a lot of especially even short term traders is that essentially when volatility spikes, there's more potentially to do. Right? And during COVID, there you had a huge volatility spike. And you saw that again with the with the tariffs, but that was in April. And, you know, if he was saying that he traded well and then, you know, more frequently more recently it started to die, the kind of the exact opposite has happened, where this is one of the lowest volatility periods of the market that I've seen in a long time.

Michael: 14:03

Ever since all that worn out, we've just been this grinding higher with a very low VIX after that. So that to me, that's what was standing out to me as well, right? If he said, oh, there's way more trades, and we were in some sort of, you know, rampant bear market where, know, the the SPY was moving around a whole bunch every day, I could see it but yeah we've been in this kind of low volatility grind up now for six months now and there really hasn't been anything other I think the biggest pullback's been like 3% so yeah there's something else out there so there's some data point, and I'm still just kind of hammering in on what you said about macro, because that makes me think again, as soon as I think macro, I think, you know, there's, they're using some sort of external reporting, you know, get a healthy economy, that kind of stuff. And that may have changed a little bit dramatically just because of the fact that the government shut down. I wonder if it's like, is it that recent where maybe the data points that he's used to relying on aren't really working out as well?

Dave: 15:04

Yeah. Well, here's one of the things that I always think about with these situations. So, you know, whenever I start, whenever I create a strategy, I'm trying to have as many trades as possible in the system. And it's usually because, like I was describing, most strategies, when they deteriorate, the strength of the signals just becomes less frequent. You end up with fewer trades.

Dave: 15:32

The fact that he's got more trades now is kind of a gift, the way you think about it, because one of the easiest things that you do is find some additional variable that is predictive with his strategy and use that to get rid of the poorest trades in this bunch and get back to what was the average number of trades all these years. I think if you could find that, you could really make this strategy go back to what it was performing before 2025 and still have the same number of trades across those periods. So that's where I would focus.

Michael: 16:16

Yeah, because we're looking at market mechanics not changing too much, right? We had a high volatility period. Now we have a low volatility period. So, you know, there's nothing really changed from a mechanical side of the market. So I'm with you that it's now time to look at the inputs of the strategy, right?

Michael: 16:34

And if it is something like a macroeconomic thing, and those have potentially changed, then that's one thing to look at. But now that you've got a lot of trades, you can you have more data because you're right, it's the annoying part about strategies is when they start to degrade. But it's because you only have a handful of trades, don't feel nearly as confident digging into your own data to try to find what the issue is because there's just not as much data there. When you have more trades, then you have more frequency of trades. To me, that's a way better problem to have because now you can go and look and say, okay, well, things are things are going bad.

Michael: 17:11

But thankfully, anyway, I've got all of this data that I can go and deal with and that I churn through and try to find an answer. So, you know, for me again, it's yeah, look at the data inputs and see if they've changed because as far as I can tell this this year in particular, market mechanics haven't. And then from there, to figure out a way to, like you said, remove some of those trades that might be might be underperforming. But yeah, thankfully, you have the data to do so.

Dave: 17:40

Yeah. So this is a good time where I would go back, look at every like you said, all the inputs. So that means all the columns in your back test. See what was important and what I made decisions upon back when I started paper trading it, see if the decisions I made then are still accurate now and if those relationships are still valid and predictive. And there's a lot of traits here, but it's not so many that you couldn't go through, at least all the recent ones, probably for the last three or four years.

Dave: 18:24

Go back With and look and

Michael: 18:26

what you said, 300 this year? Yeah. That's a weekend. Right? That's a weekend of sitting down and going through kind of if you wanted to do line by line, if you wanted to do some sort of manual work, it's not like there's 20,000 to go through.

Michael: 18:42

There's, you know, 300. So say it takes you a minute per trade. You know, it's a a day's work to go through and try to figure out if there's anything that has fundamentally changed versus if there's anything that hasn't fundamentally changed. But the other's just something as something slight has changed. And that's been enough to kind of throw it off.

Dave: 19:04

Yeah. So so I would go back and guess it was 2022 that he was his in sample set. So I would look at that. I would take that set, look at the correlations maybe between profit and the columns I added that I thought were going to be predictive. Certainly all the columns that were in there and absolutely the ones that you ended up making rules on for the system.

Dave: 19:34

Then I would take a look at the data since then, see if those are still the same. The columns that showed predictive strength before 2022, look at the trade since 2022 and see if it's still accurate and what the changes are. Maybe something else has become important. Another factor has become more predictive. If so, then take a look at that.

Dave: 20:04

We could probably do a whole episode on in sample and out of sample testing. I think, is a perfect example here of when just go ahead and use the entire set. Especially when something has changed like this. I think a lot of traders get hung up on or put too much faith in in sample and out of sample testing when it's it's useful to know about. It's useful to look at, but I've always, for all my strategies, always just look at the entire set because I have done a lot of in sample and out of sample.

Dave: 20:47

And what you find is the in sample, the rules you find are pretty much the same as when you look at the entire set and when you look at the out of sample stuff.

Michael: 20:59

Well, I just say you're naturally hamstringing your your dataset to look at when when you're doing that. Right? And that's kind of been my problem with in sample out of sample is like I'm, you know, so say I split the data in a third and I say, okay, two thirds are going to be, you know, out of sample and then one thirds in sample or vice versa. It's like, pulling out data and potentially important data go through. And I'm like you, I've never once tested something for a period of time and it was amazing, and then ran that further and had it just be completely horrible, right?

Michael: 21:33

It could be a little better, it could be a little worse, it could, you know, and that's just kind of natural ebb and flow. But yeah, I'm with you there where it's it's like I feel like I'm hamstringing myself by just taking away some of that valuable data that could really tell me something about potential column, because I'm just on purpose just shrinking my dataset. When we're trying to do the exact opposite, we're trying to as large of a dataset as we possibly can.

Dave: 21:59

Yeah, and you can, I mean, all decisions you make about in sample and how to split the data and out of sample, they're all arbitrary? There's as arbitrary as anything. So it's never made a ton of sense to me. I can hear some stats nerds in the audience saying, Oh, this is so crazy. I can't believe you do that.

Dave: 22:21

But I've thought about it a ton over the years and I've done it both ways. It's a rabbit hole. You can take yourself down and maybe worth doing one time, and maybe you could convince me that it's worth it to do in some cases. But gosh, it just seems like it's just way easier just to look at the entire set. It's just think it's useful to think about that in an example, out example.

Dave: 22:57

When you're earlier in your career and you don't really fully understand the essence of the system and what the columns really mean and how they're going to affect it. I think it's useful to do it in that case, but the better you get, the more experience you get with trading and knowing how the data is going to work in relation to profit for different signals, I think the less you have to worry about that. And just the less you need to do in sample and out of sample.

Michael: 23:37

Yeah. And I always look at it and there's some of these things that I think are carried forward from regular statistical analysis that aren't applicable to trading because of what you just said. There's it's very hard if I'm doing like, I don't know, an analysis on like how a disease spreads or mutates or or something like that. That is a completely different problem to why are more people gonna buy my stock after I buy my stock. It's just the problem we're trying to answer is different.

Michael: 24:06

So I think a lot of these statistical things sure use them, but some of them I just avoid entirely because, you know, an opening range break is an opening range break is, you know, it doesn't matter if it happened, you know, two years ago or three years ago or five years ago. The slight differences, don't think matter nearly as much in kind of the grand scheme of things. So yeah, some of the things that I find that people use in statistical analysis when they're doing like really large mega complex systems is just overcomplicating it for the needs that we have, which are like you talked about, just more understanding the why of like, why is a gap or interesting? Why are these and and you can replace some of that where if you're trying to be a 100% objective about everything about, you know, like folding a protein something, It's gonna be a it's just a different beast where we can wrap our brains around the concept we're trying to solve. So we don't need statistics to do everything.

Michael: 25:06

Whereas in some really hardcore statistical analysis, there's no way for a human being to ever comprehend the problem you're trying to solve. So you need to have all these statistical rules around everything, because you just can't infer or you can't just place. And the gravity of being wrong is way larger than, oh, this strategy just didn't work quietly in the real world quite the way I thought it would when I back tested. Doing all of these things, I think, is just overcomplicating a lot of that.

Dave: 25:37

Yeah. There's always some statistical test you can do that's gonna invalidate your strategy. So one of the things I like to say is, do you wanna be statistically perfect or do you wanna make money? That's really what it boils down to a lot of times. And it's a rabbit hole to get caught in, to go down and do all these statistical techniques to verify validity and to verify things.

Dave: 26:16

It's just, like I said, do you want to be statistically perfect, or do you want to make money?

Michael: 26:22

Yeah, and a lot, again, a lot of our questions will be answered when we just start trading the thing anyway, right? That's why we always talk about paper trading to trading small to then trading large is that there's a certain amount of testing to do and then go on from there. You know, to to kind of bring it back to this this problem, we've more or less given him, I think, a good direction to go to because he's kind of stumbled across a, a blessing, I guess, a silver lining that he has this more data to look at. Now what if the same person came to you to the same problem, but they didn't have that extra data? They went from 200 trades a year, and maybe this year they only had 50 trades, and they're trying to solve the same problem.

Michael: 27:05

Is the the direction you give them, is that different? Or is it just the same idea?

Dave: 27:12

It's a little bit harder to add more trades to the system than to take trades away from the system. It's definitely a harder problem to do. But what I would do here is start from scratch. Start with your base strategy again throw your assumptions out the window. It's been a long time since you started paper trading this like three years.

Dave: 27:37

A lot has happened since then. There's a lot more trades. You've got fresh eyes at this point and you've got a lot more experience with the strategy. You've seen a period when it has worked really well. You've seen a period where it hasn't worked as well.

Dave: 27:55

When you start from scratch and start your optimization process from scratch, you'll probably come up with a very different strategy with what you know now. And I would have to imagine that there are columns that he could add to his backtest that are going to be more predictive with this new experience that he has. It's so hard to overemphasize the value of additional columns and things that you recognize aren't being captured in your strategy that you can capture in a column. That very well could be the difference between this strategy working and this strategy being put out to pasture. I mean, no question about it.

Dave: 28:43

So as soon as you find something that's predictive and very predictive, maybe more predictive than any of the columns you have, that could be the basis for a whole another strategy from scratch using that idea. So I talk to a lot of traders that I can tell they're closer than they think they are. They're smarter than they think they are. They're closer than they think they are. And they're not very far away.

Dave: 29:16

But a lot of times they're very frustrated because, gosh, something you put a lot of work into is not working well, man, that sucks. But gosh, you're very often so much closer than you think you are, and you're really just one good column away from having a killer strategy.

Michael: 29:35

Yeah, so the moral of it is that you may have to take a step back with a lot of these things, which just means go to the basic idea, the basic strategy, gather as much data as you can and kind of reoptimize. And if it still is looking bad, then yeah, it's it's it's a core issue of your your column library. So that was kind of where I was leading you, and that's where I think is probably this person and anyone else who's having this problem's best use of their time is not trying to mess with the strategy right now until it looks pretty again, but it's to think about what columns that you might be missing. That's what I would, I think, spend most of my thinking time doing and say, you know, okay, what could have changed? What has changed in the market?

Michael: 30:20

What columns could do I think I could add? Because as soon as you answer that question, then everything else is way easier. You're just going through the the process of of back testing and optimization, which is which is simpler. So I would spend the whole, you know, the saying from Abraham Lincoln, if I had an hour to cut down a tree, I'd spend forty five minutes sharpening my eye. It's the same thing I would spend the majority of my time saying, okay, what either what have I learned since I built this strategy?

Michael: 30:48

Because he said he was paper trading it for two years. So it's been a long time since he's, you know, built the core strategy. What have I learned since then? What can I add to my system? What can what columns can I focus on?

Michael: 30:59

And I would spend all of my time until I got some really solid answers on that. And as soon as I got, you know, a handful of columns that are kind of new and different, that would be my focus. And then as soon as you got those, well, now it's just time to go to work, right? And if that doesn't yield a good result, then back to the, I got to figure out new columns and then back again, because and the good thing is with fifteen years of data and then more trades this year than normal, you have the building blocks to kind of rebuild the strategy. You just need that that kind of missing piece to put in.

Dave: 31:32

Yeah. The other thing I would do here is when I say start from scratch, start with your base strategy with basically no filtering. Do a back test for that. How that how are the number of trades changed with the base strategy over this period before you did your optimization and filtered stuff down to what you were actually going to trade? That would be a really interesting thing to see.

Dave: 31:58

Has that been consistent or has that also increased? I mean, there's a lot that could come just from that. And that's probably the first step I would take at this point.

Michael: 32:12

Yeah, because then your

Dave: 32:13

answer to see if you could remove some now. That'd be the easiest thing. Also take a step back, do your original backtest and see how that original one with no filtering, how has that held up over time? How many trades have come through that over time? What does that look like?

Dave: 32:34

That'd be super interesting.

Michael: 32:36

You're right. You're basically trying to prove whether or not the idea as as a whole, and this is why I always save a version of my base strategy, Is the idea in its core changed or is just the selection process of the strategies changed? And you know, you answer that first, and then from there you're able to go and potentially build into into fixing the idea. So as always, the answer seems to be more data. So by stripping it down, you're going get more trades, you're going to get more data that's going to give you kind of more options to play with.

Dave: 33:11

And yeah, the other thing that when you get into this point where you want to change a strategy, and it's been so long since you've went live with it and made changes to it, It's so important to have made good notes about the decisions you made two years ago, three years ago for, why did I put this rule in? What data did I look at to make this decision versus another decision? Having good notes about that is one of the fundamental things that I teach traders when I first start working with traders. I get your process for capturing that and keeping track of that very dialed in. And we use GitHub for that.

Dave: 34:01

In this situation, a goal to be able to do that because you can go back and see Having the exact mindset you were in when you made this decision, that's just so important to be able to do. Because if you don't do that, then you are not going to remember. And you're truly starting over from scratch then if haven't taken notes for the process you went through. I mean, you're totally lost.

Michael: 34:31

Well, I also find it very motivating, too. When I go back and I look at those notes, you kind of realize how far you've come. Because sometimes you read that and you're like, like, that was stupid. Why did I do that? And you're like, Oh, that was, you know, three, four years ago.

Michael: 34:47

I think kind of a good hope for life is that you always look back at yourself five years ago and think you were an idiot. That's as long as that trajectory is going, that's way better than the other way where you're like, Oh man, I was way smarter than I am now. So yeah, always look at it all the time. I like to go back and I like to look because sometimes the flaw is kind of right there, and you now know the flaw, you just you just didn't at the time. So going back and reviewing that, yeah, I think it's it's huge because you're like, oh, now I now I know what the problem was.

Michael: 35:19

It was it was x. So I just Yeah. Past me, you just didn't figure that out yet.

Dave: 35:25

Yeah. Well, I think this has been really great. I appreciate Andre giving us more details about the strategy. I mean, that was very helpful. And yeah, I wonder if there's other listeners out there that would want feedback on their strategies.

Dave: 35:41

I think this has worked well. I'm imagining that Andre is going to listen to this and have two or three work on and maybe give us some feedback on this. So yeah, I'd love to hear from more listeners with detailed specific things about their strategy that we can help with. I think that'd be fun.

Michael: 36:02

I think that I love hearing the comments and then the feedback. And as you can tell, with a lot of our episodes being around them, you know, it's great to get that input from you guys. So make sure you're commenting kind of wherever you're seeing it. We do a good job of reading them and might become an episode. So, I think this is a really cool one.

Michael: 36:23

I hope a lot of people found a lot of value from it. But as always, I'm Michael Noss.

Dave: 36:28

And I'm Dave May. We'll talk to you next week on Line Your Own Pockets.