Line Your Own Pockets

Michael: 00:00

Okay, everyone. Welcome back to another episode of Line Your Own Pockets, episode two in our little kind of mini course about about back testing. And something we kind of talked about last one, I I really liked the last episode, but it was to the point where we could have probably just recorded it for, like, three hours. So I had to stop us when we got to a certain topic, and then now we're gonna go back into this topic. And so and the reason I think that's really interesting for, what Dave's doing working with the Strategy Cruncher and all of that is the ability to or the importance of having your own kind of set of columns.

Michael: 00:36

So when you backtest something and you export those trades, that you have more than just I bought here and I sold here and here's the p and l and then the things that most backtesters give you, but you get a little bit deeper into looking at, you know, the important things of of these different column types. So, Dave, first go over and and I think explain a little better what I'm talking about, and then we'll we'll dive into what you should have for columns and and a little bit more of why this is important.

Dave: 01:04

Yeah. So, you know, last time we discussed how most people just backtest wrong. They're just doing the wrong thing. And what they do is they guess and check where they're trying to come up with the final answer on the first backtest when a much better approach is use that first backtest as a way to collect a whole bunch of data, like every instance of your signal. You're not going to trade all of it.

Dave: 01:31

You're going to trade just a small portion of them, but that first backtest should have very few filters, a very comprehensive list of all the instances of the signal that you're trying to capture. Once you get that, then you want to add columns to this one backtest. And then it basically gives you a path to find your answer, quote unquote answer, like what you're eventually actually going to trade. So that you basically say, okay, here's a list of signals. Which ones am I going to take and which ones am I going to skip?

Dave: 02:07

That's really all trading boils down to that. Yep. You've heard, I mean, remember when I first started trading, like there was a common thing was, you know, trade new highs. Yeah, when you go and back test that, well, there's all sorts of different I mean, there's new highs all the time. There's all sorts of different situations where a new high occurs.

Dave: 02:36

So there's like, that's not a complete system. And the way you come up with a complete system as a result of it find that signal and then use filters or columns to figure out, okay, under what circumstances are trading new highs profitable? And that's what we're really talking about here.

Michael: 02:57

Yeah. I think so. Ultimately, this is one of those things too that people land on naturally and but backtesting makes way quicker. So my example that came to my head when you talked about new highs is most people go, okay. So you wanna you don't wanna buy a stock that's breaking out to new highs, but then they instantly start to put caveats on that and like, okay.

Michael: 03:19

But but maybe only in the first hour of the day and maybe only on a stock that's in play, we'll use, like, you know, more vague terminology. And that is great because that's kind of step one of actually systematizing things. Right? You you take something and say, okay. Well, let's buy new highs.

Michael: 03:35

Let's have a robot buy new highs, and let's only have it work during this period of time. And now in play, let's just substitute that with relative volume. So now you've you've kind of done this anyway, and and you can see how naturally the goal is to take something that's very simple of buying new highs, just continuing with the momentum of something that's already happening, and you're removing junk to to get started with. And kinda what you're saying is to go one level deeper. You have a back test, and and hopefully that back test at least shows some edge to begin with.

Michael: 04:08

Right? It's something that you're looking at, and you're saying this, you know, makes sense. It's something that, the broad idea works. Now let me try to figure out under what scenarios it works better or even what scenarios it it kind of doesn't work at all.

Dave: 04:23

Right. So to do that, just like what you said, in play. Lot of people use that term. Of course, it doesn't mean anything and it means everything, right? Means different things to different traders.

Dave: 04:37

And it's important to quantify exactly what you mean. And This concept is really important for people going from discretionary to mechanical trading or automated trading.

Michael: 04:53

Like

Dave: 04:54

that bridging that gap is what some traders are really good at and others struggle with. There are a lot of discretionary traders who do have a sense in their mind what is in play or whatever that means to them. But it's a whole other matter of, okay, defining exactly what that is so somebody else would understand it and could follow it. That's really what you're doing because you're teaching a backtest what in play is. And the first thing I think of in this example is find some columns that you think describe in play.

Dave: 05:35

It doesn't have to be just one. It could be several.

Michael: 05:37

Yep.

Dave: 05:38

Put that in your backtest and then see what happens when you use a different combination of filter values in your backtest for what you think in play is.

Michael: 05:51

And this it it's funny just to go back to you were talking about how a lot of discretionary traders, I think, are kind of systematic traders in waiting. And it's because of of this. You can generally sit down with someone over a couple beers and say, well, what do you do in the market? And they can usually describe something that, as somebody listening, you could start to, again, the more I get into this world, the more you see it. You could start to say, okay.

Michael: 06:19

I can understand how you could potentially systematize this. Right? Like, they have the pieces there. Okay. Well, I look for stocks that are gapping up.

Michael: 06:27

So you can so in your head, go, okay. You have to define how much. And then you look at and the devil always comes down to the details if you get this broad framework. And I think part of the thing that holds a lot of people off of or maybe back from systematic trading is they don't know how to find those exact details, so they develop, you know, something they call a field. But what you're saying is that it's actually pretty easy.

Michael: 06:51

Right? If you went through even through your own trades and you had these column sets, you could go back and say, okay. Well, let me rank them by you know, do a bar chart. Right? The p and l on the bottom and and how much, how much ARVOL at the top or vice versa.

Michael: 07:07

And then just look at it and and see is there and I think the other thing that's super interesting there is that quite often, you'll find that there the answer might not be what you expect. And I think that might be another reason people avoid it as well. Because you would expect in our example that if, you know, the stock being in play matters that the more our vol generally, the better or even not worse. And what I've noticed doing this a little bit is sometimes you go in and you just kind of picked a number that works, but the number before doesn't work and the number after doesn't work. And it's just like this random kind of aberration that, you know, maybe you heard of online or someone told you or something and it just, yeah, it really doesn't mean anything.

Michael: 07:53

So it's really interesting. And I wonder, to my point, I wonder if that kind of worries people as well in which they're, they don't wanna be they don't wanna be shown with data that what they've kind of believe at their core is is not right, which, you know, there's a whole bunch of parallels we could make there for sure.

Dave: 08:14

Yeah. It's definitely you you definitely get confronted with situations that you didn't think were true. And as a trader, you actually should be seeking those out. That should be a really interesting thing that happens to you when you find something that challenges your assumptions, because obviously it helps you get better, but also it may help you improve a strategy you're working on now, but the real value there is what that implies so you can create strategy number two or three, the next strategy. When you have these market truths that you can prove with a backtest, it opens up all sorts of doors for new strategies that you're not thinking of now, but that you you that like you could be pointed to by looking at the data.

Michael: 09:07

Yeah. Or new strategies or just new filters for your own strategy. So let's use the example of of of just buying new highs. If you run that test and our vol is not doing what you think it should do, then the question is, okay, is my whole premise about this maybe wrong? Maybe the new high hits first and then the increase in volume hits second or, you know, the data should open up more questions about, okay.

Michael: 09:35

Maybe maybe there's another factor that matters. Maybe it's, you know, how much it's up or down on the day or any any number of things. But, you know, seeing that data should kind of lead you down, I think, a little bit of a better path to to get started with it. So, I think the first question from the audience is gonna be kinda in their head is, okay. That's all well and good, but how do I know what columns to to include in every single trade?

Michael: 10:01

So I think that's the first that was my first question when I was looking at this one of those.

Dave: 10:07

Yeah. So that that's a great question, and this is where the real power of this approach comes in. So analogy I have here is, as a coder, as a developer myself, one of the real ways you can leverage what you're doing in your code base is to use what's called a library. Instead of like writing, copying and pasting code to have it do something in every single instance that that function needs to happen, all these different projects that might be using this same code, you can create what's called a library and that's shared amongst many different projects. So all of a sudden you've got just one place that houses these functions.

Dave: 10:52

You can change one line of code in this one function and all these other projects automatically change. You can take that concept and do the same thing with backtesting. So you can create these columns. You don't have to come up with new ones every time you do a backtest. You can include what I call a library of columns, a growing library of columns.

Dave: 11:17

They automatically get included with all your backtests. So all you need to do when you think of something, a new column to add relative volume, like you said distance from fifty day moving average I mean, you're going to think of new ideas for these all the time, add them to your backtest, and you've got this growing library of columns that gets more and more valuable as time goes on and that you can include in the current backtest you're running. But every future backtest you run from this point forward has this growing body of columns that are automatically included with every trade you do with any backtest.

Michael: 11:59

And that I think is is the huge part is that it seems like a lot of work, and it is. But like with most things, I've noted with systematic trading, it's it's front load of work. Right? So you do the work, and then you get this kind of residual benefit forever. And it doesn't mean that your work's done, but like as you mentioned, if you have a core of these are the columns that are important to me, then it's more of right?

Michael: 12:22

Like we talked about go in the woods, go walk the dog, or whatever. And then when the idea hits you for a new column, you add that to that list and and then go and do some testing from there. Now, you know, just for the nitty gritty, I think, for the audiences, like, where should they get started? So for me, my brain instantly, like you you mentioned, said, okay. Look at, you know, the major moving averages that most people look at, volume, standard price stuff.

Michael: 12:48

I think that's, for me, the kind of the best place to get started. And then like you mentioned, you're just waiting for kind of inspiration to hit. I think, to solve a very specific problem on a very specific strategy. And then when you get that call and you get that idea, then, like, everything we do around here increases a lot of work for yourself where you now have to go back and say, okay. Did I unlock something for other strategies?

Michael: 13:14

But do you have any other reference? Because for me, was just kind of I just sat there and thought for two seconds and then threw a bunch of things together to get started.

Dave: 13:22

Yeah, well, there's certainly some foundational columns that are going to make sense with any strategy. You know, ATR, average true range, how much does the stock typically move? That's going to be foundational and that should be included in everything. I have two thoughts when it comes to creating columns. One is they should be normalized.

Dave: 13:48

And I've preached about this a lot on the newsletter. They should be normalized. So what do I mean by normalized? Well, imagine, like, so price is not normalized, right? You've got some stocks that trade below $5 even below $1 You've got some stocks that trade over $1,000 or even higher.

Dave: 14:13

If you use price as a column there, that's not going to normalize very well. Like, imagine another one, like change from the close. If you use that in a decimal format, not a percent, it's not going to generalize very well. It's not normalized across the entire market. So when you try to use a filter like that, you're going to have these, what I call, cliffs in the data where a $0.50 move in a really expensive stock is like nothing.

Dave: 14:48

It's basically a tick, but a smaller priced stock that's enormous. So that's what I mean by find columns where they're normalized and a similar value is generalized across basically the entire market.

Michael: 15:06

Mhmm. Yeah. And I think that's it's very easy to describe with, you know, just standard examples of, you know, people say, oh, I get this question all the time about, like, trailing stops. What kind of trailing stops should I use? Should be, like, 20¢ or something like that.

Michael: 15:21

And I'm like, okay. Well, 20¢ on a $1 stock is a massive trailing stop. 20¢ on Tesla or, right, MicroStrategy or something like this is is gonna be almost the spread, and you're gonna get clicked out instantly. So, yeah, you should take whatever it is and think about not comparing it to any sort of broad averages, but back to the stock itself. Right?

Michael: 15:45

So that's how I always look at it when when we're talking about standardization is everything you do has to be related to some other way of on how the stock moves, which I why I think ATR is such a good example. Because you're just saying, how much does the stock normally move for whatever bar length you're looking at daily, weekly, or or one minute? And just let's compare that and then compare that back to to what's happening now. I think that's a big one. I also think when it comes to it's why people like moving averages so much is because they're an average of of prices.

Michael: 16:18

So this is why something like a Bollinger Band or a Keltner Channel or something is so popular because it's it adapts very well based off of whatever whatever security you're looking at. So that is one that I think makes perfect sense. Also, when we're talking adapting things back to itself, like, where is it inside of a range, think, a really good column as well. Like, where is it inside of a twenty day range or a one year range or or something like this? Because, yes, it would really mess up the data if you don't do this well because you'll end up yeah.

Michael: 16:49

It really, screws things up. Right? Because a 2% move on a $1 stock is is nothing. Right? It's nothing at all.

Dave: 16:56

Yeah. So really what you're doing if you're not using normalized columns, really what you're doing is you're leaving a lot of money on the table. And here's what I mean. When you use normalized columns, you're maximizing the number of trades that you're going to have in your system. So imagine if you were using something that's price based or change from the close in a decimal format.

Dave: 17:24

What you'll end up with is a small pocket of trades with your system that look well with that particular column, but you're ignoring a whole another a huge set of trades that don't look well, but would look well if you used a generalized, normalized column to find what's predictive. And that's really what we're talking about is finding columns that are predictive to your signal. What predicts that it will be profitable? That's really what you're after, and so they have to be normalized. The second big thing I wanted to mention is they have to relate pretty directly to your signal, to the main signal for your strategy.

Dave: 18:14

So that signal needs to be something unusual. It can't be just some random event. It has to be something unusual that's occurred. A lot of people I hear a lot of people say sometimes I mean, people we know, Michael, will say, Ah, if I just have good money management and risk management, I can take a random entry and still make money. That's such BS.

Michael: 18:38

Yeah. And it's like A

Dave: 18:39

little BS.

Michael: 18:40

It's like, you know, if if I ran opened a random business and managed it correctly, I could make money. It's like, maybe, but why? I think would be the would be the biggest answer. Yes. You could probably if you just threw darts at a wall eventually, especially in the long term because things trends higher, but that's that's not the point.

Michael: 19:04

Right? That's not what we're trying to do. Right? What we're trying to do is is to beat some sort of average or to, you know, to outperform in some way. And and, yes, it's it makes perfect sense.

Michael: 19:15

Now when you say predictive to your system, there's another thing another, kind of episode that we did that people should go back on. There's a lot of things to optimize for. So are you saying, you know, you're cutting out the bad trades? Does it mean you're cutting out the big losers? Does it mean, like, what is it that when their people are trying to remove, what are they trying to to look for?

Michael: 19:37

Right? And this is more of those I'm asking as the audience because I I know the answer to it, but I think that's another thing that people could wonder is is we have what am what am I hunting for here? Because you're as you're right, you need to take a random strategy and throw it in there. It's gonna tell you some stuff that's predictive, but I think it's probably because you're sorting for probably the wrong thing.

Dave: 19:59

Yeah. So what you're looking for is are columns that are correlated with profit that you can find a rule for that would improve the system by excluding some crappy trades and leaving good trades in there. So that means coming up with columns that are So basically, have The way I think of it is there's probably two sets of columns. One are what I said before, like foundational columns that are going to apply to everything. That's one set of columns that you should be adding to your library.

Dave: 20:39

And then the second one is are ones that apply directly to your signal. What describes your signal? Under what circumstances is your signal stronger or weaker? And you should have some theories about that. And when you do have theories about that, you should be adding them as columns in your backtest and then use a tool to go through and figure out, okay, what is predictive?

Dave: 21:07

And that's really the way to do it in a way that can maximize profit across your entire strategy.

Michael: 21:14

Well and I think it's one of those that I like how you kinda talked about the lack of randomness because I think that's where people could get lost on this. I'm always trying to think of the the newer person. And, you know, the extreme example would be, you know, you're looking for a a five minute scalp on on something. And then, you know, your system says, oh, look at the quarterly dividend growth or something. Just just trying to think of an example.

Michael: 21:39

It just completely makes absolutely no sense. It's unrelated. Right? That goes back to Dave's kind of random example of could it it's not it does so in the backtest, but it's not predictive in any way in the future. So you should always look at it and not just kind of take the data at face value, but you have to think on it for a little bit and say why.

Michael: 22:00

And sometimes the answer is not obvious, and sometimes it is just gonna be random noise, and sometimes it opens up more questions than you initially thought anyway, but it should make some sense of the things that you're you're trying to filter out. And if it doesn't, again, that's a whole another line of thinking. But make sure, yeah, the these data points matter for what you're trying to do or you're just you're you're making up noise, and then that's where, you know, the really bad c word comes in of of curve fitting from there.

Dave: 22:31

Yeah. So I'm glad you brought that up because it's it's definitely the way I put it is the rule has to tell a coherent story. And you have to be able to convince yourself. And I sometimes will say, like, Can you explain this rule such that a very experienced, skeptical trader will believe it?

Michael: 23:00

And

Dave: 23:02

that's mindset you should be in when you're trying to decide, okay, is a rule good enough? Because you're right, if you put enough columns in there, you're gonna see some spurious correlations. Yeah. So you to not fool yourself into thinking, Okay, I see this. The data is telling me this.

Dave: 23:22

I'm just going to follow it. You can't simply follow the data. Thinking about it. You have to be discerning about what rules you might apply to your system. And is it going to be predictive going forward?

Dave: 23:43

That's the important thing to discover.

Michael: 23:46

Right. It's like what I say what I I really like what you say about the, you know, the level of getting that that comfort, right, with it. And at the end of the day, it may make your back test look way better. But if in the darkest moments of trading the system, it doesn't make you feel better about continuing to push the button when you're in a drawdown when things aren't going well, then you shouldn't add it to your system. And the perfect the the best times for me to actually take action, I think, on, you know, adding something from a column is kind of when I suspected it anyway.

Michael: 24:19

Like, we talked about adding a base kind of raw system into into your tester or or whatever they're doing, on their own. And the I always like you mentioned, you have in your head, these are the things I think should affect it, and then you test. And the best world I always find is when both those line up. Right? When they don't, it, again, it just creates a whole another series of questions you have to answer.

Michael: 24:46

When you're like, man, I you know, for me as a longer term trader, it's like, okay. It should probably be above its two hundred day moving average, which is a nice proxy for, like, a long term trend, blah blah blah. And when that shows it to me, I go, great. It's what I thought. It's what the system thought.

Michael: 24:59

We agree. Let's let's move on. When either of those things are not agreeing, I think that's when you really have to take a step back. So the beauty of doing this is that when you get those agreeable moment agreement moments, you can quickly move through them. It's like, it's what I thought.

Michael: 25:15

It's what the system thought. Just put it in, move on, and then retest and and see what happens. And then when you disagree, that's, where the problem creates. This actually leads to a question that I had. So as somebody who is doing now both, you know, getting into day trading and swing trading and investing, should those be different column base?

Michael: 25:36

So or do you think the user could set them up in such a way that bar size doesn't matter? It's just, you know, what it is that you're looking for after that.

Dave: 25:47

Well, I think there's most of the columns that come to my mind are gonna work across whatever time frame you happen to be trading on. If you're trading and backtesting on a one minute timeframe, you know, lot of these columns are going to work just the same as if you're doing it daily. It's just the same open, high, low, close volume bar. And a lot you know, the calculations for the whatever increment you happen to be running a backtest on are gonna be the same.

Michael: 26:22

Yeah. That was gonna be my answer as well because you're right. You know, above a two hundred day moving average could be a good signifier of a long term uptrend. A two hundred one minute moving average could be the same for, for a short term trade, which, you know, is more indicative to the fact that you're gonna do a lot of this work once. And then once you have that list, then again, it's incremental kind of from that point on.

Michael: 26:48

So, it becomes a becomes a pretty big exercise, but, again, I think more than worthwhile in the long run, imperative, I would say, in the long run.

Dave: 26:58

Yeah. So, Michael, you gave me a lot of help with the Strategy Cruncher. I I think you're still probably in the lead for the number of optimizations you've run through

Michael: 27:08

there. You you okay. So I was definitely now I'm probably. If someone's coming close, let me know. I'll I'll get back to work.

Michael: 27:14

I'm not no one's There's

Dave: 27:16

a lot of traders. Yeah. There's a lot of traders using it and, you know, getting some real good edge out of it. So as we worked on that, I had this idea and you were one of the early beta testers of it and gave me some really good feedback on it. And as I sort of showed you or sort of demonstrated concept of the Library of Columns, so I think you were working on it, I could tell you were refining it over the past few months, your own Library of Columns.

Dave: 27:48

How did you go about that? Were there any moments you had about those? Did you find stuff that did turn out to be predictive? What type of columns end up on your optimization reports more often than others? Have you detected a trend there?

Michael: 28:11

Yes. And it's a lot of the stuff, thankfully, that I that I thought it should be. So, again, I'm a a CMT or a technician by trade. So I kind of went through part of what I had to do, which again gives me a little bit of an advantage over this, is I had I was tested, and I on all of the different indicators out there, I had to know, you know, how they were calculated and what they meant and and all that kind of stuff. So I kind of had a list of favorites to begin with that, like, ATR, like you mentioned, is is just a no brainer because it's not even really an indicator.

Michael: 28:40

It's just a descriptor. And some of the ones that I I liked better than others, Bollinger Bands and and different channels that just felt more descriptive, than they were trying to predict anything. And what I found quite often was the same ones kept making it in, and I don't need to hide those, but a lot them were volatility based. Right? So you're looking for a width of a Bollinger band or a size of this, a size of that, and it it made sense a lot because, like you mentioned, you're trying to look for unusual events.

Michael: 29:15

So unusual events don't happen when, you know, everything is, right, two standard deviations make perfect sense and it's well within that and and all these things. So a lot of them came through quite a bit. And what I've done a little bit is I've started to pare things down that I've just never seen in you know, I've been doing this kind of back testing and this optimization enough. If I go through a column and I've never seen it there, then, again, it opens up a couple every everything every doorway you go into systematic trading opens up a whole bunch of questions. One is, did I program it wrong?

Michael: 29:49

And then the other is just does this not matter? And, it's actually been kind of freeing doing the cutting out thing because it does. It just shows that there are some things out there that regardless of whether it's day trading or swing trading, whatever, just don't don't matter at all.

Dave: 30:05

Yeah. That's funny because I very rarely I don't in fact, I don't think I've ever called my list of columns. And that's because the tools that I use and now the Strategy Cruncher, which I've made available, it automatically gives you a report that shows you the most predictive ones and the the exact values to use for a rule for those columns. So Well,

Michael: 30:31

and And it's super fast.

Dave: 30:33

There's no it it doesn't get slower when there's more columns that go through the report. So it's it gets marginally sort of up at hardly any.

Michael: 30:42

Yeah. And I will say now that I'm using that, I I don't feel the need to do that anymore. I just have a you know, in real tests, you just have the chevron that collapses that little trade section. I just kinda copy that into every every test that I did. So, that's another benefit, I think, of of using a software like yours or yours is that you don't need to do that.

Michael: 31:02

It was more when I was kinda brute forcing a lot of this in Excel. If I would go to that certain tab that had a a pivot table based off that one column, and I went there, you know, you go there a hundred times and there's just nothing it's just noise, then, you know, you kinda rule that out. For those for the people doing it manually, I do think it kinda makes sense. You may miss the one time out of a thousand that it was actually something important, but, the question is how it's the time side of things. But if you have something that doesn't take the time, then you might as well put in as much as possible.

Michael: 31:36

My only worry is, again, I say as much as possible, but then, right, bring people back to that, make sure you're not putting in junk. Right? Because garbage in, garbage out type of scenario. Or at least if you see something that is junk, you are smart enough to not, I guess, apply that to your backtest. That'd be the only caveat with kind of leaving as much in there as you could.

Dave: 32:00

Yeah. Yeah, when you leave stuff in there, the nice thing about the Strategy Cruncher is it's never going to show up as important, right? It's never going to show up there. It's always going be unimportant. And that report will sort of confirm that.

Dave: 32:17

The other thing is, well, there's two thoughts I have about it. The one reason I leave stuff in my columns is I know that at some point in the future, I'm going to have some moment. And I want to be able to go back from that moment with a certain strategy that has implications on other strategies that I trade. I'm going to quickly want to go back and see, all right, I've just learned something here. I'm going go back and apply it to my old strategies and see if it has an effect there too.

Dave: 32:56

That happens not that often, but when it does, it's a really powerful thing to be able to do. So you want to have sort of this little trail to go back to with your old strategies and quickly be able to sort of pick up where you left off and see if these new things you learn apply to the old strategy that you can improve.

Michael: 33:22

See, that's and that's why I always laugh about us just giving people piles of work because I've noticed the same thing. I I think I mentioned this in a in a prior episode that I have on my list somewhere once the children stop being sick all the time. A, I need to just carve out, like, a day. I need to carve out a couple hours to go through this list and add to it. And I think one of the reasons I might have been actually pushing it off is because, like, what you mentioned, if I refine this list, then I get that itch in the back of my head to say, well, now you've gotta test everything you've ever built against these new ideas and these new columns, which is equal parts, awesome and terrifying because the amount of of work that it opens up.

Michael: 34:06

But if you find edge there, it's really important. I think I think that's a huge thing where, this whole column exercise leaves almost an unlimited way to refine existing strategies if you just think of a new column. So, you know, if you're out there and you're looking at YouTube or podcasts or whatever and someone describes an indicator you've never heard of or, even a candlestick pattern or something that you've never really heard of before, then it's like, okay. Now let's go back. Let's add that in my column base.

Michael: 34:38

Let's go test this. And, again, you've just created, you know, twenty hours of work. Congratulations. Yeah. You know?

Michael: 34:44

And then, you know, we always like to either to references too because a lot of people think, great, it's still systematic. You go sit on a beach somewhere and and sip your myotis, and it's like, have the freedom to do that, but you're gonna be so deep into work, you're never gonna have the time to do that.

Dave: 35:01

Yeah. So here's something you mentioned there that reminded me of a trick I use, and I'm not sure I've mentioned this to you. One of the things I will add to my library of columns is a completely random number. Have you ever done this or or can you can you imagine why that might be useful?

Michael: 35:26

Yeah. Because if that starts showing you things, there there's a lot of data. There's actually just for I'll I'll have you explain to people better, but there is a function inside of, real test that I think does that, and it also will skip random trades and take random trades as well. So just so you know. But, yeah, explain to the audience why.

Michael: 35:51

I again, this is something that I think is I forgot about as well, but is is very, very important.

Dave: 35:57

So, you know, as you're optimizing strategies based on these columns using the Strategy Cruncher, whatever tools you might be using, you can over optimize if you're not careful, right? One of the ways I use a random column is as sort of a flag. If use the Strategy Cruncher and the random column shows up as the most important thing left, then I know that I probably tortured this data too much and that as sort of a flag, okay, I'm getting into territory where there's a danger of over optimizing. So it's a really nice way to do it because you know, of course, there's no way a random number is gonna have any effect on your system. I mean, that should be clear.

Dave: 36:48

So it's it's a good kind of level set.

Michael: 36:51

Clear to most people. Right? Clear to everyone who listens to this podcast. Let's put it that way. There's probably some people out there we could convince that a a random number would, would be impactful.

Michael: 37:00

But yeah. And that's that's the reason that I think it's so important is if I ever get that, I think I had it once because it's a random an actual random number, And then I had to rerun it, and it it went away again. But other than that, yeah, nine 99.999% of the time, hopefully, you never see it. Or or if you do, you just kinda gloss over it. It's the same as kind of skipping the random trades, and it's the same reason I think that people like Monte Carlo simulations so much as well.

Michael: 37:31

Yeah. Because you're just you're showing a bunch of you're throwing random tests at the data, and that makes, I think, people feel better. So I guess Monte Carlo, if people don't know, if you take the same trades and you just shuffle the orders, and you see if if the curve is roughly the same just to see if you've done, like, really hardcore path dependency where you had to have taken this trade and this trade and this trade or else your account would have blown up. These kind of random, tests, I think, on data are super, super important. I'm now trying to figure out, yeah, there'd definitely be a way to export that in real tests.

Michael: 38:08

I I know I do it when I was doing the Excel stuff, but I need to now add that to my real test column. So now just right? Again, you figured some more work to do. The other indication that I think you should use for when you've tortured the data enough, which should be obvious, but I think I should say anyway, is just when you start to get to the point where you're really drying up on trades. You know, say your system started with 50,000 trades, and now you're down to, like, two.

Michael: 38:38

It's probably probably time to stop. Right? You've probably done that enough. That's a pretty good indication of that that as well.

Dave: 38:45

Yeah. I mean, that's that's a great point. And, yeah, you go back to the episode on optimization that we talked about, like what to optimize on, what metric to optimize on, because yeah, you could if you choose a metric like that, if you're not careful, you'll end up with a small number of trades and really what you want. Like, if you optimize your strategy down to the two trades and then you're sitting around all year waiting for two trades to come in.

Michael: 39:12

And then one of them is

Dave: 39:13

little

Michael: 39:13

That's

Dave: 39:13

not revival. Yeah. It's not revival.

Michael: 39:16

Yeah. Unless you're full porting your whole account and, like, putting it all on black, the amount of risk you'd have to take to make that worse worth your while is just insanity. So, it's an interesting one as well. And in a conversation I have, all the time, right, where people are like, it's one of those I know it's completely off topic, but there's so many of these, like, platitudes in trading about, you know, you shouldn't trade very much or you're gonna make your broker rich and and things like this where I always like I think this is from Tastytrade that I I love there. They have a whole motto, it's it's trade small, trade often.

Michael: 39:52

And I I love it because it just they're basically saying if you have an edge, you should exploit it in in a way that can't hurt you, but as as many times as possible. So make sure with that column set when you're optimizing by it, hey. Yeah. If there's something that removes too many trades, even if it makes the equity curve looks look great, you might not wanna add it. Or, again, it leads to more thinking of, right, is there any way that I can tighten here, but then loosen up back here in order in order to let more in?

Michael: 40:22

So it's definitely a it's a dance that you have to you have to do for sure.

Dave: 40:27

Yeah. And there's you know, as you go through each optimization step, you know, like the way the optimization reports work in the Strategy Cruncher is you're going to get a list of what it's telling you is the most important according to your backtest data and your columns. And when you choose a filter to apply to your strategy, you go back, run the backtest again, do another report, because you're going to get more and different information. It's going to be different every time you apply a rule. And that's important because you'll notice quickly that you want to be careful not to remove too many trades because that's going to cut into total profit.

Dave: 41:14

And also, just things change as you apply the rule. Some relationships are going to disappear because some of those columns are correlated with other columns and you're choosing one to apply with a rule. All of a sudden the report changes quite a bit. So, yeah, when you step back and think about it, it's a really complicated, multivariable calculus problem that using optimization can simplify quite a bit.

Michael: 41:47

And just so because you touched on there, but I think it's important to highlight. Just because if you go through and your report says, here's the top five columns, or even if you're doing it in Excel and says, here's you find out that these top five have the most prediction on them, I never think that you should throw them all in at once and then go and run a test. This is probably why I'm, like, top on Dave's leaderboard because what I do is every time I make a change, I then go back and I retest that data and then and then look at that date and upload it because sometimes, I don't know, may late let some other trades in or it might not work as intended. And I always the example I give more for the the kind of layman out there is imagine you bake a cake and the cake sucks, and then you change every part of the ingredient. Right?

Michael: 42:36

You you a little little more milk, a little sugar, whatever, then you bake the cake again, it's delicious. You don't really know what you've done to make it better or worse. It's just you've changed a whole bunch of stuff at once. So, you know, a single variable at a time and then and then tweak that over time. And then eventually, you you get to the point that makes sense.

Michael: 42:57

But, yeah, I could see people looking at, in the Excel report or the Strategy Cruncher report and saying, oh, well, these are the top five things that really matter. Let me just add those all in and then run it. Oh, my equity curve looks great and then move on. It's it's not it's not elegant, I don't think, to do it that way. Right?

Dave: 43:14

It's not elegant, and you're gonna be leaving out some profitable trades, which drives me bananas. Like, if I see trades that are outside of my rule set that are profitable, that really pisses me off because I know I haven't done quite as good a job as I could have in this process. Because and ultimately you want to maximize the number of trade just I love the motto that you mentioned from Tasty Trade small and often. Think that's I love that. I mean, that's a great way to look at it.

Michael: 43:50

Yeah. So I think that's a good, so I just I feel a bit like a teacher every night, like the lame teacher because we we record this on Friday. I know you guys you guys see it on Monday. We record it on Friday. But I just feel like every Friday, I'm like, okay, listeners.

Michael: 44:05

Here's all your homework for the next, like, three months. Tune in to the next episode where we give you more homework to do. But, yes, this is a, this is a very important thing I think that everyone should do is to find that column base and, like, hold it close to your heart and, you know, keep it somewhere that regardless of, you know, any sort of computer outage or whatever, because you you've created this thing and it's for you and it's for your back testers, but it's something that you will always be adding to throughout. It's now part of the it should take that 1% of your brain that every now and then has an idea because you can slap it into that, like we mentioned, and go back and and test everything. So it's, you know, it's it's very it's a lot of work.

Michael: 44:49

It's front load of work, but I think it's very good work to do and always kinda keep top of mind because you may one day discover something that you go back and and maybe even a strategy that was suboptimal before becomes great because you're able to then test it against this kind of one call.

Dave: 45:07

Oh, no question. Yeah. That happens all the time. The more and more you the more valuable you make your library and the the more you keep doing it, the more you keep plugging away and watching the market and really trying to understand and brainstorming what's going on with your trades. What are you missing?

Dave: 45:27

What's working well? All this is part of that process, and you'll definitely improve over time.

Michael: 45:33

So listen. I'm having a lot of fun making this course. I think, I think people are feedback is gonna be fantastic, but let us let us know. Right? As always, we've got a a whole I think we could go on forever with all these topics, but we're gonna try to keep them bite sized for you and and make sure we keep going with them.

Michael: 45:53

So I think this was great. I think we've, once again, on a Friday or Monday, I guess, for them, given them a whole bunch of homework to do. But until next time, I'm Michael Doss.

Dave: 46:03

And I'm Dave Mabe. We'll talk to you next week on Line Your Own Pockets.