The New Attribution

[00:00:10] I'm saying stuff that probably they were saying in 1970s at teaching classes, and then I'm, I'm saying it like it's new and it's a new thing, but, it's just sort of a regression back to, What you should have been doing in the first place, but attribution allowed people to get fancy and not put in the effort.

[00:00:30] Hello. Hello. How's it going, everyone? So we're back again with Nathan. We've been seeing a lot of comments in the section talking about how you guys love the blue sense people. so we're back again with more brain rotting and mind numbing conversations that are hard to understand, but somehow people follow them.

[00:00:46] somehow. I think today, we're going to talk about incremental sort of testing and the new way to look at, you know, versus third party attribution tools. How do you actually find out what's working in your business, right? What's driving the revenue customers in the age old scene of like, what's the true first click, right?

[00:01:04] Cause you know, Google took away first click. Wasn't even accurate before.now it's just one of those things. People are like, well, what do we do? What's going on? Data and, and. How does that relate to actual sales and performance max and how that, you know, can be a potentially good thing or a bad thing, depending on your increment incrementality tests.

[00:01:20] So, yeah, Nathan, what is your, I guess, initial opinion when it comes to, incrementality tests and what you've seen be sort of a good starting point for a lot of agencies or, you know, clients? Yeah, I think firstly, incrementality testing, it's probably a word that will turn into a buzzword in like six months or 12 months because it's so needed because in platform doesn't really tell you a whole lot.

[00:01:43] It's good for giving you some, like triangulation as to what's going on, but you can't really figure it out unless you're baselining off backend revenue or backend new customer revenue or whatever it might be. The first thing that I normally tell clients anytime I propose, Hey, let's do an incrementality test.

[00:01:59] And so let's bump budgets and see, are we going to see a significant uptick in revenue? And then ideally enough revenue that it turns into profit. and the first suggestion is always, we need, actually big budget changes. And you have an idea that you're about to go into which actually fixes for this issue.

[00:02:16] But I always say that we need big budget changes. Because if you are going from a 5k budget a month to a 6k budget a month, but you're a 60, 000 business, And you have natural swings in Topline each month. How do you tell that you added any revenue onto Topline, right? Because your revenue looks like 55, 64, 58, 62, 50.

[00:02:40] So there's this slight variance. You add 1, 000 in spend. You can't incrementality test like that. Because the incremental uplift on Topline doesn't become visible. Unless you wait 3 6 months and then measure a very long time horizon. So that's the first thing with incrementality testing. the second is you have to keep variables relatively constant across the board, which is also really tough.

[00:03:03] And that's why attribution was a godsend by. The marketing gods about five years ago when it actually worked somewhat well, because you didn't have to go back to essentially first principles marketing. But I feel like that's where we're going. We're going, we're, we're retracting back to the old days. And now we're saying stuff, they're coming back.

[00:03:21] Yeah, I'm saying stuff that probably they were saying in 1970s at teaching classes. And then I'm, I'm saying it like it's new. It's a new thing, but, it's just sort of a regression back to what you should have been doing in the first place, but attribution allowed people to get fancy and not put in the effort.

[00:03:40] Yeah, no. And I think the, one of the key points you brought up too was benchmarking, right? Like what's your baseline. And so there's, you know, all of these little correlations that happen, like you said, naturally, because, you know, think of the stock market, the normal ebb and flow of just companies in general.

[00:03:53] Right. that's just like businesses, right? Like you have a good sale, bad sale day, and it's all just based on the user pipeline, users on the website. Like there's just so many factors that come into play there. And as you grow a business further and further, yes, that might not be as like dramatic of a change from day to day.

[00:04:08] But there still is a significant change and the budget correlation to that is going to be a big factor. And I would also argue that even dropping budget aggressively in other areas, if you don't want to scale, it's another test that you can do as well. That would kind of tell you, okay, is this pulling too much from, you know, repeat customers, it's starting my bottom line on like the actual, like NCAC on the first time user AOV, like all these other benchmarks that you have set over longer periods of time.

[00:04:32] Because, the other thing that I think is, is really important with this is seasonality. and so having, and this is, you know, my initial thought of why we believe companies that have been established tracking these numbers for years, have a way, way better estimation on what their profit is going to be because, you know, a year of COVID COVID, like all these different things.

[00:04:51] Can sort of change a year's impact. And so the more years you have, the more accurate data you have to kind of go off of, to then assume your season, right? So it's like, Hey, we're going to go into a season where, you know, we're not going to sell this product in summer cause it's a winter product. So then you drop sales.

[00:05:05] Obviously that's an, that's a very obvious one, but just being aware of those kinds of trends, helps to when you benchmark, so you know exactly what that incrementality test is going to do. The other thing that Nathan was mentioning, which was, one of the ideas to help solve this, or at least kind of.

[00:05:20] Give you more of a clear picture of what is actually happening is geo targeting. So if you notice that, you know, like your top performance state is, let's say like Texas or California or whatever, whatever state it is, if you're advertising in the U S what you can do is just run ads in specifically, you know, a state.

[00:05:38] Right. So you're trying a smaller, smaller pool. You have their address and Shopify, right? You can see like if this person is actually in that state and then seeing if that drives up revenue in that state or not, and if that actually increases it, then you have a smaller test pool. So you have more, you can make waves a lot easier in that space versus that of your full business model.

[00:05:56] So I think that's a really, really important factor. and all of this kind of highlights, you know, what Nathan was saying to like these old terms that people use. All the time back then, which is, essentially we're just looking for the halo effect, right? Like we're just looking for what is the cause of the ad to the entire business.

[00:06:12] Right. And there's, you know, all these small things that can happen as well too, right? Like you can see that organic traffic increases, direct decreases, you know, whatever the case may be. and that's going to be all based on just. How controlled your variables are or not is going to be how accurate that's going to look.

[00:06:26] Right. Cause you can launch a blog and you're, you know, your actual organic traffic is going to go up. Right. But so you have to be aware of sort of the full business picture. And I believe marketing agencies that do the full top of funnel control actually have that insight, because you don't have to worry about, you know, the Facebook team with Google team making a change when you're like, no, we don't want to change what the business is doing right now, because we're seeing if we can find a correlation to that pattern.

[00:06:50] So, yeah, hopefully that kind of. I guess summed up what you were saying, Nathan, but I don't know if you have anything else you wanted to add to that. Yeah, for sure. The only thing that I would really add is, that, and John says this a lot, SolutionsAid says this a lot, which is, if you look in platform, it won't necessarily make sense as to where you should put budget because all the attribution goes to the last click.

[00:07:13] and so, incrementality testing gives you the ability and the confidence to be able to allocate budgets to a campaign that has a one row ads and you're going, Oh. That's not a good deal. But if you do proper incrementality testing, where you're looking at top line revenue, you should be able to have the confidence to do that test, monitor your top line rev, or monitor your new customer revenue.

[00:07:35] and then see what happens out the other end. An important caveat there is you need to wait quite a while for the consumer, journey to actually. occur. Someone on average takes 14 days to purchase from you. You need to wait 14 days because if you're just increasing the top of funnel and then you don't see an uptick in revenue and you go, Oh, it didn't work.

[00:07:53] It's like, well, cause these people haven't come down the funnel yet. Like, give them some time. and so yeah, structured testing, if you're not doing it, you should be doing it. and the geo idea was fantastic. I'm actually going to start rolling that out on some accounts. Well, I would say also too, one thing that, I've done before is, You know, with clients that don't really have the capabilities of testing for that long is if I know like the average time from when they're, they're going to like the actual lag time, beginning and end.

[00:08:19] So like, if it's like a week, let's say I'll do a test super aggressively for like two to three days, like really aggressively, as long as it's like the same targeting. So we're like, let's say on manual or something like that, where it's just. Black and white. This is where it's going. We'd have to wait for it to warm up into it all.

[00:08:33] Then say, okay, we'll do the test. Have them for like three, four days in there. That's pretty aggressive. Then trim it off and see if there's uptick over that timeline. So if you, you know, you jump from like, you launch it this Monday, it launches for two days, three days, really aggressively pause it, look at the results then seven days from now.

[00:08:49] So there you capture the timeline before and after. and I think that's a good way that. You don't necessarily like it's not as accurate, but it's a very good way to kind of get a temperature check on like what that actually is doing. So then it builds your confidence. So like, okay, I want to make this last for seven days versus just, you know, that quick little bump up.

[00:09:06] and that's not to say that you have to increase, you can also decrease and test that as well too. one of the biggest things that I like to do is tell clients that come to solutions, eight is. You know, if, if you don't believe me, try it yourself. Right. So there'll be like a performance max campaign that I see that it's getting a lot of repeat traffic.

[00:09:23] I'll say, okay, we don't need to increase budget on to prove the point. Just pause. And then we'll see what happens.do it for like three days. And if you don't see any drop off, you know, during the end of your time lag, then you have your answer right there, kind of thing. So a lot of these, tests can be done to solve attribution issues, but it's the confidence level of it.

[00:09:43] It's sort of what you can kind of play with based on your tolerance to tests. So if you're, you know, not tolerant to test at all, then like your confidence level is going to be a lot shorter or a lot lower. whereas if you're, you know, very open to tests and your confidence level potential is a lot larger.

[00:09:57] So that's why I kind of go back into that thought I had earlier, which was, it's really important that if you have one agency that they're obviously communicating with one another internally, or if you have two agencies running your Facebook and your Google, they're also communicating back and forth.

[00:10:11] And ideally you'd have the system of, you just have both. Because then they have a system, they already know the incrementality test, log it at all. But, yeah, I think it's really, really important as even just a business owner to be aware of whatever tests are happening and something as simple as, you know, and this is what I do for all my clients is having a P and L sheet.

[00:10:29] And then having the actual, like test logs inside of the P and L sheet. So you can see clear correlations. We can look back years later, you know, it's like, what did we try in 20, you know, 2022 or whatever it is, and then see, okay, during summer, we actually did this test and it got us better revenue. Why did that happen?

[00:10:47] I got us more revenue. Why did that happen? and then you say, Oh, because we were targeting this theme or this creative or whatever the case may be, this demographic and maybe that demographic fatigue over that period of time. But now it's been a couple of years. We can test that same creative again, or that same audience again.

[00:11:02] And we, and because it gets refilled with a bunch of new people after that timeframe. So. It's all kind of up to you as to how you view incrementality testing, how the agency views it. But, it's definitely, I would say the number one way to solve the attribution problem. and that also goes to say that when you're benchmarking, make sure that you're benchmarking off of, even if it's flawed attribution, if it's still attribution, that's something.

[00:11:25] because if you look at a scenario where, you know, you trust an algorithm, right? So like, there's like some third party tools that will kind of try to piece together all the puzzles for you. it kind of marks where I guess money's the water because you don't necessarily know if the algorithm was set up this way during this day range or this way in this other day range.

[00:11:43] I don't know, Nathan, you've had some experience with third party apps in that sense, but that's been my experience. Yeah, for sure. yeah, I was a big fan of third party apps like six months ago. And then I went down the rabbit hole and I said, ah, this probably isn't helpful. It's not really giving us any further information.

[00:11:58] It's just confusing everyone. the, the one other thing that I'll say on incrementality is people are probably wondering why, like, why are we even using that word? What is incrementality? Where incrementality comes from is that we're trying to measure incremental improvement in profit as we scale. and that that's lost by a lot of agencies and particularly a lot of juniors when they come on board.

[00:12:21] Because. you'll always provide a blended target, right? Always. It's like, what's your target? Oh, it's a 4 MER. Or it's a 30 CAC. But that doesn't consider that if you add an incremental 2K in spend, are we still getting an incremental 4 MER? Probably not. And so if you're looking at averages, you can very easily have a junior run your account up to spending 40K a month in ad spend.

[00:12:46] Yes, your MER is 4, but that last 10K in spend that they tacked on added no revenue. And it just brought your average down and it looks fine. The average looks fine, but you just cranked your expenses up with nothing at the other end. And so, incrementality testing in the way that we're, we've been talking about and recommending it.

[00:13:02] It's, it's really making sure that you're measuring the difference that occurs in the top, rather than looking at the blended average, because. A lot of people are overspending quite a bit, particularly on bottom of funnel. And so this is a really easy way to, I easily identify, you know, Overspending and inefficiencies in your account at the moment.

[00:13:23] Yeah, I would say, I mean, literally every single client I've had come in, there's always an efficiency inefficiency when it comes to bottom funnel. Always. I've never, I've never had a scenario where that wasn't the case. So it's interesting hearing you say that. Cause yeah, like it's, it's definitely true.

[00:13:36] And like, it's always one of those debates online where I see people who are like, well, the row is, is good. Like, it's good. It's fine. It's like, yeah, but you just had all this other effects happen. And it's just a brand campaign. It's pulling in the row as like. It happens all the time. It's crazy. It honestly blows my mind that people aren't like, you know, up in arms about this, but yeah, it's crazy.

[00:13:57] Yeah. So, I don't know if you have anything else to add Nathan, but, I wanted to talk on one last point, which is earlier in terms of like, what's a good sign, what's a bad sign, what to kind of look for. and I think you've actually talked to us about your, or on your channel as well, too, which is looking at user engagement.

[00:14:13] So when it comes to like the early part of the test, right. So you don't want to wait that lag time, or you want to get some sort of signal that gives you confidence, that, you know, people are flowing through the funnel at the right rate and a lot of that's tracked with just general engagement. Right.

[00:14:26] So if you're looking at, you know, how well your ads are performing. So like, like you said before, like you need click through rate on, you know, actual link clicks. So like individual users that are new. Clicking on the link and then seeing if that holds when you scale, looking at like, for example, what Google tracks as click conversions.

[00:14:42] Yes. That's a signal of something, seeing if that changes at all. So there's a lot of these little sort of pre signals, I guess you can say that give you information on if you believe the test is going to go well or not. I don't want to. Hark too much on that because there are instances where you might be going into a new audience and that new audience might increase or shrink the lag time.

[00:15:03] But, and so you might see like worse engagement for a period of time. So that is possible as well too, but it's just a sign to kind of look in and say, okay, what, what do I think is going to happen? And sort of your, your threshold of, okay, how much more do I want to risk running this test or, you know, not kind of thing.

[00:15:20] and I think a lot of that just comes down to the client and the actual business model itself first too. so I think that's why, you know, like we were saying earlier, geotargeting or having a smaller test group, can be a way to kind of alleviate some of that stress there. Yeah. So I guess, we're just going to continue this series.

[00:15:36] it'll be one of these things where we just go over, yeah, I guess, whatever you guys want to talk about, or if we have some ideas that we see in the market that, I'll help you guys out, but, um, yeah, thanks again for coming on Nathan and, yeah, hopefully we'll see you pretty soon here.