What to analyze during Black Friday and Cyber Monday: conversion, products, payments, delivery and channels

Episode recorded in Polish ahead of Black Friday and Cyber Monday 2024 and published on 26 November 2024; the data and reports mentioned in the conversation are from that time.

Why analyze your Black Friday and Cyber Monday data?

Tomek: 67% of consumers start planning their Black Friday shopping as early as October. Yet according to Deloitte, only 24% of retailers analyze their customer data before the peak shopping season. What's more, Salesforce reports that retailers lose an average of 35% of potential sales during the peak hours of 8 to 10 pm because their infrastructure and processes aren't ready. In today's episode, we'll show you how to use your customer data to build the perfect Black Friday and Cyber Monday strategy. You'll learn which specific metrics are worth tracking and how to turn shopper behavior into real sales growth. This knowledge will put you a step, or even two, ahead of the competition. Over the next 30 minutes, we'll walk you through the key areas of Black Friday and Cyber Monday analysis: what data to track, how to interpret it and, most importantly, how to turn it into concrete action. Let's get started. Hi, Maciek.

Maciek: Hi, Tomek.

Tomek: Today's topic was prompted mostly by messages from our listeners. One of our recent episodes was about Black Friday, and it stressed that you need to prepare for this period.

Maciek: Among other things, you'll tell them what our morning routine looks like.

Tomek: We'll get to that, Maciek, no rush. But back to the point: we talked about how important it is to analyze what's happening in your store, which means measuring and looking at the data the store gives you. We got quite a few questions asking us to expand on that and explain how it should work in practice, which metrics to watch and how this data can help us plan and prepare the store better, say, for the next Black Friday. Here's my suggestion: let's start with what you think matters most when analyzing a store. Which metrics would you recommend as the basics, the ones we start with? We'll cover how to analyze them and with which tools, and I think it would also help to use some examples of why this matters and how it can translate into the business.

Which metrics should you start with?

Maciek: Sure. I'd start by saying that the most important thing is for your methodology to be repeatable. In other words, keep a year-on-year comparison of the same metrics in exactly the same place, say a single spreadsheet or report, so you can keep an eye on them. Compare year on year, of course, but also look over a longer period at how the trends are shaping up. The most basic and obvious ones are the conversion rate broken down by the marketing channels that bring in traffic, and average order value, which can also be broken down by sales channel.

Maciek: Another key aspect, specifically for Black Friday and Cyber Monday, is how sales were spread across those two days, or across all the days the promotion covered. Maybe we started our marketing early enough, built up anticipation and ran some kind of Black Week. That's worth measuring too. One critical parameter, in my view, is the peak hour: what sales looked like hour by hour. It's an interesting one, and let me give you two examples. Normally you'd conclude, and most merchants will notice this, that conversion was highest when traffic was heaviest, and sales automatically too. However, and I think this is data from 2021, ASOS, a large multi-brand clothing retailer, saw conversion drop during peak hours. That's interesting, and it's exactly why we should measure this data. For them, it was simply down to overload: the servers couldn't cope. Of course, in the following years they rolled out a whole range of technical improvements.

Maciek: As a side note, companies on good e-commerce platforms like Salesforce don't have to worry about this at all. But if you host your store yourself, you have to manage that traffic, probably forecast it too, buy the capacity you'll need, maybe higher usage, manage load balancing and so on. I think those are the absolute basics to pay attention to during this period.

Why doesn't your store's overall conversion rate tell you much?

Tomek: So to sum up this part: what comes through from what you're saying is that analyzing this data should be systematic and regular. It's good to have even the simplest Excel dashboard with a few metrics in one place that we can analyze at regular intervals, to draw conclusions about how things change and what trends appear. That's the first thing. The second thing I took away is about conversion analysis. Maybe it's obvious to many, but I think conversion analysis sometimes comes down to a single number: the store's overall conversion, meaning how many visitors end up buying. And it's important to stress that conversion, as you showed with your examples, should be measured at many different levels: per product, per product category, per channel, as you mentioned. I'd add that conversion should also be analyzed by type of traffic, mobile or desktop. Because a single overall conversion figure, which Shopify, for example, shows very simply in its analytics, can hide weaknesses, whether in our processes or in products that convert worse. It's good to pull this information at the lowest, most detailed level.

Maciek: Sure, the conversion rate on its own won't tell you much. It gives you an overall figure, but what do you do with it next? You can't work on it operationally, and you don't actually know why conversion went up or down. As you said, it's important to break it down by marketing channel, product group or individual product. As an extra dimension, it's worth adding customer segments and checking how the same parameters break down in each segment.

Which tools should you use?

Tomek: Right, and at this point could we suggest how best to measure this? What tools would you recommend?

Maciek: These are the absolute basics. Google Analytics, of course. Shopify has detailed sales reports. On top of that we can use Hotjar or Klaviyo, but those are more for observing user behavior and looking for the reasons why conversion dropped in particular channels, products or product groups.

Planned or impulse purchases?

Tomek: Sure. You've already mentioned user behavior, so we can move smoothly on to the next key metrics. Let's focus on what data we can, or really should, draw from the behavior of our users, our customers. What should we pay attention to?

Maciek: I think one key parameter to look at during Black Friday and Cyber Monday is impulse buying versus planned purchases: the spontaneous ones versus the deliberate ones. There are estimates, I think it was the Adobe Digital Commerce report for the previous year, according to which as many as 67% of users planned their purchases before Black Friday. That means they were looking at a product as early as October, and several times before buying, three or four times. That's a clear sign that people approach shopping a bit differently and that most users plan their purchases. That's valuable information for us.

Tomek: Yes, exactly. The first thought that comes to mind: impulse purchases are sometimes hard to predict, because you can't always influence that behavior. But if we're talking about such a large share of customers who prepared thoroughly for this period and planned it into their budgets, that's a very concrete signal for the store and a very concrete use of this data. A signal that the campaign, both promotional and the kind that nudges people to buy, can start much earlier. I remember that in our episode on last-minute Black Friday preparation we said you can easily run the campaign a month ahead. As the data you're citing shows, October is already a very good time to first identify these customers: to see that they've been browsing a product and keep coming back every so often to look at it again, which tells us they're interested. First, so we can encourage them at this pre-Black Friday stage, maybe even reward them early in some way for wanting to buy, so the purchase actually happens. Second, so that when they make the decision to buy, they know where to go, because I assume they may be comparing the product in several other stores at the same time. It's a great moment for our campaign to win them over early.

Maciek: Exactly. A practical example of using this knowledge is introducing notifications. We talked about that in our episode on Klaviyo automation. We can set up price drop notifications. We can build a campaign that reminds customers the product is still in their cart, or build a whole communication flow around it. It's also good practice to show how many units are left in stock, because that adds urgency to the purchase. Those are just examples off the top of my head.

Which products and categories should you analyze?

Tomek: Sure. OK, let's move on. Analyzing our products and how attractive they are is, I think, another area that's good to have measured. We always stick to the principle that e-commerce has one great feature: it can be measured down to the smallest detail, so let's make use of that. So, product analytics. Which product-specific metrics do you think should be measured, but also tracked on an ongoing basis, giving us grounds for smarter decisions and a product range that better fits our customers?

Maciek: Sure. The obvious ones that come to mind are the most popular categories during the sale. What's interesting is comparing whether they overlap with your bestsellers during the rest of the year. Second: are there any unexpected bestsellers, and were they driven by the discounts we offered? A fairly interesting metric, I think, is cross-selling between categories, meaning which products from which categories were often bought together. Of course, we can measure that all year round, but it matters when planning cross-selling during any promotion.

Tomek: Yes, I completely agree. I like that non-obvious metric, the underdogs of our range: products or categories we weren't convinced would be attractive, which often turn out to be bestsellers. That's great, because the numbers help us make the right decision. We may assume product A will definitely sell well and products C or D not so much, but then the numbers don't lie.

Maciek: Yes. A good example: if over two or three years we see a category's popularity growing steadily by 5–10%, that can help us plan stock levels for future sales.

Tomek: Yes. Or a second, very obvious action: if we see that a category is growing even faster than we planned, or is one of our bestsellers, we know how to spread our marketing and adjust budgets so the category gets properly promoted in the store.

Maciek: Yes. Taking a practical approach, you could think about adding dedicated landing pages to the site. And maybe train the customer service team to answer the questions that often come up about that product group.

Tomek: A really good example. To be honest, I hadn't even thought of that. But as you say, when a category is a bestseller, we can, for example, expand the FAQ for that category to reduce uncertainty, clear up doubts and answer customers' questions faster.

How should you analyze payment methods?

Tomek: OK, Maciek, we now have two areas related to further service, both sales and after-sales. I'd like us to stop at the metrics for payment methods and then move on to delivery. Let's start with payments. Why do you think it's important to analyze how well these methods work and how they translate into our sales?

Maciek: Let me tell you. As we know, not offering a customer's preferred payment method is very often the biggest conversion killer at checkout, right next to hidden costs. It's worth measuring how this affects sales. You can measure conversion by the payment method selected and by abandoned checkouts. What matters is offering customers the most popular payment methods first. I believe a 2023 Shopify report showed that interest in buy now, pay later solutions, like PayPo, Klarna and the like, grew by almost 100%. I think it was 85% year on year. So it's worth highlighting these payment methods, whether at checkout or by flagging them on the product page or elsewhere on the site, so users know this option is available.

Tomek: I'd add that we should approach the analysis itself by linking payment methods to placed orders and to started checkouts, to see what share of our sales they account for. That's the first thing. Second, we can go further and check whether a given payment method later led to returns. That may especially apply to deferred payments, which some people believe make customers more inclined to return things, because I didn't really buy with my own money and I'll get it back within 30 days.

Maciek: I think this is, I'm not sure how to put it, an aspect of looking at data: you also have to know how to read it. Because, of course, we might see that buy now, pay later methods raise our average order value by 40%. All well and good. We get unhealthily excited about it and start promoting this option. Then it turns out the order value is artificially inflated because it generates far more returns, and we also have to factor in the cost of the solution: the fees. It may turn out that this payment method is actually less profitable.

Tomek: Ultimately less profitable for the business.

Maciek: Exactly. So the correlation may look different than we think.

Tomek: I completely agree. That's why I brought it up, because I remember that on one of our episodes we had Marcin from the imoje payment gateway, and we talked about BNPL. He said that reports didn't, at that point, directly confirm these figures or the assumptions about returns. But we definitely encourage you, as you said, Maciek, to analyze this not just at the moment of purchase, but to then compare it with returns. Check whether that correlation is too strong, so it doesn't turn out that order value actually drops because customers buy five or 10 more products than they need, planning from the start to send some back.

What don't your channel numbers show?

Maciek: That's an interesting way of looking at data. It's also important to watch what's hidden, what you can't see. We might, for example, look at which of our sales channels or traffic sources is the most popular. It turns out to be mainly paid ads or organic, and the newsletter is only one, two or three percent. You could conclude that maybe email marketing isn't worth investing in. But if we dig deeper, it may turn out that companies similar to ours, in the same industry and of a similar size, generate 30% of their revenue from email marketing. And that's actually the opposite signal: we should focus on this channel, because there's clearly potential and we're doing something wrong.

Tomek: Yes, absolutely. On top of all this, today's consumer isn't a black-and-white customer who saw our ad, came to the store and bought.

Maciek: Because those are the impulse purchases we talked about.

Tomek: Exactly. But with those who still plan and analyze, the path to us often gets blurred or more complex. In practice, the question is how to attribute a specific purchase to a promotional channel. Because, as we said, maybe they saw an ad and that's what brought them to us for the first time. But then they bought because we ranked high enough in organic search, or they saw our ads on social media. Maybe they got a newsletter or some other email that convinced them. So, as you mentioned, that's the caveat for the data we look at: sometimes you need to see the deeper dimension and approach it in a very structured way.

How should you analyze delivery methods?

Tomek: The second thing I wanted to talk about, on the process and service side, is delivery methods. The situation here is…

Maciek: Similar.

Tomek: …very similar to payments. So again we look at which methods generate the most sales. Let's remember that logistics is still a touchpoint with the customer. So it doesn't just affect what happens at the moment of purchase, when someone chooses a delivery method. Later on we'll notice that negative reviews of the store's service are largely about how the delivery went, how long it took and whether the product arrived damaged.

Maciek: There are probably plenty of people who can't separate the delivery itself from the store.

Tomek: From the store. That's why, as we say, the analysis here is fairly similar to the one for payment methods. But here we have to stay very close to the process and analyze what we see in terms of returns. Are there too many returns with a particular courier? That may signal that products arrive in poor condition or that delivery took much longer than the customer expected, and dissatisfaction was too high.

Maciek: Exactly. I'd add that a delivery method customers choose more often is one we can promote more. And not to state the obvious, that parcel lockers, meaning InPost, are the most popular option in Poland: we can, for example, offer same-day delivery in bigger cities, for stores that can afford it because they have their own delivery or couriers who handle it. If we see that a delivery method converts better or orders are fulfilled faster, it's worth promoting it.

How do you measure marketing effectiveness?

Tomek: Sure. OK, Maciek, so let's step outside our store and focus on what we should analyze in our marketing, and why this part seems absolutely key to how effective it is. It has a very direct impact on the profitability of our business, because we have to be aware of how we bring traffic to the store. Whether it's organic or paid campaigns, that traffic costs us, and its cost translates directly into how much we earn in the end.

Maciek: Exactly. We can bring up every marketer's favorite saying: half the money spent on marketing is wasted. The problem is, we don't know which half. So it comes down to measuring how effective each channel is and checking how it translates into conversion, so we can allocate the budget better in the future.

Tomek: OK, so simply: which method is more profitable for us, and that's where we put more money.

Maciek: Exactly. Besides all the paid campaigns and organic traffic, it's also worth looking at our influencer marketing and email marketing. If we run campaigns for loyal customers, like early access, it's worth checking how they perform too. There are case studies where early access boosted the conversion rate significantly, even doubled it. That's a sign it's worth starting early access early enough, maybe building out the communication and offering customers extra benefits.

Marketplaces, social commerce and apps: analyze each sales channel separately

Tomek: Sure. As you say, maybe we didn't cover this as a separate segment, but it's also important, and closely tied to marketing, to look at how different sales channels perform, because not everyone sells only through an online store. Marketplaces often account for the vast majority of sales worldwide, especially around Black Friday. So we have marketplaces, social media shops where we sell directly, mobile apps and, as you mentioned, affiliate networks that also generate sales. Just as we recommend analyzing individual marketing channels, sales channels should also be kept separate in the analysis. That's how we can decide whether entering a given marketplace that year was a good call, or quite the opposite: the marketplace costs ate up all our potential profit, so next year we'd better focus on a different channel.

Tomek: And finally, a wrap-up of everything we've said. You started with this, Maciek: a tool, a dashboard or analysis panel, has the important advantage of letting us look at results over a longer period and see the bigger picture. Overlaying a trend or the pace of change on it shows us whether we made good decisions in a given area, or whether some of them were off and we should adjust them accordingly.

How much analytics does a small store need?

Maciek: Yes. I'd also add that I wouldn't be overly obsessed with this data if your e-commerce business is still small. You have to adapt the approach. At the start, what matters is measuring at least the basics: the key metrics we talked about and the effectiveness of each marketing channel. As the business grows, it makes sense to look at all these parameters in more detail and fine-tune things. It would be good to make it clear that analytics should look different depending on the size of the business.

Tomek: Sure. Totally agree. We don't want to turn store owners into statisticians who just scroll through Excel tables and see people as numbers.

Maciek: Sure.

Tomek: But as you say, it's more accurate to say that the level of detail should match the scale you're at at any given moment. As with any method like this, go from the general to the specific, but always keep in mind what the data is for. It frees us from subjective judgments and our own beliefs, because it's hard to argue with numbers. If we have it in black and white why a given sales channel isn't performing as it should, we don't have to wonder whether to keep it or keep investing in it; we can simply decide to change our strategy. I think that's extremely important. Maciek, anything else you'd like to add?

Maciek: I think we've covered it.

Tomek: We've covered it.

Maciek: We could keep going.

Tomek: We probably could go deeper, but that's not the point. OK, everyone, that's all from us for today. We hope this episode has added to your knowledge and sparked your curiosity about collecting data and why we really do it. We encourage you to watch our upcoming episodes. We can just hint that we have some really interesting conversations with guests coming up, so look forward to them. We'll be back soon with a really solid dose of knowledge. Thanks, Maciek.

Maciek: Thanks, Tomek.