Advanced Experimentation Analysis and Applied Neuromarketing Review
This article is part 10 of 12 reviewing the CRO Minidegree at the CXL Institute.
More CXL Institute reviews. Take a look! 1|2|3|4|5|6|7|8|9|10|11|12
This review will highlight the courses Advanced Experimentation Analysis taught by Chad Sanderson and Applied Neuromarketing instructed by Andre Morys.
Advanced Experimentation Analysis
This course is very hands-on, in each lesson R Studio is used for hypothesis testing and analyzing data (from A/B tests). It is the last course in the Testing Module and truthfully I struggled with a few of the courses in this module because of the statistics component, but I was able to grasp a few (I’m talking a very minuscule few!) key takeaways from this course.
Metric Hierarchy
A metric hierarchy is simply a system of all the metrics that we care about ranked by order of importance to our bottom-line value.
Northstar Metrics
The top of your metric hierarchy should be composed of your business’s Northstar metrics, those that are vitally important to business health. Examples include revenue per visitor (RPV), overtime, or engagement (for apps) — these are the things that we are trying to optimize for consistently!
Note: If the NorthStar metric moves drastically in a negative direction the test may need to be shut down immediately.
T2 Metrics
Tier 2 metrics are those that are better leading indicators and seem strongly correlated with lagging Metrics. For example, conversion and average order value (AOV).
These are metrics that aren’t necessarily our NorthStar metrics but might seem to correlate with our NorthStar metrics.
T3 Metrics
Tier 3 metrics are those that may have a weak correlation with Norstar metrics for example page views, clicks, etc.
These are metrics that may or may not be correlated with our NorthStar.These are things like page visits, clicks, or page views.
Binomial Metric
A binomial metric is one that has only two values, either one or zero.
Statistical comprehension
Clear interpretation of experimental data.
It’s so important to develop a foundation of statistics that allows you to understand what the data is actually telling you. And being able to read and interpret it and translate it back to people in the company that need to know.
We are also warned about the danger of averages:
“Well an average is simply a collection of a lot of different values.
Now, collecting a lot of different values isn’t necessarily a bad thing by itself, but it is a problem when you judge the whole by the average. Because the people at the far end of our tails might have drastically different behavior than the people who are in the middle. And the people in the middle might have drastically different behavior than those who are at the ends.
Servicing just the people in the middle doesn’t help us grow our business or understand our data better.
We must dive deeper into the segmentation of the data.” — Chad Sanderson
Applied Neuromarketing
This was the second course I had taught by Andre Morys. I appreciate the fact that this course didn’t make me want to unplug my brain after each lesson! Here are some of the ideas that resonated with me:
Why communication fails
Morys shares his formula on why communication fails between a business and it’s customers:
100% what you want to say > 20% what customers perceive > 5% what their interpretation is > 1% what their interpretation is
7 Principles on how to control attention
- Contrast (tells us what is important and what is not)
- Space
- People / Faces
- Movement
- Breaking Rules
- Directional Cues
- Person’s Name/ Their Own Face
How to implement control attention in your process
Wait a minute…you really thought I’d share ALL the jewels?!
Sorry, you definitely are gonna have to take the course like I did.
But I’ll definitely drop a few more jewels I picked up from Morys…
Measure emotional perception
Three ways to measure how people perceive your website emotionally.
- EEG solutions
- Con: Can’t measure was going on deeper in the 🧠 brain
- Pro: you can measure cognitive load which means the absence of emotional activity.
2. Facial recognition
- Detect micro-movements of the face that express emotions
- Con: A lot of disturbing factors that don’t make this particular test completely valid
3. Galvanic skin response
- Problem is that you know people are emotionally activated. You don’t know a lot about the valence, which means is it positive or negative the emotion that you measure here?
- Con: valance is missing frequency is low
If you wanna measure 100% on an implicit level: combine all methods to eliminate the error or leave it.
Three KPI‘s to measure
- Agility: the number of experiments per year (n)
How many optimization sprints are you able to deliver?
How much time do you need?
2. Success rate: the number of successful experiments (%)
How many experiments deliver a significant uplift?
3. Cumulative uplift: business impact ($)
How much uplift did you create in total?
How big was the business impact?
High impact, low effort equals low hanging fruits 🍉 🍎 🍌
Low impact, low effort equals high-frequency testing (A/B)
Should NOT do:
High effort, low impact equals garbage testing!
What we want:
Hi effort high impact equals high impact testing
“Prioritize high impact over low effort!” — André Morys
My thoughts
Advanced Experimentation Analysis is NOT for the faint at heart!. This class is definitely advanced as the title suggests and it is for the analytical, mathematical, statistical, coding, etc. type. I am creative I enjoy thinking “outside of the box” and this course, much like math doesn’t really allow for that so needless to say, most of this content flies way over my little peanut-sized head…
But I can and do appreciate that Chad does his absolute best to break it down to a noob like me. I REALLY enjoyed his walk through lessons on R Studio. I was able to follow along with him while I somewhat comprehended the super difficult concepts (at least for me). I also liked that he incorporated homework assignments with each lesson. While I personally will never and I do mean NEVER probably come close to being able to make this stuff make sense. I will continue to read up and study the topic until I grasp it on a very basic level. I will also hire a few people on my team that understands this intense level of thinking (ie. data scientist, data analyst, etc.) because it makes sense for someone to have a grasp of this to help CRO projects see big wins!
Applied Neuromarketing I thoroughly enjoy Andre Morys' teaching style and engaging content. I really enjoyed his in-depth lesson on using the Limbic Map to make more powerful User Personas very valuable stuff! I also love that he has very detailed and useful lesson slide decks to refer back to. This course will definitely give the student an edge.
The CXL Institute has delivered yet again! The content in these courses can give businesses the real competitive advantage that they are so desperately seeking. I feel so excited and privileged to know this stuff!
Would you like to join me by learning from the top 1% of digital marketers in the whole entire world?
Check out more of my in-depth reviews of CXL Institute’s CRO Minidegree courses:
- Intro to CRO and Best Practices Review
- Intro to Conversion Copywriting, Product Messaging, Psychology, and Social Proof Review
- Neuromarketing, Emotional Content Strategy, Influence, and Interactive Design Review
- Google Analytics for Beginners Review
- Landing Page Optimization, Conversion Research and Using Analytics to Find Conversions Review
- Google Tag Manager for Beginners, User Research, Fast and Rigorous User Personas Review
- Heuristics Analysis Frameworks for CO Audits, Google Analytics Audits, How to Run Tests
- Testing Strategies, Statistics for A/B Testing and A/B Testing Mastery Review
- Optimizing for B2B, Customer Value Optimization, Digital Psychology, and Behavioral Design Training Review
- Advanced Experimentation Analysis and Applied Neuromarketing Review
- How to Design, Roll Out, and Scale an Optimization Program; Evangelizing for Optimization in Enterprise Review
- Building Your Optimization Technology Stack and CRO Agency Masterclass Review
