The Problem with Cycling Research

We are always trying to find better ways to coach and train our athletes.

But after reading and analyzing hundreds of peer reviewed research articles, we discovered two major flaws in nearly all of these articles that made us question their validity...

1. Practice Effect Error

2. The test didn't directly measure relevant performance

But first, let's discuss a few basics of how academic research is conducted.

How Does Research Work?

The goal of research is to measure improvements to some performance metric by only changing one thing. This allows researchers to reliably correlate performance improvements with making just one single change to your training.

Independent Variable	Something you actively change
Dependent Variable	A performance metric you want to measure improvements in

But this is easier said than done.

There are lots of other things going on in the study effecting the results. Researchers want to minimize these as much as possible since they can drastically change the results but are NOT from the original change we made to the independent variable.

Confounding Variable

Something changing your results that is not your Independent variable

Confounding variables introduce uncertainty into the study and reduces its validity. If results from a study are unable to be replicated by another team of researchers when following the same method, the method's validity is compromised.

These are called errors

What Are Errors In Research Design?

Errors can be random or systematic.

Random errors are inevitable, human physiology is slightly different between every person on earth. Having a large sample of test subjects reduces the effect of this error and ensures the results of the study are representative of the average population.

Systematic errors are more damaging. Sometimes called "biases", these kinds of errors can mislead you into believing something to be true when it is not.

A broken clock is right twice a day

A clock that does not move will have random error, being incorrect more often than it's correct. You know not to trust this clock and would consider it unreliable.

But what about a clock that is slow?

A slow clock will have systematic error. It misleads you into thinking it is a different time than it really is, but you may not even realize you are being mislead. This is why systematic errors are sometimes called biases.

Practice Effect Error In Cycling Research

A common form of systematic error in endurance sport research is practice effect.

The simple act measuring the data improves it.

Every time the test subject performs a given test, they "practice" it and get better without improving the underlying metrics that the test is trying to measure.

Think of this like the kid in high school that took the ACT or SAT 5 times. They improved their score each time they did the test. But did they really get smarter and increase their knowledge between each test? No, they just got better at doing the test.

So does that mean ACT or SAT tests are a reliable measure of a student's knowledge? Not exactly.

Whenever a research article uses some sort of "time trial" to measure performance, there will be practice effect to some extent. This will change the results of both the control and the experimental group, so the difference between the two will not change substantially.

But that isn't always the case.

If the experimental group is exposed to more opportunities to experience the intensity of the time trial test, or close to it, it will help them perform better at the test without necessarily improving their underlying fitness. They will learn how their body feels at that intensity better and learn better pacing strategies.

So lets use tests that don't have a practice effect error instead...

Wingate and Graded Exercise Tests

Two tests commonly implemented in endurance sport research are the wingate and graded exercise tests.

A wingate test is simply an all out 30 second sprint. The peak power, 5 second power, and 30 second averages are often used as the dependent variable.

A graded exercise test is just a fancy name for a ramp test. It is an endless test that gets progressively more difficult until the subject can no longer continue.

	Test Duration	Intensity Level
Wingate	30 Seconds	Max level sprint
Graded Exercise	6 minutes*	Progressively more difficult to failure

*While the full test is longer than this, it is typically only "hard" for the final 4-6 minutes

But there is a different problem with using these methods to test improvements in aerobic fitness...

They're not all that aerobic.

The shorter the duration of your maximum effort, the more fast twitch muscle fibers you activate, the more phosphocreatine you use as fuel, and the more you rely on non-aerobic energy systems and processes.

If the goal of the test is to correlate improvements to aerobic fitness from some change, these tests are highly dependent on non-aerobic energy systems that skew the results.

Try doing a ramp test and a classic 20 minute FTP test in the same week. Your results will likely differ significantly. This is partially due to the pacing aspect of the 20 minute test but also the fact that the shorter ramp test more heavily weights your reliance on anaerobic energy production.

All of these flaws beg the question...

Should We Use Research In Training At All?

Short answer: Yes

But we need to be a LOT more critical of the research we read.

Instead of assuming all research is completely unbiased and fundamentally true, you should try spot these biases and flaws. Do your due diligence.

Be slow to accept the results as fact, and quick to dismiss them as flawed.

But when we do accept research, we should not live and die by it. Fundamentally, results of research represent an average result for an average population.

You are an individual, not an average population.

We use research as a starting point; a hypothesis of sorts. Then we run our own research experiment on each individual athlete.

Training Is Research

Fundamentally, training is just changing a few key variables to try and reliably get a desired outcome. The variables are our training intervention and the desired outcome is to get faster and improve performance.

So we treat every athlete we work with as an experiment with a subject size of 1.

We use research as a starting point, but then we test and measure improvements in the athletes performance directly. We try to focus on just one key training objective, or independent variable, at a time. Over the course of a training cycle we are constantly collecting data to confirm the intervention is working with the individual athlete.

This is how we developed our Full-Spectrum Performance Testing

We measure improvements in performance by testing power across the entire power-duration range. This prevents practice effect skewing results by relying on a single test. This also allows us to measure improvements to the anaerobic energy systems independently of aerobic ones. We can see exactly how a given training intervention is helping the athlete.

We can direct our training intervention more specifically to the goals of the athlete. If we relied on only a 20 minute FTP test to track performance improvements, we are missing a lot of data regarding anaerobic development. If the athlete needs a strong anaerobic engine for their target race, like an XC race or crit, we are not really preparing the athlete as best as we could.

Which leads us to developing specific tests for individual athletes.

If a key training objective for an athlete to perform well at a race is their ability quickly recover from anaerobic surges, we need to develop a method to test and measure improvements to this.

If another athlete needs to perform well on a 5 minute climb at the end of a 6 hour race, we want to develop a method to specifically measure the athlete's preparation for this objective.

Individualize Your Training

Academic research can be incredibly helpful for guiding us in the right direction with our training.

But the fact is you are an individual with unique qualities and abilities. Your training needs to be individualized and unique too.

This is what really separates the athletes who show up to a race completely on top of their game and ones who are nervous on the start line, worried about if they are truly prepared.

Want to Learn More?

If you're an elite level cyclist who wants to start winning races and further your career, click the link below and schedule a time to chat with one of our coaches. We can outline what your most relevant training objectives are and show you what you need to do to really prepare for your target race.

Talk to a Coach