You can’t manage what you don’t measure – old management adage
This is the third post in the Biohacking 101 series. The first post gives an overview of what biohacking is all about and a compendium of sources for identifying potential biohacks. The second post explains why properly-carried-out self-experiments are so important to biohacking. This third post discusses the art of quantifying the objectives of your biohack.
The objective is the primary thing that you want to influence with your biohack. In the clinical science jargon, an objective is called a clinical endpoint. These days the top medical journals will only publish results from clinical trials that register their trial before it runs. 1 This includes naming and submitting their clinical endpoint (usually restricted to just one primary endpoint) in advance. These precautions help limit biased data mining of the trial results (naming the objective after the fact allows one to measure many things and pick one that happened to have changed a lot, possibly by chance). They also insure that negative trial results are in fact reported to journals. These rules are just as wise for an individual running a self-experiment to follow as they are for big scientific trials.
It’s not usually a straightforward process to determine if a biohack is effective at improving your objective, even when there is just one objective, let alone several. It helps if your objectives are quantities that can be measured easily. This enables you to track these value before, during and after the biohack, so that they can later be analyzed for statistical significance. Objectives that are measurable quantities often appear in sports – e.g. time to run a distance, high jump height – and in health – body temperature, blood biomarkers, etc.
Often our objectives won’t be quantities themselves but will have good surrogate objectives that are quantities. A surrogate objective is an objective that we use for practical evaluation purposes instead of the objective that we would actually like to influence. Reasons to use surrogate objectives include that they are in fact measurable quantities, and moreover the ease/cost/speed/precision of measuring them. Some good examples of surrogate objectives are: steps taken for activity level, time to complete a race for cardio fitness, weight and reps in big compound lifts for overall strength. Some poor surrogates that are in common (mis)usage are: total calories for a healthy diet and total cholesterol levels for heart disease risk. 2 The latter is a major driver of the >10 billion dollar/year industry of statin drugs.
Example: hack death
Sometimes a large panel of quantified surrogate objectives may be appropriate. If we want to hack death (reduce our chance of dying), well, death is already directly quantifiable. But its value is always the same, or else the game is already up! So noting that we are currently alive is not useful in predicting whether we will be alive in 1,10, or 100 years from now. We need surrogates, and we need a population of people to correlate these surrogates with their mortality rates (even these correlations don’t guarantee a causative relationship, however).
Some biomarkers (measurable indicators of a biological state) that are commonly used as surrogates for mortality risk are: body temperature 3 (lower is better, within reason), triglycerides, fasting glucose and fasting insulin. 4 Age is perhaps the most basic biomarker, but we can’t hack that one ;). A necessary criterion for considering adding an additional biomarker as a surrogate for mortality is that the set of biomarkers including the new addition gives statistically better mortality prediction than the preexisting set.
You should always keep in mind that a biohack shown to be effective at influencing a surrogate is only as useful towards achieving your true objective as the surrogate is a good proxy for it. Good surrogates are those that cause a large part of the variation of the true objective. If changing your surrogate need not cause change in your true objective, then that surrogate may mislead you.
Surrogate gone wrong: PVCs and heart disease
Among people who have survived a heart attack, the presence of premature ventricular contractions (PVCs) is a major risk factor for death, explained largely by death from sudden heart failure. I.e. PVC correlates with future death. PVCs were deemed as good surrogates for mortality in this population. Based on this surrogate, the drugs encainide and flecainide were prescribed to heart attack survivors with PVCs, as they had been shown to be effective in clinical trials at reducing PVCs, and it was believed that they would therefore reduce sudden cardiac deaths and probably total deaths as well. However, the Cardiac Arrhythmia Suppression Trial (CAST) trial eventually demonstrated that these drugs actually increased both total mortality and death from sudden cardiac failure. 5 6 Note that in this case, it wasn’t merely that PVC correlated with future death, there was also a very plausible mechanism that the pathology causing the PVCs was also causing sudden heart failure. While encainide and flecainide proved to be effective at reducing PVCs, they worsened the true objective.
Always be skeptical of your surrogates!
Your objectives may not always be easily quantifiable, even via a surrogate. For example, if you want to hack your creativity, there is no obvious quantitative metric that captures that. In these cases, subjective quantification is one tactic at your disposal. You might ask yourself at various times of the day: How creative do I feel right now? And you could record a score from 1-10.
When using subjective quantification, however, you should be aware of the possibility of bias. Bias is more likely to be a problem for biohacks that alter your mood. For example, the amphetamine mix branded as Adderall® has been shown in at least one study to inspire exaggerated subjective feelings of enhanced productivity when compared to real measured productivity. 7 This is not surprising, given its effect of increasing dopamine, a neurotransmitter that induces a mental sense of reward and/or euphoria. Adderall can actually increase productivity at innately boring tasks via this euphoria increasing one’s engagement in those tasks. But using subjective quantification to evaluate Adderall’s effect on your productivity will lead you to a biased conclusion. One option in cases like this is to team up with a buddy, and have each other evaluate e.g. your productivity or creativity when the other is off the treatment.
An alternative to using subjective quantification of your non-quantified objectives is to use other forms of evidence, such as journal entries or comparative analysis of creative works that were composed before vs during/after the biohack. The lack of a clear quantitative measure for biohacks like this hasn’t prevented broad agreement that certain ones are effective. For example LSD is widely regarded as enhancing creative thought, with testimonials ranging from the Beatles to Steve Jobs.
The upside of some of these non-quantitative metrics is that they can provide a fuller picture than just one or several numbers. The downside is that the evaluation becomes more subjective, and we lose the ability to use statistics in figuring out to what extent the biohack caused an improvement in our objective.
For clarity in evaluating a biohack, it’s generally desirable to identify a quantified surrogate objective for measurement and tracking if your actual objective isn’t a quantity. Our whole approach to biohacking on this site will assume that our objectives are quantities.
Quantify your objectives
If you already have a list of some things that you would like to biohack (see the first post for some inspiration), then take a minute to think about how you might quantify your objectives. If you need to use surrogates, you don’t have to restrict yourself to just one (e.g. squat, deadlift, and bench press for strength). If quantified surrogates don’t seem appropriate, then textual comments/journal entries might be your best bet.
- http://www.icmje.org/news-and-editorials/ ↩
- There are much better predictors for heart disease than total cholesterol, and it is NOT the case that the higher your cholesterol is the more likely you are to die. This plot shows that 200-240mg/dl is the optimal range for men based on world population data. For data for women see: http://www.ncbi.nlm.nih.gov/pubmed/21951982 ↩
- http://www.fightaging.org/archives/2008/04/body-temperature-and-natural-longevity.php ↩
- http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2831640/ ↩
- “Preliminary Report: Effect of Encainide and Flecainide on Mortality in a Randomized Trial of Arrhythmia Suppression after Myocardial Infarction”. New England Journal of Medicine 321 (6): 406–412. 1989. PMID: 2473403 ↩
- Echt, D. S.; Liebson, P. R.; Mitchell, L. B.; Peters, R. W.; Obias-Manno, D.; Barker, A. H.; Arensberg, D.; Baker, A. et al. (1991). “Mortality and Morbidity in Patients Receiving Encainide, Flecainide, or Placebo”. New England Journal of Medicine 324 (12): 781–788. ↩
- http://www.thedailybeast.com/articles/2010/12/21/adderall-concentration-benefits-in-doubt-new-study.html ↩