Self Tracking without a Written Record

I had a great time giving an unusual talk at the Quantified Self meetup in SF last week. Several people asked me to post slides online. There were also a few questions we didn’t have time to address to which this is a partial answer.

Self-Experimentation without a Written Record 

Tracking my lifestyle changes and related symptoms on an ongoing basis has proved to be challenging.  The severity of my symptoms have never been such that I’ve made detailed note-taking a priority.  Instead, I slowly evolved a mental methodology for keeping track of my experiences by focusing on one hypothesis at a time and slowly accumulating what I consider to be informative observations and conclusions.

In practice, I mentally maintain two mental ‘records’:

  1. Working Model: a collection of beliefs about the phenomenology of my body and environment.  Often I’ll have multiple competing hypotheses about what is going on.  My experiments are performed to test or compare hypotheses.
  2. Compiled Experience: my mental summary of past experience, effectively a list of if-then casual observations I’ve taken away from my experiment.  I’m never afraid to go back and question one if future evidence is contradictory.

My Process

  1. Educate yourself  Augment your working model by talking to professionals, fellow sufferers, sharing stories, reading blogs, reading the medical literature, etc.
  2. Pick one Hypothesis  Given your current experience, pick a hypothesis to test that best fits the evidence you’ve collected and the mental model you have of what is going on.
  3. Focus  What do you need to measure?  Pick no more than 2-3 things.  I often use a 3 point scale for fatigue, mood, and skin symptoms.  I also look out for big events like onset of blurry vision, bad back pain or disturbed sleep.
  4. Confounders  What else could cause the things you are tracking to vary?  You’ll have to watch those two.  Just focus on the big ones.  For example with fatigue and mood, the amount of sleep and exercise is very important (and I do use a Zeo now and again now for this).  When I don’t sleep, I expect scores to decline.  When I exercise I expect them to improve. 
  5. Experiment  Try to keep your habits the same, change one thing implied by your theory and mentally track whether there is a change.  If you have a change in one of the big confounding variables, then I delay an observation until the next cycle or make a mental adjustment.
  6. Repeat  Do the experiment over and over until you are pretty sure you have or have not seen a consistent effect.  Pay special attention to the confounders.
  7. Update your Experience  I maintain a list of foods that seem benign or problematic, a list of symptoms that co-vary together, I’m slowly understand what problematic components different foods have (fructose, histamine, gluten, etc), how to think about my current state of being, etc.
  8. Return to Step 1  Education is critical because you are making judgements about what will or won’t be harmful and it is finding stories of people with similar conditions to you that helps you make sense of your compiled experience and find breakthroughs in your working model. 

Important limitations

This methodology only works well when several conditions hold.  You must have a relatively short window between treatment and effect, the treatment must not have significant short-term effects on your overall condition, etc.  This is typically true for sleep problems, dietary sensitivities, supplements, mild skin and mood conditions, etc.  IF you aren’t going to be aggressive about measuring or tracking, you’ll need patience instead.  This model only leads to discovery after many repeated trials over months or even years.  Repeated experiments are the only way to start peering through the complexity of your body and the limitations of your mind’s ability to correctly recall past events.

There are a few key things to keep an eye out for that can significantly improve the experimental process.

  1. Find a Baseline   Can you get to a healthy state?  Can you find a stable condition (aka “normal”).  If so, you can use this as a basis for experiments.  Some methods relevant to discovering a baseline includes:
    a. Elimination diets / fasting.  Removing all possible dietary triggers can be helpful.
    b. Symptom suppressing drugs (separate condition from other life factors)
    c. Regularity of schedule.  These experiments are easier to perform when your daily routine is extremely consistent.
  2. Co-varying symptoms  Identifying which of your minor ailments or symptoms all vary together is highly informative
    a. Clustering symptoms may help identify multiple conditions
    b. Symptoms that normally cluster may sometimes not cluster which tells you something about your condition or possible confounders.

QS Talk Video

QS Talk Slides



5 thoughts on “Self Tracking without a Written Record

  1. Excellent article, Ian. I’m very interested in the process of self-experimentation (my simple starting point when explaining it to people is ‘Think, Try, Learn’) and your experience helps my thinking. Thank you. For someone wanting to track multiple experiments at once, keeping it in the head wouldn’t work well, I think.You address one of the challenges I’ve faced in my own tracking – that there are so many possible causes and factors. In my case it’s around mood and sleep (separate experiments for each). Being patient and only changing one thing at a time are a few of my challenges. I listed some others recently in my QS blog post Designing good experiments: Some mistakes and lessons – . In particular, as an amateur scientist I found thinking about ABA designs helped to clarify process – what you’re getting at in Find a Baseline.

  2. Matthew, thank you for the comment! That’s an excellent article you reference and as you point out, one of the primary methods for N:1 trials is the ABA design (using your own baseline state as a control). I think one of the key refinements I’m exploring is the issue of managing confounding effects. In population experiments confounding is controlled for by randomization over large populations, or managed through stratification and regression analysis. In a self-trial, you either have to do many ABA repetitions (a large "population") or identify a way to do regression analysis over measurements of likely confounding variables. If we know what variables are most likely to influence our primary dependent variable, then we should measure treatment, outcome and those confounders. One rule of thumb I’ve used to approximate regression analysis (since I haven’t been recording numbers) is to inhibit measuring outcomes when confounding factors are likely to be dominant.

  3. I would argue that _most_ seasoned professionals have problem with this, thus the emphasis in clinical science on double-blinded experiments. Unfortunately double-blinding is hard to achieve in many contexts – and even when you can, doctors often can detect treatment effects and human curiosity drives them to guess who is treated and who is not (since they expect an effect to begin with). That has been shown in some studies to influence patients and even to create false positives.I think the hypothesis space notion can be important for this. When you’re experimenting on yourself to see if something has a change you want, you are also looking to invalidate something else you may believe.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s