Self Tracking and Methodology

What is Self Tracking??????????

Self-tracking is a process through which we attempt to uncover patterns in our daily lives or environment. Tracking can be used for a variety of purposes, including exploratory (what correlations do I see), explanatory (why does this happen) or experimental (If I change X, Y should happen). Regardless of the specific purpose, our ultimate goal is almost always to develop some model of cause and effect that we can use to inform our future decisions. The discovery of cause-effect relationships and the consequences of interventions is the essential aim of the scientific method. It takes years of education and practical training to understand how to apply methodology to gain valid insights into fundamental questions about cause and effect in some natural or artificial system. Methodology is crucial to avoid developing incorrect conclusions.

However, we must also acknowledge that tracking, modeling and intervening in system is a fundamental human exercise. As children we play with blocks to learn the rules of physical cause and effect and engage in a variety of behaviors that help us understand how our interactions influence the world. As adults we continue this process developing ideas about the systems we interact with: natural, artificial, or human.  The models that we build about these systems are often (usually?) wrong.  For example, Piaget’s writing is full of examples of how children incorrectly model the world, yet invariably figure much of it out.  As humans we build models that are effective or good enough.  As scientists, we seek to discover models that are formal and valid.  Formality and validity are crucial elements of the scientific process because we need to share our models and have a disciplined way of testing whether and when they have explanatory power about the world.

The goals of self-tracking lie somewhere in-between these two extremes, and thus are less ambitious than that of the formal scientific method.

1. Purpose: We are interested in efficacy, not validity. There are many models that provide the same answer to specific questions, progress is made when we compare two models with a specific experiment to see which provides the more accurate prediction of observed effects
2. Accuracy: We are willing to be wrong, for awhile. In self-tracking there are no review boards or publication committees. The primary goal is to make practical progress towards a concrete end.
3. Generalizability: We are interested in the individual, or in a select population, not everyone. We don’t necessarily need to identify or elucidate an underlying scientific truth, we simply need to identify the interventions that works for us.

As individuals we make the best progress when our hypotheses are well founded and the conclusions drawn from our experiments are accurate.  As a community we make the best progress when we can share our ideas in ways that accurately predict how someone else will fare.  However, like the child learning how to be a functioning member of society, self-tracking can be a process of slow improvement and false starts – an ongoing process of learning how to interact with ourselves and our environment that converges towards accuracy.

Methodology in Self-Tracking

There are few generally accepted guidelines today that help us engage in a rigorous, structured process of self-experimentation.  We can draw on classic statistics, and a few leading lights like Seth Robertson who demonstrate how to iterate hypothesis based on observations of recorded evidence.  However, there is literally no methodological guidance for how we can best learn from each other other than through the exchange of anecdotal stories. Medicine, Psychology, Sociology and other disciplines have developed a variety of methodologies and trial designs aimed at combating well known biases which obfuscate or mislead during the experimental process. Many of the most powerful techniques applied to ensure validity in scientific experimentation are difficult or impossible to apply to the self-tracking model.

Most scientists, when confronting this fundamental challenge, say that we have no choice but to go to a professional for help in working through problems. In situations where there is scientific results as well as professionals available who know how to address our problem, I agree. The reality is that there are many subtle conditions (depression, bowel problems, food sensitivities, skin disorders) that are amenable to lifestyle intervention, for which the average doctor is ill-equipped. The internet is rife with stories of how hard finding the right expert can be. Not all doctors are equally equipped to solve any given problem. Moreover, many people don’t go to the doctor at all. Instead, millions of people turn to popular medical articles, blog posts, discussion forums, alternative practitioners and snake oil sites all having a high variance of underlying methodological rigor.

The darker side of saying that everyone must consult a doctor is the presumption that there are no conditions under which individuals can responsibly engage in methodology designed to improve their own condition.  Given the massive unmet need, the decreasing availability of doctors, do we give up? Is there no scientific methodology for the undiagnosed condition, or the disenfranchised individual? Can the scientific community offer nothing more than “you are on your own” when there is no published journal article addressing an untested patient hypothesis?  That was the answer my last doctor gave me at my annual exam. I at the least appreciated that he was honest about his limitations.

I believe that most scientists look at this challenge the wrong way. The question is not whether self-tracking is a valid means to yield scientific insight; the jury will undoubtably be out on that for some time. The question is whether self-tracking can be better structured such that it yields more effective results over shorter periods of time.  If the efficacy of a methodology can be measured, then it can be subject to scientific analysis and we can compare one methodology to another across a population to determine ‘best practices’ for self-tracking.  A community of motivated, and highly self-educated individuals may serve as an adequate regulator on methodology and information sharing, particularly if tools are available to help monitor and evaluate both data and methodology.

Health-related questions people use self-tracking for include:

1. What specialist should I go see?
2. Should I switch doctors?
3. How can I best manage side effects of the care I’ve being given? (Where there is no scientific data)
4. I’ve seen 10 doctors, and none have been helpful. How do I chose which Internet suggestion to try next?
5. I think I know what’s going on, how do I test my hypothesis? What do I measure? How do I measure it? When do I have enough data to be confident?

The closest scientific methodology to the self experiment is the “N of 1” trial used in some areas of medicine to identify the specific treatment among a set of approved treatments. In this methodology the patient is used as their own control and repeated cycles of randomized and blinded treatments are given to evaluate the relative strength of response of each treatment.

To be successful, these trials require certain conditions to hold:

• Relatively stable condition we are seeking to manage, that is the intervention or treatment doesn’t change the overall condition.
• Measurable or verifiable baseline conditions, i.e. how do we know when the last treatment is out of our system?
• Short half-life of treatment (we can run multiple cycles in reasonable timeframes)
• Rapid onsite / offset of treatme
  nt effects

In my own process of ad-hoc self-tacking, I discovered that identifying these parameters and using them to structure the way I engaged in my daily life and the way I accumulated evidence for and against my competing hypotheses, dramatically accelerated my ability to make progress in understanding the appropriate means to manage my own quality of life. I hope that sharing my own recipe, and the limitations I was working with, can serve as an example methodology that is better than random and can lead us towards “best practices” for self-tracking. See my prior post on Self Tracking without a Written Record.