Tracking My Personal Reliability

Tracking My Personal Reliability

Hi, I’m Daniel Reeves, co-founder of Beeminder. This talk is mostly not about Beeminder, but the common theme is committing to things.

First, in case you don’t know about Beeminder it’s a tool for ongoing usually open-ended quantifiable commitments like getting 10,000 steps per day measured by Fitbit, or 40 hours per week on your job as measured by RescueTime or staying under a 100 kilograms as measured by your Withings scale. Or even a number of items marked done in Hibitica, or messages in Gmail. Beeminder has dozens of integrations with other Quantified Self-tools and less QSE apps that generate data about you incidentally.

Oh yeah, and the commitment part is that you put in a credit card and agree to get charged money if you don’t keep all of your data points on track for your goal every day.

But what if you want to make, if you want to a specific onetime thing like, ‘I will send in the slides for this talk.’ I noticed that I would casually make ‘I will’ statements a lot, ‘I’ll see if I can reproduce that bug’ or ‘I’ll let you know if I can make it to the meetup’, or ‘I’ll send you those photos.’

And it was really bugging me that my future tense statements can sometimes be falsehoods, so I can try to assist them so that instead of making a potentially false statement about what I’ll do, I can instead, impeccably, truthfully, and more informatively always give an exact probability that I’ll do the thing.

My current reliability for doing for what I say is about 97%. Before I explain that I’ll give some more background. If you know much about eh behavior change literature, then you’ll have heard of implementation tension or in much simpler terms trigger action plans.

Here’s one I’m proud of. Anytime I hear myself saying I should do something, I should write more code, I should run a marathon someday etc. then I take some tiny immediate action that makes that thing more likely to eventually happen. That’s been working well for me for years. Mostly of the effect of me not making I should statement in the first place, and instead rephrasing them like, ‘It might be a good idea to do X, but’, and then listing costs or downsize.

I feel like this is still a win if only because it makes me feel less guilty about all the things that I ‘should’ be doing but aren’t.

So last year I had the idea of doing something similar with ‘I will’ statements. I started the following trigger action plan. Anytime I heard myself making an I will statement, even the most casual I’ll take a look.

I had to explicitly mention a deadline, calendar it and add a data point to Beeminder graph. Initially the Beeminder graph was enforcing that I fulfill eight promises per week. No stipulation of the denominator, just that I fulfill per week. That Beeminder graph is still going, but astute observers will notice a few things about it.

For one, I backed off from the eight per week and have averaged just over five per week overall. But more interestingly, there are 364 promises logged hear. Actually, a few less than that because of dummy data points, and each time I derailed and paid Beeminder money. But I only have credit for 384.3 of them so what’s up with that? Well funny you should ask.

When I started tracking this I soon realized although fundamentally I care about the probability of the fraction of commitment fulfilled. I needed a way to quantify fulfilled but late. And I wanted to distill it all into a single metric for reliability because that’s the whole point. So this is what I came up with.

I figured if I missed a deadline by mere seconds, that 99.99% count. If I missed it by some minutes, that’s still 99.9% counts. Hours late isn’t a big deal that almost like counts days late, it’s like, well it got done and that’s the main thing, 90%. Being weeks late kind of half defeats the point, so that’s 50% credit. Months late mostly doesn’t count, 10% and years is like, better late than never but just barely.

Okay, so the real “nerdery” is in defining this function for exactly how much credit you get as a function of how late you are. There are three properties we need. One is strict monotonicity meaning that you always see your reliability score visibly ticking down second by second whenever you have an overdue commitment. Two is that it as it approaches zero, so you always get some epsilon of credit no matter how late you are. And the cleaver behavioral property is that there are these sudden drops at every shelling fence.

We don’t want you to feel like once you’ve missed another deadline by an hour or day or week won’t matter. That’s a slippery slope to not finishing ever. So I’ll come back to this if people have questions about it but you don’t need to draw on this to understand the rest of the system.

So how do you actually use it? So, suppose your name is Alice and your coworker or friend is Bob. Whenever Alice says something like, I’ll send you my edits tomorrow, she types the URL like so. She literally types that in on the fly directly to Bob manually when she’s making a commitment, and we’re assuming they’re talking by email or Slack or whatever.

When she or Bob clicks that URL, the commitment is created in the app and the entry is added to Alice’s calendar, and ideally, a data point is sent to Beeminder if that person is on manual. The system lets you mark the commitment completed and tracks your liability and shows it up to anyone whoever follows one of those links that you give them.

So, here’s me telling Steven Jonas that I sent him my QS slides. I added a note to remind myself that I promised in an email to Steven. And again, the UI here was literally typing that URL in the email to Steven. Three dot kiss dot two slash send underscore slides etc.

Clicking that link was a commitment which I can then later edit, or mark fulfilled. And if you visit just three dot commits dot two, you’ll see this , my gallery of commitments.

So some quick numbers of myself. That’s my liability based on 391 commitments plus seven that don’t yet count for or against my score because they’re not due yet. But my liability is gradually ticking down second by second right now because I have 10, when I wrote this slide it was 11 before I actually sent the slides, commitments that are currently overdue.

So some quick numbers for commits overall. We have about 10 active beta users with a bit over a thousand commitments in the database altogether. And we’ve been working on this for exactly a year now.

I’ve learned a lot from working on this and using it. Like in a way it adds stress. That’s kind of the point. But yet, it’s also weirdly relaxing I think for a similar reason that people find David Allen’s getting things done system relaxing.

So there are definitely some failure modes, and of course, one lesson and that was part of the point all along is you promise less, beak the habit of saying you’ll do something and really mean it. Our friend and hard-core beta user Kim making that point.

Another lesson is to promise specifics. I’ve been burned by promising things that seemed straightforward but turned out to be ill-defined. Better to promise a very concrete first step and then make another promise for the next step if it still makes sense.

And finally, the idea of commits is to be motivated by your reliability score, and seeing it tick down when you miss a deadline. But for me anyway that’s not enough. My scores been slipping lately and despite those shelling defenses for the individual score, each individual score doesn’t contribute much to my overall liability, so I’m still on a slippery slope. Which just means I still need Beeminder, so that’s good.

As with Beeminder, I think it’s only a certain kind of personality who wants to put them through this kind of thing, but I can only assume that personality type is overrepresented here, so.

This is still in enclosed beta, but if you want to commit to helping build it then you can start using it now. Just try your name dot commits dot to and either you’ll see someone’s gallery and commitments if that names taken, or you’ll see instructions for how to learn more and get involved.

Thank you.