Tracking the fitness trackers: a mixed bag of results

The results are in: head on over to page two to see how each fitness tracker coped with the 10k, and page three for a 3k second opinion

Around a month ago, an innocuous Slack message appeared. It was my Editor David asking if I wanted to take part in Run in the Dark – a five or ten kilometre jaunt around Battersea Park, which – as the name suggests – is done at night. Being a pretty regular 5k runner, and still taking on the advice from my Science of Running piece, I agreed. Then I was signed up for 10km, despite my protests.

Anyway, between us, we ran 45km (one of us, who will remain nameless, stuck to 5km), and immediately the question became whether we should cover it on the site, or take our inevitable uninspired, sweaty performance to our graves. In the end, we decided it was a great opportunity to test fitness trackers, but given we’d be traveling at different speeds, it made sense to stick them all on one person.

Readers, I was that guinea pig.

We dug around our drawers, cupboards, homes and wrists to dig deep and pile as many fitness trackers onto my arm as possible. What you’re looking at there is (from left to right) is a

Misfit Flash, a Jawbone UP3, an Apple Watch, a Fitbit Flex, a first generation Moto 360 and a Sony Smartband 2, all talking to a HTC One M8 (and David’s iPhone 6s). My arm has never been this valuable, with a recommended retail price of £895 on wearables alone – though over half of that is on Apple. There was talk of including a Google Glass as well, but perhaps taking pity on me, it was overruled.

For the race itself, I divided them up a bit more neatly than that. Watches on the left wrist, step counters on the right, with the exception of the Misfit which – appropriately given its name – was worn on my shoe.

On Thursday morning, feeling like my legs were attached by string, I began to analyse the results…

Right. Anyone looking for something quick and easy to digest has come to the wrong place. The data were messy, inconclusive and incomplete, but here’s what was tracked and how.

First and foremost, I came last of the Alphr 10k runners, with a finishing time – according to the offical stats – of 59 minutes and 26 seconds. Hooray!

Okay, great. Let’s look at what Runkeeper – tracked through the phone and via the Moto 360 – had to say about that:

Right. That makes sense: cruelly the start line wasn’t clear, so I ended up running to it, meaning I went a little longer than I intended to. 

Apple Watch, do you agree?

Not so much. There’s a mystery extra 0.36km here – I’ll give it a pass on the calorie discrepancy, as it’s set up for David’s height and build rather than mine, but distance should be near identical. 

Things get stranger when we look into the splits. On the left is Runkeeper via Android, on the right is Apple Watch:

Now granted, I don’t have enough arms to set all my trackers off simultaneously, but those are pretty big differences. Kilometres 4, 5 and 6 are pretty close (less than 10 seconds) but the rest are all over the place.

Okay. Steps, then. This gets a little complicated because most of the apps don’t bother to break down steps by time of day. Take a bow Fitbit for letting me isolate which steps were which:

Now in theory, green is supposed to represent energetic, orange moderate and red lazy, but given the race didn’t begin until just after 20:00 and I sure as hell wasn’t running to get there, I think you can take that with a pinch of salt.

Still, isolating the specific race start time I can tell you Fitbit tracked 8,750 steps, and because it breaks it down into five minute segments, I can roughly say that at 8:35 and 8:50 I took less steps – 678 and 673 respectively. This roughly matches up with Runkeeper’s slowest kilometres, which makes sense.

Now we turn to another step counter. The Misfit Flash:

That’s… not very helpful. There’s no option to break down by time, so I can’t tell exactly how many steps were carried over from the run, and how many were after heaving over the finish line (the time stamp at the top was my edit, though the app had a good old guess at when I stopped.) Around 700 more steps than FitBit, but it was worn on the shoe, rather than the wrist which would suggest higher accuracy, perhaps? Oh, except you’ll notice that 5.7 miles does not equal 10km. I’ve lost half a mile somewhere.

The Moto 360’s step rate isn’t too helpful – it gives you a daily total: 20,522, but no way of breaking that down to the specific time. Or at least not on the watch itself.

Which brings me to heartrate. Or rather, it should, but only the Apple Watch bothered to track it in a usable way. The Jawbone – due to a misunderstanding – wasn’t installed on either me or David’s phone, and the Sony Smartband 2 logged it, but then forgot it when I didn’t install the Lifelog app. In any case, the latter was constantly disconnecting throughout the race.

All of which leads to the main point… I need to do this again. Tomorrow morning, I’m going to run 5k before work, with all the fitness trackers on again. Everything should work, and more importantly, as they’ll be my first steps of the day, they won’t get muddled up with my usual wanderings.

Did you hear that? That was my legs screaming out in terror…

In the end, I decided to run three kilometres, rather than five: I’m was still exhuasted from the 10k, there’s loads to do and… look, I don’t need to justify myself to you. You’re not my personal trainer, okay?

I can be 100% confident that you aren’t my personal trainer, because I don’t have one. But plenty of people are considering making their wearable their fitness mentor of sorts: a gentle nudging to actually head outside and get active. But how much should you trust them? Let’s find out.

To keep things as impartial as possible, I used the online version of Map My Run to measure my exact route before setting out, and it came out at 3.36km. For a second opinion, Mapometre called it 3.3km. Even the elevation was similar between the two:

Good enough for me.

To avoid additional steps counting, I did the second run first thing on Friday morning, and had my laptop to hand as soon as I stumbled through the door. This time, accompanying me on my run were the Misfit Flash on the foot, the Fitbit Flex, Jawbone UP3 and Sony Smartband 2 on my right wrist, and the Moto 360 and special guest star of the Gear S2 Watch on my left. Alas, the Apple Watch refused to pair with a test iPhone 5 I was all set to use, so missed out this time around.

So how did I do? Runkeeper, tracked via the HTC One M8, pretty much confirmed the online mapping estimates, and tracked me with a time of 15:53. Much better than I managed on Wednesday when it actually mattered. Ho hum.

As for the rest, here’s everything I learned, one point at a time:

Unless your tracker has GPS, distance traveled is pure guesswork

Okay, so we know I ran 3.25km this morning, but clearly nobody bothered telling my fitness trackers, which took a ‘how many sweets in the jar’ approach to guessing the magic number.

Here’s a quick chart with each tracker’s guess at the distance, and – as a constant reminder – the actual figure. I’ve put it in ascending order of accuracy:

How long did each tracker think I ran?

Tracker	Estimated distance	Actual distance
Moto 360 via Google Fit	2.0km	3.3km
Sony Smartband 2	2.1km	3.3km
Misfit Flash	2.4km	3.3km
Gear S2 Watch	2.57km	3.3km
FitBit Flex	2.67km	3.3km
Jawbone UP3	3.04km	3.3km
Runkeeper (on handset)	3.25km	3.3km

Without a GPS, all these fitness trackers are just guessing – and guessing pretty poorly, with the exception of the Jawbone UP3. Every one except the Gear S2 watch, all of these had my measurements, so why are they so far over the place?

Perhaps a glance at the steps will provide some answers?

One tracker’s step is another’s shuffle

Steps, it turns out, aren’t much better. Again, every tracker was aware of my height and weight (Gear S2 Watch aside), and yet the numbers of steps taken were wildly inconsistent.

This time, I don’t have a ‘real’ figure to quote (when running, I find it hard enough to concentrate on putting one foot in front of the other, without bringing counting into the equation), so it’s hard to guess which is ‘right’ here, if any:

How many steps did I take?

Tracker	Estimated steps
Moto 360 via Google Fit	1,773
Sony Smartband 2	2,034
FitBit Flex	2,171
Gear S2 Watch	2,353
Misfit Flash	2,360
Jawbone UP3	2,532

Now, wisdom of crowds suggests that with only seven steps between them, the Gear S2 Watch and Misfit Flash are right on the money here, but given the Jawbone UP3 was the only one to come close to the actual distance, I’m inclined to say it was near the top end. Certainly, the Sony Smartband 2 and Moto 360 seem particuarly poor. Both of them are supposed to track without any intervention, but then again so did the Jawbone, so it’s not an excuse.

Over short distances, non-pro heartrate monitors appear only slightly better than pointless

Seriously, if this test is representative, then I wouldn’t pay them much attention. I won’t include a table for this one, given only four of the wearables ‘tracked’ it. To be honest, the ones that didn’t may as well have just guessed a number, and they wouldn’t have looked out of place.

An important caveat, of course, is that this was only 3km, and other tests have suggested they take a while to warm up, but even so: these figures are way off.

The Smartband 2 capped me at 78 beats per minute, the Jawbone UP3 at 61 and the Gear S2 watch at 87. Given on the previous page, the Apple Watch detected an average of 151 and that these figures aren’t horrible resting heartrates, I think these are all rubbish.

After sitting down to write this for half an hour, the Jawbone jumped to 90. I really don’t get that excited compiling HTML tables. You may as well take my word for that – it’s not like the trackers could give you a consensus.

Nobody knows how long it takes to burn off a bag of Mini Cheddars

Each app has my weight and height (again, except the Gear), so a calorie burn consensus should be possible. Turns out, it isn’t as simple as that, as the chart below demonstrates.

How many calories did I burn?

Tracker	Calorie burn (kcal)
Gear S2 Watch	204
Moto 360 (via Google Fit)	215
Misfit Flash	311
Fitbit Flex	303
Runkeeper (on handset)	323
Sony Smartband 2	352
Jawbone UP3	402

That might not seem like a big deal, but to put that into context, a 25g bag of Mini Cheddars contains 128 kilocalories. That means that the difference between the Moto 360 Watch’s estimate and the Jawbone UP3’s total is around one and a half bags of cheesy goodness.

But ultimately, it doesn’t matter too much – as long as the wearable is consistent

Most people won’t do what I did this week: if you take one thing from this short series, please let it be that nobody needs to wear more than one fitness tracker. Seriously, don’t do it: it’s a right old pain.

The point is that it doesn’t really matter how inconsistent these trackers are against each other, as long as each is consistent with itself.

All a Fitbit Flex or Sony Smartband 2 really needs to tells you is when you’ve had a good or a bad day. If it encourages you to take more steps than you would normally, then it has done its job. The trouble comes when you’re comparing with friends, and that’s when a GPS really comes in handy as a metric to really sense-check the rest of the data.

Without sounding too schmaltzy, getting out there and getting fit is the important thing. Any one of these trackers will give you that bit of incentive, and as long as you don’t take any single metric as gospel, then they can be an invaluable tool on the road to a fitter, healthier you.

Now if you’ll excuse me, I’m going to go home, lie down, and not think about running for a couple of weeks. I’ll might even reaquaint myself with what my arms look like when not covered with screens and LEDs.

READ THIS NEXT: The science of running: how can you go faster and longer?