R Package to Download Fitbit Data

Edit 4/8/2015: Updated to use newer functions…
Edit 3/26/2015: fitbitScraper 0.1.2 is now on CRAN with new heart rate data capabilities.
Edit 2/2/2015: fitbitScraper 0.1.1 is now on CRAN

Fitbit is a device that tracks your daily activity. It's basically a pedometer, but it does a little more. It has an altimeter, so it can count flight of stairs climbed. It can detect your sleeping activity and give you a read on how often you are tossing and turning. Any how, I received a Fitbit One for Christmas and have been unexpectedly into this thing. I absolutely did not expect to be motivated by monitoring my fitness activity. It's been fun.

The problem is that Fitbit doesn't allow you to export your data under their “free” data tracking tier. It costs $50 a month to upgrade to “premium” to allow for data export. It's my data, I should be able to get it. So, I made an R package to do just that.

fitbitScraper is my new R package, hosted on Github for now, to download Fitbit data. It uses the internal API that fitbit uses to populate your dashboard. Here, they give data as granular as every 15 minutes. Also, there's daily aggregate data available. Basically, the package logs into fibit.com using your email / password, and is returned a cookie. Using that cookie, the internal API is accessible. For the time being, I can only get that cookie if you use email / password to login. Facebook and Google login are available, but I haven't put any effort into figuring out how to get a cookie if one of those methods are used to login. One could copy and paste their cookie information into R and use this package that way…

Here's a quick example of downloading and graphing step data for a given day. On this particular day, I played basketball from about 6am until 9am.

install.packages("fitbitScraper")  
library("fitbitScraper")
# just reading from file to hide pw and to make .Rmd document to work...
mypassword <- readLines("pw.txt")
cookie <- login(email="corynissen@gmail.com", password=mypassword) 
df <- get_intraday_data(cookie, what="steps", date="2015-01-10")  
library("ggplot2")  
ggplot(df) + geom_bar(aes(x=time, y=steps, fill=steps), stat="identity") + 
             xlab("") +ylab("steps") + 
             theme(axis.ticks.x=element_blank(), 
                   panel.grid.major.x = element_blank(), 
                   panel.grid.minor.x = element_blank(), 
                   panel.grid.minor.y = element_blank(), 
                   panel.background=element_blank(), 
                   panel.grid.major.y=element_line(colour="gray", size=.1), 
                   legend.position="none") 

Here's an example of the daily data.

df <- get_daily_data(cookie, what="steps", start_date="2015-01-13", end_date="2015-01-20")  
library("ggplot2")  
ggplot(df) + geom_bar(aes(x=time, y=steps), stat="identity") + 
             xlab("") +ylab("steps") + 
             theme(axis.ticks.x=element_blank(), 
                   panel.grid.major.x = element_blank(), 
                   panel.grid.minor.x = element_blank(), 
                   panel.grid.minor.y = element_blank(), 
                   panel.background=element_blank(), 
                   panel.grid.major.y=element_line(colour="gray", size=.1), 
                   legend.position="none") 

I have to give a should out to this guy for inspiration. I don't know perl, so I couldn't really use anything here, but it gave me confidence that I could figure it out.

I hope this is of some use to some of you R hackers out there…

Using R to Analyze Yahoo Fantasy Football Data

Updated: 9/16/2014

I recently created a gist that demonstrated how to authenticate with the Yahoo API, using the httr package. In this post, I will expand on this a little to downloading personal Yahoo fantasy football data and creating a graph showing my regular season results.

The first step, obviously, is to authenticate with the Yahoo API and get a signature to attach to all of the requests. This entails registering an application with Yahoo and getting API keys. This page has information about the auth process. I have my credentials saved to a text file that I read in, but you can use environment variables too, whatever you are comfortable with.

library(httr)
library(XML)
library(RJSONIO)
library(ggplot2)

# saved my yahoo keys to a file, now, read them in...
creds <- read.table("~/cn/personal/keys/yahoo.txt", stringsAsFactors=F)
consumer.key <- creds[1,1]
consumer.secret <- creds[2,1]
oauth_endpoints("yahoo")
myapp <- oauth_app("yahoo", key = consumer.key, secret = consumer.secret)
token <- oauth1.0_token(oauth_endpoints("yahoo"), myapp)

Next, you have to discover your game key, league key, and team key. You will need your league id that is in the URL when you visit your normal Yahoo fantasy football page. My league id should not work for you, as mine is a private league.

# need to get game id for my league...
ff.url <- "http://fantasysports.yahooapis.com/fantasy/v2/game/nfl?format=json"
game.key.json <- GET(ff.url, config(token = token))
game.key.list <- fromJSON(as.character(game.key.json), asText=T)
game.key <- game.key.list$fantasy_content$game[[1]]["game_key"]

# my personal leagueid, you will have to use your own, mine is private
league.id <- "262101"
league.key <- paste0(game.key, ".l.", league.id)
league.url <- "http://fantasysports.yahooapis.com/fantasy/v2/league/"

Next, I had to figure out my team id. This was trial and error for me. I'm not sure if there's a better way. Using the matchups endpoint, I can get the data for all of my games for the regular season (2 games as of today). The data is available as XML or JSON. I choose JSON with the “?format=json” at the end of the URL. Then, using the RJSONIO package, I parsed the data to create two vectors, my score for every week, and my opponent's score for every week. The JSON is nested deeply, so this is kinda ugly…

my.team.id <- "4"
my.team.key <- paste0(league.key, ".t.", my.team.id)
team.url <- "http://fantasysports.yahooapis.com/fantasy/v2/team/"
# lots of endpoints to play with, more here... 
# http://developer.yahoo.com/fantasysports/guide/
my.team.stats.json <- GET(paste0(team.url, my.team.key, "/stats?format=json"), 
                          config(token = token))
my.team.standings.json <- GET(paste0(team.url, my.team.key, 
  "/standings?format=json"), config(token = token))
my.team.matchups.json <- GET(paste0(team.url, my.team.key, 
  "/matchups?format=json"), config(token = token))
my.team.matchups.list <- fromJSON(as.character(my.team.matchups.json), asText=T)

# number of games played
game.num <- 2

# get the opponent scores for my matchups for the entire season
tmp <- my.team.matchups.list$fantasy_content["team"][[1]][[2]]$matchups
opp.score <- tmp$'0'$matchup$`0`$teams$`1`$team[[2]]$team_points["total"]
opp.score <- c(opp.score, sapply(as.character(1:(game.num-1)),   
  function(x)tmp[x][[x]]$matchup$`0`$teams$`1`$team[[2]]$team_points$total))
my.score <- tmp$'0'$matchup$`0`$teams$`0`$team[[2]]$team_points["total"]
my.score <- c(my.score, sapply(as.character(1:(game.num-1)),   
  function(x)tmp[x][[x]]$matchup$`0`$teams$`0`$team[[2]]$team_points$total))

Almost there… now we just have to make a dataframe in the format that ggplot2 likes to have…

my.df <- data.frame(cbind(game=rep(1:length(my.score), 2), 
  team=c(rep("me", length(my.score)), rep("them", length(my.score))),
  score=as.numeric(c(my.score, opp.score))))
my.df$game <- factor(my.df$game, levels=1:game.num)
my.df$score <- as.numeric(as.character(my.df$score))

Then we can graph it using ggplot2. Here's the code and output…

p1 <- ggplot(my.df, aes(x=game, y=score, color=team, group=team)) + 
  geom_point() + geom_line() + scale_y_continuous()
ggsave("FF_regular_season.jpg")

There's a bunch more that can be done with this API, the point of this post was to show how to get through the auth stuff and show a simple demonstration of the API.

Full code can be found on github

Follow me on twitter for more R tidbits like this… https://twitter.com/corynissen