Friday, December 6, 2013

Using R to Analyze Yahoo Fantasy Football Data

Updated: 9/16/2014

I recently created a gist that demonstrated how to authenticate with the Yahoo API, using the httr package. In this post, I will expand on this a little to downloading personal Yahoo fantasy football data and creating a graph showing my regular season results.

The first step, obviously, is to authenticate with the Yahoo API and get a signature to attach to all of the requests. This entails registering an application with Yahoo and getting API keys. This page has information about the auth process. I have my credentials saved to a text file that I read in, but you can use environment variables too, whatever you are comfortable with.

library(httr)
library(XML)
library(RJSONIO)
library(ggplot2)

# saved my yahoo keys to a file, now, read them in...
creds <- read.table("~/cn/personal/keys/yahoo.txt", stringsAsFactors=F)
consumer.key <- creds[1,1]
consumer.secret <- creds[2,1]
oauth_endpoints("yahoo")
myapp <- oauth_app("yahoo", key = consumer.key, secret = consumer.secret)
token <- oauth1.0_token(oauth_endpoints("yahoo"), myapp)

Next, you have to discover your game key, league key, and team key. You will need your league id that is in the URL when you visit your normal Yahoo fantasy football page. My league id should not work for you, as mine is a private league.

# need to get game id for my league...
ff.url <- "http://fantasysports.yahooapis.com/fantasy/v2/game/nfl?format=json"
game.key.json <- GET(ff.url, config(token = token))
game.key.list <- fromJSON(as.character(game.key.json), asText=T)
game.key <- game.key.list$fantasy_content$game[[1]]["game_key"]

# my personal leagueid, you will have to use your own, mine is private
league.id <- "262101"
league.key <- paste0(game.key, ".l.", league.id)
league.url <- "http://fantasysports.yahooapis.com/fantasy/v2/league/"

Next, I had to figure out my team id. This was trial and error for me. I'm not sure if there's a better way. Using the matchups endpoint, I can get the data for all of my games for the regular season (2 games as of today). The data is available as XML or JSON. I choose JSON with the “?format=json” at the end of the URL. Then, using the RJSONIO package, I parsed the data to create two vectors, my score for every week, and my opponent's score for every week. The JSON is nested deeply, so this is kinda ugly…

my.team.id <- "4"
my.team.key <- paste0(league.key, ".t.", my.team.id)
team.url <- "http://fantasysports.yahooapis.com/fantasy/v2/team/"
# lots of endpoints to play with, more here... 
# http://developer.yahoo.com/fantasysports/guide/
my.team.stats.json <- GET(paste0(team.url, my.team.key, "/stats?format=json"), 
                          config(token = token))
my.team.standings.json <- GET(paste0(team.url, my.team.key, 
  "/standings?format=json"), config(token = token))
my.team.matchups.json <- GET(paste0(team.url, my.team.key, 
  "/matchups?format=json"), config(token = token))
my.team.matchups.list <- fromJSON(as.character(my.team.matchups.json), asText=T)

# number of games played
game.num <- 2

# get the opponent scores for my matchups for the entire season
tmp <- my.team.matchups.list$fantasy_content["team"][[1]][[2]]$matchups
opp.score <- tmp$'0'$matchup$`0`$teams$`1`$team[[2]]$team_points["total"]
opp.score <- c(opp.score, sapply(as.character(1:(game.num-1)),   
  function(x)tmp[x][[x]]$matchup$`0`$teams$`1`$team[[2]]$team_points$total))
my.score <- tmp$'0'$matchup$`0`$teams$`0`$team[[2]]$team_points["total"]
my.score <- c(my.score, sapply(as.character(1:(game.num-1)),   
  function(x)tmp[x][[x]]$matchup$`0`$teams$`0`$team[[2]]$team_points$total))

Almost there… now we just have to make a dataframe in the format that ggplot2 likes to have…

my.df <- data.frame(cbind(game=rep(1:length(my.score), 2), 
  team=c(rep("me", length(my.score)), rep("them", length(my.score))),
  score=as.numeric(c(my.score, opp.score))))
my.df$game <- factor(my.df$game, levels=1:game.num)
my.df$score <- as.numeric(as.character(my.df$score))

Then we can graph it using ggplot2. Here's the code and output…

p1 <- ggplot(my.df, aes(x=game, y=score, color=team, group=team)) + 
  geom_point() + geom_line() + scale_y_continuous()
ggsave("FF_regular_season.jpg")

Drawing

There's a bunch more that can be done with this API, the point of this post was to show how to get through the auth stuff and show a simple demonstration of the API.

Full code can be found on github

Follow me on twitter for more R tidbits like this… https://twitter.com/corynissen

13 comments:

  1. Cory, this is fantastic, and I enjoy following your blog. I'm looking to analyze my own league's data, but I'm having trouble getting R to authenticate the API. The problem comes when I submit the token assignment statement. Here's what R returns:

    > token <- oauth1.0_token(yahoo, myapp)
    Use a local file to cache OAuth access credentials between R sessions?
    1: Yes
    2: No


    No matter which option I select, R returns the following:

    Error in init_oauth1.0(endpoint, app, permission = params$permission) :
    client error: (401) Unauthorized

    When I then go to submit the sig assignment statement, I get the following:

    > sig <- sign_oauth1.0(myapp, token$oauth_token, token$oauth_token_secret)
    Error in sign_oauth1.0(myapp, token$oauth_token, token$oauth_token_secret) :
    object 'token' not found

    Do you have any idea what might be the problem here? From looking at my Yahoo application page, it seems that I may need to verify the callback domain, but I'm not sure how to do that. Is there anything in particular I need to provide to Yahoo as my application URL and callback domain or is there something I need to do to activate the application? Any help would be greatly appreciated.

    ReplyDelete
    Replies
    1. I updated the code a couple months ago, make sure you are using the current stuff... It should look like this...

      consumer.key <- "your key"
      consumer.secret <- "your secret"
      myapp <- oauth_app("yahoo", key = consumer.key, secret = consumer.secret)
      token <- oauth1.0_token(oauth_endpoints("yahoo"), myapp)

      # need to get game id for my league...
      ff.url <- "http://fantasysports.yahooapis.com/fantasy/v2/game/nfl?format=json"
      game.key.json <- GET(ff.url, config(token = token))

      A couple things... When you sign up for a key, on the page that gives you the keys, there's an option at the bottom for permissions. Check Fantasy Sports and click "Save and change consumer key".

      The yahoo page says this about a 401 error: 401 Error: Unauthorized. This error is normally caused by a problem with the OAuth parameters attached to your request. The OAuth signature or OAuth Verifier may be incorrect or one of the tokens is in a bad state (e.g. you used a request token instead of an access token).

      Hope this helps...

      Cory

      Delete
    2. Have you solved this issue. I'm running into the same problem. The token command always generates a 401 error.

      Delete
    3. After working a little bit step 3 from http://nullinfo.wordpress.com/oauth-yahoo/ (and the original flow chart) seems to indicate that we should have to get an user authentication key, however R is not passing the oauth_token to us to go authenticate. How did you work around that?

      Delete
    4. The code is working for me. Did you create an application and get the keys correctly? You have to give read permission to "Fantasy Sports" in order for this to work. Also, install the latest version of the httr package. Maybe you have an older version. It was updated a few months back and I had to change my original fantasy football code to get it working again.

      Delete
    5. httr caches the token, but Yahoo tokens expire, unlike twitter and others. So, don't use the cache... token <- oauth1.0_token(oauth_endpoints("yahoo"), myapp, cache=FALSE)

      Delete
  2. I too am getting the 401 error. I have a feeling the issue is related to how I registered my app on YDN. Is there something special you used for your callback domain? I am trying to use localhost as my callback domain and I feel that is why I'm getting a 401 unauthorized error.

    ReplyDelete
    Replies
    1. I just registered a new dummy app and didn't see a place to enter a callback domain. I remember doing this using the google api, but not yahoo.

      Delete
  3. Hi Cory, it worked for me with a new app and with httpuv loaded in R. I also used XML instead of JSON, here is that code if people are curious.

    https://github.com/bcrossman/yahoofantasyscrape

    ReplyDelete
  4. Has anybody gotten past the 401 unauthorized error yet, I am stuck in the same place as most. I tried to register my APP a few different ways but no luck.

    ReplyDelete
    Replies
    1. I tested my code above again... and the authorization worked. It opened a browser window and saved the auth token. Here's my sessionInfo() if it'll help...

      R version 3.1.3 (2015-03-09)
      Platform: x86_64-pc-linux-gnu (64-bit)
      Running under: Ubuntu 14.04.2 LTS

      locale:
      [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
      [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
      [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
      [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
      [9] LC_ADDRESS=C LC_TELEPHONE=C
      [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

      attached base packages:
      [1] stats graphics grDevices utils datasets methods base

      other attached packages:
      [1] ggplot2_1.0.1 RJSONIO_1.3-0 XML_3.98-1.1 httr_0.6.1

      Delete
  5. I ran into the 401 issue and was able to work through it by 1- installing the httpuv library and 2- adding 'options("httr_oob_default" = T)'. Hope that helps!

    ReplyDelete
  6. Hi Cory,

    Have you had any success making a more specific call using the sub-resources in JSON format? For example, making calls to a particular player from a fantasy league team:

    http://fantasysports.yahooapis.com/fantasy/v2/league/223.l.431/players;player_keys=223.p.5479 (yahoo's URI example)

    Thanks!

    ReplyDelete

Note: Only a member of this blog may post a comment.