KDD Competition 2010: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
Line 21: | Line 21: | ||
== Notes == | == Notes == | ||
* to zip the file on OSX: use command line, otherwise will complain about __MACOSX file: e.g.: zip asdf.zip algebra_2008_2009_submission.txt | * to zip the file on OSX: use command line, otherwise will complain about __MACOSX file: e.g.: zip asdf.zip algebra_2008_2009_submission.txt | ||
== Ideas == | |||
* Add new features by computing their values from existing columns -- e.g. correlation between skills based on their co-occurence within problems. Could use Decision tree to define boundaries between e.g. new "good student, medium student, bad student" feature | |||
* Dimensionality reduction -- transform into numerical values appropriate for consumption by SVM |
Revision as of 21:01, 19 May 2010
We're interested in working on the KDD Competition, as a way to focus our machine learning exploration -- and maybe even finding some interesting aspects to the data! If you're interested, drop us a note, show up at a weekly Machine Learning meeting, and we'll use this space to keep track of our ideas.
Resources
TODOs
Next week, after the Hadoop presentation, we'll show each other how to get the tools working on the data (what one needs to download, any data transformations needed, how to produce submission output) and share any insights on the data gleaned so far
- Vikram -- will present on Hadoop next week!
- Thomas -- will get libsvm working on the data and put together a "how to" guide for doing so
- put together a perl script which will take random samples from the data, for working on smaller instances
- put together a simple R script for loading the data
- Andy -- will get Weka working on the data and put together a "how to" guide for doing so
- Erin -- will work on data transformations and ways to create better representations of the data
- We will need to make sure we don't get disqualified for people belonging to multiple teams!
Notes
- to zip the file on OSX: use command line, otherwise will complain about __MACOSX file: e.g.: zip asdf.zip algebra_2008_2009_submission.txt
Ideas
- Add new features by computing their values from existing columns -- e.g. correlation between skills based on their co-occurence within problems. Could use Decision tree to define boundaries between e.g. new "good student, medium student, bad student" feature
- Dimensionality reduction -- transform into numerical values appropriate for consumption by SVM