Machine Learning/OmniscopeVisualization

From Noisebridge
Jump to navigation Jump to search

How to visualize data in a SQLITE database using Omniscope[edit]

To get an immediate sense of what this tutorial allows you to do, see the screencast.


Download the Omniscope Trial, which will be good for a few weeks. I have heard people say that they just create a new OSX user to install the program anew and to continue using Omniscope trial beyond its expiration date.

Download the Zentus driver -- sqlitejdbc-v054.jar is the version that works on my OSX 10.5/10.6, your mileage may vary.

Connect to Database -> JDBC database -> enter details:

     /path/to/sqlitejdbc-v054.jar
     org.sqlite.JDBC
     jdbc:sqlite:/path/to/dbKdd.dbev
   

No authentication, click just next; then select table name -- in our case bridge or algebra. Click next in customize SQL;

Then you can select the fields to exclude in your visualization bu unchecking them: In the KDD case, I chose to remove all fields that are not relevant in test data:

all time/duration fields
correct first attempt
incorrects
hints
corrects

Click next then finish. It'll import your data into an IOK file -- which you need to save; Once you've done the conversion into the IOK file you won't need the DB any more for omniscope (but can refresh if data changes).

If necessary, delete the first row (right click) which may contain header information, not actual data.

You can then format your data: Click Data -> Manage Fields. Select the proper data types; e.g. row,problemview fields are an Integer in our KDD data

To slice and dice the data:

  • In the top left dropdown you can select Chart view instead of Bar view. This will give you histograms. Many other possible.
  • change filters on the right to narrow down the data you are looking at.
  • etc...

Note: Many operations in Omniscope take a long time; expect to wait for a few minutes when you click certain things or make selections. This is annoying but it occurs when you deal with datasets greater than a few hundred MB. Save often, so that when your Omniscope hangs you can Force Quit it and open it again if necessary.

Andy