Shindig Data Integration Development Guide

This guide is designed to help you get familiar with working with Shindig as well as show an example of how Shindig can be extended to communicate with an external database. The four tutorials are broken down as: 

Basics
  1. Tutorial 1: Downloading, building and running Shindig
  2. Tutorial 2: Editing and Debugging Shindig in Eclipse
Intermediate
  1. Tutorial 3: Creating a MySQL backend database for OpenSocial data
  2. Tutorial 4: Using extended Java classes within Shindig to talk to your SQL database

Important Notice: Since Shindig is a moving target with frequent changes which sometimes break the code, a clean version of Shindig originally downloaded on May 9th is downloadable here:

http://chrisschalk.com/shindig_docs/shindig_sql_tutorial/shindig-5-9-save.tar.gz

This code bundle has been is guaranteed to work with this tutorial.


Tutorial 1: Downloading, building and running Shindig

Shindig can be checked out of the publicly accessible subversion server from the Apache incubator website.

These steps were taken adapted from: http://incubator.apache.org/shindig/#tab-building

Prequisites before building Shindig

In order to build Shindig, you must have the following:

Getting the code

Create a subdirectory and checkout the Shindig code from its Subversion repository.

  1. mkdir ~/src/shindig (or wherever you'd like to put it)
  2. cd ~/src/shindig
  3. svn co http://svn.apache.org/repos/asf/incubator/shindig/trunk/ .

    Alternatively, you can just download the May 9th shindig (shindig-5-9-save.tar.gz) mentioned earlier, and unzip it into this shindig directory. This version has been tested to work with this tutorial.

Building and running the code (with Maven)

To build the entire Shindig and run tests, perform the following:

  1. Make sure you have the prerequisites installed first.
  2. cd ~/src/shindig/
  3. mvn
  4. Once the build completes, you can run Shindig with:
    mvn -Prun

This will start Shindig in a local Jetty server that will run at localhost:8080. Note: mvn -Prun is the same as changing to the ..shindig/java/server and executing:

  1. mvn jetty:run-war

To run the Jetty server on a different port, use:

  1. mvn -Djetty.port=<port> jetty:run-war

Testing your running Shindig instance using the sample container

Once you are running locally using the Jetty server, you can check to see if Shindig is running by visiting:

samplecontainer

 

Some further facts about the SampleContainer

You'll notice that the samplecontainer as a default points to a test OpenSocial application at:


This is a very basic HelloWorld OpenSocial application example.

Feel free to inspect the source files for the sample container. They are located in <shindig>/javascript/samplecontainer


These include:



To experiment with the samplecontainer a bit, you can load other OpenSocial applications described at the code.google.com/opensocial documentation pages. For example, feel free to try out the tutorials listed at:
http://code.google.com/apis/opensocial/docs/0.7/devguide.html

A first example test is to try out the simple listfriends application. The source code for this application is listed here:
http://code.google.com/apis/opensocial/docs/0.7/devguide.html#ListFriends_Complete


Tutorial 2: Editing and Debugging Shindig in Eclipse

To improve the Java development experience, it's helpful to use an IDE such as Eclipse. It is relatively easy to setup Shindig to be able to be run or debugged directly from Eclipse as opposed to running from a command line.

The following steps for running Shindig from Eclipse are based on these original instructions:
http://incubator.apache.org/shindig/#eclipseProject

Installing Maven to work with your Eclipse IDE

The first steps to enabling Eclipse to work with Shindig is to install the Maven plugin for Eclipse. This will allow you build use Maven's dependencies. To get started, in your home directory, create an empty Maven settings file:



Installing the Maven plugin and setting up your Shindig workspace

In order to easily build, package and launch Shindig directly from Eclipse for debugging purposes, you can install the Eclipse Maven plugin.

Creating projects in Eclipse to map to the Shindig java projects

Before creating Eclipse projects that map to Shindig code, it's important to understand the Shindig project organization. The Java code for Shindig is divided into 3 separate code projects and a utility Web project for running Shindig:

Creating projects in Eclipse based on any of the existing Shindig code projects is done in a similar manner. You can create projects for each code project that you wish to examine the code.

For each code project of Shindig [common | gadgets | social-api |server], you can create Eclipse projects:

Enabling Maven for your new Eclipse projects


Creating Jetty run and Debug profiles in Eclipse

These steps allow you to run and debug Shindig directly from Eclipse which allows for easy debugging of Shindig source code. The steps involve creating run profiles to package Shindig, and start Jetty as well as a debug profile to launch the webapp and attach the debugger.

Creating a run profile to execute: "mvn package"

This step is optional, but allows you to execute the command "mvn package" from Eclipse without having to open a shell window and do this manually.




Tutorial 3: Creating a MySQL backend database for OpenSocial State data

Important Note: Tutorials 3 and 4 use code that is downloadable from: http://chrisschalk.com/shindig_docs/shindig_sql_tutorial/shindig_sql.tar.gz

You can either refer to the included READMEs in this file, or follow along here. The content here is derived from the READMEs provided in the downloadable file. Before continuing, a few notes about the downloadable archive: shindig_sql.tar.gz.

As you download and open the contents of this archive, you'll see the following:

The remainder tutorials explain the same information provided in the README files in the downloadable bundle.


Installing MySQL on your system

These steps largely mirror what is provided in: <shindig_sql>/sql_script/README.txt

If you don't have a MySQL database, the steps for downloading and installing a MySQL database can be found here:
http://www.mysql.com/

The free "community server" edition of MySQL which runs on all the major OS platforms is sufficient for these exercises and can be found at: http://dev.mysql.com/downloads/

After installing MySQL database, you might want to also install the GUI Tools available for MySQL. These are available at:
http://dev.mysql.com/downloads/gui-tools/5.0.html

Although the GUI Tools are not required, the 'MySQL Administrator' tool is extremely helpful when setting up any MySQL database. It can be downloaded from the MySQL website.

Setting up a MySQL schema for OpenSocial

1. Setup 'opensocial' schema .
As mysql root, create a new database: 

mysql> create database opensocial;

After completion of the new database, create a new user opensocial with password opensocial;

mysql>use opensocial;
mysql>create user opensocial identified by 'opensocial';
mysql>grant all on opensocial.* to 'opensocial'@'localhost' identified by 'opensocial';

2. Install SQL objects using provided script: opensocial_mysql.sql: (be sure to return to the OS prompt again using 'mysql>quit;' and then execute:)

>mysql -u opensocial --password="opensocial" opensocial < /<path_to>/opensocial_mysql.sql

4. Check to see if database installed correctly.

>mysql -u opensocial --password="opensocial" opensocial
mysql> show tables;
Should see:
   +----------------------+
   | Tables_in_opensocial |
   +----------------------+
   | activities           | 
   | activity_media_items | 
   | activity_streams     | 
   | addresses            | 
   | movies               | 
   | music                | 
   | organizations        | 
   | people               | 
   | person_appdata       | 
   | person_friends       | 
   | quotes               | 
   | urls                 | 
   +----------------------+

Feel free to issue some more queries:

To see all people preloaded:

mysql>select os_id, name from people;

To see all activities:
mysql>select * from activities;


Congratulations! You now have OpenSocial installed on your MySQL database!

Tutorial 4: Using extended SQL Java classes within Shindig to talk to your MySQL database.

These steps largely mirror what is provided in: <shindig_sql>/sql/README.txt

Now that your MySQL database is set up with OpenSocial data, it's time to enable the SampleContainer within Shindig to be able to use the database instead of working directly with the static XML file. To that end, a set of custom Java classes have been created for this tutorial and provide a simple way to connect to MySQL instead of reading the the XML file.

The custom classes that have been added to Shindig are:

Classes that implement the opensocial interfaces:
Implements GadgetDataHandler interface:

Called by SQLDataHandler to extract state from JDBC source:


These classes have been added to the new package: org.apache.shindig.social.samplecontainer.sql

In addition to these supplemental classes, in order invoke these classes you'll have to do some minor edits to the class <shindig>/java/social-api/src/main/java/org/apache/shindig/social/SocialApiGuiceModule class to bind the SQL implementations.


Adding the new SQL implementation classes to Shindig

1. Download the custom classes from here and store this file in a temporary location.
2. In your Shindig code, cd to the directory java/gadgets/src/main/java/org/apache/shindig/social/samplecontainer
3. Create a new directory call 'sql': mkdir sql
4. Copy the java files from the downloaded Shindig_SQL bundle (<shindig_sql>/sql/*.java) into this new sql subdirectory.

Editing the existing Shindig code to use your new SQL classes

Edit the existing SocialApiGuiceModule file to invoke these new classes. 

 >cd <shindig_home>/java/social-api/src/main/java/org/apache/shindig/social/

Then in your favorite editor change the code in SocialApiGuiceModule.java:

from:

bind(PeopleService.class).to(BasicPeopleService.class);
bind(DataService.class).to(BasicDataService.class);
bind(ActivitiesService.class).to(BasicActivitiesService.class);

to: 

bind(PeopleService.class).to(SQLPeopleService.class);
bind(DataService.class).to(SQLDataService.class);
bind(ActivitiesService.class).to(SQLActivitiesService.class);

Before finishing editing, you must invoke the new datahandler (SQLDataHandler) instead of the old one (OpenSocialDataHandler). You'll also remove the StateFileHandler handler as well since it's no longer needed.

Change the code:

 @Inject
public GadgetDataHandlersProvider(OpenSocialDataHandler
  openSocialDataHandler , StateFileDataHandler stateFileHandler) {
  handlers = new ArrayList<GadgetDataHandler>();
  handlers.add(openSocialDataHandler);
  handlers.add(stateFileHandler);
}

to:

@Inject
public GadgetDataHandlersProvider(SQLDataHandler
  openSocialDataHandler ) {
  handlers = new ArrayList<GadgetDataHandler>();
  handlers.add(openSocialDataHandler);
}

 

2. Before exiting the editor, you'll need to correct the import statements in your modified SocialApiGuiceModule.java file to now refer to these new samplecontainer/sql/SQL*.java classes instead of the old ones.

For example, you'll need these imports:

  import org.apache.shindig.social.samplecontainer.sql.SQLActivitiesService;
  import org.apache.shindig.social.samplecontainer.sql.SQLDataHandler;
  import org.apache.shindig.social.samplecontainer.sql.SQLDataService;
  import org.apache.shindig.social.samplecontainer.sql.SQLPeopleService;

You can comment out or remove these imports:

//import org.apache.shindig.social.opensocial.OpenSocialDataHandler;
//import org.apache.shindig.social.samplecontainer.BasicActivitiesService;
//import org.apache.shindig.social.samplecontainer.BasicDataService;
//import org.apache.shindig.social.samplecontainer.BasicPeopleService;
//import org.apache.shindig.social.samplecontainer.StateFileDataHandler;

Save and exit.

3. In order to successfully build Shindig, you'll also need to add a MySQL dependency to the social-api project's pom.xml file. This will pull down the necessary MySQL JDBC jar file and package it with Shindig. 

  cd <shindig_home>/java/social-api

Using your favorite editor, add the following dependency to the pom.xml file:

 <dependency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
  <version>5.1.5</version>
  </dependency>

4. Finally, if needed, edit the JDBC connection information in the class file SQLDataLayer.java.

5. Once these changes are applied, you can compile and package the code by issuing the maven build commands directly in an OS shell, or from Eclipse using your new run and debug profiles.

Adding the new simplecontainer.html file to Shindig

In order to simplify understanding of how to build a front end Web page that renders an OpenSocial Gadget, a simplified version of the samplecontainer.html file is provided. It is located in the root directory of the shindig_sql bundle.

To install it to your build of Shindig, simply copy it from the shindig_sql bundle and place it in the <shindig_home>/javascript/samplecontainer directory.

Upon subsequent builds and packaging, this file will be packaged into the WAR file and will be available just as the existing samplecontainer.html file is and can be accessed at:

http://localhost:8080/gadgets/files/samplecontainer/simplecontainer.html



Testing your new Shindig w/ MySQL backend database!

Now to test your new modified Shindig, start up the Jetty server and then launch the debugger from Eclipse, or launch from the command line.

Once launched, you can use either:
http://localhost:8080/gadgets/files/samplecontainer/simplecontainer.html

or

http://localhost:8080/gadgets/files/samplecontainer/samplecontainer.html

Using the "SimpleContainer.html" Web page

samplecontainer

Incidentally: Viewing the original samplecontainer.html will still work, except you'll see an alert error upon viewing this page.
This is because this SQL modified Shindig version does not have a handler for "SET_STATE" since the data is retrieved from the database as needed.

Feel free to try out the different OpenSocial gadgets listed in the simplecontainer.html page. To see the changes in the database, open an OS shell and invoke the mysql command line prompt. As you create activities or add persistent data, you can watch the database tables get updated!

Also, feel free to use Firebug to monitor the outgoing HTTP posts and subsequent responses from the Shindig server.


This concludes the Tutorials on enabling MySQL with Shindig.