Beyond Social Media-Creating Data Models Using RoR and APIs

Wall Street, public companies, governments, privately held businesses, and start-ups are trying to find trends and indicators that they can act on today to improve operations and sales.  These groups are spending billions to glean business intelligence from vast and growing public and private data repositories.  Never before has there been so much data available nor has it been so relatively easy to access and analyze this data.  There is an incredible amount of opportunity to create value from this data.  If you can learn to create applications that utilize, sort, merge, and organize multiple data with dynamic applications like Ruby on Rails (RoR), you will never have a shortage of demand for your services.

Seize the Opportunity to Reorganize and Analyze Data Streams

I recently spoke with the co-founder and COO of an apparel company.  He told me his number one challenge was business intelligence.  To put his comments in perspective, his revenues are well into the 9 figures and yet all of their forecasting, demand planning, and replenishment is done in Excel.  He is actively looking for a better way to respond to his own data to improve day to day operations and to uncover trends in the greater marketplace.  His situation is not unique.  These two challenges are two of the largest hurdles business face.  As data increases exponentially and change occurs in increasingly shortening cycles, businesses are struggling more than ever to find actionable business intelligence in the mountain of data.  And where there is a challenge of this magnitude, opportunity abounds.

I have been a supply chain manager for nearly a decade and the only way to efficiently manage a supply chain with thousands of daily transactions is with database backed analytical software.  To realize numerous efficiencies and cost savings including batch data import, data integrity, management by exception, trend and business intelligence discovery, leveraging demand planning, inventory, and replenishment algorithms across hundreds and thousands of SKUs and multiple stocking locations, and bottom up actuals and projection reporting can only be done well and accurately with analytical software.  Excel is not scalable or flexible enough to easily update, provide management by exception, or roll up numbers elegantly or in multiple ways for analytics.

Enterprise Software Companies Pay Billions to Acquire Cloud Analytics Startups

All of the billion dollar acquisitions IBM, SAP, and Oracle have made in the cloud computing analytics space have a new perspective.  Because there is so growth in the cloud analytics space and these companies can’t meet it organically, they are gobbling cloud analytics companies.  While the enterprise ERP market is very saturated, the current market for Software as a Service (SaaS) is only $12 billion with projected growth to $21 billion by 2015.  While there is currently significant development of social SaaS sites directed at consumers, the small and medium sized business market in particular has a large runway because of the large ROI available from good analytics.  Few companies have either the budget to buy a traditional software package or the software or business expertise on staff to build their own software package to analyze their data effectively.  Even large companies are having difficulty finding software or employees who can access this data as detailed in “Fumbling Over Data.”

Companies Who Are Innovating with Data

1. Consolidate and Present existing Data in Highly Usable and Valuable Ways-Yipit, Wufoo, and MailChimp.
2. Provide Data Compilation, Analytics, Automation, and Processing-Salesforce.com, Intuit, and Freshbooks.
3. Offer New Services for Storing, Sharing, and Presenting Data in valuable and or/fun ways-Dropbox, Foursquare, and Twitter.

You can mix and match these features in any way you like to create original and compelling mashups, automation, deliverability, analytics, operational efficiency, and fun to create a ton of value.

Getting access to this data has never been easier.  There are thousands of APIs available at Programmable Web you can find to create applications.  If there isn’t an API but the data is publicly available for fair use, you can scrape it.  Or you can take your own dataset and build a custom application that is accessible over the internet or an intranet that gives access, analytics, data integrity, and much more to your internal and external customers.

You can build something like this in days or weeks if you utilize open source web application frameworks Ruby on Rails (RoR) or Django for Python.  Even if you have no background in programming or web application development, you can learn to hack together a working prototype in six months with RoR.  I started learning RoR in August very part-time while working full time and taking a college course in Statistics.  I have purchased and read multiple books, used multiple online tutorials, and worked and built one big application and several more simple applications.  Based on my trial of all of these resources, I am going to give you my ranked list of resources to use to learn RoR.  You can change the order and mix and match based on how you like to learn.

1. Programming Ruby-Called the Pickaxe book by RoR developers, it is the definitive resource for RoR.  I could not recommend it more highly.  This book has been in continuous print for 11 years.  Most of the other RoR books are either derived from this book or assume you know the knowledge  in this book.  The first edition of the Pickaxe book (2001) is available for free online here and the basics are still the same but realize many of the code samples won’t work in Rails 3.  The only caveat I offer is that if you have no prior programming or web experience, you may want to begin with “Beginning Rails 3″ below:  It is much more basic.   While anyone can learn RoR from scratch with the Pixaxe book, it jumps right into the deep end in a step by step fashion.
2. Beginning Rails 3-This is the best, basic introductory book on Rails.  It gives you simple examples or apps to create (including a Rails app in one line early in the book) and it really explains clearly how Rails works so that even a beginner can understand.
3. Web Applications with John Ousterhout-This is a free online course that provides videos, course slides, resources including a 200+ page CSS PDF , and assignments that will provide you with a working web application at the end.  This is an excellent course.
4. Rails Recipes-An amazing resource that will be in its third printing next month.  The 2nd edition is generally unusable to new programmers because it is 4 years old but I have this 3^rd edition beauty pre-ordered and can’t wait to get my hands on it in Feburary.
5. Rails Casts with Ryan Bates is an excellent resource for using gems to perform specific tasks in Ruby.  Resources on the site include screencasts and the actual code you need to implement the screencasts on your own site.
6. Learn Ruby the Hard Way-If you like getting your hands dirty with drills and learning the skills to build equations, this is a good resource to get you started.  Did I mention the entire book is posted free online.
7. Ruby Toolbox is a comprehensive catalog of Ruby and Rails plug-ins, gems, tools and resources for Ruby developers with a scoring system to help find you the best resources for your project.
8. One of the most interesting online Tutorials I’ve found is written by Dan Nguyen.  While it is still rough I like how  documents his ongoing scraping, compiling, data analytics on his projects page.
9. Ruby on Rails Tutorial-Learn Rails by Example by Michael Hartl-I bought the physical book but ended up using the free online version much more because it was much more current.  While this is a good resource and has more code examples and detailed descriptions for an end to end app than anything I’ve seen, it isn’t the best first resource to use.  I used the book to learn Ruby because I read inaccurate reviews of the book online.  Some of my issues with using this book as a first resource:

• The RoR installation instructions were useless for Windows users and there are countless lines of code out of date i.e. they don’t work.  So you end up spending hours and hours over the course of the book trying to figure out if you made a mistake of if the code in the book is just incorrect such as using specific versions of gems.  No, you shouldn’t use specific versions but the book says you should.  The CSS folder was in a completely different area than specified, and much more.
• While test driven development (TDD) is the best way to perform application development, it was extremely unwieldy to implement as a newbie and after several chapters I couldn’t get some of the tests to work anymore.  I spent hours hacking at them but I finally I gave up.  If you were to use this resource as a first book to learn Ruby, I would skip TDD as it is detrimental to getting your application running and the code in the book is incorrect in too many areas.
• Because far too much coverage is give to TDD, Active Record and other critical components of how and why RoR works are either omitted or not covered adequately.
• You build a working version of Twitter.  If you want to build a social media app, then this is a great book.  If you want to work with APIs and perform analytics on data then there are glaring omissions.
  I recommend the book because it is both free and has extensive coverage of security and authentication.  But I recommend that it be a resource only after you have learned some RoR.

Ruby on Rails is an amazing open source, web application framework because you can build a simple working app in minutes and hours.  Utilizing the principles of Lean Startup and 4 Steps to Epiphany, you can quickly build and test and validate new business and data models.  This model employs the minimum viable product (MVP) strategy of building the minimum feature set to test with real users.  After test the initial test you begin a cycle of quick iterations while considering a final take to market or discontinuation decision at the end of a cycle.

Quick and Easy RoR Installation

New Macs come with Ruby preinstalled but you will probably need to update to Rails 3.1.  Since you are using Ruby for the first time, I recommend using the  “gem update rails” command.  Windows users should use the simple bundle install available at RailsInstaller.  Take from someone who struggled through hours of a manual piecemeal setup:  You will save a lot of frustration and time if you use the RailsInstaller package.  There are no downsides for a newbie using the package installer.

You will also need to use a good text editor to develop.  There are many available but I use Sublime Text and love it.  It has a free unlimited, and full featured evaluation period.

If you can find a friend, live teacher, or a group of people to bounce ideas off that would be the best case scenario.  Living Social (through  Hungry Academy) was offering to pay 25 people with no RoR experience to learn RoR for 5 months and then offer them a a job at the end of the training.  Yes, Rails developers are in that much demand that they have to train their own from scratch.  My future sister-in-law is the VP of Operations at Plum District (a daily deals site catering to women) whose company also uses RoR and she confirmed that developers straight out of college are getting 6 figures+ to start.  Learning RoR will give you the ability to create and utilize existing data to create new ways to see and interact with the world and when done correctly build a ton of value.