Train, Model, Action! Using machine learning to create hollywood block busters.

August 13, 2013 at 1:20 AMBuddy James
Should machine learning be used to make decisions about movie scripts? You heard it right.  Machine learning has been used in many applications in the past few years from suggesting products on to recommending movies on  These algorithms are complex to the core, however, at a high level they all work basically the same.  You provide historical data to be "learned" by the algorithms, and then you test the model to make a prediction based on the model.  These methods have proved to work exceptionally well in finding patterns in user's choices and behavior.   But the big question is, what happens when you apply these methods to things that aren't focused on a particular person?  I feel confident that if an algorithm has enough data consisting of movies that I've seen, it should be able to provide a recommendation of other movies that I may like.  But what about analyzing data on previous block buster films?  Just because these movies did well at the box office, does that mean that a computer program and construct an award winning, enjoyable movie?  I have my doubts about this but at the same time, I believe this stems from a fundamental issue that man kind will have to face in this age of Big Data.  The more that we take away the human element from decision making, the more people will begin to worry until the we see the proof that the data is enough to predict a great story from a mediocre, redundant one. What do you think?  Is this a good use of technology, or will this create lazy producers and predictable film scripts? Here is a link to the original story  Solving Equation of a Hit Film Script, With Data

Posted in: Analytics | machine learning | Entertainment | You be the judge

Tags: , , ,

RapidMiner tips and tricks #1 How to use SQL Server named instances with RapidMiner Read/Write database operators

July 14, 2013 at 7:29 PMBuddy James
 Tips and tricks. Tip #1 How to use SQL Server named instances with RapidMiner Read/Write to database operators Hello and welcome to my first of many tips and tricks for RapidMiner.  If you are unfamiliar with RapidMiner, it's a Open Source Java based data mining solution.  You can visit the official RapidMiner website by clicking here.  My plan is to write a short article to provide solutions to problems that I encounter as I learn more about this awesome application.   RapidMiner and database connectivity There are many operators in RapidMiner that take input data sets and generate models for prediction and analysis.  Often, you will want to write the result set of the model to a database.  To do this you use the "Write Database" operator. I was using RapidMiner for web mining by way of the Crawl Web operator.  The Example set output of the Crawl Web operator was connected to the input of the Write Database operator.  At the time I was using a SQL Server database that I pay for through my web hosting account.  Just like most everything in RapidMiner, the setup was easy and worked like a charm.  My database size quota was 200MB with my current hosting plan and it became apparent to me that I would quickly run out of space.  As such, I decided to use the local SQL Express 2012 named instanced on my machine.  This is where the problem was introduced.  I couldn't figure out how to successfully setup the database connection in RapidMiner.   RapidMiner, Named Instances, and Integrated Security The issues that I encountered when trying to setup my local SQL Server 2012 named instanced were as follows: If I used the named instance for the server name(localhost\SQLExpress), I was unable to connect.  I didn't encounter this problem with my hosting server's database because it was a direct hostname (  There was no instance name and so the configuration was easy. I wasn't sure how to specify integrated security as this is something that you usually specify in the connection string.  I didn't encounter this problem either using my hosting database server because I was given a user name and password to connect to the server. After some research and banging my head against my laptop, I finally figured out the resolution to my problems and I'm here to save someone else the headache. For the named instance issue, there is a trick that is not readily apparent to get this to work.  You set your database server name as per usual, in my case, localhost, however, when you specify the database name, you include a semicolon (;) followed by instance=<instance name>.  So for my local server instance (localhost\sqlexpress), I set the Host value to localhost and the Database scheme value to mydatabasename;instance=sqlexpress .   As far as the integrated security requirement, all you need to do is make sure that you have the latest JTDS SQL Server driver from here.  Once you download the zip file, you'll need to extract the file\x86\SSO\ntlmauth.dll and place it in your windows\system32 directory.  This will insure that you have the driver with the capabilities of using the integrated security.  Once this file is in place, you simply leave the username and password values blank. Here is a screen shot of the Manage Database Connections window in RapidMiner for your reference.   Well that about wraps it up.  Please leave a comment if you have any questions. Until next time, Buddy James

Posted in: Analytics | BI | Data mining | RapidMiner | SQL Server | Tutorial

Tags: , , , ,