Review of An Introduction to Machine Learning with Web Data by Hilary Mason

An Introduction to Machine Learning with Web Data by Hilary Mason; O'Reilly Media.

The video itself is presented in five sections: (1) Introduction, (2) Classifying Web Documents - The Theory, (3) Classifying Web Documents - The Code, (4) Clustering , Recommendations, and Probability, and (5) Conclusion. In short, the video Hilary uses web based data to show the audience how to work with data to solve problems you may have by using basic machine learning techniques. The video is particularly directed at programmers who do not have statistical training.  

The viewer will sit with a group of a few other students and feel the imtimate setting of a small classroom. For myself, a video where you can re-watch segments, stop the video to reference a suggested resource, or pause to experiment with a variant of the code is both helpful and handy. For example, in the introduction, Hilary references a link (http://bit.ly/9RYQEF) that explans the concept of "data science." At that point, I paused the video and browsed to the link and found it quiet informative. By the second section, Classifying Web Documents - The Theory, the audience is gently taken into statistical techniques such as naive bayes and shown a step-by-step approach in how the math is applied. In the  Classifying Web Documents - The Code, the participant utilizes python code and the New York Time API to classify words from the New York Times web site. Within the Clustering, Recommendations, and Probability video the viewer is taken through code that demonstrates how to take data with which little is known and learn from the clustering results. Finally, the conclusion section deals briefly with the concepts of probability and then reviews the entire sessions content. 

While being able to navigate around in python is beneficial and by following along with the running of the code one can learn and retain more information, the participant can just view the video content as both the code and concepts are displayed and explained. What is nice is Hilary provides the code used in the video from her Git repository at https://github.com/hmason/ml_class. If the viewer wants to participate she will need to make sure that they have the proper python modules installed. 

In conclusion, the software developer that has little more than the required stats college class would do well to purchase this video. Seeing the actual application of code to the basic statistical algorithms is extremely informative and applicable in various problem domains.

Posted via email from Mark's Musings

Playing my Daily Healthmonth Turn and to Posting to 750words via my Droid

For over the past three months I have been using a few sites that help me to reach my personal goals within a "gamified" process.
 
The first site, healthmonth, is according to its creator Buster Benson, "an online adaptation of a game some friends used to play every year right after New Years."
 
The second, 750words also created by Benson, is a site that encourages the daily writing of...you guessed it, 750 words. The site provides an online, private space to compose an essay, journal, or just write what is on your mind. As you type, the site saves your work, tracks and displays your word count, and lets you know when you've passed the coveted 750 word mark. The fun aspect is in the fact that 750words keeps a score card of points for the days you enteract with the site. 750words also keeps track of the consecutive number of days (or streaks that) you submit completed posts. You can also see how others are doing point and streak-wise to fuel your competitive nature.
 
My wife and I were away from home for a few days and I was not certain if we would have reliable wifi access to play my daily healthmonth turns and to post to 750words to keep my 100 day plus streak alive. So, I wanted to find away to use my Droid to utilize the sites. 

 

I was able to play my turn on the healthmonth site easily with the Droid via the native web browser. However, posting to 750words was a little more tricky. While I could type a 750 word submission with the droid keyboard, I found it more feasible to create the post on my desktop, laptop, or netbook and import them into the Droid via a text app I use, AK Notepad. Then, once in AK Notepad I could copy and paste the text into the 750words site via the web browser. Note that any text editor should work in a similar manner in your Smart Phone. Specifically, here are the steps I took:
 

First you have to import your 750word posting into the AK Notepad. To import text (.txt) files into the Andriod AK Notepad app you will need to do the following:

1) Have a folder that is created during an export. You will need to use this folder as AK Notepad will not see folders created manually. If it is not there, create a directory to use by selecting and holding down a note from the list of notes, then select the "Export to text file" menu option.

2) With the droid connected to a computer via a USB cable place the files that you desire to import into AK Notepad in the directory in step 1 above.

3) Disconnect the USB cable and opened the app on the phone.

4) Select "Menu" and then "Settings"

5) Under Import/Export section select "Import notes" and then browse to the folder in step 1 above. NOTE: each file in the selected folder will be imported.

6) Select and hold the folder and then from the popup menu select "Use this folder." This should import all the files in the folder.

Next, open the note that you want to post on the 750words site and select and hold down the text until the popup menu is displayed and select the "Copy all" option. Now that the text is copied to memory, open the web browser, browse to the 750words site, and paste the text into the site page. The site will auto-save the entry.

 

Now, if I plan to be away from reliabe internet access I can still play my daily healthmonth turn and to post to 750words to keep my streak alive.

 

Posted via email from Mark's Musings

Not Being the Expert

It is interesting to note as you move throughout your day, we are fearful to appear as not being the expert. Why? Perhaps there is an expectation that you are to be the person who is considered the expert in this or that? Maybe you have a self-image that you are the go-to-person on this subject or that topic?
 
Regardless, why is it OK to not be the expert? Isn't there value in just being the person that is trying to learn as much as he/she can and enjoying the journey? What is more important, being seen as the person that is the expert or being known as the individual that is open to learning, realizes that there is always more to understand, and is known to be gracious to others who lack their level of knowledge? I think the latter.

Posted via email from Mark's Musings

Where's that taboo when I need it? To Social Media I go....

Historically, communities with their various cultural taboos, acted as societal controls and helped its citizens curb harmful behaviors. Granted, while come taboos were used as instruments of oppression, others did and do help one restrain oneself unto positive outcomes.

In the New York Times article by Stephanie Rosenbloom, I Resolve. World, Don’t Fail Me Now, Rosenbloom deal with New Year's resolutions:

Certainly, you don’t need cyberspace to share your resolutions. But the potential humiliation of failure is more potent online. Social networking sites also enable you to formalize and, in many cases, regularly track your goals, which makes it harder to blow them off. Some sites reward you with a badge on your Facebook page or a congratulatory message. Nearly every site makes it easy to tell friends about your ambitions as well as to help find strangers who share your goals. The latter is useful if you don’t want to discuss your resolutions with friends and family.

A site that I started using recently, 750words, has assisted me in journaling daily. The site, create by Buster Benson offers stats and analysis of one's writing. This feature is what drew me to use the site as I am on a year long personal data collection and analysis venture to test the value of personal data analytics. In addition to the 750word site, starting in March I will be using another site created by Benson, HealthMonth.com. There, one can set up rules to help change behavior in either a public or private manner. This site, along with others on the web, act as both a daily motivator and reminder of the goals.

In summary, the visible nature of these sites assist with self-imposed, virtual taboos that help motivate me.

Posted via email from Mark's Musings

The Future of Business Intelligence--Everyone a Data Analyst?

I follow a local LinkedIn group, Cincinnati Regional Business Intelligence Networking Group, where the question “Where do you think the future of Business Intelligence is headed?” was posted. My response was:

Personally, in addition to the normal predictive data analysis, I see a real strength is to have everyone that deals with data, the administrative assistant to the CEO, understanding how to properly interpret the data that they deal with. Therefore, everyone in the organization has the ability to provide insight into better decision making at their level.

  

Another member of the group then asked in response, “Mark, what data tools do you see being developed or expanded upon to reach this goal? I believe that total company comprehension of corporate data could also be the way of the future.”

  

This caused me to further reflect and consider how enterprise wide data analysis, by most if not all employees, might look. I replied:

Foremost, standard data understanding needs to be basic training for employees. For example, training them how to: disassemble complex problems in order to better evaluate and solve the issues, develop hypothesis and experiment with the data to find unknown causal relationships, use of current common tools such as spread sheet apps (for example, using the Solver function for data optimization) and free tools such as R, understanding how to use data visualization, understanding and using Bayesian rules, etc.

    

Along with the standard extract, transform and load (ETL) and reporting tools, apps that can assist in the above would be very valuable. Given that the above are really processes that can be applied to various situations, tools that shorten the steps is what I would consider. To be honest, I am not sure that the tools would be as valuable as training employees to understand and do basic data analysis with the data they work with.

  

The consideration of the importance of understanding data and how to interpret it came from an interesting and informative talk by Danah Boyd at Gov 2.0 entitled "Transparency Is Not Enough." The talk is available at http://goo.gl/GraT

  

Per Boyd, "The #1 goal of transparency is to empower people, to give them an opportunity to be informed as citizens, to allow them to be a check to power. But when those in power release data in a way that allows them to maintain power, we've got a huge problem....We must also help people develop the skills to interpret data.....Information is most useful to people when they have the skills to interpret it....Because of this, making information available alone is not the great democratizer. It must be coupled with enabling people to have the skills to interpret it."

  

Applying this premise to the Business Intelligence arena, having access to terabytes of data in and of itself does not result in helping the organization make better decisions. “It must be coupled with enabling people to have the skills to interpret it.”

Posted via email from Mark's Musings

Lending Kindle eBooks Now Enabled

You can now lend Kindle eBooks. It is still controlled by Digital Rights Management set by the publisher, etc. but it is a next step!

Posted via email from Mark's Musings

HTML5 Canvas Tag Basic Example II - Gradients and Images

Per my previous post, working with the canvas tag in HTML5, here is some more code with gradients and an image.
  
Thanks again to Mark Pilgrim and his wonderful Dive into HTML5 site.
  
Enjoy: 
  

Loving Posterous' use of Github Gist to post code

Loving Posterous' use of Github Gist to post code in your Posterous blog. Just paste a Github Gist URL on its own line and Posterous pulls in the contents automatically, even for your RSS feed and autopost sites. 

Posted via email from Mark's Musings

HTML5 Canvas Tag Basic Example

It has been a while since I posted any example code. Was messing with the canvas tag in HTML5 and thought I would post it up.
  

Thanks to Mark Pilgrim and his wonderful Dive into HTML5 site, I was working through the canvas tag page and put this together (only got as far as the gradient section though). Enjoy:

"How good software makes us stupid"-You're still smart, what does that say about your software?

In his post to BBC News Technology, Dave Lee writes about "How good software makes us stupid." Like it or not, the research outcomes consistently show that the tools we use to learn also shape our ability to learn. The premise of the article is that good software makes things easier and us more stupid.
 
In retrospect, I know, hang around, and work around a lot of smart people. Therefore, their software must not be that good! Gaaa! I develop software! Must get better at making people dumber...or I a mean make things easier for them!

Posted via email from Mark's Musings

RSS - The Really Unknown Syndication

At work, on the corporate intranet site, there is a daily survey question. Today's question was, "Do you use an RSS feed to capture news or similar info from websites?" The possible responses were:
Yes – I like that it sends me news I request. 
Yes – I’m just learning how this works.
No – I’ve never used it. 
No – I don’t know what that is.
 
With most survey components, you do not see the tabulated results until after you answer the question. Before I answered I thought that the response rate would be about 60% answering either "Yes – I like that it sends me news I request" or "Yes – I’m just learning how this works." Wow, was I off. Here are the actual results as of the time of this post:
The "Yes – I like that it sends me news I request" had 719 out of 13,856 or  5% of total.
The "Yes – I’m just learning how this works" received 300 responses out of 13,856 for a 2% affirmative rate.
The "No – I’ve never used it" got a 2,766 out of 13,856 or 20% of the total.
Finally, the "No – I don’t know what that is" answer received a whopping 10,071 out of 13,856 for a 73% response rate.
 
Again, wow! 93% of the respondents are not using RSS even though it has been broadly available since 2005. Given that the RSS acronym means Really Simple Syndication there are a good many of my fellow employees that have yet to discover the convenience and pleasure of using RSS to gather and consume their news and information.

Posted via email from Mark's Musings

Wired Mag is bumming me also!

From his August 23, 2010 blog entry, Nicolas Carr states that "he is bummed." Why? Because of Wired magazine's statements over the last 5 years:
 
Wired magazine cover story, August 2005: "We Are the Web"
Wired magazine cover story, September 2010: "The Web Is Dead"
Unavoidable conclusion: "We Are Dead"
 
He is right. If we are something and that something is dead, then irresistibly, we are dead. Wow! I could easily become bummed too!

Posted via email from Mark's Musings

The Citizen Developer

I have been thinking about the rise of user friendly development tools. Recently, Google released the Android App Inventor tool which according to Google, "App Inventor requires NO programming knowledge. This is because instead of writing code, you visually design the way the app looks and use blocks to specify the app's behavior."
 
In addition to the Android App Inventor, Microsoft as released their beta version of WebMatrix, which "is for developers, students, or just about anyone who just wants a small and simple way to build Web sites."
 
So I did a search for the phrase "Citizen Programmer" or "Citizen Developer" to see what was out there. I found where last October, Gartner Group posted an article where they shared that by 2014, 25% of business applications will be done by these citizen developers. Interesting! The article defines a citizen developer as "a user operating outside of the scope of enterprise IT and its governance who creates new business applications for consumption by others either from scratch or by composition."
 
Historically, CASE tools were to enable the business person to do basic application development. In fact the programming language COBOL was meant to provide that ease of use and comprehension given its verbose and common-language structure.
 
I consider myself to be an example of a Citizen Developer as my educational background shows that you do not have to have formal development training to be a software developer. Since I received a B.S. in Education from Ohio University in 1993, I have basically learned to program by taking the time to learn to think computationally while learning and using the various interpreted and compiled languages and database systems.

The pattern in the past has been that tools that were marketed to enable business line personnel to create business applications actually required some development skills (such as Microsoft's WebMatrix discussed above). The result was that the tool was moved into the development teams as another tool in the software programming arsenal, leaving the users still depending on the software team to produce the programs.

However, it is my hope that tools such as Android App Inventor will proliferate and result in the growth of the Citizen Developer.
 

Posted via email from Mark's Musings

Lions, tigers, tracking, oh my!

Just looked at a Wall Street Journal Blog article What They Know. While I agree that apps that record and retrieve keystrokes are evil, in my view, most tracking is for data purposes to better serve customers with more relevant advertisements. Agreed?

Posted via email from Mark's Musings

Time...between compiles

Remember in the first Jurassic Park movie, Dennis Nedry the programmer who attempted to sell/smuggle dino-embryos off the island? He stated that it would take several minutes for the section of the software that he had just added code to compile, and that certain systems would be down during this compile. It was during this gap in time that he attempted to smuggle the embryos.
 
Anyway, I often have 5 minutes here, 15 minutes there waiting for compiles to complete. During these times I refuse to be involved in illegal activity! My employer will be glad to know that I am sure. But seriously, I try to get other things done such as planning the next section of code to write, consider more optimal database queries, and complete emails, etc in that time.
 
The point is that I have to plan the break or I catch myself surfing the net and wasting time. What do you do with your time in between tasks to stay focused? A list of items to complete? Does your type of work govern the web and flow of the day for you?
 
Oh... the build is complete...got to get going...bye!
 

Posted via email from Mark's Musings

Future of Tech: Mobile or the Living Room?

Was listening to the Pipeline podcast, episode 20 where "Dan Benjamin interviews Clayton Morris, self-described casual geek and anchor of Fox and Friends about inspiration and creativity, authenticity online and on TV, the way we as viewers are changing the shape of media, pursuing your dreams, balancing work and personal life, and more."
 
Towards the end of the show the discussion arose as to where the future of technology would be focused. The two spaces discussed were the mobile space and the living room. I aways thought that while mobile and home based technologies would have there own areas of products and services, the two would converge together into a "personal, electronically assisted living" that was not restricted by location but existed and functioned in your home as well as transportation.
 
We shall see.

Posted via email from Mark's Musings

Never Trust a Programmer

"Programming is a bit of mystery to people. It breeds a lot of distrust in an organization. In general, when you don’t understand what it takes to make something, anything seems plausible. You might think a house can be built in couple of weeks...." Good stuff! The rest here.
 

Posted via email from Mark's Musings

Who is the "Bitsmith" in your domain?

Harvard Business Review's Management Tip of the Day today (Thursday, July 8, 2010) is Get Yourself a Bitsmith. A Bitsmith is defined as, "a person who understands both the work content and the tools needed to support the job."
 
One thing that organizations need to understand is that IT is more than a group of code and reboot monkeys that can be simply plugged into a position like a warm body in the Matrix. Successful IT in any company understands the objectives of the business and what it needs to do to see it happen. "By understanding the domain and the technology, bitsmiths can quickly take an idea from concept to implementation, speeding up productivity. Find someone on your team or bring someone in from IT who can serve as your bitsmith."

Posted via email from Mark's Musings

Ubuntu Lucid Lynx DVD Watching...ba da bing, ba da boom

Got my new Rush: Beyond the Lighted Stage movie DVD the other day. I wanted to watch it on my Ubuntu Desktop. Alas, I did not have the proper libraries loaded. Solution, query the smart and generous Ubuntu community. A quick search yielded How to play video dvd Movies in Ubnutu 10.04 Lucid Lynx.
 
In short, do the following from the terminal prompt:

$ sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list

$ sudo apt-get --quiet update

$ sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring

$ sudo apt-get --quiet update

 
And, ba da bing, ba da boom...I was watching the DVD on my Ubuntu Desktop.

Posted via email from Mark's Musings

Free Ubuntu 10.04 Manual

For those of you who are using Ubnutu Lucid Lynx (10.04) or are at least considering it, check out the free Ubuntu Manual. The Ubuntu Manual is available in many languages! Very nice!
 

Posted via email from Mark's Musings