Mashup Patterns

My latest post over at SearchSOA, When Do I Get to Build My Own Portal, , describes an interesting mashup use case. We don’t often think of portals as mashups, because there is this notion that a mashup communicates information via a single point.

For example, in the classic “Show me nearby apartments on a map” mashup, the two underlying components may be craiglist and Google Maps. But you only see 1 output: A map with data points on it. A portal typically has a bunch of “little boxes” (portlets) that can be populated from a variety of different places, and they don’t necessarily interact with one another. So is it even proper to call a portal a mashup?

I think so. Although there might not be integration at the data level, the fact is the various portlets are mashed together at the presentation level. A portal provides a unified container to view disparate systems, even if the views inside that container aren’t necessarily mashed together. And the fact is that many “portal-enabling” tools that help you get content into a portal are in fact mashup products.

If you’ve read any of my previous posts, you know I am a big fan of users being able to create their own mashups inside corporate environments. I think that “self-serve IT” is the only way users will get many of the solutions they need since IT departments can’t afford to dedicate resource to every project out there. In my SearchSOA piece though, I might seem to contradict myself, since I don’t think users should build their own Portals.

My experience is that users who want a portal are really voicing a concern about information availability. If you just give them a portal framework, they will still have the same problem. You should attack the underlying issue first and expose more of the information your users want. If they then choose to build a portal with that data, so be it. But they might also build other tools (information dashboards, monitors, etc) that are actually what they really need. You don’t solve a problem by throwing a tool or a framework at it.

Fellow New York CTO Club member Dan Woods has c0-authored a book for O’Reilly titled, “APIs: A Strategy Guide”. Since APIs are the lifeblood of mashups, it seemed only appropriate to do a review. I’ve posted it over on SearchSoA:

http://searchsoa.techtarget.com/answer/Wheres-the-ROI-in-a-public-API

One issue not covered in the book (which Dan and I briefly discussed) is the resistence to API development I have seen in large organizations. For example, management will argue:

“I can’t afford to take the time to expose our team’s assets! We have to meet the requirements of our specific business users”
-or-
“I can’t risk our systems slowing down because other teams are leveraging our API”

These are comments straight out of silo-ville. This is an example of management thinking more about protecting a fiefdom of data and functionality rather than working for the good of the entire firm.

Yes – there are instances where security or “mission-critical” issues override the desire for an API. But in my experience the amount of systems that could potentially be opened up is far greater. A closed application inventory hampers innovation and -by extension- is a negative impact on the firm.

Dan’s book takes on the API issues at a non-technical level, and has been useful in helping me educate managers to see the benefits of opening up their products to outside collaborators.

For the past several months I’ve been blogging over at SearchSOA, in absence of any major advances in the world of mashups. In fact, it seemed that even as the tech industry as a whole continued to move towards the APIs that make mashups possible, it had abandoned the notion of mashups as a new development paradigm. Within the past couple of weeks there has been a veritable explosion of mashup-related announcements that demonstrate otherwise.

Item 1: Microsoft Re-Enters the World of Mashups

I totally did not see this one coming. After Microsoft shuttered Popfly, its own mashup environment, I had assumed their flirtation with mashups was over. Yet from out of nowhere comes ExcelMashup.com, a platform for mashing together excel workbooks and public APIs.  Clearly Microsoft has not abandonded mashups and now sees Excel as a means to drive their creation. I recall a 2008 Web 2.0 Expo talk given by John Musser, founder of ProgrammableWeb, where he predicted Enterprise Mashups would be the Excel of our era. In his prescient way, it looks like John might have been closer to the truth than he thought!

Item 2: An Enterprise-worthy Open Source Mashup Tool

The good news continued with the release of Convertigo’s Enterprise Mashup Server Community Edition (press release )

I have always felt that the mashup community needed to get traction with the open-source community in order to be successful in enterprise environments (I had high hopes for the Open Mashup Alliance). Obviously, people could always manually connect various APIs and code an interface around the results, but even a small amount of tooling can accelerate this process dramatically. And using a mashup product like Convertigo CE can expose this discipline to a much broader class of non-developer. Convertigo CE has 4 main functions:

• Connectors — for any SQL, REST, SOAP, RSS, ATOM or Microsoft Excel file;
• Sequencer — combines, orchestrates and exposes new REST, SOAP or JSON services;
• Gadgetizer — feed portals with services and widgets for Microsoft SharePoint, Oracle Web Center, IBM WebSphere portal or any open source portal;
• Mobilizer — build and deploy cross-platform mobile native or web applications for iOS, Android, BlackBerry and Windows Mobile.

I used an earlier version of Convertigo (along with tools from JackBe and Kapow) at a Web 2.0 presentation I gave on mashups and it was very powerful. I plan on testing the Community Edition and posting an in-depth review in the future.

Item 3: If This… Then That…

The last announcement comes in the spirit of web-based mashup creation tools like Dapper and Yahoo Pipes. “If This then That” (http://ifttt.com) is a new site that provides a way to mash services together to perform simple, specific tasks. For example, “IF it is going to rain, THEN send me an SMS”. The condition would use a weather API to check the forecast for your area, and then an SMS API to send you a notification if needed. These types of simple automated tasks are the equivalent of the parameciums of the mashup world, and yet the site is hugely popular and tens of thousands of these tasks have been created by the community. As I wrote in my post, “Dumbing-down Mashups: A Good Thing?” the spread of simpler mashup products like ifttt is ultimately what allows a more complex ecosystem to develop. Non-developers will be casually drawn to a site like ifttt at home, fall in love with it, and ultimately demand similar capabilities in their workplace.

From this USA Today article comes the story of a Johan Bollen, a professor at Indiana University who has authored a book on extracting sentiment information from Twitter.

Bollen analyzed the text of daily Twitter feeds (9.6 million in total) over a nine-month period in 2008 with two mood-tracking tools. One simply measures whether the daily tweets were positive or negative. The other tool sought to measure human mood states by categorizing tweets under six terms equated with different mood types: calm, alert, sure, vital, kind and happy.

He claims that using this technique allowed him to anticipate Dow stock prices 3-4 days later with 87% accurracy. I havent looked at the study in detail, so I don’t know if he’s measuring the overall movement of the DOW, or specific issues.

Mining Twitter sentiment seems to be all the rage. Streambase recently announced a feed based on Twitter sentiment, and Thompson Reuters’ Open Calais has offered similar features for a while now.

What’s news here is not that these techniques are being applied.. I wrote about a similar mashup last June. No; the thing I think is interesting is that you are now seeing this approach become mainstream.

I’m certainly not the first person to advodate this approach, but I gave clear examples in Mashup Patterns (back in 2009) of using this technique to get “Advance Knowledge of an Industry’s Performance” and to “Spot the Underlying Causes of Trends in the Housing Market”. Besides market performance, I wrote about how firms could use these techniques to uncover issues that might threaten the reputation of their brand (or of a competitor’s).

Between Innovation and Commoditization lies a Competitive Gap

Like most innovations, there is an “S” curve that takes you from humble beginnings to ubiquity. The gap in-between these points is where you can create a competitive advantage. I think this latest news demonstrates that Leading Indicator mashups are nearing the right side of this chart.

Firms are perpetually caught in a struggle between experimenting with new products and tools that can give them an edge in their market, and playing catch-up with companies that have proven-out new ideas and deployed them more quickly.

What’s interesting about Mashups is that while some applications of the technology are clearly moving to the commodity stage, many others are not. And “Enterprise Mashups” as a discipline unto itself is definitely still down to the right of the chart.

“Managing Innovation” may seem like an oxymoron, but firms should recognize this maturation curve and try to move items along it more quickly. This helps maximize the time they can enjoy the competitive advantages the new technology yields.

For the “Leading Indicator” pattern, I think that window is closing quickly (if it’s not already closed). But there are still large problem spaces where mashups can offer new and valuable solutions that will help you leap ahead.

Having a little trouble posting directly from my iPad (from my top-secret vacation spot lol) so I’m going to punt from my regular mashup exposition and direct readers over to my latest podcast on SearchSOA:

http://searchsoa.techtarget.com/podcast/Application-mashups-and-data-migration-in-the-cloud

It’s a short discussion of some of the risks of using external SaaS and Cloud services, and how mashups can help.