Hans Rosling: health insights, presentation excellence

Insights - wisdom and knowledge - are precious.  Too often information is lacking context, or context is lacking information.  News media is particularly guilty of this; most reports give scant weighting to the why over the what, and the event becomes mere spectacle.

Hans Rosling is a Swedish professor of global health.  I stumbled across his presentations in the context of software tools, but found myself riveted by Rosling's ability to communicate on his subject matter - something he brings alive, even for those who may not have an immediate interest.

Those presentations, available on the website of the excellent organisation TED (devoted to "ideas worth spreading"), are every one of them worth watching: entertaining yet full of information and insight.

In the first presentation in the above series, Rosling's discussion revolves around four dimensions: time, health, wealth, and location (region/country).  He gives his audience a good understanding of how the other three factors affect health outcomes, yet argues cogently for a more complex perspective on factors that affect health.

His second presentation is briefer, but includes an impressive feat which might seem gratuitous, yet he does it with purpose: to illustrate his point on achieving better health outcomes that "the seemingly impossible is possible".  I won't divulge the climax: something that has to be seen for itself.


Rosling is, first and foremost, a Subject Matter Expert.  But crucially he is a very effective communicator.  He presents with knowledge and clarity, in a way that engages the audience.  Part of the 'wow' factor lies in the fluid use of  the presentation software he uses, which leaves the world's Powerpoints for dead.  And if you explore the links, you'll find out that that software was originally developed by Rosling's foundation, no doubt to achieve the sort of communication at which Rosling excels.

Business Intelligence on the cheap

Here's a pointer to this technical post.

Tech: Data Provisioning

I grant that some of my technical pieces may be rather esoteric for a general audience. So I have sequestered them off to a separate blog for my professional pursuits. You'll find the first post here, on a neologistic use of the term Data Provisioning for a successor model to the Enterprise Data Warehouse.

Business Intelligence trends: HP's take

Buried amidst the consolidation of business intelligence (software) vendors, HP appears to base its solution on Knightsbridge, which it purchased in 2007.

It can't be too surprising if HP (and, previously Knightsbridge) aren't familiar names in the BI field. They do not figure regularly in industry reports and forums. HP's profile is quite low - so low, in fact, that they're not even mentioned in Wikipedia's list of BI tools. I expect their marketing strategy is limited to supplementing their provision of whole-of-business solutions to the marketplace: when selling to enterprises, something akin to "Oh, and we also supply business intelligence solutions - and consulting services. No need to go to the market for that". (However, for a counter-view on HP's BI profile, see this blog by Shawn Rogers.)


Notwithstanding, they recently released their take on BI trends for 2009, as follows.

Trend #1: Consumerisation of IT
In effect, business to adopt consumer-level technologies such as facebook and twitter. BI-specific effects in collaboration, visualisations, new data sources.

Trend #2: Post-Western tech economy
"Emerging regions" will transition from being simply "suppliers of low-cost talent" to being developers of best practice and global standard-setting consumers. Benefits to BI in terms of innovation - in analytics, unstructured data, etc.

Trend #3: BI importance increases; data governance and quality to become critical
So says HP - but see BI Survey's comments on less than expected adoption. However, it's to be expected that in hard times businesses would turn to BI for efficiency gains.

Trend #4: BI Buyers more scrutinising
- linking projects to business outcomes.

Trend #5: Market demands lower BI complexity
Commodified BI, standards for data marts. Here, HP contradicts its own earlier comment about SaaS/Cloud issues not yet figuring prominently in BI. They note it under "consideration".


Trend #6: Analytics moves to the front office; business users get greater sophistication
- including data modelling in the hands of business managers - scary!

Trend #7: Data integration increases in importance
Consolidating data from traditionally disparate sources; increasing focus on enterprise-level information management strategies.

Trend #8: A blurring between data warehouses and operations systems/data
A need for realtime operational reporting - enter, data hubs, Enterprise Service Buses, etc.

Trend #9: Convergence of structured/unstructured data
- this is a brave call, as business are only just starting coming to grips with the unstructured data buried in documents, notes, etc. My call is that it will be some time before unstructured data sees much effective use, let alone convergence.

Trend #10: CEP (Complex Event Processing) comes of age
- this seems to be an amalgam of alerting and data mining, nearing real-time.

As they later note: "To make the most of BI, first you need to get the data right". Much as that sounds a truism, it is a point that needs to be hammered at every opportunity, from data modelling to quality/management/governance.

HP's full report here.

BI Survey 8: business intelligence trends

Out now is the new edition of the BI Survey - once known as the OLAP Survey, now up to edition 8, covering 2008.

As a specific term, Business Intelligence is more widely known than Online Analytical Processing, but the Survey found that even on this, the software vendors are over-optimistic. They estimate about 14% of employees (of a "typical" organisation) use BI tools, but the Survey finds a reality closer to 8%.

Other points noted in the preview copy I have:
- BI implementations that follow a competitive evaluation of different vendor toolsets are more successful in every way - however, the Survey shows a slight drop in the number of competitive evaluations. My comment: this is not fully surprising, and I can give a couple of reasons: a) the competitive process is seen as too hard, costly, or lengthy; and b) encumbents with BI skills will inevitably veer towards the toolsets with which they are most familiar;

- the most common issue with BI tools is (query) performance - although vendors have a rosier view than business users. ;

- small software vendors provide better support than large ones.


The spate of vendor amalgamations is somewhat represented in the survey. Gone are explicit listings of tools such as ProClarity and Brio (subsequently Hyperion Intelligence), but still represented are TM1 (as Cognos TM1) and Hyperion (via Essbase - all their other tools were subsumed in the Oracle toolset).


Another blogger, Chris Webb, has seen the full report, and makes some comments here.

One interesting point he notes is about MicroSoft's marketing strategies vis-a-vis different product offering. He strongly suggests that MicroSoft is leveraging their BI tools (SQL Server services SSAS, SSIS and SSRS) to promote greater adoption of the MS Office suite - to the ultimate detriment of its BI tools. In effect, their BI toolset is less than could be in the interests of encouraging greater sales of MS Office. This is of note because MicroSoft are increasingly pushing their BI solutions out to Office products. Client-end tools in Excel were only the start.

The degradation of outcomes for one product in favour of another is certainly plausible for an organisation selling multiple products, especially if their marketing strategies are sufficiently sophisticated. However, I would in turn suggest that greater adoption of BI tools is hampered by specialist knowledge requirements, and that the more users are able to work within a familiar paradigm, the easier they find it to use the tools.

On the other hand, MicroSoft's BI solutions have always been far too heavily geared to the technical environment to the detriment of business-level stakeholders, and I'm not yet convinced their Office-related tools are a good fix.

Webb's other notable reading of BI Survey 8 is that MicroSoft's BI products come squarely in the middle of all rankings (eg usability, scalability, etc etc). This is not surprising. While they are capable of producing competent product, I don't think MS's BI products are stellar in the scheme of things - ubiquity is the word that springs closer to mind, since they come free with all enterprise edition databases. Thus MicroSoft will perpetually remain too big to ignore in the BI marketplace.

Business Intelligence 2.0

A provocative article on B.I. dating back a year and a half makes an interesting juxtaposition between the development of Web 2.0 and business intelligence. Or at least, where he thinks B.I. will be. Or ought to be.


It's by a bloke called Neil Raden, and you can read it here. There are limits to the analogy, of course. Web 2.0 is about a ramping up due to interactivity, collaboration. Business Intelligence 2.0, from what the author says, is simply about what B.I. would be like if the professionals and software developers just got on with the job and came up with more sophisticated tools.

At first glance, I thought it was pie in the sky. Second glance, it all made sense in a general way: the toolset is not what it should be in an ideal world, and there's no reason the B.I. space shouldn't go that way. But in reality, such evolution would take quite some time. And in fact I query how susceptible to simplification something as mercurial as data - and people's approach to it - can be. Raden acknowledges here the value of the semantic web project, and that may actually be whence the best initiatives emerge.

Still, it's challenging, and worth reading for those in the industry, or who are touched by the issue.

Oracle swallows Hyperion

In a poetic mood, perhaps, Oracle Corporation is taking over Hyperion Solutions.

But it's a hard fit to make sense of it in the context of Greek mythology. The Delphic Oracle was a divine presence; Hyperion was a god, father of Helios the sun, and the twain never met.

Oracle is best known as a database company, maker of the eponymous Database Management System. However, it has been on the warpath in recent years, expanding its reach into all areas of database and business software.

Hyperion is a software company, focused on Business Performance Management and Business Intelligence products.

A few short years agon, Hyperion took over the Business Intelligence company Brio Software.

Brio once had a nifty little desktop client (back in version 6) called Explorer. It was particularly good for two reasons.

First, it could actually operate standalone. It didn’t need server-side software to dish up databases or functionality. It could provide querying, analysis and reporting on anything you could configure via ODBC (or import). This could be a database (such as Oracle), but it could also work on an excel file – or even a text file.

Second, you could create a .bqy file that could operate as a standalone application with data behind it. Once the file was refreshed during the day, the sales manager could take it home over the weekend, and query and analyse that data, if necessary via a high-level user interface that duplicated the functionality seen in web browsers (eg listboxes, dropdown menus).

I don’t know if the new version of Brio (now called Hyperion Intelligence) has that functionality. But I know Oracle bought Hyperion for its Business Performance Management capabilities, not Business Intelligent. Oracle already has its BI tools – such as they are – and may not be interested in the Brio tools it acquired in the process of swallowing Hyperion.

And right now, I’m longing for some decent standalone software that I can use as a database query, analysis and reporting tool. And I’m remembering HP who swallowed Compaq who swallowed Digital, who had a few nifty products at the time.

It's like a Data Warehousing genie

I met with someone this week who was working on a large Data Warehouse/Business Intelligence project.

At the large enterprise level, it's hard to get close to the ideal in a Data Warehouse. The task is positively Hurculean, to schedule feeds from a large number of data sources, cleanse the data, integrate it, and store it. The data integration task alone would be daunting, and could conceivably take forever. Why? Because if you do the right thing and involve all interested business units, it could be a neverending talkfest. Especially if there are competing interests. And much as people can talk about eliminating stovepipes (around the various business areas), there is always going to be competing interests, people concerned with building, maintaining, expanding their domains.

So I was unsurprised to hear there was a certain amount of descoping. Not enough, to my mind, to match the realities of a large business environment.

But on the positive side, ever business or technical issue I raised (on a broad level) was matched with a plan of action, or at least a firm awareness of the issue. So I'm happy to let them run with it – I'm not an enterprise architect.

But on the even more positive side, look what they were promising the data user! An array of different tools to access the DW, from (almost raw) SQL coding to (vanilla) data mining tools, and various options in between. But wait – that's not all. They were also offering the data user access to several levels of data, from modelled representations to raw tables, even as far as allowing access to the [ETL] staging area.

What does that mean? On one level, if the tool you're using doesn't give you the productivity you're after, you can go for something more gui-based. Conversely, if it doesn't give you the level of control you're after, you can go more towards SQL. And if you're not confident about the data you're receiving, you can drill behind the model, to the tables, then to the staging data.

What better access could you ask for? It's like wish fulfilment.

Prototypes are a matter of months away. I'm keeping my fingers crossed.

Tech: MicroStrategy - BI strengths and weaknesses

I attended a MicroStrategy presentation a few days ago. It was a good opportunity to position it against other players in the Business Intelligence market. There were some surprises, and some to-be-expecteds.

MicroStrategy is a relatively small player. It would seem to be playing into niches - they pretty much said as much - which is surprising giving its strengths.

They position their competitive advantage as being performance and scalability - which should make the product industrial strength, up against the biggest players (Cognos, Business Objects, Hyperion/Brio, Microsoft). In fact, the stats they give suggest as much, with average data volumes and seats being significantly ahead of the major players. In particular, customer loyalty is very strong, by their measures.

The architecture is good, also industrial strength. Object reuse is extensive, they do caching at multiple levels, connect to multiple (heterogeneous) data sources, and - something that impressed me - their use of SQL is multi-pass. For my money this would mean their generated SQL is less convoluted to achieve the same results, and MicroStrategy's also capable of more complex querying.

Of course, they have industry standard features such as web interfaces, zero-footprint clients, and extensive use of metadata - albeit their proprietary implementation.

They're also focused on ROLAP, which is a plus for people like me who want to get directly stuck into the database, rather than work through manufactured cubes.

Where's the rub? I was betting that implementation and use would be consequentially more difficult. And this was confirmed by another observer at the presentation.

Thus, they wouldn't be a casual investment. But then, is any BI?, you ask.

It all begs the question why they aren't bigger in the market place, despite any usability and implementation issues. But markets are not axiomatically logical.

I look forward to any further critical analysis I can lay my hands on.

Tech: Don't waste your time on a Data warehouse!

Dratz did an interesting post on data warehousing.

In effect: why bother?

In concept, a data warehouse is laudible: one place where you can store all the organisation's data, integrated across business units...

But why? Do you have a business case for it? A good reason you can't extract the intelligence you want from the existing databases?

Well maybe you can't. But maybe you can. Dratz pointed out that the "if you build it they will come" philosophy is the ruin of many a technologist. It just doesn't necessarily happen.

At my workplace, there's a data warehouse. But does it contain feeds from all the company's repositories? Noooo. In fact, much as you'd like to think the organisation is all one big happy family, there remain concepts of ownership. Competition, even.

Break down the information silos? Look at the business silos first. Work on the business issues, the internal politics. And tell me integration is axiomatic.


All I'm saying is, think more carefully about what you're doing. If there's a green light, start small. Don't try to bring on board business units that aren't yet interested - or even jealously guard their IP. Datawarehouse by all means, but only as and when there are specific business returns.

Tech: Business intelligence: basic practice for print reports

I came across a post which discussed a list of must-haves for identifying information on printed reports. I came up with my own to see how they matched. All this sounds really simple, but too often the basics are missing.

1. Report title
2. Data source
3. Data refresh date/time
4. Print date/time
5. Page number (of n pages)
6. Filter values, if any


Now, what did Mitch Wheat have?

He adds:
- who printed it
- alternate shading of data lines
- don't print empty reports
- URL to the live report
Apart from those, we're substantially in agreement. Personally I don't think those additions are must-haves - although they're useful options. He adds to the title a "unique identifier" which he rightly points out is essential when the stock of reports is large.

What else makes you grind your teeth in frustration when it's lacking in the report? (apart from the quality or absence of data!)

Too often, there's a communication disconnect when someone responsible for BI services is working through issues with information users. It's not always possible to even agree whether they are working off the same page. Are all six identifiers above present? If the software tools aren't able to generate all those identifiers, it's time to rethink the toolset. If this is not possible, proxies (through either data or business processes) need to be engaged. For example: strict controls over when data is refreshed; information users only having access to a single data source. In other words, eliminate all possible sources of ambiguity.

All this is something that ideally should be worked out between the report developers and information users early on in the development process, at the point of business analysis. However, in mitigation it must be said that some ambiguities can creep in later, for example when extra capabilities are added to either reportage or the data environment - or the information user's environment. Be mindful of the ramifications of such changes.

Tech: Database the world with XML (Semantic web, part 2)

(part 1 was Semantic web, super web.)


I have a vision: I want to see the whole digital world databased.

Why? Databases are wonderfully associative tools. We can make connections, sort, and list. We can gain new insights into our information with rapid querying and analysis tools (business intelligence tools in particular).

Now, databases are rather inefficient for storing information, as a colleague pointed out to me. But once upon a time, relational databases were said to be impractical in the real world for much the same reason. Then precipitous drops in CPU and storage costs brought the theoretical into the real world, to the point where you’d be hard-pressed to find a database not predicated on the relationship model.

My vision will prevail (although I’m in for a bit of a wait). The web will become a virtual database, thanks to semantic web and XML technology. We will see a gradual takeup of the concept, through the markup of new and existing pages in XML, which will define the web page semantically, giving machine-readable meaning to the information on the page. Search engines will need to be more powerful to process that meaning, to integrate an open set of disparate pages. This is the power of the semantic web paradigm, this is how true integration will happen.

Finally, the whole of human knowledge will be integrated, and we’ll all be experts on everything… whoops, getting ahead of myself here. (We only think we’re experts.)
Seriously, there’s no reason we won’t go down this path. Of course, beyond a certain point much of this information will remain specific and privatised, sensitive to organisations or individuals. Yet what remains in the public domain – even now – is powerful. We just need the tools in place to boost the value of this chaotic, cluttered web.

Tech: Business Intelligence misconceptions (CeBIT part 2)

Part two in matters arising from CeBIT Australia 2006. Part one is my overview of future trends (revised). Part three is on the future of broadband, a crucial aspect of information integration.


What is “Business Intelligence”? In a nutshell, it encompasses database query, data analysis, and reporting functions, usually wrapped in a single software package. Typically, it runs off a data warehouse or data mart, a database (often denormalised) that is fed information from, and is offline from, an organisation’s transaction database(s). This can often utilise data ‘cubes’ extracted from the database for ease of analysis, but it doesn’t have to.

Now that we’ve got that out of the way. The term is widely misunderstood or abused, going by its categorisation at CeBIT Australia 2006. Some of it was likely to be due to over-zealous marketing, but I think there are some genuine misconceptions, as would happen with any term that has its fifteen minutes of fame as a buzzword.

The most common misuse is to describe any reporting as BI. A suite of reports is not BI if you can’t dynamically query data, analyse it (with pivot tables, for example), and create custom reports. Some of the self-styled BI exhibitors had a small set of pre-canned reports; some could create custom reports - in a limited way. Only one - Yellowfin - could perform most BI functions.
Pronto said they had a business intelligence module – but it wasn’t installed at the show. Others, including Sage, swore they were in development as we spoke. However, the overall record is poor. I talked to 13 companies purporting to do BI, and found only one had a product to demonstrate that remotely approached Business Intelligence.
There was even one or two - Baycorp being an egregious example - that categorised themselves Business Intelligence because they provided an information service to business. Well yes, but it’s not BI in an IT sense.

Aggressive marketers aside, it sounds like some education is needed, even within the IT world.

Tech: Business Intelligence: Hyperion and Microsoft seek shelter in each other’s arms

Unexpected news today that Hyperion and Microsoft are to integrate their Business Intelligence offerings.

Hmm. That report wasn’t greatly helpful: integration covers a multitude of sins. I had a closer look, via the respective companies’ websites. The press release is identical (except that Hyperion’s site was easier to navigate!); in fact, the announcement was made at Hyperion’s global Solutions 2006 conference.

The key word here is ‘interoperability’. They are planning to allow their respective products to work with each other. In particular, people will be able to access components of Microsoft’s SQL Server with Hyperion tools, and access Hyperion components from SQL Server Reporting Services.

Although the proof of the pudding etc, this is a significant initiative. In a number of ways, the two companies have become direct competitors. Like few others, Hyperion has built up a vertical solution package encompassing database, business intelligence, OLAP (online analytical processing), ETL (extract, transform, load) and a healthy range of tools relating to Business Performance Management. They’ve done this both by internal development and acquisition – with a particularly useful find in Brio for business intelligence [unfortunately, they’ve drowned the brand name, but you can find it as significant parts of Hyperion Intelligence v8, or Hyperion 9 BI+].

On the other hand, Microsoft is simply Microsoft. They mostly build their own (the SQL Server database technology was acquired from Sybase). As the elephant in the living room, they’ve diverged from other BI toolsets - as I’ve discussed before. Their delivery model tends to be “here’s the components in pieces on the floor. Oh, would you like one of our partners to do some consulting?”

However, by now they both have the full toolset and are going head to head.

Which is one of the reasons the news was unexpected – but good. Interoperability is good. It’s going to be essential for the survival of software companies, and it’s a boon for the information consumer. Hyperion and Microsoft each have something to gain from this initiative – although Hyperion more so, given the disparity in market presence. In fact, Hyperion has already integrated Microsoft's .NET development framework into a number of its products.

Hyperion seems to be doing what is needed to survive in its core areas. By comparison, a takeover of one BI company by another (as with Crystal Reports and Business Objects) is risible. Expect Hyperion and Microsoft to remain two of the main players in this space.

Tech: The future of Business Intelligence

In yet another Business Intelligence consolidation, Microsoft is buying ProClarity (see ITToolbox's report). ProClarity makes front-end, visualisation tools for use with Microsoft technology. This buyout makes sense, but it will no doubt scare other BI vendors - as does everything Microsoft does in their space, no doubt.

My feeling has been that the major BI products have been converging for some time, in terms of both look-and-feel and functionality. Apart from Microsoft, who has always had the muscle to go it alone. Yet this should scare the vendors more than most. Traditionally, Microsoft's solutions haven't really been solutions: they've been technology, which then needs to be implemented. Typically, Microsoft recommend one of their partners to actually design and implement a solution, which entails a fair bit of coding and tailoring. Sometimes a customer will have their own team of Microsoft programmers - with SQL Server 2005, .NET-based development plus the customer's inhouse database resources will normally suffice. However, the implementation will certainly take time and resources.

With the takeover of ProClarity, Microsoft will now have a better front-end - ie something more presentable for the customer, something that should take less time and resource to implement - albeit probably with increased training requirements.

I've said consolidation is good; it helps technology converge, and clears the field somewhat. I may be in the minority on this, as a Data Warehouse Institute poll has a majority of opinions saying it's bad for customers. But DMReview has a more sanguine view that simply points out the need to be aware of impending upheavals as mergers take hold.

For my money, we don't need a rash of competing technologies requiring evaluating customers to get their heads around a disparate range of tools. Ideally, we're moving towards more intuitive paradigms that don't require expert training, and convergence helps this. To date, though, convergence is going to snag on Microsoft vs everyone else. Indeed, we should expect more vertical consolidation (Microsoft+ProClarity, Hyperion+Brio) than horizontal (Business Objects+Crystal), simply because standalone BI vendors will need to seek shelter within an umbrella solution, rather than Business Intelligence in isolation. Why would customers really want the pain of selecting a database tool, then ETL, then BI, performance management, etc. At least Microsoft has that right.

I think the era of standalone BI vendors rising above niche markets is drawing to a close.