Friday, June 30, 2006

Tech: Google Maps: The new web 2.0 in action

Last night I attended a W3 Consortium meeting. Most of this page will discuss Google Maps, but there’s a little at the end about W3 activity.

The best value was in the presentation by Lars Rasmussen from Google Maps (here's a Wikipedia description of the concepts). Now I’ve used this gadget a few times, and it struck me as interesting, but how useful? It’s a mapping application (based on satellite images and map data) which allows you to search, navigate, and zoom. Particularly powerful in that:
a) you can drag the map around, rather than navigating and waiting for a reload. Makes a big difference;
b) you can choose a hybrid satellite/map format - overlays actual pictures with streets/names. Try it - find out how useful that is. (As one who likes to get one's bearings, I find this is the new geography! I could navigate from my house all the way to Trafalgar Square in London.)
c) you can search for an address and it will point to it. However, Although this works for the US, it doesn't yet work for too many other places (I had no luck with Australia, New Zealand, or England).

Lars’ talk was a very good illustration of the rapidly changing nature of the internet, and how Web 2.0 heralds a radical ramp-up in technology. Just when you thought you’d come to grips with what you can achieve, another paradigm shift is here to wow you.

Any inaccuracies in the following reflect the quality of my notes taken on the night. I’m open to corrections.

Lars gave a story of an application in search of venture capital. They’d spent a couple of years working on the mapping application, and hawking it around. Then they spoke to Google. The spark was a passing comment from one of the Google blokes, that it would be cool to see that application in a web browser. The developers frantically converted their C++ application to a web-based application (from a client-based to a server-based mechanism), and won over Google.

A few days after it was released to the world, they found a blog which took it apart and detailed exactly how it worked. A few days after that, another blog described how anyone could import it to a web site and create a mashup, that is, overlay your own data on theirs. Now we’re starting to see the power in the Web 2.0 concept: collaborative power.

All this without specific authorisation from Google, however they were happy to come to the party, and made available the API details (that’s the programmatic hooks into it), and instructions on how to apply it. In fact, they followed the instructions on the original blog that worked it out!

Conceptually, my favourite application is to be able to attach photos to particular geographical locations. Powerful!!

Although detailed mapping data is currently focused on the developed world (North America, Europe and Australasia), there are already 30,000-odd sites that provide mashups.

Lars: “When you work for Google, it’s enough that someone likes it – you don’t need to do the market research”. He added that current directions for Google Maps are based on the strands of development that are most popular at the moment. Tracking by Google noted that one site that has consistently rated high is one that identifies the exact antipode of any given point on the globe – that is, where you’d end up if you drilled to the other side of the world. Meaningless, but popular.

Lars then mentioned one site Quik Maps – which uses vector graphics to allow people to draw on a Google Map on the fly. This used a Vector Markup Language, VML, that Microsoft had hidden in Internet Explorer. It’s apparently quite buggy, but can be useful if you work around the bugs.

They’re currently working on allowing non-programmers to do mashups by simplifying the API; as well as latitude/longitude issues. All this is free to the public, with the proviso that the applications developed are in turn made freely available to the public. They’ve developed a “Google Maps for Enterprise” package, that provides service and support, and allows internal business uses – eg intranets.

Lars commented that this sort of [collaborative] work is putting more life into the web, and will speed up dramatically. He’s very much looking forward to this future.

For some examples of good Google Maps mashups, look at this page: it has a "top ten" (US-centric, but try Tagzania as well), and lots of pointers to resources. Also look at this page at it's got more description of the concept, pointers to some good sites, and resources for implementing mashups - for programmers and non-programmers.

I can think of a couple of applications I’d be immediately interested. The most useful would be applying this to genealogy: marking a map with significant locations in an ancestor’s life. This can be a powerful visual aid in some of the detective work involved in genealogy, as well as simply presenting a good overview of someone’s life.

Dean Jackson from the W3 Consortium then discussed standardisation of new tools – here, this means enabling a common language to make life easier for developers, rather than simple ‘enforcement’. For example, providing descriptions to enable VML and web widgets to be implemented in any browser.
This is pertinent to an earlier comment of Lars’, that it’s easier to update a web site than to distribute client application updates to millions of people.
There’s no telling where the browser can take us.

Thursday, June 29, 2006

World: How to change the world

Warren Buffett's pledge of US$37 billion (odd) is said to be the biggest act of philanthropy in history.

It's generous. But He's actually pledged US$1.5 billion per year to the Gates foundation on the priviso that it's spent in the year it's given.

It sounds like Buffett didn't know what to do with his money. I wouldn't exactly know either. I'd be concerned that the money wouldn't be used carefully enough - at this magnitude, a few hundred million tends to get dropped here and there. Look what happened when the US poured cash into Iraq. A lot of it spilt all over the ground. Metaphorically.

But with money of that magnitude, surely something structural can be achieved, rather than a few band-aids. Despite poor aid records for the western world in recent times, government aid dwarfs the Buffet billions. And that's generally band-aid, with some minor structural tinkering (eg micro-loans to women). Curing 20 diseases is laudable, but what if you could put the structures in place such that fundamental issues are addressed more permanently?

Perhaps the Gates Foundation is doing a little of that? I heard one of their briefs was to improve U.S. education outcomes.

First thing I'd do: reverse the tide of false information put out by vested interests regarding global warming. There's surely enough money there to influence world opinion and put the issue at the top of the international agenda.

What other structural changes could be effected with such massive sums? Issues spring to mind such as sanitation, education, biodiversity, over-population, etc. That last one is problematic. The cure for overpopulation has been well proven to be escape from poverty. (people have fewer children as wealth increases.)

But that requires a wholesale redistribution of wealth, and that's a bit trickier. And a bit Marxist.

Tuesday, June 27, 2006

Tech: XML for DBAs

“XML is not for every use or occasion. But when you need flexibility and neutrality, it can be ideal” – Ari Kaplan

I’m not a Database Administrator, by temperament. But I have to be across some of the issues, working in business intelligence.

An interesting use of XML in this month’s Oracle magazine. You want to archive data from previous years, but the table’s schema may change. You want to drop the archived data, but you want to be able to restore it again.

The solution is to archive the data into XML format. This involves extracting and wrapping it in XML tags. The data can be placed in an archival database as XML, able to be mined later.

Here, Oracle uses two specific functions: XMLFOREST to convert relational into XML data, XMLELEMENT to wrap user-defined tags around the data. However, I can’t see any difficulty achieving the desired results in other ways, if a given database product doesn’t have such functions.


World: East Timor teeters a little closer to stability

East Timor's PM Mari Alkatiri finally resigned today, giving the beleaguered country a chance to settle down - somewhat.

Alkateri has been identified a major culprit in recent violence, having armed and motivated a fair amount of the militia-fuelled killing and arson this year.

It's sad to see this further tragedy befall a country so freshly reeling from Indonesia’s hate-filled rampage of death and destruction as Timor struggled to emerge from Indonesia’s yoke.

Why did Alkateri take so long to resign? Once his situation became untenable, he wanted to wait for a key Fretelin meeting to re-affirm him, so that he could pretend to be doing the honorable thing. I doubt he did it for president Gusmao; he hasn't been acting outside self-interest for a while now.

If Jose Ramos Horta (resigned from Fretelin a while back) is not made PM despite being outside the party, he will be there one day. He seems to be one of the few with sufficient stature and nous.

The echoes of a nation’s trauma take a long time to subside. Violence remains within the culture, which gives some explanation for, but in no way excuses, Alkateri’s crimes.

When violence and destruction has been so widespread, it’s hard to bring everyone to account. That’s why South Africa had a Truth and Reconciliation Commission, which worked as successfully as any other mechanism could, under the circumstances. That process seems to have been less successful in East Timor. Where there has not been substantive reconciliation, recovery from entrenched violence will take so much longer, as with Iraq.

2-Jul: A further note: Indonesia's rampage in East Timor in its dying days of rule was a classic example of a scorched earth policy. But as I'm often reminded, Indonesia per se is not the culprit: it's a sub-strata - the Javanese elite.

Monday, June 26, 2006

Tech: Web 2.0: Social networking on the rise

There's been a lot of talk about "Web 2.0" - what is it, and why should you care?

It's Internet-based collaboration, via new tools, technologies and websites. At one end of the scale, it's been adopted by teens (e.g. - similar to their rapid adoption of mobile phone technologies. At the other end, it's web-based business collaboration: both within an organisation, and between businesses.

So you don't fit into those marketplaces right now, so why should you care? Because it signals an interactive approach to the internet - not humans interacting with computers, but humans with humans - computers are just tools (as they should be!). And it will become pervasive. You will catch up with it, or it will catch up with you. So far, it's largely with early adoptors - like teens and tech companies.

For lots of examples of Web 2.0 applications, try Web2.0Slides. Meanwhile, I have a few examples below.

A simple example is Wikipedia: already the best encyclopaedia in the world, because it's created collaboratively by a critical mass of people that care enough to make it what it is.

Try also a news carrier which is fed information by the general public; items in turn are given prominence according to popularity.

Blogging is also, of course, another example of interactive collaboration. Enthusiastic bloggers on this topic include Luis Suarez and Edmon Begoli. (But of course, without sufficient readership and interaction, a blog is just a soapbox - like this one :)
Meanwhile, Matt Moran blogs a criticism of this: that social isolation is growing, and we need more face to face. But it's horses for courses really: a) yes, direct contact is important, but b) technology is providing additional avenues for communication - Web 2.0 is not a replacement for face-to-face, but an additional mechanism. Like the phone. Some people are isolated anyway, and technology helps break it down.

Again, I feel critical mass is the key. Smalltime info sharing and collaboration have limited uses, but when it explodes, the effects are awesome.

Film: The rest of the film festival

I saw a total of eight films at the 2006 Sydney Film Festival. Not as good as the old days, when I had fewer time and money pressures.

In order of merit:
An Inconvenient Truth (USA, 2006)
Memory for Max, Ida, Clair and Company (Canada, 2005)
Secuestro Express (Venezuela, 2005)
Feast of the Goat (Spain, 2005)
Metal: A Headbanger’s Journey (USA, 2005)
A Pervert’s Guide to the Cinema (UK, 2006)
United 93 (USA, 2006)

- This list is actually ordered by what I feel affected me the most. I’ve already made notes on An Inconvenient Truth (a significant film); Memory For Max… (moving), and Feast Of The Goat (powerful). I’ll make a few notes on the rest here.

Secuestro Express is also a powerful film. Brutal, too. It depicts the volatile mixture of human experience that is Venezuela’s capital, Caracas. A large city where poverty and obscene wealth rub shoulders, fuelled by an abundance of guns, drugs and corruption. The medium is the kidnapping of a spoilt rich boy and a woman who volunteers in a health clinic. One aspect of the moralism is rather hollywood – the rich boy gets his, the woman emerges alive and unraped. On the other hand, the players are brutalised – terrified – by hyper-drug-crazed thugs with guns. But the hollywood pretence remains, to some extent: that if you’re alive and unscathed, everything’s all right. Yet the scars will remain.
I certainly didn’t feel unscathed when I came out of this film. It was quite nasty at times. But as the film directly points out, half the world is starving while the other half is dying of obesity; few of us are unscathed in the end. I recommend you read the review on IMDB.

Metal: A Headbanger’s Journey is a documentary about heavy metal, from the personal perspective of a (relatively young) fan. It’s quite well-done, although it helps to have at least a passing interest in the genre. It gives a good rundown on its evolution, and all the various sub-genres. I guess I lose out, because the genre is illustrated as starting with Black Sabbath, which is already somewhat askant from my musical tastes. I can tolerate only a little of them, or Motorhead or Deep Purple derivatives (Dio, etc). Some I like that might get honorary mention were not included, so what do I know? I never could stand the posturing of bands from Twisted Sister at one extreme to the death metal at the other (now Lemmy from Motorhead, there’s a good example of metal with no posturing).
I didn’t find the filmmaker entirely successful in addressing the several controversies surrounding heavy metal, such as satanism and violence. Still, it was worth it to see some of the ‘satanist’ bands back away from that stance at the rate of knots when properly questioned.

A Pervert’s Guide to the Cinema was promising, but didn’t deliver. Touted as a Freudian look at film, it was more than enough to put me off Freudian analysis for life. Far too long and ponderous for theatrical release, it should go back to the tv screens where it was obviously aimed. A parade of Hitchcock and Lynch films didn’t redeem the rambling so-called “philosopher and psycho-analyst” with the thick european accent.

United 93 portrayed the 911 plane that crashed into the field when the hijackers were overpowered. I read the festival notes as presenting a documentary, but it was a dramatisation. Starting out with a “we’re all going to die” music soundtrack, didn’t encourage me to continue. I would actually have got more out of a documentary. Still, I can’t blame Americans too much for making a film like this. As George Bush said recently "In Europe [and elsewhere] September 11 was a moment; for us, it was a change of thinking.”

Saturday, June 24, 2006

Tech: That stolen sidekick and the power of the internet

By posting online, a bloke in New York got his friend's phone/PDA back.

I'll recap in case you haven't heard the story. A woman left her Sidekick in a taxi. (A Sidekick is a convergent device, combining mobile phone, pocket computer and camera.) This device was sold by the taxi driver (or another passenger), and ended up with someone who took pictures of herself, etc, not knowing that the phone company automatically backs up the owner's memory to a web site.

To cut a long story short, the owner's friend was able to identify the possessor, and requested it be returned. This was met with various responses, including physical threats and demands for money. He then started a web site on this, that got picked up all over the place. He requested people not hassle/threaten the possessor, just wanted the device returned. Many offers of help and legal advice, and media attention. Eventually the device was returned after the police charged the possessor with a misdemeanor.

The bloke had spent a fair bit of time and energy on this, largely on principle. Throughout, he tried to be fair, legal and honest. The possessor was aware she was in illegal possession, and that it was illegally bought. The NY police did not come out of this very well, because it took public pressure before they acted, despite having evidence of who had it and where it was.

Some things on the internet just take off. No guarantee of successful results for you. But it's an interesting example of what can be achieved.

Climate Change: making your money talk - painlessly

Something further to add to the "things to do" list for global warming: put your superannuation in an ethical investment fund.

Sometimes called "socially responsible investment", this refers to funds and/or fund management that use specific social criteria for funds/superannuation investment.

Choosing how your own money is invested can make a significant difference. And various surveys I've read have shown that ethical investment funds frequently do better, and mostly do as well as, non-ethical funds.

Myself, I've been with Australian Ethical Investments for about ten years now, and have no complaints at all. But there are plenty of others to choose from. They have various investment criteria, and fund structures (eg large vs small company investment, and Australian vs Global) tend to mirror non-ethical funds. I would be surprised if any ethical fund doesn't include global warming on its criteria, but it's not hard to check.

The link to the Wikipedia article at the top lists a number of such funds around the world. In Australia, there is also an Ethical Investment Association, and a web site called Ethical Investor. Any more useful hints on this burgeoning market, I'll post here.

Thursday, June 22, 2006

Tech: Anti-spyware software can be problematic

Second of two overnight reports from the BBC dealt with spyware.

Now, life is difficult for the average punter. At the very least, you need a firewall (for broadband) and virus protection. Beyond that, spam filters and spyware/adware detectors would be useful. But how useful? I’ve been getting too much spam coming through, despite the filters.

And spyware detectors? Useful, but only to a point.

The vendors have a vested interest in providing a free scanning service, then telling you shock horror! there's a lot of malware on your computer. Just pay for this product, and we’ll get rid of it. Not to mention spoofing anti-spyware software, which is just as bad as the spyware/adware itself. It’s worth reading the Wikipedia article on Spyware, at least for the list of spoofers.

I tried scanning through three vendors yesterday: XoftSpy, NoAdware, and Spyware Doctor. They all gave different results; each of them identified a handful of “high risk” situations and a large number of minor risks. Mostly the minor risks were cookies. None of them agreed on the major risks. What they probably identified was vestiges of risky software, but nothing that could activate and cause problems. I suspect my firewall and anti-virus software took care of them. If you don’t have any installed, I suggest you start out with some open source/free products such as AVG (antivirus) or Sygate (firewall). There’s no excuse not to have at least these two types of protection.

The BBC’s “expert” suggested ignoring all alarms unless your system wasn’t working properly. Yes, in the main. If you have the anti-virus and firewall protection.

Life sure is tough for the average punter.

World: Biofuel - ethanol in petrol

First of two overnight BBC reports spoke to something I was doing yesterday.

They were talking about biofuels: fuel made from organic products such as ethanol from sugar cane (as opposed to petrol distilled through thousands of centuries of geological pressure). The consensus view was that at at a maximum 10% mix with normal petrol, it was no problem. In fact, the expert said, biofuels are less harmful for global warming.

Yesterday, prior to hearing this report, I tried to find out for myself the effect of an ethanol mix in petrol, since I’ve noticed more petrol stations in recent times have been offering an ethanol mix, in addition to the normal unleaded petrol (with bland irony, they tend to call this mix “unleaded plus”, whereas you’re actually getting less petrol with each litre).

Specifically, I wanted to find out:
a) Is it bad for my car?
b) What effect does it have on performance?
c) What is the nett effect on greenhouse gas emissions?

In Australia in recent times, there had been some furore over additives to petrol, particularly ethanol*. More recently, the federal government had legislated a 10% cap on ethanol in petrol, and required consumers be informed of that mix.

The answers? I rang the car manufacturer, and was told mine was fine for ethanol – at a maximum 10% mix. However, they were at a loss on the question of performance, and said I was the first one to ever ask about that!

An internet search verified that at that mix, there was no performance loss; however, the literature I read suggested that on a whole of lifecycle basis, the effect of the mix on greenhouse gas emissions was neutral (see here). The Wikipedia article on Ethanol fuel suggests similar (in referring to 100% ethanol). Perhaps the BBC expert hadn’t looked at ‘whole of lifecycle’; perhaps the difference is small enough to be negligible.

It’s worth looking at that Wikipedia article – Brazil and Columbia are being particularly innovative on biofuel.

*Part of the problem was the chief beneficiary was a company called Manildra, who had a virtual monopoly on ethanol distribution in Australia, and who was a major donor to PM John Howard’s Liberal Party, in return gaining great benefit from some Howard policy changes.

Tuesday, June 20, 2006

Tech: Why use a snowflake schema?

The gurus of Data Warehousing are, of course, Ralph Kimball and Bill Inmon, in whatever order you like. (Kimball's company is the Kimball Group.)

The Snowflake Schema gets mentioned an awful lot, but is understood less and implemented relatively rarely. Even Wikipedia’s entry is uncharacteristically terse (any takers for improving it?).

Kimball has a good article on this architecture, and where it would be useful.

Here's a quick FAQ on BI/DW, but to recap: data warehouses experience different usage patterns from normal transaction databases; they typically involve a lot more reads, fewer writes and updates, and few or no deletions. Most writes are at ETL time, batched. Because of this: a) efficiency issues are different from production databases; and because data is loaded from a database that is already normalised, so data redundancy has already been addressed. Typically, data warehouses are centred around a fine-tuned fact table with dimension tables coming off it in a star.

The snowflake involves removing from a dimension table “low cardinality” attributes, into secondary dimension tables. Literature seems to differ on whether the resultant schema is normalised or not, so you can take your pick (data warehousing is not an exact science, despite Inmon and Kimball – who often differ anyway).

Even if you have a snowflake, Kimball strongly favours presenting information users with a simpler star schema representation – unless the snowflake is demonstrably simple in concept.

Kimball’s first example of a useful snowflake is a customer table which mixes browsers with actual shoppers – shoppers would have many more attributes, so it’s best to delineate early. He advocates separating out into subdimensions those dimensions that are specific to actual buyers – who will have much more data recorded about them.

A somewhat similar example is a set of financial products. The varying products may have completely different sets of attributes, so each product-specific set is separated out into a different dimension tables – again, subdimensions.

Finally, he deals with calendar dimensions, where an organisation crosses jurisdictional boundaries, and needs to deal with a variety of settings relating to holidays, fiscal periods, etc. From the base calendar dimension comes subdimensions specifying those different jurisdictions and their attributes.

All that discussion ignores issues of normalisation or denormalisation, although literature tends to indicate snowflakes are normalised and stars (which are really a special case of a snowflake) are not.

The ultimate lesson is that a snowflake is not a trivial solution, and shouldn’t get used willy-nilly.

Film: Memory for Max, Claire, Ida and Company (Canada, 2005)

Continuing at the Sydney Film Festival...

This is a documentary about some residents at a retirement home. The director, Allan King, is said to be one of Canada’s great documentary makers. Cinematography doesn’t look great, in fact the film stock looks rather cheap. But the power in this film lies in the words of the old people themselves. A stirring work that puts the humanity into a group of people that are usually kept out of sight; a reminder that they are people, with memories. There’s a lot of humour, a lot of pathos – usually mixed together.

Despite the memory failures, the crankiness and sadness, you learn to respect these people, you learn that behind the fa├žade they had full lives (eg “she was an intelligent businesswoman”), and that they should be forgiven for their deterioration: time will do that to all of us.

It’s a Jewish retirement home, which adds a certain familiar flavour to the mix. And of course, there’s more women than men.

- “How do I look with these glasses?” from a woman old enough that it doesn’t make much difference.
- [A resident asked about Max, and was told “He passed away five days ago.”] “That I don’t remember.”
- [A resident asked why she was upset]: “I’m lonely as the devil. I can’t take it.”
- “You know you said that when you’re 80, you can forget what you like.” “How d’ya like that? I made it up.”
- If I can’t do that I’d go crazy.
- At least I’m not bothering anyone.
- Everybody here is confused. If you weren’t confused, you wouldn’t be here.
- You like the chocolate? - I hate ya. - I’m sorry.
- Will you keep quiet? Will you keep quiet? They’ll put you out of here. (Aside:) God forbid it should happen to any of us. It’s because she married a goy.

- I used to take blood. I learnt a lot. I’m sure there’s something here I could do. Cheer them up a little bit... and now I’m in that position myself. Life’s funny. But I can’t complain, I’ve had a wonderful life. The last years are not good... it’s very wonderful that I can think back.

Monday, June 19, 2006

Tech: Plagiarism and the net: new paradigms needed

News that one in five U.S. teenagers plagiarises from the internet? The nearest thing to a surprise is that the figure is so low.

This represents a very clear paradigm shift that the real world hasn't properly caught up with. The information is there on the internet, and learning can no longer involve simply regurgitating it. There are sites that have banks of model essays to download and crib; there are sites that charge a little more to write your essay to order.

But the easiest sin is to copy and paste something found on a web page. It happens, it will happen, it's inevitable.

Teaching needs to take this into account. Possibilities:
1) Show me that you can access the knowledge by reproducing it, with the links referenced;
2) Demonstrate in some other fashion that you are on top of the course work.

It's not enough to reproduce knowledge - you're just being marked on your ability to access it. Students should be able to prove - somehow - that they can make use of this knowledge: for example, by synthesising new knowledge or information, or demonstrating how it can be applied.

We are beyond the era where knowledge is a virtue in itself. Say you have perfect access to human knowledge. Now learn something.

That's the paradigm shift.

Film: Feast of the Goat (Spain, 2005)

Another screening from the Sydney Film Festival, Feast of The Goat (La Fiesta del Chivo) is based on the eponymous novel by Mario Vargas Llosa. It is set in the Dominican Republic at the time of the dictator Trujillo and - unusually - it's a Spanish film shot in English, with Isabella Rossellini as the central character. Although the story’s strands - in depicting an archetypical latin american dictator and his eventual downfall - are not unfamiliar, the film is very well executed and thought-provoking.

What is your response to life under a ruthless, despotic dictator? Would you escape, fight back, or knuckle down? This story is about those who chose the latter path, and the ramifications of that choice - which are many and often involved compromising one’s very humanity.

The film revolves around a daughter’s return to confront her now immobilised father, who had sacrificed her at the dictator’s altar when young. This resulted in permanent damage to her, body and spirit.

Others compromise themselves - or others - in a variety of ways. One character is made to break off a wedding because his fiancee's brother is said to be a marxist. Later, that character is made to kill a "marxist", who turns out to be that brother. Elsewhere, a couple of men that are killed are smeared as being homosexual lovers. Reminiscent of the Malaysian Mahathir regime's smear on Anwar Ibrahim, the weapons of choice of despots become almost banal cliche. Nothing changes.

Why would you compromise yourself? If not fight, why wouldn’t you run? The reasons aren’t heavily explored here. In some cases, it was a lifestyle choice. In other cases, a small compromise is followed by a bigger and a bigger one, until one is swallowed whole by the enormity of the brutality.

The conspirators who killed the tyrant were those selfsame who had compromised their souls. Yet destruction of the regime doesn’t constitute redemption. Everyone is scarred.

Friday, June 16, 2006

Tech: How to breach security with a memory stick - so simple!

We've heard in the past how USB drives/memory sticks constitute a real security risk. Well, someone's demonstrated the simplest method yet for infiltrating any company's computer systems, using cheap memory sticks.

As SecurityMonkey has pointed out numerous times, the human factor is a major issue - if not the biggest problem - in security risk.

In a nutshell, you put a trojan - really, anything executable - on a memory stick. On lots of them. Scatter them around in the employee carpark of the target organisation. Then just sit back and watch people plug them into their computers ("Look. Free memory. What's on it? Does it work? One way to find out..."). Watch the company's data just walk out the door. Or watch the network collapse. The possibilities are endless.

PS Full marks to my wife: she said she wouldn't pick one up. But then, she's been bitten before. On her offday, some bloke "borrowed" the use of her computer, and plugged in an infected memory stick. It totally tanked the computer.

World: Windschuttle: it's like having a holocaust denier oversee the BBC

Startling news that the Howard government has appointed Keith Windschuttle, radical rightwing history revisionist, to the board of the ABC, Australia's public broadcaster.

This is really shocking. A lurch to the extreme for Australian cultural life. Windschuttle is the equivalent of a holocaust denier, pretending that wholesale slaughter of aboriginals by early Australian settlers didn't happen. Surprising that the Wikipedia article on him doesn't do justice to the heinous ideas that he perpetrates. Well, here's a critique of his fabrication of history in a mainstream paper; just to be fair, here too is a critique from the right: Gerald Henderson, arch conservative, has a go at him.

The worst of all is that, even if his views have marginal effect on the ABC, this appointment helps legitimise his nonsense. Do you think even Howard intended to go that far?

Thursday, June 15, 2006

Tech: DB2: an impressive new database release

This blog’s been slow because I attended a particularly good course on the new release of DB2 – called Viper on pre-release; now simply DB2 9. The main presenter was one
Boris Bialek, who was particularly knowledgable and entertaining.

Technology competitors are constantly leapfrogging each other. It’s hard to say that product X is consistently better than product Y when the following year, product Y brings out a new release that trumps competition.

Having said that, I think DB2 now has the edge that IBM’s competitors ( Oracle and Microsoft's SQL Server) will be struggling to match – Microsoft in particular, since they’ve only just released a major upgrade, SQL Server 2005.

In fact, IBM has made a realistic effort to keep Microsoft at bay with the release of an Express version, which is effectively free – but limited to 2 CPUs and 4 gigabytes of data. This is enough for them to prevent revenue leakage, but at the same time provide a small entry point for developers and small business.

In order of merit, the chief points about the new DB2 are:

  • Native XML support – this is not the half-baked implementations of its competitors; it’s a true hybrid relational/XML database, storing XML documents intact – no shredding – and providing proper indexing to the XML fields. Each XML document is a field in a row.
  • Autonomics – memory management is now largely automatic: just set the upper threshold of DB2’s entire memory needs, and its internal management will produce optimal results – among multiple instances – better than a DBA’s manual tuning efforts. It’s so good, IBM plan to get rid of all other memory management parameters.
  • Free at entry level with Express versions
  • Backup/restore – a host of improvements to handle partial/failure situations
  • Range partitioning – ability to partition tables by key values
  • Granular security – Label Based Access Control allows administrators to define access levels within tables
  • granular compression – data compression can be defined up to the row level (note that this is not an exact equivalence to granular security
  • other improvements – including capacity improvements at page level and below

Other databases will have some of these features already, but the true Native XML support is a first for a relational (non-specialised XML) database. The support for XPATH and XQuery structures is good – very good – as is XML schema support. All way better than anything currently on the market.

That hybrid model may cause some rethinking of the general concepts of relational databases. XQuery and SQL constructs can be embedded within each other, but you can’t precisely treat fields within XML documents as database fields – the document structures are too flexible. First Normal Form is instantly broken if tables and XML documents are treated as a continuum.

Interesting to see where this will all take us. And good to see the technology is there. Although this is invisible to most people, the world is already exploding with XML.

Wednesday, June 14, 2006

World: Louis Tillett: he’s back - more Brian than Elvis

Louis Tillett is back, in spades.

Not that he’s been absent for years - he always seems to pop up sporadically - it’s his renewed musical vigour and command that is noteworthy.

He’s been in the Sydney music scene for over 25 years, in various states and with various bands. I first saw him at the Sando in Paris Green nearly 20 years ago, and he’s there again, with a Sunday night residency. He has a superb band (double bass, drums and saxophone accompanying his keyboards and singing). He was playing the keyboards as an instrument of rhythm and of melody, sometimes both at the same time.

He was in fine form: older, yes, but obviously healthier than he’s been for a long time. He must be on the wagon, and it really shows up in the music, which was tight, complex, and strong. The crowd was small but vociferous; the songs surged, died, and were reborn, to loud acclaim.

The first piece I heard took a while to sink in, then resolved to Sailors Dream, and what more could I want? A magnificent journey, turbulent and peripatetic, fading out and in. It was followed by another of his Wet Taxis tunes, Clock On The Wall. The two together seemed to last over 20 minutes – and if that sounds like jazz, well it isn’t and it is. Jazz fusion has a bad name, and it wasn’t that. Although he veers through blues (and Paris Green had jazz tinges), I listen to him for his rock; Wet Taxis was certainly rock, and that’s what this was - albeit with hint of the sensibilities and musicianship of jazz.

Wherever you read about Louis, he’s trying to conquer his demons, drink and depression. But he has clearly risen above, in the last few years in particular. The tortured artist may be a quaint picture, but give me the artist in proper form, as Louis Tillett was and is now: a consummate musician and songwriter, pulling together an impressive band.

He’s got a great-looking web site, and you can download some music from it. He has a bio there, and a site called Divine Rites has a discography.

Well worth another visit to Newtown, another Sunday evening.

One note: The Sando – Newtown’s Sandringham Hotel – once an institution, is now a foreign country to me, despite its re-emergence as a venue. It’s been totally gutted and is unrecognisable from the venue and local that I knew and loved in the late 1980s. I’ll blog something on it soon.

Sunday, June 11, 2006

World: The bottom line of Global Warming: An inconvenient Truth

"the era of procrastination, of half measures, of soothing and baffling expedients, of delays, is coming to a close. In its place, we are entering a period of consequences."
- Winston Churchill

Global Warming is An Inconvenient Truth - and that's the name of a dynamite film (which kicked off my sojourn at this year's Sydney Film Festival).

This film is the clearest and most comprehensive overview of global warming that I have ever seen. Its basis is a graphical presentation given by Al Gore, who has constantly refined and updated it over the thousand plus times he has given the talk.

I am suspicious of activist films, simply because I don’t expect to learn anything new. But the power of the presentation is undeniable: if Gore gave his talk in Sydney, I’d be there in an instant. The message is direct, the most succinct it’s ever been, and the presenter has won my renewed respect.

The outcome is frightening: even if we act now, the oceans will likely rise 6 metres, which will result in 100 MILLION refugees from lowlying areas in India, Bangladesh, China, the Pacific, and so on. The west is barely able to acknowledge current refugee situations (Darfur, for instance); we will become a spiritually poorer world as we knock back their suffering and focus on our own problems. Massive infrastructure investment will be needed to protect and mitigate.

A survey of a random 10% of over 9000 published (peer-reviewed) scientific articles gave NO dissent from the fact of global warming; a similar review of newspaper articles showed 53% expressed doubts about this reality. That is a very stark illustration of the extent to which the public is misled. A clear parallel was made between this situation and the “doubt” spread by tobacco companies when the links between tobacco and cancer were scientifically drawn.

The film clearly shows other climactic effects that we are already seeing. Desertification, extremes of weather: increased drought, heat and violent storms in particular.

This is the first work in any medium that draws together all strands of the issue in a complete summation of the issues. It even deals with the temptation to move straight from uncertainty to do-nothing despair: there are always things you can do. At the very minimum, a) vote strategically to make this issue top of the agenda; b) change your electricity source to (certified) green energy – most energy companies now have this option. More than this, the film and web site has many suggestions.

The web site for the film also has a trailer - take a look.

Whether you’re a climate change skeptic or think you know it already, this film is a must-see. It should be text-book material in all schools.

What will you say to your children about your action at this pivotal point in history?

Thursday, June 01, 2006

Tech: Gartner's Database market shares

Apropros to my comments about databasing everything, I note that the latest Gartner database report says the market (ie, relational DBMSs) has grown by 8% last year.

Is that all? Well, that’s the year-on-year increase in sales of new systems (as I read it), not a measure of the amount of information stored in databases.

However, of more interest is the market share. As you’d expect, Oracle’s far ahead, followed by IBM, with Microsoft only managing two thirds of IBM’s share. The actual proportions are 49% to Oracle, 22% to IBM, with Microsoft at 13% and Teradata only 3%. Surprisingly, Teradata’s market share has been static. Its tools were somewhat rudimentary when last I played with it, but it is quite an industrial-strength product for high-end data warehousing – quite a growth area. Probably indicative that IBM and Oracle are successful in maintaining their presence in that market segment.

What about open source? Growth is significant, but from quite a small base, so products like MySQL and Ingres are not quite on the radar yet.

Year-on-year movement in market share is also surprisingly static. There has been a slight increase from Microsoft, at the expense of most others, but the figures show they’re having more of a struggle than they bargained for, in trying to match the product – and brandnames – of Oracle and IBM. Still, those figures definitely count for something, as Microsoft’s entry point is considerable lower than all their direct competitors, so unit sales must have shown some oomph.

Of more relevance to my earlier comments, I guess, would be number of installations, amount of data held in each system, and year-on-year changes in those figures. If I hear anything useful, I’ll publish here.

World: The Snowys: privatising a major community resource

I was accosted today by saturation advertising, encouraging people to register for shares in Snowy Hydro Limited. It's sad to see Australian governments* collude to sell off this essential piece of infrastructure.

This is a hydro-electric and river diversion system – one of the biggest in the world - that provides a massive amount of renewable electricity (over 32,000 Gigawatt hours per year at capacity), and thence governs distribution of a very important – and contentious - water resource.

On sentiment alone, the Snowy Mountains Scheme is a crucial part of Australian history. From 1949 to 1974, thousands of new migrants toiled on it, in the process finding their place in Australian culture, proving themselves as Australians. Over 100,000 people worked on it over those 25 years, and 120 people died on the job. I cannot say enough about how important to Australia's growth as a nation - both economically and culturally - the successive waves of migration have been.

However, history is history, not an argument for public ownership. It is as an essential infrastructure resource – controlling seven power stations, 16 dams, and a significant amount of renewable energy and water resources - that its control should be in the hands of the people. Legislation alone cannot provide sufficient oversight of the complexities of these resources, and the communities of both people, flora and fauna that it directly affects.

As a sop, there has been some legislation to cap foreign ownership. This will have no practical effect, as private hands have the same profit imperatives, whether those controlling it live overseas or not.

Opposition to the sale has come from both community and prominent Australians, including Justice Marcus Einfeld, ex-Reserve Bank governor Bernie Fraser, and ex-PM Malcolm Fraser. There’s also some adverse legal advice on the constitutionality of the sale. For what it's worth, a Herald online poll showed 85% opposition, at last count.

I'll also add here an alternative perspective from the Herald’s business writer Alan Kohler, who depicts Snowy as a hedging company rather than a marshaller of resources. One could also argue that its fortunes depend on rainfall, which will probably decrease with global warming. Kohler says the water resource is only a "rented" commodity, but that doesn't stop it becoming a political football, with the sale increasing pressure for commercial (rather than community) outcomes.

Still, I remain quite concerned that legislation is not a sufficient instrument to ensure that Snowy’s water, electricity and other infrastructure resources are marshalled for the benefit of the public. Shareholder interests and public interests do not generally coincide.

*The Snowy company is currently in joint ownership of the federal, Victorian and New South Wales governments – surprisingly, it was the latter (the majority shareholder) rather than the former that initiated the sale process.

2nd June: Federal backdown
Some quite exemplary political opportunism from both the NSW and federal governments. NSW premier Iemma stood resolute on the sale, claiming "we need the money for hospitals" (oh, so you wouldn't be properly funding hospitals were it not for the sale?). For the federal government, John Howard said he was responding to public concern, and withdrawing sale of their 13% share. Of course, that rump portion is good for nothing, and will be very quietly sold off once the furore has abated. Of course, Howard wins the prize here for biggest political effect for smallest cost.

4th June: The sale is off
The NSW government followed the federal lead, and the sale is off. I must say, I expected them all to weather the storm but federally, Howard's got quite a few shaky electorates that could be affected by this, and NSW's due for an election next year, so nobody wanted unnecessary voter aggregation. The Victoria government followed suit.
All in all, a good result for public activism - for the moment. We can still expect the NSW government to try to put it back on the table later. Once started, pushes for privatisation never seem to go away.
A couple of minor questions:
1) Who backed down first out of NSW and Victoria? Probably not important, as Victoria was only acceding to everyone else's plans.
2) What reason did the NSW government give for abandoning the sale? They could still have proceeded without the 13% federal share on the table.