One of the hardest things about solving a problem is keeping your eye on the problem while a cacophony of distraction whirls around you. People want to tell everybody it isn't their fault. You want to tell everybody that it isn't your fault. Random people will want to find out whose fault it was so that they can be seen to have blamed somebody, as if that constituted a sensible contribution to the issue. These things are luxuries few can afford while the problem remains unsolved.

The only sane approach is to find the bug, fix the bug, and move on with your life.

The reward for fixing the last bug is to see the system sing, to watch the engine purr, precisely as designed. The reward is to hear the cacophony cease as the attention of the peanut gallery gets shifted elsewhere, with blissful silence to follow. The reward is the look on the end user's face, when they find the problem that put a dent into their day is now gone.

Stuck?

JAXB2 has a deprecated the javax.xml.bind.Validator class, and the person who documented this didn't bother to document the mechanism to replace it.

What I did manage to find was the javax.xml.bind.util.ValidationEventCollector class, which you can add to your Marshaller object, like this:

			ValidationEventCollector collector = new ValidationEventCollector();
			marshaller.setEventHandler(collector);

Now the marshaling works! And your errors can be retrieved from ValidationEventCollector, like this:

			collector.getEvents();

Hopefully this will save someone some pain.

This is fixed in Java 6, at long last.

The solution is to convert your existing certificate and key into a PKCS12 file, and then use the keytool functionality to merge one keystore with another one. Java 6 can treat a PKCS12 file as a keystore, so putting this together, you get this:

keytool -importkeystore -deststorepass changeit -destkeypass changeit -destkeystore my-keystore.jks -srckeystore cert-and-key.p12 -srcstoretype PKCS12 -srcstorepass cert-and-key-password -alias 1

The alias of 1 is required to choose the certificate in the source PKCS12 file, keytool isn't clever enough to figure out which certificate you want in a store containing one certificate.

At last, it works.

At last, a solution to wifi bandwidth problems. It's the Wifi Speed Spray!

Why was this never thought of before?

Can you spot the difference?

The problem is "bleed", where areas of intense brightness starts to leak across the CCD into areas of darkness. This in turn is caused by the need for the CCD in the Coolscan to be so small, right up against the film. Drum scanners solve the problem by magnifying the image onto a much larger detector "far away" from the image, working like a standard camera.

So how do the modern slide scanners match up? Apparently the bleed problem has been reduced, but not eliminated, so your bleed problem remains. If you shoot medium format film, or even 35mm, and you have the option, go for the drum scan.

No really, real 3D, promise.

I did one serious double take and can honestly say I was blown away.

At the entrance to one of the display areas they were showing off a multi screen display, and this photo completely fails to do it justice:

Philips calls it WOWvx.

You need to find a sweet spot to view the 3D effect, too far away and the image reverts to the 2D image that you wouldn't notice as out of the ordinary in any way. But come closer, and a threshold is reached where beyond the screen depth appears.

Left and right matters too. Walk past, and there is a "switchover" as you move back into the sweetspot. If this thing finds its way into your living room, you could finding yourself fighting for the best spots on the couch.

And content?

Some of the content took the form of tentative attempts at 3D menus that looked a bit bland: Menu options on hovering squares, or 3D titling hovering in front of 2D images. But when you switched to pure 3D content, the effect was breathtaking.

When you look at television, people have got incredibly excited by the idea of HD, which is basically the same as what we've been watching for years, only much, much clearer. WOWvx however does justice to the wow in its name: HD becomes 3D.

If you get the chance to see one of these things demoed for you live, go take a look. No image on the web is likely to do this justice at this point. Not till we get 3D on our computer screens at least.

The only issue left remaining for me was the annoyance of carrying around two portable handheld devices - an iPod, and a mobile phone.

At some point though Nokia and Sony Ericsson were destined to absorb the iPod's features, and potentially offer a phone that would become sole occupant of the wearer's pocket.

Or Apple could have released a mobile phone of their own.

And they just did.

Image source: http://www.apple.com/iphone/technology/specs.html

The site claims "This device has not been authorized as required by the rules of the Federal Communications Commission. This device is not, and may not be, offered for sale or lease, or sold or leased, until authorization is obtained.", but throwing caution to the wind they have announced it anyway.

Running a handheld version of MacOS X, I am hoping that the feel of the phone is not as sluggish as the Nokias of late, and the lack of support for 3G networks is a potential oversight.

But if this device is anything like the iPod, Apple will do well.

Simply put, Harmony wants to build a Java JVM, a faster and more modular drop in replacement for the existing JVM's released by Sun, IBM and BEA.

Following the announcement of the project's beginnings, what followed was one of the most intense discussions on any open source project ever, and questions were raised as to whether it was all just talk, and no actual code. Slowly the noise died down, and Harmony plodded on.

Earlier today, I attended a talk by Geir Magnussen at ApacheCon 2006 where a summary was given of the project so far, and to be honest I am blown away.

Harmony runs Apache Tomcat and Eclipse, no mean feat.

The most impressive is the JAPI class libraries completeness test hosted here, which shows at this writing that 94% of the class libraries are complete.

The importance of the Harmony project and its place in the Java universe is probably under appreciated by most people in it. For the first time, a JVM will exist that can be ported to any platform, including embedded or future platforms, without the buyin of a major (and possibly competing) vendor. And in addition, Java can become as ubiquitous on platforms like Linux and FreeBSD as perl is today, without the need to navigate the legal minefield of the distribution licenses of the existing JVMs.

Despite the fact that Harmony has yet to graduate from the incubator, Harmony looks to be well on its way to becoming one of the ASF's key projects.

Today I got to get the real work done. I got to see the new HD set top boxes from Amino, and I managed to get hold of the last pieces of the IPTV puzzle we have been building.

The real bling for me was the ranges of camera equipment being shown by Sony and Grass Valley. They had both set up and lit a complex set, with each camera showing the resulting HD above it. The crispness of detail in the HD pictures was stunning.

More fun stuff was the HD virtual set being being shown off by Orad, called Proset. They demonstrated their system live, with a news presenter in a combination real and chroma key studio, inserting all the graphics that are usually seen overlaid on top of the screen into the scene itself. The cameras were free moving, but motion tracked using sensors on the cameras and in the studio. TV graphics are cheesy at best though, now the cheese is part of the set.

The big showcase of the day however was the screening of Pirates of the Caribbean, Dead Man's Chest on a Christie 2K DLP Cinema projection system. Digital cinema technology, being called D Cinema, has come about in response to the rise of HD, and as a solution to the high cost of film print distribution. The 2K refers to the number of lines on the screen. It was impressive stuff. The only flaw I picked up that may have been caused by the projector, was that the end credits didn't scroll smoothly for some reason - there seemed to be a slight amount of jitter. Still, at data rates of 250Mbps, D Cinema is impressive.

Today's goal was to find where the major players were exhibiting, and what they were showing off, but first - you noticed what they weren't showing off - SD was nowhere to be seen. Actually I found one plasma at a stand, showing a grainy version of ESPN, but that was it.

In short, HD is king.

This project is attempting to come up with a way to insert virtual 3D rendered objects into a live video stream, without the use of special motion tracking and position tracking studio equipment. The demo they were showing off was a bit shaky, but the concept was very clever. They were able to virtually insert a checkers board onto a table forming part of a live set, which stayed put while the camera was tracked and panned, and even while the camera was taken off the tripod and shoulder held. It was some interesting stuff.

While wondering around some more I was pounced on by people from Forbidden, a UK based company punting a video editing system, written in Java and delivered via the web. An interesting idea.

I then found the Apple stand, and to be honest I was expecting more bling. They were showing off the HD versions of Final Cut Pro and others, but I was destined to be sucked into a demo by Autodesk.

My interest in IBC was more on the content delivery side, but it wasn't difficult getting sucked into the demos for Flame, and Smoke. Their systems on demo were all Redhat Linux based, which was interesting to see, and were being demoed at full HD throughout.

Autodesk were punting the Infiniband system for interconnects between clustered systems. They showed how a Smoke installation, and a second Flame installation, were able to remotely access the same HD video, at full HD frame rate, at the same time, with virtually no latency, and with room to spare. Autodesk were at pains to point out that they support Gigabit Ethernet and 10GBE too, so they are probably hedging their bets somewhat as to which technology might win in future.

The fun continues tomorrow.

Damn right they are, that's what you're paying them for, isn't it?

Does your technical or engineering team revisit what they know? Continually add to that knowledge with new research? Do they challenge conventional wisdom? Do they take note of what their competitors are doing in the marketplace, and are they ready with a response in good time?

Are you sure?

This is a cautionary tale.

"What is your IPTV strategy?"

At the time, the world internet, telephony and television industries were undergoing a period of reinvention. Some called it convergence. Others called it Triple Play. The bottom line was, each of the three industries had realised that with minimal reengineering, they could provide the services of the other two.

No longer did a household have to pay three amounts against three bills for three different services. Now they could buy all three services from just one supplier. One supplier could charge more money, on condition they could boot out the other two suppliers.

This house just ain't no longer big enough for the three of us.

I was researching IP Television at the time, to find out what headend hardware was available, what Video on Demand (VoD) hardware was available, what set top box (STB) hardware was available, and a range of new products were on the horizon. UK based Amino Communications led the field with their beautifully designed range of set top boxes, an industry lead they maintain today. To the end user, the set top box plugged into your internet connection on one end, your television on the other end, and you used a remote control in the middle. IPTV was indistinguishable from normal satellite or cable, just like users wanted.

A brief history of the South African telecommunications market: Hampered by ill advised legislation, an incumbent monopoly operator Telkom was allowed to run roughshod all over the South African telecommunications industry, charging exhorbitant fees for antique services. But the winds of change were coming. The South African government were committed to opening up the market to competition, and once that happened, competitors would break open the market, leading to a long awaited catch up.

But here lay a problem. In the stifling regulatory environment at the time, technicians and engineers in the South African industry knew that high speed internet access, defined as "anything faster than a dialup modem", was unheard of in South Africa. And they knew that this would never change. Ever.

And so in answer to my question of the settop box manufacturer in Durban, the person billing themselves as their chief engineer laughed me off the phone.

You're crazy. IPTV can never work. You're wasting your time.

My research indicated otherwise, and so I persisted. To the lasting shock of the South African internet industry, the incumbent Minister for Telecommunications for South Africa triggered some exit legislation, effectively ending the incumbent operator's state sanctioned monopoly. The much publicised awarding of a second telecommunications license to Neotel was imminent, and unleased from regulatory constraints, existing mobile operators and internet service providers could enter the industry. The game was hotting up.

Internationally, the race for Triple Play was on. In the UK, Telewest and Homechoice battled it out with conspicuous media coverage. Get your television, your telephone and your internet from one place. Existing operators like Sky got into providing internet access alongside their satellite based offering. The same story played out throughout Europe. At the international IBC exhibition in Amsterdam in September of 2005, the hot topics were HD and IPTV.

About three months ago, I made contact with the Durban based company again. The contact was through a colleague, a friend of theirs had just started working there, and out of curiosity I asked again:

What is your IPTV strategy?

They said they'd ask and find out, and the answer came back.

We looked a making satellite cards to receive internet and television on a PC, but we didn't pursue it.

Turns out they still had no idea what IPTV was.

Over the past few years, the biggest question on the telecommunications industry's lips was how to break the enormous lead held by the incumbent Telkom. But a more burning question was, how would Telkom respond to this threat from competitors?

In our research over the past few years, we have found that wireless technologies simply didn't cut the mustard when compared to the power and convenience of fixed line networks. If it couldn't do full bandwidth television, the technology just wasn't good enough. This fact didn't escape the watchful eye of the incumbent Telkom. To maintain their widespread dominance, IPTV was the key.

So, this last thursday, when the second South African telecommunications operator Neotel launched with great fanfare to the South African public, Telkom responded by playing their trump card: Telkom would introduce IPTV to the South African market, and compete with the dominant satellite provider Multichoice.

On friday, I contacted the Durban based manufacturer again to ask what they thought. The answer came back:

We had not heard Telkom's announcement, where did you hear it from?

Three years later, and they still knew that IPTV was not possible, and they still knew that IPTV would never happen in South Africa.

So confident were they in their knowledge, that they saw no value in research, no value in development. They saw no value in monitoring technological developments overseas, nor did they see value in monitoring the local political and legislative environment.

Their technicians and engineers already knew, what point was there in learning something new?

So, ask yourself: What new things did your technicians learn this week? What do they think their field looks like today? What do they think their field will look like a year from now, five years from now? Is there a change in the air, and do they understand what that change might mean?

And has technical opinion inside your organisation changed over the last few years? If not, you may find yourself quietly exiting your chosen market, while others outpace you.

Forget what you know. It's what you learn that is important.

Interviews, I have had all kinds, but this last week I had two that were polar opposites of each other, one where the interviewers did everything right, the second where they did everything wrong.

"I hear you're looking, because we're looking." said they, "keen to have a chat?". I was keen. "How about now?" I said, and in a flash I was in an office on the 3rd floor meeting with the two key players on a risk management project for one of South Africa's big four banks.

There are three things that make me nervous, one is public speaking, the second is appearing on television, and the third is dealing with strangers, so I appreciate it when people put me at ease, which is exactly what the person leading the interview did.

They started with some idle chit chat, and then began with where they had found my CV - via a project manager I had worked with previously.

First step, establish context, I now know why I am there.

Then the selling started. They needed to find somebody that fulfilled two key requirements:

- Someone who could do the job.

- Someone who wanted to do the job.

They kicked off with some brief questions, what kind of technologies had I worked on in the past, what kind of projects? could I do Java? No really, could I do Java?

I wasn't a Dot Net person masquerading in the other camp, they had been sent a couple of those.

Next, a description of their system, starting with an apology: "We're sorry, we haven't followed the full J2EE thing, we wanted to keep it simple".

Simple. I like simple. 'We kept it simple' means they understood the task at hand and hadn't wasted time on unnecessary complexity.

Their system allows traders to trade exotics, and handle the day to day trading and the risk analysis behind it. The more reliable the system, the more numbers they crunch for the buck, the more competitive the division is within financial markets.

Some key parts thrown in, they use RMI on their server, and they use an application developed for the Eclipse platform on the client.

Wow, an approach I have not seen before.

Their problem is their system was built in a bit of a hurry, they needed to bring the system under control, clean it up, relook at security and encryption, and make it faster, and they needed someone with experience to add to their existing already experienced team. Was I keen?

Looks good so far.

What then followed was a long chat about distributed computing, what had they tried, what approaches others in the industry have tried, what could work in their case.

Throughout the interview the two people conducting it showed confidence they knew what they were doing, something not entirely widespread within large organisations, and they showed that it was clear this was a job they needed to get done properly. They weren't there to waste my time, and didn't want to hire someone who was going to waste theirs. They had done a thorough vetting of me by the time they had finished.

"The director has a large wallet" they said.

They were spending their money on people.

Was I interested? I was. But I had an interview the next morning, could I give an answer after then? Sure no problem. "Don't sign anything" they said.

So, the next morning at 8am I found myself at an interview for a software company doing systems for financial markets. I had been there some months before to do an "exam" of sorts, a written test to work out if I knew my Java from my coffee. I type sixty plus words a minute, but was forced to use a pen, something I haven't done for years. The "hour test" took two and a half, and I left with an aching wrist. Their test was some standard stuff, with a focus more on theoretical computer science than engineering and system delivery, but they included some probing questions.

Hmmm. Ok. Don't seem too in tune with the people they hire, don't know anybody in this business who can still use a pen.

But regardless, there I was, at 8am, ready for an interview. Some helpful people at reception tried to find the people I was there to see, none of whom were there. I loitered for 20 minutes before two people finally arrived and said "this way". We sat down in a meeting room, an interviewer and their manager, with a pile of paper, my CV, and what looked like that exam from way back when.

Clearly I was here to be tested, I had had an unfriendly reception and that made me nervous. But no worries.

The interviewer began with the major companies I have worked with, started ten years ago with VWV Interactive. "What did you build for them?". Given time, more coffee, and a less nervous environment, I could probably reminisce for hours about the country's first auction site we built for SAA, Nando's 98, built for Nando's, or the Y Files, an online game built for newsagent CNA. I started by thinking back and saying "ooh gosh", and mentioning the auction system, but I was cut off, we were moving onto the next one on the list, apparently that was all I did back then.

The next item on my CV is a useful little litmus test as to whether the interviewer knows software engineering. Since 1999, I have been associated with The Apache Software Foundation, a not for profit organisation that offers a legal umbrella for the development of projects like the httpd web server, and have been a member of the Foundation since 2002. The acceptance of the use of free software by the business community has been a key part of the politics of software engineering over the past decade, and the ASF has been at the forefront of this issue. Today, business trusts Apache, and its projects are widely used, even though developed by volunteers, and overseen by unpaid members. This makes the question "So how long have you worked for the Apache Software Foundation" ridiculous. The ASF doesn't employ its members.

The interviewer asked anyway, and giving them the benefit of the doubt, I asked them to confirm whether they were familiar with the foundation. "Of course" they said curtly.

Not.

The employment history angle wasn't going well, so they opted for the curve ball. "Don't you think that not doing maintenance meant you lost the opportunity to learn?".

I learn for a living, and own a research company, what a silly question.

I said "no".

Ok, it was time for a practical test. They would give me a scenario, I was to solve it. The problem was a pretty straightforward one, the delivery of a continuous rates feed via HTTP for 5000 users. When a client says "5000 users" what they really mean is "we want to begin with 5000 users and scale up from there". Another classic case for a distributed system, that scales up as customers scale up. You start with a cluster of one machine, nice and cheap, and the new customers pay for new capacity, which you don't need to shell out for at the beginning. A bit of careful design makes sure that each additional user is as cheap as possible, but other than that none of this is rocket science.

But the interviewer already had their own ideas, and weren't interested in what I had to suggest. They were convinced the task meant very high system load, and the large number of threads was a showstopper. They switched into lecture mode, and started laying out their plans to write a custom webserver, based on an event driven model, that ran on one box. It was quite clear that the interviewer's design was correct, and mine was wrong. "The key issues here are performance, and latency, don't you agree?" they asked, reading from the textbook. "Er, no." I answered, "It's a lot more involved than that". Having been involved in building a real webserver, I could see they were getting into uncertain territory, something clients are seldom happy to pay for. They weren't impressed at being disagreed with, time to change strategy.

When you need to catch someone out in an interview, the classic fallback is the old "make up some new computing term on the spot" trick. I remember being asked at an interview "what I did think of OT?"

Occupational Therapy? I don't see what this has to do with anything, can you elaborate?

They meant "object technology", what everyone else calls "OO".

Today's question was "how well do you know your databases? What is an asset transaction?".

Um... to be honest, I don't know, tell me.

And I got the definition for a "transaction". Um... ok, why didn't you just call it what it was? I checked on Google afterwards, and could find no mention of a computing term "asset transaction", only lots of references to bank transactions involving assets. Restricting the search to "database asset transaction" found nothing relevant either. Cheap trick.

In hindsight, what had happened in the interview was that the interviewer's view of computing was that each question had just one correct answer. The "what if" analysis that is so key to the engineering approach had no place in this organisation - what the architect said was final, and my disagreement with their design approach had no place within their organisation. The problem with this attitude is that the architects are like glory boys on a sports team. You've seen the people who won't pass the ball so they can score themselves, when everyone else can see the defender bearing down on them? In the world of business, clients want solutions to problems, not excuses as to why the solution the architect dreamt up failed. And this means that a successful software team needs people who are sharp enough to come up with a design, but flexible enough to bury their ego, and modify their design to solve issues raised by others.

It was clear at this point that their organisation needed no people upsetting the apple cart, and I was not suited at all to their culture. That meant the first interview won.

I called the bank up and said "if you need me, I'm in".

I start monday.