How many ways can you link a node?

Got a call from a friend last week that wanted to chat about some work we did a few years back to migrate data from one Product Data Management (PDM) system to another. Not for the first (or last) time we have come full circle and they are now looking to prise the data out and pop it back somewhere else.

PDM systems typically store mechanical design data – parts of things, if you like – along with meta data relating to the part (part number, material, weight, colour, edibility…). Perhaps the most fun thing they tend to hold is relational and positional and configuration data – what connects to what and where it is in a particular variant of whatever is being designed. Actually, edibility is quite fun too when it comes to designing motor vehicles or nuclear submarines.

Strangely, earlier in the year I had been thinking of the approach we had used to transform the data to get it in a useable form between these systems. Back in 2005 we developed a concept that Andreas Tsiotsias had dreamed up as part of some thesis or other.

Andreas had created a number of Java classes around something called PLMStructuredData in a collective called DataAdapter. Essentially there were three elements to PLMStructuredData – nodes, nodeLinks and nodeLinkInstances.

Nodes represent “something”. Which can be just about anything. A part, a tree, a picture of a part in a tree – that kind of thing. You might also hear nodes called “entities” by lesser mortals.

NodeLinks are the basic relationships between two nodes. They don’t convey very much other than that two nodes might be linked in some circumstances. There is also a hierarchy – one is a parent and one is a child. So you might have a bolt and a nut that have a nodeLink between them where the bolt is the parent and the nut is the child – and they both may also have a nodeLink to a “crazed robot mechanical arm” assembly.

The amazing trick with nodeLinks is that they are very fast to navigate to uncover relationships between stuff. In fact, they are not just amazing – they are amazingly amazing. At connecting stuff that is. I’m not so sure they would be much good to defend yourself against a bear or to borrow a fiver for your lunch from.

NodeLinkInstances define the conditions where a particular nodeLink applies. This might be “always” or if a particular attribute is true or if the root parent node in an assembly is a particular flavour. So – you might only include a link between “dashboard” and “Sennboseheiser Omni Megga Thump 2000 Sub-woofter” if the parent root node at the top of all the parents is “Ridiculously stoopid jacked up neon green drugdealermobile”. But it might also only apply if the build date was between 1986 and 1992. NodeLinkInstances transform buckets of loosely related nodes in to particular things that have meaning.

Armed with these three things, we were able to model just about anything in the universe – including the universe itself. As it was, we used it to model data in PDM systems – mostly of motor cars. At least, we were able to once I had put in place a documented and clearly defined class structure (UML with forward and reverse updating between the diagrams and the method stubs), built all the code with jUnit and dbUnit test cases and written education presentations so that the folks in India could understand what the hell we were on about.

Oh, and commented the code too. Although Dick Stephen might point out that my comments reflected the same tone as this blog entry and so some of the time were very entertaining, if a little confusing.

I also added a set of transformation plugins, a new language for defining how they operated on data (although interestingly it is the only language I know of that doesn’t actually have a name!) and invented what we called flavoured data.

When you want to transform data from one system to another – you are really just changing it’s flavour – and instead of “transformation” you are “flavouring”. It is remarkably easy to change the flavour of data once you know what you want to change it to.

There are, I know, many variants around in one form or another of what I have described above. I know this because I have seen and read about them – not always quite as obvious – but the same none the less. I also know they are the same so I won’t get sued for publishing this.

I did love the beautiful simplicity of our approach – and the innocent arrogance we had in delivering in a few months what many vendors have spent years developing.

But with the passage of time a few thoughts have been forming in my head. Yes, it doesn’t happen very often – and even when it does, it usually involves sheep or rockets or sheep powered rockets. But thoughts have formed none the less.

First of all, a confession. We were all out and out Java zealots at the time. Well, the ones who were important and knew what was what were. I’d have specified Java if I was being asked to paint a house or light a fire. Java was awesome for doing PLMStructredData – and once we had twigged the memory stuff and dropped a lot of the unwanted overheads like multi-threading when you don’t want or need multi-threading, we were able to make it really cook. But I have at last come to the realisation that maybe, just maybe, there are circumstances when Java might not be the bees knees. Not many circumstances mind – I’m painting my burning house in Java next week.

You see, one of the problems we had was that we ran all this in memory. Which is fine for smaller, lighter structures – but once the data gets hefty we had to resort to techniques such as the BOM skeleton – where you hold the structural information in memory and then make calls out to a relational database for the heftier bits of attribute data only when you need it.

But in working recently with loads and loads of less complex data (vastly less complex – here I am, brain the size of a planet…) another thought has occurred to me – why not just hold the whole of PLMStructredData in a relational database in the first place and use Database caching and indexing to achieve what we did with our in memory techniques.

I really hope that Mike Forest isn’t reading this as this is something along the lines that he maintained we should have been doing all along (having not been quite as beguiled by Java as normal people were).

Some may remember the tree walk race that Mike and I had – and if not, may remember the constant bitching from me with a variety of excuses I dreamed up for having lost the race. But only by a few seconds. And in any real world scenario I would have won. Or if applied across multiple vehicle variants he would have been toast. And anyway – stored procedures are so 90’s and not platform or database independent. So there.

One of the standard out of the box input and output plugins for DataAdapter was a means to store and retrieve data from a relational database. But this really was just to dump the data rather than to operate on it. It would be much more interesting to enhance these and the transformation plugins for operations in the database itself.

So I have been tinkering with a few Java classes (alas – without DataAdapter as it is no longer mine). It’s quite good fun to crack open eclipse again and give it a whirl. To be honest, I haven’t the foggiest if what I come up with will be useful – but it will be fun and if it doesn’t come off – it just gives me another reason why I was right all along.

What happened to the Pandas?

The more astute amongst you may have noticed that in my recent post on “Coffee and Pandas“, the title characters – “The Pandas” – didn’t feature particularly heavily in the actual article itself.

Of course, many of you wouldn’t have noticed anything special about Pandas apart from the cute panda picture at the start.

That’s probably being a bit liberal with the word “many” – as “many” is a subset of a universal set “people with nothing better to do” that read this blog. Come to think of it, the strict definition of the universal set I have used (the bit in quotes) is a bit lax too – as it would include quite a large percentage of the world’s populace. My hit counter doesn’t support such a position… unless it has flicked back around to zero a couple of times. That must have been what has happened.

We should also consider the intersect of those that did spot that pandas were not in my article, but know me well enough to be aware that, from time to time, I will freely hazard a panda or two for no rational or explicable reason – and so they choose not to comment.

I don’t particularly like pandas, by the way. They just seem a little less threatening than grizzlies and are more obstinate than teddies.

Oh, and if you are browsing this article for technical content – the bit about set theory above is about as far as this one goes. Note that I have filed it under by “Whimsical” category to indicate that it includes no useful content of any sort.

Anyway, I did start the previous article with pandas in mind. But as is oft the case, I found myself wrapped up in so much of the preamble, that I never made it to the bit about pandas. And this entry seems to be heading in the same direction. Although, technically, there is quite a lot about pandas already. Just not the bits that I had in mind at the start. Of the previous article. And this one.

Back to coffee (don’t worry, I am sure the pandas are on the way). So I go to the coffee machine. This is a different occasion to my previous coffee social disaster – and this time, there is no one at the coffee machine. No opportunity to redefine the boundaries of polite conversation – just me, the coffee machine and 60p in exact change.

I put a 50p coin in to the machine. It drops straight through to the coin return. I’m not exactly cognisant as this is early in the day so as a reflex reaction I scoop up the coin and pop it back in. It drops straight through to the coin return. Again.

Somewhere in a cortex or some kind of lobe – electrons dance around and blow raspberries at each other.

I don’t have another 50p – but I do have a cunning plan. If I can entice the machine to accept a 10p, maybe it will get greedy and swallow the 50p quickly afterwards. There is no technological or rational explanation for why this should work. But, admit it – you have fallen for this appealing logic yourself, haven’t you? Well, if not – perhaps a slight deviation of it. This includes warming a coin up with your hands, blowing on it, licking it or giving it a little back spin with your finger as you pop it in. I really hope that no one quotes me on this out of context.

I confidently pop the 10p in to the coffee machine.

It drops straight through to the coin return.

One day – just one day – you would think I might get coffee without social anxiety or a nasty metallic taste on my tongue.

A sudden thought occurs before I resort to consuming small change. Maybe something written on the front of the coffee machine might offer some helpful advice?

Before you scoff – have you ever read the instructions on a coffee machine? No?

Well, although we all take coffee machines for granted, they are advanced devices managing harmful, noxious and searing substances to channel them in to polyurethane conical receptors which are transitioned across a failsafe hazard interface. It has taken decades of painstaking development and refinement to evolve a level of environmental manipulation, artificial intelligence and raw computing power to control all this with such utter sublime subtlety that we can take it for granted. But all the same, I’m sure they read the instructions for the Space Shuttle before take off.

I scan the instructions and notice the words “Insert Mug” on the coffee machine display.

How absolutely bloody stupid. And what is that supposed to mean anyway? Is it inviting me to squeeze my face past the drip dray? I certainly feel like a mug. Is it any wonder that people get themselves killed with these idiotic contraptions?!!!?!

That’s when I notice the stack of thick foamy cups next to the machine. I’m sure that they weren’t there yesterday. And they are cups for crying out loud.

Now my definition of a mug is a heavy ceramic affair with a suitable loopy handle. Either that, or someone who fails to get coffee out of a perfectly serviceable coffee machine.

I just stand there, confused. Is it a mug, or isn’t it?

“Hi Mark, how’s the coffee?”, a friend chirps in as he walks over.

I have completely no answer.

Damn, damn, damn, damn….

Coffee and Cake!

In order to recover from my ‘near coffee experience’ as documented in my previous post and to celebrate the first drop of code at a major project I am working on, I have bought cakes.

If you aren’t working on a major project, then there is no chance of cake.

If you are working on a major project, then your chance of cake increases depending on how close to Swindon you are.

To compensate those that are neither near Swindon or on a major project – or those that are, but aren’t in Swindon today, here is a picture of a cake.

Cake
...and I don't even like chocolate

 

Mmmmmmmmmm.

Coffee and Pandas

Cute PandaOk, so the app production line has been eerily silent for the past couple of months, I’d have to admit. I have been developing a concept that I thought was quite unique and froody – but then saw that there was not just one app, but a whole genre of apps that essentially were based on the same root idea. And what is more, they sucked.

So, back to the drawing board. Although I was out at ‘Zizzi’ in Kenilworth last night having a rather pleasant Italian in some rather pleasant company and happened across the germ of an idea that was, in a nutshell, rather pleasant. Or at least it will be when I have figured out how to do it.

So that leaves me at something of a loose end as to what to write on the blog just now. Earlier in the week I had a rather pleasant evening in Leamington at ‘Voodoo’. Actually, you know what – it wasn’t pleasant, because nothing pleasant ever happens at Voodoo. In fact, it was more of an entertaining evening in the fun-fun-fun sense of the word. I happened across a leaving do for an old friend who had a dozen or so other old friends along to see him off. Why is it that friends don’t get younger? Something to do with the uni-directional Shott vector on the space-time inversion axis I suspect. Anyway, Martin Favor was one of these friends and he gently grabbed my by the lapels and politely asked why I hadn’t done any blog entries recently. Now there’s an avid fan for you.

To be fair to Martin, he was not the first to remark on the blog. I’ve literally had an email demanding to know when the next issuance will be forthcoming. It was from Nick Dyer. Thank’s Nick – if ever I am famous, you will be on the VIP list. I will do my best not to confuse this with the RIP list or things might turn out bad.

Anyway, I finally hit upon and idea for this entry. As with most things when trying to actually do something or think of something – I decided to get a coffee.

That was the idea, by the way – get a coffee.

Now most people wander up to a coffee machine and press a button (sometimes two or even three on the posh ones), get their coffee and wander off again. Occasionally, they are pressed in to polite conversation that will cover either:

  1. The weather
  2. What happened on the weekend
  3. What is going to happen on the next weekend
  4. What the weather was like on either the preceding or will be on the next weekend
  5. What day of the week it is and why this is a good or bad thing (…including weather)
  6. How bad the coffee is.

Technically, it is a complete social taboo to talk about anything other than these things. You probably don’t even realise that you are conforming to these social norms.

“Ah Ha!”, I hear you say (though not actually as that would mean I was stalking you or the placing of listening devices), “but I was talking about football at the coffee machine – so there!”.

Uh huh? And when exactly did this football take place? Did you, by any chance, happen to mention the weather too – and, come to think of it, this coffee is pretty awful, isn’t it?

I like to think that I am a bit of a maverick – not constrained by social norms. Having absolutely no regard for convention I set it in my mind to get a coffee and to break the rules. Ok, hardly trail blazing stuff – but I am sure that even Monty Python had to start somewhere. You don’t think they were able to openly discuss taxidermy in public without first hazarding the use of a spoon instead of a fork for the consumption of peas, do you?

But how to break this mould – what could I do at a coffee machine that hadn’t ever been done before? Thinking like you are doing right now might get me arrested – and remember this was just a first step, right?

So first I had to approach the coffee machine. The plan extended about as far as that – walk to the machine.

I was in luck – there were two people I knew at the coffee machine. How terrifying might it have been if someone unknown was lurking there. I mean, a stranger might not exactly react predictably. Which isn’t to say that a friend would – only they would have the context of the rest of my unusual behaviour with which to place this event. This has the reassuring effect of cushioning the impact of social anarchism on their psyche – and quite possibly averting brain damage.

I approached the coffee machine and steeled myself for the radical.

“I hear the weather is going to be good this weekend”, my friend proffered.

My mind raced.

“Yeah”, I reply, “better than last weekend. Thank goodness it’s Thursday – how’s the coffee?”

Damn, damn, damn, damn…

Home and away with a NAS and a MacBook

Two left feet
These boots are made for walkin' ... but not too far from my Wi-Fi or my mounts will hang my MacBook

If you have read a couple of my posts on here before, you will know that I’ve gone to some particular trouble to procure myself a development environment so that I can start writing apps – and I have taken my first faltering steps to create the first test app that runs on my iPhone.Well, being as it has taken so much time and effort to set this environment up, there was nothing for it, but to go and buy myself a MacBook Pro anyway and have the development environment handed to me on a plate. Actually, it wasn’t quite that decadent or pointless – I had a change of employer and my new employer, a gracious gentleman of taste and sophistication, gave me the option of procuring my own laptop. Sweet.

So, now I had the tools, the time and and a completed expense form. It was really tricky coming up with new and different things to avoid knuckling down to writing the app – but I managed to come up some good ones. First there was settling in to the new job, then there was getting my TOGAF Practitioner certification, then applying for my Chartered Professional status with the British Computer Society, taking the BCS “Breadth of Knowledge” test and finally booking my PRINCE2 Practitioner certification exams. When I found myself researching the Certificate in Finance, Accounting and Business (CFAB) with the Institute of Chartered Accountants in England and Wales, I knew I had taken the whole avoidance through self improvement tactic too far and needed something to get the focus back where it should be.

There was nothing for it, but to open up that MacBook, fire up xCode and crack on! It was in a slightly chilly hotel room (the radiators had been turned off and turning them on had only improved things a little bit). The lights were that particular shade of yellow that comes from super eco energy saving bulbs that are identical to the 100 watt pearescent volt glutton ones in every way except, of course, the way that they consume energy and emit light. The MacBook high definition ultra screen seared through the murk and my retinas. You just got to love a display that is only matched in its crisp sharpness by its glossy reflectiveness. Fortunately, in this room there wasn’t much to reflect. Certainly nothing that a MacBook as cool as this one would entertain. Except of course the light bulb hanging from the ceiling in the centre of the room which, by a staggering coincidence, also aligned with the centre of my screen. Leaning forwards just a fraction placed my silhouette between the light and the screen and added just enough discomfort to my lower back to give me the satisfaction that I was suffering for my art.

That’s when I noticed it for the first time – a little pop up window saying that the network drives that I had mounted from my NAS at home for photos, video and music respectively were no longer responding and asking me politely if I would like to disconnect from them. Well, given that I was many miles away from my NAS which wasn’t hooked up to the Internet, this didn’t seem entirely unreasonable, so I disconnected them. But that started me wondering what would happen when I got back home. Would they just magically come back by themselves, would they ping back to life the next time I tried to access data on them, or would they expect me to manually remount the disks every bloody time I came home. I wasn’t sure, but I suspected that the answer had the word “bloody” in it. With a sense of moral outrage and welling satisfaction – I now had reason to avoid writing an app – there was nothing to do but to spend the rest of the evening figuring out how to fix this show stopper.

As it turns out, when I got back home, things were not quite as bad as I expected. Time Machine was quite clever and mounted the drive from the NAS it needed when it ran backups every three or four minutes. Sometimes it just runs continuously with occasional breaks when you don’t need the system to do anything quickly. But even if it’s inconvenient, it’s not going to get me out of writing an app, is it?

Surely iTunes wouldn’t let me down – I clicked the familiar blue circle and up it came without a hitch even though I knew my music was on the NAS. Darn, since when did iTunes do something the way you wanted it to? In actual fact, I realised later that the iTunes library was on the local hard disk which is why it came up – it was only my music files that were on the NAS. Had I tried to play something, to my glee it would have failed – but I didn’t because I was getting cross and wasn’t in the mood for Chas and Dave. So, disconsolately, I plugged in my iPhone to upload the photos from it and was delighted to find that Image Capture couldn’t find the NAS mounted directory to save them to. Bingo -prevarication gold!

When I started looking in to this, I was pretty sure that this must be a common issue. How many people have NAS drives? Loads! How may people have MacBooks? OK, don’t answer that one, but more than three people. And sure enough the Internet is littered with articles about how to mount your NAS when at home – along with a number of applications, both free and fee for making all the necessary adjustments. But hell, this is OSX and I started doing UNIX back in 1986 so this wasn’t going to be a problem to code up myself, even if I was cheating by searching the Internet!

The strange thing was that most of the articles seemed to very nearly do what I wanted to do, but not quite. I wanted something that would connect to my NAS when at home, and disconnect when I was away – automatically. Or at least, if it didn’t disconnect, not cause any nasty problems when the NAS wasn’t there any more.

My first thought was to use automounts. These will make the mounts to servers when you access a mounted directory and disconnect if you don’t use them for a certain time. …and then reconnect if attempt to use them again. I set this up easily enough (it involves editing /etc/auto_master and creating an /etc/auto_smb. Plenty of guidance elsewhere if you want it – but here are mine:

/etc/auto_master:

#
# Automounter master map
#
+auto_master # Use directory service
/net -hosts -nobrowse,hidefromfinder,nosuid
/home auto_home -nobrowse,hidefromfinder
/Network/Servers -fstab
/- -static
/- auto_smb

## Automounter master map#+auto_master # Use directory service/net -hosts -nobrowse,hidefromfinder,nosuid/home auto_home -nobrowse,hidefromfinder/Network/Servers -fstab/- -static/- auto_smb/etc/auto_smb:

/Users/<username>/Volumes/music -fstype=smbfs ://<user>:<password>@<NAS>/music

/Users/<username>/Volumes/photo -fstype=smbfs ://<user>:<password>@<NAS>/photo

/Users/<username>/Volumes/video -fstype=smbfs ://<user>:<password>@<NAS>/video

I found the problem with this, however was that if you strayed in to a directory used by automounter whilst away from home, it hung around for a very long time attempting to connect. And heaven forbid if you tried to save anything using Word, Powerpoint or Excel. They just seems to smell that there might be a mount you could conceivably attempt to use – even if you weren’t using it – and hang for even longer than Finder did.

Two things eventually came to my rescue. First was the excellent article here (http://tech.inhelsinki.nl/locationchanger/) which explains how to set up a LaunchAgent to detect when your WAN changes and a locationchanger script that to detect when your WAN is at home and when it is somewhere else. I modified the script to mount my NAS directories when at home. However, when away from home, I found that the mounts did not unmount automatically and we were back to the Finder and programs hanging when going near the mounts.

That’s when I remembered back to my days working at Rover Group in the late 90’s under the expert tutelage of Obe Wan Kenobe – otherwise known as Paul Manton. I never did make Jedi, but back in those days we had problems with clients hanging when there were brief network outages and so we changed our mounts to be “soft” mounts. Soft mounts will drop from beneath applications if the network connection is lot. Of course, this can be bad news for an application with open file handles. Soft mounts think of them more like love handles in that it can choose to ignore them rather than hang around awaiting perfection. This can result in unsaved data being lost – but if you haven’t saved something that is being written to a NAS device and then you move 76 miles away and expect it to magic the data with pixie magic then you probably deserve to lose that data anyway. What are you expecting, magic?

So here is my modified locationchanger script. With the soft mounts it pops up the dialogue asking if you want to disconnect when you move away from your WAN and automatically reconnects to the NAS when at home.

#!/bin/bash

# automatically change configuration of Mac OS X based on location

# author: Onne Gorter <o.gorter@gmail.com>

# url: http://tech.inhelsinki.nl/locationchanger/

# version: 0.4

# set -x

# redirect all IO to /dev/null (comment this out if you want to debug)

exec 1>/dev/null 2>/dev/null

# get a little breather before we get data for things to settle down

sleep 2

# get various system information

SSID=`/System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport -I\

| grep ‘ SSID:’ | cut -d ‘:’ -f 2 | tr -d ‘ ‘`

EN0IP=`ifconfig en0 | grep ‘inet ‘ | cut -d’ ‘ -f 2`

EN1IP=`ifconfig en1 | grep ‘inet ‘ | cut -d’ ‘ -f 2`

LOCATION=

# locations (use to be used Location name here)

ATHOME=Home

# detect HOME

ATHOME_SSID=<my SSID>

ATHOME_EN0IP=<home network address>

ATHOME_EN1IP=<home network address>

if [ -z $LOCATION ]; then

case $SSID in

$ATHOME_SSID ) LOCATION=”$ATHOME”;;

esac

REASON=SSID

fi

if [ -z $LOCATION ]; then

case $EN0IP in

$ATHOME_EN0IP* ) LOCATION=”$ATHOME”;;

esac

REASON=EN0IP

fi

if [ -z $LOCATION ]; then

case $EN1IP in

$ATHOME_EN1IP* ) LOCATION=”$ATHOME”;;

esac

REASON=EN1IP

fi

if [ -z $LOCATION ]; then

# still didn’t get a location, so do automatic

LOCATION=”automatic”

REASON=Fallback

fi

case $LOCATION in

$ATHOME )

echo “At home…”

if [ -z “`mount | grep ‘/Users/<userid>/Volumes/music’`” ]

then

mount_smbfs -o soft //<user>:<password>@<NAS>/music /Users/<userid>/Volumes/music

fi

if [ -z “`mount | grep ‘/Users/<userid>/Volumes/photo’`” ]

then

mount_smbfs  -o soft //<user>:<password>@<NAS>/photo /Users/<userid>/Volumes/photo

fi

if [ -z “`mount | grep ‘/Users/<userid>/Volumes/video’`” ]

then

mount_smbfs -o soft //<user>:<password>@<NAS>/video /Users/<userid>/Volumes/video

fi

;;

automatic )

echo “Not at home…”

;;

esac

# do some stuff here that needs to happen after every network change

# echo `date` “Location: $LOCATION – $REASON” >> $HOME/.locationchanger.log

exit 0

Of course, this now means that I have no excuse but to get on with coding the app…