trog’s haus

We’ve just set up a Zimbra server as a trial in the office to use as a groupware solution - hopefully avoiding the Microsoft tax of Exchange.

First glance seems good, but it really highlights the deficiencies of the open source world in this area - Thunderbird/Lightning/Sunbird need a lot of work to catch up to Outlook/Exchange. Looks like they’re on the right track though.

Anyway, the first thing I tried to do was connect my Thunderbird/Lightning install with the Zimbra server. The documented method on the official wiki only included reference to connecting to your calendar via ICS using the webcal:// format.

Not really sure what it’s doing on the back end there, but I immediately ran into problems deleting events - a quick Google indicates many other users have the same problem. Fortunately in some other forum post (which I can’t find any more) I saw reference to Zimbra getting CalDAV support, which seems to work fine.

I’ve updated the wiki page above, but figure it’s worth saying again because it doesn’t appear to be widely known.

The CalDAV URL for your calendar is just:

http://zimbra.url.goes.here/dav/username/Calendar

It should possibly also be noted that you cannot map your Tasks in this way - ICS seems to be the only way to do that.

I was looking at other people’s Facebook profiles and kept seeing the “Interested in” where you declare your sexuality. Sadly this isn’t one of the free-form text entry fields.

Imagine it on your profile! “Interested in: Your Mother”! It’d never get old.

Google’s LatLong blog has announced Google MapMaker, their foray into community-driven mapping for the Google Maps site.

It’s only open for editing in a few countries at the moment - places like Iceland, Jamaica, and the Barbados, which we can assume have limited coverage already. It seems safe to assume this will be expanded in the near future to include more areas.

The OpenGeoData blog has a few scathing things to say about it:Google have launched MapMaker, a kind of faux OpenStreetMap where they own all the data and you’re only allowed to map in certain Freedom Of Speech Zones.

Like Knol, the mooted ‘wikipedia killer’, Google refuse to acknowledge existing communities, trample on their hard work and lack the mindset to engage with an open project. The OpenStreetMap project is a ridiculously awesome open collaboration between people all around the world that are trying to solve the problem of outdated and expensive maps for GPS systems.

They have a Google Maps-esque system where you can look at the maps online, as well as a comprehensive Wiki that explains how to use the maps. Obviously there’s a lot of incentive for the normal consumer GPS devices to not be readily compatible with these things as they no doubt make a killing selling you maps - but hopefully this sort of data will inevitably become the norm.

Check out this animation showing how the level of available maps has grown over the last few years (it’ll look like a black page for a while, but give it a second - it’s loading).

As soon as time permits I would love to get involved; if not directly then certainly by looking at some sort of sponsorship or competition via AusGamers to get other people interested that might have a bit more spare time than I do.

In the meantime, if you’re going to spend your time working on maps that you want to make available, I’d strongly encourage you to give Google a miss on this one. Putting it on OpenStreetMap means your work will be available to be used much more freely - all work is licensed under a Creative Commons license.

Why doesn’t Google just make all their maps available under Creative Commons? Well, the most likely explanation is that they’re not Google’s maps - they presumably don’t own them outright and just have a license to use them on their site.

That said, there’s nothing stopping them from using the OpenStreetMaps system as a supplementary data system and encouraging people to contribute to it - well, other than the fact that this way they get to own all the data and use it as they see fit. Maybe they’ll open it up in other ways later down the track though.

I have a Gmail account which is based on my real name. Since the advent of the Internet, I realised just how common my real name is around the world, which really should have come as no real surprise - but for some reason it did.

Gmail doesn’t pay attention to full stops in email addresses. That is, alicebob@gmail.com is the same address as alice.bob@gmail.com. This was reported ages ago and has been the subject of a lot of discussion, because it seemed like a bug - why would you want to get email that’s not addressed exactly to you?

At least one other person bearing my name has signed up for a Gmail account. Not an unreasonable thing for them to do. They no doubt got through the sign-up process with few problems and managed to create a Gmail account.

Or at least, they think they did. Unfortunately, they also think their email address is the same as mine (albeit with a full stop in the middle of it somewhere). Not a real drama, until they start giving that email address out to friends and family and using it for things like hotel reservations and business.

After all my time on the Internet, I’m long accustomed to getting email that I don’t want. I get literally hundreds of spams a day to my work and personal addresses that I ignore more or less completely.

However, emails like this tend to bust through my spam filter, because they’re often very similar to actual emails that I’d get myself. They’re definitely not spam, but they’re definitely emails that shouldn’t have made their way into my inbox.

I go to pains to NOT read these emails, and almost always hit reply to let the sender know (after a quick check to make sure they’re not spam that crept through) that their email was misdirected. When its a personal email or something from a business contact, I usually get a reply thanking me. But when its an automated email from a mailing list or some other non-human sending process, I’m a little bit torn about what to do.

I don’t really want to get any more emails from here, but often my only recourse from an automated email is to click a link in it that takes me to some sort of online profile, helpfully logging me in to someone else’s account. While there’s probably no real damage I could do (I’m sure, for example, that I couldn’t get my alternate namesake’s credit card details), if I was a little more malicious I could probably at least make his life a little uncomfortable or embarrassing.

Needless to say, I don’t want to do that. I just want the emails to stop. So this raises the question - can I ethically (and legally) claim some ownership of emails that are accidentally sent to an address that - while it isn’t mine per se, is still delivered to me - so that I can try to make sure the sender knows they’re sending it to the wrong person?

Case study:

My alternate namesake created a profile on an international dating site. He, no doubt, put in all sorts of personal information into this site. I could have probably gone in and messed with his profile and made him a she-male seeking furry companionship or something, but instead I went through this arduous and painful process of trying to contact the site through normal means to ask they take me off.

This process took weeks - they floundered around for a while trying to verify it, told me they’d removed me, I still got emails every few days, floundered around again, etc.

It would have been vastly easier for me to just log into the guy’s profile and delete his account. But I couldn’t do that - even though he’d used my email address to (somehow) create a profile, it wasn’t my account.

While I went through the process then, this guy just keeps signing up for services using my email address - thinking it’s his. I’m getting all sorts of stuff I don’t want. At some point, I’m just going to start deleting them, meaning they’ll go into a black hole until he finally figures it out.

I’m sure this is happening to a lot of other users. It’s crazy how much personal information I could have obtained from this guy without him even having the slightest idea about it - if I was maliciously inclined.

Obviously, you should be careful when deciding when to give someone your email address - the last thing you want is spam or more useless crap filling it up. But remember - also be careful that you’re giving it to them correctly, because it’s probably worse that your personal and private information is going to someone completely different.

David Harrison of the UK, I’m talking to you.

(Further - as a web developer-type, I find it somewhat objectionable that several sites have let this guy sign up to various emails and services without first verifying his email address.)

Revision3 got back to me and have confirmed that their content is still indeed released under a Creative Commons license.

It should be noted again though that redistribution of their “early access” releases - the shows made available for subscribers - is still strictly forbidden by their license agreement. In the interests of keeping the free content coming it’s obviously in everyone’s best interests to respect this policy.

Revision3.com is one of the biggest content creators in the burgeoning “Internet TV” market. They create and distribute a variety of popular shows, including Diggnation (as in digg.com, the pervasive social bookmarking site), PixelPerfect, Internet Superstar, and many others.

The Revision3.com site originally sported the Creative Commons logo - you might’ve seen it around:

This logo implies that the works on the site are made available under a Creative Commons license. While there are several different types of CC licenses, the most commonly used license is one that allows redistribution of content. This is extremely handy for us in Australia, as it means ISPs can easily mirror their content to make it available for their users usage-free. This is kind of a big deal, as any Australian broadband user will tell you, and I suspect will become a bigger deal for people in the US as they clamp down more on wild bandwidth usage.

Recently the site got a big overhaul - I don’t know when; I only visit it every couple months when I see they have a new show, and the Wayback Machine only goes back to August 2007.

Conspicuously absent from the new design - the Creative Commons logo.

Closer inspection also reveals the Creative Commons information is now not available in their content RSS feeds. I am not sure if it ever was, but I have a vague recollection it used to be - backed up by the fact that the RSS XML includes a references to a Creative Commons namespace. However, the feed doesn’t appear to be using that namespace at all - there’s no license applied to the relevant sections of the RSS document.

Some Googling also reveals that there has been a bit of angst from the Revision3 guys towards redistribution - part of their revenue model is to make episodes available early via a private access system to paying subscribers. However, it seems some dastardly types would get those early release videos and then distribute them publicly, citing the Creative Commons as their reason for doing so. Legal issues aside, it’s obviously a douchebag thing to do, and it’s unsurprising that Revision3 took legal action to try and stamp it out.

A quick read of the the Terms of use for the revision3.com site doesn’t really mention much about the content. It does specifically mention that you can’t redistribute “member only, not publicly released downloads” - hopefully putting a stop to people leaking those subscriber-only early access videos.

There’s one reference to Creative Commons:

By uploading, submitting or otherwise disclosing or distributing Content of any kind at or on the Site or otherwise through the Services, unless source quoted, you represent and warrant that you own all rights in the Content and you agree that the Content will be subject to the Creative Commons Public Domain License, available at http://creativecommons.org/licenses/publicdomain/. All of our non-member downloads comply with creative commons 2.5.

(Emphasis mine).

This raises the question of what exactly constitues “non-member downloads”. It seems that is it probably the video content that we’re after, but it’s just a little bit too vague, especially in the context of a legal agreement.

The current situation and the net result of the above is that, at least to my eyes, the licensing of the Revision3 content is now more nebulous than it was previously, simply because of the removal of the Creative Commons logo.

While it is possible that they’re trying to distance themselves from the Creative Commons license with a view to better control and accounting of the distribution of their content, it seems that this is. It’d also be a huge shame, because it would probably dramatically decrease their audience, the size of which is arguably a direct result of their distribution model.

Last week I emailed Revision3’s official contact address to try and clarify this position - as yet I haven’t received a reply, so I have emailed them again today (from a different address, in case spam filtering was an issue) and I have also posed the same question publicly on their forums.

While I was staring at the output of an Apache server-status page from a server that was getting smashed by hundreds of users simultaneously downloading the latest Ubuntu release, I was thinking about how a web server reads files off the disk.

When multiple clients are connected all downloading the same file - as often happens, given that we host many popular files - they’re all downloading at varying speeds (depending on their Internet connection and server load), so the web server is reading different parts of the file off the disk and sending it to heaps of different users, all at different rates.

This results in a lot of seeking as the hard drives spin furiously to try and keep up with the demand from the HTTP server, which in turn is trying desperately to service the requests of users. This turns into load, which turns into slower downloads.

Our experience is that - under peak usage in general conditions for a typically configured web server - users can suck down files much faster than we can send them out. Even though we have bitchin’ servers on awesome connections, it only takes so many cable users with their 30mbit connections trying to suck down at full capacity before we’re hitting the wall - bear in mind 30 such users will use around 900mbit of bandwidth!

The obvious solution to this problem is to stuff the file into RAM and serve it from there. There are many ways to do this - RAM drives, mmapping with an Apache module, etc. This is probably the best way, but requires you have a buttload of RAM available to do it for large files.

The Crazy Idea

The crazy idea I was thinking about was - instead of having the disks bottleneck up the webserver from all the different clients downloading at different speeds in different parts of the file, you come up with a way to synchronise the file reads across user downloads.

The desired outcome of this idea is to reduce the overhead on the disks by sacrificing the download speed of some specific users. Obviously, such a method would result in users with faster downloads getting worse performance - but there exists a point at which the server is loaded so heavily they’re getting slow downloads anyway.

The complicated part (at least, the only complicated part I can think of - this idea might completely suck for other low-level reasons I’m not aware of/haven’t considered) would be keeping track of all the users, their download speed, and the current “position” of their download (ie, the amount of bytes they’ve already been sent), then slowing down the faster connections to bring them in line with the slower ones - so the web server is only reading from the disk ONCE, but sending it to MULTIPLE users.

With some clever programming it seems feasible to me that you could come up with rough user groups as well - for example, users downloading between 50-75 kbytes/sec could all be lumped together and set to download at 50kbytes/sec, users downloading between 200 and 300kbytes/sec could be grouped and set at 200 kbytes/sec, and so on.

Optimally, such a system would be smart enough to only turn itself on when it detects disk activity reaching a certain threshold - so under normal circumstances everyone is downloading at full speed, but once processes start getting stuck waiting for I/O, this magical new mode would get enabled, multiple connections for the same file in the same speed categories would start syncing, and then disk wait times would go back down.

I would love to know if such a system has already been implemented in an existing web server (I’ve had a quick search and can’t find anything like it), or if there’s any reasons why such a system would be impossible to implement - or if indeed it’s just a dumb idea.

The AusGamers Grand Theft Auto 4 review - on our new whiz-bang website - is now up. Steve has given it a thorough playing before deigning to write a single word, I think with excellent results. If you want to really know how the game plays before you drop your hard earned cash, then you should definitely check it out.

It was interesting to see how all the major players in the review space had their reviews up. A brand like GTA is going to sell no matter what they put in the box and put on the shelves, so I can’t help but feel that it almost doesn’t matter what people write about it - everyone is going to buy it regardless. But at the same time, it’d be nice to think that critical reviews of entertainment software are going to be done by someone that has at least spent several days playing the game - especially when it is a title crammed with such variety as your average GTA instance.

The echoes of the release of GTA4 will continue to rebound around the universe for months, if not years. A title that cost around US$100 million to make and raked in around US$500 million in sales in the first few nanoseconds after release - well, at title like that is something that will affect the industry in significant ways.

EA has definitely caught a taste of blood in the water, and are circling Take Two to try and get a few bites of that tasty Grand Theft Auto meat that appears to be ever-so-slightly out of their reach.

Another effect is a little more personal, but I suspect it’s actually fairly common. GTA4 is, for me, the killer app that is going to drag me kicking and screaming into the “next generation” console world. The reluctance on my part is really just because I know what is “next generation” in consoles is really “several generations old” in terms of PC hardware - while I enjoy running down pedestrians from the comfort of my couch, I can’t deny I’d rather be paying GTA4 on the PC.

But it looks like that dream is a while off, so I’ve reconciled myself with my fate - I need to buy an Xbox 360 (I have ruled out a PlayStation 3 as part of a general anti-Sony bias that shows no sign of abating).

Now, I don’t want to go to all the trouble of buying a next generation console and the hottest game of the year only to play it in a blurry mess on my old CRT TV - so that means I need to get a new TV. As far as console accessories go, this is a big one - instantly my ~$600 next generation upgrade becomes a massive investment, not just into my game playing but into my TV watching. I need to factor in how big a deal 1080p is, whether or not I want a BluRay player, should I get surround sound…

I tried to solve all these problems in a single sweep last weekend, making the painful decision about what TV I wanted, committing myself to spending a large sum of money all so I could play one particular game. By the time I’d committed to the decision I felt like a large weight had been lifted off my chest - I was finally moving forward into the next generation, so my peers would no longer look pityingly upon me when I said I only had a 80cm CRT TV that wasn’t even widescreen on which I played GTA: San Andreas on a PlayStation 2.

Sadly, about 10 minutes after I’d decided to jump in feet first, it turned out that the plasma TV I had decided on was not available in the store - or any other stores from around town. Instantly, my whole plan became unravelled - I’d spent ages researching this particular TV and had my heart set on it as I knew it was a pretty decent one. Should I risk gambling on another just to solve the problem? Or go back to the research drawing board?

The adrenaline rush and mental effort of committing to this purchase had drained me of all ability to think further on the issue, so I walked out of the store a broken man - no doubt much to the disappointment of the salesman I was dealing with, who was looking forward to a nice combo sale to wrap up his day.

So I’m back at square one - no GTA4, no nice new plasma, no Xbox 360. I’ll probably go out again this weekend after doing some more research on TVs, but reflecting on the whole process and then extrapolating it out to the Rest of the World makes me curious - how many other people are going through similar pains because of the lure of just one game? Are we a small group of just crazy people with a bit too much disposable income? Or is this game such a big deal that it has the potential to not only drive people into the next generation console world, but also move people into the next generation television world, prompting a more significant uptake of big screen, high definition televisions than ever before seen?

Either way, the impact GTA4 has had already in terms of its own sales - regardless of anything else it’s done to or for anyone else - is simply mind-boggling. The effect it has had on gamers is no less astounding - every website is awash with stories of how great the game is, how much it’s selling, how many people are going to go crazy and kill people because of it, how it’s not going to incite riots, etc, etc.

And I have only played it for a few minutes. When will I get my turn?!

A follow-up to my earlier post about problems with the Linksys WAG325N ADSL modem device - after going through a couple of hours of Live Chat with the Linksys team, they have finally admitted defeat and asked me to send it back in, as it seems a hardware fault is responsible for at least the 10mbit fallback problem.

Several other users are still reporting the problems; some have also sent theirs back only to get another faulty unit and have to send it back again - so hopefully at some point in the process we’ll end up getting working units.

I am hoping there’s just a bad batch of these things out there and it’s a relatively simple hardware fault that can be magically fixed. This is my first time buying Linksys hardware so needless to say I’m a little disappointed; I’m especially not looking forward to sending this back and being without a modem/switch/wifi device for the duration.

…and available for download for BigPond Broadband customers on BigPond Files.

At the moment I’ve just got all the major/popular ISOs up - i386 and AMD64 architecture ones. The SPARC, HHPC and PowerPC ones I’ll do a bit later (although if you really want them now, just hit the File Request page to let us know).

I believe there’s also a way to upgrade an existing distribution using the repository mirror (BP users only) we have, but I’m not sure on the process. (Update: Nathan has provided some instructions on the process on his site.)

Unfortunately given the release happened on a public holiday it caused a bit more havoc than normal; I underestimated the demand (which was HUGE, which is great to see) so for the first hour or so downloads were a little bit slow until I got another distribution server into the mix - then that got hammered as well.

I’m interested in feedback from users having performance problems; feel free to post details about your issues here and I’ll see if I can figure out a way to speed things up for you.