Encouraging More Flexible Government Procurement

Last year, Jon Stewart interviewed Nancy Pelosi on The Daily Show discussing how government procurement sort of sucks. A lot of it was the result of issues with the Affordable Care Act’s healthcare.gov site, which struggled at its launch with a series of technical problems that would make any web developer wince.

The full interview is, as always, an excellent watch.

While the scale of the deployment and the bizarre state of the American healthcare system is almost completely alien to most of us in Australia, there was one part of the interview that resonated very strongly with me as a small business owner who has tried to approach government to solve their technical problems.

Stewart brought up the point that the complexity of the procurement process limits the accessibility of it to large companies – small companies are excluded simply because they don’t have the resources to devote to the tendering process.

The relevant text for those nerds (like me) who dislike video:

Stewart: “Obama’s IT guy, small company, clearly a brilliant guy – he arranged all of Obama’s Internet campaign stuff … That guy couldn’t figure out the process. He couldn’t figure out how to bid for that contract.

He said it was a 300 page document and it seems like it’s obscured like that purposefully so that the larger companies have an advantage because they have teams of lawyers and things that can do it.

I’m presenting it as – “Do we have a foundational problem? Is there a corruption in the system that needs to be addressed to give us the confidence that moving forward, we can execute these programmes better?”

Pelosi: “I don’t think there’s a corruption. There may be a risk aversion with going with the known and then just not being entrepreneurial enough to say, question whether that is really going to do the job.”

If you’re a small ICT company in Queensland looking to expand your customer base, you’ve almost certainly looked at the QTenders site every now and then to see what stuff is on the table.

In general I’ve found the tender documents to be very well written. They’re well organised; it’s clear what they’re trying to do and how they want to do it.

Unfortunately many of the documents are very long – just reading them can be a full day exercise.

For a small company, responding to a tender is the work time equivalent of running an entire project. They often require several people to work on. Given the often short timeframes for tender response – maybe a month – it can mean taking up a huge amount of time servicing other customers.

Arguably, this is just part of the sales process. But large companies have entire teams of sales people that do this. A mate I played soccer with actually lead a team (in mining or something, not IT), the sole purpose of which was to just reply to government tenders, having (over the years) developed a keen insight into how many they’d win versus how many they’d lose.

If you’re a large company, you can afford to do this. You can build your sales process over time and simply absorb the losses incurred by blowing a few human-months of time on responding to a tender. But if you’re a startup or an SME, you’re potentially losing 10-20% of your entire sales effort for a YEAR, working on something that you’re almost certainly not going to win – because the big companies have the tender process so streamlined. They basically have human machines for churning out responses to these things; responses that they probably know statistically how likely they are to win.

The tender process is an important part of open government procurement. But watching the giant companies that win the tenders fail again and again costing taxpayers billions of dollars is starting to wear a bit thin. We need to look at better solutions – dividing large projects up unto smaller and more manageable components and figuring out a way to let our SMEs and startups compete effectively for them, instead of them being excluded because they simply can’t afford a seat at the table.

Idle thoughts on Mozilla, Firefox and Thunderbird.

I’m a big fan of Mozilla and have been a Firefox and Thunderbird user and advocate for many years. The last few years of development on these projects have left me somewhat disillusioned. Firefox seems to be slowly converging on Chrome, with disruptive UI changes making each update irritating, rather than exciting. Thunderbird, despite regular updates, feels like it has stagnated.

I feel like Mozilla have already won the browser wars. I’d love to see more effort going into Thunderbird and Lightning – groupware being something that open source is still really struggling with despite many valiant efforts.

It’s hard to convince myself this is a big deal; web-based groupware is pretty good these days. But I use Thunderbird every day. I’ve become almost dependent on a bunch of excellent extensions. I love having the option to be in complete control of my email.

I wrote about this in a bit more detail at Medium.

Update:

This seemed to resonate with a few people – ended up being the 18th “Most Read” article on Medium and was featured in their Technology section. Also spawned interesting discussion on reddit and Slashdot.

forgetting-firefox-stats

WPUpCheck – keeping WordPress up-to-date

WordPress is a great piece of software, but it’s popularity and superficial ease-of-use combined with the fact that computers are hard means running a site on WordPress is not always as simple as it seems.

I wrote about some of the ways to reduce the risk with WordPress over on the Mammoth blog a while back.

One of the biggest risks is a WordPress site that is out of date. There are three main components to the WordPress site:

– Core: the base functionality you get on a brand new installation.
– Plugins: all the other stuff you install for functionality
– Themes: what things look like

Each component is typically its own code base, requiring maintenance and updates. Many users only know they have updates available when they log in – and many of them don’t log in that often, especially if their site is primarily static.

WPUpCheck is a simple Windows tool that polls a WordPress site periodically to check for updates in any of these three components. If it detects available updates it will bring it to your attention via a balloon in the system tray.

The goal is simple – try to ensure a larger number of WordPress sites are no longer running obsolete, out-of-date, potentially vulnerable software.

Anyone interested in beta testing it can download WPUpCheck now.

Irritation, Confusion and Deception in the Android Permission System

The Android permission system seemed like a great thing at first – crystal clear understanding about what each application can do on your device. However, with the latest round of updates, it has become even clear that the permission system is confusing at best and deliberately harmful for users at worst.

If you’re a privacy/security conscious, tech savvy user, you might end up poring over individual permissions for each application before deciding to install it. This can be time consuming – even experienced users might have to hit up Google to see exactly what a permission means. Sometimes you might even veto an app or an upgrade because of onerous permissions. But if you watch normal civilians use their phone, they barely even glance at the security options, during install or upgrade. I’ll even confess to giving up dealing with permissions and just installing an app because I felt I “needed” it.

Everything looked good for a while back in Android 4.3 with the discovery of the permission tweaking system which lead to “App Ops”, allowing users to selectively enable/disable permissions on a per-application basis, granting total control over what they had access to. This was perfect, but sadly was pulled in a later release, with Google saying wasn’t intended to be available. The clever hackers over at CyanogenMod restored it as part of their release, but there’s basically no way to have fine-grained control over your apps – meaning you accept everything permission that they want, or you do without.

Here’s an example – I just received notice of an update for PasswdSafe:

2014-09-29 17.21.57

Now, the “What’s New” notes are provided by the application developer. Sometimes they are nice enough to include details about why there was a permission change, but I would say this is generally pretty rare.

Here’s what happens when you click ‘Update’:

2014-09-30 01.04.50

You only get this sort of popup when there’s a permission change. So this popup implies – to me, at least – that there’s some sort of permission change involving media, or files, or whatever.

But if you scroll to the bottom of the app page to click on the ‘Permission Details’ icon, you’ll see this:

2014-09-29 17.22.46

Here you can see the new permission actually has something to do with Near-Field Communication! (So in this case, the changelog provided by the developer actually does relate directly to the permission change, though it’s not really clear until you manually inspect the permissions like this.)

While you can find the information, this current flow is totally broken. It’s infuriating that the recent change now seems to actively hide and thus mislead the user.

This UI change is a massive step back for Android; it compromises the ability of the user to make informed decisions about the software on their device.

Setting Up Infobox Templates in MediaWiki v1.23

This article explains how to add the “Infobox” template to your MediaWiki installation. It is primarily intended for people who have installed v1.23 from source.

This is an updated version of this older post about setting up Infobox on earlier versions of MediaWiki. It is basically the same but has been modified to be suitable for the current (at the time of writing) version of MediaWiki, v1.23. Please see the older post for more info and background as well as helpful commentary from other users in different circumstances.

Here are the basic steps necessary to add working Infoboxes to a freshly installed version of MediaWiki. Note that the original steps required the install of ParserFunctions; this is no longer required as it ships with recent versions of MediaWiki by default.

  1. Download the Scribuntu extension into your extensions folder and add it to your LocalSettings.php as described in the ‘Installation’ section.
  2. Copy the CSS required to support the infobox from Wikipedia.org to your Wiki. The CSS is available in Common.css. You’ll probably need to create the stylesheet – it will be at http://your_wiki/wiki/index.php?title=MediaWiki:Common.css&action=edit – and then you can just copy/paste the contents in there. (I copied the whole file; you can probably just copy the infobox parts.)
  3. Export the Infobox Template from the Wikipedia.org:
    1. Go to Wikipedia’s Special:Export page
    2. Leave the field for ‘Add pages from category’ empty
    3. In the big text area field, just put in “Template:Infobox”.
    4. Make sure the three options – “Include only the current revision, not the full history”, “Include templates”, and “Save as file” – are all checked
    5. Hit the ‘Export’ button; it will think for a second then spit out an XML file containing all the Wikipedia Templates for the infobox for you to save to your PC.
  4. Now you have the Template, you need to integrate them into your MediaWiki instance. Simply go to your Import page – http://your_wiki/wiki/index.php/Special:Import – select the file and then hit ‘Upload file’.
  5. With the Templates and styles added you should be able to now add a simple infobox. Pick a page and add something like this to the top:{{Infobox
    |title = An amazing Infobox
    |header1 = It works!
    |label2 = Configured by
    |data2 = trog
    |label3 = Web
    |data3 = [https://trog.qgl.org trog.qgl.org]
    }}
  6. Save, and you should end up with something that looks like this:

Farewell Robin Williams, an oldschool video gamer

Very sad to hear about the passing of Robin Williams.

I loved his movies, but I also loved his lesser-known passion for video games.

He did an interview years ago where he talked about playing Quake – I’ve held on to this mp3 file since I got it and always enjoyed listening to it every time I stumbled across it when I was deep-diving in my archived files.

It’s embedded below.

The failure of the Australian state to gracefully deal with copyright

Nic Suzor has written another insightful piece for The Conversation which looks at the Australian government’s leaked plan to combat piracy.

In conjunction with research assistant Alex Button-Sloan, the article users several words that I think fairly accurately sum up this plan: “unrealistic”, “vague”, “unlikely to help”. Some of the outcomes are equally depressing: “likely to raise the price of internet access”.

Everyone in Australia should read this article. This plan does nothing to address the root cause of Australian piracy – the simple fact that content is not made available on the same terms as it is in other markets.

The fact that this plan could “massively increase the potential risks for companies that provide legitimate services” also fills me with dread; Mammoth – in addition to being an Internet hosting provider – has been actively involved in legitimate content distribution since the late 90s, across video games, music, and movies. With our clients, we have struggled for years to try to figure out how to make sure Australians have access to the latest content.

These are not technical problems; they are licensing and rights problems caused by media companies carefully choosing how to maximise their profits. The fact that they are now cooperating with out government to literally change the laws to better suit their business model is kind of irritating.

Disruption: why is the US so different for startups?

AVC’s¬†Fred Wilson wrote a post recently looking at platform monopolies and why they’re great targets for disruption.

It’s clear that almost everything about the US is different for startups. A lot of it is just its sheer size – having ten times more people than Australia changes everything. Probably the biggest complaint in Australia though is the lack of VC funding and a reduced appetite for risk.

It’s not clear to me which one of these is the chicken and which one is the egg. Is the startup scene awesome because of the VC? Or does the VC exist because of the startups?

Since moving to the US, the most striking thing I’ve noticed here is the prevalence of these large, giant “platform monopolies” – but not just in the tech space. Many aspects of daily life in the US seem to be managed by these giant institutions. Navigating these large institutions is cumbersome and tiring (especially if you’re a foreigner and have no idea how things are glued together) – and as a result, there are a lot of middlemen that try to make the process easier.

For example, if you’re looking for health insurance, trying to deal with the insurance companies is a real pain. So, there are hordes of health insurance agents and brokers that sit in between you and the insurance companies to try to make that easier. They take a small fee.

If you’re starting a company and need to pay your employees, you need to be aware of the specific tax issues in your state and federally. Even for a one person company this is challenging; I can’t imagine what a headache it’d be if you were trying to set up in multiple states. But don’t worry – there are many payroll companies that specialise in this, all for a small cut. (The first time I ran a payroll here in the US I was staggered by the fact that the company used UPS to courier me an actual payslip on an actual piece of paper. )

Near where I live there’s an entire business that appears to exist solely to cash cheques – a form of payment that is basically extinct in Australia, but because it’s so common here there are these weird cheque companies that exist. I assume that they must make a bit of money on each cheque they cash.

The list goes on. Almost anything complicated you want to do, there’s a support system that seems to sit in the middle to make your life easier.

Fred Wilson comment on this in his article:

The Internet, at its core, is a marketplace that, over time, removes the need for the middleman. That is very good news for the talent that has been giving up a fairly large part of its value to all of the toll takers in between them and their end customers.

This is where a lot of the interesting disruption is happening in the US – people tired of these old, monolithic systems looking to make a change. The payroll issue is being addressed by companies like ZenPayroll. I have no idea what people are doing about cheques but I thought it was entertaining to see that you can deposit a cheque here now by taking a photo of it. And everyone¬†is trying to crack the healthcare nut – health industry startups abound and appear to be highly favoured as targets for funding (hey, the US spend more money per person on healthcare than any other country – it’s good business).

There are many other examples, including ones with global application – Uber and Lyft are probably the most significant and most recent examples; their impact on the transport industry is still being felt. The music industry is another one – all those poor record company executives are going to be a relic of days gone by.

Every country, every society, every community has its own entrenched systems, their own way of doing things. The bigger the environment and the longer they’ve been around, the more likely you are to find middlemen. Technology makes it possible to go back and re-evaluate the old way of doing things. Better automation, communication and integration means a lot of the old ways of doing things are ready to be swept aside – and that’s where a lot of the disruption is happening.

Ultimately I think that it’s this potential for disruption that makes the US startup scene so vibrant and interesting. It’s the fact that every opportunity to change the status quo has the potential to pack a huge punch – even capturing a small percentage of the market here can mean a big business. The bigger the established players in a sector become, the more naturally resistant they become to change – meaning a lot of opportunity for smaller players to start to carve out a niche.

When you encounter someone saying “this is just how we’ve always done it”, pay attention – you might be on the verge of something big. These are tough fights to pick, but – as is being shown by Uber – if you make the right moves at the right time with the right technology, you can revolutionise a space.

MySQL Server fails to upgrade in Debian/Ubuntu

Just did a relatively normal ‘apt-get upgrade’ on my simple WordPress HHVM test server and got a horrible bunch of errors, leaving it in a non-working state:

E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install. Trying to recover:
Setting up mysql-server-5.1 (5.1.54-1ubuntu4) …
start: Job failed to start
invoke-rc.d: initscript mysql, action “start” failed.
dpkg: error processing mysql-server-5.1 (–configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
mysql-server depends on mysql-server-5.1; however:
Package mysql-server-5.1 is not configured yet.
dpkg: error processing mysql-server (–configure):
dependency problems – leaving unconfigured
Errors were encountered while processing:
mysql-server-5.1
mysql-server

There are many reports of this on various forums and blogs with many different solutions. Here’s mine.

In my case, MySQL logged the following error:

May 4 07:04:22 trog mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
May 4 07:04:22 trog mysqld: 140504 7:04:22 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
May 4 07:04:22 trog mysqld: 140504 7:04:22 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
May 4 07:04:22 trog mysqld: 140504 7:04:22 [Note] Plugin ‘FEDERATED’ is disabled.
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: The InnoDB memory heap is disabled
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Compressed tables use zlib 1.2.7
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Using Linux native AIO
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Initializing buffer pool, size = 128.0M
May 4 07:04:22 trog mysqld: InnoDB: mmap(137363456 bytes) failed; errno 12
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Completed initialization of buffer pool
May 4 07:04:22 trog mysqld: 140504 7:04:22 InnoDB: Fatal error: cannot allocate memory for the buffer pool
May 4 07:04:22 trog mysqld: 140504 7:04:22 [ERROR] Plugin ‘InnoDB’ init function returned error.
May 4 07:04:22 trog mysqld: 140504 7:04:22 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
May 4 07:04:22 trog mysqld: 140504 7:04:22 [ERROR] Unknown/unsupported storage engine: InnoDB
May 4 07:04:22 trog mysqld: 140504 7:04:22 [ERROR] Aborting
May 4 07:04:22 trog mysqld:
May 4 07:04:22 trog mysqld: 140504 7:04:22 [Note] /usr/sbin/mysqld: Shutdown complete
May 4 07:04:22 trog mysqld:

 

I’ve bolded the important lines – in my case, MySQL was starting and trying to allocate some memory for InnoDB, but it was failing. So when the package manager updated MySQL and restarted it, it would simply exit as failed, returning failure as part of the update process.

I solved this hackishly (but easily) by stopping a memory-hungry process on the server (HHVM), then re-running the upgrade. Without HHVM running there was plenty of spare memory, the update was able to successfully restart MySQL, then I just restarted HHVM.

This is of course indicative that this system probably doesn’t have enough memory – but it’s a small-scale test environment. So if you’re looking for a cheap easy fix when Debian/Ubuntu fail to upgrade MySQL, this might work.

Initial Binary Lane Feedback: Positive

So far the feedback for the newly launched Binary Lane has been really positive:

Feedback on Whirlpool has been similarly positive:

bl-do

bl-wp-1

… and also a good thread on LowEndTalk.com, a developer-focused community for infrastructure services.

There’s still a lot of work going on behind the scenes. New features are still be developed – most recently, a new BYO ISO system, allowing people to install their own operating systems, including things we haven’t supported before like FreeBSD.

Stay tuned.