Hacking big data — there’s never been a better time

While the term “data science” has been around since the 1960s, it’s only in the last 10 years that the job of data scientist has emerged as “the sexiest job of the 21st century” (according to the Harvard Business Review at least). 

Today, all of our major universities offer data science courses, and job-finding websites show hundreds of open positions looking for staff with these skills.

In addition, recent advances in easy-to-use software tools for data analysis and visualisation have now been joined by widely available, often open source, machine learning toolkits.

And while Google AlphaGo has now retired, having humiliated humankind by beating our greatest Go player, artificial intelligence systems are increasingly delivering results that seem so futuristic they could be described as magic.

Why should we care about data?

Consider the role of data in a broader context: with the traditional business model of old media, where journalism was funded by the “rivers of gold” provided by classified advertising, more or less gone, we’re increasingly witnessing global events such as the US political win influenced and supported by the spouting of “alternative facts”.

We can no longer challenge such events by simply rejecting news or information as “fake news”; we need to base our arguments on data.

In many instances, that data comes from government. Thankfully, here in Australia, we have a strong culture of open government that continues to be supported by our political leaders and senior public servants.

At this year’s GovHack event, running this weekend, contestants will get an opportunity to make creative use of available government data. The benefits of initiatives like this are twofold: by publicly creating knowledge out of data, government gets encouraged to make more data available in the future, while simultaneously getting feedback on the format and usefulness of existing data.

Better yet, in many instances, contestants are going on to commercialise their projects and taking new skills back to their workplaces.

Startups that found a break in hacking data

Participation in hackathons, such as GovHack, can be a launching pad for a highly sought after career, with participants developing valuable skills and providing key insights about life in Australia.

Here’s some examples:

• “Rain Parrot”, a prize winner from 2014, is still going strong and is regularly the top paid weather app in the Apple App Store;

• Chamonix, which has won three GovHack awards, has developed Healthi, which is used by more than 1000 consumers and health professionals. It’s an idea that originated when the Australian Government opened a digital portal for third party providers to connect to the My Health Record System; and

• The Sydney Citizen Journalism Meetup was founded to lobby for better data after Rosie Williams created the first ever implementation of a searchable federal budget, which was inspired by an early GovHack project called theopenbudget.org.

This is part of a 12-part series on hackathons that SmartCompany is publishing in association with GovHack.