Collecting data is the easy part - what’s next?

Florian Douetteau, chief executive officer, DATAIKU SAS

Today, businesses in almost any industry generate staggering amounts of data every day. Whether it’s customer interactions, social media, personal information, transaction data, or data streaming in from any number of internet-connected devices as part of the internet of things (IoT) era, there’s more data available than many organisations know what to do with. We’re talking trillions and trillions of gigabytes every year - that’s a lot of data (to put it mildly).

Having a vast sea of raw information might feel empowering, but the act of just collecting data isn’t valuable - it’s the ease with which important insights come out of that data that matters. On top of that, it’s not enough to gain important insights just one time - the challenge lies in the fact that businesses have to figure out a way continually to analyse, iterate, learn from and reiterate, at scale, all the time. 

Businesses that don’t set themselves up for successful data projects are particularly disadvantaged because, ultimately, they won’t get what they need out of that data and it will be further discounted as a non-critical part of the business. On the other hand, companies that produce one successful data project are encouraged to continue and resources for hiring staff and expanding tools continues to grow.

While it might cost more upfront in terms of time or resources, building a solid data lab is critical to making sure that money being spent to collect vast amounts of data sees returns in crucial insights and improvements grown from that data. It can be daunting to get started, but business that take these three steps have the best foundation for ensuring a data lab and data projects are successful:

1.Get organized;

2.Build a team;

3.Arm the team with the right tool(s).

Get organised

Being organised seems like an elementary tip, especially for established businesses and experienced employees. But it’s worth discussing this aspect of setting up a data lab because it’s absolutely critical. The value in big data and with data labs lies in being able to deliver results quickly. Many industries don’t just need insights sometime - they need them in real-time.

So, if any part of the team, process, communication, or access to data is not working smoothly, projects get delayed, and they break down. To prevent breakdowns and get organised:

  • Know exactly where data comes from, how reliable it is, and how often it gets updated. Working with unreliable or stale data is worse than not doing anything with the data at all.
  • Establish reproducible workflows. Data labs should not break down when someone leaves the team or goes on vacation. Plus, reproducible workflows add efficiency and transparency.
  • Ensure access to all data is open and not siloed. Data labs should never have to spend time figuring out how to access datasets.
  • Create clear communication guidelines and paths. Members of the data lab should always be communicating and know what each other is working on. Additionally, communication with other teams involved in data projects or with clients should be clear - especially when it comes to aligning expectations with data science realities.

Build a team

In an ideal world (or maybe a dream world), companies would have unlimited resources to hire the best data scientists who would work together seamlessly to provide the best insights. In the real world, it’s not possible to hire an ideal combination of data scientists and analysts with similar backgrounds and who all work exactly the same way. 

Staff with business versus IT backgrounds traditionally don’t get along because they use fundamentally different tools and have different perspectives on business needs. But that doesn’t mean it’s impossible to build a great, efficient, effective team with resources from all backgrounds. The most important qualities in team members should be willingness to collaborate, effective communication, and willingness to learn. Team members should be able to look at projects holistically and prioritise the success of data projects over all else. 

Arm the team with the right tool(s)

One of the most important ways to ensure data science is successful in a business is to give the team what they need to test and deploy projects quickly and efficiently. Part of this, again, is providing the right tools for communication. If all members of the data lab are working in a silo in their own spreadsheet and operating independently of each other, the project is likely doomed to fail. Endless issues can arise, from versioning problems while emailing around content to more serious data security policy breaches.

But, communication aside, it’s also critical to give all members of the team (whether clickers or coders) the ability to work in a way that suits them and that they are comfortable with. Because the reality is, when you have a data lab full of business and IT staff (which is the ideal combination for differing business points of view), it’s very likely they will have completely different skill sets when it comes to manipulating and training data. So this means choosing a tool that:

  • Allows each member of the team to grow his or her skills individually.
  • Allows the team to clean data and develop/train models in the way most comfortable for them.
  • Provides transparency as well as different levels of access so that less experienced staff can work alongside the more experienced without making critical errors.
  • Allows data science projects to be deployed easily to production so that the business actually sees the fruits of the team’s labour.
  • Allows for quick and efficient onboarding of new hires, whether experienced or not.

From a sea of data to organised output

By following these key components of setting up a successful data lab, teams can continue to deliver value over and over again to a business, which further justifies its existence and allows for more resources and even more challenging projects. While going from simply collecting data to doing something valuable with it can be a scary leap, putting the right team with the right tools in place for the job will ensure success.

Please note that blogs are the sole view of the author and that they are not neccesarily the view of IQ ddg Ltd and should not be interpreted as advice. Please read our full disclaimer

Chief executive officer , DATAIKU SAS

You have....



to be GDPR compliant.

Register with us for all the news

Sign-up to hear about the latest DataIQ news, content and events.