The benefits of in-house analysis at Tech Nation UK

Toni Sekinah, research analyst and features editor, DataIQ

The Tech Nation report is an annual audit of the UK’s tech sector “with an international angle”. The 2018 edition showed that London ranked third in the league table of global tech eco-systems and highlighted dozens of tech clusters across the country, as well as “digital suburbs” such as Guildford and Aldershot.

To produce the report, Tech Nation UK has a host of data partners, including Adzuna, Startup Genome and Eventbrite, among others. Senior insights manager Henri Egle Sorotos explained that a five-strong insights team gathers data on everything from talent and skills to investment, and dives deep into Meetup, GitHub repositories as well as Stackoverflow questions and data from the aforementioned partners.

Digital investment in technology by UK cityTwo years ago, when Tech Nation UK didn’t have an insights team, the data for this report was compiled in a very different way, with work being outsourced. This was problematic because it meant relying on the relevant person from the outside company to remain in their job and not take annual leave for the duration of the project.

Previously all the research was taken to a partnership organisations such as Nesta, which had some “quite serious implications” financially, as well as in terms of internal knowledge retention and innovation.

Egle Sorotos also said that in the past the Tech Nation report was very static and used a closed methodology, so it missed out some of the nuances of the UK tech scene.

"We've made all the data open."

However, despite the high initial cost of bringing insights in-house, Egle Sorotos believes there are no longer cross-sectional silos with the data from different years not linking up. “We've made all the data open so that you can now bring it together on a year-by-year basis to provide longitudinal analysis,” he said.

With all the analysis taking place in house, Tech Nation now has full control of the methodology, “something that was previously quite tricky for the organisation.”

Tech Nation UK’s commitment to make all its data open and take a web-first approach to its output means that now “we don’t produce reports in a PDF format any more, everything is driven by JavaScript products.” There is a series of JavaScript-driven visualisations online that move dynamically, but viewers can also look at the raw open data underneath.

The new insights team has not only revamped the annual report, but also is also helping with the recruitment of start-up companies to Tech Nation UK’s accelerator programmes.

"Querying the Companies House API has been really innovative."

London's digital investment by sector according to the Tech Nation Report 2018“We've recently been doing call-outs for our early stage company. I've been querying the Companies House API to get every single company that is remotely eligible for that in the UK and it has been a really innovative technique. We've managed to get 12,000 unique companies that could attend that accelerator programme.”

Data is drawn from the Office for National Statistics as well as APIs and by scraping using Beautiful Soup, a Python library for extracting data from HTML and XML files, as well as using open data and organisations like Eurostat. Then R and Python is used to do everything from pivoting data to running descriptors and doing a bit of machine learning. Finally, the visualisations are produced using D3 and Data.World.

Egle Sorotos said that his role as a quant is to do programming. The rest of his team consists of an insights lead who “does the qual and some quants,” an ecosystem insights specialist who is “purely a qual researcher,” and a data design executive who does “wonderful and pretty visualisations,” as well as a freelance data scientist.

"Organisations are bringing research and data client-side."

He sees what Tech Nation UK has done with bringing the insights team in-house as part of a growing trend. “Data-driven advocacy is ubiquitous. It’s incredibly important. More and more, you are seeing organisations that used to outsource lots of research and data bring it client-side and ensure they are retaining the knowledge internally.”

Henri Egle Sorotos was speaking at the Open Data Institute.