Data scientists are explorers. They use Jupyter Notebooks, one of the most popular environments for data science analysis, to begin work toward creative solutions to big problems. But once those…
No one likes being dependent on someone else for their day-to-day work. It leads to inefficiencies, a cycle of endless change requests, and a general feeling of powerlessness.
Unfortunately, this type of dependency often occurs when data scientists and decision-makers are working together to tackle a problem. Data scientists hold the keys to data stored in complex databases, as well as the tool sets for analysis, so non-technical decision-makers have to send requests for new models, analyses, and visualizations that they cannot produce themselves.
At Civis, we love it when users ask data-driven questions. However, we seek to avoid the dreaded situation of dependency as much as possible by building tools that hand over data to business users and key decision-makers. They can explore trends and generate insights without depending on data scientists, providing data teams resources to improve their data pipelines, test out new models, and implement better approaches to analysis and visualization.
My team recently found ourselves with a dependency problem. We partner with our client to conduct weekly survey research, measuring KPIs such as brand sentiment, product usage and consideration, understanding and awareness of new features, and other data for strategic decision making.
Our client is very data-driven and technically savvy, which leads them to request many complex survey questions that cannot be answered with a single static memo or presentation. Rather, Civis has developed custom analysis methods and visualizations over multiple years. Since our client could not independently explore the data, Civis Applied Data Scientists had to generate custom tables and visualizations through multiple iterations of analysis and feedback, which led to inefficiency.
Realizing that we needed a way to give decision-makers the ability to explore the data and answer key questions quickly and directly, my team invested the past several months in building a custom application, Research Toolkit. Research Toolkit is a custom R Shiny application, hosted in Civis Platform, which has transformed the way that our team collaborates with our client.
As we collect new survey data, the responses are automatically fed into Research Toolkit, where our client can easily track key metrics and identify important trends. When we add a new question to the survey, the data is readily available for our client. They can immediately answer business questions and generate visualizations of the findings to present to key stakeholders. Rather than having the Civis team analyze the data based on our understanding of the business questions and then coming back with follow-up requests, our client now has an uninterrupted workflow of data exploration and analysis, all without writing a single line of code.
In the last few months of building Research Toolkit, my team has ventured into the realm of product development and gathered some best practices for building applications for data scientists and business users.