Web application development with R Shiny: how to scale data products quickly and reliably

Oct 11, 2019·
Gabriel Teotonio
· 2 min read
The Developer’s Conference 2019 - Recife
Abstract
R is a programming language developed initially to deal with statistical computing and graphics. However, over the years, many features and packages have been added, offering users more tools to create entire environments. An example of this is the Shiny package: a framework for building web applications using R. Shiny applications are a powerful and easy way to share and communicate your analysis and allow people to interact with it. At Inloco, we have a series of applications built using such a framework, both for external and internal customers. In this lecture we will share the challenges and solutions applied during development.
Event
Location

Universidade Católica de Pernambuco, Recife - Brazil

At Inloco, we often need to create data validation applications very quickly for the validation process of the customers and teams involved. And because of this speed, these data pass only through the hands of our data analysts and scientists before being presented or exposed. About 16 TB of data are processed daily and creating architectures that support that amount is essential. One of the challenges is to create dashboards and applications that can be robust in various aspects such as scalability, security and performance in a short period of time and give the freedom for data scientists to transmit their analysis and development in a transparent manner.
The lecture will initially address the Shiny framework, exploring its characteristics and exemplifying how it can be an outlet for sharing analysis and data modeling. In addition, it will be shown the possibility of using Shiny in the construction of data products without losing scalability and gaining speed in product development. We will now show a real use case for Shiny at Inloco: OOH Planner, a platform for analyzing the performance of offline media campaigns, and we will share the challenges we had in implementing it. Soon after, time will be spent explaining how to structure the backend and deploy these applications. At first, focusing on the choice of the database and good practices in determining the schema of the tables, in particular. Also used are tools used to build queries, perform performance and responsiveness tests, such as dplyr and shinyloadtest. In a second step, the deployment structure using docker. How to create containers for applications in Shiny and management of dependencies in R. Use of plumber APIs as good practice for code reproducibility, given that, possibly, applications will be processing classification and prediction models in their backend.