DeepOps: an approach to project management in the age of AI
Data science projects are increasingly complex, relying on extensive, high quality information to build useful applications. Data scientists can spend up to 30 per cent of their time building infrastructure, and are often limited by inefficient or outdated tools. They must then carry out extensive version control to manage any changes made to the information.
In the age of deep learning and AI, data science will only become more complex. How do data scientists manage deep learning operations without drowning in the data?
The answer could lie in DeepOps (deep learning operations). DeepOps involves a collection of tools and methodologies to help build faster and more reliable deep learning pipelines. DeepOps begins with automating the data processes needed for AI driven apps, taking on management tasks that compromise developer creativity. This includes tracking all code changes to form a complete history, building a system that can take on new data without corrupting the information that already exists, and automating the creation of pre-configured computers.
DeepOps has its origins in DevOps, an approach to software development that merges the skills of developers with IT operations teams. An important aspect of DevOps is that it encourages communication across organisations, drawing on a diverse range of skills to speed up time to market and improve quality. In the next iteration of software development, the deep learning community is making a similar cultural shift. Rather than working in silos, teams work collaboratively to empower data scientists. Although still in its infancy, DeepOps is expected to lead to deep learning applications that are more robust, faster to deploy, and easier to manage. As ever, the devil is in the data…
Want to find out more? Sign up for our free, weekly newsletter.