AIoT DevOps and Infrastructure

Revision as of 13:44, 15 September 2020 by Joshua Kugele (talk | contribs)
Jump to navigation Jump to search
Ignite AIoTArtificial IntelligenceInternet of ThingsBusiness ModelProduct ArchitectureDevOps & InfrastructureTrust & SecurityReliability & ResilienceVerification & ValidationIgnite AIoT - DevOps and Infrastructure

Ignite AIoT: DevOps and Infrastructure

The introduction of DevOps - together with Continuous Integration / Continuous Delivery (CI/CD) - has fundamentally changed the way how software is developed, integrated, tested, and deployed. DevOps and CI/CD are key enablers of agile development. However, today`s DevOps practices are predominantly focusing on cloud and enterprise application development. For successfull AIoT products, DevOps will have to be extended to include AI and IoT.

Agile DevOps for Cloud and Enterprise Applications

DevOps organizations break down the traditional barriers between development and operations, focusing on cross-functional teams which support all aspects of development, testing, integration and deployment. Successful DevOps organizations avoid overspecialization and instead focus on cross-training and open communication between all DevOps stakeholders.

The DevOps culture is usually closely aligned with the agile culture, both are required for incremental and explorative development.

Continuous Integration / Continuous Delivery (CI/CD) emphasize automation tools that drive build and test, ultimately enabling a highly efficient and agile software life cycle. The Continuous Integration (CI) process typically requires commitment of all code changes to a central code repository. Each new check-in triggers an automated process which re-builds the system, automatically performs unit tests, and executes automated code-quality checks. The resulting software packages are deployed to a CI server, with optional notification of a repository manager.

Continuous Testing (CT) goes beyond simple unit tests, and utilizes complex test suites which combine different test scripts to simulate and test complex interactions and processes.

Finally, Continuous Delivery (CD) uses Infrastructure-as-Code (IaC) concepts to deploy updated software packages to the different test and production environments.

Agile DevOps for Cloud and Enterprise Applications

Agile DevOps for AI

The introduction of AI to the traditional development process is adding many new concepts, which are create challenges for DevOps:

  • New roles: data scientist, AI engineer
  • New artefacts (in addition to code): Data, Models
  • New methods / processes: AI/data-centric, e.g. „Agile CRISP-DM“, Cognitive Project Management for AI (CPMAI)
  • New AI tools + infrastructure

The development of AI-based systems is also introducing a number of new requirements from a DevOps perspective:

  • Reproduceability of models: Creating reproduceable models is a key prerequisite for a stable DevOps process
  • Model validation: Validating models from a functional and business perspective is key
  • Explainability (XAI, or 'explainable AI'): How to ensure that the results of the AI are comprehensible for humans?
  • Testing and test automation: AI requires new methods and infrastructure
  • Versioning: Models, code, data
  • Lineage: Track evolution of models over time
  • Security: Deliberately skewed models as new attack vector / adversarial attacks
  • Monitoring and re-training: Model decay requires constant monitoring and re-training

The figure below provides an overview of how an AI-specific DevOps process can help addressing many of the issues outlined above.

DevOps for AI

Agile DevOps for IoT

Finally, we need to look at the DevOps challenges from an IoT point of view. The main factors are:

  • OTA: Over-the-Air updates (OTA) require a completely different infrastructure and process than traditional, cloud-based DevOps approaches
  • Embedded Software & Hardware: The lifecylcle of embedded hardware and software is very different from cloud-based software. Testing and test automation are possible, but require special efforts and techniques

The OTA update process is described in more detail here. The figure below provides a high-level overview. The OTA Update process usually comprises of three phases. During the authoring phase, new versions of the software (or AI models or other content) are created. The distribution phase is responsible for physical distribution (e.g. between different global regions) and the management of update campaigns. Finally, once arrived on the asset, the local distribution process ensures that the updates are securely deployed, and the updated system is validated.

OTA Overview

Looking again at the 4 quadrant DevOps overview, this time from the IoT perspective, a number of differences compared to the standard DevOps approach can be noticed:

  • The agile development is structured to match the needs of an IoT organization, as discussed here
  • Continuous Integration (CI) will usually have to cover a much more diverse set of development environments, since it needs to cover cloud and embedded development
  • Continuous Testing (CT) will have to address test automation of embedded components, e.g. by utilizing different abstraction and simulation techniques like HIL (hardware in the loop), SIL (software in the loop) and MIL (model in the loop)
  • Continuous Delivery (CD) will have to utilize OTA not only for the production system, but already for Quality Assurance and User Acceptance Tests

Finally, all of the above will also have to be looked at from the perspective of Verification and Validation.

DevOps for IoT

Agile DevOps for AIoT

The AIoT DevOps approach will have to combine all 3 perspectives outlined in the previous sections: Cloud DevOps, AI DevOps and IoT DevOps. Each of these three topics in itself is complex, and integrating the three into a single, homogeneous and highly automated DevOps approach will be one of the main challenges of each AIoT product. However, without succeeding in this effort, it will be near impossible to deliver an attractive and feature rich product which can also evolve over time - as far as the limitations of hardware deployed in the field will allow for. Utilizing OTA to evolve the software and AI deployed on the assets in the field will be a key success factor for smart, connected products in the future.

Authors and Contributors


Dirk Slama is VP and Chief Alliance Officer at Bosch Software Innovations (SI). Bosch SI is spearheading the Internet of Things (IoT) activities of Bosch, the global manufacturing and services group. Dirk has over 20 years experience in very large-scale distributed application projects and system integration, including SOA, BPM, M2M and most recently IoT. He is representing Bosch at the Industrial Internet Consortium and is active in the Industry 4.0 community. He holds an MBA from IMD Lausanne as well as a Diploma Degree in Computer Science from TU Berlin.