AIoT DevOps and Infrastructure
The introduction of DevOps -- together with Continuous Integration/Continuous Delivery (CI/CD) -- has fundamentally changed the way software is developed, integrated, tested, and deployed. DevOps and CI/CD are key enablers of agile development. However, today's DevOps practices predominantly focus on cloud and enterprise application development. For successful AIoT products, DevOps will need to be extended to include AI and IoT.
Agile DevOps for Cloud and Enterprise Applications
DevOps organizations breakdown the traditional barriers between development and operations, focusing on cross-functional teams that support all aspects of development, testing, integration and deployment. Successful DevOps organizations avoid overspecialization and instead focus on cross-training and open communication between all DevOps stakeholders.
DevOps culture is usually closely aligned with agile culture; both are required for incremental and explorative development.
Continuous Integration / Continuous Delivery (CI/CD) emphasize automation tools that drive building and testing, ultimately enabling a highly efficient and agile software life cycle. The Continuous Integration (CI) process typically requires commitment of all code changes to a central code repository. Each new check-in triggers an automated process that rebuilds the system, automatically performs unit tests, and executes automated code-quality checks. The resulting software packages are deployed to a CI server, with optional notification of a repository manager.
Continuous Testing (CT) goes beyond simple unit tests, and utilizes complex test suites that combine different test scripts to simulate and test complex interactions and processes.
Finally, Continuous Delivery (CD) uses Infrastructure-as-Code (IaC) concepts to deploy updated software packages to the different test and production environments.
Agile DevOps for AI: MLOps
The introduction of AI to the traditional development process is adding many new concepts, which create challenges for DevOps:
- New roles: data scientist, AI engineer
- New artefacts (in addition to code): Data, Models
- New methods/processes: AI/data-centric, e.g., "Agile CRISP-DM", Cognitive Project Management for AI (CPMAI)
- New AI tools + infrastructure
The development of AI-based systems also introduces a number of new requirements from a DevOps perspective:
- Reproducibility of models: Creating reproducible models is a key prerequisite for a stable DevOps process
- Model validation: Validating models from a functional and business perspective is key
- Explainability (XAI, or 'explainable AI'): How to ensure that the results of the AI are comprehensible for humans?
- Testing and test automation: AI requires new methods and infrastructure
- Versioning: Models, code, data
- Lineage: Track evolution of models over time
- Security: Deliberately skewed models as new attack vector/adversarial attacks
- Monitoring and retraining: Model decay requires constant monitoring and retraining
The figure below provides an overview of how an AI-specific DevOps process can help in addressing many of the issues outlined above.
Agile DevOps for IoT
Finally, we need to look at the DevOps challenges from an IoT point of view. The main factors are:
- OTA: Over-the-Air updates (OTA) require a completely different infrastructure and process than traditional, cloud-based DevOps approaches
- Embedded Software & Hardware: The lifecycle of embedded hardware and software is very different from cloud-based software. Testing and test automation are possible, but require special efforts and techniques.
The OTA update process is described in more detail here. The figure following provides a high-level overview. The OTA Update process usually comprises three phases. During the authoring phase, new versions of the software (or AI models or other content) are created. The distribution phase is responsible for physical distribution (e.g., between different global regions) and the management of update campaigns. Finally, once arrived on the asset, the local distribution process ensures that the updates are securely deployed, and the updated system is validated.
Looking again at the 4 quadrant DevOps overview, this time from the IoT perspective, a number of differences compared to the standard DevOps approach can be seen:
- Agile development is structured to match the needs of an IoT organization, as discussed here
- Continuous Integration (CI) will usually have to cover a much more diverse set of development environments since it needs to cover cloud and embedded development
- Continuous Testing (CT) will have to address the test automation of embedded components, e.g. by utilizing different abstraction and simulation techniques such as HIL (hardware in the loop), SIL (software in the loop) and MIL (model in the loop)
- Continuous Delivery (CD) will have to utilize OTA not only for the production system but also for Quality Assurance and User Acceptance Tests
Finally, all of the above will also have to be examined from the perspective of Verification and Validation.
Agile DevOps for AIoT
The AIoT DevOps approach will need to combine all three perspectives outlined in the previous sections: Cloud DevOps, AI DevOps and IoT DevOps. Each of these three topics in itself is complex, and integrating the three into a single, homogeneous and highly automated DevOps approach will be one of the main challenges of each AIoT product. However, without succeeding in this effort, it will be nearly impossible to deliver an attractive and feature-rich product that can also evolve over time, as far as the limitations of hardware deployed in the field will allow. Utilizing OTA to evolve the software and AI deployed on the assets in the field will be a key success factor for smart, connected products in the future.
Authors and Contributors