• Written by Azhar Aliyar, Director

A hospital's IT department is preparing to deploy AsscherAi, an enterprise AI platform that will allow clinicians to query patient data in natural language and receive answers in real time. Before deployment, the procurement team must settle several questions about security and privacy, given the strict obligations of clinical operations.

Healthcare provider constraints

Healthcare providers present unique challenges for enterprise AI deployment, particularly in terms of scheduling and utilisation. Clinical operations are often subject to unpredictable variability, such as emergency admissions or changes in patient flow, which can impact the availability of staff and resources. Additionally, supply consumption can be highly variable, with different wards or departments having different needs and usage patterns. This variability can make it difficult to anticipate and plan for the demands on the AI system.

The documentation burden is another significant constraint in healthcare providers. Clinical staff must maintain accurate and detailed records of patient care, which can be time-consuming and may divert attention away from direct patient care. Any AI system must be designed to minimise additional documentation requirements, while also ensuring that all interactions are properly logged and auditable.

Data processing and storage

One of the key questions for procurement teams is where data will be processed and stored. Will the AI system be deployed on-premise, or will data be transmitted to a cloud-based service for processing? In either case, the team must ensure that all data handling and storage comply with relevant privacy regulations, such as the General Data Protection Regulation (GDPR) in the European Union. This includes ensuring that all data are properly encrypted, both in transit and at rest.

The team must also consider whether the AI system will train on patient data, and if so, how this training will be managed and audited. Any training data must be properly anonymised and aggregated to prevent re-identification of individual patients. For more information on AsscherAi's data handling and storage capabilities, please visit our website via this link.

Prompt injection as a data-boundary problem

Prompt injection is a potential security risk for AI systems, where an attacker attempts to manipulate the system by injecting malicious input. In the context of healthcare providers, prompt injection can be seen as a data-boundary problem, where the attacker seeks to breach the boundaries between different data domains or systems. For example, an attacker might attempt to use the AI system to access sensitive patient data, or to manipulate clinical decisions.

To mitigate this risk, the procurement team must ensure that the AI system is designed with robust data boundaries and access controls. This includes implementing strict authentication and authorisation mechanisms, as well as ensuring that all data interactions are properly logged and audited. By doing so, the team can help prevent prompt injection attacks and protect sensitive patient data.

Least privilege for AI assistants

Another key consideration for procurement teams is the principle of least privilege, which states that any system or user should only have the minimum necessary privileges to perform its intended function. In the context of AI assistants like AsscherAi, this means ensuring that the system only has access to the data and systems necessary to answer clinical questions, and no more. This helps to prevent potential security risks, such as data breaches or unauthorised access.

The team must work with clinical staff to identify the specific systems and data that the AI assistant will need to access, and ensure that all access is properly controlled and audited. This may involve implementing role-based access controls, where the AI system is assigned a specific role with associated privileges. For more information on implementing least privilege in AsscherAi, please contact our support team via this link.

Limitations and wrong choices

While AsscherAi can be a powerful tool for clinical decision support, it is not a replacement for human clinical judgement or expertise. In situations where complex or high-stakes decisions are required, human clinicians must always be involved. Additionally, AsscherAi is not designed to handle emergency or critical care situations, where rapid and decisive action is required.

In some cases, AsscherAi may not be the best choice for a particular healthcare provider. For example, small or rural providers may not have the necessary infrastructure or technical expertise to support an enterprise AI deployment. In such cases, alternative solutions or partnerships may be more suitable. By carefully considering these factors, procurement teams can ensure that AsscherAi is deployed effectively and securely, and that it supports the delivery of high-quality patient care.