Here’s the next excerpt from Auditoria’s latest eBook “Demystifying AI Governance and Control for Finance: A Guide to AI Governance.”
In Part 2 of 4 of our social and blog posts, we’ll cover selecting and processing data with AI and protecting and managing AI data.
Selecting and Processing Data with AI
AI systems must adhere to policies that prioritize ethical data selection and processing. This includes obtaining informed consent for data collection, ensuring data accuracy and relevance, and respecting privacy laws. The systems should use diverse and representative datasets to avoid biases and implement robust security measures to protect data integrity and confidentiality.
Additionally, they must be transparent about data usage, allowing for accountability and facilitating user rights such as access, correction, and deletion of their data. These policies are crucial for maintaining trust and ethical standards in AI applications.
Explainability and Interpretability
AI systems often operate as “black boxes,” making it challenging to understand how they arrive at specific decisions. Generative AI models, particularly deep learning models, can be complex and opaque, making it difficult to understand how they arrive at their decisions. The underlying assumption is that perceptions of negative risk stem from a lack of ability to make sense of, or contextualize, system output appropriately.
Ethical AI development emphasizes the need for transparency and explainability, allowing users and stakeholders to understand the underlying reasoning and factors that influence AI outputs. Explainability refers to a representation of the mechanisms underlying AI systems’ operation – how to take an ML model and explain the behavior in human terms. Interpretability refers to the meaning of AI systems’ output in the context of their designed functional purposes.
Explainable and interpretable AI systems offer information that will help end users understand the purposes and potential impact of an AI system.
Together, explainability and interpretability assist those operating or overseeing an AI system, as well as users of an AI system, to gain deeper insights into the functionality and trustworthiness of the system, including its outputs.
To properly assess an AI system’s explainability and interpretability, organizations should:
- Understand the purposes and potential impact of an AI system
- Verify techniques for model interpretability
- Request transparency in decision-making processes
Fairness and Bias
Generative AI models are susceptible to biases if the training data is not diverse or representative. This may lead to discriminatory outcomes, such as biased decisions. It is crucial to ensure that training data is unbiased and that models are regularly monitored and audited to identify and mitigate any potential biases.
Fairness in AI includes concerns for equality and equity by addressing issues such as harmful bias and discrimination. Standards of fairness may be complex and difficult to define because perceptions of fairness differ among cultures and could shift depending on application. Organizations’ risk management efforts will be enhanced by recognizing and considering these differences.
Systems in which harmful biases are mitigated are not necessarily fair. For example, systems in which predictions are somewhat balanced across demographic groups may still be inaccessible to individuals with disabilities or affected by the digital divide or may exacerbate existing disparities or systemic biases.
To properly assess an AI system’s fairness and bias, organizations should:
- Confirm training data is representative
- Regularly monitor and audit models
- Recognize and consider the differences
- Understand that bias is always present
Protecting and Managing AI Data
AI systems should safeguard data using strong security measures such as encryption and adhere to privacy laws, ensuring transparent, consent-based data usage. They must also employ data minimization and anonymization to protect individual privacy and reduce misuse risks.
Privacy and Safeguards
AI systems should also offer anonymization and pseudo-anonymization of both personally identifiable information (PII) and business identifiable information (BII) data attributes to ensure compliance and privacy control management for personal information as well as information that could identify a business entity. Protecting user privacy and maintaining data security are essential ethical considerations.
Privacy generally refers to the norms and practices that help to safeguard human autonomy, identity, confidentiality, and dignity. These norms and practices typically address freedom from intrusion, limiting observation, or individuals’ agency to consent to disclosure or control of facets of their identities.
AI relies on extremely vast amounts of data, raising concerns about privacy and data protection. Ethical AI development involves implementing robust privacy measures, obtaining appropriate consent for data usage, and ensuring secure handling of sensitive information.
For an AI system, privacy values should help guide choices for AI system design, development, and deployment. Data privacy policies should clearly outline what data is collected, how it is collected, and for what purposes. Systems should only collect the data necessary for the stated purpose and avoid using it for other, unrelated purposes. This principle reduces the risk of misuse and potential harm to individuals.
To properly assess an AI system’s privacy tactics and safeguards, organizations should be sure the software:
- Adheres to laws and regulations
- Establishes robust data governance practices
- Employs anonymization of PII and BII
- Obtains appropriate consent for data usage
Data Security, Resilience, and Compliance
Financial data is highly regulated, and implementing generative AI in the finance office must comply with regulatory requirements. Organizations need to ensure that their generative AI system’s use of data adheres to applicable laws and regulations, and that these systems provide the necessary documentation and audits to demonstrate compliance.
Security and resilience are related but distinctly important characteristics. While resilience is the ability to return to normal function after an unexpected adverse event, security includes resilience but also encompasses protocols to avoid, protect against, respond to, or recover from attacks.
A secure AI systems must maintain confidentiality, integrity, and availability through protection mechanisms that prevent unauthorized access. AI systems need to have secure data storage and transfer mechanisms in place and comply with relevant regulations such as General Data Protection Regulation (GDPR) or California Consumer Privacy Act (CCPA).
Resiliency in AI security is the ability of AI systems to withstand, adapt to, and recover from various types of threats, disruptions, or attacks while maintaining their intended functionality. This concept is crucial in AI security due to the increasing reliance on AI systems in critical areas and their potential vulnerabilities to various forms of cyber threats.
The security must be robust enough so that the software does not become a conduit for bad actors to infiltrate systems.
To properly assess an AI system’s data security and compliance, organizations should be sure the software:
- Uses data encryption and protections
- Implements proper secure storage
- Provides documentation to demonstrate compliance
- Secures end-points, plug-ins, inputs, and outputs
Fill out the form and download your complimentary copy of the Auditoria.AI eBook.
"Demystifying AI Governance and Control for Finance - A Guide to AI Governance"
