In our continuing review of the emerging US AI regulatory regime (previously the OMB memo on AI governance and the NTIA paper on disclosure and audit), this week we review the proposed US regulation of non-US parties training AI models on cloud platforms of US-based providers (Infrastructure as a Service or IaaS providers).

As the US IaaS providers, such as Amazon, dominate cloud services globally, the proposed US regulation is likely to have an impact across the world. As the proposed regulation is in the final stages of the rule-making process, it seems likely to come into force later this year much as it currently stands.

The harm to be addressed

The proposed regulation notes that, with their growing importance and reach, IaaS platforms provide both the training ground for and are a target of AI-based cyber attacks by foreign malicious parties:

“U.S. entities providing IaaS products, such as network management or data storage, can create multiple opportunities for foreign adversaries to exploit potential vulnerabilities in the ICTS [Information and Communications Technology and Services] ecosystem….Further, a foreign adversary could target vulnerable IaaS products to implement denial of service attacks, potentially causing widespread disruptions to critical industries…Sophisticated cyber-attacks are often obfuscated, making it difficult to establish the exact number of attacks that have leveraged IaaS product vulnerabilities against the U.S. ICTS supply chain. Such attacks, however, are increasing in frequency, exacting heavy tolls on U.S. consumers and businesses…..[I]f if the use of IaaS products is expected to increase in the future, so too would the possibility of attacks.”

The proposed regulation essentially applies a strict ‘know your customer’ (KYC) regime on US IaaS providers and, through them, their resellers around the world, backed up by extraordinary ‘black banning’ powers which can apply to foreign individuals or whole countries.

The US Commerce Department says it believes that many US IaaS providers and foreign resellers already collect the required information from their customers, and that “the proposed rule would set a baseline for data collection that would help all providers effectively verify and document the identities of their customers". That maybe a pious hope given the reach of the proposed KYC requirements.

What and who is caught?

The proposed regulation would apply the KYC obligations when a non-US person proposes to train a "large LLM" on a US provider’s IaaS platform, including where the training takes place ‘outside’ the US (as much as any geographic nexus exists with global cloud services).

The LLMs covered are defined as a "dual use foundation model" that:

• is trained on broad data and contains at least tens of billions of parameters (while this sounds a lot, GPT4 has over a trillion parameters and models with less than 100 million parameters are now typically considered small language models);
• generally uses self-supervision;
• is applicable across a wide range of contexts; and
• exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, including:
  • substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;
  • enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or
  • permitting the evasion of human control or oversight through means of deception or obfuscation.

The ‘dark side’ to which an LLM could be turned is broadly defined as “including but not limited to social engineering attacks, vulnerability discovery, denial-of-service attacks, data poisoning, target selection and prioritisation, disinformation or misinformation generation and/or propagation, and remote command-and-control of cyber operations". This does not involve a judgment of the motivations of the foreign party training the AI model, but rather of the technical capability of the model to be turned to the ‘dark side’ – which pretty much will capture most LLMs.

The LLM definition will capture LLMs “even if they are provided to end users with technical safeguards that attempt to prevent users from taking advantage of the relevant unsafe capabilities".

The primary KYC obligations

The KYC obligations apply unless the customer and each of its "beneficial owners" is a US person. A beneficial owner is defined as a person who exercises substantial control over a customer or owns or controls at least 25 percent of the ownership interests of a customer.

The US IaaS provider is required to implement and ensure that each of its foreign resellers implements a Customer Information Program (CIP). The CIP “must be appropriate for the IaaS providers’ size, type of IaaS products offered, and relevant risks”, which notionally provides some wriggle room, but the proposed regulation goes on to specify a comprehensive list of mandatory minimum requirements for all CIPs:

• the information collected and the risk-based analysis undertaken by the US IaaS provider or its foreign reseller must provide a “sound basis…to form a reasonable belief that it knows the true identity of each customer and each beneficial owner".
• the minimum information to be collected about the customer and each of its beneficial owners not only includes expected details such as name, incorporation details, governing jurisdiction, and principal business address, but also each address from which the IaaS platform will be accessed, payment details and IP addresses “used for access or administration and the date and time of each such access or administrative action, related to ongoing verification of such foreign person’s ownership or control of [its] account".
• supply terms must provide that customers are to notify the US IaaS provider or foreign reseller of changes in beneficial ownership, and the CIP must provide for an updated risk based reassessment of the customer as ownership changes occur.
• the CIP must specify the methodologies for verifying the documentary and/or non-documentary means used to identify customers and their beneficial owners, and in deciding to adopt and apply these methodologies, a risk-based assessment must be made of their reliability and limitations.
• the CIP must set out the terms and conditions (including technical limits) on which any interim access to the IaaS platform is provided pending completion of verification, and making interim access available must itself be tested on a risk assessment basis.
• the CIP must provide that accounts will not be opened if the customer or its beneficial owners cannot be verified, and the procedures for rejecting the customer application must be set out.

Foreign resellers

As they are outside the jurisdiction of the US, the proposed regulation indirectly enforces its requirements on foreign resellers by requiring the US IaaS provider to ensure that its foreign resellers have a compliant CIP in place and that they comply with it.

If requested by the Commerce Department, the US IaaS must provide a foreign reseller’s CIP to the Department within 10 days of the request.

The US IaaS provider must terminate its relationship with a foreign reseller if the US IaaS has evidence that the reseller is not complying with its CIP or, the US IaaS provider having identified issues to be remediated, the reseller fails to do so in 30 days.

However, the ‘policing’ role of the US IaaS provider is expressed much more broadly than addressing simple legal non-compliance. A US IaaS provider is required to terminate a foreign reseller where:

• there is evidence of a lack of good faith efforts by the foreign reseller to prevent the use of US IaaS products for malicious cyber-enabled activities; or
• continuation of the reseller relationship otherwise increases the risk its US IaaS products may be used for malicious cyber-enabled activity.

How to give these highly qualitative judgments contractual and procedural forms will be an interesting challenge for the US IaaS providers.

Record keeping and reporting

A US IaaS provider must give the Commerce Department an initial notification of the implementation of its own CIP and each of its foreign resellers’ CIPs. This initial report is to include the following:

• how changes in the beneficial owners of customers will be kept track of;
• the mechanisms, services, software, systems, or tools used to detect malicious cyber activity;
• the mechanisms, services, software, systems, or tools used by the IaaS provider to detect a training run that could result in the training of a large AI model with potential capabilities that could be used in malicious cyber-enabled activity;
• the criteria and methodology used to determine that an AI model proposed to be trained by a customer on the US provider’s IaaS platform carries the potential risk of being used in malicious cyber-enabled activities; and
• the timeline over which the CIP will be applied to satisfactorily identify existing foreign customers, including a ‘drop dead’ disconnection date if customers do not provide the required details.

The US IaaS provider is then required to annually certify compliance by itself and each of its foreign resellers with their CIPs and to provide timely ad hoc notice of materials changes in their CIPs: for example, a material change in the documentary or non-documentary methods of identity verification or in the procedures for handling unverified accounts.

A US IaaS provider also must give the US Commerce Department notice of an individual "covered transaction" involving a foreign party relating to the training of an AI model on its US IaaS platform. A covered transaction not only includes a contract that directly provides for training of an AI model which, from the ‘get go’, carries the potential risk of malicious cyber-enabled activity, but also a larger set of notifiable transactions, e.g.:

• the AI model as originally specified by the customer is assessed by the US IaaS provider or its foreign reseller not to carry a malicious cyber-risk but the US IaaS or reseller forms the view during training that there is a potential malicious cyber security risk because of changes or adaptations made by the customer or simply because the US IaaS provider or reseller has a better understanding of what the customer is up to or what the AI model is capable of.
• the US IaaS provider or foreign reseller takes an equity stake in a foreign AI developer and provides the developer with computing time as part of the deal, and at the time of the deal or later, the US IaaS provider or reseller forms the view that the developer will use the computing time to train an AI model with potential malicious cyber-risk capabilities.

Again, it is important to remember that the obligation to report is not based on an imputation that the foreign customer has intentions to undertake malicious cyber-enabled activities (i.e. that the customer is or could be a bad actor), but only that the model itself has the technical capabilities to be used (including by persons to whom the AI model is eventually licensed or supplied) to undertake malicious cyber-enabled activities.

Records of applying a CIP to individual customers are to be kept for two years. Access to the records is restricted, but the US IaaS providers can share “share security best practices or other threat information with other US IaaS providers”, which possibly could include a list of high risk foreign customers.

Black listing

The US Commerce Secretary may direct a US IaaS provider that neither it nor its foreign resellers are to provide access (or are only to provide limited access) to the IaaS platform to a particular foreign person if the Secretary has reasonable grounds for believing that the foreign person has established “a pattern of conduct of offering US IaaS products that are used for malicious cyber-enabled activities or directly obtaining US IaaS products for use in malicious cyber-enabled activities”. In deciding whether to banish an individual customer from the cloud, the Secretary is to consider:

• the extent to which US IaaS products offered by a foreign person are used by third parties to facilitate or promote malicious cyber-enabled activities;
• the extent to which US IaaS products offered by a foreign person are used for legitimate business purposes in the foreign jurisdiction; and
• the extent to which actions short of a ban would address the cyber risk harm.

Perhaps more extraordinarily, the US Commerce Secretary can black list entire countries from having access (or impose conditions on access) if the Secretary is satisfied that the foreign jurisdiction is “found to have any significant number of foreign persons offering US IaaS products used for malicious cyber-enabled activities, or by any US IaaS provider of US IaaS to US IaaS platforms for or on behalf of a foreign person". This would prevent or restrict access by anyone from a black listed country, many or most of whom might be legitimate users.

Before ‘de-clouding’ an entire country, the US Commerce Secretary must consider:

• evidence that foreign malicious cyber actors have obtained US IaaS products from persons offering US IaaS products in that foreign jurisdiction (including foreign resellers);
• the extent to which that foreign jurisdiction is a source of malicious cyber-enabled activities; and
• whether the United States has a mutual legal assistance treaty with that foreign jurisdiction, and the practical experience of the degree of assistance that regulators in that jurisdiction provide US law enforcement and regulators.

In the case of both special measures dealing with foreign individuals and with whole countries, the US Commerce Secretary has to consider the significant competitive disadvantage, including any undue cost or burden associated with compliance, for the US IaaS provider.

Incentive regulation

The US Commerce Secretary can grant exemptions from the CIP requirements to a US IaaS provider or its foreign reseller if satisfied with an alternative Abuse of IaaS Products Deterrence Program (IDP) offered by the US IaaS provider or foreign reseller. Essentially, this allows the US IaaS provider or foreign reseller to spin the customer verification process around from KYC checking everyone to a ‘by exceptions’ approach in which the ‘suspect’ customers or situations requiring detailed verification are identified.

The IDP is to consistent of two broad elements:

• ‘red flags’ triggering the more detailed verification process, which could include certain types of identification evidence provided by a customer, activities on the account, complaints from third parties or law enforcement authorities, and analytics methods monitoring the use of the IaaS platform; and
• the mitigation activities triggered by a red flag, including contacting the customer for an explanation, suspending or closing the account, changing passwords etc.

The IDP has to be approved and monitored by the IaaS provider’s or foreign reseller’s board or a board sub-committee. As cyber threats are constantly evolving, the IDP needs clear mechanisms for review and updating, including in the implementation of technological responses.

As a further incentive, the US Commerce Secretary is to look favourably at requests for exemption based on an IDP where the US IaaS provider or the foreign reseller is a member of an industry consortium to “develop and maintain privacy-preserving data sharing and analytics to enable improved detection and mitigation of malicious cyber-enabled activities". The consortium has to be committed to making available tools to smaller (non-member) IaaS providers, including to help them improve their IDPs.

Read more: Taking Additional Steps To Address the National Emergency With Respect to Significant Malicious Cyber-Enabled Activities