Buying the Black Box: How Dealmakers Can Diligence, Price, and Close AI Transactions

Introduction: AI Diligence Requires a Different Deal Mindset

An AI target's true value is often difficult to see from the outside—the trained model, the training data, the weights, the AI architecture, and the algorithms do not reveal themselves the way source code does. AI systems are often called "black boxes" because their outputs cannot always be traced to specific inputs or decision pathways, yet users of the systems and regulators increasingly expect explainability and transparency. Before signing, opening that black box takes more than the traditional diligence playbook: it requires pricing what cannot be fully audited, allocating residual risk through the purchase agreement, and building the remediation and integration path that will let the acquired technology perform after closing—all of which demand a shift in mindset.

Applying traditional software diligence frameworks alone can miss AI-specific risks, generate issue lists that do not advance the deal, and produce deal terms that fail to address what actually matters.

In traditional software deals, diligence asks whether the target owns the code—confirming IP assignments from employees and contractors, reviewing customer agreements, verifying chain of title, and auditing the target's open source contributions—and whether third-party licenses or other contractual restrictions reflect the target's current use cases and whether they limit future use. In AI deals, those questions are only the beginning. The additional questions are whether the target has adequate rights to the training data for the buyer's intended use, whether the target actually owns the model weights or merely has a license, how the target's customer agreements allocate AI-specific risks, and whether the parties can allocate the IP risks that traditional diligence cannot fully resolve.

Define the AI Asset Before Pricing the Deal

Locate the Source of Value

The threshold question sounds simple: what is being purchased? The answer may be a proprietary model (one the target built and owns), a customized version of a publicly available open-source model, a system that combines AI with the target's proprietary knowledge base, a workflow layer embedding AI into customer operations, or a data asset that makes otherwise available technology commercially valuable. Each answer carries a different diligence profile and risk allocation strategy.

To test the seller's valuation narrative, counsel should map each component of the AI system and ask: Is the base model proprietary, open source, or licensed? Were its parameters trained from scratch or derived from existing models? Does the target have documented rights to use the training data? Does the target own or have adequate rights to the prompts, logs, and other components needed to operate and continue to develop the product? Does the application code incorporate third-party libraries with conflicting license terms? Is the product dependent on third-party AI services that could be terminated or repriced?

This mapping determines whether the buyer is acquiring a defensible technology asset, a dependent commercial application, or a business whose AI claims rest on fragile third-party access. It also determines what the purchase agreement must protect: model rights, data rights, retraining rights, trade secret controls, regulatory compliance, or personnel retention.

Separate the Defensible Moat from the AI Wrapper

The wrapper-versus-proprietary distinction is often the critical valuation issue. A "wrapper" is an application built on top of a third-party AI model, adding a user interface or workflow. A target commanding an "AI company" valuation may rely primarily on engineered prompts and an Application Programming Interface ("API") connection (a software link) to a commercial model. That dependency exposes the buyer to platform risk: if the model provider changes terms, or a competitor can replicate the workflow using the same publicly available model, the valuation premium may warrant closer scrutiny. But a wrapper is not automatically low value. A workflow-specific application layer may be defensible if it reflects proprietary data, deep customer integration, domain-specific customization, or validated performance in a regulated environment.

The diligence task is to identify the real competitive advantage. If the product stops working when a single API provider changes terms, the buyer should treat that dependency as a valuation issue. If a competitor could rebuild the core functionality within months using publicly available models, premium pricing should be tested carefully. If the value lies in customer data, annotated datasets (data labeled for AI training), or proprietary datasets or workflows, diligence should focus on ownership, licensing and contractual rights, copyright, database and unfair competition issues, Computer Fraud and Abuse Act ("CFAA")-concerns, consent and privacy issues, and the buyer's right to continue using those assets.

Assess Protectability Without Overstating Certainty

Even genuinely proprietary AI technology may receive uncertain protection under existing IP regimes—and that uncertainty should feed directly into valuation and risk allocation.

Under current case law, patent protection may be limited where core algorithms are characterized as abstract ideas or where infringement is hard to detect. Copyright presents a different problem: under current Copyright Office guidance, content generated entirely by AI (without human creative input) may lack the human authorship required for protection, while the legal treatment of a model's trained parameters remains unsettled. These issues may influence how much value the buyer assigns to exclusivity.

Trade secret protection often becomes the more practical strategy. A model's trained parameters, the composition of training datasets, training techniques, and prompt libraries (collections of instructions used to guide AI outputs) may all be protectable if the target maintained reasonable secrecy measures. Diligence should test confidentiality agreements, access controls, and security protocols. Weak trade secret controls may affect valuation, closing conditions, or indemnity treatment.

Price the Unknowns When Diligence Cannot Produce Certainty

Move from Issue Elimination to Risk Classification

AI diligence often reaches its hardest point when the buyer confronts risks that cannot be fully audited before signing. Training data provenance is the clearest example. A model may have been trained on datasets compiled from many sources, some of which may be subject to disputed licensing, privacy, or contractual use rights. A complete review against every potential claim is rarely realistic.

Risk classification depends on getting the right information. Counsel should request a complete inventory of all third-party data sources used to train or fine-tune the model, along with the terms governing each source. Being well-versed in recent data scraping-related case law across relevant jurisdictions is a must. Where a third party trained or hosts the model, diligence should examine what steps the target took to obtain transparency into the training process, the provenance of the underlying data, and the third party's ability to explain model behavior. Without answers to these questions, the buyer cannot classify exposure.

The answers then drive deal structure. If the target cannot produce a complete data inventory, the buyer may treat that gap as a valuation issue, risk-allocation trigger, closing condition, or post-closing action item. Diligence that surfaces incomplete answers can still advance the deal—so long as those answers shape how the parties allocate risk and how the buyer views the target's value proposition.

Regulatory compliance diligence deserves distinct attention. AI-specific regulations are emerging across jurisdictions, with the EU AI Act as the most prominent example. These frameworks often classify AI systems by risk level—the EU AI Act, for instance, imposes heightened requirements on "high-risk" systems used in areas such as employment, credit decisions, and critical infrastructure. A system classified as high-risk may face mandatory conformity assessments, transparency obligations, human oversight requirements, and ongoing monitoring duties. Other jurisdictions are developing their own frameworks, and sector-specific regulators are increasingly focused on AI governance.

Assess Regulatory Compliance as a Distinct Diligence Workstream

Diligence should determine which regulations apply to the target's AI systems based on how those systems are used, where they are deployed, and what decisions they inform. The buyer should then assess whether the target has evaluated its own regulatory posture, implemented required compliance measures, and developed the policies and technical capabilities needed to satisfy ongoing obligations. Gaps in regulatory compliance—or uncertainty about classification—can affect valuation, require remediation commitments, or become conditions to closing. As with data provenance, the goal is not to eliminate regulatory uncertainty before signing but to classify the exposure and build the appropriate response into the deal.

Translate AI Risk into Deal Terms

Standard IP representations remain necessary but are often not sufficient for AI transactions. Supplemental coverage should depend on what the AI system does and which risks matter. Counsel should resist adding generic AI representations without first understanding the target's use case. A quality-control vision system, a customer service chatbot, and a predictive platform in a regulated industry warrant different treatment.

Address Output Reliability Where It Changes the Risk Profile

Output reliability shows why AI representations should be tailored. Generative AI systems (which create text, images, or other content) may produce confident but inaccurate results; predictive systems (which forecast outcomes or classify data) may generate biased predictions or misidentifications. The legal significance depends on the intended use. An internal tool and an AI system used in healthcare, employment, or lending should not be treated the same way.

The practical solution is often to adapt existing software representations rather than create disconnected AI provisions. For example, a "no material defects" representation can be modified to address output inaccuracies that materially impair the system’s intended use.

Use Indemnities, Escrows, and Holdbacks to Allocate Residual Risk

Because AI diligence often leaves residual uncertainty, indemnification structure can be unusually consequential. The parties may know that certain risks exist—uncertain training data origins, possible open-source issues, or privacy gaps—without knowing whether those risks will mature into claims or enforcement actions.

Market practice matters. In private equity and many strategic deals, insurance-backed no-recourse transactions often set the baseline. Buyers may have limited ability to obtain special indemnities or escrows for AI issues unless the risk is specific and substantial. The practical question then shifts to how the identified risk should be addressed elsewhere in the deal.

Where seller recourse is available, the parties may consider separate indemnification for defined categories—such as IP claims arising from training data or open-source compliance failures. Distinct baskets, caps, or escrows should be driven by the actual risk profile. Where recourse is unavailable, the buyer should address the risk through valuation, purchase price adjustments, or closing conditions.

Escrows and holdbacks may fund retraining, replacement of problematic components, or regulatory remediation. If those tools are unavailable, similar discipline may be reflected through price adjustments, earnouts, or milestone payments.

Use Remediation to Preserve the Deal

Treat Fixable Problems as Deal-Structure Issues

Identified AI issues do not automatically require abandoning an otherwise attractive deal. Affected datasets, unverified model parameters, or open-source concerns may be serious, but some can be isolated, priced, or remediated. The key is to determine during diligence whether the issue is fatal, value-reducing, or manageable through post-closing controls.

AI systems can sometimes be fixed in ways that differ from traditional software. A model may be retrained on different datasets; components may be replaced; and particular data sources may be excluded. Technical feasibility, cost, and timing should be assessed before signing so that the deal structure reflects the practical path to remediation.

Build the Fix into the Transaction

When diligence identifies remediable issues, the transaction agreement should make the remediation path concrete. Ring-fencing may exclude problematic assets until retraining is completed. Restricted-use covenants may permit limited deployment while prohibiting customer-facing use until conditions are met.

Remediation covenants or earnouts should identify the components to be cleaned or replaced, performance benchmarks, testing procedures, and consequences if milestones are not met. If the target's technical team is needed, personnel retention may become a closing condition.

The economics should follow the risk: whichever risk-allocation tool the parties use, its terms should be tied to remediation milestones or the absence of claims during a survival period. Used carefully, these tools let parties close through uncertainty.

Plan Integration Before Signing, Not After Closing

Integration readiness is not unique to AI, but AI can make it more critical. Ownership alone does not equal usable value. The buyer may need continuing rights to use data, trained model parameters, and the processes for updating or retraining the AI system. The buyer may also need the system to operate within its privacy, cybersecurity, and regulatory environment. If those elements do not align, the buyer may acquire the asset but be unable to deploy it.

Integration planning should begin during diligence. Counsel should test whether third-party terms permit the buyer's intended use; whether the system can satisfy required monitoring, privacy, and security controls; whether performance may degrade when migrated; and whether key personnel are required to maintain or retrain the system. These issues can affect valuation, closing conditions, and whether the deal delivers the value priced.