Japan Weighs Safety Duties and Penalties for Developers of Advanced AI

Binding orders could give Japan new powers over dangerous AI

Japan is considering requiring businesses behind advanced artificial intelligence to ensure safety, with penalties for those that disobey government orders after serious problems arise. The proposed approach would give authorities stronger tools against companies that refuse to cooperate over risks such as cyberattacks, criminal misuse and systems acting outside their intended instructions.

The direction emerged on the 11th following discussions in a closed working group on AI legislation. Officials aim to present the full framework for institutional changes, including legal amendments, by the end of the year. The penalties under consideration would focus on incidents involving highly capable AI.

The proposed enforcement sequence is central: when a problem occurs and a business refuses discussions, the government could issue a binding order where necessary. Disobeying that order could then trigger a penalty. Developers, researchers and companies supplying services to the public could face different responsibilities because each controls a different part of an AI system.

Japan's Cabinet Office had already outlined the foundations for stronger oversight in its May 2024 AI governance policy paper. Published by the AI Strategy Team, it proposed examining whether legal requirements should supplement voluntary guidelines for AI with substantial risks. It also recorded calls for improvement orders and public identification of businesses where misuse was particularly likely.

The present debate brings that earlier policy question into sharper focus: how can Japan make companies address serious dangers while preserving room for research and useful services? For users, the practical issue is whether authorities can obtain cooperation and corrective action when an advanced system creates a threat.

Who would have to follow the new rules?

AI businesses operate across several stages. Researchers develop techniques, model developers build the underlying systems, and service providers turn those systems into products that people use. A foundation model is an underlying AI trained for many tasks that other businesses can adapt for particular services.

The government still has to settle which businesses would be covered, what would qualify as sufficiently risky AI and the form or size of any penalty. The timetable announced on the 11th gives no month or year, while the separate June 19 policy timetable gives no year, leaving their precise calendar relationship unspecified.

The May 2024 policy paper argues that model size, intended use and risk should be considered together. It warns that future technical advances could allow smaller models to achieve greater capabilities, making size alone an unreliable way to set a regulatory boundary.

Foreign developers would also matter. The paper says a Japanese framework should cover AI supplied to Japan from abroad and consider effective responses when businesses fail to comply. Including major American technology companies would make international cooperation and the ability to enforce orders practical concerns for Japanese users.

Why voluntary cooperation became a concern

The AI Strategy Council and AI Institutional Study Group set out a selective approach to binding regulation in their February 4, 2025 interim policy report. They recommended respecting business autonomy and limiting legal regulation to matters that companies could not be expected to address through voluntary efforts.

The report identifies AI that causes, or is highly likely to cause, serious problems for life, physical safety, property, public safety or national security as requiring consideration according to the severity of the risk.

Government investigations depend on information held by companies. The report therefore calls for a legal framework through which authorities can seek cooperation, including information from domestic and foreign businesses. It also says government guidelines and investigations into actual AI use should have a legal basis to make them effective.

Penalties would add force to that structure. The report explains that legal sanctions can make rules easier to enforce, while also warning that rigid requirements can slow development and take time to define precisely. The proposed response to uncooperative businesses follows this focus on cases where voluntary action is insufficient.

Cybersecurity is driving the latest urgency

Advanced AI can perform technical tasks with less detailed human direction. An autonomous system can pursue an assigned goal through several steps, choosing actions along the way. That ability can help with legitimate work, but it also makes safety controls more demanding when the system can interact with computers or networks.

Unexpected unauthorized access by an advanced AI under development has been reported in the United States. AI misuse is also suspected in some recent cyberattacks. These concerns have strengthened the case for a government response when developers will not cooperate over dangerous behavior.

A draft revision of Japan's AI Basic Plan described on June 19 places cybersecurity risks at the center of the proposed changes. It identifies a US model called Claude Mythos as capable of autonomously finding weaknesses in computer systems. A vulnerability is a flaw that an attacker may exploit to gain access or disrupt a service.

Finding such weaknesses can help defenders repair software. The same capability creates risks when used to support unauthorized access. The draft therefore combines stronger safety assessment with plans to strengthen defensive capabilities led by the National Police Agency.

Japan plans a larger AI Safety Institute

The AI Safety Institute, known as AISI, would play a larger role under the draft basic plan revision. Japan intends to expand its capabilities and work with foreign governments and global developers to verify advanced models from Japan and abroad.

Japan established AISI within the Information Technology Promotion Agency in February 2024. The May 2024 policy paper describes its work on practical safety guidelines, evaluation methods and technical research, alongside cooperation with similar bodies in the United States and Britain.

The policy paper also suggests an arrangement in which legislation sets basic responsibilities for risk management and information sharing, while specialist institutions help develop detailed guidelines and standards with government and industry. Such an arrangement would allow technical practices to change as AI develops.

AISI could also support responses to harmful systems. The paper proposes examining a mechanism resembling Japan's existing software cybersecurity process, under which the Information Technology Promotion Agency investigates problems and publishes measures for businesses or users to take.

How the policy debate has developed

Japan's proposals have developed through voluntary guidance, institutional preparation and consideration of legal duties. The official documents show a continuing preference for combining these tools rather than applying the same requirements to every AI business.

• May 2023: The AI Strategy Council published an initial discussion of AI risks, describing appropriate safeguards as necessary for wider development and use.
• December 2023: G7 leaders approved the Hiroshima AI Process policy framework, including international principles and a code of conduct for advanced AI developers.
• February 2024: Japan established AISI within the Information Technology Promotion Agency.
• April 2024: Japan published version 1.0 of its AI Guidelines for Business.
• May 2024: The Cabinet Office AI Strategy Team outlined options for supplementing voluntary guidance with legal requirements.
• February 4, 2025: The AI Strategy Council and AI Institutional Study Group recommended targeted legislation where voluntary efforts would be insufficient.

The June 19 draft basic plan revision envisages Cabinet approval in July after public comment. It follows an initial plan adopted the preceding December, putting the proposed revision a little more than six months after that plan. The separate working group process aims to present its broader institutional proposals by year end.

Existing laws already govern some AI uses

Japan's May 2024 policy paper distinguishes rules aimed at advanced AI developers from laws governing particular products and activities. It states that Japan then had no binding framework specifically covering large or risky AI comparable to those being developed in Europe and the United States. Criminal law, privacy law and product rules already applied to relevant conduct.

Medical devices using AI are subject to safety and effectiveness requirements under the Pharmaceuticals and Medical Devices Act. Devices presenting particularly high risks require approval following an assessment that can include clinical trial data. The paper also states that doctors remain responsible for their final medical judgments when using AI as an aid.

Vehicles using automated driving systems must meet requirements under the Road Transport Vehicle Act. Their automated systems are assessed for the conditions in which they are designed to operate.

Employment and credit decisions present different concerns. The paper describes human review of rejected applicants in an example involving AI screening of video interviews. It also notes that advanced methods used to assess credit repayment capacity must avoid unfair discrimination, prejudice and other serious disadvantages. Across these examples, the degree of human control matters: replacing a person's decision can create different risks from supplying information for that person to consider.

Deepfakes, creative work and education are also in scope

The draft basic plan revision includes development of deepfake detection technology. Deepfakes are convincing images, audio or video created or manipulated with AI, which can be used to impersonate people or spread false claims.

The May 2024 policy paper identifies several technical responses: watermarks that signal AI generation, information recording where content came from, and labels applied by online platforms. It also connects AI risks to broader efforts against fraudulent advertisements and harmful online information.

Creative work has a separate legal and policy track. The paper records the March 15, 2024 adoption of guidance on AI and copyright by a Cultural Affairs Agency council subcommittee, alongside discussions by the Cabinet Office on intellectual property in the AI era. Those discussions cover legal rules, technical protection and contracts providing compensation. Concerns also extend to voices, likenesses and artistic styles.

Education appears in the proposed basic plan revision because heavy reliance on AI may weaken independent thinking and judgment. At the same time, Japan intends to support early commercialization of autonomous AI. The policy therefore seeks both greater use of capable systems and measures that preserve human skills and control.

The balance Japan is trying to strike

Japan's official papers describe a narrower role for binding regulation than a blanket system covering all AI development. The May 2024 paper favors voluntary guidance for developers whose work presents limited risk, including many researchers and startups, while considering stronger requirements for systems with greater potential impact.

The same paper contrasts international approaches as they stood at the time. The European Union's AI legislation included obligations based on risk and substantial financial sanctions. The United States combined voluntary commitments with reporting requirements for certain large models under existing legal powers. Both approaches placed some binding duties on developers of powerful systems.

Japan's proposed orders and penalties would address a concrete weakness in an approach centered on cooperation: a dangerous system may require action even when its supplier refuses to engage. Defining the businesses covered, the duties they owe and the conditions for an order will determine how that power works for users and developers.

Key Points

• Japan is considering safety duties and penalties focused on serious risks from advanced AI.
• The proposed process would allow binding orders against uncooperative businesses, with penalties for violations.
• Official policy documents favor targeted legal requirements where voluntary efforts are insufficient.
• A draft AI Basic Plan revision includes expanded safety testing, cybersecurity defenses, deepfake detection and education measures.
• The government aims to present a broader framework for institutional changes by year end.