# The AI Act through AI’s eyes: using the most advanced text models—or trying to—to analyse the EU law in full Author: Fernando Nieto Lobato Original publication: 2024-03-20 Spanish original: https://estrategiabyaleph.substack.com/p/estrategia-25-explorando-la-ai-act English URL: https://elcontemplador.github.io/estrategia-english/essays/025/ Status: Published translation This is a translation of the original Spanish essay published on 20 March 2024. Its claims, examples and forecasts retain that historical context. English publication: 2026-09-29 *Archive note: this article records an experiment published on 20 March 2024, following the European Parliament’s vote. The questions and model answers are translated from the Spanish source and preserved as historical AI output, not as verified legal analysis or current guidance. The two screenshot answers about Article 30 differ; both are reproduced faithfully below.* Last Wednesday, the European Parliament approved the AI Act, the general law that will regulate artificial intelligence in the European Union and become the world’s first major comprehensive transnational law on the subject. As [we already examined the text agreed at the end of last year in depth in issue 11 of the newsletter](https://open.substack.com/pub/estrategiabyaleph/p/estrategia-11-la-union-europea-se?r=2tyybn&utm_campaign=post&utm_medium=web), **this time we decided to use the AI Act’s approval as an opportunity for a small experiment with one of AI’s potentially most useful capabilities in politics: analysing and working with legislation**. In this way, we can bring you directly the analysis that some of today’s most advanced AI systems make of the new law. ## Methodology We began with GPT-4, Gemini Ultra—using it in English and translating afterwards, for the reasons explained in [issue 20 of estrategIA](https://open.substack.com/pub/estrategiabyaleph/p/estrategia-20-google-lanza-el-modelo?r=2tyybn&utm_campaign=post&utm_medium=web)—and Claude 3, the three most advanced “textual” AI systems currently available. The first task, which all completed successfully, was to act as journalists and generate interesting questions about the AI Act. From the full set, we eventually selected five generated by GPT-4, perhaps because it is the tool we use most often and whose prompts we have the best feel for, and because we tried it first—many questions were quite similar—plus one from Gemini Ultra and another from Claude 3. So far, so good. [The problem came when we began working with the law and realised it ran to more than 450 pages](https://www.europarl.europa.eu/doceo/document/TA-9-2024-0138_EN.html). Unfortunately, this let us establish directly that neither GPT-4 nor Gemini Ultra had a large enough context window to work with such a long text. In our tests, GPT-4 could not remember and retrieve complete articles beyond the first 17. The situation with Gemini Ultra was similar: although it searched the internet and could provide some information about a given article, it was not retrieving it directly from the source and therefore analysing it first-hand, which was what we wanted on this occasion. **Fortunately, every version of Claude 3 has a context window of up to 200,000 tokens**, with more than 99% accuracy when retrieving content. **This allows it to work with much larger texts, so we could use it to answer our previously selected questions and produce the analysis below**. There was one further setback. Working with so much information—more than 450 pages—has a **very high computational cost**, and we soon exhausted our available credits for Claude 3 Opus, the most powerful version, in just two queries. One was a context-window check in which we asked it to retrieve a particular article of the law. Fortunately, after the first answer, we were able to generate the rest free using Claude 3 Haiku. It performs very well and may currently be the most recommendable free model. We used it through the [Poe website](https://poe.com/), which we continue to recommend. ## Analysis of the law The original article reproduces the questions generated by the three large language models and Claude 3’s answers without alteration. Their full translations follow. The first answer is by Opus; the remaining answers are by Haiku, as explained above. ### 1. Innovation and its impact on AI development > **Question:** From your data-analysis perspective, what are the most innovative aspects of the AI Act, and how do you think they will change the development and deployment of AI systems in the European Union? > > **Claude 3 Opus:** After analysing the full text of the AI Act approved by the European Parliament in detail, I consider the following among its most innovative aspects and those likely to have the greatest impact on AI development and deployment in the EU: > > 1. The risk-based approach, classifying AI systems into categories according to their level of risk: unacceptable, high, limited and minimal. This will allow proportionate regulation, focusing on the highest-risk systems. > 2. The explicit prohibition of certain AI practices deemed unacceptable, such as social scoring, subliminal manipulation, exploitation of vulnerabilities or real-time remote biometric identification in public spaces, subject to exceptions. This sets clear limits on harmful AI applications. > 3. Strict requirements for high-risk systems concerning risk management, data quality, documentation, transparency, human oversight, accuracy and cybersecurity. These will significantly raise quality and safety standards. > 4. Regulatory sandboxes allowing innovative AI systems to be developed, tested and validated in controlled, supervised environments. These will facilitate innovation and the development of trustworthy AI. > 5. Support for SMEs and start-ups, including priority access to sandboxes, awareness-raising activities and dedicated communication channels. This will encourage a more diverse, inclusive AI ecosystem. > 6. Specific regulation of general-purpose AI models, with transparency and systemic-risk management requirements. Given their considerable impact, these systems needed to be addressed specifically. > 7. An EU-level public database of high-risk AI systems, to promote transparency and oversight. > > Taken together, I believe the AI Act will create a robust, balanced regulatory framework that increases trust in AI, minimising risks and harm without stifling innovation. It will raise standards of quality, safety, transparency and accountability throughout the EU while providing flexibility according to risk level. > > It will require a rethink of how AI is designed, developed and used, incorporating safeguards from the outset. More testing, documentation, traceability and human involvement will be needed. AI will have to be more robust, equitable and explainable. > > It will provide legal certainty and a level playing field for businesses. Harmonising EU rules will make it easier to operate throughout the single market. > > Ultimately, I believe it will encourage more trustworthy, human-centred AI in the EU, becoming an international reference point. However, compliance will also require investment and new capabilities from businesses and public authorities. ### 2. Ethical and social concerns > **Question:** Considering the ethical debates and social concerns surrounding artificial intelligence, how does the AI Act address these issues, and what are its potential strengths and weaknesses in ensuring ethical, responsible AI technology? > > **Claude 3 Haiku:** The EU AI Act addresses ethical and social issues in several key ways: > > **Strengths:** > > 1. **Prohibition of unacceptable AI practices:** The law explicitly prohibits certain AI practices considered abusive or manipulative, such as social scoring or remote identification of emotions in the workplace. This establishes clear boundaries for unacceptable AI uses. > 2. **Requirements for high-risk AI systems:** Requirements such as fundamental-rights risk assessments, human oversight and transparency seek to mitigate these systems’ ethical and social risks. > 3. **Emphasis on human-centred AI:** The law insists that AI must serve human well-being and EU values, helping to guide ethical AI development. > 4. **Stakeholder participation:** The law provides for experts, civil society organisations and other actors to participate in developing voluntary AI codes of conduct. > > **Weaknesses:** > > 1. **Limited scope:** The law focuses mainly on high-risk AI systems, leaving out other AI uses that also raise ethical concerns. > 2. **Voluntary compliance:** Ethical requirements for lower-risk systems are largely left to voluntary compliance through codes of conduct. > 3. **Implementation challenges:** AI’s technical complexity and rapid evolution may make it difficult to apply ethical principles consistently. > 4. **Lack of strong penalties:** Although substantial, fines for non-compliance might not be sufficiently deterrent for large technology companies. ### 3. Regulation, innovation and competitiveness > **Question:** Analysing the balance between regulation and innovation that the AI Act seeks, how would you assess its potential impact on European companies’ competitiveness in the global AI landscape? > > **Claude 3 Haiku:** The European Union’s AI Act attempts to balance AI regulation with the encouragement of innovation, with potentially mixed effects on European companies’ global competitiveness: > > **Positive aspects for competitiveness:** > > 1. **A harmonised EU regulatory framework:** This provides greater certainty and reduces fragmentation, facilitating the development and marketing of AI systems in the single market. > 2. **Innovation support:** Regulatory sandboxes and targeted assistance for SMEs can help European companies develop and test innovative AI solutions. > 3. **Emphasis on trust and ethical AI:** By setting minimum safety and responsibility requirements, the law can improve European AI’s reputation and consumer acceptance. > > **Negative aspects for competitiveness:** > > 1. **Additional compliance requirements:** Providers of high-risk AI systems must meet a series of obligations, potentially increasing costs and complexity, especially for SMEs. > 2. **Restrictions on critical applications:** Limits on AI use for remote biometric identification, criminal-risk assessment and similar applications may constrain the development of competitive solutions in certain sectors. > 3. **Uncertainty over interpretation and practical application:** The flexibility left to Member States and rapid technological change may produce implementation disparities and complexity for operators. > 4. **Possible delays in AI adoption:** The transition period and time needed to develop standards could slow the introduction of AI solutions to the European market. ### 4. Challenges and limitations > **Question:** Based on your analysis of large quantities of data and previous discussions of AI regulation, what do you consider the greatest challenges and limitations facing the AI Act in its aim of creating an effective regulatory framework for AI? > > **Claude 3 Haiku:** Some of the greatest challenges and limitations facing the EU AI Act in its aim of creating an effective regulatory framework for artificial intelligence are: > > **1. The rapid pace of AI innovation:** > > - Rapid technological change may quickly make some provisions obsolete. > - Keeping regulation up to date and relevant will be challenging. > > **2. AI systems’ technical complexity:** > > - The opacity and sophistication of AI algorithms make compliance assessment and monitoring difficult. > - Appropriate methods and tools for assessing conformity will need to be developed. > > **3. Scope limited to “high-risk” systems:** > > - Many “low-risk” AI applications also raise ethical and social-impact concerns. > - Limiting the scope may leave problematic uses of the technology uncovered. > > **4. Difficulties in harmonised implementation:** > > - Differences in interpretation and application by Member States may cause fragmentation. > - Consistent implementation across Europe will be essential. > > **5. Effective oversight and enforcement:** > > - Ensuring adequate oversight and penalties throughout the single market will be a challenge. > - Fines, although high, may not sufficiently deter large companies. > > **6. Balancing regulation and innovation:** > > - Finding the middle ground between protecting rights and safety and avoiding undue barriers to innovation. > - Preventing European regulation from losing competitiveness against more flexible frameworks. ### 5. Adaptation to future innovation > **Question:** Given the rapid pace of development in artificial intelligence, how do you think the AI Act will adapt to future technological innovations, and what provisions or mechanisms in the law enable this adaptability? > > **Claude 3 Haiku:** The EU AI Act provides several mechanisms to adapt to future technological innovations in artificial intelligence: > > **1. Periodic review and updating:** > > - Article 112 establishes that the European Commission must evaluate and review the law every four years, taking technical advances into account. > - This will allow it to identify needs to amend or extend the regulation’s scope. > > **2. Delegated acts procedure:** > > - The law empowers the Commission to adopt delegated acts to update certain aspects, such as the conditions for considering an AI system “low risk”, the list of high-risk AI systems, technical requirements and so on. > - This provides flexibility to adapt the rules without revising the entire law. > > **3. Development of harmonised standards:** > > - The law calls on the Commission to request the development of harmonised standards reflecting the state of the art in AI. > - These standards can be updated more quickly than the law itself. > > **4. Common specifications:** > > - The Commission may adopt implementing acts establishing “common specifications” when harmonised standards prove insufficient. > - This allows new technical requirements to be addressed more quickly. > > **5. Expert and stakeholder participation:** > > - The law provides for expert groups and advisory forums to contribute technical knowledge. > - This facilitates early identification of emerging trends and challenges. > > **6. Phased entry into force:** > > - Some provisions will enter into force before others, allowing more time for implementation. > - This helps gradual adaptation to changes. > > Together, these mechanisms give the AI Act some flexibility and capacity to adapt, although the rapid pace of AI innovation will remain a significant practical challenge. Constant review and updating will be fundamental. ### 6. Risk categories > **Question:** Do you consider classifying AI systems into risk categories—unacceptable, high, limited and minimal—an appropriate approach? Why or why not? > > **Claude 3 Haiku:** The AI Act’s approach of classifying AI systems into risk categories—unacceptable, high, low and minimal—is generally appropriate and effective for several reasons: > > **1. Proportionality and adaptability:** > > - It allows requirements and obligations proportionate to each AI system’s risk level. > - This avoids unnecessary burdens on lower-risk technologies while maintaining flexibility. > > **2. A risk-based approach:** > > - Focusing regulation on systems posing the greatest risks to health, safety and fundamental rights is pragmatic. > - It allows oversight and control efforts to be prioritised in areas with the greatest potential impact. > > **3. Transparency and predictability:** > > - Explicit classification of unacceptable, high-risk and other systems creates greater clarity and legal certainty. > - Operators can better understand which requirements will apply to them. > > **4. Capacity to adapt:** > > - The law allows periodic updates to lists of high-risk systems and unacceptable practices. > - This makes it easier to incorporate new technological developments and emerging risks. > > Nevertheless, there are also possible limitations: > > **1. Blurred boundaries between categories:** > > - It can sometimes be difficult to determine precisely which category an AI system belongs in. > - This could create uncertainty and litigation. > > **2. Exclusion of “lower-risk” systems:** > > - Relevant ethical and social concerns also arise in “lower-risk” applications outside the scope. > - Broader coverage would be desirable, at least through transparency obligations. > > **3. Rapid technological change:** > > - The pace of innovation may outstrip the capacity to update risk classifications. > - Agile review and adaptation mechanisms will be essential. ### 7. Global impact > **Question:** Looking ahead, what global impact do you think the AI Act will have once fully in force? Do you expect it to significantly influence how other countries and regions regulate AI, or to become a de facto international standard? > > **Claude 3 Haiku:** Once the European Union’s AI Act is fully in force, it is very likely to have a significant global impact on AI regulation. Some of the principal effects that can be expected are: > > **1. Leadership and a model to follow:** > > - The AI Act will establish a comprehensive, pioneering regulatory framework worldwide. > - Other countries and regions will probably take note and tend to align their own rules with the European model. > > **2. Spillover effects in the global market:** > > - Given the size and economic weight of the European single market, companies will have to comply with AI Act requirements to operate in the EU. > - This will create pressure to adopt these standards even outside the EU. > > **3. Effects on interoperability and competitiveness:** > > - Harmonised European rules will facilitate the development and marketing of AI systems in the single market. > - This may give European companies a competitive advantage over competitors in fragmented markets. > > **4. Influence on international standard-setting:** > > - The EU will seek to promote global adoption of the technical and governance standards developed under the AI Act. > - This could make them de facto international reference points. > > **5. Pressure towards greater global regulatory convergence:** > > - A robust EU regulatory framework will pressure other jurisdictions to develop similar rules. > - This could encourage greater international harmonisation around common AI principles and requirements. > > However, there are also risks and limitations: > > - Divergent implementation at national level within the EU. > - Potential conflicts with other technological powers’ regulatory approaches. > - Difficulties in regulating AI effectively and uniformly on a global scale. > > In summary, the EU AI Act has considerable potential to become an influential global model for AI regulation, although its actual impact will depend on consistent implementation and its ability to adapt to a constantly evolving technological environment. ## Additional observations As a point of interest, here is evidence that both Claude 3 Opus and Haiku could correctly retrieve and translate Article 30 of the law into Spanish. We worked with the English text because it was not yet translated into our language, which is understandable given its length. Article 30 appears on page 248. Until now, models with very large context windows have usually had particular difficulty with material in the middle, while remembering the beginning and end especially well. As we saw in one answer, the model also retrieved correct information from Article 112, on page 411 of the document. ### Claude 3 Opus: screenshot [![Historical Claude 3 Opus response describing the notification procedure for conformity assessment bodies under Article 30.](https://elcontemplador.github.io/estrategia-english/assets/images/54462183-bdad-46f7-917c-1843b0a32c41_1566x880.png)](https://substackcdn.com/image/fetch/$s_!DwcF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54462183-bdad-46f7-917c-1843b0a32c41_1566x880.png) *Full translation of the Spanish model response in the screenshot:* > Article 30 of the AI Act concerns the notification procedure for conformity assessment bodies. It establishes the following: > > 1. Notifying authorities may designate only conformity assessment bodies that meet the requirements laid down in Article 31. > 2. Notifying authorities shall inform the Commission and the other Member States of the conformity assessment bodies they have designated, using the electronic notification tool developed and managed by the Commission. > 3. The notification shall include full details of the conformity assessment activities, the conformity assessment module or modules, the type or types of AI systems concerned and the corresponding attestation of competence. > 4. The conformity assessment body concerned may perform the activities of a notified body only if the Commission or other Member States raise no objections within two weeks of a notifying authority’s notification when it includes an accreditation certificate, or within two months when it does not include such a certificate. > 5. The Commission and the other Member States shall be informed of any subsequent relevant changes to the notification. > > In summary, this article establishes the process by which national authorities designate and notify the European Commission and other Member States of the bodies that will assess AI systems’ conformity with the AI Act’s requirements, as well as the periods within which the Commission and other countries may object to these notifications. ### Claude 3 Haiku, via Quora: screenshot [![A Poe exchange with Claude-3-Haiku-200k answering a question about Article 30 of the supplied document.](https://elcontemplador.github.io/estrategia-english/assets/images/e41751e7-6a65-44f1-b169-fd71ae10054d_1166x1057.png)](https://substackcdn.com/image/fetch/$s_!GLfu!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe41751e7-6a65-44f1-b169-fd71ae10054d_1166x1057.png) *Full translation of the Spanish exchange in the screenshot, whose interface is labelled Poe:* > **User:** What does Article 30 of the law I gave you in the document say? > > **Claude-3-Haiku-200k:** According to the document, Article 30 of the EU AI Act concerns the notification procedure for notified bodies. The main points are: > > 1. Conformity assessment bodies must submit an application for notification to the notifying authority of the Member State in which they are established. > 2. The application for notification must be accompanied by a description of the conformity assessment activities, the conformity assessment modules and the types of AI systems for which the body claims competence, as well as an accreditation certificate issued by a national accreditation body. > 3. If the conformity assessment body cannot provide an accreditation certificate, it must give the notifying authority all the supporting documentation necessary to verify, recognise and regularly monitor its compliance with the requirements. > 4. For notified bodies designated under any other Union harmonisation legislation, all documents and certificates relating to those designations may be used to support their designation procedure under this Regulation. > 5. The notifying authority may notify only conformity assessment bodies that have met the requirements set out in Article 31. > > This establishes the formal process through which Member States notify the European Commission of conformity assessment bodies competent to assess high-risk AI systems’ conformity with the law. And here is evidence of how Claude 3 Opus used up the $5 promotional credit in just two interactions. As shown, although the unit price per million tokens is much lower for input—the information we give the model—than for output—the information it generates—these prices remain much higher than competitors’ because this is the newest language model. Two requests with such a large document loaded carry an enormous cost. Fortunately, as we have already seen elsewhere, particularly with smaller, open-source models, these prices will fall very substantially in the coming months and years through technical optimisation of algorithms and processes, and increased available power from [new graphics card designs specifically adapted to AI with much greater computing power, such as those Nvidia unveiled last Monday](https://hipertextual.com/2024/03/nvidia-blackwell-b200-y-gb200). [![Historical Claude 3 Opus usage statement showing input and output token quantities, unit prices and totals.](https://elcontemplador.github.io/estrategia-english/assets/images/153ffa08-5fd7-4ec1-ab95-f3e84deafc18_1085x321.jpeg)](https://substackcdn.com/image/fetch/$s_!0tMQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F153ffa08-5fd7-4ec1-ab95-f3e84deafc18_1085x321.jpeg) *Accessible transcription of the usage screenshot; quantities, rounding and totals are preserved as displayed:* | Description | Quantity, millions of tokens | Unit price | Total | | --- | --- | --- | --- | | Input tokens | 0.38 | US$15.00 | US$5.70 | | Output tokens | 0.001 | US$75.00 | US$0.10 | Fernando Nieto Lobato *Director of Digital Innovation at Institución Educativa ALEPH*